-
A Review of Cybersecurity Incidents in the Food and Agriculture Sector
Authors:
Ajay Kulkarni,
Yingjie Wang,
Munisamy Gopinath,
Dan Sobien,
Abdul Rahman,
Feras A. Batarseh
Abstract:
The increasing utilization of emerging technologies in the Food & Agriculture (FA) sector has heightened the need for security to minimize cyber risks. Considering this aspect, this manuscript reviews disclosed and documented cybersecurity incidents in the FA sector. For this purpose, thirty cybersecurity incidents were identified, which took place between July 2011 and April 2023. The details of…
▽ More
The increasing utilization of emerging technologies in the Food & Agriculture (FA) sector has heightened the need for security to minimize cyber risks. Considering this aspect, this manuscript reviews disclosed and documented cybersecurity incidents in the FA sector. For this purpose, thirty cybersecurity incidents were identified, which took place between July 2011 and April 2023. The details of these incidents are reported from multiple sources such as: the private industry and flash notifications generated by the Federal Bureau of Investigation (FBI), internal reports from the affected organizations, and available media sources. Considering the available information, a brief description of the security threat, ransom amount, and impact on the organization are discussed for each incident. This review reports an increased frequency of cybersecurity threats to the FA sector. To minimize these cyber risks, popular cybersecurity frameworks and recent agriculture-specific cybersecurity solutions are also discussed. Further, the need for AI assurance in the FA sector is explained, and the Farmer-Centered AI (FCAI) framework is proposed. The main aim of the FCAI framework is to support farmers in decision-making for agricultural production, by incorporating AI assurance. Lastly, the effects of the reported cyber incidents on other critical infrastructures, food security, and the economy are noted, along with specifying the open issues for future development.
△ Less
Submitted 12 March, 2024;
originally announced March 2024.
-
ACWA: An AI-driven Cyber-Physical Testbed for Intelligent Water Systems
Authors:
Feras A. Batarseh,
Ajay Kulkarni,
Chhayly Sreng,
Justice Lin,
Siam Maksud
Abstract:
This manuscript presents a novel state-of-the-art cyber-physical water testbed, namely: The AI and Cyber for Water and Agriculture testbed (ACWA). ACWA is motivated by the need to advance water supply management using AI and Cybersecurity experimentation. The main goal of ACWA is to address pressing challenges in the water and agricultural domains by utilising cutting-edge AI and data-driven techn…
▽ More
This manuscript presents a novel state-of-the-art cyber-physical water testbed, namely: The AI and Cyber for Water and Agriculture testbed (ACWA). ACWA is motivated by the need to advance water supply management using AI and Cybersecurity experimentation. The main goal of ACWA is to address pressing challenges in the water and agricultural domains by utilising cutting-edge AI and data-driven technologies. These challenges include Cyberbiosecurity, resources management, access to water, sustainability, and data-driven decision-making, among others. To address such issues, ACWA consists of multiple topologies, sensors, computational nodes, pumps, tanks, smart water devices, as well as databases and AI models that control the system. Moreover, we present ACWA simulator, which is a software-based water digital twin. The simulator runs on fluid and constituent transport principles that produce theoretical time series of a water distribution system. This creates a good validation point for comparing the theoretical approach with real-life results via the physical ACWA testbed. ACWA data are available to AI and water domain researchers and are hosted in an online public repository. In this paper, the system is introduced in detail and compared with existing water testbeds; additionally, example use-cases are described along with novel outcomes such as datasets, software, and AI-related scenarios.
△ Less
Submitted 26 September, 2023;
originally announced October 2023.
-
ExClaim: Explainable Neural Claim Verification Using Rationalization
Authors:
Sai Gurrapu,
Lifu Huang,
Feras A. Batarseh
Abstract:
With the advent of deep learning, text generation language models have improved dramatically, with text at a similar level as human-written text. This can lead to rampant misinformation because content can now be created cheaply and distributed quickly. Automated claim verification methods exist to validate claims, but they lack foundational data and often use mainstream news as evidence sources t…
▽ More
With the advent of deep learning, text generation language models have improved dramatically, with text at a similar level as human-written text. This can lead to rampant misinformation because content can now be created cheaply and distributed quickly. Automated claim verification methods exist to validate claims, but they lack foundational data and often use mainstream news as evidence sources that are strongly biased towards a specific agenda. Current claim verification methods use deep neural network models and complex algorithms for a high classification accuracy but it is at the expense of model explainability. The models are black-boxes and their decision-making process and the steps it took to arrive at a final prediction are obfuscated from the user. We introduce a novel claim verification approach, namely: ExClaim, that attempts to provide an explainable claim verification system with foundational evidence. Inspired by the legal system, ExClaim leverages rationalization to provide a verdict for the claim and justifies the verdict through a natural language explanation (rationale) to describe the model's decision-making process. ExClaim treats the verdict classification task as a question-answer problem and achieves a performance of 0.93 F1 score. It provides subtasks explanations to also justify the intermediate outcomes. Statistical and Explainable AI (XAI) evaluations are conducted to ensure valid and trustworthy outcomes. Ensuring claim verification systems are assured, rational, and explainable is an essential step toward improving Human-AI trust and the accessibility of black-box systems.
△ Less
Submitted 21 January, 2023;
originally announced January 2023.
-
Rationalization for Explainable NLP: A Survey
Authors:
Sai Gurrapu,
Ajay Kulkarni,
Lifu Huang,
Ismini Lourentzou,
Laura Freeman,
Feras A. Batarseh
Abstract:
Recent advances in deep learning have improved the performance of many Natural Language Processing (NLP) tasks such as translation, question-answering, and text classification. However, this improvement comes at the expense of model explainability. Black-box models make it difficult to understand the internals of a system and the process it takes to arrive at an output. Numerical (LIME, Shapley) a…
▽ More
Recent advances in deep learning have improved the performance of many Natural Language Processing (NLP) tasks such as translation, question-answering, and text classification. However, this improvement comes at the expense of model explainability. Black-box models make it difficult to understand the internals of a system and the process it takes to arrive at an output. Numerical (LIME, Shapley) and visualization (saliency heatmap) explainability techniques are helpful; however, they are insufficient because they require specialized knowledge. These factors led rationalization to emerge as a more accessible explainable technique in NLP. Rationalization justifies a model's output by providing a natural language explanation (rationale). Recent improvements in natural language generation have made rationalization an attractive technique because it is intuitive, human-comprehensible, and accessible to non-technical users. Since rationalization is a relatively new field, it is disorganized. As the first survey, rationalization literature in NLP from 2007-2022 is analyzed. This survey presents available methods, explainable evaluations, code, and datasets used across various NLP tasks that use rationalization. Further, a new subfield in Explainable AI (XAI), namely, Rational AI (RAI), is introduced to advance the current state of rationalization. A discussion on observed insights, challenges, and future directions is provided to point to promising research opportunities.
△ Less
Submitted 21 January, 2023;
originally announced January 2023.
-
Proceedings of AAAI 2022 Fall Symposium: The Role of AI in Responding to Climate Challenges
Authors:
Feras A. Batarseh,
Priya L. Donti,
Ján Drgoňa,
Kristen Fletcher,
Pierre-Adrien Hanania,
Melissa Hatton,
Srinivasan Keshav,
Bran Knowles,
Raphaela Kotsch,
Sean McGinnis,
Peetak Mitra,
Alex Philp,
Jim Spohrer,
Frank Stein,
Meghna Tare,
Svitlana Volkov,
Gege Wen
Abstract:
Climate change is one of the most pressing challenges of our time, requiring rapid action across society. As artificial intelligence tools (AI) are rapidly deployed, it is therefore crucial to understand how they will impact climate action. On the one hand, AI can support applications in climate change mitigation (reducing or preventing greenhouse gas emissions), adaptation (preparing for the effe…
▽ More
Climate change is one of the most pressing challenges of our time, requiring rapid action across society. As artificial intelligence tools (AI) are rapidly deployed, it is therefore crucial to understand how they will impact climate action. On the one hand, AI can support applications in climate change mitigation (reducing or preventing greenhouse gas emissions), adaptation (preparing for the effects of a changing climate), and climate science. These applications have implications in areas ranging as widely as energy, agriculture, and finance. At the same time, AI is used in many ways that hinder climate action (e.g., by accelerating the use of greenhouse gas-emitting fossil fuels). In addition, AI technologies have a carbon and energy footprint themselves. This symposium brought together participants from across academia, industry, government, and civil society to explore these intersections of AI with climate change, as well as how each of these sectors can contribute to solutions.
△ Less
Submitted 29 January, 2023; v1 submitted 27 December, 2022;
originally announced December 2022.
-
Cybersecurity Law: Legal Jurisdiction and Authority
Authors:
Feras A. Batarseh
Abstract:
Cybersecurity threats affect all aspects of society; critical infrastructures (such as networks, corporate systems, water supply systems, and intelligent transportation systems) are especially prone to attacks and can have tangible negative consequences on society. However, these critical cyber systems are generally governed by multiple jurisdictions, for instance the Metro in the Washington, D.C.…
▽ More
Cybersecurity threats affect all aspects of society; critical infrastructures (such as networks, corporate systems, water supply systems, and intelligent transportation systems) are especially prone to attacks and can have tangible negative consequences on society. However, these critical cyber systems are generally governed by multiple jurisdictions, for instance the Metro in the Washington, D.C. area is managed by the states of Virginia and Maryland, as well as the District of Columbia (DC) through Washington Metropolitan Area Transit Authority (WMATA). Additionally, the water treatment infrastructure managed by DC Water consists of waste water input from Fairfax and Arlington counties, and the district (i.e. DC). Additionally, cyber attacks usually launch from unknown sources, through unknown switches and servers, and end up at the destination without much knowledge on their source or path. Certain infrastructures are shared amongst multiple countries, another idiosyncrasy that exacerbates the issue of governance. This law paper however, is not concerned with the general governance of these infrastructures, rather with the ambiguity in the relevant laws or doctrines about which authority would prevail in the context of a cyber threat or a cyber-attack, with a focus on federal vs. state issues, international law involvement, federal preemption, technical aspects that could affect lawmaking, and conflicting responsibilities in cases of cyber crime. A legal analysis of previous cases is presented, as well as an extended discussion addressing different sides of the argument.
△ Less
Submitted 20 July, 2022; v1 submitted 19 June, 2022;
originally announced June 2022.
-
AI Assurance using Causal Inference: Application to Public Policy
Authors:
Andrei Svetovidov,
Abdul Rahman,
Feras A. Batarseh
Abstract:
Developing and implementing AI-based solutions help state and federal government agencies, research institutions, and commercial companies enhance decision-making processes, automate chain operations, and reduce the consumption of natural and human resources. At the same time, most AI approaches used in practice can only be represented as "black boxes" and suffer from the lack of transparency. Thi…
▽ More
Developing and implementing AI-based solutions help state and federal government agencies, research institutions, and commercial companies enhance decision-making processes, automate chain operations, and reduce the consumption of natural and human resources. At the same time, most AI approaches used in practice can only be represented as "black boxes" and suffer from the lack of transparency. This can eventually lead to unexpected outcomes and undermine trust in such systems. Therefore, it is crucial not only to develop effective and robust AI systems, but to make sure their internal processes are explainable and fair. Our goal in this chapter is to introduce the topic of designing assurance methods for AI systems with high-impact decisions using the example of the technology sector of the US economy. We explain how these fields would benefit from revealing cause-effect relationships between key metrics in the dataset by providing the causal experiment on technology economics dataset. Several causal inference approaches and AI assurance techniques are reviewed and the transformation of the data into a graph-structured dataset is demonstrated.
△ Less
Submitted 1 December, 2021;
originally announced December 2021.
-
Outlier Detection using AI: A Survey
Authors:
Md Nazmul Kabir Sikder,
Feras A. Batarseh
Abstract:
An outlier is an event or observation that is defined as an unusual activity, intrusion, or a suspicious data point that lies at an irregular distance from a population. The definition of an outlier event, however, is subjective and depends on the application and the domain (Energy, Health, Wireless Network, etc.). It is important to detect outlier events as carefully as possible to avoid infrastr…
▽ More
An outlier is an event or observation that is defined as an unusual activity, intrusion, or a suspicious data point that lies at an irregular distance from a population. The definition of an outlier event, however, is subjective and depends on the application and the domain (Energy, Health, Wireless Network, etc.). It is important to detect outlier events as carefully as possible to avoid infrastructure failures because anomalous events can cause minor to severe damage to infrastructure. For instance, an attack on a cyber-physical system such as a microgrid may initiate voltage or frequency instability, thereby damaging a smart inverter which involves very expensive repairing. Unusual activities in microgrids can be mechanical faults, behavior changes in the system, human or instrument errors or a malicious attack. Accordingly, and due to its variability, Outlier Detection (OD) is an ever-growing research field. In this chapter, we discuss the progress of OD methods using AI techniques. For that, the fundamental concepts of each OD model are introduced via multiple categories. Broad range of OD methods are categorized into six major categories: Statistical-based, Distance-based, Density-based, Clustering-based, Learning-based, and Ensemble methods. For every category, we discuss recent state-of-the-art approaches, their application areas, and performances. After that, a brief discussion regarding the advantages, disadvantages, and challenges of each technique is provided with recommendations on future research directions. This survey aims to guide the reader to better understand recent progress of OD methods for the assurance of AI.
△ Less
Submitted 1 December, 2021;
originally announced December 2021.
-
Public Policymaking for International Agricultural Trade using Association Rules and Ensemble Machine Learning
Authors:
Feras A. Batarseh,
Munisamy Gopinath,
Anderson Monken,
Zhengrong Gu
Abstract:
International economics has a long history of improving our understanding of factors causing trade, and the consequences of free flow of goods and services across countries. The recent shocks to the free trade regime, especially trade disputes among major economies, as well as black swan events, such as trade wars and pandemics, raise the need for improved predictions to inform policy decisions. A…
▽ More
International economics has a long history of improving our understanding of factors causing trade, and the consequences of free flow of goods and services across countries. The recent shocks to the free trade regime, especially trade disputes among major economies, as well as black swan events, such as trade wars and pandemics, raise the need for improved predictions to inform policy decisions. AI methods are allowing economists to solve such prediction problems in new ways. In this manuscript, we present novel methods that predict and associate food and agricultural commodities traded internationally. Association Rules (AR) analysis has been deployed successfully for economic scenarios at the consumer or store level, such as for market basket analysis. In our work however, we present analysis of imports and exports associations and their effects on commodity trade flows. Moreover, Ensemble Machine Learning methods are developed to provide improved agricultural trade predictions, outlier events' implications, and quantitative pointers to policy makers.
△ Less
Submitted 14 November, 2021;
originally announced November 2021.
-
A Survey on AI Assurance
Authors:
Feras A. Batarseh,
Laura Freeman
Abstract:
Artificial Intelligence (AI) algorithms are increasingly providing decision making and operational support across multiple domains. AI includes a wide library of algorithms for different problems. One important notion for the adoption of AI algorithms into operational decision process is the concept of assurance. The literature on assurance, unfortunately, conceals its outcomes within a tangled la…
▽ More
Artificial Intelligence (AI) algorithms are increasingly providing decision making and operational support across multiple domains. AI includes a wide library of algorithms for different problems. One important notion for the adoption of AI algorithms into operational decision process is the concept of assurance. The literature on assurance, unfortunately, conceals its outcomes within a tangled landscape of conflicting approaches, driven by contradicting motivations, assumptions, and intuitions. Accordingly, albeit a rising and novel area, this manuscript provides a systematic review of research works that are relevant to AI assurance, between years 1985 - 2021, and aims to provide a structured alternative to the landscape. A new AI assurance definition is adopted and presented and assurance methods are contrasted and tabulated. Additionally, a ten-metric scoring system is developed and introduced to evaluate and compare existing methods. Lastly, in this manuscript, we provide foundational insights, discussions, future directions, a roadmap, and applicable recommendations for the development and deployment of AI assurance.
△ Less
Submitted 14 November, 2021;
originally announced November 2021.
-
Measuring Outcomes in Healthcare Economics using Artificial Intelligence: with Application to Resource Management
Authors:
Chih-Hao Huang,
Feras A. Batarseh,
Adel Boueiz,
Ajay Kulkarni,
Po-Hsuan Su,
Jahan Aman
Abstract:
The quality of service in healthcare is constantly challenged by outlier events such as pandemics (i.e. Covid-19) and natural disasters (such as hurricanes and earthquakes). In most cases, such events lead to critical uncertainties in decision making, as well as in multiple medical and economic aspects at a hospital. External (geographic) or internal factors (medical and managerial), lead to shift…
▽ More
The quality of service in healthcare is constantly challenged by outlier events such as pandemics (i.e. Covid-19) and natural disasters (such as hurricanes and earthquakes). In most cases, such events lead to critical uncertainties in decision making, as well as in multiple medical and economic aspects at a hospital. External (geographic) or internal factors (medical and managerial), lead to shifts in planning and budgeting, but most importantly, reduces confidence in conventional processes. In some cases, support from other hospitals proves necessary, which exacerbates the planning aspect. This manuscript presents three data-driven methods that provide data-driven indicators to help healthcare managers organize their economics and identify the most optimum plan for resources allocation and sharing. Conventional decision-making methods fall short in recommending validated policies for managers. Using reinforcement learning, genetic algorithms, traveling salesman, and clustering, we experimented with different healthcare variables and presented tools and outcomes that could be applied at health institutes. Experiments are performed; the results are recorded, evaluated, and presented.
△ Less
Submitted 14 November, 2021;
originally announced November 2021.
-
DeepAg: Deep Learning Approach for Measuring the Effects of Outlier Events on Agricultural Production and Policy
Authors:
Sai Gurrapu,
Feras A. Batarseh,
Pei Wang,
Md Nazmul Kabir Sikder,
Nitish Gorentala,
Gopinath Munisamy
Abstract:
Quantitative metrics that measure the global economy's equilibrium have strong and interdependent relationships with the agricultural supply chain and international trade flows. Sudden shocks in these processes caused by outlier events such as trade wars, pandemics, or weather can have complex effects on the global economy. In this paper, we propose a novel framework, namely: DeepAg, that employs…
▽ More
Quantitative metrics that measure the global economy's equilibrium have strong and interdependent relationships with the agricultural supply chain and international trade flows. Sudden shocks in these processes caused by outlier events such as trade wars, pandemics, or weather can have complex effects on the global economy. In this paper, we propose a novel framework, namely: DeepAg, that employs econometrics and measures the effects of outlier events detection using Deep Learning (DL) to determine relationships between commonplace financial indices (such as the DowJones), and the production values of agricultural commodities (such as Cheese and Milk). We employed a DL technique called Long Short-Term Memory (LSTM) networks successfully to predict commodity production with high accuracy and also present five popular models (regression and boosting) as baselines to measure the effects of outlier events. The results indicate that DeepAg with outliers' considerations (using Isolation Forests) outperforms baseline models, as well as the same model without outliers detection. Outlier events make a considerable impact when predicting commodity production with respect to financial indices. Moreover, we present the implications of DeepAg on public policy, provide insights for policymakers and farmers, and for operational decisions in the agricultural ecosystem. Data are collected, models developed, and the results are recorded and presented.
△ Less
Submitted 6 November, 2021; v1 submitted 22 October, 2021;
originally announced October 2021.
-
The history and future prospects of open data and open source software
Authors:
Feras A. Batarseh,
Abhinav Kumar,
Sam Eisenberg
Abstract:
Open data for all New Yorkers is the tagline on New York City's open data website. Open government is being promoted at most countries of the western world. Government transparency levels are being measured by the amount of data they share through their online public repositories. Additionally, open source software is promoted at governments, academia, and the industry. This is the new digital sto…
▽ More
Open data for all New Yorkers is the tagline on New York City's open data website. Open government is being promoted at most countries of the western world. Government transparency levels are being measured by the amount of data they share through their online public repositories. Additionally, open source software is promoted at governments, academia, and the industry. This is the new digital story of this century, and the new testament between the Gods of technology and there users. Data and software openness will redefine the path forward and aim to rekindle our collective intelligence. Data and software openness can redefine Data Democracy and be the catalyst for its progress. This chapter provides a historical insight into data and software openness, the beginnings, the heroes, prospects for the future, and all things we cannot afford to negotiate or lose.
△ Less
Submitted 3 August, 2021;
originally announced August 2021.
-
The application of artificial intelligence in software engineering: a review challenging conventional wisdom
Authors:
Feras A. Batarseh,
Rasika Mohod,
Abhinav Kumar,
Justin Bui
Abstract:
The field of artificial intelligence (AI) is witnessing a recent upsurge in research, tools development, and deployment of applications. Multiple software companies are shifting their focus to developing intelligent systems; and many others are deploying AI paradigms to their existing processes. In parallel, the academic research community is injecting AI paradigms to provide solutions to traditio…
▽ More
The field of artificial intelligence (AI) is witnessing a recent upsurge in research, tools development, and deployment of applications. Multiple software companies are shifting their focus to developing intelligent systems; and many others are deploying AI paradigms to their existing processes. In parallel, the academic research community is injecting AI paradigms to provide solutions to traditional engineering problems. Similarly, AI has evidently been proved useful to software engineering (SE). When one observes the SE phases (requirements, design, development, testing, release, and maintenance), it becomes clear that multiple AI paradigms (such as neural networks, machine learning, knowledge-based systems, natural language processing) could be applied to improve the process and eliminate many of the major challenges that the SE field has been facing. This survey chapter is a review of the most commonplace methods of AI applied to SE. The review covers methods between years 1975-2017, for the requirements phase, 46 major AI-driven methods are found, 19 for design, 15 for development, 68 for testing, and 15 for release and maintenance. Furthermore, the purpose of this chapter is threefold; firstly, to answer the following questions: is there sufficient intelligence in the SE lifecycle? What does applying AI to SE entail? Secondly, to measure, formulize, and evaluate the overlap of SE phases and AI disciplines. Lastly, this chapter aims to provide serious questions to challenging the current conventional wisdom (i.e., status quo) of the state-of-the-art, craft a call for action, and to redefine the path forward.
△ Less
Submitted 3 August, 2021;
originally announced August 2021.
-
Foundations of data imbalance and solutions for a data democracy
Authors:
Ajay Kulkarni,
Deri Chong,
Feras A. Batarseh
Abstract:
Dealing with imbalanced data is a prevalent problem while performing classification on the datasets. Many times, this problem contributes to bias while making decisions or implementing policies. Thus, it is vital to understand the factors which cause imbalance in the data (or class imbalance). Such hidden biases and imbalances can lead to data tyranny and a major challenge to a data democracy. In…
▽ More
Dealing with imbalanced data is a prevalent problem while performing classification on the datasets. Many times, this problem contributes to bias while making decisions or implementing policies. Thus, it is vital to understand the factors which cause imbalance in the data (or class imbalance). Such hidden biases and imbalances can lead to data tyranny and a major challenge to a data democracy. In this chapter, two essential statistical elements are resolved: the degree of class imbalance and the complexity of the concept; solving such issues helps in building the foundations of a data democracy. Furthermore, statistical measures which are appropriate in these scenarios are discussed and implemented on a real-life dataset (car insurance claims). In the end, popular data-level methods such as random oversampling, random undersampling, synthetic minority oversampling technique, Tomek link, and others are implemented in Python, and their performance is compared.
△ Less
Submitted 30 July, 2021;
originally announced August 2021.
-
Panel: Economic Policy and Governance during Pandemics using AI
Authors:
Feras A. Batarseh,
Munisamy Gopinath
Abstract:
The global food supply chain (starting at farms and ending with consumers) has been seriously disrupted by many outlier events such as trade wars, the China demand shock, natural disasters, and pandemics. Outlier events create uncertainty along the entire supply chain in addition to intervening policy responses to mitigate their adverse effects. Artificial Intelligence (AI) methods (i.e. machine/r…
▽ More
The global food supply chain (starting at farms and ending with consumers) has been seriously disrupted by many outlier events such as trade wars, the China demand shock, natural disasters, and pandemics. Outlier events create uncertainty along the entire supply chain in addition to intervening policy responses to mitigate their adverse effects. Artificial Intelligence (AI) methods (i.e. machine/reinforcement/deep learning) provide an opportunity to better understand outcomes during outlier events by identifying regular, irregular and contextual components. Employing AI can provide guidance to decision making suppliers, farmers, processors, wholesalers, and retailers along the supply chain, and policy makers to facilitate welfare-improving outcomes. This panel discusses these issues.
△ Less
Submitted 20 October, 2020;
originally announced October 2020.
-
Context-Driven Data Mining through Bias Removal and Data Incompleteness Mitigation
Authors:
Feras A. Batarseh,
Ajay Kulkarni
Abstract:
The results of data mining endeavors are majorly driven by data quality. Throughout these deployments, serious show-stopper problems are still unresolved, such as: data collection ambiguities, data imbalance, hidden biases in data, the lack of domain information, and data incompleteness. This paper is based on the premise that context can aid in mitigating these issues. In a traditional data scien…
▽ More
The results of data mining endeavors are majorly driven by data quality. Throughout these deployments, serious show-stopper problems are still unresolved, such as: data collection ambiguities, data imbalance, hidden biases in data, the lack of domain information, and data incompleteness. This paper is based on the premise that context can aid in mitigating these issues. In a traditional data science lifecycle, context is not considered. Context-driven Data Science Lifecycle (C-DSL); the main contribution of this paper, is developed to address these challenges. Two case studies (using data-sets from sports events) are developed to test C-DSL. Results from both case studies are evaluated using common data mining metrics such as: coefficient of determination (R2 value) and confusion matrices. The work presented in this paper aims to re-define the lifecycle and introduce tangible improvements to its outcomes.
△ Less
Submitted 18 October, 2019;
originally announced October 2019.
