Showing 1–27 of 27 results for author: Flynn, D

Uncovering implementable dormant pruning decisions from three different stakeholder perspectives

Authors: Deanna Flynn, Abhinav Jain, Heather Knight, Cristina G. Wilson, Cindy Grimm

Abstract: Dormant pruning, or the removal of unproductive portions of a tree while a tree is not actively growing, is an important orchard task to help maintain yield, requiring years to build expertise. Because of long training periods and an increasing labor shortage in agricultural jobs, pruning could benefit from robotic automation. However, to program robots to prune branches, we first need to understa… ▽ More Dormant pruning, or the removal of unproductive portions of a tree while a tree is not actively growing, is an important orchard task to help maintain yield, requiring years to build expertise. Because of long training periods and an increasing labor shortage in agricultural jobs, pruning could benefit from robotic automation. However, to program robots to prune branches, we first need to understand how pruning decisions are made, and what variables in the environment (e.g., branch size and thickness) we need to capture. Working directly with three pruning stakeholders -- horticulturists, growers, and pruners -- we find that each group of human experts approaches pruning decision-making differently. To capture this knowledge, we present three studies and two extracted pruning protocols from field work conducted in Prosser, Washington in January 2022 and 2023. We interviewed six stakeholders (two in each group) and observed pruning across three cultivars -- Bing Cherries, Envy Apples, and Jazz Apples -- and two tree architectures -- Upright Fruiting Offshoot and V-Trellis. Leveraging participant interviews and video data, this analysis uses grounded coding to extract pruning terminology, discover horticultural contexts that influence pruning decisions, and find implementable pruning heuristics for autonomous systems. The results include a validated terminology set, which we offer for use by both pruning stakeholders and roboticists, to communicate general pruning concepts and heuristics. The results also highlight seven pruning heuristics utilizing this terminology set that would be relevant for use by future autonomous robot pruning systems, and characterize three discovered horticultural contexts (i.e., environmental management, crop-load management, and replacement wood) across all three cultivars. △ Less

Submitted 7 May, 2024; originally announced May 2024.

Comments: 36 pages; 21 figures

Modelling the Formation of Peer-to-Peer Trading Coalitions and Prosumer Participation Incentives in Transactive Energy Communities

Authors: Ying Zhang, Valentin Robu, Sho Cremers, Sonam Norbu, Benoit Couraud, Merlinda Andoni, David Flynn, H. Vincent Poor

Abstract: Peer-to-peer (P2P) energy trading and energy communities have garnered much attention over in recent years due to increasing investments in local energy generation and storage assets. However, the efficiency to be gained from P2P trading, and the structure of local energy markets raise many important challenges. To analyse the efficiency of P2P energy markets, in this work, we consider two differe… ▽ More Peer-to-peer (P2P) energy trading and energy communities have garnered much attention over in recent years due to increasing investments in local energy generation and storage assets. However, the efficiency to be gained from P2P trading, and the structure of local energy markets raise many important challenges. To analyse the efficiency of P2P energy markets, in this work, we consider two different popular approaches to peer-to-peer trading: centralised (through a central market maker/clearing entity) vs. fully decentralised (P2P), and explore the comparative economic benefits of these models. We focus on the metric of Gains from Trade (GT), given optimal P2P trading schedule computed by a schedule optimiser. In both local market models, benefits from trading are realised mainly due to the diversity in consumption behaviour and renewable energy generation between prosumers in an energy community. Both market models will lead to the most promising P2P contracts (the ones with the highest Gains from Trade) to be established first. Yet, we find diversity decreases quickly as more peer-to-peer energy contracts are established and more prosumers join the market, leading to significantly diminishing returns. In this work, we aim to quantify this effect using real-world data from two large-scale smart energy trials in the UK, i.e. the Low Carbon London project and the Thames Valley Vision project. Our experimental study shows that, for both market models, only a small number of P2P contracts, and only a fraction of total prosumers in the community are required to achieve the majority of the maximal potential Gains from Trade. We also study the effect that diversity in consumption profiles has on overall trading potential and dynamics in an energy community. △ Less

Submitted 18 November, 2023; originally announced November 2023.

Journal ref: Authors' pre-print of paper published in Applied Energy, Vol. 355 (1 February 2024), 122173

Non-contact Sensing for Anomaly Detection in Wind Turbine Blades: A focus-SVDD with Complex-Valued Auto-Encoder Approach

Authors: Gaëtan Frusque, Daniel Mitchell, Jamie Blanche, David Flynn, Olga Fink

Abstract: The occurrence of manufacturing defects in wind turbine blade (WTB) production can result in significant increases in operation and maintenance costs and lead to severe and disastrous consequences. Therefore, inspection during the manufacturing process is crucial to ensure consistent fabrication of composite materials. Non-contact sensing techniques, such as Frequency Modulated Continuous Wave (FM… ▽ More The occurrence of manufacturing defects in wind turbine blade (WTB) production can result in significant increases in operation and maintenance costs and lead to severe and disastrous consequences. Therefore, inspection during the manufacturing process is crucial to ensure consistent fabrication of composite materials. Non-contact sensing techniques, such as Frequency Modulated Continuous Wave (FMCW) radar, are becoming increasingly popular as they offer a full view of these complex structures during curing. In this paper, we enhance the quality assurance of manufacturing utilizing FMCW radar as a non-destructive sensing modality. Additionally, a novel anomaly detection pipeline is developed that offers the following advantages: (1) We use the analytic representation of the Intermediate Frequency signal of the FMCW radar as a feature to disentangle material-specific and round-trip delay information from the received wave. (2) We propose a novel anomaly detection methodology called focus Support Vector Data Description (focus-SVDD). This methodology involves defining the limit boundaries of the dataset after removing healthy data features, thereby focusing on the attributes of anomalies. (3) The proposed method employs a complex-valued autoencoder to remove healthy features and we introduces a new activation function called Exponential Amplitude Decay (EAD). EAD takes advantage of the Rayleigh distribution, which characterizes an instantaneous amplitude signal. The effectiveness of the proposed method is demonstrated through its application to collected data, where it shows superior performance compared to other state-of-the-art unsupervised anomaly detection methods. This method is expected to make a significant contribution not only to structural health monitoring but also to the field of deep complex-valued data processing and SVDD application. △ Less

Submitted 19 June, 2023; originally announced June 2023.

Comments: 17 pages, 5 figures, 3 Tables

Bayesian Learning for the Robust Verification of Autonomous Robots

Authors: Xingyu Zhao, Simos Gerasimou, Radu Calinescu, Calum Imrie, Valentin Robu, David Flynn

Abstract: Autonomous robots used in infrastructure inspection, space exploration and other critical missions operate in highly dynamic environments. As such, they must continually verify their ability to complete the tasks associated with these missions safely and effectively. Here we present a Bayesian learning framework that enables this runtime verification of autonomous robots. The framework uses prior… ▽ More Autonomous robots used in infrastructure inspection, space exploration and other critical missions operate in highly dynamic environments. As such, they must continually verify their ability to complete the tasks associated with these missions safely and effectively. Here we present a Bayesian learning framework that enables this runtime verification of autonomous robots. The framework uses prior knowledge and observations of the verified robot to learn expected ranges for the occurrence rates of regular and singular (e.g., catastrophic failure) events. Interval continuous-time Markov models defined using these ranges are then analysed to obtain expected intervals of variation for system properties such as mission duration and success probability. We apply the framework to an autonomous robotic mission for underwater infrastructure inspection and repair. The formal proofs and experiments presented in the paper show that our framework produces results that reflect the uncertainty intrinsic to many real-world systems, enabling the robust verification of their quantitative properties under parametric uncertainty. △ Less

Submitted 11 December, 2023; v1 submitted 15 March, 2023; originally announced March 2023.

Comments: Accepted by Communications Engineering

Efficient Methods for Approximating the Shapley Value for Asset Sharing in Energy Communities

Authors: Sho Cremers, Valentin Robu, Peter Zhang, Merlinda Andoni, Sonam Norbu, David Flynn

Abstract: With the emergence of energy communities, where a number of prosumers invest in shared generation and storage, the issue of fair allocation of benefits is increasingly important. The Shapley value has attracted increasing interest for redistribution in energy settings - however, computing it exactly is intractable beyond a few dozen prosumers. In this paper, we first conduct a systematic review of… ▽ More With the emergence of energy communities, where a number of prosumers invest in shared generation and storage, the issue of fair allocation of benefits is increasingly important. The Shapley value has attracted increasing interest for redistribution in energy settings - however, computing it exactly is intractable beyond a few dozen prosumers. In this paper, we first conduct a systematic review of the literature on the use of Shapley value in energy-related applications, as well as efforts to compute or approximate it. Next, we formalise the main methods for approximating the Shapley value in community energy settings, and propose a new one, which we call the stratified expected value approximation. To compare the performance of these methods, we design a novel method for exact Shapley value computation, which can be applied to communities of up to several hundred agents by clustering the prosumers into a smaller number of demand profiles. We perform a large-scale experimental comparison of the proposed methods, for communities of up to 200 prosumers, using large-scale, publicly available data from two large-scale energy trials in the UK (UKERC Energy Data Centre, 2017, UK Power Networks Innovation, 2021). Our analysis shows that, as the number of agents in the community increases, the relative difference to the exact Shapley value converges to under 1% for all the approximation methods considered. In particular, for most experimental scenarios, we show that there is no statistical difference between the newly proposed stratified expected value method and the existing state-of-the-art method that uses adaptive sampling (O'Brien et al., 2015), although the cost of computation for large communities is an order of magnitude lower. △ Less

Submitted 31 December, 2022; originally announced January 2023.

Comments: 34 pages, 10 figures, Published in Elsevier Applied Energy

Journal ref: Applied Energy, 331 (2022)

Millimeter-Wave Sensing for Avoidance of High-Risk Ground Conditions for Mobile Robots

Authors: Jamie Blanche, Shivoh Chirayil Nandakumar, Daniel Mitchell, Sam Harper, Keir Groves, Andrew West, Barry Lennox, Simon Watson, David Flynn, Ikuo Yamamoto

Abstract: Mobile robot autonomy has made significant advances in recent years, with navigation algorithms well developed and used commercially in certain well-defined environments, such as warehouses. The common link in usage scenarios is that the environments in which the robots are utilized have a high degree of certainty. Operating environments are often designed to be robot friendly, for example augment… ▽ More Mobile robot autonomy has made significant advances in recent years, with navigation algorithms well developed and used commercially in certain well-defined environments, such as warehouses. The common link in usage scenarios is that the environments in which the robots are utilized have a high degree of certainty. Operating environments are often designed to be robot friendly, for example augmented reality markers are strategically placed and the ground is typically smooth, level, and clear of debris. For robots to be useful in a wider range of environments, especially environments that are not sanitized for their use, robots must be able to handle uncertainty. This requires a robot to incorporate new sensors and sources of information, and to be able to use this information to make decisions regarding navigation and the overall mission. When using autonomous mobile robots in unstructured and poorly defined environments, such as a natural disaster site or in a rural environment, ground condition is of critical importance and is a common cause of failure. Examples include loss of traction due to high levels of ground water, hidden cavities, or material boundary failures. To evaluate a non-contact sensing method to mitigate these risks, Frequency Modulated Continuous Wave (FMCW) radar is integrated with an Unmanned Ground Vehicle (UGV), representing a novel application of FMCW to detect new measurands for Robotic Autonomous Systems (RAS) navigation, informing on terrain integrity and adding to the state-of-the-art in sensing for optimized autonomous path planning. In this paper, the FMCW is first evaluated in a desktop setting to determine its performance in anticipated ground conditions. The FMCW is then fixed to a UGV and the sensor system is tested and validated in a representative environment containing regions with significant levels of ground water saturation. △ Less

Submitted 30 March, 2022; originally announced March 2022.

Comments: 6 pages, 9 figures

Addressing Non-Intervention Challenges via Resilient Robotics utilizing a Digital Twin

Authors: Sam Harper, Shivoh Nandakumar, Daniel Mitchell, Jamie Blanche, Theodore Lim, David Flynn

Abstract: Multi-robot systems face challenges in reducing human interventions as they are often deployed in dangerous environments. It is therefore necessary to include a methodology to assess robot failure rates to reduce the requirement for costly human intervention. A solution to this problem includes robots with the ability to work together to ensure mission resilience. To prevent this intervention, rob… ▽ More Multi-robot systems face challenges in reducing human interventions as they are often deployed in dangerous environments. It is therefore necessary to include a methodology to assess robot failure rates to reduce the requirement for costly human intervention. A solution to this problem includes robots with the ability to work together to ensure mission resilience. To prevent this intervention, robots should be able to work together to ensure mission resilience. However, robotic platforms generally lack built-in interconnectivity with other platforms from different vendors. This work aims to tackle this issue by enabling the functionality through a bidirectional digital twin. The twin enables the human operator to transmit and receive information to and from the multi-robot fleet. This digital twin considers mission resilience and autonomous and human-led decision making to enable the resilience of a multi-robot fleet. This creates the cooperation, corroboration, and collaboration of diverse robots to leverage the capability of robots and support recovery of a failed robot. △ Less

Submitted 27 November, 2022; v1 submitted 29 March, 2022; originally announced March 2022.

Comments: 6 pages, 7 figures, preprint conference submission for ICRA 2023

Millimeter-wave Foresight Sensing for Safety and Resilience in Autonomous Operations

Authors: Daniel Mitchell, Jamie Blanche, Sam T. Harper, Theodore Lim, Valentin Robu, Ikuo Yamamoto, David Flynn

Abstract: Robotic platforms are highly programmable, scalable and versatile to complete several tasks including Inspection, Maintenance and Repair (IMR). Mobile robotics offer reduced restrictions in operating environments, resulting in greater flexibility; operation at height, dangerous areas and repetitive tasks. Cyber physical infrastructures have been identified by the UK Robotics Growth Partnership as… ▽ More Robotic platforms are highly programmable, scalable and versatile to complete several tasks including Inspection, Maintenance and Repair (IMR). Mobile robotics offer reduced restrictions in operating environments, resulting in greater flexibility; operation at height, dangerous areas and repetitive tasks. Cyber physical infrastructures have been identified by the UK Robotics Growth Partnership as a key enabler in how we utilize and interact with sensors and machines via the virtual and physical worlds. Cyber Physical Systems (CPS) allow for robotics and artificial intelligence to adapt and repurpose at pace, allowing for the addressment of new challenges in CPS. A challenge exists within robotics to secure an effective partnership in a wide range of areas which include shared workspaces and Beyond Visual Line of Sight (BVLOS). Robotic manipulation abilities have improved a robots accessibility via the ability to open doorways, however, challenges exist in how a robot decides if it is safe to move into a new workspace. Current sensing methods are limited to line of sight and are unable to capture data beyond doorways or walls, therefore, a robot is unable to sense if it is safe to open a door. Another limitation exists as robots are unable to detect if a human is within a shared workspace. Therefore, if a human is detected, extended safety precautions can be taken to ensure the safe autonomous operation of a robot. These challenges are represented as safety, trust and resilience, inhibiting the successful advancement of CPS. This paper evaluates the use of frequency modulated continuous wave radar sensing for human detection and through-wall detection to increase situational awareness. The results validate the use of the sensor to detect the difference between a person and infrastructure, and increased situational awareness for navigation via foresight monitoring through walls. △ Less

Submitted 24 March, 2022; originally announced March 2022.

Comments: 7 pages, 4 figures

Bio-inspired Multi-robot Autonomy

Authors: Shivoh Chirayil Nandakumar, Samuel Harper, Daniel Mitchell, Jamie Blanche, Theodore Lim, Ikuo Yamamoto, David Flynn

Abstract: Increasingly, high value industrial markets are driving trends for improved functionality and resilience from resident autonomous systems. This led to an increase in multi-robot fleets that aim to leverage the complementary attributes of the diverse platforms. In this paper we introduce a novel bio-inspired Symbiotic System of Systems Approach (SSOSA) for designing the operational governance of a… ▽ More Increasingly, high value industrial markets are driving trends for improved functionality and resilience from resident autonomous systems. This led to an increase in multi-robot fleets that aim to leverage the complementary attributes of the diverse platforms. In this paper we introduce a novel bio-inspired Symbiotic System of Systems Approach (SSOSA) for designing the operational governance of a multi-robot fleet consisting of ground-based quadruped and wheeled platforms. SSOSA couples the MR-fleet to the resident infrastructure monitoring systems into one collaborative digital commons. The hyper visibility of the integrated distributed systems, achieved through a latency bidirectional communication network, supports collaboration, coordination and corroboration (3C) across the integrated systems. In our experiment, we demonstrate how an operator can activate a pre-determined autonomous mission and utilize SSOSA to overcome intrinsic and external risks to the autonomous missions. We demonstrate how resilience can be enhanced by local collaboration between SPOT and Husky wherein we detect a replacement battery, and utilize the manipulator arm of SPOT to support a Clearpath Husky A200 wheeled robotic platform. This allows for increased resilience of an autonomous mission as robots can collaborate to ensure the battery state of the Husky robot. Overall, these initial results demonstrate the value of a SSOSA approach in addressing a key operational barrier to scalable autonomy, the resilience. △ Less

Submitted 15 March, 2022; originally announced March 2022.

Comments: A preprint submit to IROS 2022

A Review: Challenges and Opportunities for Artificial Intelligence and Robotics in the Offshore Wind Sector

Authors: Daniel Mitchell, Jamie Blanche, Sam Harper, Theodore Lim, Ranjeetkumar Gupta, Osama Zaki, Wenshuo Tang, Valentin Robu, Simon Watson, David Flynn

Abstract: A global trend in increasing wind turbine size and distances from shore is emerging within the rapidly growing offshore wind farm market. In the UK, the offshore wind sector produced its highest amount of electricity in the UK in 2019, a 19.6% increase on the year before. Currently, the UK is set to increase production further, targeting a 74.7% increase of installed turbine capacity as reflected… ▽ More A global trend in increasing wind turbine size and distances from shore is emerging within the rapidly growing offshore wind farm market. In the UK, the offshore wind sector produced its highest amount of electricity in the UK in 2019, a 19.6% increase on the year before. Currently, the UK is set to increase production further, targeting a 74.7% increase of installed turbine capacity as reflected in recent Crown Estate leasing rounds. With such tremendous growth, the sector is now looking to Robotics and Artificial Intelligence (RAI) in order to tackle lifecycle service barriers as to support sustainable and profitable offshore wind energy production. Today, RAI applications are predominately being used to support short term objectives in operation and maintenance. However, moving forward, RAI has the potential to play a critical role throughout the full lifecycle of offshore wind infrastructure, from surveying, planning, design, logistics, operational support, training and decommissioning. This paper presents one of the first systematic reviews of RAI for the offshore renewable energy sector. The state-of-the-art in RAI is analyzed with respect to offshore energy requirements, from both industry and academia, in terms of current and future requirements. Our review also includes a detailed evaluation of investment, regulation and skills development required to support the adoption of RAI. The key trends identified through a detailed analysis of patent and academic publication databases provide insights to barriers such as certification of autonomous platforms for safety compliance and reliability, the need for digital architectures for scalability in autonomous fleets, adaptive mission planning for resilient resident operations and optimization of human machine interaction for trusted partnerships between people and autonomous assistants. △ Less

Submitted 13 December, 2021; originally announced December 2021.

Comments: 36 figures, 49 pages

Assessing the Reliability of Deep Learning Classifiers Through Robustness Evaluation and Operational Profiles

Authors: Xingyu Zhao, Wei Huang, Alec Banks, Victoria Cox, David Flynn, Sven Schewe, Xiaowei Huang

Abstract: The utilisation of Deep Learning (DL) is advancing into increasingly more sophisticated applications. While it shows great potential to provide transformational capabilities, DL also raises new challenges regarding its reliability in critical functions. In this paper, we present a model-agnostic reliability assessment method for DL classifiers, based on evidence from robustness evaluation and the… ▽ More The utilisation of Deep Learning (DL) is advancing into increasingly more sophisticated applications. While it shows great potential to provide transformational capabilities, DL also raises new challenges regarding its reliability in critical functions. In this paper, we present a model-agnostic reliability assessment method for DL classifiers, based on evidence from robustness evaluation and the operational profile (OP) of a given application. We partition the input space into small cells and then "assemble" their robustness (to the ground truth) according to the OP, where estimators on the cells' robustness and OPs are provided. Reliability estimates in terms of the probability of misclassification per input (pmi) can be derived together with confidence levels. A prototype tool is demonstrated with simplified case studies. Model assumptions and extension to real-world applications are also discussed. While our model easily uncovers the inherent difficulties of assessing the DL dependability (e.g. lack of data with ground truth and scalability issues), we provide preliminary/compromised solutions to advance in this research direction. △ Less

Submitted 2 June, 2021; originally announced June 2021.

Comments: Accepted by the AISafety'21 Workshop at IJCAI-21. To appear in a volume of CEUR Workshop Proceedings

Machine learning pipeline for battery state of health estimation

Authors: Darius Roman, Saurabh Saxena, Valentin Robu, Michael Pecht, David Flynn

Abstract: Lithium-ion batteries are ubiquitous in modern day applications ranging from portable electronics to electric vehicles. Irrespective of the application, reliable real-time estimation of battery state of health (SOH) by on-board computers is crucial to the safe operation of the battery, ultimately safeguarding asset integrity. In this paper, we design and evaluate a machine learning pipeline for es… ▽ More Lithium-ion batteries are ubiquitous in modern day applications ranging from portable electronics to electric vehicles. Irrespective of the application, reliable real-time estimation of battery state of health (SOH) by on-board computers is crucial to the safe operation of the battery, ultimately safeguarding asset integrity. In this paper, we design and evaluate a machine learning pipeline for estimation of battery capacity fade - a metric of battery health - on 179 cells cycled under various conditions. The pipeline estimates battery SOH with an associated confidence interval by using two parametric and two non-parametric algorithms. Using segments of charge voltage and current curves, the pipeline engineers 30 features, performs automatic feature selection and calibrates the algorithms. When deployed on cells operated under the fast-charging protocol, the best model achieves a root mean squared percent error of 0.45\%. This work provides insights into the design of scalable data-driven models for battery SOH estimation, emphasising the value of confidence bounds around the prediction. The pipeline methodology combines experimental data with machine learning modelling and can be generalized to other critical components that require real-time estimation of SOH. △ Less

Submitted 1 February, 2021; originally announced February 2021.

Comments: Peer review, pre-print to be published in Nature Machine Intelligence - 32 pages and 24 figures (including supplementary material)

ACM Class: C.4; I.5.1; I.2.6

Symbiotic System of Systems Design for Safe and Resilient Autonomous Robotics in Offshore Wind Farms

Authors: Daniel Mitchell, Jamie Blanche, Osama Zaki, Joshua Roe, Leo Kong, Samuel Harper, Valentin Robu, Theodore Lim, David Flynn

Abstract: To reduce Operation and Maintenance (O&M) costs on offshore wind farms, wherein 80% of the O&M cost relates to deploying personnel, the offshore wind sector looks to Robotics and Artificial Intelligence (RAI) for solutions. Barriers to Beyond Visual Line of Sight (BVLOS) robotics include operational safety compliance and resilience, inhibiting the commercialization of autonomous services offshore.… ▽ More To reduce Operation and Maintenance (O&M) costs on offshore wind farms, wherein 80% of the O&M cost relates to deploying personnel, the offshore wind sector looks to Robotics and Artificial Intelligence (RAI) for solutions. Barriers to Beyond Visual Line of Sight (BVLOS) robotics include operational safety compliance and resilience, inhibiting the commercialization of autonomous services offshore. To address safety and resilience challenges we propose a Symbiotic System Of Systems Approach (SSOSA), reflecting the lifecycle learning and co-evolution with knowledge sharing for mutual gain of robotic platforms and remote human operators. Our novel methodology enables the run-time verification of safety, reliability and resilience during autonomous missions. To achieve this, a Symbiotic Digital Architecture (SDA) was developed to synchronize digital models of the robot, environment, infrastructure, and integrate front-end analytics and bidirectional communication for autonomous adaptive mission planning and situation reporting to a remote operator. A reliability ontology for the deployed robot, based on our holistic hierarchical-relational model, supports computationally efficient platform data analysis. We demonstrate an asset inspection mission within a confined space through Cooperative, Collaborative and Corroborative (C3) governance (internal and external symbiosis) via decision-making processes and the associated structures. We create a hyper enabled human interaction capability to analyze the mission status, diagnostics of critical sub-systems within the robot to provide automatic updates to our AI-driven run-time reliability ontology. This enables faults to be translated into failure modes for decision-making during the mission. △ Less

Submitted 22 July, 2021; v1 submitted 23 January, 2021; originally announced January 2021.

Comments: A preprint submit to IEEE Access Reliability Society Section

BayLIME: Bayesian Local Interpretable Model-Agnostic Explanations

Authors: Xingyu Zhao, Wei Huang, Xiaowei Huang, Valentin Robu, David Flynn

Abstract: Given the pressing need for assuring algorithmic transparency, Explainable AI (XAI) has emerged as one of the key areas of AI research. In this paper, we develop a novel Bayesian extension to the LIME framework, one of the most widely used approaches in XAI -- which we call BayLIME. Compared to LIME, BayLIME exploits prior knowledge and Bayesian reasoning to improve both the consistency in repeate… ▽ More Given the pressing need for assuring algorithmic transparency, Explainable AI (XAI) has emerged as one of the key areas of AI research. In this paper, we develop a novel Bayesian extension to the LIME framework, one of the most widely used approaches in XAI -- which we call BayLIME. Compared to LIME, BayLIME exploits prior knowledge and Bayesian reasoning to improve both the consistency in repeated explanations of a single prediction and the robustness to kernel settings. BayLIME also exhibits better explanation fidelity than the state-of-the-art (LIME, SHAP and GradCAM) by its ability to integrate prior knowledge from, e.g., a variety of other XAI techniques, as well as verification and validation (V&V) methods. We demonstrate the desirable properties of BayLIME through both theoretical analysis and extensive experiments. △ Less

Submitted 29 May, 2021; v1 submitted 5 December, 2020; originally announced December 2020.

Comments: Preprint accepted by UAI2021. The final version to appear in the UAI2021 volume of Proceedings of Machine Learning Research

Assessing Safety-Critical Systems from Operational Testing: A Study on Autonomous Vehicles

Authors: Xingyu Zhao, Kizito Salako, Lorenzo Strigini, Valentin Robu, David Flynn

Abstract: Context: Demonstrating high reliability and safety for safety-critical systems (SCSs) remains a hard problem. Diverse evidence needs to be combined in a rigorous way: in particular, results of operational testing with other evidence from design and verification. Growing use of machine learning in SCSs, by precluding most established methods for gaining assurance, makes operational testing even mor… ▽ More Context: Demonstrating high reliability and safety for safety-critical systems (SCSs) remains a hard problem. Diverse evidence needs to be combined in a rigorous way: in particular, results of operational testing with other evidence from design and verification. Growing use of machine learning in SCSs, by precluding most established methods for gaining assurance, makes operational testing even more important for supporting safety and reliability claims. Objective: We use Autonomous Vehicles (AVs) as a current example to revisit the problem of demonstrating high reliability. AVs are making their debut on public roads: methods for assessing whether an AV is safe enough are urgently needed. We demonstrate how to answer 5 questions that would arise in assessing an AV type, starting with those proposed by a highly-cited study. Method: We apply new theorems extending Conservative Bayesian Inference (CBI), which exploit the rigour of Bayesian methods while reducing the risk of involuntary misuse associated with now-common applications of Bayesian inference; we define additional conditions needed for applying these methods to AVs. Results: Prior knowledge can bring substantial advantages if the AV design allows strong expectations of safety before road testing. We also show how naive attempts at conservative assessment may lead to over-optimism instead; why extrapolating the trend of disengagements is not suitable for safety claims; use of knowledge that an AV has moved to a less stressful environment. Conclusion: While some reliability targets will remain too high to be practically verifiable, CBI removes a major source of doubt: it allows use of prior knowledge without inducing dangerously optimistic biases. For certain ranges of required reliability and prior beliefs, CBI thus supports feasible, sound arguments. Useful conservative claims can be derived from limited prior knowledge. △ Less

Submitted 19 August, 2020; originally announced August 2020.

Comments: Accepted by Information and Software Technology. arXiv admin note: substantial text overlap with arXiv:1908.06540

VPR-Bench: An Open-Source Visual Place Recognition Evaluation Framework with Quantifiable Viewpoint and Appearance Change

Authors: Mubariz Zaffar, Sourav Garg, Michael Milford, Julian Kooij, David Flynn, Klaus McDonald-Maier, Shoaib Ehsan

Abstract: Visual Place Recognition (VPR) is the process of recognising a previously visited place using visual information, often under varying appearance conditions and viewpoint changes and with computational constraints. VPR is related to the concepts of localisation, loop closure, image retrieval and is a critical component of many autonomous navigation systems ranging from autonomous vehicles to drones… ▽ More Visual Place Recognition (VPR) is the process of recognising a previously visited place using visual information, often under varying appearance conditions and viewpoint changes and with computational constraints. VPR is related to the concepts of localisation, loop closure, image retrieval and is a critical component of many autonomous navigation systems ranging from autonomous vehicles to drones and computer vision systems. While the concept of place recognition has been around for many years, VPR research has grown rapidly as a field over the past decade due to improving camera hardware and its potential for deep learning-based techniques, and has become a widely studied topic in both the computer vision and robotics communities. This growth however has led to fragmentation and a lack of standardisation in the field, especially concerning performance evaluation. Moreover, the notion of viewpoint and illumination invariance of VPR techniques has largely been assessed qualitatively and hence ambiguously in the past. In this paper, we address these gaps through a new comprehensive open-source framework for assessing the performance of VPR techniques, dubbed "VPR-Bench". VPR-Bench (Open-sourced at: https://github.com/MubarizZaffar/VPR-Bench) introduces two much-needed capabilities for VPR researchers: firstly, it contains a benchmark of 12 fully-integrated datasets and 10 VPR techniques, and secondly, it integrates a comprehensive variation-quantified dataset for quantifying viewpoint and illumination invariance. We apply and analyse popular evaluation metrics for VPR from both the computer vision and robotics communities, and discuss how these different metrics complement and/or replace each other, depending upon the underlying applications and system requirements. △ Less

Submitted 1 October, 2021; v1 submitted 16 May, 2020; originally announced May 2020.

Comments: Accepted version of our IJCV paper

Journal ref: International Journal of Computer Vision. 2021 May 7:1-39

A Safety Framework for Critical Systems Utilising Deep Neural Networks

Authors: Xingyu Zhao, Alec Banks, James Sharp, Valentin Robu, David Flynn, Michael Fisher, Xiaowei Huang

Abstract: Increasingly sophisticated mathematical modelling processes from Machine Learning are being used to analyse complex data. However, the performance and explainability of these models within practical critical systems requires a rigorous and continuous verification of their safe utilisation. Working towards addressing this challenge, this paper presents a principled novel safety argument framework f… ▽ More Increasingly sophisticated mathematical modelling processes from Machine Learning are being used to analyse complex data. However, the performance and explainability of these models within practical critical systems requires a rigorous and continuous verification of their safe utilisation. Working towards addressing this challenge, this paper presents a principled novel safety argument framework for critical systems that utilise deep neural networks. The approach allows various forms of predictions, e.g., future reliability of passing some demands, or confidence on a required reliability level. It is supported by a Bayesian analysis using operational data and the recent verification and validation techniques for deep learning. The prediction is conservative -- it starts with partial prior knowledge obtained from lifecycle activities and then determines the worst-case prediction. Open challenges are also identified. △ Less

Submitted 6 June, 2020; v1 submitted 7 March, 2020; originally announced March 2020.

Comments: Accepted by SafeComp2020

Consider ethical and social challenges in smart grid research

Authors: Valentin Robu, David Flynn, Merlinda Andoni, Maizura Mokhtar

Abstract: Artificial Intelligence and Machine Learning are increasingly seen as key technologies for building more decentralised and resilient energy grids, but researchers must consider the ethical and social implications of their use Artificial Intelligence and Machine Learning are increasingly seen as key technologies for building more decentralised and resilient energy grids, but researchers must consider the ethical and social implications of their use △ Less

Submitted 26 November, 2019; originally announced December 2019.

Comments: Preprint of paper published in Nature Machine Intelligence, vol. 1 (25 Nov. 2019)

Journal ref: Nature Machine Intelligence, vol. 1, 25 November 2019

Towards Integrating Formal Verification of Autonomous Robots with Battery Prognostics and Health Management

Authors: Xingyu Zhao, Matt Osborne, Jenny Lantair, Valentin Robu, David Flynn, Xiaowei Huang, Michael Fisher, Fabio Papacchini, Angelo Ferrando

Abstract: The battery is a key component of autonomous robots. Its performance limits the robot's safety and reliability. Unlike liquid-fuel, a battery, as a chemical device, exhibits complicated features, including (i) capacity fade over successive recharges and (ii) increasing discharge rate as the state of charge (SOC) goes down for a given power demand. Existing formal verification studies of autonomous… ▽ More The battery is a key component of autonomous robots. Its performance limits the robot's safety and reliability. Unlike liquid-fuel, a battery, as a chemical device, exhibits complicated features, including (i) capacity fade over successive recharges and (ii) increasing discharge rate as the state of charge (SOC) goes down for a given power demand. Existing formal verification studies of autonomous robots, when considering energy constraints, formalise the energy component in a generic manner such that the battery features are overlooked. In this paper, we model an unmanned aerial vehicle (UAV) inspection mission on a wind farm and via probabilistic model checking in PRISM show (i) how the battery features may affect the verification results significantly in practical cases; and (ii) how the battery features, together with dynamic environments and battery safety strategies, jointly affect the verification results. Potential solutions to explicitly integrate battery prognostics and health management (PHM) with formal verification of autonomous robots are also discussed to motivate future work. △ Less

Submitted 22 August, 2019; originally announced September 2019.

Journal ref: Proceedings of 17th International Conference on Software Engineering and Formal Methods (SEFM 2019), Oslo, Norway (September 2019)

Gibbs sampling for game-theoretic modeling of private network upgrades with distributed generation

Authors: Merlinda Andoni, Valentin Robu, David Flynn, Wolf-Gerrit Fruh

Abstract: Renewable energy is increasingly being curtailed, due to oversupply or network constraints. Curtailment can be partially avoided by smart grid management, but the long term solution is network reinforcement. Network upgrades, however, can be costly, so recent interest has focused on incentivising private investors to participate in network investments. In this paper, we study settings where a priv… ▽ More Renewable energy is increasingly being curtailed, due to oversupply or network constraints. Curtailment can be partially avoided by smart grid management, but the long term solution is network reinforcement. Network upgrades, however, can be costly, so recent interest has focused on incentivising private investors to participate in network investments. In this paper, we study settings where a private renewable investor constructs a power line, but also provides access to other generators that pay a transmission fee. The decisions on optimal (and interdependent) renewable capacities built by investors, affect the resulting curtailment and profitability of projects, and can be formulated as a Stackelberg game. Optimal capacities rely jointly on stochastic variables, such as the renewable resource at project location. In this paper, we show how Markov chain Monte Carlo (MCMC) and Gibbs sampling techniques, can be used to generate observations from historic resource data and simulate multiple future scenarios. Finally, we validate and apply our game-theoretic formulation of the investment decision, to a real network upgrade problem in the UK. △ Less

Submitted 22 August, 2019; originally announced August 2019.

Comments: Preprint of final submitted version. arXiv admin note: text overlap with arXiv:1908.10313

Journal ref: Proceedings of 2018 IEEE PES Innovative Smart Grid Technologies Conference Europe (ISGT-Europe)

Game-theoretic modeling of curtailment rules and network investments with distributed generation

Authors: Merlinda Andoni, Valentin Robu, Wolf-Gerrit Fruh, David Flynn

Abstract: Renewable energy has achieved high penetration rates in many areas, leading to curtailment, especially if existing network infrastructure is insufficient and energy generated cannot be exported. In this context, Distribution Network Operators (DNOs) face a significant knowledge gap about how to implement curtailment rules that achieve desired operational objectives, but at the same time minimise d… ▽ More Renewable energy has achieved high penetration rates in many areas, leading to curtailment, especially if existing network infrastructure is insufficient and energy generated cannot be exported. In this context, Distribution Network Operators (DNOs) face a significant knowledge gap about how to implement curtailment rules that achieve desired operational objectives, but at the same time minimise disruption and economic losses for renewable generators. In this work, we study the properties of several curtailment rules widely used in UK renewable energy projects, and their effect on the viability of renewable generation investment. Moreover, we propose a new curtailment rule which guarantees fair allocation of curtailment amongst all generators with minimal disruption. Another key knowledge gap faced by DNOs is how to incentivise private network upgrades, especially in settings where several generators can use the same line against the payment of a transmission fee. In this work, we provide a solution to this problem by using tools from algorithmic game theory. Specifically, this setting can be modelled as a Stackelberg game between the private transmission line investor and local renewable generators, who are required to pay a transmission fee to access the line. We provide a method for computing the empirical equilibrium of this game, using a model that captures the stochastic nature of renewable energy generation and demand. Finally, we use the practical setting of a grid reinforcement project from the UK and a large dataset of wind speed measurements and demand to validate our model. We show that charging a transmission fee as a proportion of the feed-in tariff price between 15%-75% would allow both investors to implement their projects and achieve desirable distribution of the profit. △ Less

Submitted 22 August, 2019; originally announced August 2019.

Comments: Preprint of final submitted version

Journal ref: Applied Energy, Volume 201, 1 September 2017, Pages 174-187

Assessing the Safety and Reliability of Autonomous Vehicles from Road Testing

Authors: Xingyu Zhao, Valentin Robu, David Flynn, Kizito Salako, Lorenzo Strigini

Abstract: There is an urgent societal need to assess whether autonomous vehicles (AVs) are safe enough. From published quantitative safety and reliability assessments of AVs, we know that, given the goal of predicting very low rates of accidents, road testing alone requires infeasible numbers of miles to be driven. However, previous analyses do not consider any knowledge prior to road testing - knowledge wh… ▽ More There is an urgent societal need to assess whether autonomous vehicles (AVs) are safe enough. From published quantitative safety and reliability assessments of AVs, we know that, given the goal of predicting very low rates of accidents, road testing alone requires infeasible numbers of miles to be driven. However, previous analyses do not consider any knowledge prior to road testing - knowledge which could bring substantial advantages if the AV design allows strong expectations of safety before road testing. We present the advantages of a new variant of Conservative Bayesian Inference (CBI), which uses prior knowledge while avoiding optimistic biases. We then study the trend of disengagements (take-overs by human drivers) by applying Software Reliability Growth Models (SRGMs) to data from Waymo's public road testing over 51 months, in view of the practice of software updates during this testing. Our approach is to not trust any specific SRGM, but to assess forecast accuracy and then improve forecasts. We show that, coupled with accuracy assessment and recalibration techniques, SRGMs could be a valuable test planning aid. △ Less

Submitted 18 August, 2019; originally announced August 2019.

Journal ref: Proceedings of 30th IEEE International Symposium on Software Reliability Engineering (ISSRE 2019)

Predicting the Voltage Distribution for Low Voltage Networks using Deep Learning

Authors: Maizura Mokhtar, Valentin Robu, David Flynn, Ciaran Higgins, Jim Whyte, Caroline Loughran, Fiona Fulton

Abstract: The energy landscape for the Low-Voltage (LV) networks are beginning to change; changes resulted from the increase penetration of renewables and/or the predicted increase of electric vehicles charging at home. The previously passive `fit-and-forget' approach to LV network management will be inefficient to ensure its effective operations. A more adaptive approach is required that includes the predi… ▽ More The energy landscape for the Low-Voltage (LV) networks are beginning to change; changes resulted from the increase penetration of renewables and/or the predicted increase of electric vehicles charging at home. The previously passive `fit-and-forget' approach to LV network management will be inefficient to ensure its effective operations. A more adaptive approach is required that includes the prediction of risk and capacity of the circuits. Many of the proposed methods require full observability of the networks, motivating the installations of smart meters and advance metering infrastructure in many countries. However, the expectation of `perfect data' is unrealistic in operational reality. Smart meter (SM) roll-out can have its issues, which may resulted in low-likelihood of full SM coverage for all LV networks. This, together with privacy requirements that limit the availability of high granularity demand power data have resulted in the low uptake of many of the presented methods. To address this issue, Deep Learning Neural Network is proposed to predict the voltage distribution with partial SM coverage. The results show that SM measurements from key locations are sufficient for effective prediction of voltage distribution. △ Less

Submitted 19 June, 2019; originally announced June 2019.

Comments: 9th IEEE International Conference on Innovative Smart Grid Technologies (IEEE ISGT Europe 2019)

Probabilistic Model Checking of Robots Deployed in Extreme Environments

Authors: Xingyu Zhao, Valentin Robu, David Flynn, Fateme Dinmohammadi, Michael Fisher, Matt Webster

Abstract: Robots are increasingly used to carry out critical missions in extreme environments that are hazardous for humans. This requires a high degree of operational autonomy under uncertain conditions, and poses new challenges for assuring the robot's safety and reliability. In this paper, we develop a framework for probabilistic model checking on a layered Markov model to verify the safety and reliabili… ▽ More Robots are increasingly used to carry out critical missions in extreme environments that are hazardous for humans. This requires a high degree of operational autonomy under uncertain conditions, and poses new challenges for assuring the robot's safety and reliability. In this paper, we develop a framework for probabilistic model checking on a layered Markov model to verify the safety and reliability requirements of such robots, both at pre-mission stage and during runtime. Two novel estimators based on conservative Bayesian inference and imprecise probability model with sets of priors are introduced to learn the unknown transition parameters from operational data. We demonstrate our approach using data from a real-world deployment of unmanned underwater vehicles in extreme environments. △ Less

Submitted 15 February, 2019; v1 submitted 10 December, 2018; originally announced December 2018.

Comments: Version accepted at the 33rd AAAI Conference on Artificial Intelligence, Honolulu, Hawaii, 2019

Lidar Cloud Detection with Fully Convolutional Networks

Authors: Erol Cromwell, Donna Flynn

Abstract: In this contribution, we present a novel approach for segmenting laser radar (lidar) imagery into geometric time-height cloud locations with a fully convolutional network (FCN). We describe a semi-supervised learning method to train the FCN by: pre-training the classification layers of the FCN with image-level annotations, pre-training the entire FCN with the cloud locations of the MPLCMASK cloud… ▽ More In this contribution, we present a novel approach for segmenting laser radar (lidar) imagery into geometric time-height cloud locations with a fully convolutional network (FCN). We describe a semi-supervised learning method to train the FCN by: pre-training the classification layers of the FCN with image-level annotations, pre-training the entire FCN with the cloud locations of the MPLCMASK cloud mask algorithm, and fully supervised learning with hand-labeled cloud locations. We show the model achieves higher levels of cloud identification compared to the cloud mask algorithm implementation. △ Less

Submitted 11 July, 2018; v1 submitted 2 May, 2018; originally announced May 2018.

Comments: Updated for full version of paper. 10 pages, submitted to NIPS 2018 Conference (in review)

Temporal Convolutional Networks for Action Segmentation and Detection

Authors: Colin Lea, Michael D. Flynn, Rene Vidal, Austin Reiter, Gregory D. Hager

Abstract: The ability to identify and temporally segment fine-grained human actions throughout a video is crucial for robotics, surveillance, education, and beyond. Typical approaches decouple this problem by first extracting local spatiotemporal features from video frames and then feeding them into a temporal classifier that captures high-level temporal patterns. We introduce a new class of temporal models… ▽ More The ability to identify and temporally segment fine-grained human actions throughout a video is crucial for robotics, surveillance, education, and beyond. Typical approaches decouple this problem by first extracting local spatiotemporal features from video frames and then feeding them into a temporal classifier that captures high-level temporal patterns. We introduce a new class of temporal models, which we call Temporal Convolutional Networks (TCNs), that use a hierarchy of temporal convolutions to perform fine-grained action segmentation or detection. Our Encoder-Decoder TCN uses pooling and upsampling to efficiently capture long-range temporal patterns whereas our Dilated TCN uses dilated convolutions. We show that TCNs are capable of capturing action compositions, segment durations, and long-range dependencies, and are over a magnitude faster to train than competing LSTM-based Recurrent Neural Networks. We apply these models to three challenging fine-grained datasets and show large improvements over the state of the art. △ Less

Submitted 16 November, 2016; originally announced November 2016.

Design Methodology and Manufacture of a Microinductor

Authors: D. Flynn, Marc Desmulliez

Abstract: Potential core materials to supersede ferrite in the 0.5-10 MHz frequency range are investigated. The performance of electrodeposited nickel-iron, cobalt-iron-copper alloys and the commercial alloy Vitrovac 6025 have been assessed through their inclusion within a custom-made solenoid microinductor. Although the present inductor, at 500 KHz, achieves 77% power efficiency for 24.7W/cm3 power densi… ▽ More Potential core materials to supersede ferrite in the 0.5-10 MHz frequency range are investigated. The performance of electrodeposited nickel-iron, cobalt-iron-copper alloys and the commercial alloy Vitrovac 6025 have been assessed through their inclusion within a custom-made solenoid microinductor. Although the present inductor, at 500 KHz, achieves 77% power efficiency for 24.7W/cm3 power density, an optimized process predicts a power efficiency of 97% for 30.83W/cm3 power density. The principle issues regarding microinductor design and performance are discussed. △ Less

Submitted 7 May, 2008; originally announced May 2008.

Comments: Submitted on behalf of EDA Publishing Association (http://irevues.inist.fr/handle/2042/16838)

Journal ref: Dans Symposium on Design, Test, Integration and Packaging of MEMS/MOEMS - DTIP 2008, Nice : France (2008)