-
Trust in Shared Automated Vehicles: Study on Two Mobility Platforms
Authors:
Shashank Mehrotra,
Jacob G Hunter,
Matthew Konishi,
Kumar Akash,
Zhaobo Zheng,
Teruhisa Misu,
Anil Kumar,
Tahira Reid,
Neera Jain
Abstract:
The ever-increasing adoption of shared transportation modalities across the United States has the potential to fundamentally change the preferences and usage of different mobilities. It also raises several challenges with respect to the design and development of automated mobilities that can enable a large population to take advantage of this emergent technology. One such challenge is the lack of…
▽ More
The ever-increasing adoption of shared transportation modalities across the United States has the potential to fundamentally change the preferences and usage of different mobilities. It also raises several challenges with respect to the design and development of automated mobilities that can enable a large population to take advantage of this emergent technology. One such challenge is the lack of understanding of how trust in one automated mobility may impact trust in another. Without this understanding, it is difficult for researchers to determine whether future mobility solutions will have acceptance within different population groups. This study focuses on identifying the differences in trust across different mobility and how trust evolves across their use for participants who preferred an aggressive driving style. A dual mobility simulator study was designed in which 48 participants experienced two different automated mobilities (car and sidewalk). The results found that participants showed increasing levels of trust when they transitioned from the car to the sidewalk mobility. In comparison, participants showed decreasing levels of trust when they transitioned from the sidewalk to the car mobility. The findings from the study help inform and identify how people can develop trust in future mobility platforms and could inform the design of interventions that may help improve the trust and acceptance of future mobility.
△ Less
Submitted 16 March, 2023;
originally announced March 2023.
-
Localization & Mapping Requirements for Level 2+ Autonomous Vehicles
Authors:
Tyler G. R. Reid,
Andrew Neish,
Brian Manning
Abstract:
Autonomous vehicles are being deployed with a spectrum of capability, extending from driver assistance features for the highway in personal vehicles (SAE Level 2+) to fully autonomous fleet ride sharing services operating in complex city environments (SAE Level 4+). This spectrum of autonomy often operates in different physical environments with different degrees of assumed driver in-the-loop over…
▽ More
Autonomous vehicles are being deployed with a spectrum of capability, extending from driver assistance features for the highway in personal vehicles (SAE Level 2+) to fully autonomous fleet ride sharing services operating in complex city environments (SAE Level 4+). This spectrum of autonomy often operates in different physical environments with different degrees of assumed driver in-the-loop oversight and hence have very different system and subsystem requirements. At the heart of SAE Level 2 to 5 systems is localization and mapping, which ranges from road determination for feature geofencing or high-level routing, through lane determination for advanced driver assistance, to where-in-lane positioning for full vehicle control. We assess localization and mapping requirements for different levels of autonomy and supported features. This work provides a framework for system decomposition, including the level of redundancy needed to achieve the target level of safety. We examine several representative autonomous and assistance features and make recommendations on positioning requirements as well map georeferencing and information integrity.
△ Less
Submitted 5 January, 2023; v1 submitted 15 December, 2022;
originally announced December 2022.
-
Web Archiving as Entertainment
Authors:
Travis Reid,
Michael L. Nelson,
Michele C. Weigle
Abstract:
We want to make web archiving entertaining so that it can be enjoyed like a spectator sport. To this end, we have been working on a proof of concept that involves gamification of the web archiving process and integrating video games and web archiving. Our vision for this proof of concept involves a web archiving live stream and a gaming live stream. We are creating web archiving live streams that…
▽ More
We want to make web archiving entertaining so that it can be enjoyed like a spectator sport. To this end, we have been working on a proof of concept that involves gamification of the web archiving process and integrating video games and web archiving. Our vision for this proof of concept involves a web archiving live stream and a gaming live stream. We are creating web archiving live streams that make the web archiving process more transparent to viewers by live streaming the web archiving and replay sessions to video game live streaming platforms like Twitch, Facebook Gaming, and YouTube. We also want to live stream gameplay from games where the gameplay is influenced by web archiving and replay performance. So far we have created web archiving live streams that show the web archiving and replay sessions for two web archive crawlers and gaming live streams that show gameplay influenced by the web archiving performance from the web archiving live stream. We have also applied the gaming concept of speedruns, where a player attempts to complete a game as quickly as possible. This could make a web archiving live stream more entertaining, because we can have a competition between two crawlers to see which crawler is faster at archiving a set of URIs.
△ Less
Submitted 3 November, 2022;
originally announced November 2022.
-
The Interaction Gap: A Step Toward Understanding Trust in Autonomous Vehicles Between Encounters
Authors:
Jacob G. Hunter,
Matthew Konishi,
Neera Jain,
Kumar Akash,
Xingwei Wu,
Teruhisa Misu,
Tahira Reid
Abstract:
Shared autonomous vehicles (SAVs) will be introduced in greater numbers over the coming decade. Due to rapid advances in shared mobility and the slower development of fully autonomous vehicles (AVs), SAVs will likely be deployed before privately-owned AVs. Moreover, existing shared mobility services are transitioning their vehicle fleets toward those with increasingly higher levels of driving auto…
▽ More
Shared autonomous vehicles (SAVs) will be introduced in greater numbers over the coming decade. Due to rapid advances in shared mobility and the slower development of fully autonomous vehicles (AVs), SAVs will likely be deployed before privately-owned AVs. Moreover, existing shared mobility services are transitioning their vehicle fleets toward those with increasingly higher levels of driving automation. Consequently, people who use shared vehicles on an "as needed" basis will have infrequent interactions with automated driving, thereby experiencing interaction gaps. Using human trust data of 25 participants, we show that interaction gaps can affect human trust in automated driving. Participants engaged in a simulator study consisting of two interactions separated by a one-week interaction gap. A moderate, inverse correlation was found between the change in trust during the initial interaction and the interaction gap, suggesting people "forget" some of their gained trust or distrust in automation during an interaction gap.
△ Less
Submitted 21 September, 2022;
originally announced September 2022.
-
SmOOD: Smoothness-based Out-of-Distribution Detection Approach for Surrogate Neural Networks in Aircraft Design
Authors:
Houssem Ben Braiek,
Ali Tfaily,
Foutse Khomh,
Thomas Reid,
Ciro Guida
Abstract:
Aircraft industry is constantly striving for more efficient design optimization methods in terms of human efforts, computation time, and resource consumption. Hybrid surrogate optimization maintains high results quality while providing rapid design assessments when both the surrogate model and the switch mechanism for eventually transitioning to the HF model are calibrated properly. Feedforward ne…
▽ More
Aircraft industry is constantly striving for more efficient design optimization methods in terms of human efforts, computation time, and resource consumption. Hybrid surrogate optimization maintains high results quality while providing rapid design assessments when both the surrogate model and the switch mechanism for eventually transitioning to the HF model are calibrated properly. Feedforward neural networks (FNNs) can capture highly nonlinear input-output mappings, yielding efficient surrogates for aircraft performance factors. However, FNNs often fail to generalize over the out-of-distribution (OOD) samples, which hinders their adoption in critical aircraft design optimization. Through SmOOD, our smoothness-based out-of-distribution detection approach, we propose to codesign a model-dependent OOD indicator with the optimized FNN surrogate, to produce a trustworthy surrogate model with selective but credible predictions. Unlike conventional uncertainty-grounded methods, SmOOD exploits inherent smoothness properties of the HF simulations to effectively expose OODs through revealing their suspicious sensitivities, thereby avoiding over-confident uncertainty estimates on OOD samples. By using SmOOD, only high-risk OOD inputs are forwarded to the HF model for re-evaluation, leading to more accurate results at a low overhead cost. Three aircraft performance models are investigated. Results show that FNN-based surrogates outperform their Gaussian Process counterparts in terms of predictive performance. Moreover, SmOOD does cover averagely 85% of actual OODs on all the study cases. When SmOOD plus FNN surrogates are deployed in hybrid surrogate optimization settings, they result in a decrease error rate of 34.65% and a computational speed up rate of 58.36 times, respectively.
△ Less
Submitted 7 September, 2022;
originally announced September 2022.
-
Physics-Guided Adversarial Machine Learning for Aircraft Systems Simulation
Authors:
Houssem Ben Braiek,
Thomas Reid,
Foutse Khomh
Abstract:
In the context of aircraft system performance assessment, deep learning technologies allow to quickly infer models from experimental measurements, with less detailed system knowledge than usually required by physics-based modeling. However, this inexpensive model development also comes with new challenges regarding model trustworthiness. This work presents a novel approach, physics-guided adversar…
▽ More
In the context of aircraft system performance assessment, deep learning technologies allow to quickly infer models from experimental measurements, with less detailed system knowledge than usually required by physics-based modeling. However, this inexpensive model development also comes with new challenges regarding model trustworthiness. This work presents a novel approach, physics-guided adversarial machine learning (ML), that improves the confidence over the physics consistency of the model. The approach performs, first, a physics-guided adversarial testing phase to search for test inputs revealing behavioral system inconsistencies, while still falling within the range of foreseeable operational conditions. Then, it proceeds with physics-informed adversarial training to teach the model the system-related physics domain foreknowledge through iteratively reducing the unwanted output deviations on the previously-uncovered counterexamples. Empirical evaluation on two aircraft system performance models shows the effectiveness of our adversarial ML approach in exposing physical inconsistencies of both models and in improving their propensity to be consistent with physics domain knowledge.
△ Less
Submitted 7 September, 2022;
originally announced September 2022.
-
Probabilistic Iterative Methods for Linear Systems
Authors:
Jon Cockayne,
Ilse C. F. Ipsen,
Chris J. Oates,
Tim W. Reid
Abstract:
This paper presents a probabilistic perspective on iterative methods for approximating the solution $\mathbf{x}_* \in \mathbb{R}^d$ of a nonsingular linear system $\mathbf{A} \mathbf{x}_* = \mathbf{b}$. In the approach a standard iterative method on $\mathbb{R}^d$ is lifted to act on the space of probability distributions $\mathcal{P}(\mathbb{R}^d)$. Classically, an iterative method produces a seq…
▽ More
This paper presents a probabilistic perspective on iterative methods for approximating the solution $\mathbf{x}_* \in \mathbb{R}^d$ of a nonsingular linear system $\mathbf{A} \mathbf{x}_* = \mathbf{b}$. In the approach a standard iterative method on $\mathbb{R}^d$ is lifted to act on the space of probability distributions $\mathcal{P}(\mathbb{R}^d)$. Classically, an iterative method produces a sequence $\mathbf{x}_m$ of approximations that converge to $\mathbf{x}_*$. The output of the iterative methods proposed in this paper is, instead, a sequence of probability distributions $μ_m \in \mathcal{P}(\mathbb{R}^d)$. The distributional output both provides a "best guess" for $\mathbf{x}_*$, for example as the mean of $μ_m$, and also probabilistic uncertainty quantification for the value of $\mathbf{x}_*$ when it has not been exactly determined. Theoretical analysis is provided in the prototypical case of a stationary linear iterative method. In this setting we characterise both the rate of contraction of $μ_m$ to an atomic measure on $\mathbf{x}_*$ and the nature of the uncertainty quantification being provided. We conclude with an empirical illustration that highlights the insight into solution uncertainty that can be provided by probabilistic iterative methods.
△ Less
Submitted 11 January, 2021; v1 submitted 23 December, 2020;
originally announced December 2020.
-
Ford Highway Driving RTK Dataset: 30,000 km of North American Highways
Authors:
Sarah E. Houts,
Nahid Pervez,
Umair Ibrahim,
Gaurav Pandey,
Tyler G. R. Reid
Abstract:
There is a growing need for vehicle positioning information to support Advanced Driver Assistance Systems (ADAS), Connectivity (V2X), and Autonomous Driving (AD) features. These range from a need for road determination ($<$5 meters), lane determination ($<$1.5 meters), and determining where the vehicle is within the lane ($<$0.3 meters). This paper presents the Ford Highway Driving RTK (Ford-HDR)…
▽ More
There is a growing need for vehicle positioning information to support Advanced Driver Assistance Systems (ADAS), Connectivity (V2X), and Autonomous Driving (AD) features. These range from a need for road determination ($<$5 meters), lane determination ($<$1.5 meters), and determining where the vehicle is within the lane ($<$0.3 meters). This paper presents the Ford Highway Driving RTK (Ford-HDR) dataset. This dataset includes nearly 30,000 km of data collected primarily on North American highways during a driving campaign designed to validate driver assistance features in 2018. This includes data from a representative automotive production GNSS used primarily for turn-by-turn navigation as well as an Inertial Navigation System (INS) which couples two survey-grade GNSS receivers with a tactical grade Inertial Measurement Unit (IMU) to act as ground truth. The latter utilized networked Real-Time Kinematic (RTK) GNSS corrections delivered over a cellular modem in real-time. This dataset is being released into the public domain to spark further research in the community.
△ Less
Submitted 5 October, 2020;
originally announced October 2020.
-
Human Trust-based Feedback Control: Dynamically varying automation transparency to optimize human-machine interactions
Authors:
Kumar Akash,
Griffon McMahon,
Tahira Reid,
Neera Jain
Abstract:
Human trust in automation plays an essential role in interactions between humans and automation. While a lack of trust can lead to a human's disuse of automation, over-trust can result in a human trusting a faulty autonomous system which could have negative consequences for the human. Therefore, human trust should be calibrated to optimize human-machine interactions with respect to context-specifi…
▽ More
Human trust in automation plays an essential role in interactions between humans and automation. While a lack of trust can lead to a human's disuse of automation, over-trust can result in a human trusting a faulty autonomous system which could have negative consequences for the human. Therefore, human trust should be calibrated to optimize human-machine interactions with respect to context-specific performance objectives. In this article, we present a probabilistic framework to model and calibrate a human's trust and workload dynamics during his/her interaction with an intelligent decision-aid system. This calibration is achieved by varying the automation's transparency---the amount and utility of information provided to the human. The parameterization of the model is conducted using behavioral data collected through human-subject experiments, and three feedback control policies are experimentally validated and compared against a non-adaptive decision-aid system. The results show that human-automation team performance can be optimized when the transparency is dynamically updated based on the proposed control policy. This framework is a first step toward widespread design and implementation of real-time adaptive automation for use in human-machine interactions.
△ Less
Submitted 29 June, 2020;
originally announced June 2020.
-
Satellite Navigation for the Age of Autonomy
Authors:
Tyler G. R. Reid,
Bryan Chan,
Ashish Goel,
Kazuma Gunning,
Brian Manning,
Jerami Martin,
Andrew Neish,
Adrien Perkins,
Paul Tarantino
Abstract:
Global Navigation Satellite Systems (GNSS) brought navigation to the masses. Coupled with smartphones, the blue dot in the palm of our hands has forever changed the way we interact with the world. Looking forward, cyber-physical systems such as self-driving cars and aerial mobility are pushing the limits of what localization technologies including GNSS can provide. This autonomous revolution requi…
▽ More
Global Navigation Satellite Systems (GNSS) brought navigation to the masses. Coupled with smartphones, the blue dot in the palm of our hands has forever changed the way we interact with the world. Looking forward, cyber-physical systems such as self-driving cars and aerial mobility are pushing the limits of what localization technologies including GNSS can provide. This autonomous revolution requires a solution that supports safety-critical operation, centimeter positioning, and cyber-security for millions of users. To meet these demands, we propose a navigation service from Low Earth Orbiting (LEO) satellites which deliver precision in-part through faster motion, higher power signals for added robustness to interference, constellation autonomous integrity monitoring for integrity, and encryption / authentication for resistance to spoofing attacks. This paradigm is enabled by the 'New Space' movement, where highly capable satellites and components are now built on assembly lines and launch costs have decreased by more than tenfold. Such a ubiquitous positioning service enables a consistent and secure standard where trustworthy information can be validated and shared, extending the electronic horizon from sensor line of sight to an entire city. This enables the situational awareness needed for true safe operation to support autonomy at scale.
△ Less
Submitted 18 May, 2020;
originally announced May 2020.
-
Developments in Modern GNSS and Its Impact on Autonomous Vehicle Architectures
Authors:
Niels Joubert,
Tyler GR Reid,
Fergus Noble
Abstract:
This paper surveys a number of recent developments in modern Global Navigation Satellite Systems (GNSS) and investigates the possible impact on autonomous driving architectures. Modern GNSS now consist of four independent global satellite constellations delivering modernized signals at multiple civil frequencies. New ground monitoring infrastructure, mathematical models, and internet services corr…
▽ More
This paper surveys a number of recent developments in modern Global Navigation Satellite Systems (GNSS) and investigates the possible impact on autonomous driving architectures. Modern GNSS now consist of four independent global satellite constellations delivering modernized signals at multiple civil frequencies. New ground monitoring infrastructure, mathematical models, and internet services correct for errors in the GNSS signals at continent scale. Mass-market automotive-grade receiver chipsets are available at low Cost, Size, Weight, and Power (CSWaP). The result is that GNSS in 2020 delivers better than lane-level accurate localization with 99.99999% integrity guarantees at over 95% availability. In autonomous driving, SAE Level 2 partially autonomous vehicles are now available to consumers, capable of autonomously following lanes and performing basic maneuvers under human supervision. Furthermore, the first pilot programs of SAE Level 4 driverless vehicles are being demonstrated on public roads. However, autonomous driving is not a solved problem. GNSS can help. Specifically, incorporating high-integrity GNSS lane determination into vision-based architectures can unlock lane-level maneuvers and provide oversight to guarantee safety. Incorporating precision GNSS into LiDAR-based systems can unlock robustness and additional fallbacks for safety and utility. Lastly, GNSS provides interoperability through consistent timing and reference frames for future V2X scenarios.
△ Less
Submitted 19 April, 2020; v1 submitted 2 February, 2020;
originally announced February 2020.
-
Standalone and RTK GNSS on 30,000 km of North American Highways
Authors:
Tyler G. R. Reid,
Nahid Pervez,
Umair Ibrahim,
Sarah E. Houts,
Gaurav Pandey,
Naveen K. R. Alla,
Andy Hsia
Abstract:
There is a growing need for vehicle positioning information to support Advanced Driver Assistance Systems (ADAS), Connectivity (V2X), and Automated Driving (AD) features. These range from a need for road determination (<5 meters), lane determination (<1.5 meters), and determining where the vehicle is within the lane (<0.3 meters). This work examines the performance of Global Navigation Satellite S…
▽ More
There is a growing need for vehicle positioning information to support Advanced Driver Assistance Systems (ADAS), Connectivity (V2X), and Automated Driving (AD) features. These range from a need for road determination (<5 meters), lane determination (<1.5 meters), and determining where the vehicle is within the lane (<0.3 meters). This work examines the performance of Global Navigation Satellite Systems (GNSS) on 30,000 km of North American highways to better understand the automotive positioning needs it meets today and what might be possible in the near future with wide area GNSS correction services and multi-frequency receivers. This includes data from a representative automotive production GNSS used primarily for turn-by-turn navigation as well as an Inertial Navigation System which couples two survey grade GNSS receivers with a tactical grade Inertial Measurement Unit (IMU) to act as ground truth. The latter utilized networked Real-Time Kinematic (RTK) GNSS corrections delivered over a cellular modem in real-time. We assess on-road GNSS accuracy, availability, and continuity. Availability and continuity are broken down in terms of satellite visibility, satellite geometry, position type (RTK fixed, RTK float, or standard positioning), and RTK correction latency over the network. Results show that current automotive solutions are best suited to meet road determination requirements at 98% availability but are less suitable for lane determination at 57%. Multi-frequency receivers with RTK corrections were found more capable with road determination at 99.5%, lane determination at 98%, and highway-level lane departure protection at 91%.
△ Less
Submitted 28 August, 2019; v1 submitted 19 June, 2019;
originally announced June 2019.
-
Localization Requirements for Autonomous Vehicles
Authors:
Tyler G. R. Reid,
Sarah E. Houts,
Robert Cammarata,
Graham Mills,
Siddharth Agarwal,
Ankit Vora,
Gaurav Pandey
Abstract:
Autonomous vehicles require precise knowledge of their position and orientation in all weather and traffic conditions for path planning, perception, control, and general safe operation. Here we derive these requirements for autonomous vehicles based on first principles. We begin with the safety integrity level, defining the allowable probability of failure per hour of operation based on desired im…
▽ More
Autonomous vehicles require precise knowledge of their position and orientation in all weather and traffic conditions for path planning, perception, control, and general safe operation. Here we derive these requirements for autonomous vehicles based on first principles. We begin with the safety integrity level, defining the allowable probability of failure per hour of operation based on desired improvements on road safety today. This draws comparisons with the localization integrity levels required in aviation and rail where similar numbers are derived at 10^-8 probability of failure per hour of operation. We then define the geometry of the problem, where the aim is to maintain knowledge that the vehicle is within its lane and to determine what road level it is on. Longitudinal, lateral, and vertical localization error bounds (alert limits) and 95% accuracy requirements are derived based on US road geometry standards (lane width, curvature, and vertical clearance) and allowable vehicle dimensions. For passenger vehicles operating on freeway roads, the result is a required lateral error bound of 0.57 m (0.20 m, 95%), a longitudinal bound of 1.40 m (0.48 m, 95%), a vertical bound of 1.30 m (0.43 m, 95%), and an attitude bound in each direction of 1.50 deg (0.51 deg, 95%). On local streets, the road geometry makes requirements more stringent where lateral and longitudinal error bounds of 0.29 m (0.10 m, 95%) are needed with an orientation requirement of 0.50 deg (0.17 deg, 95%).
△ Less
Submitted 3 June, 2019;
originally announced June 2019.
-
A Classification Model for Sensing Human Trust in Machines Using EEG and GSR
Authors:
Kumar Akash,
Wan-Lin Hu,
Neera Jain,
Tahira Reid
Abstract:
Today, intelligent machines \emph{interact and collaborate} with humans in a way that demands a greater level of trust between human and machine. A first step towards building intelligent machines that are capable of building and maintaining trust with humans is the design of a sensor that will enable machines to estimate human trust level in real-time. In this paper, two approaches for developing…
▽ More
Today, intelligent machines \emph{interact and collaborate} with humans in a way that demands a greater level of trust between human and machine. A first step towards building intelligent machines that are capable of building and maintaining trust with humans is the design of a sensor that will enable machines to estimate human trust level in real-time. In this paper, two approaches for developing classifier-based empirical trust sensor models are presented that specifically use electroencephalography (EEG) and galvanic skin response (GSR) measurements. Human subject data collected from 45 participants is used for feature extraction, feature selection, classifier training, and model validation. The first approach considers a general set of psychophysiological features across all participants as the input variables and trains a classifier-based model for each participant, resulting in a trust sensor model based on the general feature set (i.e., a "general trust sensor model"). The second approach considers a customized feature set for each individual and trains a classifier-based model using that feature set, resulting in improved mean accuracy but at the expense of an increase in training time. This work represents the first use of real-time psychophysiological measurements for the development of a human trust sensor. Implications of the work, in the context of trust management algorithm design for intelligent machines, are also discussed.
△ Less
Submitted 26 March, 2018;
originally announced March 2018.
-
Maintenance in Probabilistic Knowledge-Based Systems
Authors:
Thomas F. Reid,
Gregory S. Parnell
Abstract:
Recent developments using directed acyclical graphs (i.e., influence diagrams and Bayesian networks) for knowledge representation have lessened the problems of using probability in knowledge-based systems (KBS). Most current research involves the efficient propagation of new evidence, but little has been done concerning the maintenance of domain-specific knowledge, which includes the probabilistic…
▽ More
Recent developments using directed acyclical graphs (i.e., influence diagrams and Bayesian networks) for knowledge representation have lessened the problems of using probability in knowledge-based systems (KBS). Most current research involves the efficient propagation of new evidence, but little has been done concerning the maintenance of domain-specific knowledge, which includes the probabilistic information about the problem domain. By making use of conditional independencies represented in she graphs, however, probability assessments are required only for certain variables when the knowledge base is updated. The purpose of this study was to investigate, for those variables which require probability assessments, ways to reduce the amount of new knowledge required from the expert when updating probabilistic information in a probabilistic knowledge-based system. Three special cases (ignored outcome, split outcome, and assumed constraint outcome) were identified under which many of the original probabilities (those already in the knowledge-base) do not need to be reassessed when maintenance is required.
△ Less
Submitted 27 March, 2013;
originally announced April 2013.