Showing 1–36 of 36 results for author: Saripalli, S

Exploring Unstructured Environments using Minimal Sensing on Cooperative Nano-Drones

Authors: Pedro Arias-Perez, Alvika Gautam, Miguel Fernandez-Cortizas, David Perez-Saura, Srikanth Saripalli, Pascual Campoy

Abstract: Recent advances have improved autonomous navigation and mapping under payload constraints, but current multi-robot inspection algorithms are unsuitable for nano-drones due to their need for heavy sensors and high computational resources. To address these challenges, we introduce ExploreBug, a novel hybrid frontier range bug algorithm designed to handle limited sensing capabilities for a swarm of n… ▽ More Recent advances have improved autonomous navigation and mapping under payload constraints, but current multi-robot inspection algorithms are unsuitable for nano-drones due to their need for heavy sensors and high computational resources. To address these challenges, we introduce ExploreBug, a novel hybrid frontier range bug algorithm designed to handle limited sensing capabilities for a swarm of nano-drones. This system includes three primary components: a mapping subsystem, an exploration subsystem, and a navigation subsystem. Additionally, an intra-swarm collision avoidance system is integrated to prevent collisions between drones. We validate the efficacy of our approach through extensive simulations and real-world exploration experiments involving up to seven drones in simulations and three in real-world settings, across various obstacle configurations and with a maximum navigation speed of 0.75 m/s. Our tests demonstrate that the algorithm efficiently completes exploration tasks, even with minimal sensing, across different swarm sizes and obstacle densities. Furthermore, our frontier allocation heuristic ensures an equal distribution of explored areas and paths traveled by each drone in the swarm. We publicly release the source code of the proposed system to foster further developments in mapping and exploration using autonomous nano drones. △ Less

Submitted 9 July, 2024; originally announced July 2024.

Comments: Submitted to IEEE Robotics and Automation Letters

Reflectivity Is All You Need!: Advancing LiDAR Semantic Segmentation

Authors: Kasi Viswanath, Peng Jiang, Srikanth Saripalli

Abstract: LiDAR semantic segmentation frameworks predominantly leverage geometry-based features to differentiate objects within a scan. While these methods excel in scenarios with clear boundaries and distinct shapes, their performance declines in environments where boundaries are blurred, particularly in off-road contexts. To address this, recent strides in 3D segmentation algorithms have focused on harnes… ▽ More LiDAR semantic segmentation frameworks predominantly leverage geometry-based features to differentiate objects within a scan. While these methods excel in scenarios with clear boundaries and distinct shapes, their performance declines in environments where boundaries are blurred, particularly in off-road contexts. To address this, recent strides in 3D segmentation algorithms have focused on harnessing raw LiDAR intensity measurements to improve prediction accuracy. Despite these efforts, current learning-based models struggle to correlate the intricate connections between raw intensity and factors such as distance, incidence angle, material reflectivity, and atmospheric conditions. Building upon our prior work, this paper delves into the advantages of employing calibrated intensity (also referred to as reflectivity) within learning-based LiDAR semantic segmentation frameworks. We initially establish that incorporating reflectivity as an input enhances the existing LiDAR semantic segmentation model. Furthermore, we present findings that enable the model to learn to calibrate intensity can boost its performance. Through extensive experimentation on the off-road dataset Rellis-3D, we demonstrate notable improvements. Specifically, converting intensity to reflectivity results in a 4% increase in mean Intersection over Union (mIoU) when compared to using raw intensity in Off-road scenarios. Additionally, we also investigate the possible benefits of using calibrated intensity in semantic segmentation in urban environments (SemanticKITTI) and cross-sensor domain adaptation. △ Less

Submitted 19 March, 2024; originally announced March 2024.

3DGS-ReLoc: 3D Gaussian Splatting for Map Representation and Visual ReLocalization

Authors: Peng Jiang, Gaurav Pandey, Srikanth Saripalli

Abstract: This paper presents a novel system designed for 3D mapping and visual relocalization using 3D Gaussian Splatting. Our proposed method uses LiDAR and camera data to create accurate and visually plausible representations of the environment. By leveraging LiDAR data to initiate the training of the 3D Gaussian Splatting map, our system constructs maps that are both detailed and geometrically accurate.… ▽ More This paper presents a novel system designed for 3D mapping and visual relocalization using 3D Gaussian Splatting. Our proposed method uses LiDAR and camera data to create accurate and visually plausible representations of the environment. By leveraging LiDAR data to initiate the training of the 3D Gaussian Splatting map, our system constructs maps that are both detailed and geometrically accurate. To mitigate excessive GPU memory usage and facilitate rapid spatial queries, we employ a combination of a 2D voxel map and a KD-tree. This preparation makes our method well-suited for visual localization tasks, enabling efficient identification of correspondences between the query image and the rendered image from the Gaussian Splatting map via normalized cross-correlation (NCC). Additionally, we refine the camera pose of the query image using feature-based matching and the Perspective-n-Point (PnP) technique. The effectiveness, adaptability, and precision of our system are demonstrated through extensive evaluation on the KITTI360 dataset. △ Less

Submitted 17 March, 2024; originally announced March 2024.

Comments: 8 pages, 7 figures

Off-Road LiDAR Intensity Based Semantic Segmentation

Authors: Kasi Viswanath, Peng Jiang, Sujit PB, Srikanth Saripalli

Abstract: LiDAR is used in autonomous driving to provide 3D spatial information and enable accurate perception in off-road environments, aiding in obstacle detection, mapping, and path planning. Learning-based LiDAR semantic segmentation utilizes machine learning techniques to automatically classify objects and regions in LiDAR point clouds. Learning-based models struggle in off-road environments due to the… ▽ More LiDAR is used in autonomous driving to provide 3D spatial information and enable accurate perception in off-road environments, aiding in obstacle detection, mapping, and path planning. Learning-based LiDAR semantic segmentation utilizes machine learning techniques to automatically classify objects and regions in LiDAR point clouds. Learning-based models struggle in off-road environments due to the presence of diverse objects with varying colors, textures, and undefined boundaries, which can lead to difficulties in accurately classifying and segmenting objects using traditional geometric-based features. In this paper, we address this problem by harnessing the LiDAR intensity parameter to enhance object segmentation in off-road environments. Our approach was evaluated in the RELLIS-3D data set and yielded promising results as a preliminary analysis with improved mIoU for classes "puddle" and "grass" compared to more complex deep learning-based benchmarks. The methodology was evaluated for compatibility across both Velodyne and Ouster LiDAR systems, assuring its cross-platform applicability. This analysis advocates for the incorporation of calibrated intensity as a supplementary input, aiming to enhance the prediction accuracy of learning based semantic segmentation frameworks. https://github.com/MOONLABIISERB/lidar-intensity-predictor/tree/main △ Less

Submitted 2 January, 2024; originally announced January 2024.

Comments: Accepted to ISER 2023

GPT-4V Takes the Wheel: Promises and Challenges for Pedestrian Behavior Prediction

Authors: Jia Huang, Peng Jiang, Alvika Gautam, Srikanth Saripalli

Abstract: Predicting pedestrian behavior is the key to ensure safety and reliability of autonomous vehicles. While deep learning methods have been promising by learning from annotated video frame sequences, they often fail to fully grasp the dynamic interactions between pedestrians and traffic, crucial for accurate predictions. These models also lack nuanced common sense reasoning. Moreover, the manual anno… ▽ More Predicting pedestrian behavior is the key to ensure safety and reliability of autonomous vehicles. While deep learning methods have been promising by learning from annotated video frame sequences, they often fail to fully grasp the dynamic interactions between pedestrians and traffic, crucial for accurate predictions. These models also lack nuanced common sense reasoning. Moreover, the manual annotation of datasets for these models is expensive and challenging to adapt to new situations. The advent of Vision Language Models (VLMs) introduces promising alternatives to these issues, thanks to their advanced visual and causal reasoning skills. To our knowledge, this research is the first to conduct both quantitative and qualitative evaluations of VLMs in the context of pedestrian behavior prediction for autonomous driving. We evaluate GPT-4V(ision) on publicly available pedestrian datasets: JAAD and WiDEVIEW. Our quantitative analysis focuses on GPT-4V's ability to predict pedestrian behavior in current and future frames. The model achieves a 57% accuracy in a zero-shot manner, which, while impressive, is still behind the state-of-the-art domain-specific models (70%) in predicting pedestrian crossing actions. Qualitatively, GPT-4V shows an impressive ability to process and interpret complex traffic scenarios, differentiate between various pedestrian behaviors, and detect and analyze groups. However, it faces challenges, such as difficulty in detecting smaller pedestrians and assessing the relative motion between pedestrians and the ego vehicle. △ Less

Submitted 25 January, 2024; v1 submitted 24 November, 2023; originally announced November 2023.

Radar-Only Off-Road Local Navigation

Authors: Timothy Overbye, Srikanth Saripalli

Abstract: Off-road robotics have traditionally utilized lidar for local navigation due to its accuracy and high resolution. However, the limitations of lidar, such as reduced performance in harsh environmental conditions and limited range, have prompted the exploration of alternative sensing technologies. This paper investigates the potential of radar for off-road local navigation, as it offers the advantag… ▽ More Off-road robotics have traditionally utilized lidar for local navigation due to its accuracy and high resolution. However, the limitations of lidar, such as reduced performance in harsh environmental conditions and limited range, have prompted the exploration of alternative sensing technologies. This paper investigates the potential of radar for off-road local navigation, as it offers the advantages of a longer range and the ability to penetrate dust and light vegetation. We adapt existing lidar-based methods for radar and evaluate the performance in comparison to lidar under various off-road conditions. We show that radar can provide a significant range advantage over lidar while maintaining accuracy for both ground plane estimation and obstacle detection. And finally, we demonstrate successful autonomous navigation at a speed of 2.5 m/s over a path length of 350 m using only radar for ground plane estimation and obstacle detection. △ Less

Submitted 26 October, 2023; originally announced October 2023.

Comments: 7 pages, 17 figures, ITSC 2023

ROSS: Radar Off-road Semantic Segmentation

Authors: Peng Jiang, Srikanth Saripalli

Abstract: As the demand for autonomous navigation in off-road environments increases, the need for effective solutions to understand these surroundings becomes essential. In this study, we confront the inherent complexities of semantic segmentation in RADAR data for off-road scenarios. We present a novel pipeline that utilizes LIDAR data and an existing annotated off-road LIDAR dataset for generating RADAR… ▽ More As the demand for autonomous navigation in off-road environments increases, the need for effective solutions to understand these surroundings becomes essential. In this study, we confront the inherent complexities of semantic segmentation in RADAR data for off-road scenarios. We present a novel pipeline that utilizes LIDAR data and an existing annotated off-road LIDAR dataset for generating RADAR labels, in which the RADAR data are represented as images. Validated with real-world datasets, our pragmatic approach underscores the potential of RADAR technology for navigation applications in off-road environments. △ Less

Submitted 20 October, 2023; originally announced October 2023.

Comments: 10 pages, 6 figures, accepted by the 18th International Symposium on Experimental Robotics (ISER 2023)

Online Multi-IMU Calibration Using Visual-Inertial Odometry

Authors: Jacob Hartzer, Srikanth Saripalli

Abstract: This work presents a centralized multi-IMU filter framework with online intrinsic and extrinsic calibration for unsynchronized inertial measurement units that is robust against changes in calibration parameters. The novel EKF-based method estimates the positional and rotational offsets of the system of sensors as well as their intrinsic biases without the use of rigid body geometric constraints. A… ▽ More This work presents a centralized multi-IMU filter framework with online intrinsic and extrinsic calibration for unsynchronized inertial measurement units that is robust against changes in calibration parameters. The novel EKF-based method estimates the positional and rotational offsets of the system of sensors as well as their intrinsic biases without the use of rigid body geometric constraints. Additionally, the filter is flexible in the total number of sensors used while leveraging the commonly used MSCKF framework for camera measurements. The filter framework has been validated using Monte Carlo simulation as well as experimentally. In both simulations and experiments, using multiple IMU measurement streams within the proposed filter framework outperforms the use of a single IMU in a filter prediction step while also producing consistent and accurate estimates of initial calibration errors. Compared to current state-of-the-art optimizers, the filter produces similar intrinsic and extrinsic calibration parameters for each sensor. Finally, an open source repository has been provided at https://github.com/unmannedlab/ekf-cal containing both the online estimator and the simulation used for testing and evaluation. △ Less

Submitted 3 January, 2024; v1 submitted 18 October, 2023; originally announced October 2023.

WiDEVIEW: An UltraWideBand and Vision Dataset for Deciphering Pedestrian-Vehicle Interactions

Authors: Jia Huang, Alvika Gautam, Junghun Choi, Srikanth Saripalli

Abstract: Robust and accurate tracking and localization of road users like pedestrians and cyclists is crucial to ensure safe and effective navigation of Autonomous Vehicles (AVs), particularly so in urban driving scenarios with complex vehicle-pedestrian interactions. Existing datasets that are useful to investigate vehicle-pedestrian interactions are mostly image-centric and thus vulnerable to vision fail… ▽ More Robust and accurate tracking and localization of road users like pedestrians and cyclists is crucial to ensure safe and effective navigation of Autonomous Vehicles (AVs), particularly so in urban driving scenarios with complex vehicle-pedestrian interactions. Existing datasets that are useful to investigate vehicle-pedestrian interactions are mostly image-centric and thus vulnerable to vision failures. In this paper, we investigate Ultra-wideband (UWB) as an additional modality for road users' localization to enable a better understanding of vehicle-pedestrian interactions. We present WiDEVIEW, the first multimodal dataset that integrates LiDAR, three RGB cameras, GPS/IMU, and UWB sensors for capturing vehicle-pedestrian interactions in an urban autonomous driving scenario. Ground truth image annotations are provided in the form of 2D bounding boxes and the dataset is evaluated on standard 2D object detection and tracking algorithms. The feasibility of UWB is evaluated for typical traffic scenarios in both line-of-sight and non-line-of-sight conditions using LiDAR as ground truth. We establish that UWB range data has comparable accuracy with LiDAR with an error of 0.19 meters and reliable anchor-tag range data for up to 40 meters in line-of-sight conditions. UWB performance for non-line-of-sight conditions is subjective to the nature of the obstruction (trees vs. buildings). Further, we provide a qualitative analysis of UWB performance for scenarios susceptible to intermittent vision failures. The dataset can be downloaded via https://github.com/unmannedlab/UWB_Dataset. △ Less

Submitted 27 September, 2023; originally announced September 2023.

Learning Pedestrian Actions to Ensure Safe Autonomous Driving

Authors: Jia Huang, Alvika Gautam, Srikanth Saripalli

Abstract: To ensure safe autonomous driving in urban environments with complex vehicle-pedestrian interactions, it is critical for Autonomous Vehicles (AVs) to have the ability to predict pedestrians' short-term and immediate actions in real-time. In recent years, various methods have been developed to study estimating pedestrian behaviors for autonomous driving scenarios, but there is a lack of clear defin… ▽ More To ensure safe autonomous driving in urban environments with complex vehicle-pedestrian interactions, it is critical for Autonomous Vehicles (AVs) to have the ability to predict pedestrians' short-term and immediate actions in real-time. In recent years, various methods have been developed to study estimating pedestrian behaviors for autonomous driving scenarios, but there is a lack of clear definitions for pedestrian behaviors. In this work, the literature gaps are investigated and a taxonomy is presented for pedestrian behavior characterization. Further, a novel multi-task sequence to sequence Transformer encoders-decoders (TF-ed) architecture is proposed for pedestrian action and trajectory prediction using only ego vehicle camera observations as inputs. The proposed approach is compared against an existing LSTM encoders decoders (LSTM-ed) architecture for action and trajectory prediction. The performance of both models is evaluated on the publicly available Joint Attention Autonomous Driving (JAAD) dataset, CARLA simulation data as well as real-time self-driving shuttle data collected on university campus. Evaluation results illustrate that the proposed method reaches an accuracy of 81% on action prediction task on JAAD testing data and outperforms the LSTM-ed by 7.4%, while LSTM counterpart performs much better on trajectory prediction task for a prediction sequence length of 25 frames. △ Less

Submitted 22 May, 2023; originally announced May 2023.

Comments: 8 pages, 9 figures

Improving Extrinsics between RADAR and LIDAR using Learning

Authors: Peng Jiang, Srikanth Saripalli

Abstract: LIDAR and RADAR are two commonly used sensors in autonomous driving systems. The extrinsic calibration between the two is crucial for effective sensor fusion. The challenge arises due to the low accuracy and sparse information in RADAR measurements. This paper presents a novel solution for 3D RADAR-LIDAR calibration in autonomous systems. The method employs simple targets to generate data, includi… ▽ More LIDAR and RADAR are two commonly used sensors in autonomous driving systems. The extrinsic calibration between the two is crucial for effective sensor fusion. The challenge arises due to the low accuracy and sparse information in RADAR measurements. This paper presents a novel solution for 3D RADAR-LIDAR calibration in autonomous systems. The method employs simple targets to generate data, including correspondence registration and a one-step optimization algorithm. The optimization aims to minimize the reprojection error while utilizing a small multi-layer perception (MLP) to perform regression on the return energy of the sensor around the targets. The proposed approach uses a deep learning framework such as PyTorch and can be optimized through gradient descent. The experiment uses a 360-degree Ouster-128 LIDAR and a 360-degree Navtech RADAR, providing raw measurements. The results validate the effectiveness of the proposed method in achieving improved estimates of extrinsic calibration parameters. △ Less

Submitted 17 May, 2023; originally announced May 2023.

Comments: accepted in IV 2023

Online Multi Camera-IMU Calibration

Authors: Jacob Hartzer, Srikanth Saripalli

Abstract: Visual-inertial navigation systems are powerful in their ability to accurately estimate localization of mobile systems within complex environments that preclude the use of global navigation satellite systems. However, these navigation systems are reliant on accurate and up-to-date temporospatial calibrations of the sensors being used. As such, online estimators for these parameters are useful in r… ▽ More Visual-inertial navigation systems are powerful in their ability to accurately estimate localization of mobile systems within complex environments that preclude the use of global navigation satellite systems. However, these navigation systems are reliant on accurate and up-to-date temporospatial calibrations of the sensors being used. As such, online estimators for these parameters are useful in resilient systems. This paper presents an extension to existing Kalman Filter based frameworks for estimating and calibrating the extrinsic parameters of multi-camera IMU systems. In addition to extending the filter framework to include multiple camera sensors, the measurement model was reformulated to make use of measurement data that is typically made available in fiducial detection software. A secondary filter layer was used to estimate time translation parameters without closed-loop feedback of sensor data. Experimental calibration results, including the use of cameras with non-overlapping fields of view, were used to validate the stability and accuracy of the filter formulation when compared to offline methods. Finally the generalized filter code has been open-sourced and is available online. △ Less

Submitted 7 October, 2023; v1 submitted 28 September, 2022; originally announced September 2022.

CAMEL: Learning Cost-maps Made Easy for Off-road Driving

Authors: Kasi Vishwanath, P. B. Sujit, Srikanth Saripalli

Abstract: Cost-maps are used by robotic vehicles to plan collision-free paths. The cost associated with each cell in the map represents the sensed environment information which is often determined manually after several trial-and-error efforts. In off-road environments, due to the presence of several types of features, it is challenging to handcraft the cost values associated with each feature. Moreover, di… ▽ More Cost-maps are used by robotic vehicles to plan collision-free paths. The cost associated with each cell in the map represents the sensed environment information which is often determined manually after several trial-and-error efforts. In off-road environments, due to the presence of several types of features, it is challenging to handcraft the cost values associated with each feature. Moreover, different handcrafted cost values can lead to different paths for the same environment which is not desirable. In this paper, we address the problem of learning the cost-map values from the sensed environment for robust vehicle path planning. We propose a novel framework called as CAMEL using deep learning approach that learns the parameters through demonstrations yielding an adaptive and robust cost-map for path planning. CAMEL has been trained on multi-modal datasets such as RELLIS-3D. The evaluation of CAMEL is carried out on an off-road scene simulator (MAVS) and on field data from IISER-B campus. We also perform realworld implementation of CAMEL on a ground rover. The results shows flexible and robust motion of the vehicle without collisions in unstructured terrains. △ Less

Submitted 18 October, 2022; v1 submitted 26 September, 2022; originally announced September 2022.

Contrastive Learning of Features between Images and LiDAR

Authors: Peng Jiang, Srikanth Saripalli

Abstract: Image and Point Clouds provide different information for robots. Finding the correspondences between data from different sensors is crucial for various tasks such as localization, mapping, and navigation. Learning-based descriptors have been developed for single sensors; there is little work on cross-modal features. This work treats learning cross-modal features as a dense contrastive learning pro… ▽ More Image and Point Clouds provide different information for robots. Finding the correspondences between data from different sensors is crucial for various tasks such as localization, mapping, and navigation. Learning-based descriptors have been developed for single sensors; there is little work on cross-modal features. This work treats learning cross-modal features as a dense contrastive learning problem. We propose a Tuple-Circle loss function for cross-modality feature learning. Furthermore, to learn good features and not lose generality, we developed a variant of widely used PointNet++ architecture for point cloud and U-Net CNN architecture for images. Moreover, we conduct experiments on a real-world dataset to show the effectiveness of our loss function and network structure. We show that our models indeed learn information from both images as well as LiDAR by visualizing the features. △ Less

Submitted 24 June, 2022; originally announced June 2022.

Comments: accepted in CASE2022

Extrinsic Calibration of LiDAR, IMU and Camera

Authors: Subodh Mishra, Srikanth Saripalli

Abstract: In this work we present a novel method to jointly calibrate a sensor suite consisting a 3D-LiDAR, Inertial Measurement Unit (IMU) and Camera under an Extended Kalman Filter (EKF) framework. We exploit pairwise constraints between the 3 sensor pairs to perform EKF update and experimentally demonstrate the superior performance obtained with joint calibration as against individual sensor pair calibra… ▽ More In this work we present a novel method to jointly calibrate a sensor suite consisting a 3D-LiDAR, Inertial Measurement Unit (IMU) and Camera under an Extended Kalman Filter (EKF) framework. We exploit pairwise constraints between the 3 sensor pairs to perform EKF update and experimentally demonstrate the superior performance obtained with joint calibration as against individual sensor pair calibration. △ Less

Submitted 17 May, 2022; originally announced May 2022.

Comments: Workshop on Challenges in Sensor Calibration for Robotics Applications at the 17th INTERNATIONAL CONFERENCE ON INTELLIGENT AUTONOMOUS SYSTEMS (IAS-17), ZAGREB, CROATIA

OTTR: Off-Road Trajectory Tracking using Reinforcement Learning

Authors: Akhil Nagariya, Dileep Kalathil, Srikanth Saripalli

Abstract: In this work, we present a novel Reinforcement Learning (RL) algorithm for the off-road trajectory tracking problem. Off-road environments involve varying terrain types and elevations, and it is difficult to model the interaction dynamics of specific off-road vehicles with such a diverse and complex environment. Standard RL policies trained on a simulator will fail to operate in such challenging r… ▽ More In this work, we present a novel Reinforcement Learning (RL) algorithm for the off-road trajectory tracking problem. Off-road environments involve varying terrain types and elevations, and it is difficult to model the interaction dynamics of specific off-road vehicles with such a diverse and complex environment. Standard RL policies trained on a simulator will fail to operate in such challenging real-world settings. Instead of using a naive domain randomization approach, we propose an innovative supervised-learning based approach for overcoming the sim-to-real gap problem. Our approach efficiently exploits the limited real-world data available to adapt the baseline RL policy obtained using a simple kinematics simulator. This avoids the need for modeling the diverse and complex interaction of the vehicle with off-road environments. We evaluate the performance of the proposed algorithm using two different off-road vehicles, Warthog and Moose. Compared to the standard ILQR approach, our proposed approach achieves a 30% and 50% reduction in cross track error in Warthog and Moose, respectively, by utilizing only 30 minutes of real-world driving data. △ Less

Submitted 5 October, 2021; originally announced October 2021.

G-VOM: A GPU Accelerated Voxel Off-Road Mapping System

Authors: Timothy Overbye, Srikanth Saripalli

Abstract: We present a local 3D voxel mapping framework for off-road path planning and navigation. Our method provides both hard and soft positive obstacle detection, negative obstacle detection, slope estimation, and roughness estimation. By using a 3D array lookup table data structure and by leveraging the GPU it can provide online performance. We then demonstrate the system working on three vehicles, a C… ▽ More We present a local 3D voxel mapping framework for off-road path planning and navigation. Our method provides both hard and soft positive obstacle detection, negative obstacle detection, slope estimation, and roughness estimation. By using a 3D array lookup table data structure and by leveraging the GPU it can provide online performance. We then demonstrate the system working on three vehicles, a Clearpath Robotics Warthog, Moose, and a Polaris Ranger, and compare against a set of pre-recorded waypoints. This was done at 4.5 m/s in autonomous operation and 12 m/s in manual operation with a map update rate of 10 Hz. Finally, an open-source ROS implementation is provided. https://github.com/unmannedlab/G-VOM △ Less

Submitted 27 September, 2021; originally announced September 2021.

Comments: 7 pages, 14 figures

SemCal: Semantic LiDAR-Camera Calibration using Neural MutualInformation Estimator

Authors: Peng Jiang, Philip Osteen, Srikanth Saripalli

Abstract: This paper proposes SemCal: an automatic, targetless, extrinsic calibration algorithm for a LiDAR and camera system using semantic information. We leverage a neural information estimator to estimate the mutual information (MI) of semantic information extracted from each sensor measurement, facilitating semantic-level data association. By using a matrix exponential formulation of the $se(3)$ transf… ▽ More This paper proposes SemCal: an automatic, targetless, extrinsic calibration algorithm for a LiDAR and camera system using semantic information. We leverage a neural information estimator to estimate the mutual information (MI) of semantic information extracted from each sensor measurement, facilitating semantic-level data association. By using a matrix exponential formulation of the $se(3)$ transformation and a kernel-based sampling method to sample from camera measurement based on LiDAR projected points, we can formulate the LiDAR-Camera calibration problem as a novel differentiable objective function that supports gradient-based optimization methods. We also introduce a semantic-based initial calibration method using 2D MI-based image registration and Perspective-n-Point (PnP) solver. To evaluate performance, we demonstrate the robustness of our method and quantitatively analyze the accuracy using a synthetic dataset. We also evaluate our algorithm qualitatively on an urban dataset (KITTI360) and an off-road dataset (RELLIS-3D) benchmark datasets using both hand-annotated ground truth labels as well as labels predicted by the state-of-the-art deep learning models, showing improvement over recent comparable calibration approaches. △ Less

Submitted 21 September, 2021; originally announced September 2021.

Comments: 7 pages, 10 figures, accepted by The 2021 IEEE International Conference on Multisensor Fusion and Integration (MFI 2021). arXiv admin note: substantial text overlap with arXiv:2104.12023

ROOAD: RELLIS Off-road Odometry Analysis Dataset

Authors: George Chustz, Srikanth Saripalli

Abstract: The development and implementation of visual-inertial odometry (VIO) has focused on structured environments, but interest in localization in off-road environments is growing. In this paper, we present the RELLIS Off-road Odometry Analysis Dataset (ROOAD) which provides high-quality, time-synchronized off-road monocular visual-inertial data sequences to further the development of related research.… ▽ More The development and implementation of visual-inertial odometry (VIO) has focused on structured environments, but interest in localization in off-road environments is growing. In this paper, we present the RELLIS Off-road Odometry Analysis Dataset (ROOAD) which provides high-quality, time-synchronized off-road monocular visual-inertial data sequences to further the development of related research. We evaluated the dataset on two state-of-the-art VIO algorithms, (1) Open-VINS and (2) VINS-Fusion. Our findings indicate that both algorithms perform 2 to 30 times worse on the ROOAD dataset compared to their performance in structured environments. Furthermore, OpenVINS has better tracking stability and real-time performance than VINS-Fusion in the off-road environment, while VINS-Fusion outperformed OpenVINS in tracking accuracy in several data sequences. Since the camera-IMU calibration tool from Kalibr toolkit is used extensively in this work, we have included several calibration data sequences. Our hand measurements show Kalibr's tool achieved +/-1 degree for orientation error and +/-1 mm at best (x- and y-axis) and +/-10 mm (z-axis) at worse for position error in the camera frame between the camera and IMU. This novel dataset provides a new set of scenarios for researchers to design and test their localization algorithms on, as well as critical insights in the current performance of VIO in off-road environments. ROOAD Dataset: github.com/unmannedlab/ROOAD △ Less

Submitted 7 April, 2022; v1 submitted 16 September, 2021; originally announced September 2021.

Comments: 7 pages, 6 figures, 5 tables, IV 2022 conference

Vehicular Teamwork: Collaborative localization of Autonomous Vehicles

Authors: Jacob Hartzer, Srikanth Saripalli

Abstract: This paper develops a distributed collaborative localization algorithm based on an extended kalman filter. This algorithm incorporates Ultra-Wideband (UWB) measurements for vehicle to vehicle ranging, and shows improvements in localization accuracy where GPS typically falls short. The algorithm was first tested in a newly created open-source simulation environment that emulates various numbers of… ▽ More This paper develops a distributed collaborative localization algorithm based on an extended kalman filter. This algorithm incorporates Ultra-Wideband (UWB) measurements for vehicle to vehicle ranging, and shows improvements in localization accuracy where GPS typically falls short. The algorithm was first tested in a newly created open-source simulation environment that emulates various numbers of vehicles and sensors while simultaneously testing multiple localization algorithms. Predicted error distributions for various algorithms are quickly producible using the Monte-Carlo method and optimization techniques within MatLab. The simulation results were validated experimentally in an outdoor, urban environment. Improvements of localization accuracy over a typical extended kalman filter ranged from 2.9% to 9.3% over 180 meter test runs. When GPS was denied, these improvements increased up to 83.3% over a standard kalman filter. In both simulation and experimentally, the DCL algorithm was shown to be a good approximation of a full state filter, while reducing required communication between vehicles. These results are promising in showing the efficacy of adding UWB ranging sensors to cars for collaborative and landmark localization, especially in GPS-denied environments. In the future, additional moving vehicles with additional tags will be tested in other challenging GPS denied environments. △ Less

Submitted 7 October, 2023; v1 submitted 29 April, 2021; originally announced April 2021.

AutoCone: An OmniDirectional Robot for Lane-Level Cone Placement

Authors: Jacob Hartzer, Srikanth Saripalli

Abstract: This paper summarizes the progress in developing a rugged, low-cost, automated ground cone robot network capable of traffic delineation at lane-level precision. A holonomic omnidirectional base with a traffic delineator was developed to allow flexibility in initialization. RTK GPS was utilized to reduce minimum position error to 2 centimeters. Due to recent developments, the cost of the platform i… ▽ More This paper summarizes the progress in developing a rugged, low-cost, automated ground cone robot network capable of traffic delineation at lane-level precision. A holonomic omnidirectional base with a traffic delineator was developed to allow flexibility in initialization. RTK GPS was utilized to reduce minimum position error to 2 centimeters. Due to recent developments, the cost of the platform is now less than $1,600. To minimize the effects of GPS-denied environments, wheel encoders and an Extended Kalman Filter were implemented to maintain lane-level accuracy during operation and a maximum error of 1.97 meters through 50 meters with little to no GPS signal. Future work includes increasing the operational speed of the platforms, incorporating lanelet information for path planning, and cross-platform estimation. △ Less

Submitted 7 October, 2023; v1 submitted 29 April, 2021; originally announced April 2021.

Target-free Extrinsic Calibration of a 3D-Lidar and an IMU

Authors: Subodh Mishra, Gaurav Pandey, Srikanth Saripalli

Abstract: This work presents a novel target-free extrinsic calibration algorithm for a 3D Lidar and an IMU pair using an Extended Kalman Filter (EKF) which exploits the \textit{motion based calibration constraint} for state update. The steps include, data collection by motion excitation of the Lidar Inertial Sensor suite along all degrees of freedom, determination of the inter sensor rotation by using rotat… ▽ More This work presents a novel target-free extrinsic calibration algorithm for a 3D Lidar and an IMU pair using an Extended Kalman Filter (EKF) which exploits the \textit{motion based calibration constraint} for state update. The steps include, data collection by motion excitation of the Lidar Inertial Sensor suite along all degrees of freedom, determination of the inter sensor rotation by using rotational component of the aforementioned \textit{motion based calibration constraint} in a least squares optimization framework, and finally, the determination of inter sensor translation using the \textit{motion based calibration constraint} for state update in an Extended Kalman Filter (EKF) framework. We experimentally validate our method using data collected in our lab and open-source (https://github.com/unmannedlab/imu_lidar_calibration) our contribution for the robotics research community. △ Less

Submitted 28 September, 2021; v1 submitted 25 April, 2021; originally announced April 2021.

Comments: Published at 2021 IEEE International Conference on Multisensor Fusion and Integration (MFI 2021)

Calibrating LiDAR and Camera using Semantic Mutual information

Authors: Peng Jiang, Philip Osteen, Srikanth Saripalli

Abstract: We propose an algorithm for automatic, targetless, extrinsic calibration of a LiDAR and camera system using semantic information. We achieve this goal by maximizing mutual information (MI) of semantic information between sensors, leveraging a neural network to estimate semantic mutual information, and matrix exponential for calibration computation. Using kernel-based sampling to sample data from c… ▽ More We propose an algorithm for automatic, targetless, extrinsic calibration of a LiDAR and camera system using semantic information. We achieve this goal by maximizing mutual information (MI) of semantic information between sensors, leveraging a neural network to estimate semantic mutual information, and matrix exponential for calibration computation. Using kernel-based sampling to sample data from camera measurement based on LiDAR projected points, we formulate the problem as a novel differentiable objective function which supports the use of gradient-based optimization methods. We also introduce an initial calibration method using 2D MI-based image registration. Finally, we demonstrate the robustness of our method and quantitatively analyze the accuracy on a synthetic dataset and also evaluate our algorithm qualitatively on KITTI360 and RELLIS-3D benchmark datasets, showing improvement over recent comparable approaches. △ Less

Submitted 24 April, 2021; originally announced April 2021.

Comments: 5 pages, 6 figures, submitted to ICRA 2021 workshop

OFFSEG: A Semantic Segmentation Framework For Off-Road Driving

Authors: Kasi Viswanath, Kartikeya Singh, Peng Jiang, Sujit P. B., Srikanth Saripalli

Abstract: Off-road image semantic segmentation is challenging due to the presence of uneven terrains, unstructured class boundaries, irregular features and strong textures. These aspects affect the perception of the vehicle from which the information is used for path planning. Current off-road datasets exhibit difficulties like class imbalance and understanding of varying environmental topography. To overco… ▽ More Off-road image semantic segmentation is challenging due to the presence of uneven terrains, unstructured class boundaries, irregular features and strong textures. These aspects affect the perception of the vehicle from which the information is used for path planning. Current off-road datasets exhibit difficulties like class imbalance and understanding of varying environmental topography. To overcome these issues we propose a framework for off-road semantic segmentation called as OFFSEG that involves (i) a pooled class semantic segmentation with four classes (sky, traversable region, non-traversable region and obstacle) using state-of-the-art deep learning architectures (ii) a colour segmentation methodology to segment out specific sub-classes (grass, puddle, dirt, gravel, etc.) from the traversable region for better scene understanding. The evaluation of the framework is carried out on two off-road driving datasets, namely, RELLIS-3D and RUGD. We have also tested proposed framework in IISERB campus frames. The results show that OFFSEG achieves good performance and also provides detailed information on the traversable region. △ Less

Submitted 23 March, 2021; originally announced March 2021.

Path Optimization for Ground Vehicles in Off-Road Terrain

Authors: Timothy Overbye, Srikanth Saripalli

Abstract: We present a method for path optimization for ground vehicles in off-road environments at high speeds. This path optimization considers the kinematic constraints of the vehicle. By thinking in the actuator space we can represent these constraints as limits in the space rather than derived properties of the path. In this paper we present a actuator space approach to path optimization for off-road g… ▽ More We present a method for path optimization for ground vehicles in off-road environments at high speeds. This path optimization considers the kinematic constraints of the vehicle. By thinking in the actuator space we can represent these constraints as limits in the space rather than derived properties of the path. In this paper we present a actuator space approach to path optimization for off-road ground vehicles. This is done by representing and operation on the path as a list of steering angles over the path length. This transforms the set of kinematic constraints into constraints on the steering angle. We then put this path into a gradient descent solver. This produced paths that are kinematically feasible and optimized in accordance with our cost function. Finally, we tested the system both in simulation and on an off-road vehicle at speeds of 5 m/s. △ Less

Submitted 3 January, 2021; originally announced January 2021.

Comments: 6 pages, 13 figures, submitted to ICRA 2021

RELLIS-3D Dataset: Data, Benchmarks and Analysis

Authors: Peng Jiang, Philip Osteen, Maggie Wigness, Srikanth Saripalli

Abstract: Semantic scene understanding is crucial for robust and safe autonomous navigation, particularly so in off-road environments. Recent deep learning advances for 3D semantic segmentation rely heavily on large sets of training data, however existing autonomy datasets either represent urban environments or lack multimodal off-road data. We fill this gap with RELLIS-3D, a multimodal dataset collected in… ▽ More Semantic scene understanding is crucial for robust and safe autonomous navigation, particularly so in off-road environments. Recent deep learning advances for 3D semantic segmentation rely heavily on large sets of training data, however existing autonomy datasets either represent urban environments or lack multimodal off-road data. We fill this gap with RELLIS-3D, a multimodal dataset collected in an off-road environment, which contains annotations for 13,556 LiDAR scans and 6,235 images. The data was collected on the Rellis Campus of Texas A\&M University and presents challenges to existing algorithms related to class imbalance and environmental topography. Additionally, we evaluate the current state-of-the-art deep learning semantic segmentation models on this dataset. Experimental results show that RELLIS-3D presents challenges for algorithms designed for segmentation in urban environments. This novel dataset provides the resources needed by researchers to continue to develop more advanced algorithms and investigate new research directions to enhance autonomous navigation in off-road environments. RELLIS-3D is available at https://github.com/unmannedlab/RELLIS-3D △ Less

Submitted 25 May, 2022; v1 submitted 17 November, 2020; originally announced November 2020.

Comments: 7 pages, 7 figures

ACM Class: I.2.9; I.2.10; I.4.8

An Iterative LQR Controller for Off-Road and On-Road Vehicles using a Neural Network Dynamics Model

Authors: Akhil Nagariya, Srikanth Saripalli

Abstract: In this work we evaluate Iterative Linear Quadratic Regulator(ILQR) for trajectory tracking of two different kinds of wheeled mobile robots namely Warthog (Fig. 1), an off-road holonomic robot with skid-steering and Polaris GEM e6 [1], a non-holonomic six seater vehicle (Fig. 2). We use multilayer neural network to learn the discrete dynamic model of these robots which is used in ILQR controller t… ▽ More In this work we evaluate Iterative Linear Quadratic Regulator(ILQR) for trajectory tracking of two different kinds of wheeled mobile robots namely Warthog (Fig. 1), an off-road holonomic robot with skid-steering and Polaris GEM e6 [1], a non-holonomic six seater vehicle (Fig. 2). We use multilayer neural network to learn the discrete dynamic model of these robots which is used in ILQR controller to compute the control law. We use model predictive control (MPC) to deal with model imperfections and perform extensive experiments to evaluate the performance of the controller on human driven reference trajectories with vehicle speeds of 3m/s- 4m/s for warthog and 7m/s-10m/s for the Polaris GEM △ Less

Submitted 28 July, 2020; originally announced July 2020.

Experimental Evaluation of 3D-LIDAR Camera Extrinsic Calibration

Authors: Subodh Mishra, Philip R. Osteen, Gaurav Pandey, Srikanth Saripalli

Abstract: In this paper we perform an experimental comparison of three different target based 3D-LIDAR camera calibration algorithms. We briefly elucidate the mathematical background behind each method and provide insights into practical aspects like ease of data collection for all of them. We extensively evaluate these algorithms on a sensor suite which consists multiple cameras and LIDARs by assessing the… ▽ More In this paper we perform an experimental comparison of three different target based 3D-LIDAR camera calibration algorithms. We briefly elucidate the mathematical background behind each method and provide insights into practical aspects like ease of data collection for all of them. We extensively evaluate these algorithms on a sensor suite which consists multiple cameras and LIDARs by assessing their robustness to random initialization and by using metrics like Mean Line Re-projection Error (MLRE) and Factory Stereo Calibration Error. We also show the effect of noisy sensor on the calibration result from all the algorithms and conclude with a note on which calibration algorithm should be used under what circumstances. △ Less

Submitted 3 July, 2020; originally announced July 2020.

Comments: Accepted at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October 25-29, 2020 | Las Vegas, NV, USA

Extrinsic Calibration of a 3D-LIDAR and a Camera

Authors: Subodh Mishra, Gaurav Pandey, Srikanth Saripalli

Abstract: This work presents an extrinsic parameter estimation algorithm between a 3D LIDAR and a Projective Camera using a marker-less planar target, by exploiting Planar Surface Point to Plane and Planar Edge Point to back-projected Plane geometric constraints. The proposed method uses the data collected by placing the planar board at different poses in the common field of view of the LIDAR and the Camera… ▽ More This work presents an extrinsic parameter estimation algorithm between a 3D LIDAR and a Projective Camera using a marker-less planar target, by exploiting Planar Surface Point to Plane and Planar Edge Point to back-projected Plane geometric constraints. The proposed method uses the data collected by placing the planar board at different poses in the common field of view of the LIDAR and the Camera. The steps include, detection of the target and the edges of the target in LIDAR and Camera frames, matching the detected planes and lines across both the sensing modalities and finally solving a cost function formed by the aforementioned geometric constraints that link the features detected in both the LIDAR and the Camera using non-linear least squares. We have extensively validated our algorithm using two Basler Cameras, Velodyne VLP-32 and Ouster OS1 LIDARs. △ Less

Submitted 19 May, 2020; v1 submitted 2 March, 2020; originally announced March 2020.

Comments: Accepted at the 31st IEEE Intelligent Vehicles Symposium October 20-23, 2020 | Las Vegas, NV, United States

LiDARNet: A Boundary-Aware Domain Adaptation Model for Point Cloud Semantic Segmentation

Authors: Peng Jiang, Srikanth Saripalli

Abstract: We present a boundary-aware domain adaptation model for LiDAR scan full-scene semantic segmentation (LiDARNet). Our model can extract both the domain private features and the domain shared features with a two-branch structure. We embedded Gated-SCNN into the segmentor component of LiDARNet to learn boundary information while learning to predict full-scene semantic segmentation labels. Moreover, we… ▽ More We present a boundary-aware domain adaptation model for LiDAR scan full-scene semantic segmentation (LiDARNet). Our model can extract both the domain private features and the domain shared features with a two-branch structure. We embedded Gated-SCNN into the segmentor component of LiDARNet to learn boundary information while learning to predict full-scene semantic segmentation labels. Moreover, we further reduce the domain gap by inducing the model to learn a mapping between two domains using the domain shared and private features. Additionally, we introduce a new dataset (SemanticUSL\footnote{The access address of SemanticUSL:\url{https://unmannedlab.github.io/research/SemanticUSL}}) for domain adaptation for LiDAR point cloud semantic segmentation. The dataset has the same data format and ontology as SemanticKITTI. We conducted experiments on real-world datasets SemanticKITTI, SemanticPOSS, and SemanticUSL, which have differences in channel distributions, reflectivity distributions, diversity of scenes, and sensors setup. Using our approach, we can get a single projection-based LiDAR full-scene semantic segmentation model working on both domains. Our model can keep almost the same performance on the source domain after adaptation and get an 8\%-22\% mIoU performance increase in the target domain. △ Less

Submitted 24 April, 2021; v1 submitted 2 March, 2020; originally announced March 2020.

Comments: 8 pages, 3 figures

ACM Class: I.2.10; I.4.6

Fast Local Planning and Mapping in Unknown Off-Road Terrain

Authors: Timothy Overbye, Srikanth Saripalli

Abstract: In this paper, we present a fast, on-line mapping and planning solution for operation in unknown, off-road, environments. We combine obstacle detection along with a terrain gradient map to make simple and adaptable cost map. This map can be created and updated at 10 Hz. An A* planner finds optimal paths over the map. Finally, we take multiple samples over the control input space and do a kinematic… ▽ More In this paper, we present a fast, on-line mapping and planning solution for operation in unknown, off-road, environments. We combine obstacle detection along with a terrain gradient map to make simple and adaptable cost map. This map can be created and updated at 10 Hz. An A* planner finds optimal paths over the map. Finally, we take multiple samples over the control input space and do a kinematic forward simulation to generated feasible trajectories. Then the most optimal trajectory, as determined by the cost map and proximity to A* path, is chosen and sent to the controller. Our method allows real time operation at rates of 30 Hz. We demonstrate the efficiency of our method in various off-road terrain at high speed. △ Less

Submitted 18 October, 2019; originally announced October 2019.

Comments: 7 pages, 14 figures

Autonomous Shuttles for Last-Mile Connectivity

Authors: Garrison Neel, Amir Darwesh, Quang Le, Srikanth Saripalli

Abstract: This paper describes an autonomous shuttle which targets providing last-mile transportation. Often, this involves operation in crowded areas with high levels of pedestrian traffic, and little to no lane markings or traffic control. We aim to create a functional shuttle to be improved upon in the future as new robust solutions are developed to replace the current components. An initial implementati… ▽ More This paper describes an autonomous shuttle which targets providing last-mile transportation. Often, this involves operation in crowded areas with high levels of pedestrian traffic, and little to no lane markings or traffic control. We aim to create a functional shuttle to be improved upon in the future as new robust solutions are developed to replace the current components. An initial implementation of such a shuttle presented, detailing the overall architecture, controller structure, waypoint following, obstacle detection and avoidance, LiDAR based sign detection, and pedestrian communication. The performance of each component is evaluated, and future improvements are discussed. △ Less

Submitted 11 October, 2019; originally announced October 2019.

Collaborative Localization for Micro Aerial Vehicles

Authors: Sai Vemprala, Srikanth Saripalli

Abstract: In this paper, we present a framework for performing collaborative localization for groups of micro aerial vehicles (MAV) that use vision based sensing. The vehicles are each assumed to be equipped with a forward-facing monocular camera, and to be capable of communicating with each other. This collaborative localization approach is developed as a decentralized algorithm and built in a distributed… ▽ More In this paper, we present a framework for performing collaborative localization for groups of micro aerial vehicles (MAV) that use vision based sensing. The vehicles are each assumed to be equipped with a forward-facing monocular camera, and to be capable of communicating with each other. This collaborative localization approach is developed as a decentralized algorithm and built in a distributed fashion where individual and relative pose estimation techniques are combined for the group to localize against surrounding environments. The MAVs initially detect and match salient features between each other to create a sparse reconstruction of the observed environment, which acts as a global map. Once a map is available, each MAV performs feature detection and tracking with a robust outlier rejection process to estimate its own pose in 6 degrees of freedom. Occasionally, one or more MAVs can be tasked to compute poses for another MAV through relative measurements, which is achieved by exploiting multiple view geometry concepts. These relative measurements are then fused with individual measurements in a consistent fashion. We present the results of the algorithm on image data from MAV flights both in simulation and real life, and discuss the advantages of collaborative localization in improving pose estimation accuracy. △ Less

Submitted 10 May, 2020; v1 submitted 7 May, 2019; originally announced May 2019.

Comments: Supplementary video at https://www.youtube.com/watch?v=LvaTOWuTOPo

Drone Detection Using Depth Maps

Authors: Adrian Carrio, Sai Vemprala, Andres Ripoll, Srikanth Saripalli, Pascual Campoy

Abstract: Obstacle avoidance is a key feature for safe Unmanned Aerial Vehicle (UAV) navigation. While solutions have been proposed for static obstacle avoidance, systems enabling avoidance of dynamic objects, such as drones, are hard to implement due to the detection range and field-of-view (FOV) requirements, as well as the constraints for integrating such systems on-board small UAVs. In this work, a data… ▽ More Obstacle avoidance is a key feature for safe Unmanned Aerial Vehicle (UAV) navigation. While solutions have been proposed for static obstacle avoidance, systems enabling avoidance of dynamic objects, such as drones, are hard to implement due to the detection range and field-of-view (FOV) requirements, as well as the constraints for integrating such systems on-board small UAVs. In this work, a dataset of 6k synthetic depth maps of drones has been generated and used to train a state-of-the-art deep learning-based drone detection model. While many sensing technologies can only provide relative altitude and azimuth of an obstacle, our depth map-based approach enables full 3D localization of the obstacle. This is extremely useful for collision avoidance, as 3D localization of detected drones is key to perform efficient collision-free path planning. The proposed detection technique has been validated in several real depth map sequences, with multiple types of drones flying at up to 2 m/s, achieving an average precision of 98.7%, an average recall of 74.7% and a record detection range of 9.5 meters. △ Less

Submitted 1 August, 2018; originally announced August 2018.

Comments: Accepted at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2018), Madrid, Spain

Monocular Vision based Collaborative Localization for Micro Aerial Vehicle Swarms

Authors: Sai Vemprala, Srikanth Saripalli

Abstract: In this paper, we present a vision based collaborative localization framework for groups of micro aerial vehicles (MAV). The vehicles are each assumed to be equipped with a forward-facing monocular camera, and to be capable of communicating with each other. This collaborative localization approach is built upon a distributed algorithm where individual and relative pose estimation techniques are co… ▽ More In this paper, we present a vision based collaborative localization framework for groups of micro aerial vehicles (MAV). The vehicles are each assumed to be equipped with a forward-facing monocular camera, and to be capable of communicating with each other. This collaborative localization approach is built upon a distributed algorithm where individual and relative pose estimation techniques are combined for the group to localize against surrounding environments. The MAVs initially detect and match salient features between each other to create a sparse reconstruction of the observed environment, which acts as a global map. Once a map is available, each MAV performs feature detection and tracking with a robust outlier rejection process to estimate its own six degree-of-freedom pose. Occasionally, the MAVs can also fuse relative measurements with individual measurements through feature matching and multiple-view geometry based relative pose computation. We present the implementation of this algorithm for MAVs and environments simulated within Microsoft AirSim, and discuss the results and the advantages of collaborative localization. △ Less

Submitted 7 April, 2018; originally announced April 2018.

Comments: 9 pages, 12 figures, IEEE conference format

Toward a Science of Autonomy for Physical Systems: Aerial Earth Science

Authors: M. Ani Hsieh, Srikanth Saripalli, Gaurav Sukhatme, Vijay Kumar

Abstract: Unmanned Aerial Vehicles (UAVs) equipped with LiDAR, electro-optical and infrared cameras, SAR and atmospheric sensors have transformed the way we acquire high spatio-temporal resolution data. For example, UAVs equipped with these sensors have been able to obtain topography at resolutions of less than one meter, revolutionizing earth sciences. Surface processes act at spatial scales on the order o… ▽ More Unmanned Aerial Vehicles (UAVs) equipped with LiDAR, electro-optical and infrared cameras, SAR and atmospheric sensors have transformed the way we acquire high spatio-temporal resolution data. For example, UAVs equipped with these sensors have been able to obtain topography at resolutions of less than one meter, revolutionizing earth sciences. Surface processes act at spatial scales on the order of a meter to produce intricate landforms and UAVs equipped with these sensors are able to measure the three dimensional spatio-temporal geometry of the earths surface and overlying anthropogenic features and vegetation at resolutions appropriate to document these processes. In addition, surface changes due to erosion, transport and sedimentation, as well as earthquakes, landslides, volcanoes can be quantified with this data. △ Less

Submitted 19 September, 2016; originally announced September 2016.

Comments: A Computing Community Consortium (CCC) white paper, 10 pages