-
Model Predictive Path Integral Control for Agile Unmanned Aerial Vehicles
Authors:
Michal Minarik,
Robert Penicka,
Vojtech Vonasek,
Martin Saska
Abstract:
This paper introduces a control architecture for real-time and onboard control of Unmanned Aerial Vehicles (UAVs) in environments with obstacles using the Model Predictive Path Integral (MPPI) methodology. MPPI allows the use of the full nonlinear model of UAV dynamics and a more general cost function at the cost of a high computational demand. To run the controller in real-time, the sampling-base…
▽ More
This paper introduces a control architecture for real-time and onboard control of Unmanned Aerial Vehicles (UAVs) in environments with obstacles using the Model Predictive Path Integral (MPPI) methodology. MPPI allows the use of the full nonlinear model of UAV dynamics and a more general cost function at the cost of a high computational demand. To run the controller in real-time, the sampling-based optimization is performed in parallel on a graphics processing unit onboard the UAV. We propose an approach to the simulation of the nonlinear system which respects low-level constraints, while also able to dynamically handle obstacle avoidance, and prove that our methods are able to run in real-time without the need for external computers. The MPPI controller is compared to MPC and SE(3) controllers on the reference tracking task, showing a comparable performance. We demonstrate the viability of the proposed method in multiple simulation and real-world experiments, tracking a reference at up to 44 km/h and acceleration close to 20 m/s^2, while still being able to avoid obstacles. To the best of our knowledge, this is the first method to demonstrate an MPPI-based approach in real flight.
△ Less
Submitted 13 July, 2024;
originally announced July 2024.
-
Gesture-Controlled Aerial Robot Formation for Human-Swarm Interaction in Safety Monitoring Applications
Authors:
Vít Krátký,
Giuseppe Silano,
Matouš Vrba,
Christos Papaioannidis,
Ioannis Mademlis,
Robert Pěnička,
Ioannis Pitas,
Martin Saska
Abstract:
This paper presents a formation control approach for contactless gesture-based Human-Swarm Interaction (HSI) between a team of multi-rotor Unmanned Aerial Vehicles (UAVs) and a human worker. The approach is intended for monitoring the safety of human workers, especially those working at heights. In the proposed dynamic formation scheme, one UAV acts as the leader of the formation and is equipped w…
▽ More
This paper presents a formation control approach for contactless gesture-based Human-Swarm Interaction (HSI) between a team of multi-rotor Unmanned Aerial Vehicles (UAVs) and a human worker. The approach is intended for monitoring the safety of human workers, especially those working at heights. In the proposed dynamic formation scheme, one UAV acts as the leader of the formation and is equipped with sensors for human worker detection and gesture recognition. The follower UAVs maintain a predetermined formation relative to the worker's position, thereby providing additional perspectives of the monitored scene. Hand gestures allow the human worker to specify movements and action commands for the UAV team and initiate other mission-related commands without the need for an additional communication channel or specific markers. Together with a novel unified human detection and tracking algorithm, human pose estimation approach and gesture detection pipeline, the proposed approach forms a first instance of an HSI system incorporating all these modules onboard real-world UAVs. Simulations and field experiments with three UAVs and a human worker in a mock-up scenario showcase the effectiveness and responsiveness of the proposed approach.
△ Less
Submitted 22 March, 2024;
originally announced March 2024.
-
Energy-aware Multi-UAV Coverage Mission Planning with Optimal Speed of Flight
Authors:
Denys Datsko,
Frantisek Nekovar,
Robert Penicka,
Martin Saska
Abstract:
This paper tackles the problem of planning minimum-energy coverage paths for multiple UAVs. The addressed Multi-UAV Coverage Path Planning (mCPP) is a crucial problem for many UAV applications such as inspection and aerial survey. However, the typical path-length objective of existing approaches does not directly minimize the energy consumption, nor allows for constraining energy of individual pat…
▽ More
This paper tackles the problem of planning minimum-energy coverage paths for multiple UAVs. The addressed Multi-UAV Coverage Path Planning (mCPP) is a crucial problem for many UAV applications such as inspection and aerial survey. However, the typical path-length objective of existing approaches does not directly minimize the energy consumption, nor allows for constraining energy of individual paths by the battery capacity. To this end, we propose a novel mCPP method that uses the optimal flight speed for minimizing energy consumption per traveled distance and a simple yet precise energy consumption estimation algorithm that is utilized during the mCPP planning phase. The method decomposes a given area with boustrophedon decomposition and represents the mCPP as an instance of Multiple Set Traveling Salesman Problem with a minimum energy objective and energy consumption constraint. The proposed method is shown to outperform state-of-the-art methods in terms of computational time and energy efficiency of produced paths. The experimental results show that the accuracy of the energy consumption estimation is on average 97% compared to real flight consumption. The feasibility of the proposed method was verified in a real-world coverage experiment with two UAVs.
△ Less
Submitted 16 February, 2024;
originally announced February 2024.
-
Agilicious: Open-Source and Open-Hardware Agile Quadrotor for Vision-Based Flight
Authors:
Philipp Foehn,
Elia Kaufmann,
Angel Romero,
Robert Penicka,
Sihao Sun,
Leonard Bauersfeld,
Thomas Laengle,
Giovanni Cioffi,
Yunlong Song,
Antonio Loquercio,
Davide Scaramuzza
Abstract:
Autonomous, agile quadrotor flight raises fundamental challenges for robotics research in terms of perception, planning, learning, and control. A versatile and standardized platform is needed to accelerate research and let practitioners focus on the core problems. To this end, we present Agilicious, a co-designed hardware and software framework tailored to autonomous, agile quadrotor flight. It is…
▽ More
Autonomous, agile quadrotor flight raises fundamental challenges for robotics research in terms of perception, planning, learning, and control. A versatile and standardized platform is needed to accelerate research and let practitioners focus on the core problems. To this end, we present Agilicious, a co-designed hardware and software framework tailored to autonomous, agile quadrotor flight. It is completely open-source and open-hardware and supports both model-based and neural-network--based controllers. Also, it provides high thrust-to-weight and torque-to-inertia ratios for agility, onboard vision sensors, GPU-accelerated compute hardware for real-time perception and neural-network inference, a real-time flight controller, and a versatile software stack. In contrast to existing frameworks, Agilicious offers a unique combination of flexible software stack and high-performance hardware. We compare Agilicious with prior works and demonstrate it on different agile tasks, using both model-based and neural-network--based controllers. Our demonstrators include trajectory tracking at up to 5g and 70 km/h in a motion-capture system, and vision-based acrobatic flight and obstacle avoidance in both structured and unstructured environments using solely onboard perception. Finally, we demonstrate its use for hardware-in-the-loop simulation in virtual-reality environments. Thanks to its versatility, we believe that Agilicious supports the next generation of scientific and industrial quadrotor research.
△ Less
Submitted 12 July, 2023;
originally announced July 2023.
-
MRS Drone: A Modular Platform for Real-World Deployment of Aerial Multi-Robot Systems
Authors:
Daniel Hert,
Tomas Baca,
Pavel Petracek,
Vit Kratky,
Robert Penicka,
Vojtech Spurny,
Matej Petrlik,
Matous Vrba,
David Zaitlik,
Pavel Stoudek,
Viktor Walter,
Petr Stepan,
Jiri Horyna,
Vaclav Pritzl,
Martin Sramek,
Afzal Ahmad,
Giuseppe Silano,
Daniel Bonilla Licea,
Petr Stibinger,
Tiago Nascimento,
Martin Saska
Abstract:
This paper presents a modular autonomous Unmanned Aerial Vehicle (UAV) platform called the Multi-robot Systems (MRS) Drone that can be used in a large range of indoor and outdoor applications. The MRS Drone features unique modularity with respect to changes in actuators, frames, and sensory configuration. As the name suggests, the platform is specially tailored for deployment within a MRS group. T…
▽ More
This paper presents a modular autonomous Unmanned Aerial Vehicle (UAV) platform called the Multi-robot Systems (MRS) Drone that can be used in a large range of indoor and outdoor applications. The MRS Drone features unique modularity with respect to changes in actuators, frames, and sensory configuration. As the name suggests, the platform is specially tailored for deployment within a MRS group. The MRS Drone contributes to the state-of-the-art of UAV platforms by allowing smooth real-world deployment of multiple aerial robots, as well as by outperforming other platforms with its modularity. For real-world multi-robot deployment in various applications, the platform is easy to both assemble and modify. Moreover, it is accompanied by a realistic simulator to enable safe pre-flight testing and a smooth transition to complex real-world experiments. In this manuscript, we present mechanical and electrical designs, software architecture, and technical specifications to build a fully autonomous multi UAV system. Finally, we demonstrate the full capabilities and the unique modularity of the MRS Drone in various real-world applications that required a diverse range of platform configurations.
△ Less
Submitted 12 June, 2023;
originally announced June 2023.
-
Metaheuristic planner for cooperative multi-agent wall construction with UAVs
Authors:
Basel Elkhapery,
Robert Pěnička,
Michal Němec,
Mohsin Siddiqui
Abstract:
This paper introduces a wall construction planner for Unmanned Aerial Vehicles (UAVs), which uses a Greedy Randomized Adaptive Search Procedure (GRASP) metaheuristic to generate near-time-optimal building plans for even large walls within seconds. This approach addresses one of the most time-consuming and labor-intensive tasks, while also minimizing workers' safety risks. To achieve this, the wall…
▽ More
This paper introduces a wall construction planner for Unmanned Aerial Vehicles (UAVs), which uses a Greedy Randomized Adaptive Search Procedure (GRASP) metaheuristic to generate near-time-optimal building plans for even large walls within seconds. This approach addresses one of the most time-consuming and labor-intensive tasks, while also minimizing workers' safety risks. To achieve this, the wall-building problem is modeled as a variant of the Team Orienteering Problem and is formulated as Mixed-Integer Linear Programming (MILP), with added precedence and concurrence constraints that ensure bricks are built in the correct order and without collision between cooperating agents. The GRASP planner is validated in a realistic simulation and demonstrated to find solutions with similar quality as the optimal MILP, but much faster. Moreover, it outperforms all other state-of-the-art planning approaches in the majority of test cases. This paper presents a significant advancement in the field of automated wall construction, demonstrating the potential of UAVs and optimization algorithms in improving the efficiency and safety of construction projects.
△ Less
Submitted 25 May, 2023;
originally announced May 2023.
-
CTopPRM: Clustering Topological PRM for Planning Multiple Distinct Paths in 3D Environments
Authors:
Matej Novosad,
Robert Penicka,
Vojtech Vonasek
Abstract:
In this paper, we propose a new method called Clustering Topological PRM (CTopPRM) for finding multiple homotopically distinct paths in 3D cluttered environments. Finding such distinct paths, e.g., going around an obstacle from a different side, is useful in many applications. Among others, using multiple distinct paths is necessary for optimization-based trajectory planners where found trajectori…
▽ More
In this paper, we propose a new method called Clustering Topological PRM (CTopPRM) for finding multiple homotopically distinct paths in 3D cluttered environments. Finding such distinct paths, e.g., going around an obstacle from a different side, is useful in many applications. Among others, using multiple distinct paths is necessary for optimization-based trajectory planners where found trajectories are restricted to only a single homotopy class of a given path. Distinct paths can also be used to guide sampling-based motion planning and thus increase the effectiveness of planning in environments with narrow passages. Graph-based representation called roadmap is a common representation for path planning and also for finding multiple distinct paths. However, challenging environments with multiple narrow passages require a densely sampled roadmap to capture the connectivity of the environment. Searching such a dense roadmap for multiple paths is computationally too expensive. Therefore, the majority of existing methods construct only a sparse roadmap which, however, struggles to find all distinct paths in challenging environments. To this end, we propose the CTopPRM which creates a sparse graph by clustering an initially sampled dense roadmap. Such a reduced roadmap allows fast identification of homotopically distinct paths captured in the dense roadmap. We show, that compared to the existing methods the CTopPRM improves the probability of finding all distinct paths by almost 20% in tested environments, during same run-time. The source code of our method is released as an open-source package.
△ Less
Submitted 28 September, 2023; v1 submitted 23 May, 2023;
originally announced May 2023.
-
MRS Modular UAV Hardware Platforms for Supporting Research in Real-World Outdoor and Indoor Environments
Authors:
Daniel Hert,
Tomas Baca,
Pavel Petracek,
Vit Kratky,
Vojtech Spurny,
Matej Petrlik,
Matous Vrba,
David Zaitlik,
Pavel Stoudek,
Viktor Walter,
Petr Stepan,
Jiri Horyna,
Vaclav Pritzl,
Giuseppe Silano,
Daniel Bonilla Licea,
Petr Stibinger,
Robert Penicka,
Tiago Nascimento,
Martin Saska
Abstract:
This paper presents a family of autonomous Unmanned Aerial Vehicles (UAVs) platforms designed for a diverse range of indoor and outdoor applications. The proposed UAV design is highly modular in terms of used actuators, sensor configurations, and even UAV frames. This allows to achieve, with minimal effort, a proper experimental setup for single, as well as, multi robot scenarios. Presented platfo…
▽ More
This paper presents a family of autonomous Unmanned Aerial Vehicles (UAVs) platforms designed for a diverse range of indoor and outdoor applications. The proposed UAV design is highly modular in terms of used actuators, sensor configurations, and even UAV frames. This allows to achieve, with minimal effort, a proper experimental setup for single, as well as, multi robot scenarios. Presented platforms are intended to facilitate the transition from simulations, and simplified laboratory experiments, into the deployment of aerial robots into uncertain and hard-to-model real-world conditions. We present mechanical designs, electric configurations, and dynamic models of the UAVs, followed by numerous recommendations and technical details required for building such a fully autonomous UAV system for experimental verification of scientific achievements. To show strength and high variability of the proposed system, we present results of tens of completely different real-robot experiments in various environments using distinct actuator and sensory configurations.
△ Less
Submitted 9 February, 2023;
originally announced February 2023.
-
Power Line Inspection Tasks with Multi-Aerial Robot Systems via Signal Temporal Logic Specifications
Authors:
Giuseppe Silano,
Tomas Baca,
Robert Penicka,
Davide Liuzza,
Martin Saska
Abstract:
A framework for computing feasible and constrained trajectories for a fleet of quad-rotors leveraging on Signal Temporal Logic (STL) specifications for power line inspection tasks is proposed in this paper. The planner allows the formulation of complex missions that avoid obstacles and maintain a safe distance between drones while performing the planned mission. An optimization problem is set to g…
▽ More
A framework for computing feasible and constrained trajectories for a fleet of quad-rotors leveraging on Signal Temporal Logic (STL) specifications for power line inspection tasks is proposed in this paper. The planner allows the formulation of complex missions that avoid obstacles and maintain a safe distance between drones while performing the planned mission. An optimization problem is set to generate optimal strategies that satisfy these specifications and also take vehicle constraints into account. Further, an event-triggered replanner is proposed to reply to unforeseen events and external disturbances. An energy minimization term is also considered to implicitly save quad-rotors battery life while carrying out the mission. Numerical simulations in MATLAB and experimental results show the validity and the effectiveness of the proposed approach, and demonstrate its applicability in real-world scenarios.
△ Less
Submitted 9 February, 2023;
originally announced February 2023.
-
Autonomous Drone Racing: A Survey
Authors:
Drew Hanover,
Antonio Loquercio,
Leonard Bauersfeld,
Angel Romero,
Robert Penicka,
Yunlong Song,
Giovanni Cioffi,
Elia Kaufmann,
Davide Scaramuzza
Abstract:
Over the last decade, the use of autonomous drone systems for surveying, search and rescue, or last-mile delivery has increased exponentially. With the rise of these applications comes the need for highly robust, safety-critical algorithms which can operate drones in complex and uncertain environments. Additionally, flying fast enables drones to cover more ground which in turn increases productivi…
▽ More
Over the last decade, the use of autonomous drone systems for surveying, search and rescue, or last-mile delivery has increased exponentially. With the rise of these applications comes the need for highly robust, safety-critical algorithms which can operate drones in complex and uncertain environments. Additionally, flying fast enables drones to cover more ground which in turn increases productivity and further strengthens their use case. One proxy for developing algorithms used in high-speed navigation is the task of autonomous drone racing, where researchers program drones to fly through a sequence of gates and avoid obstacles as quickly as possible using onboard sensors and limited computational power. Speeds and accelerations exceed over 80 kph and 4 g respectively, raising significant challenges across perception, planning, control, and state estimation. To achieve maximum performance, systems require real-time algorithms that are robust to motion blur, high dynamic range, model uncertainties, aerodynamic disturbances, and often unpredictable opponents. This survey covers the progression of autonomous drone racing across model-based and learning-based approaches. We provide an overview of the field, its evolution over the years, and conclude with the biggest challenges and open questions to be faced in the future.
△ Less
Submitted 8 July, 2024; v1 submitted 4 January, 2023;
originally announced January 2023.
-
Learning Perception-Aware Agile Flight in Cluttered Environments
Authors:
Yunlong Song,
Kexin Shi,
Robert Penicka,
Davide Scaramuzza
Abstract:
Recently, neural control policies have outperformed existing model-based planning-and-control methods for autonomously navigating quadrotors through cluttered environments in minimum time. However, they are not perception aware, a crucial requirement in vision-based navigation due to the camera's limited field of view and the underactuated nature of a quadrotor. We propose a learning-based system…
▽ More
Recently, neural control policies have outperformed existing model-based planning-and-control methods for autonomously navigating quadrotors through cluttered environments in minimum time. However, they are not perception aware, a crucial requirement in vision-based navigation due to the camera's limited field of view and the underactuated nature of a quadrotor. We propose a learning-based system that achieves perception-aware, agile flight in cluttered environments. Our method combines imitation learning with reinforcement learning (RL) by leveraging a privileged learning-by-cheating framework. Using RL, we first train a perception-aware teacher policy with full-state information to fly in minimum time through cluttered environments. Then, we use imitation learning to distill its knowledge into a vision-based student policy that only perceives the environment via a camera. Our approach tightly couples perception and control, showing a significant advantage in computation speed (10 times faster) and success rate. We demonstrate the closed-loop control performance using hardware-in-the-loop simulation.
△ Less
Submitted 3 March, 2023; v1 submitted 4 October, 2022;
originally announced October 2022.
-
Learning Minimum-Time Flight in Cluttered Environments
Authors:
Robert Penicka,
Yunlong Song,
Elia Kaufmann,
Davide Scaramuzza
Abstract:
We tackle the problem of minimum-time flight for a quadrotor through a sequence of waypoints in the presence of obstacles while exploiting the full quadrotor dynamics. Early works relied on simplified dynamics or polynomial trajectory representations that did not exploit the full actuator potential of the quadrotor, and, thus, resulted in suboptimal solutions. Recent works can plan minimum-time tr…
▽ More
We tackle the problem of minimum-time flight for a quadrotor through a sequence of waypoints in the presence of obstacles while exploiting the full quadrotor dynamics. Early works relied on simplified dynamics or polynomial trajectory representations that did not exploit the full actuator potential of the quadrotor, and, thus, resulted in suboptimal solutions. Recent works can plan minimum-time trajectories; yet, the trajectories are executed with control methods that do not account for obstacles. Thus, a successful execution of such trajectories is prone to errors due to model mismatch and in-flight disturbances. To this end, we leverage deep reinforcement learning and classical topological path planning to train robust neural-network controllers for minimum-time quadrotor flight in cluttered environments. The resulting neural network controller demonstrates substantially better performance of up to 19\% over state-of-the-art methods. More importantly, the learned policy solves the planning and control problem simultaneously online to account for disturbances, thus achieving much higher robustness. As such, the presented method achieves 100% success rate of flying minimum-time policies without collision, while traditional planning and control approaches achieve only 40%. The proposed method is validated in both simulation and the real world, with quadrotor speeds of up to 42km/h and accelerations of 3.6g.
△ Less
Submitted 17 June, 2022; v1 submitted 28 March, 2022;
originally announced March 2022.
-
Time-Optimal Online Replanning for Agile Quadrotor Flight
Authors:
Angel Romero,
Robert Penicka,
Davide Scaramuzza
Abstract:
In this paper, we tackle the problem of flying a quadrotor using time-optimal control policies that can be replanned online when the environment changes or when encountering unknown disturbances. This problem is challenging as the time-optimal trajectories that consider the full quadrotor dynamics are computationally expensive to generate (order of minutes or even hours). We introduce a sampling-b…
▽ More
In this paper, we tackle the problem of flying a quadrotor using time-optimal control policies that can be replanned online when the environment changes or when encountering unknown disturbances. This problem is challenging as the time-optimal trajectories that consider the full quadrotor dynamics are computationally expensive to generate (order of minutes or even hours). We introduce a sampling-based method for efficient generation of time-optimal paths of a point-mass model. These paths are then tracked using a Model Predictive Contouring Control approach that considers the full quadrotor dynamics and the single rotor thrust limits. Our combined approach is able to run in real-time, being the first time-optimal method that is able to adapt to changes on-the-fly. We showcase our approach's adaption capabilities by flying a quadrotor at more than 60 km/h in a racing track where gates are moving. Additionally, we show that our online replanning approach can cope with strong disturbances caused by winds of up to 68 km/h.
△ Less
Submitted 21 July, 2022; v1 submitted 18 March, 2022;
originally announced March 2022.
-
Minimum-Time Quadrotor Waypoint Flight in Cluttered Environments
Authors:
Robert Penicka,
Davide Scaramuzza
Abstract:
We tackle the problem of planning a minimum-time trajectory for a quadrotor over a sequence of specified waypoints in the presence of obstacles while exploiting the full quadrotor dynamics. This problem is crucial for autonomous search and rescue and drone racing scenarios but was, so far, unaddressed by the robotics community \emph{in its entirety} due to the challenges of minimizing time in the…
▽ More
We tackle the problem of planning a minimum-time trajectory for a quadrotor over a sequence of specified waypoints in the presence of obstacles while exploiting the full quadrotor dynamics. This problem is crucial for autonomous search and rescue and drone racing scenarios but was, so far, unaddressed by the robotics community \emph{in its entirety} due to the challenges of minimizing time in the presence of the non-convex constraints posed by collision avoidance. Early works relied on simplified dynamics or polynomial trajectory representations that did not exploit the full actuator potential of a quadrotor and, thus, did not aim at minimizing time. We address this challenging problem by using a hierarchical, sampling-based method with an incrementally more complex quadrotor model. Our method first finds paths in different topologies to guide subsequent trajectory search for a kinodynamic point-mass model. Then, it uses an asymptotically-optimal, kinodynamic sampling-based method based on a full quadrotor model on top of the point-mass solution to find a feasible trajectory with a time-optimal objective. The proposed method is shown to outperform all related baselines in cluttered environments and is further validated in real-world flights at over 60km/h in one of the world's largest motion capture systems. We release the code open source.
△ Less
Submitted 8 February, 2022;
originally announced February 2022.
-
Multi-goal path planning using multiple random trees
Authors:
Jaroslav Janoš,
Vojtěch Vonásek,
Robert Pěnička
Abstract:
In this paper, we propose a novel sampling-based planner for multi-goal path planning among obstacles, where the objective is to visit predefined target locations while minimizing the travel costs. The order of visiting the targets is often achieved by solving the Traveling Salesman Problem (TSP) or its variants. TSP requires to define costs between the individual targets, which - in a map with ob…
▽ More
In this paper, we propose a novel sampling-based planner for multi-goal path planning among obstacles, where the objective is to visit predefined target locations while minimizing the travel costs. The order of visiting the targets is often achieved by solving the Traveling Salesman Problem (TSP) or its variants. TSP requires to define costs between the individual targets, which - in a map with obstacles - requires to compute mutual paths between the targets. These paths, found by path planning, are used both to define the costs (e.g., based on their length or time-to-traverse) and also they define paths that are later used in the final solution. To enable TSP finding a good-quality solution, it is necessary to find these target-to-target paths as short as possible. We propose a sampling-based planner called Space-Filling Forest (SFF*) that solves the part of finding collision-free paths. SFF* uses multiple trees (forest) constructed gradually and simultaneously from the targets and attempts to find connections with other trees to form the paths. Unlike Rapidly-exploring Random Tree (RRT), which uses the nearest-neighbor rule for selecting nodes for expansion, SFF* maintains an explicit list of nodes for expansion. Individual trees are grown in a RRT* manner, i.e., with rewiring the nodes to minimize their cost. Computational results show that SFF* provides shorter target-to-target paths than existing approaches, and consequently, the final TSP solutions also have a lower cost.
△ Less
Submitted 7 June, 2021;
originally announced June 2021.
-
Autonomous Cooperative Wall Building by a Team of Unmanned Aerial Vehicles in the MBZIRC 2020 Competition
Authors:
Tomas Baca,
Robert Penicka,
Petr Stepan,
Matej Petrlik,
Vojtech Spurny,
Daniel Hert,
Martin Saska
Abstract:
This paper presents a system for autonomous cooperative wall building with a team of Unmanned Aerial Vehicles (UAVs). The system was developed for Challenge 2 of the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2020. The wall-building scenario of Challenge 2 featured an initial stack of bricks and wall structure where the individual bricks had to be placed by a team of three UAVs. T…
▽ More
This paper presents a system for autonomous cooperative wall building with a team of Unmanned Aerial Vehicles (UAVs). The system was developed for Challenge 2 of the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2020. The wall-building scenario of Challenge 2 featured an initial stack of bricks and wall structure where the individual bricks had to be placed by a team of three UAVs. The objective of the task was to maximize collected points for placing the bricks within the restricted construction time while following the prescribed wall pattern. The proposed approach uses initial scanning to find a priori unknown locations of the bricks and the wall structure. Each UAV is then assigned to individual bricks and wall placing locations and further perform grasping and placement using onboard resources only. The developed system consists of methods for scanning a given area, RGB-D detection of bricks and wall placement locations, precise grasping and placing of bricks, and coordination of multiple UAVs. The paper describes the overall system, individual components, experimental verification in demanding outdoor conditions, the achieved results in the competition, and lessons learned. The presented CTU-UPenn-NYU approach achieved the overall best performance among all participants to won the MBZIRC competition by collecting the highest number of points by correct placement of a high number of bricks.
△ Less
Submitted 10 December, 2020;
originally announced December 2020.
-
The MRS UAV System: Pushing the Frontiers of Reproducible Research, Real-world Deployment, and Education with Autonomous Unmanned Aerial Vehicles
Authors:
Tomas Baca,
Matej Petrlik,
Matous Vrba,
Vojtech Spurny,
Robert Penicka,
Daniel Hert,
Martin Saska
Abstract:
We present a multirotor Unmanned Aerial Vehicle control (UAV) and estimation system for supporting replicable research through realistic simulations and real-world experiments. We propose a unique multi-frame localization paradigm for estimating the states of a UAV in various frames of reference using multiple sensors simultaneously. The system enables complex missions in GNSS and GNSS-denied envi…
▽ More
We present a multirotor Unmanned Aerial Vehicle control (UAV) and estimation system for supporting replicable research through realistic simulations and real-world experiments. We propose a unique multi-frame localization paradigm for estimating the states of a UAV in various frames of reference using multiple sensors simultaneously. The system enables complex missions in GNSS and GNSS-denied environments, including outdoor-indoor transitions and the execution of redundant estimators for backing up unreliable localization sources. Two feedback control designs are presented: one for precise and aggressive maneuvers, and the other for stable and smooth flight with a noisy state estimate. The proposed control and estimation pipeline are constructed without using the Euler/Tait-Bryan angle representation of orientation in 3D. Instead, we rely on rotation matrices and a novel heading-based convention to represent the one free rotational degree-of-freedom in 3D of a standard multirotor helicopter. We provide an actively maintained and well-documented open-source implementation, including realistic simulation of UAV, sensors, and localization systems. The proposed system is the product of years of applied research on multi-robot systems, aerial swarms, aerial manipulation, motion planning, and remote sensing. All our results have been supported by real-world system deployment that shaped the system into the form presented here. In addition, the system was utilized during the participation of our team from the CTU in Prague in the prestigious MBZIRC 2017 and 2020 robotics competitions, and also in the DARPA SubT challenge. Each time, our team was able to secure top places among the best competitors from all over the world. On each occasion, the challenges has motivated the team to improve the system and to gain a great amount of high-quality experience within tight deadlines.
△ Less
Submitted 23 September, 2022; v1 submitted 18 August, 2020;
originally announced August 2020.
