Adaptive Control Strategy for Quadruped Robots in Actuator Degradation Scenarios
Authors: 
Xinyuan Wu, Wentao Dong, Hang Lai, Yong Yu, and Ying WenAuthors Info & Claims
Xinyuan Wu, Wentao Dong, Hang Lai, Yong Yu, and Ying WenAuthors Info & ClaimsNovember 2023
Article No.: 14, Pages 1 - 13
Abstract
Quadruped robots have strong adaptability to extreme environments but may also experience faults. Once these faults occur, robots must be repaired before returning to the task, reducing their practical feasibility. One prevalent concern among these faults is actuator degradation, stemming from factors like device aging or unexpected operational events. Traditionally, addressing this problem has relied heavily on intricate fault-tolerant design, which demands deep domain expertise from developers and lacks generalizability. Learning-based approaches offer effective ways to mitigate these limitations, but a research gap exists in effectively deploying such methods on real-world quadruped robots. This paper introduces a pioneering teacher-student framework rooted in reinforcement learning, named Actuator Degeneration Adaptation Transformer (Adapt), aimed at addressing this research gap. This framework produces a unified control strategy, enabling the robot to sustain its locomotion and perform tasks despite sudden joint actuator faults, relying exclusively on its internal sensors. Empirical evaluations on the Unitree A1 platform validate the deployability and effectiveness of Adapt on real-world quadruped robots, and affirm the robustness and practicality of our approach.
Supplementary Material
Presentation video
- Download
- 83.50 MB
References
[1]
Maxime Allard, Simón C Smith, Konstantinos Chatzilygeroudis, Bryan Lim, and Antoine Cully. 2023. Online Damage Recovery for Physical Robots with Hierarchical Quality-Diversity. ACM Transactions on Evolutionary Learning 3, 2 (2023), 1–23.
[2]
Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, and Anil Anthony Bharath. 2017. Deep reinforcement learning: A brief survey. IEEE Signal Processing Magazine 34, 6 (2017), 26–38.
[3]
C Dario Bellicoso, Marko Bjelonic, Lorenz Wellhausen, Kai Holtmann, Fabian Günther, Marco Tranzatto, Peter Fankhauser, and Marco Hutter. 2018. Advances in real-world applications for legged robots. Journal of Field Robotics 35, 8 (2018), 1311–1326.
[4]
Matteo Bettini, Ajay Shankar, and Amanda Prorok. 2023. Heterogeneous Multi-Robot Reinforcement Learning. arxiv:2301.07137 [cs.RO]
[5]
Priyaranjan Biswal and Prases K Mohanty. 2021. Development of quadruped walking robots: A review. Ain Shams Engineering Journal 12, 2 (2021), 2017–2031.
[6]
Adrian Boeing and Thomas Bräunl. 2012. Leveraging multiple simulators for crossing the reality gap. In 2012 12th international conference on control automation robotics & vision (ICARCV). IEEE, 1113–1119.
[7]
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. 2016. Openai gym. arXiv preprint arXiv:1606.01540 (2016).
[8]
Jinlin Chen, Jiannong Cao, Zhiqin Cheng, and Wei Li. 2023. Mitigating Imminent Collision for Multi-Robot Navigation: A TTC-Force Reward Shaping Approach. In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems (London, United Kingdom) (AAMAS ’23). International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 1448–1456.
[9]
Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. 2021. Decision transformer: Reinforcement learning via sequence modeling. Advances in neural information processing systems 34 (2021), 15084–15097.
[10]
Zhijun Chen, Qingxing Xi, Feng Gao, and Yue Zhao. 2022. Fault-tolerant gait design for quadruped robots with one locked leg using the GF set theory. Mechanism and Machine Theory 178 (2022), 105069.
[11]
Junwen Cui, Zhan Li, Jing Qiu, and Tianxiao Li. 2022. Fault-tolerant motion planning and generation of quadruped robots synthesised by posture optimization and whole body control. Complex & Intelligent Systems 8, 4 (2022), 2991–3003.
[12]
Jared Di Carlo, Patrick M Wensing, Benjamin Katz, Gerardo Bledt, and Sangbae Kim. 2018. Dynamic locomotion in the mit cheetah 3 through convex model-predictive control. In 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, 1–9.
[13]
Alejandro Escontrela, Xue Bin Peng, Wenhao Yu, Tingnan Zhang, Atil Iscen, Ken Goldberg, and Pieter Abbeel. 2022. Adversarial motion priors make good substitutes for complex reward functions. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 25–32.
[14]
Junsong FAN, Yuxi WANG, He GUAN, Chunfeng SONG, and Zhaoxiang ZHANG. 2022. Toward few-shot domain adaptation with perturbation-invariant representation and transferable prototypes. Frontiers of Computer Science 16, 3, Article 163347 (2022). https://doi.org/10.1007/s11704-022-2015-7
[15]
Dongdong Guo, Li Fu, and Lingling Wang. 2019. Robots solving the urgent problems by themselves: A review. In 2019 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW). IEEE, 1–2.
[16]
Jonathan Ho and Stefano Ermon. 2016. Generative adversarial imitation learning. Advances in neural information processing systems 29 (2016).
[17]
Carolin Kemper and Michael Kolain. 2022. K9 Police Robots-Strolling Drones, RoboDogs, or Lethal Weapons?. In Accepted paper at WeRobot2022 conference.
[18]
Eliahu Khalastchi and Meir Kalech. 2018. A sensor-based approach for fault detection and diagnosis for robotic systems. Autonomous Robots 42 (2018), 1231–1248.
[19]
Sylvain Koos, Jean-Baptiste Mouret, and Stéphane Doncieux. 2010. Crossing the reality gap in evolutionary robotics by promoting transferable controllers. In Proceedings of the 12th annual conference on Genetic and evolutionary computation. 119–126.
[20]
Eric Krotkov, Douglas Hackett, Larry Jackel, Michael Perschbacher, James Pippine, Jesse Strauss, Gill Pratt, and Christopher Orlowski. 2017. The DARPA Robotics Challenge Finals: Results and Perspectives. Journal of Field Robotics 34, 2 (2017), 229–240. https://doi.org/10.1002/rob.21683 arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1002/rob.21683
[21]
Ashish Kumar, Zipeng Fu, Deepak Pathak, and Jitendra Malik. 2021. Rma: Rapid motor adaptation for legged robots. arXiv preprint arXiv:2107.04034 (2021).
[22]
Hang Lai, Weinan Zhang, Xialin He, Chen Yu, Zheng Tian, Yong Yu, and Jun Wang. 2023. Sim-to-Real Transfer for Quadrupedal Locomotion via Terrain Transformer. In 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 5141–5147.
[23]
Joonho Lee, Jemin Hwangbo, Lorenz Wellhausen, Vladlen Koltun, and Marco Hutter. 2020. Learning quadrupedal locomotion over challenging terrain. Science robotics 5, 47 (2020), eabc5986.
[24]
Dikai Liu, Tianwei Zhang, Jianxiong Yin, and Simon See. 2022. Saving the Limping: Fault-tolerant Quadruped Locomotion via Reinforcement Learning. arXiv preprint arXiv:2210.00474 (2022).
[25]
Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, 2021. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108.10470 (2021).
[26]
Gabriel B Margolis, Ge Yang, Kartik Paigwar, Tao Chen, and Pulkit Agrawal. 2022. Rapid locomotion via reinforcement learning. arXiv preprint arXiv:2205.02824 (2022).
[27]
Takahiro Miki, Joonho Lee, Jemin Hwangbo, Lorenz Wellhausen, Vladlen Koltun, and Marco Hutter. 2022. Learning robust perceptive locomotion for quadrupedal robots in the wild. Science Robotics 7, 62 (2022), eabk2822.
[28]
I Made Aswin Nahrendra, Byeongho Yu, and Hyun Myung. 2023. DreamWaQ: Learning robust quadrupedal locomotion with implicit terrain imagination via deep reinforcement learning. In 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 5078–5084.
[29]
Wataru Okamoto and Kazuhiko Kawamoto. 2020. Reinforcement learning with randomized physical parameters for fault-tolerant robots. In 2020 Joint 11th International Conference on Soft Computing and Intelligent Systems and 21st International Symposium on Advanced Intelligent Systems (SCIS-ISIS). IEEE, 1–4.
[30]
Xue Bin Peng, Marcin Andrychowicz, Wojciech Zaremba, and Pieter Abbeel. 2018. Sim-to-real transfer of robotic control with dynamics randomization. In 2018 IEEE international conference on robotics and automation (ICRA). IEEE, 3803–3810.
[31]
Xue Bin Peng, Erwin Coumans, Tingnan Zhang, Tsang-Wei Lee, Jie Tan, and Sergey Levine. 2020. Learning agile robotic locomotion skills by imitating animals. arXiv preprint arXiv:2004.00784 (2020).
[32]
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, 2018. Improving language understanding by generative pre-training. (2018).
[33]
Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, 2022. A generalist agent. arXiv preprint arXiv:2205.06175 (2022).
[34]
Unitree Robotics. 2020. Unitree A1. Online. https://www.unitree.com/a1
[35]
Alejandro Romero, Gianluca Baldassarre, Richard J. Duro, and Vieri Giuliano Santucci. 2023. Learning Multiple Tasks with Non-Stationary Interdependencies in Autonomous Robots. In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems (London, United Kingdom) (AAMAS ’23). International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 2547–2549.
[36]
Nikita Rudin, David Hoeller, Philipp Reist, and Marco Hutter. 2022. Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning. In Proceedings of the 5th Conference on Robot Learning(Proceedings of Machine Learning Research, Vol. 164), Aleksandra Faust, David Hsu, and Gerhard Neumann (Eds.). PMLR, 91–100. https://proceedings.mlr.press/v164/rudin22a.html
[37]
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017).
[38]
Prasanth Sengadu Suresh, Yikang Gui, and Prashant Doshi. 2023. Dec-AIRL: Decentralized Adversarial IRL for Human-Robot Teaming. In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems (London, United Kingdom) (AAMAS ’23). International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 1116–1124.
[39]
Jie Tan, Tingnan Zhang, Erwin Coumans, Atil Iscen, Yunfei Bai, Danijar Hafner, Steven Bohez, and Vincent Vanhoucke. 2018. Sim-to-real: Learning agile locomotion for quadruped robots. arXiv preprint arXiv:1804.10332 (2018).
[40]
Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel. 2017. Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, 23–30.
[41]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).
[42]
Muning WEN, Runji LIN, Hanjing WANG, Yaodong YANG, Ying WEN, Luo MAI, Jun WANG, Haifeng ZHANG, and Weinan ZHANG. 2023. Large sequence models for sequential decision-making: a survey. Frontiers of Computer Science 17, 6, Article 176349 (2023). https://doi.org/10.1007/s11704-023-2689-5
[43]
Jinze Wu, Guiyang Xin, Chenkun Qi, and Yufei Xue. 2023. Learning Robust and Agile Legged Locomotion Using Adversarial Motion Priors. IEEE Robotics and Automation Letters (2023).
[44]
Zichen Yan, Junbo Tan, Bin Liang, Houde Liu, and Jun Yang. 2022. Active Fault-Tolerant Control Integrated with Reinforcement Learning Application to Robotic Manipulator. In 2022 American Control Conference (ACC). IEEE, 2656–2662.
[45]
Chen Yu, Weinan Zhang, Hang Lai, Zheng Tian, Laurent Kneip, and Jun Wang. 2023. Multi-embodiment Legged Robot Control as a Sequence Modeling Problem. In 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 7250–7257.
[46]
Wenhao Yu, Jie Tan, C Karen Liu, and Greg Turk. 2017. Preparing for the unknown: Learning a universal policy with online system identification. arXiv preprint arXiv:1702.02453 (2017).
[47]
Yu Zhang. 2023. From Abstractions to Grounded Languages for Robust Coordination of Task Planning Robots. In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems (London, United Kingdom) (AAMAS ’23). International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 2535–2537.
[48]
Rui Zhao, Xu Liu, Yizheng Zhang, Minghao Li, Cheng Zhou, Shuai Li, and Lei Han. 2023. CraftEnv: A Flexible Collective Robotic Construction Environment for Multi-Agent Reinforcement Learning. In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems (London, United Kingdom) (AAMAS ’23). International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 1164–1172.
[49]
Yong Yong Zhao, Jing Hua Wang, Bao Wen Zhang, Xu Yao, and Guo Hua Cao. 2023. Gait planning and fault-tolerant control of quadruped robots. In Third International Conference on Mechanical, Electronics, and Electrical and Automation Control (METMS 2023), Vol. 12722. SPIE, 841–846.
Index Terms
- Adaptive Control Strategy for Quadruped Robots in Actuator Degradation Scenarios
Index terms have been assigned to the content through auto-classification.
Recommendations
Responsive CPG-Based Locomotion Control for Quadruped Robots
Neural Information ProcessingAbstractQuadruped robots with flexible movement are gradually replacing traditional mobile robots in many spots. To improve the motion stability and speed of the quadruped robot, this paper presents a responsive gradient CPG (RG-CPG) approach. ...
Distributed-force-feedback-based reflex with online learning for adaptive quadruped motor control
AbstractBiological motor control mechanisms (e.g., central pattern generators (CPGs), sensory feedback, reflexes, and motor learning) play a crucial role in the adaptive locomotion of animals. However, the interaction and integration of these ...
Spinal joint compliance and actuation in a simulated bounding quadruped robot
Spine movements play an important role in quadrupedal locomotion, yet their potential benefits in locomotion of quadruped robots have not been systematically explored. In this work, we investigate the role of spinal joint actuation and compliance on the ...
Comments
Information & Contributors
Information
Published In
November 2023
139 pages
ISBN:9798400708480
DOI:10.1145/3627676
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 30 December 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
DAI '23
DAI '23: The Fifth International Conference on Distributed Artificial Intelligence
November 30 - December 3, 2023
Singapore, Singapore
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 67Total Downloads
- Downloads (Last 12 months)67
- Downloads (Last 6 weeks)12
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format