Schedule

Reinforcement learning (RL) has shown great promise for enabling agents to learn complex behaviors in robotics, but real-world deployment faces challenges like generalization across dynamics, robustness to disturbances, and efficient task adaptation. This talk explores these challenges in the context of ground and aerial robotics, presenting innovative RL- and Imitation Learning-based frameworks for single- and multi-agent problems. For single-agent flight control, we introduce the Robust Tube Model Predictive Control (RTMPC) framework, which generates robust trajectories under uncertainty. Using Tube-Guided Data Augmentation, RTMPC trains policies resilient to disturbances, significantly reducing the need for large datasets. Building on this, we present GRAM, a deep RL framework for dynamics generalization, combining in-distribution adaptation and out-of-distribution robustness with an uncertainty-aware module for enhanced performance without expert-designed robustness. For multiagent planning, we highlight frameworks for decentralized and collaborative operations in uncertain environments. These include PUMA, a perception- and uncertainty-aware trajectory planner, PRIMER, which accelerates planning with imitation learning, and DYNUS, an optimization-based approach for fast, safe replanning. Finally, we discuss policy safety through uncertainty-aware planning and safety certification, introducing CARV for scalable reachability analysis and EVORA for probabilistic models for terrain navigation. Together, these frameworks demonstrate RL’s potential to address real-world challenges in robotics.

Bio: Jonathan P. How is the Richard C. Maclaurin Professor of Aeronautics and Astronautics at the Massachusetts Institute of Technology. He received a B.A.Sc. (Aerospace) from the University of Toronto in 1987, and his S.M. and Ph.D. in Aeronautics and Astronautics from MIT in 1990 and 1993, respectively, and then studied for 1.5 years at MIT as a postdoctoral associate. Prior to joining MIT in 2000, he was an assistant professor in the Department of Aeronautics and Astronautics at Stanford University.

Dr. How was the editor-in-chief of the IEEE Control Systems Magazine (2015-19), an associate editor for the AIAA Journal of Aerospace Information Systems (2012-21) and IEEE Transactions on Neural Networks and Learning Systems (2018-21). He was the Program Vice-chair (tutorials) for the 2021 Conference on Decision and Control and is the Program Chair for the American Control Conference in 2025. He was elected to the Board of Governors of the IEEE CSS for 2020- 22, is a member of the IEEE CSS Executive Committee (VP Finance) (2023-24), is on the IEEE CSS Long Range Planning Committee (2022 – ), is a member of the IEEE CSS Technical Committee on Aerospace Control and the Technical Committee on Intelligent Control, was a member of the IEEE Fellows Selection committee for CSS (2021-22), and since 2021, he serves as the AIAA Director on the American Automatic Control Council. He was a member of the USAF Scientific Advisory Board (SAB) from 2014-17.

His research focuses on robust planning and learning under uncertainty with an emphasis on multiagent systems, and he was the planning and control lead for the MIT DARPA Urban Challenge team. His work has been recognized with multiple awards, including receiving the IEEE Transactions on Robotics King-Sun Fu Memorial Best Paper Award for 2022, being inducted into the University of Toronto Engineering Hall of Distinction (2022), receiving the 2020 IEEE CSS Distinguished Member Award, the 2020 AIAA Intelligent Systems Award, the 2015 AeroLion Technologies Outstanding Paper Award for Unmanned Systems, the 2015 IEEE CSS Video Clip Contest, the 2011 IFAC Automatica award for best applications paper, and the 2002 Institute of Navigation Burka Award. He also received the Air Force Commander’s Public Service Award in 2017. He is a Fellow of IEEE (2018) and AIAA (2016) and was elected to the National Academy of Engineering in 2021.

The idea of reinforcement learning (RL) as a key principle of animal learning has been around at least since Edward Thorndike proposed the “Law of Effect” in 1898. Machine implementation of this principle began with electro-mechanical machines in the 1930s, and the earliest idea for computer implementation was probably Turing’s 1948 proposal of a computer implementation of a “pleasure-pain system”.  In this talk I review what has followed Turing’s unimplemented proposal, starting with the first computer experiments in 1954, up to what we now know as modern computational RL.

Bio: Andrew Barto is Professor Emeritus of Information and Computer Sciences, University of Massachusetts Amherst, having retired in 2012. He served as Chair of the UMass Amherst Department of Computer Science from 2007 to 2011. He received a BS with distinction in mathematics from the University of Michigan in 1970, and a PhD in Computer and Communication Sciences in 1975, also from the University of Michigan. He joined the UMass Computer Science Department in 1977 as a Postdoctoral Research Associate, became an Associate Professor in 1982, and a Full Professor in 1991. Before retiring he co-directed the UMass Autonomous Learning Laboratory, which focused on reinforcement learning and produced many notable machine learning researchers.. Professor Barto is a Fellow of the American Association for the Advancement of Science, a Fellow and Life Member of the IEEE, and has published over one hundred papers or chapters in journals, books, and conference and workshop proceedings. He is co-author of the book “Reinforcement Learning: An Introduction,” MIT Press, 1998, which has received over 25,000 citations. A much expanded second edition was published in 2018.

We address the problem of exploration versus exploitation that lies at the heart of reinforcement learning of dynamic systems. We describe the Biased Maximum Likelihood Method proposed to address this challenge. We present a comparative study of its regret performance in a variety of contexts ranging from Bandits to Markov Decision Processes to LQG systems. We also provide an account of regulation problems where there is no intrinsic conflict between exploration and exploitation, and present a historical account of results on stability, asymptotic behavior and robustness. [Joint work with Akshay Mete, Rahul Singh, Ping-Chun Hsieh, Yu-Heng Hung, Xi Liu, and Anirban Bhattacharya].

Bio: P. R. Kumar, B. Tech (1973, IIT Madras) and D.Sc. (1977, Washington Univ., St. Louis), was a faculty member in the Math Dept at University of Maryland, Baltimore County (1977-84), ECE and CSL at the University of Illinois, Urbana-Champaign (1985-2011), and has been at Texas A&M University since 2011. He has worked on problems in game theory, adaptive control, simulated annealing, machine learning, queueing networks, manufacturing systems, scheduling wafer fabrication plants, wireless networks and network information theory. His current research focus includes renewable energy, power systems, security, automated transportation, unmanned aerial vehicle traffic management, millimeter wave 5G, and cyber-physical systems. He is a member of the U.S. National Academy of Engineering, The World Academy of Sciences, and Indian National Academy of Engineering. He was awarded an honorary doctorate by ETH, Zurich. He received the Alexander Graham Bell Medal of IEEE, the IEEE Field Award for Control Systems, the Donald Eckman Award of the American Automatic Control Council, the Ellersick Prize of IEEE Communication Society, the Outstanding Contribution Award of ACM SIGMOBILE, the Infocom Achievement Award, the ACM SIGMOBILE Test-of-Time Paper Award, and COMSNETS Outstanding Contribution Award. He is a Fellow of IEEE, ACM and IFAC. He is an Honorary Professor at IIT Hyderabad.