Equivariant Reinforcement Learning Frameworks for Quadrotor Low-Level Control
Written by
Beomyeol Yu
Table of Contents
Our research focuses on how differential geometry can be utilized across various topics in dynamical systems theory and control systems engineering, including computational mechanics, optimization, estimation, control, and machine learning, with primary applications in aerospace engineering and robotics.
We assert that by embracing the geometric structure of dynamical systems, we can construct more efficient, robust, and elegant control strategies for modern, complex systems.
Falling cat problem: When a cat falls, it appears to “rotate” in mid-air to reorient itself and land on its feet, even if it starts from rest with no initial angular momentum. How does the cat achieve this without violating the conservation of angular momentum, which states that the total angular momentum of a system remains constant in the absence of external torques?
Geometric Deep Learning
Geometric deep learning refers to a class of machine learning methods that generalize deep learning to data structured in non-Euclidean domains, such as graphs, manifolds, and other geometric spaces. These approaches leverage the underlying geometry of the problem to improve learning efficiency and generalization.
In particular, we have shown that symmetry and equivariance properties of dynamical systems can be exploited to significantly improve the data efficiency of deep reinforcement learning for unmanned aerial vehicles.
Recommended
Geometric Control
Control systems formulated in local coordinates of a configuration manifold often suffer from singularities, ambiguities, and increased complexity when representing aggressive or large-scale maneuvers.
By explicitly leveraging geometric properties of nonlinear manifolds in Lyapunov-based stability analysis, we construct intrinsic control laws that are globally defined and avoid coordinate singularities, while ensuring rigorous stability guarantees.
Uncertainty Propagation and Estimation
We develop intrinsic representations of probability distributions on manifolds, moving beyond Gaussian approximations in local coordinates that are commonly used in engineering practice.
In particular, we have constructed the matrix Fisher–Gaussian distribution on the product space of the special orthogonal group and Euclidean space, as well as approaches based on non-commutative harmonic analysis. These methods enable accurate uncertainty propagation and Bayesian estimation while respecting the underlying geometry of the state space.
Computational Geometric Optimal Control
This formalism enables efficient computation of optimal maneuvers for complex dynamical systems by leveraging principles from computational geometric mechanics.
We have shown that discretizing optimal control problems using geometric integrators significantly improves numerical efficiency and preserves key structural properties. Furthermore, we have developed accelerated optimization methods on manifolds by discretizing Bregman Lagrangian mechanics.
Computational Geometric Mechanics
We develop geometric numerical integrators on manifolds that preserve the intrinsic structure of dynamical systems, such as symplecticity, momentum, and energy behavior.
These structure-preserving properties are critical for ensuring long-term stability, accuracy, and physical fidelity in numerical simulations of complex systems.
Acknowledgments
Our research has been supported by AFOSR, AFRL, NASA, NAVAIR, NRL, NSF, and ONR.