This book presents the latest approximate dynamic programming (ADP) techniques for decision-making and control in engineered systems. ADP, inspired by learning mechanisms in biological and animal systems, is a reinforcement machine learning approach that bridges adaptive and optimal control methods. The book demonstrates how ADP can be used to develop adaptive optimal control algorithms that learn in real time, converging to optimal solutions by using data collected along system trajectories.
Traditionally, adaptive controllers and optimal controllers are treated as separate approaches: adaptive controllers learn online but may not achieve optimal performance, while optimal controllers require offline design with complete system knowledge. This book shows how ADP unifies these approaches, enabling real-time, adaptive, and optimal control. It also explains how ADP can be applied to online multi-player differential games, which are important for H-infinity robust control and for coordinating multiple agents in networked systems.
The book focuses on continuous-time systems, with models derived from physical principles using Hamiltonian or Lagrangian dynamics.
