This book may be regarded as consisting of two parts. In Chapters I-IV we pre sent what we regard as essential topics in an introduction to deterministic optimal control theory. This material has been used by the authors for one semester graduate-level courses at Brown University and the University of Kentucky. The simplest problem in calculus of variations is taken as the point of departure, in Chapter I. Chapters II, III, and IV deal with necessary conditions for an opti mum, existence and regularity theorems for optimal controls, and the method of dynamic programming. The beginning reader may find it useful first to learn the main results, corollaries, and examples. These tend to be found in the earlier parts of each chapter. We have deliberately postponed some difficult technical proofs to later parts of these chapters. In the second part of the book we give an introduction to stochastic optimal control for Markov diffusion processes. Our treatment follows the dynamic pro gramming method, and depends on the intimate relationship between second order partial differential equations of parabolic type and stochastic differential equations. This relationship is reviewed in Chapter V, which may be read inde pendently of Chapters I-IV. Chapter VI is based to a considerable extent on the authors' work in stochastic control since 1961. It also includes two other topics important for applications, namely, the solution to the stochastic linear regulator and the separation principle.
This monograph deals with various classes of deterministic and stochastic continuous time optimal control problems that are defined over unbounded time intervals. For these problems the performance criterion is described by an improper integral and it is possible that, when evaluated at a given admissible element, this criterion is unbounded. To cope with this divergence new optimality concepts, referred to here as overtaking optimality, weakly overtaking optimality, agreeable plans, etc. , have been proposed. The motivation for studying these problems arises primarily from the economic and biological sciences where models of this type arise naturally. Indeed, any bound placed on the time hori zon is artificial when one considers the evolution of the state of an economy or species. The responsibility for the introduction of this interesting class of problems rests with the economists who first studied them in the modeling of capital accumulation processes. Perhaps the earliest of these was F. Ramsey  who, in his seminal work on the theory of saving in 1928, considered a dynamic optimization model defined on an infinite time horizon. Briefly, this problem can be described as a Lagrange problem with unbounded time interval. The advent of modern control theory, particularly the formulation of the famous Maximum Principle of Pontryagin, has had a considerable impact on the treat ment of these models as well as optimization theory in general.
This edited volume comprises invited contributions from world-renowned researchers in the subject of stochastic control and inverse problems. There are several contributions on stochastic optimal control and stochastic inverse problems covering different aspects of the theory, numerical methods, and applications.
"This volume is a textbook on linear control systems with an emphasis on stochastic optimal control with solution methods using spectral factorization in line with the original approach of N. Wiener. Continuous-time and discrete-time versions are presented in parallel.... Two appendices introduce functional analytic concepts and probability theory, and there are 77 references and an index. The chapters (except for the last two) end with problems.... [T]he book presents in a clear way important concepts of control theory and can be used for teaching." —Zentralblatt Math "This is a textbook intended for use in courses on linear control and filtering and estimation on (advanced) levels. Its major purpose is an introduction to both deterministic and stochastic control and estimation. Topics are treated in both continuous time and discrete time versions.... Each chapter involves problems and exercises, and the book is supplemented by appendices, where fundamentals on Hilbert and Banach spaces, operator theory, and measure theoretic probability may be found. The book will be very useful for students, but also for a variety of specialists interested in deterministic and stochastic control and filtering." —Applications of Mathematics "The strength of the book under review lies in the choice of specialized topics it contains, which may not be found in this form elsewhere. Also, the first half would make a good standard course in linear control." —Journal of the Indian Institute of Science
Stochastic and Deterministic Problems (Pure and Applied Mathematics: A Series of Monographs and Textbooks/221)
Author: Gennadii E. Kolosov
Publisher: CRC Press
"Covers design methods for optimal (or quasioptimal) control algorithms in the form of synthesis for deterministic and stochastic dynamical systems-with applications in aerospace, robotic, and servomechanical technologies. Providing new results on exact and approximate solutions of optimal control problems."
"The first part of this book presents the essential topics for an introduction to deterministic optimal control theory. The second part introduces stochastic optimal control for Markov diffusion processes. It also includes two other topics important for applications, namely, the solution to the stochastic linear regulator and the separation principle"--Publisher description.
As is well known, Pontryagin's maximum principle and Bellman's dynamic programming are the two principal and most commonly used approaches in solving stochastic optimal control problems. * An interesting phenomenon one can observe from the literature is that these two approaches have been developed separately and independently. Since both methods are used to investigate the same problems, a natural question one will ask is the fol lowing: (Q) What is the relationship betwccn the maximum principlc and dy namic programming in stochastic optimal controls? There did exist some researches (prior to the 1980s) on the relationship between these two. Nevertheless, the results usually werestated in heuristic terms and proved under rather restrictive assumptions, which were not satisfied in most cases. In the statement of a Pontryagin-type maximum principle there is an adjoint equation, which is an ordinary differential equation (ODE) in the (finite-dimensional) deterministic case and a stochastic differential equation (SDE) in the stochastic case. The system consisting of the adjoint equa tion, the original state equation, and the maximum condition is referred to as an (extended) Hamiltonian system. On the other hand, in Bellman's dynamic programming, there is a partial differential equation (PDE), of first order in the (finite-dimensional) deterministic case and of second or der in the stochastic case. This is known as a Hamilton-Jacobi-Bellman (HJB) equation.
Perturbation methods provide a powerful technique for treating many problems of applied mathematics. These problems occur very frequently in solid and fluid mechanics, in physics, in engineering and also in economics. The purpose of this book is to describe, analyse and to some extent generalise the principal results concerning perturbation methods in optimal control for systems governed by deterministic or stochastic differential equations. The author aims to present a unified account of the available results. The first two chapters cover the main results in deterministic and stochastic optimal control theory, and in ergodic control theory. The remaining chapters deal with the applications of perturbation methods in deterministic and stochastic optimal control. Regular and singular perturbations are treated separately. Two broad categories of methods are used: the theory of necessary conditions, leading to Pontryagin's maximum principle, and the theory of sufficient conditions, leading to dynamic programming. The book will be of great interest to researchers and practitioners working in applied mathematics, solid and fluid mechanics, engineering, physics and economics.
This book proposes, for the first time, a basic formulation for structural control that takes into account the stochastic dynamics induced by engineering excitations in the nature of non-stationary and non-Gaussian processes. Further, it establishes the theory of and methods for stochastic optimal control of randomly-excited engineering structures in the context of probability density evolution methods, such as physically-based stochastic optimal (PSO) control. By logically integrating randomness into control gain, the book helps readers design elegant control systems, mitigate risks in civil engineering structures, and avoid the dilemmas posed by the methods predominantly applied in current practice, such as deterministic control and classical linear quadratic Gaussian (LQG) control associated with nominal white noises.
Linear Systems Control provides a very readable graduate text giving a good foundation for reading more rigorous texts. There are multiple examples, problems and solutions. This unique book successfully combines stochastic and deterministic methods.
Control theory provides a large set of theoretical and computational tools with applications in a wide range of ?elds, running from ”pure” branches of mathematics, like geometry, to more applied areas where the objective is to ?nd solutions to ”real life” problems, as is the case in robotics, control of industrial processes or ?nance. The ”high tech” character of modern business has increased the need for advanced methods. These rely heavily on mathematical techniques and seem indispensable for competitiveness of modern enterprises. It became essential for the ?nancial analyst to possess a high level of mathematical skills. C- versely, the complex challenges posed by the problems and models relevant to ?nance have, for a long time, been an important source of new research topics for mathematicians. The use of techniques from stochastic optimal control constitutes a well established and important branch of mathematical ?nance. Up to now, other branches of control theory have found comparatively less application in ?n- cial problems. To some extent, deterministic and stochastic control theories developed as di?erent branches of mathematics. However, there are many points of contact between them and in recent years the exchange of ideas between these ?elds has intensi?ed. Some concepts from stochastic calculus (e.g., rough paths) havedrawntheattentionofthedeterministiccontroltheorycommunity.Also, some ideas and tools usual in deterministic control (e.g., geometric, algebraic or functional-analytic methods) can be successfully applied to stochastic c- trol.
Optimal control of a class of single-input, discrete, stochastic bilinear systems is discussed. The control is assumed to be unbounded and the cost functional quadratic in state. A closed-form solution has been obtained for the stochastic control problem with perfect state observation, and with additive and multiplicative noise in the state equation. It is demonstrated that the presence of noise considerably simplifies the analysis compared to the deterministic case by virtue of integration over certain sets of measure zero. When the state equation has additive noise and the observation equation is noisy, a perturbation controller is obtained to minimize the instantaneous mean-square departure from the nominal, which is chosen to be the solution to the deterministic optimal control problem.
This book presents cutting-edge contributions in the areas of control theory and partial differential equations. Over the decades, control theory has had deep and fruitful interactions with the theory of partial differential equations (PDEs). Well-known examples are the study of the generalized solutions of Hamilton-Jacobi-Bellman equations arising in deterministic and stochastic optimal control and the development of modern analytical tools to study the controllability of infinite dimensional systems governed by PDEs. In the present volume, leading experts provide an up-to-date overview of the connections between these two vast fields of mathematics. Topics addressed include regularity of the value function associated to finite dimensional control systems, controllability and observability for PDEs, and asymptotic analysis of multiagent systems. The book will be of interest for both researchers and graduate students working in these areas.