Abstract
This report presents a unified approach for the study of constrained Markov decision processes with a countable state space and unbounded costs. We consider a single controller having several objectives; it is desirable to design a controller that minimize one of cost objective, subject to inequality constraints on other cost objectives. The objectives that we study are both the expected average cost, as well as the expected total cost (of which the discounted cost is a special case). We provide two frameworks: the case were costs are bounded below, as well as the contracting framework. We characterize the set of achievable expected occupation measures as well as performance vectors. This allows us to reduce the original control dynamic problem into an infinite Linear Programming. We present a Lagrangian approach that enables us to obtain sensitivity analysis. In particular, we obtain asymptotical results for the constrained control problem: convergence of both the value and the pol...
Keywords
Mathematical optimizationMarkov decision processDiscountingCountable setState spaceBounded functionDynamic programmingMarkov processAverage costBellman equationComputer scienceTime horizonMathematicsOptimal controlController (irrigation)
Related Publications
A Probabilistic Production and Inventory Problem
R. Howard (R. Howard.1960. Dynamic Programming and Markov Processes. John Wiley and Sons, Inc., New York.) and A. Manne (A. Manne. 1960. Linear programming and sequential decisi...
Markov Decision Processes: Discrete Stochastic Dynamic Programming.
From the Publisher: The past decade has seen considerable theoretical and applied research on Markov decision processes, as well as the growing use of these models in ecology, ...
Contraction Mappings in the Theory Underlying Dynamic Programming
Next article Contraction Mappings in the Theory Underlying Dynamic ProgrammingEric V. DenardoEric V. Denardohttps://doi.org/10.1137/1009030PDFBibTexSections ToolsAdd to favorite...
Plant-wide predictive control for a thermal power plant based on a physical plant model
A constrained non-linear, physical model-based, predictive control (NPMPC) strategy is developed for improved plant-wide control of a thermal power plant. The strategy makes use...
The optimal control of partially observable Markov processes
The report studies the control of a finite-state, discrete-time Markov process characterized by incomplete state observation. The process is viewed through a set of outputs such...
Publication Info
- Year
- 2021
- Type
- book
- Citations
- 1412
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
1412
OpenAlex
Cite This
Eitan Altman
(2021).
Constrained Markov Decision Processes.
.
https://doi.org/10.1201/9781315140223
Identifiers
- DOI
- 10.1201/9781315140223