Abstract
In this paper, we describe the partially observable Markov decision process (pomdp) approach to finding optimal or near-optimal control strategies for partially observable stochastic environments, given a complete model of the environment. The pomdp approach was originally developed in the operations research community and provides a formal basis for planning problems that have been of interest to the AI community. We found the existing algorithms for computing optimal control strategies to be highly computationally inefficient and have developed a new algorithm that is empirically more efficient. We sketch this algorithm and present preliminary results on several small problems that illustrate important properties of the pomdp approach. Introduction Agents that act in real environments, whether physical or virtual, rarely have complete information about the state of the environment in which they are working. It is necessary for them to choose their actions in partial ignorance and o...
Keywords
Partially observable Markov decision processObservableSketchComputer scienceMarkov decision processMathematical optimizationOptimal controlMarkov processProcess (computing)Control (management)Markov chainArtificial intelligenceAlgorithmMarkov modelMachine learningMathematics
Affiliated Institutions
Related Publications
Partially Observable<scp>MDPs</scp>(<scp>POMDPS</scp>): Introduction and Examples
Abstract A partially observable Markov decision process (POMDP) is a generalization of a Markov decision process where the states of the model are not completely observable by t...
The optimal control of partially observable Markov processes
The report studies the control of a finite-state, discrete-time Markov process characterized by incomplete state observation. The process is viewed through a set of outputs such...
Markov: A methodology for the solution of infinite time horizon markov decision processes
Abstract Algorithms are described for determining optimal policies for finite state, finite action, infinite discrete time horizon Markov decision processes. Both value‐improvem...
Scalable multi-objective optimization test problems
After adequately demonstrating the ability to solve different two-objective optimization problems, multi-objective evolutionary algorithms (MOEAs) must show their efficacy in ha...
Applying the genetic approach to simulated annealing in solving some NP-hard problems
A stochastic approach called the annealing-genetic algorithm is presented for solving some well-known combinatorial optimization problems. This approach incorporates genetic alg...
Publication Info
- Year
- 1994
- Type
- article
- Pages
- 1023-1028
- Citations
- 561
- Access
- Closed
External Links
Citation Metrics
561
OpenAlex
Cite This
Anthony R. Cassandra,
Leslie Pack Kaelbling,
Michael L. Littman
(1994).
Acting Optimally in Partially Observable Stochastic Domains.
, 1023-1028.