Control and Analysis of Stochastic Systems
How does uncertainty affect a robot when attempting to generate a control policy to achieve some objective? How sensitive is the obtained control policy to changes in the description of the system? These are the central questions we wish to address. For most practical robotic systems, the state of the system is observed only indirectly through various sensors Since the actual state of the robot is not fully observable, the partially observable information is all that is available to the robot to infer its state and to use to make decisions. Further complicating matters, the system may be subject to disturbances that not only perturb the evolution of the system but also the observations. In such cases determining policies to effectively and efficiently govern the behavior of the system relative to a stated objective becomes computationally burdensome and impractical for the exact case. Thus, much research has been devoted to determining approximately optimal solutions for these partially observed Markov decision processes (POMDPs). We have developed a technique that is versatile: the majority of the computational effort is performed offline and the cost function and/or the initial hyperbelief can be changed with a minimal additional computational cost for any given system. In the future, we will use this framework to explore the sensitivity of the optimal policies to variation in the parameters describing the system. Ideally, this will endow us with an understanding of interplay between process model, measurement model, and the objective being optimized and the uncertainty therein as well as provide us with a means to derive a compact representation for the set of optimal policies over the space of of system parameters.