《Stochastic Optimal Control Problems with a Bounded Memory》由会员分享，可在线阅读，更多相关《Stochastic Optimal Control Problems with a Bounded Memory（16页珍藏版）》请在金锄头文库上搜索。
1、Stochastic Optimal Control Problems with a Bounded MemoryMou-Hsiung ChangTao PangMoustapha PemyJune 2, 2006AbstractThis paper treats a finite time horizon optimal control problem in which the controlled state dynamics is governed by a general systemof stochastic functional differential equations wit
2、h a bounded memory.An infinite-dimensional HJB equation is derived using a Bellman-type dynamic programming principle. It is shown that the value function is the unique viscosity solution of the HJB equation.In addition, the computation issues are also studied.More particularly, a finitedifference s
3、cheme is obtained to approximate the viscosity solution ofthe infinite dimensional HJB equation. The convergence of the schemeis proved using the Banach fixed point theorem. The computational algorithm is also provided based on the scheme obtained.Keywords: Stochastic control, stochastic functional
4、differential equations,viscosity solutions, finite difference approximation.AMS 2000 subject classifications: primary 93E20, 60H35; secondary 34K50, 49L25The research of this paper is partially supported by a grant W911NF-04-D-0003 fromthe U. S. Army Research Office Mathematics Division, U. S. Army
5、Research Office, P. O. Box 12211, RTP, NC 27709, USA, mouhsiung.changus.army.mil Department of Mathematics and Center for Research in Scientific Computation, North Carolina State University, Raleigh, NC 27695 USA, tpangunity.ncsu.edu. Corresponding author. Department of Mathematics and Center for Re
6、search in Scientific Computation, North Carolina State University/SAMSI, Raleigh, NC 27695 USA, mnpemyunity.ncsu.edu11IntroductionThe theory of stochastic functional differential equations has been widely used to describe the stochastic systems whose evolution depend on the past history of the state
7、. It has many applications in real world applications (see Mohammed 20, 21 and Kolmanovskii and Shaikhet 13 for basic theory and some applications). The linear-quadratic regulatory problem involvingstochastic delay equations was first studied in Kolmanovskii and Maizen- berg 12, and optimal control
8、problems for a class of nonlinear stochastic equations that involve a continuous delay of the following typedX(s)=(s,X(s),Y (s),u(s)ds+ (s,X(s),Y (s),u(s)dW(s),s t,T,(1)have been studied in recent literature (see e.g.Elsanousi 8, Elsanousi et al 9, and Larssen 16, Oksendal and Sulem 22), in which Y
9、(s) =R0reX(s + )d .In this paper, we will consider more general forms of the system of stochastic equations in RndX(s) = f(s,Xs,u(s)ds + g(s,Xs,u(s)dW(s),s t,T(2)in which W(s) is a standard m-dimensional Brownian motion. In addition, the drift f(s,Xs,u(s) and the diffusion coefficients g(s,Xs,u(s) d
10、epend explicitly on the window of the state process Xsover the time interval s r,s, where Xs: r,0 Rnis defined by Xs() = X(s + ), r,0. The consideration of such a system enable us to model many real worldproblems that have aftereffects (see 13). Apparently, equation (1) is only a special case of (2)
11、. In our recent work 4, we studied the system (2) using the viscosity solution concept introduced by Crandall and Lions 2, 18, 19 in order to characterize the value function as the unique viscosity solution of theassociated HJB equation. In 5, we considered the finite difference method to solve the
12、associated Hamilton-Jacobi-Bellman equation numerically. One thing we would like to point out is that the Markov Chain approximation method (see 15 for basic theory) can also be used to obtain the numerical solution for stochastic systems with delay (see Kusher 14). In addition, in 3, 6, we consider
13、ed optimal stopping time for stochastic systems with a bounded memory. In 7, we studied the application in Black-Scholes for-mula when the stock price is described with a stochastic delayed differential equations.2This paper is organized as follows. Notation and the statement of theproblem are conta
14、ined in Section 2. In Section 3, the infinite dimensional Hamilton-Jacobi-Bellman (HJB) equation for the value function is given. In Section 4, we consider the viscosity solution of the HJB equation. It is shown in Section 4 that the value function is a viscosity solution of the HJB equation. The un
15、iqueness result for viscosity solution of the HJB equationis also given there. In Section 5, we present a finite difference method to approximate the viscosity solution of the HJB equation. The convergence results are given. An computational algorithm is given in Section 6.2Problem FormulationLet T
16、0 denote a fixed terminal time, and let t 0,T denote an initialtime. We study the finite time horizon optimal control problem for a generalsystem of stochastic functional differential equations on the interval t,T.Let r 0 be a fixed constant, and let J = r,0 denote the duration of the bounded memory of the equations considered in this paper. For the sake of simplicity, denote C(J; 0 such that|f(t,
