- Research Article
- Open access
- Published:
On Properties of Solutions for Two Functional Equations Arising in Dynamic Programming
Fixed Point Theory and Applications volume 2010, Article number: 905858 (2010)
Abstract
We introduce and study two new functional equations, which contain a lot of known functional equations as special cases, arising in dynamic programming of multistage decision processes. By applying a new fixed point theorem, we obtain the existence, uniqueness, iterative approximation, and error estimate of solutions for these functional equations. Under certain conditions, we also study properties of solutions for one of the functional equations. The results presented in this paper extend, improve, and unify the results according to Bellman, Bellman and Roosta, Bhakta and Choudhury, Bhakta and Mitra, Liu, Liu and Ume, and others. Two examples are given to demonstrate the advantage of our results over existing results in the literature.
1. Introduction and Preliminaries
The existence, uniqueness, and successive approximations of solutions for the following functional equations arising in dynamic programming:
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ1_HTML.gif)
were first introduced and discussed by Bellman [1, 2]. Afterwards, further analyses on the properties of solutions for the functional equations (1.1) and (1.2) and others have been studied by several authors in [3–7] and [8–11] by using various fixed point theorems and monotone iterative technique, where (1.2) are as follows:
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ2_HTML.gif)
The aim of this paper is to investigate properties of solutions for the following more general functional equations arising in dynamic programming of multistage decision processes:
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ3_HTML.gif)
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ4_HTML.gif)
where and
are real Banach spaces,
is the state space,
is the decision space, opt denotes the sup or inf,
and
stand for the state and decision vectors, respectively,
,
represent the transformations of the processes, and
denotes the optimal return function with initial state
. The rest of the paper is organized as follows. In Section 2, we state the definitions, notions, and a lemma and establish a new fixed point theorem, which will be used in the rest of the paper. The main results are presented in Section 3. By applying the new fixed point theorem, we establish the existence, uniqueness, iterative approximation, and error estimate of solutions for the functional equation (1.3) and (1.4). Under certain conditions, we also study other properties of solutions for the functional equations (1.4). The results present in this paper extend, improve, and unify the corresponding results according to Bellman [1], Bellman and Roosta [5], Bhakta and Choudhury [6], Bhakta and Mitra [7], Liu [8], Liu and Ume [11], and others. Two examples are given to demonstrate the advantage of our results over existing results in the literature.
Throughout this paper, we assume that ,
, and
. For any
,
denotes the largest integer not exceeding
. Define
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ5_HTML.gif)
2. A Fixed Point Theorem
Let be a countable family of pseudometrics on a nonvoid set
such that for any two different points
,
for some
. For any
, let
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ6_HTML.gif)
then is a metric on
. A sequence
in
is said to converge to a point
if
as
for any
and to be a Cauchy sequence if
as
for any
.
Theorem 2.1.
Let be a complete metric space, and let
be defined by (2.1). If
satisfies the following inequality:
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ7_HTML.gif)
where is some element in
, then
(i) has a unique fixed point
and
for any
,
(ii)if, in addition, , then
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ8_HTML.gif)
Proof.
Given and
, define
for each
. In view of (2.2), we know that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ9_HTML.gif)
Since , by (2.4) we easily conclude that
is nonincreasing. It follows that
has a limit
. We claim that
. Otherwise,
. On account of (2.4) and
, we deduce that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ10_HTML.gif)
which is impossible. That is, . We now show that
is a Cauchy sequence. Suppose that
is not a Cauchy sequence, then there exist
,
, and two sequences of positive integers
and
with
and
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ11_HTML.gif)
which yields that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ12_HTML.gif)
As in (2.7), we derive that
. Note that (2.2) and (2.7) mean that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ13_HTML.gif)
for any . Letting
in (2.8), we see that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ14_HTML.gif)
This is a contradiction. By completeness of , there exists a point
, such that
. Using (2.1), (2.2), and
, we obtain that for each
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ15_HTML.gif)
which yields that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ16_HTML.gif)
that is, is a fixed point of
. If
has a fixed point
different from
, then there exists
such that
. By (2.2), we have
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ17_HTML.gif)
which is a contradiction. Consequently, is a unique fixed point of
.
Suppose that . By (2.2), we get that for any
,
, and
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ18_HTML.gif)
This completes the proof.
Remark 2.2.
Theorem 2.1 extends Theorem 2.1 of Bhakta and Choudhury [6] and Theorem 1 of Boyd and Wong [12].
Lemma 2.3 (see [11]).
Let ,
,
, and
be in
, then
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ19_HTML.gif)
3. Properties of Solutions
In this section, we assume that and
are real Banach spaces,
is the state space, and
is the decision space. Define
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ20_HTML.gif)
For any positive integer and
, let
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ21_HTML.gif)
where and
, then
is a countable family of pseudometrics on
. It is clear that
is a complete metric space.
Theorem 3.1.
Let and
be mappings, and let
be in
, such that
(C1)for any and
,
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ22_HTML.gif)
(C2)for any and
, there exists
satisfying
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ23_HTML.gif)
then the functional equation (1.3) possesses a unique solution , and
converges to
for each
, where
is defined by
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ24_HTML.gif)
In addition, if is in
, then
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ25_HTML.gif)
Proof.
It follows from (C2) and (3.4) that maps
into itself. Given
,
,
, and
, suppose that
, then there exist
such that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ26_HTML.gif)
In view of (3.3), (3.5), and (3.7), we deduce that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ27_HTML.gif)
which implies that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ28_HTML.gif)
Similarly, we can show that (3.9) holds for . As
in (3.9), we get that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ29_HTML.gif)
Notice that the functional equation (1.3) possesses a unique solution if and only if the mapping
has a unique fixed point
. Thus, Theorem 3.1 follows from Theorem 2.1. This completes the proof.
Remark 3.2.
The conditions of Theorem 3.1 are weaker than the conditions of Theorem 3.1 of Bhakta and Choudhury [6].
Theorem 3.3.
Let and
be mappings for
. Assume that the following conditions are satisfied:
(C3)for each , there exists
such that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ30_HTML.gif)
(C4), for all
,
(C5)there exists a constant such that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ31_HTML.gif)
then the functional equation (1.4) possesses a unique solution , and
converges to
for each
, where
is defined by
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ32_HTML.gif)
Moreover,
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ33_HTML.gif)
Proof.
Set
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ34_HTML.gif)
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ35_HTML.gif)
It follows from (C3)–(C5) and (3.15) that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ36_HTML.gif)
for any and
. Consequently,
is a self mapping on
. By Lemma 2.3, (C4), and (C5), we obtain that for any
and
,
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ37_HTML.gif)
where for
. Thus, Theorem 3.3 follows from Theorem 3.1. This completes the proof.
Remark 3.4.
Theorem 2 of Bellman [1, page 121], the result of Bellman and Roosta [5, page 545], Theorem 3.3 of Bhakta and Choudhury [6], and Theorems  3.3 and  3.4 of Liu [8] are special cases of Theorem 3.3. The example below shows that Theorem 3.3 extends properly the results in [1, 5, 6, 8].
Example 3.5.
Let and
. Put
,
, and
for any
. It follows from Theorem 3.3 that the functional equation
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ38_HTML.gif)
possesses a unique solution . However, the results in [1, 5, 6, 8] are not applicable.
Theorem 3.6.
Let and
be mappings for
, and,
be in
satisfying
(C6), for all
,
(C7), for all
,
(C8),
then the functional equation (1.4) possesses a solution that satisfies the following conditions:
(C9)the sequence defined by
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ39_HTML.gif)
converges to ,
(C10) for any
and
,
(C11) is unique with respect to condition (C10).
Proof.
Let and
be defined by (3.15) and (3.16), respectively. We now claim that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ40_HTML.gif)
If not, then there exists some such that
. On account of
, we know that for any
,
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ41_HTML.gif)
whence
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ42_HTML.gif)
which is a contradiction since .
Next, we assert that the mapping is nonexpansive on
. Let
and
. It is easy to see that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ43_HTML.gif)
by (C7) and (3.21). Consequently, there exists a constant satisfying
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ44_HTML.gif)
In view of (C6), (3.16), and (3.25), we derive that for any ,
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ45_HTML.gif)
which yields that maps
into itself. Given
,
,
, and
, suppose that
, then there exist
such that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ46_HTML.gif)
Using (C6)–(C8), (3.15) and (3.27), and Lemma 2.3, we deduce that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ47_HTML.gif)
which means that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ48_HTML.gif)
Similarly, we can conclude that the above inequality holds for . Letting
, we get that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ49_HTML.gif)
which implies that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ50_HTML.gif)
That is, is nonexpansive.
We show that for each ,
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ51_HTML.gif)
In terms of (C6) and (C9), we obtain that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ52_HTML.gif)
which means that (3.32) holds for . Suppose that (3.32) holds for some
. It follows from (C6)–(C8) and (3.25) that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ53_HTML.gif)
Therefore, (3.32) holds for any .
Next, we prove that is a Cauchy sequence in
. Given
,
,
,
, and
, suppose that
. We select that
with
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ54_HTML.gif)
According to (C6)–(C8) and (3.35), we have
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ55_HTML.gif)
for some and
. In a similar way, we can conclude that (3.36) holds for
. Proceeding in this way, we select
and
for
such that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ56_HTML.gif)
In terms of (C7), (3.21), (3.32), (3.36), and (3.37), we know that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ57_HTML.gif)
which implies that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ58_HTML.gif)
Letting in the above inequality, we have
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ59_HTML.gif)
which means that is a Cauchy sequence in
because
for each
. Let
. By the nonexpansivity of
, we get that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ60_HTML.gif)
which implies that . That is,
is a solution of the functional equation (1.4).
Now, we show that (C10) holds. Given ,
,
, and
,
for
, set
. It is easy to verify that there exists a positive integer
satisfying
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ61_HTML.gif)
Notice that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ62_HTML.gif)
for any . Consequently, we infer immediately that, for
,
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ63_HTML.gif)
which yields that .
At last, we show that (C11) holds. Suppose that the functional equation (1.4) possesses another solution , which satisfies (C10). Given
and
, suppose that
, then there are
satisfying
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ64_HTML.gif)
Whence there exists and
such that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ65_HTML.gif)
by (C8). Proceeding in this way, we select and
for
satisfying
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ66_HTML.gif)
It follows that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ67_HTML.gif)
which yields that
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ68_HTML.gif)
by letting . Similarly, (3.49) also holds for
. As
, we know that
. This completes the proof.
Remark 3.7.
Theorem 3.6 generalizes Theorem 1 of Bellman [1, page 119], Theorem 3.5 of Bhakta and Choudhury [6], Theorem 2.4 of Bhakta and Mitra [7], Theorem 3.5 of Liu [8] and Theorem 3.1 of Liu and Ume [11]. The following example reveals that Theorem 3.6 is indeed a generalization of the results in [1, 6–8, 11].
Example 3.8.
Let ,
. Define
by
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ69_HTML.gif)
It is easy to verify that the following functional equation:
![](http://media.springernature.com/full/springer-static/image/art%3A10.1155%2F2010%2F905858/MediaObjects/13663_2010_Article_1361_Equ70_HTML.gif)
satisfies conditions (C6)–(C8). Consequently, Theorem 3.6 ensures that it has a solution that satisfies conditions (C9)–(C11). However, Theorem 1 of Bellman [1, page 119], Theorem 3.5 of Bhakta and Choudhury [6], Theorem 2.4 of Bhakta and Mitra [7], Theorem 3.5 of Liu [8], and Theorem 3.1 of Liu and Ume [11] are not applicable.
References
Bellman R: Dynamic Programming. Princeton University Press, Princeton, NJ, USA; 1957:xxv+342.
Bellman R: Methods of Nonlinear Analysis. Volume 2. Academic Press, New York, NY, USA; 1973:xvii+261.
Belbas SA: Dynamic programming and maximum principle for discrete Goursat systems. Journal of Mathematical Analysis and Applications 1991,161(1):57–77. 10.1016/0022-247X(91)90362-4
Bellman R, Lee ES: Functional equations in dynamic programming. Aequationes Mathematicae 1978,17(1):1–18. 10.1007/BF01818535
Bellman R, Roosta M: A technique for the reduction of dimensionality in dynamic programming. Journal of Mathematical Analysis and Applications 1982,88(2):543–546. 10.1016/0022-247X(82)90212-8
Bhakta PC, Choudhury SR: Some existence theorems for functional equations arising in dynamic programming. II. Journal of Mathematical Analysis and Applications 1988,131(1):217–231. 10.1016/0022-247X(88)90201-6
Bhakta PC, Mitra S: Some existence theorems for functional equations arising in dynamic programming. Journal of Mathematical Analysis and Applications 1984,98(2):348–362. 10.1016/0022-247X(84)90254-3
Liu Z: Existence theorems of solutions for certain classes of functional equations arising in dynamic programming. Journal of Mathematical Analysis and Applications 2001,262(2):529–553. 10.1006/jmaa.2001.7551
Liu Z: Coincidence theorems for expansion mappings with applications to the solutions of functional equations arising in dynamic programming. Acta Scientiarum Mathematicarum 1999,65(1–2):359–369.
Liu Z: Compatible mappings and fixed points. Acta Scientiarum Mathematicarum 1999,65(1–2):371–383.
Liu Z, Ume JS: On properties of solutions for a class of functional equations arising in dynamic programming. Journal of Optimization Theory and Applications 2003,117(3):533–551. 10.1023/A:1023945621360
Boyd DW, Wong JSW: On nonlinear contractions. Proceedings of the American Mathematical Society 1969, 20: 458–464. 10.1090/S0002-9939-1969-0239559-9
Acknowledgment
This research is financially supported by Changwon National University in 2009-2010.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
About this article
Cite this article
Liu, Z., Ume, J. & Kang, S. On Properties of Solutions for Two Functional Equations Arising in Dynamic Programming. Fixed Point Theory Appl 2010, 905858 (2010). https://doi.org/10.1155/2010/905858
Received:
Accepted:
Published:
DOI: https://doi.org/10.1155/2010/905858