I just want to reiterate how dynamic programming problems can be solved in Maple.

Especially dynamic programming models that frequently appears in economic models.

First of all it is important to note that is close to impossible to find an easy to understand

and step-by-step road maps to dynamic programming. Why is that ?!  The below Maple

code was basically "discovered" by trial and error and pure stubbornness (caveman 101).

It is interesting to note that when people have finally understand how the mechanics of

dynamic programming works they usually don't want to give it away. An alternative reason

might be that they don't understand it but they want to communicate that they do.

Students (yes I was a uni student for 10 years) are presented with a shit load of

highly complex and obscure mathematical equations that usually make no sense at all :-) 

No effort is put into the language of process and no value is allocated to mechanics.

The reasons for that are simple. It makes sure no useful information is communicated but it also

make sure that the students is subjugated to the mathematical complexity (no questions asked, trembling in fear). 

ok, lets close the psychotherapy session for today and move on to the problem at hand.

This is my preferred way of solving a constrained consumption dynamic programming problems in Maple.

Note that it involves an objective function, dynamic budget constraints,  the Bellman equation, value functions,

backward induction and discounting  :-)

restart:

ob := ln(c[0])+B^1*ln(c[1])+B^2*ln(c[2]):

a[3] := (1+r)*a[2]-c[2]:   a[2] := (1+r)*a[1]-c[1]:    a[1] := (1+r)*a[0]-c[0]:

V[3] := 0:

V[2] := ln(c[2])+B*V[3]:     c[2] := solve(a[3], c[2]);

V[1] := ln(c[1])+B*V[2]:     c[1] := solve(diff(V[1], c[1]), c[1]);

V[0] := ln(c[0])+B*V[1]:     c[0] := solve(diff(V[0], c[0]), c[0]);

a[0] := 1000:     r := .1:    B := 1/(1+r):

evalf([ob, 'c[0]' = c[0], 'c[1]' = c[1], 'c[2]' = c[2]], 6);

                         [16.4043, c[0] = 402.114, c[1] = 402.116, c[2] = 402.115]

 

In this case the optimal solution is the amount of consumption for each time period that maximize the sum of

discounted utility from consumption over time where a[t]=(1+r)*a[t-1]-c[t-1] is the budget constraint which says

that the amount of assets in this period must be equal to the interest we have earned on assets in the previous

period minus the consumption in the previous period. We can now confirm that the above solution indeed is optimal by

using lagrange optimization and the maximize procedure (which is essentially lagrange) in Maple as seen below.

restart:

ob := ln(c[0])+B*ln(c[1])+B^2*ln(c[2]):

con := (1+r)*((1+r)*((1+r)*a[0]-c[0])-c[1])-c[2]:

LL := ob+lambda*con:

x1 := diff(LL, c[0]) = 0:      x2 := diff(LL, c[1]) = 0:     x3 := diff(LL, c[2]) = 0:     x4 := diff(LL, lambda) = 0:

sol := evalf(solve({x1, x2, x3, x4}, {lambda, c[0], c[1], c[2]}), 4);

a[0] := 1000:     r := .1:     B := 1/(1+r):    

sol;

with(Optimization):

evalf(Maximize(ob, [con = 0]), 4);

     {lambda = 0.002055249579, c[0] = 402.1148036, c[1] = 402.1148036, c[2] = 402.1148036}
         
                                 [16.404, [c[0] = 402.12, c[1] = 402.12, c[2] = 402.11]]

 

Which is the same result we had previously. The optimal solution can also be derived by using

optimal control (generalization of calculus of variation) but I think I will leave that for another post.

Please follow the below link for a further discussion about optimal control and it economic applications.

http://www.maplesoft.com/applications/view.aspx?SID=6933