Introduction to Modern Economic Growth 6.1 Brief Review of Dynamic Programming Using abstract but simple notation, the canonical dynamic optimization program in discrete time can be written as Problem A1 ∗ : V (x (0)) = sup {x(t+1)}∞ t=0 ∞ X β t U (x (t) , x (t + 1)) t=0 subject to x (t + 1) ∈ G(x (t)), for all t ≥ x (0) given where β ∈ (0, 1), and x (t) is a vector of variables, or more formally, x (t) ∈ X ⊂ RK for some K ≥ G(x) is a set-valued mapping, or a correspondence, also written as G:X ⇒X (see the Mathematical Appendix), thus the first constraint basically specifies what values of x (t + 1) are allowed given the value x (t) For this reason, we can think of x (t) as the state variable (state vector) of the problem, while x (t + 1) plays the role of the control variable (control vector) at time t Therefore, the constraint x (t + 1) ∈ G(x (t)) determines which control variables can be chosen given the state variable The real-valued function U : X × X → R is the instantaneous payoff function of this problem, and we have imposed that overall payoff (objective function) is a discounted sum of instantaneous payoffs In the problem formulation, we used “sup” rather than max, since there is no guarantee that the maximal value is attained by any feasible plan However, in all cases studied in this book the maximal value will be attained, so the reader may wish to substitute “max” for “sup” When the maximal value is attained by some ∞ sequence {x∗ (t + 1)}∞ t=0 ∈ X , we refer to this as a solution or as an optimal plan (where X ∞ is the infinite product of the set X, so that an element of X ∞ is a sequence with each member in X) Notice that this problem is stationary in the sense that the instantaneous payoff function U is not time-dependent; it only depends on x (t) and x (t + 1) A more general formulation would be to have U (x (t) , x (t + 1) , t), but for most economic 256