Natural gas supply management and contract allocation and valuation for a power generation company

NATURAL GAS SUPPLY MANAGEMENT AND CONTRACT ALLOCATION AND VALUATION FOR A POWER GENERATION COMPANY WENG RENRONG NATIONAL UNIVERSITY OF SINGAPORE 2015 NATURAL GAS SUPPLY MANAGEMENT AND CONTRACT ALLOCATION AND VALUATION FOR A POWER GENERATION COMPANY WENG RENRONG (B.Eng., Shanghai Jiao Tong University) A THESIS SUBMITTED FOR THE DEGREE OF DOCTOR OF PHILOSOPHY DEPARTMENT OF INDUSTRIAL AND SYSTEMS ENGINEERING NATIONAL UNIVERSITY OF SINGAPORE 2015 Acknowledgements First and foremost, I would like to express my deepest gratitude to my supervisor Dr. Kim Sujin for her aspiring guidance, constructive criticism and invaluable advice throughout my Ph.D career. Without her persistent help and encouragement, there would be no way to accomplish this dissertation. In addition, I am sincerely grateful to my committee members, Prof. Ng Kien Ming and Dr. Tan Chin Hon for their illuminating views and brilliant comments on this thesis. Furthermore, I would also like to express my great appreciation to all my friends for their companion and continuous support. Finally, a special thanks to my family. Their endless love and encouragement drive me to face challenges and strive towards my goal. iv Contents Declaration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii Introduction 1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.1 Natural Gas Prices . . . . . . . . . . . . . . . . . . . . . . 1.1.2 Natural Gas Contracts . . . . . . . . . . . . . . . . . . . . 1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Natural Gas Supply Portfolio and Contract Allocation and Valuation: A Brief Review . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.1 Natural Gas Supply Portfolio . . . . . . . . . . . . . . . . 1.3.2 Single-Time Scale Contract Allocation and Valuation . . . 1.3.3 Multi-Time Scale Contract Allocation and Valuation . . . . 11 1.4 Research Gaps and Objectives . . . . . . . . . . . . . . . . . . . . 12 1.5 Outline of Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Literature Review 16 2.1 Stochastic Dynamic Programming . . . . . . . . . . . . . . . . . . 16 2.2 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.2.1 Backward Dynamic Programming . . . . . . . . . . . . . . 19 v 2.2.2 Approximate Dynamic Programming . . . . . . . . . . . . 21 2.2.3 Least-Squares Monte Carlo . . . . . . . . . . . . . . . . . 25 2.2.4 Algorithmic Strategy Comparison . . . . . . . . . . . . . . 27 2.3 Adaptive Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.3.1 Base Stock Policy . . . . . . . . . . . . . . . . . . . . . . . 28 2.3.2 Bang-Bang Policy . . . . . . . . . . . . . . . . . . . . . . . 30 2.4 Multi-Time Scale Markov Decision Process . . . . . . . . . . . . . 31 2.4.1 Decision Dependent Uncertainty . . . . . . . . . . . . . . 33 Short-Term Natural Gas Supply Management 34 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 3.2 Problem Description and Model Formulation . . . . . . . . . . . . 37 3.3 Optimal Base Stock Policy and Monotonicity . . . . . . . . . . . . 40 3.3.1 Price Monotonicity . . . . . . . . . . . . . . . . . . . . . . 44 3.4 Price Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 3.4.1 Mean Reverting Model . . . . . . . . . . . . . . . . . . . . 48 3.4.2 Trinomial Tree Construction . . . . . . . . . . . . . . . . . 49 3.4.3 Parameter Calibration and Monte Carlo Simulation . . . . 52 3.5 Numerical Study . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.5.1 Experimental Settings . . . . . . . . . . . . . . . . . . . . 54 3.5.2 Value of Stochastic Solution . . . . . . . . . . . . . . . . . 54 3.6 Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . . 57 Contract Negotiation and Price Determination 58 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 4.2 Contract Valuation . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4.2.1 Contract Valuation for the GENCO . . . . . . . . . . . . . 61 4.2.2 Contract Valuation for the Gas Supplier . . . . . . . . . . 62 4.3 Contract Price Determination . . . . . . . . . . . . . . . . . . . . 65 4.3.1 Relationship Between Jm (Q) and Js (Q) . . . . . . . . . . . 65 vi 4.3.2 Nash Equilibrium . . . . . . . . . . . . . . . . . . . . . . . 4.4 Numerical Experiments 68 . . . . . . . . . . . . . . . . . . . . . . . 70 4.4.1 Optimal Contract Price and Price Indexes . . . . . . . . . 71 4.4.2 Contract Price Determination . . . . . . . . . . . . . . . . 73 4.5 Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . . 76 Multi-time Scale Markov Decision Process for Natural Gas Contract Allocation and Valuation 77 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 5.2 Problem Description and Model Formulation . . . . . . . . . . . . 81 5.2.1 Upper Time Level Markov Decision Process . . . . . . . . 82 5.2.2 Lower Time Level Markov Decision Process . . . . . . . . 85 5.3 Threshold Policy for Lower Time Level MDP . . . . . . . . . . . . 87 5.4 Least-Squares Policy Iteration Algorithm for Upper Time Level MDP 92 5.4.1 Policy Evaluation . . . . . . . . . . . . . . . . . . . . . . . 94 5.4.2 Policy Improvement . . . . . . . . . . . . . . . . . . . . . 96 5.4.3 Basis Function for Value Function Approximation . . . . . 99 5.4.4 Finite Difference Stochastic Approximation (FDSA) Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 5.5 Convergence and Error Bound . . . . . . . . . . . . . . . . . . . . 102 5.6 Numerical Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 108 5.6.1 Experiment Setup . . . . . . . . . . . . . . . . . . . . . . 108 5.6.2 Value of Make Up Clause . . . . . . . . . . . . . . . . . . . 110 5.6.3 Performance of LSPI Algorithm . . . . . . . . . . . . . . . 117 5.7 Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Conclusions and Future Research 125 Bibliography 129 vii A Proofs in Chapter 137 A.1 Proof of Lemma 3.5 . . . . . . . . . . . . . . . . . . . . . . . . . . 137 A.2 Proof of Lemma 3.6 . . . . . . . . . . . . . . . . . . . . . . . . . . 138 A.3 Proof of Lemma 3.7 . . . . . . . . . . . . . . . . . . . . . . . . . . 139 B Proofs in Chapter 143 B.1 Proof of Lemma 5.4 . . . . . . . . . . . . . . . . . . . . . . . . . . 143 B.2 Proof of Proposition 5.7 . . . . . . . . . . . . . . . . . . . . . . . 144 B.3 Proof of Proposition 5.8 . . . . . . . . . . . . . . . . . . . . . . . 146 viii Summary In Singapore, about 80 % of electricity is generated from natural gas. Natural gas supply for a power generation company (GENCO) is usually regulated through the use of provision contracts in addition to market transactions. With increasingly volatile gas prices in the deregulated environment, the profitability of a GENCO heavily relies on its ability to manage the portfolio of natural gas contract and spot trading. This thesis mainly concerns about the management of natural gas supply and contract gas allocation and valuation taking into account the volatile gas prices and various contractual flexibilities. In this thesis, we first study the optimization problem of dynamically allocating contracted gas over a short-term horizon. It is shown that a price and stage dependent base stock policy is optimal and the related optimal target levels monotonically decrease with the spot price. With a trinomial price scenario tree, these target levels can be easily computed to facilitate prompt contracted gas allocation. Numerical analyses demonstrate the importance of taking price volatility into consideration in the decision making process. Subsequently, we develop a novel scheme to price a bilateral gas contract. By incorporating the contract valuations for both the GENCO and the natural gas supplier, we unveil that there is always a possibility for both contracting parties to negotiate and reach a unique mutually acceptable equilibrium. The feasibility of the proposed pricing framework is validated by numerical results under various market conditions. Lastly, we consider a medium-term contract allocation problem with hierarchically structured sequential decision making induced by emerging make up clause. A multi-time scale Markov decision process (MMDP) model is proposed to address the interaction of decision makings in two different time scales of short-term and medium-term. We also contribute to developing a least-squares policy iteration (LSPI) algorithm in conjunction with a finite difference stochastic approximation (FDSA) method to solve the MMDP problem involving decision dependent uncertainty. Moreover, ix we rigorously establish the convergence guarantee and performance bound of the proposed algorithm. Extensive numerical experiments show that our LSPI algorithm outperforms the standard DP method, especially for a realistically sized problem. In summary, this thesis may provide valuable insights on dynamic energy contract allocation and valuation in the presence of spot trading both in shortterm and medium-term. x Laughton, D. G. and Jacoby, H. D. (1995). The effects of reversion on commodity projects of different length. Real options in capital investments: Models, strategies, and applications. Ed. by L. Trigeorgis. Westport: Praeger Publisher, pages 185–205. 48 Lee, S., Homem-de Mello, T., and Kleywegt, A. J. (2012). Newsvendor-type models with decision-dependent uncertainty. Mathematical Methods of Operations Research, 76(2):189–221. 33 Løland, A. and Lindqvist, O. (2008). Valuation of commodity-based swing options: a survey. Norsk, Regnesentral. 6, 11, 35, 78 Longstaff, F. A. and Schwartz, E. S. (2001). Valuing american options by simulation: A simple least-squares approach. Review of Financial Studies, 14(1):113– 147. 10, 25, 26 Ma, J. and Powell, W. B. (2008). Convergence proofs of least squares policy iteration algorithm for high-dimensional infinite horizon markov decision process problems. Technical report, working paper, Princeton University. 25 Ma, J. and Powell, W. B. (2011). Approximate policy iteration algorithms for continuous, multidimensional applications and convergence analysis. Princeton University. 25 Magee, L. (1998). Nonlocal behavior in polynomial regressions. The American Statistician, 52(1):20–22. 119 Meinshausen, N. and Hambly, B. M. (2004). Monte carlo methods for the valuation of multiple-exercise options. Mathematical Finance, 14(4):557–583. 10, 26 Morton, T. E. (1978). The nonstationary infinite horizon inventory problem. Management Science, 24(14):1474–1482. 29 Muppala, J. K., Malhotra, M., and Trivedi, K. S. (1996). Markov dependability models of complex systems: Analysis techniques. In Reliability and Maintenance of Complex Systems, pages 442–486. Springer. 32 Nadarajah, S., Margot, F., and Secomandi, N. (2013). Improved least squares monte carlo for term structure option valuation with energy applications. Technical report, Citeseer. 27, 99, 146 Nascimento, J. M. and Powell, W. B. (2009). An optimal approximate dynamic programming algorithm for the energy dispatch problem with grid-level storage. Preprint, Princeton University. 23 Nash, J. (1950). The bargaining problem. Econometrica: Journal of the Econometric Society, pages 155–162. 68 Nash, J. (1951). Non-cooperative games. The Annals of Mathematics, 54(2):286– 295. 68 134 Nash, J. (1953). Two-person cooperative games. Econometrica: Journal of the Econometric Society, pages 128–140. 68 Nedi´c, A. and Bertsekas, D. P. (2003). Least squares policy evaluation algorithms with linear function approximation. Discrete Event Dynamic Systems, 13(12):79–110. 25 O’Neill, R. P., Williard, M., Wilkins, B., and Pike, R. (1979). A mathematical programming model for allocation of natural gas. Operations Research, 27(5):857– 873. Palamarchuk, S. (2012). Compromise scheduling of bilateral contracts in electricity market environment. In Handbook of Networks in Power Systems I, pages 241–262. Springer. 59 Pflug, G. C. (1990). On-line optimization of simulated markovian processes. Mathematics of Operations Research, 15(3):381–395. 33 Popescu, D. G. and Seshadri, S. (2013). Demand uncertainty and excess supply in commodity contracting. Management Science, 59(9):2135–2152. 36, 60 Powell, W. B. (2007). Approximate Dynamic Programming: Solving the curses of dimensionality, volume 703. John Wiley & Sons. 18, 19, 20, 21, 22, 23, 24 Powell, W. B. (2009). What you should know about approximate dynamic programming. Naval Research Logistics (NRL), 56(3):239–249. 20 Puterman, M. L. (2009). Markov decision processes: discrete stochastic dynamic programming, volume 414. John Wiley & Sons. 19 Resnick, S. (1999). A probability path. Birkh¨ auser. 141 Roth, A. (1979). Axiomatic models of bargaining. Technical report, David K. Levine. 68 Schwartz, E. (1997). The stochastic behavior of commodity prices: Implications for valuation and hedging. Journal of Finance, 52:923–974. 38 Secomandi, N. (2010). Optimal commodity trading with a capacitated storage asset. Management Science, 56(3):449–467. xi, 9, 10, 30, 35, 36, 48, 91 Sen, S., Yu, L., and Genc, T. (2006). A stochastic programming approach to power portfolio optimization. Operations Research, 54(1):55–72. 5, 31 Sethi, S. P. and Zhang, Q. (1994). Hierarchical decision making in stochastic manufacturing systems. Springer. 32 Shapiro, A. and Philpott, A. (2007). A tutorial on stochastic programming. Manuscript. Available at www2.isye.gatech.edu/˜ ashapiro/publications.html. 17 Spall, J. C. (2005). Introduction to stochastic search and optimization: estimation, simulation, and control, volume 65. John Wiley & Sons. 101, 102 135 Stentoft, L. (2004). Convergence of the least squares monte carlo approach to american option valuation. Management Science, 50(9):1193–1203. 26 Sutton, R. S. (1988). Learning to predict by the methods of temporal differences. Machine Learning, 3(1):9–44. 96 Tan, R. B., Wijaya, D., and Khoo, H. H. (2010). Lci (life cycle inventory) analysis of fuels and electricity generation in singapore. Energy, 35(12):4910–4916. Taylor, J. (1997). An introduction to error analysis: the study of uncertainties in physical measurements. Univ Science Books. 52 Thanawalla, R. K. (2006). Valuation of gas swing options using an extended least squares Monte Carlo algorithm. PhD thesis, Heriot-Watt University,. 10 Thompson, A. C. (1995). Valuation of path-dependent contingent claims with multiple exercise decisions over time: The case of take-or-pay. Journal of Financial and Quantitative Analysis, 30(02):271–293. 5, Topkis, D. (1998). Supermodularity and complementarity. Princeton University Press, Princeton, NJ. 36, 44, 45, 47, 137, 138, 140, 141 Tsitsiklis, J. N. (2003). On the convergence of optimistic policy iteration. The Journal of Machine Learning Research, 3:59–72. 25, 106 Wu, D. and Kleindorfer, P. (2005). Competitive options, supply contracting, and electronic markets. Management Science, 51(3):452–466. 59 Xu, X., Hu, D., and Lu, X. (2007). Kernel-based least squares policy iteration for reinforcement learning. IEEE Transactions on Neural Networks, 18(4):973–992. 25 Yin, G. (1999). Rates of convergence for a class of global stochastic optimization algorithms. SIAM Journal on Optimization, 10(1):99–120. 102 136 Appendix A Proofs in Chapter A.1 Proof of Lemma 3.5 Proof: Consider period t ∈ T . Since both xt and at are defined in , it holds that X and At are latticed by definition. Hence, X ×At is also a lattice, following from the result that the direct product of lattices is a lattice (Topkis, 1998). Define Vt+1 (xt − at , pt , Dt ) as V t+1 (xt , at ) for given pt and Dt . Pick any feasible inventory and action pairs (xt , at ), (xt , at ) ∈ X × At . To verify the claimed submodularity of Ut (xt − at , pt ), it suffices to show the submodularity of V t+1 (xt , at ) in (xt , at ): V t+1 ((xt , at ) ∨ (xt , at )) + V t+1 ((xt , at )) ∧ (xt , at ) ≤ V t+1 (xt , at ) + V t+1 (xt , at ) (A.1) Without loss of generality, two cases need to be considered: When xt < xt and at < at , it is easy to derive that V t+1 ((xt , at ) ∨ (xt , at )) + V t+1 ((xt , at )) ∧ (xt , at ) − V t+1 (xt , at ) − V t+1 (xt , at ) = 0. For the other scenario that xt < xt and at < at , inequality (A.1) is now shown to be the case by Lemma 2.6.2 in Topkis (1998). Let yt = xt − at , yt = xt − at and δ = at − at . It is clear that δ > and yt < yt . V t+1 ((xt , at ) ∨ (xt , at )) + V t+1 ((xt , at ) ∧ (xt , at )) − V t+1 (xt , at ) − V t+1 (xt , at ) = Vt+1 (xt ∨ xt − at ∨ at , pt+1 , Dt+1 ) + Vt+1 (xt ∧ xt − at ∧ at , pt+1 , Dt+1 ) 137 −Vt+1 (xt − at , pt+1 , Dt+1 ) − Vt+1 (xt − at , pt+1 , Dt+1 ) Vt+1 (xt − at , pt+1 , Dt+1 ) + Vt+1 (xt − at , pt+1 , Dt+1 ) = −Vt+1 (xt − at , pt+1 , Dt+1 ) − Vt+1 (xt − at , pt+1 , Dt+1 ) Vt+1 (yt + δ, pt+1 , Dt+1 ) − Vt+1 (yt , pt+1 , Dt+1 ) = −[Vt+1 (yt + δ, pt+1 , Dt+1 ) − Vt+1 (yt , pt+1 , Dt+1 )] ≤ (A.2) The inequality holds because the convexity of Vt (xt , pt , Dt ) in xt implies that Vt (yt + δ, pt , Dt ) − Vt (yt , pt , Dt ) is increasing in yt for given pt , Dt and δ > 0. Therefore, inequality (A.1) holds for any (xt , at ) ∈ X × At . Corollary 2.6.2 in Topkis (1998) implies that Ut (xt − at , pt ) is submodular in (xt , at ) ∈ X × At , since expectation preserves submodularity. A.2 Proof of Lemma 3.6 Proof: The claimed piecewise linearity of Vt (xt , pt , Dt ) can be shown by mathematical induction. In stage T + 1, VT +1 (xT +1 , pT +1 , DT +1 ) is obviously piecewise linear in xT +1 . Make the induction hypothesis that the stated property also holds in stages t + 1, · · · , T . Now focus on stage t. The optimal base stock policy implies that the expression of Vt (xt , pt , Dt ) depends on the relationship between the remaining contract level xt and the base stock target BSt (pt ). The relationship between the revealed demand Dt and the withdraw capacity Cd further partitions the feasible action into two scenarios by whether the base stock target level BSt (pt ) is achievable from xt or not. Therefore two cases are necessary to be considered: Dt ≥ Cd and 138 Dt < Cd . In the former case of Dt ≥ Cd , it holds that Vt (xt , pt , Dt ) =     pt Dt + Ut (xt , pt ),     if xt ≤ BSt (pt ); pt (Dt − xt + BSt (pt )) + Ut (BSt (pt ), pt ), if BSt (pt ) < xt ≤ BSt (pt ) + Cd ;       pt (Dt − Cd ) + Ut (xt − Cd , pt ), if xt > BSt (pt ) + Cd ; (A.3) Consider the three cases in (A.3) separately. Case 1: xt ≤ BSt (pt ), Vt (xt , pt , Dt ) = pt Dt + Ut (xt , pt ). Piecewise linearity of Vt+1 (xt , pt+1 , Dt+1 ) in xt indicates that Ut (xt , pt ) is also piecewise linear in xt for given pt . Case 2: BSt (pt ) < xt ≤ BSt (pt ) + Cd . Vt (xt , pt , Dt ) = pt (Dt − xt + BSt (pt )) + Ut (BSt (pt ), pt )). Ut (BSt (pt ), pt )) is independent of xt , since the base stock target level BSt (pt ) is independent of xt . Also pt (Dt − xt + BSt (pt )) is trivially linear in xt . Case 3: xt > BSt (pt ) + Cd , Vt (xt , pt , Dt ) = pt (Dt − Cd ) + Ut (xt − Cd , pt ). Similar to Case 1, Ut (xt − Cd , pt ) is piecewise linear in xt for given pt . Accordingly, it is not difficult to verify that Vt (xt , pt , Dt ) is also piecewise linear in xt . On the other hand, when Dt < Cd , we can achieve the same result in a similar way. Therefore the claimed property holds for all periods by the principle of mathematical induction. A.3 Proof of Lemma 3.7 Proof: According to Lemma 3.6, Vt (xt , pt , Dt ) is piecewise linear in xt , hence the left and right derivatives of Vt (xt , pt , Dt ) in terms of xt exists for all xt ∈ X except for the boundary, where the left derivative is unavailable at xt = and the right derivative is unavailable at xt = Q. Without loss of generality, define Vt (xt , pt , Dt ) as the right derivative of Vt (xt , pt , Dt ) at xt ∈ [0, Q) and its left 139 derivative at xt = Q. Vt (xt , pt , Dt ) = lim Vt (xt + , pt , Dt ) − Vt (xt , pt , Dt ) →0 Vt (xt , pt , Dt ) = lim Vt (xt , pt , Dt ) − Vt (xt − , pt , Dt ) →0 , t ∈ T , xt ∈ [0, Q) (A.4) , t ∈ T , xt = Q (A.5) Ut (xt+1 , pt ) can be defined in a similar manner. To prove that Vt (xt , pt , Dt ) has decreasing differences in (xt , pt ) for given Dt , it suffices to show that [Vt (xt + , pt , Dt ) − Vt (xt , pt , Dt )] decreases in pt for any positive such that (xt + ) ∈ X (Topkis, 1998), which is equivalent to show that the right derivative Vt (xt , pt , Dt ) decreases in pt for given Dt when approaches to zero. This can be shown by mathematical induction. It is trivial that the claimed property holds in period T + 1. Make the hypothesis assumption that it also holds in periods t + 1, · · · , T . Now focus on period t. Similar to the proof in Lemma 3.6, two mutually exclusive cases Dt ≥ Cd and Dt < Cd need to be considered. In the former scenario Dt ≥ Cd , the corresponding expression of Vt (xt , pt , Dt ) is given by equation (A.3). Denote I(A) as an indicator function which equals to if the containing argument A is true and otherwise. Taking derivative with respect to xt in both sides of equation (A.3), we obtain that Vt (xt , pt , Dt ) = Ut (xt , pt )I(xt ≤ BSt (pt )) −pt I(BSt (pt ) < xt ≤ BSt (pt ) + Cd ) +Ut (xt − Cd , pt )I(xt > BSt (pt ) + Cd ) (A.6) To proceed, we need to show that Ut (xt , pt ) is decreasing in pt . Notice that the optimal value function Vt+1 (xt+1 , pt+1 , Dt+1 ) can be rewritten as the summation of the risk neutral value of the cash flows from period t + to the expiration of the planning horizon. Vt+1 (xt+1 , pt+1 , Dt+1 ) = (Dt+1 − at+1 )pt+1 + E at+1 ∈At+1 140 ˜ t+2 − at+2 )˜ (D pt+2 at+2 ∈At+2 ˜ T − aT )˜ +E · · · + E[ (D pT ] (A.7) aT ∈AT For any x1t+1 , x2t+1 ∈ Xt+1 , the difference between Vt+1 (x1t+1 , pt+1 , Dt+1 ) and Vt+1 (x2t+1 , pt+1 , Dt+1 ) is the benefit by allocating the additional |x1t+1 − x2t+1 | amount of contract gas in the remaining periods. It is not difficult to derive that T |Vt+1 (x1t+1 , pt+1 , Dt+1 ) − Vt+1 (x2t+1 , pt+1 , Dt+1 )| E(˜ pk |pt+1 )|x1t+1 − x2t+1 | ≤ k=t+1 (A.8) where T k=t+1 E(˜ pk |pt+1 ) < ∞ holds by Assumption 3.4. Hence Vt+1 (xt+1 , pt+1 , Dt+1 ) is Lipschitz continuous in xt+1 . In particular, pick any xt+1 in [0, Q) and any positive such that (xt+1 + ) ∈ [0, Q), and we have |Vt+1 (xt+1 + , pt+1 , Dt+1 ) − Vt+1 (xt+1 , pt+1 , Dt+1 )| T E(˜ pk |pt+1 ) < ∞ ≤ k=t+1 (A.9) The Dominated Convergence Theorem (Resnick, 1999) implies that the expectation and derivative are interchangeable in this case. Ut (xt+1 , pt ) = lim Ut (xt+1 + , pt ) − Ut (xt+1 , pt ) →0 = lim E[ ˜ t+1 ) − Vt+1 (xt+1 , p˜t+1 , D ˜ t+1 ) Vt+1 (xt+1 + , p˜t+1 , D →0 = E[lim ˜ t+1 ) − Vt+1 (xt+1 , p˜t+1 , D ˜ t+1 ) Vt+1 (xt+1 + , p˜t+1 , D →0 ˜ t+1 )|pt ] = E[Vt+1 (xt+1 , p˜t+1 , D |pt ] |pt ] (A.10) It is not difficult to verify that the equation (A.10) also holds at xt+1 = Q in terms of left derivative. The induction hypothesis that Vt+1 (xt+1 , pt+1 , Dt+1 ) decreases in pt+1 and Assumption 3.4 that the distribution F (˜ pt+1 |pt ) stochastically ˜ t+1 )|pt ] deincreases in pt together imply that Ut (xt+1 , pt ) = E[Vt+1 (xt+1 , p˜t+1 , D creases in pt , following from Corollary 3.9.1 in Topkis (1998). Similarly, we can show that Ut+1 (xt − Cd , pt ) also decreases in pt . It follows from equation (A.6) that Vt (xt , pt , Dt ) is decreasing in pt . On the other hand, the same result can 141 be achieved in the counterpart scenario Dt < Cd in a similar manner. Hence, the stated property also holds in stage t. Therefore the mathematical induction principle implies that it holds for all periods . 142 Appendix B Proofs in Chapter B.1 Proof of Lemma 5.4 Proof: We show the bounds by backward induction. At the terminal stage t = T + 1, it is evident that J¯Tπn+1 (sT +1 ) = JTn−1 +1 (sT +1 ), since no decision is involved at this stage. Make the hypothesis that the bound performance (5.43) also holds for stage t + 1, · · · , T . For stage t πn J¯tπn (st , ωt ) = E[Ct (st , at ) + γ J¯t+1 (F (st , ant , ωt ), ωt+1 )] T n−1 Jt+1 F (st , ant , ωt ), ωt+1 ) ≤ E Ct (st , at ) + γ γ τ −t−1 ςτn−1 +1 + τ =t+1 T n−1 γJt+1 (F (st , ant , ωt ), ωt+1 ) = E Ct (st , at ) + γ τ −t ςτn−1 +1 + τ =t+1 T = J¨tn (st , ωt ) + γ τ −t ςτn−1 +1 τ =t+1 T ≤ Jtn−1 (st , ωt ) + ςtn−1 γ τ −t ςτn−1 +1 + τ =t+1 T γ τ −t ςτn−1 = Jtn−1 (st , ωt ) + (B.1) τ =t where the first equality is based on the definition of J¯tπn (st , ωt ), while the inequality follows from hypothesis assumption (5.43). The third equality holds because n−1 ant = argminat E[Ct (st , at ) + γJt+1 (F (st , ant , ωt ), ωt+1 )], and the second inequality 143 follows from the definition of ςtn−1 in (5.42). The induction proof is completed by the principle of mathematical induction. B.2 Proof of Proposition 5.7 n+1 , ωt ). By the Proof: For the notational simplicity, we first define sn+1 t+1 = Ft (st , at definition of J¨tn+1 (st , ωt ), we have n+1 J¨tn+1 (st , ωt ) = Ct (st , at ) + γE[Jt+1 (Ft (st , at , ωt ), ωt+1 )] at n+1 n+1 ≤ Ct (st , an+1 ) + γE[Jt+1 (st+1 , ωt+1 )] t n = Ct (st , an+1 ) + γE[(1 − αn )Jt+1 (sn+1 t t+1 , ωt+1 ) πn+1 n+1 n (sn+1 +αn (J¯t+1 (st+1 , ωt+1 ) + wt+1 t+1 , ωt+1 ) − n n+1 i+1 (st+1 , ωt+1 ))] n+1 n (sn+1 ) ) + γE[Jt+1 = (1 − αn ){Ct (st , an+1 t t+1 , ωt+1 )]} + αn Ct (st , at π n+1 n n+1 +αn γE[J¯t+1 (sn+1 t+1 , ωt+1 ) + wt+1 (st+1 , ωt+1 ) − n n+1 t+1 (st+1 , ωt+1 )] π = (1 − αn )J¨tn (st , ωt ) + αn J¯t n+1 (st , ωt ) (B.2) π πn+1 n+1 The last equality follows from J¯t n+1 (st , ωt ) = E[Ct (st , an+1 ) + γ J¯t+1 (st+1 , ωt+1 )] t and Assumption 5.5 that approximation errors wtn and n t are white noise with zero mean. Combining (5.41) and (B.2) yields J¨tn+1 (st , ωt ) − Jtn+1 (st , ωt ) ≤ π (1 − αn )J¨tn (st , ωt ) + αn J¯t n+1 (st , ωt ) π − (1 − αn )Jtn (st , ωt ) + αn (J¯t n+1 (st , ωt ) + wtn+1 (st , ωt ) − = (1 − αn )[J¨tn (st , ωt ) − Jtn (st , ωt )] − αn [wtn+1 (st , ωt ) − n+1 (st , ωt )) t n+1 (st , ωt )] t (B.3) Let YTn (st , ωt ) = J¨tn (st , ωt ) − Rtn (st , ωt ). We establish that Ytn+1 (st , ωt ) ≤ (1 − αn )Ytn (st , ωt ) + αn [ 144 n+1 (st , ωt ) t − wtn+1 (st , ωt )] (B.4) n+1 Construct an auxiliary sequence Y t n+1 Yt (st , ωt ) of the form n (st , ωt ) = (1 − αn )Y t (st , ωt ) + αn ( n+1 (st , ωt ) t − wtn+1 (st , ωt )) (B.5) with Y t (st , ωt ) = Yt0 (st , ωt ) for all st ∈ St ×Ωt and t = 1, 2, · · · , T . It is not difficult n to check that Ytn (st , ωt ) ≤ Y t (st , ωt ) holds for all n. Rearranging equation (B.5) results in n+1 Yt n n (st , ωt ) = Y t (st , ωt ) − αn (Y t (st , ωt ) + wtn+1 (st , ωt ) − n+1 (st , ωt )) t (B.6) It resembles to the updating rule of a stochastic gradient algorithm for mini2 mizing f (Y t (st , ωt )) = Y t (st , ωt )/2. According to Assumption 5.5, it is evident that E[wtn+1 (st , ωt ) − E[wtn+1 (st , ωt )]2 + E[ n+1 (st , ωt )] t n+1 (st , ωt )]2 t = and E[wtn+1 (st , ωt ) − n+1 (st , ωt )]2 t = < ∞. Example 4.3 in Bertsekas and Tsitsiklis n (1996) implies that the process Y t (st , ωt ) converges to ∇f (Y t (st , ωt )) = 0, under the stepsize Assumption 5.6. In other words, n lim Y t (st , ωt ) = 0, ∀(st , ωt ) ∈ St × Ωt , t = 1, 2, · · · , T n→∞ (B.7) Therefore, lim Ytn (st , ωt ) ≤ for all (st , ωt ) and t = 1, 2, · · · , T . It is equivalent n→∞ to lim ςtn = lim n→∞ max n→∞ (st ,ωt )∈St ×Ωt Ytn (st , ωt ) ≤ (B.8) The convergence of ςtn to some scalar less than or equal to zero implies that there exists an iteration count nk such that ςtn ≤ 0, ∀n ≥ nk . From (5.45), we can conclude that E[Jtn+1 (st , ωt )|Jtn (st , ωt )] ≤ Jtn (st , ωt ) for a subsequence n = nk , nk+1 , · · · for all (st , ωt ) and t = 1, 2, · · · , T . That is, Jtn (st , ωt ) is a supermartingale for n = nk , nk+1 , · · · . As Jtn (st , ωt ) = (θtn )T Ψ(st , ωt ), it is clear that E[|Jtn (st , ωt )|] < ∞. It then follows from Doob’s martingale convergence theorem (Doob, 1953) that Jtn (st , ωt ) → Jt∞ (st , ωt ) almost surely for all (st , ωt ) and t = 1, 2, · · · , T . This completes the proof. 145 B.3 Proof of Proposition 5.8 Before proceeding, we first introduce a useful lemma that is required in the proof. Lemma B.1. Let g(z) and h(z) be any functions defined on a closed and bounded set Z. It holds that | g(z) − h(z)| ≤ max |g(z) − h(z)| z∈Z z∈Z (B.9) z∈Z Proof: To show the desired result, it suffices to show that min(−|g(z)−h(z)|) = − max |g(z)−h(z)| ≤ g(z)−min h(z) ≤ max |g(z)−h(z)| z∈Z z∈Z z∈Z z∈Z z∈Z (B.10) Denote z1 ∈ argmin g(z) and z2 ∈ argmin h(z) . Let z ∈ argmin(g(z) − h(z)) and z¯ ∈ argmax(g(z) − h(z)). It is clear that g(z) − h(z) ≤ g(z1 ) − h(z1 ). Adding h(z2 ) to both sides, we can obtain that h(z2 )+g(z)−h(z) ≤ h(z2 )+g(z1 )−h(z1 ) ≤ g(z1 ). That is, g(z) − h(z) ≥ min(g(z) − h(z)) ≥ min(−|g(z) − h(z)|) z∈Z z∈Z z∈Z z∈Z (B.11) On the other hand, we have g(¯ z ) − h(¯ z ) ≥ g(z2 ) − h(z2 ) ≥ g(z1 ) − h(z2 ). That is, g(z) − h(z) ≤ max(g(z) − h(z)) ≤ max |g(z) − h(z)| z∈Z z∈Z z∈Z z∈Z (B.12) Hence, it holds that | g(z) − h(z)| ≤ max |g(z) − h(z)|. z∈Z z∈Z Define an auxiliary variable t = z∈Z ||Jt∞ − J¨t∞ ||∞ = max |Jt∞ (st , ωt ) − J¨t∞ (st , ωt )| st ,ωt for the ease of ensuing analysis. The first part in the proof of Proposition 5.8 resembles to that of Lemma 6.1 in Nadarajah et al. (2013). Proof: For each stage t = 1, 2, · · · , T , we have et = ||Jt∞ − Jt∗ ||∞ ≤ ||Jt∞ − J¨t∞ ||∞ + ||J¨t∞ − Jt∗ ||∞ 146 = t + ||J¨t∞ − Jt∗ ||∞ (B.13) Now, we bound the second term ||J¨t∞ − Jt∗ ||∞ = max J¨t∞ (st , ωt ) − Jt∗ (st , ωt ) = ∗ ∞ (st+1 , ωt+1 )]} (st+1 , ωt+1 )]} − {min Ct (st , at ) + γE[Ji+1 max {min Ct (st , at ) + γE[Jt+1 ≤ ∞ ∗ (st+1 , ωt+1 )] max max γE[Jt+1 (st+1 , ωt+1 )] − γE[Jt+1 ≤ ∞ ∗ γ max max Jt+1 (st+1 , ωt+1 ) − Jt+1 (st+1 , ωt+1 ) ≤ γ = γ max = ∞ ∗ γ||Jt+1 − Jt+1 ||∞ = γet+1 st ,ωt st ,ωt at at st ,ωt at st ,ωt at ,ωt+1 max st ,at ,ωt ,ωt+1 st+1 ,ωt+1 ∞ ∗ Jt+1 (st+1 , ωt+1 ) − Ji+1 (st+1 , ωt+1 ) ∞ ∗ Jt+1 (st+1 , ωt+1 ) − Jt+1 (st+1 , ωt+1 ) (B.14) where the first inequality follows from lemma B.1 while the second inequality holds because we use maximum in place of expectation. The third inequality follows by | maxz∈Z g(z)| ≤ maxz∈Z |g(z)|. Therefore, we can obtain that et ≤ γet+1 + t, from which we can easily derive the error bound et by induction T γ τ −t et ≤ τ (B.15) τ =t We now channel the auxiliary variable t to sampling approximation error and regression error. The value function updating rule (5.41) can be rearranged as π Jtn+1 (st , ωt )−Jtn (st , ωt ) = −αn [Jtn (st , ωt )−J¯t n+1 (st , ωt )]+αn [wtn+1 (st , ωt )− n+1 (st , ωt )] t When n approaches to infinity, the left hand side Jtn+1 (st , ωt )−Jtn (st , ωt ) converges to zero. Therefore, we have 147 π lim max Jtn (st , ωt ) − J¯t n+1 (st , ωt ) n→∞ st ,ωt lim max wtn+1 (st , ωt ) − = n→∞ st ,ωt n+1 (st , ωt ) t lim max |wtn+1 (st , ωt )| + | ≤ n→∞ st ,ωt wt∞ + = n+1 (st , ωt )| t ∞ t (B.16) and π lim max E Jtn (st , ωt ) − J¯t n+1 (st , ωt ) n→∞ st ,ωt lim max E wtn+1 (st , ωt ) − = n→∞ st ,ωt = By the definition of t = = = = ≤ n+1 (st , ωt ) t (B.17) t, we have lim max Jtn (st , ωt ) − J¨tn (st , ωt ) n→∞ st ,ωt n lim max Jtn (st , ωt ) − Ct (st , at ) + γE[Jt+1 (Ft (st , at , ωt ), ωt+1 )] n→∞ st ,ωt at n lim max Jtn (st , ωt ) − Ct (st , an+1 ) − γE[Jt+1 (Ft (st , an+1 , ωt ), ωt+1 )] t t n→∞ st ,ωt π πn+1 n+1 n lim max Jtn (st , ωt ) − J¯t n+1 (st , ωt ) − γ J¯t+1 (st+1 , ωt+1 ) − γE[Jt+1 (sn+1 t+1 , ωt+1 )] n→∞ st ,ωt π n ¯πn+1 n+1 lim max Jtn (st , ωt ) − J¯t n+1 (st , ωt ) + γ E Jt+1 (sn+1 t+1 , ωt+1 ) − Jt+1 (st+1 , ωt+1 ) n→∞ st ,ωt ≤ wt∞ + ∞ t (B.18) where the third equality follows from (2.8) and the fourth equality holds because π πn+1 n+1 J¯t n+1 (st , ωt ) = Ct (st , an+1 ) + γ J¯t+1 (st+1 , ωt+1 ). Combining (B.15) and (B.18) t yields T γ τ −t (wτ∞ + et ≤ ∞ τ ) (B.19) τ =t In the case of approximating the immediate cost Ct (st , at ) with C¯t (st , at ), the second term of (B.13) becomes 148 ||J¨t∞ − Jt∗ ||∞ = max J¨t∞ (st , ωt ) − Jt∗ (st , ωt ) = ∞ ∗ max {min C¯t (st , at ) + γE[Jt+1 (st+1 , ωt+1 )]} − {min Ct (st , at ) + γE[Ji+1 (st+1 , ωt+1 )]} ≤ ∗ ∞ (st+1 , ωt+1 )] (st+1 , ωt+1 )] − γE[Jt+1 max max C¯t (st , at ) − Ct (st , at ) + γE[Jt+1 ≤ ∗ ∞ (st+1 , ωt+1 ) (st+1 , ωt+1 ) − Jt+1 max C¯t (st , at ) − Ct (st , at ) + γ max max Jt+1 ≤ γ = esub + γ max t = esub t st ,ωt st ,ωt st ,ωt at at at st ,at st ,ωt max st ,at ,ωt ,ωt+1 at ,ωt+1 ∞ ∗ Jt+1 (st+1 , ωt+1 ) − Ji+1 (st+1 , ωt+1 ) st+1 ,ωt+1 ∞ ∗ Jt+1 (st+1 , ωt+1 ) − Jt+1 (st+1 , ωt+1 ) ∞ ∗ + γ||Jt+1 − Jt+1 ||∞ = esub + γet+1 t Therefore, we obtain that et ≤ t (B.20) + γet+1 . Together with (B.18), we have + esub t T γ τ −t (esub + wτ∞ + t et ≤ τ =t This completes the proof. 149 ∞ τ ) (B.21) [...]... Stochastic Dynamic Programming TOP Take-Or-Pay xiv Chapter 1 Introduction This chapter introduces natural gas supply management and contract allocation and valuation for a power generation company The background of this research is presented in Section 1.1 Section 1.2 clarifies the motivation for natural gas supply management and contract allocation and valuation In Section 1.3, we review the literature... contractual flexibilities 1.3 Natural Gas Supply Portfolio and Contract Allocation and Valuation: A Brief Review Depending on whether multiple contracts or a single contract is involved in the optimization framework, the literature can be broadly classified into two branches: natural gas supply portfolio and contract allocation and valuation The former branch (Section 1.3.1) mainly addresses natural gas. .. States (Tan et al., 2010) With gas- fired power plants becoming increasingly popular, natural gas supply management has attracted considerable interest from researchers and practitioners The relevance of the topic is apparent for the power generation companies (GENCOs), as fuel cost accounts for more than 70% of total generation cost for gas- fired power plants (Chang and Hin Tay, 2006) Natural gas has... variability in the deregulated environment 1.1.2 Natural Gas Contracts A gas contract is a purchase and sale agreement between a buyer and a supplier that specifies the total amount of contract gas for delivery over a finite time horizon at a predetermined contract price Engaging into a contract allows the market participants to lock in prices for a portion of gas supply in advance Thus, the contract can... is highly imperative to develop a plan for strategic gas supply management to reap more fuel cost savings Natural gas supply management is crucial for natural gas- fired GENCOs, especially after the structural deregulation of natural gas industry It has been reported that natural gas prices are substantially volatile, ranking second only to electricity among commonly traded commodities (Hale, 2002) The... the computational effort explodes quickly as the problem size grows The incorporation of both make up clause and carry forward clause in a unified contract valuation framework was first proposed by Chiarella et al (2011), where the rights of variable contract gas delivery are exercised on a daily basis and decisions on the make up and carry forward usage are made on an annual basis A major weakness of... dynamically allocate the natural gas contract in response to the frequently changing market conditions • Develop a novel scheme for contract valuation and contract price determination to aid the contract negotiation between the GENCO and the gas supplier • Propose a multi-time scale Markov decision process model for contract nomination and allocation with a particular hierarchical structure Develop a. .. strategy of selecting a combination of multiple supply contracts and storage facilities (if available), while the latter branch (Section 1.3.2 and 1.3.3) tackles the optimization problem of single contract allocation and valuation in the presence of market transaction 1.3.1 Natural Gas Supply Portfolio The study of natural gas supply portfolio can be dated back to O’Neill et al (1979), where a large-scale... (Guldmann, 1986), contract pricing terms and storage pump capability (Avery et al., 1992), deliverability and security (Bopp et al., 1996) and market curtailment and trade-off between contract characteristics (Guldmann and Wang, 1999) The works above focused more on the natural gas portfolio for a local distribution company, whereas another stream of research aimed to develop gas contract portfolio strategies... natural gas supply management problem in Chapter 3 is extended to a multi-time scale natural gas contract nomination and allocation problem taking into account additional contractual flexibility introduced by 14 make up clause We develop a multi-time scale Markov decision process (MMDP) model to integrate the monthly nomination in a coarse time scale and daily contract allocation in a fine time scale . NATURAL GAS SUPPLY MANAGEMENT AND CONTRACT ALLOCATION AND VALUATION FOR A POWER GENERATION COMPANY WENG RENRONG NATIONAL UNIVERSITY OF SINGAPORE 2015 NATURAL GAS SUPPLY MANAGEMENT AND CONTRACT. natural gas contract and spot trading. This thesis mainly concerns about the management of natural gas supply and contract gas allocation and valuation taking into account the volatile gas prices. 1.2 clarifies the motivation for natural gas supply management and contract allocation and valuation. In Section 1.3, we review the literature related to the development of natural gas supply