Time series analysis

Time Series Analysis C5967_FM.indd 10/22/07 10:39:05 AM CHAPMAN & HALL/CRC Texts in Statistical Science Series Series Editors Bradley P Carlin, University of Minnesota, USA Julian J Faraway, University of Bath, UK Martin Tanner, Northwestern University, USA Jim Zidek, University of British Columbia, Canada Analysis of Failure and Survival Data P J Smith The Analysis and Interpretation of Multivariate Data for Social Scientists D.J Bartholomew, F Steele, I Moustaki, and J Galbraith The Analysis of Time Series — An Introduction, Sixth Edition C Chatfield Applied Bayesian Forecasting and Time Series Analysis A Pole, M West and J Harrison Applied Nonparametric Statistical Methods, Fourth Edition P Sprent and N.C Smeeton Applied Statistics — Handbook of GENSTAT Analysis E.J Snell and H Simpson Applied Statistics — Principles and Examples D.R Cox and E.J Snell Bayes and Empirical Bayes Methods for Data Analysis, Second Edition B.P Carlin and T.A Louis Bayesian Data Analysis, Second Edition A Gelman, J.B Carlin, H.S Stern and D.B Rubin Beyond ANOVA — Basics of Applied Statistics R.G Miller, Jr Computer-Aided Multivariate Analysis, Fourth Edition A.A Afifi and V.A Clark A Course in Categorical Data Analysis T Leonard A Course in Large Sample Theory T.S Ferguson Data Driven Statistical Methods P Sprent Decision Analysis — A Bayesian Approach J.Q Smith Elementary Applications of Probability Theory, Second Edition H.C Tuckwell C5967_FM.indd Elements of Simulation B.J.T Morgan Epidemiology — Study Design and Data Analysis, Second Edition M Woodward Essential Statistics, Fourth Edition D.A.G Rees Extending the Linear Model with R: Generalized Linear, Mixed Effects and Nonparametric Regression Models J.J Faraway A First Course in Linear Model Theory N Ravishanker and D.K Dey Generalized Additive Models: An Introduction with R S Wood Interpreting Data — A First Course in Statistics A.J.B Anderson An Introduction to Generalized Linear Models, Second Edition A.J Dobson Introduction to Multivariate Analysis C Chatfield and A.J Collins Introduction to Optimization Methods and Their Applications in Statistics B.S Everitt Introduction to Randomized Controlled Clinical Trials, Second Edition J.N.S Matthews Introduction to Statistical Methods for Clinical Trials Thomas D Cook and David L DeMets Large Sample Methods in Statistics P.K Sen and J da Motta Singer Linear Models with R J.J Faraway Markov Chain Monte Carlo — Stochastic Simulation for Bayesian Inference, Second Edition D Gamerman and H.F Lopes Mathematical Statistics K Knight Modeling and Analysis of Stochastic Systems V Kulkarni 10/22/07 10:39:05 AM Modelling Binary Data, Second Edition D Collett Modelling Survival Data in Medical Research, Second Edition D Collett Multivariate Analysis of Variance and Repeated Measures — A Practical Approach for Behavioural Scientists D.J Hand and C.C Taylor Multivariate Statistics — A Practical Approach B Flury and H Riedwyl Practical Data Analysis for Designed Experiments B.S Yandell Practical Longitudinal Data Analysis D.J Hand and M Crowder Practical Statistics for Medical Research D.G Altman Probability — Methods and Measurement A O’Hagan Problem Solving — A Statistician’s Guide, Second Edition C Chatfield Randomization, Bootstrap and Monte Carlo Methods in Biology, Third Edition B.F.J Manly Readings in Decision Analysis S French Sampling Methodologies with Applications P.S.R.S Rao Statistical Analysis of Reliability Data M.J Crowder, A.C Kimber, T.J Sweeting, and R.L Smith C5967_FM.indd Statistical Methods for Spatial Data Analysis O Schabenberger and C.A Gotway Statistical Methods for SPC and TQM D Bissell Statistical Methods in Agriculture and Experimental Biology, Second Edition R Mead, R.N Curnow, and A.M Hasted Statistical Process Control — Theory and Practice, Third Edition G.B Wetherill and D.W Brown Statistical Theory, Fourth Edition B.W Lindgren Statistics for Accountants S Letchford Statistics for Epidemiology N.P Jewell Statistics for Technology — A Course in Applied Statistics, Third Edition C Chatfield Statistics in Engineering — A Practical Approach A.V Metcalfe Statistics in Research and Development, Second Edition R Caulcutt Survival Analysis Using S —Analysis of Time-to-Event Data M Tableman and J.S Kim Time Series Analysis H Madsen The Theory of Linear Models B Jørgensen 10/22/07 10:39:05 AM C5967_FM.indd 10/22/07 10:39:05 AM Texts in Statistical Science Time Series Analysis Henrik Madsen Technical University of Denmark Boca Raton London New York Chapman & Hall/CRC is an imprint of the Taylor & Francis Group, an informa business C5967_FM.indd 10/22/07 10:39:05 AM CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2007 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S Government works Version Date: 20140313 International Standard Book Number-13: 978-1-4200-5968-7 (eBook - PDF) This book contains information obtained from authentic and highly regarded sources Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint Except as permitted under U.S Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers For permission to photocopy or use material electronically from this work, please access www.copyright com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400 CCC is a not-for-profit organization that provides licenses and registration for a variety of users For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com “C5967 Final 1st Edition” — 2007/10/24 — 14:13 — page vii — #1 Contents Preface xiii Notation Introduction 1.1 Examples of time series 1.1.1 Dollar to Euro exchange rate 1.1.2 Number of monthly airline passengers 1.1.3 Heat dynamics of a building 1.1.4 Predator-prey relationship 1.2 A first crash course 1.3 Contents and scope of the book xv 2 Multivariate random variables 2.1 Joint and marginal densities 2.2 Conditional distributions 2.3 Expectations and moments 2.4 Moments of multivariate random variables 2.5 Conditional expectation 2.6 The multivariate normal distribution 2.7 Distributions derived from the normal distribution 2.8 Linear projections 2.9 Problems 13 13 14 15 17 20 22 23 24 29 Regression-based methods 3.1 The regression model 3.2 The general linear model (GLM) 3.2.1 Least squares (LS) estimates 3.2.2 Maximum likelihood (ML) estimates 3.3 Prediction 3.3.1 Prediction in the general linear model 3.4 Regression and exponential smoothing 3.4.1 Predictions in the constant mean model 31 31 33 34 40 44 45 47 48 “C5967 Final 1st Edition” — 2007/10/24 — 14:13 — page viii — #2 3.4.2 3.5 3.6 3.7 Locally constant mean model and simple exponential smoothing 3.4.3 Prediction in trend models 3.4.4 Local trend and exponential smoothing Time series with seasonal variations 3.5.1 The classical decomposition 3.5.2 Holt-Winters procedure Global and local trend model—an example Problems Linear dynamic systems 4.1 Linear systems in the time domain 4.2 Linear systems in the frequency domain 4.3 Sampling 4.4 The z-transform 4.5 Frequently used operators 4.6 The Laplace transform 4.7 A comparison between transformations 4.8 Problems 50 52 56 59 60 61 62 65 69 70 73 78 80 87 90 94 96 Stochastic processes 5.1 Introduction 5.2 Stochastic processes and their moments 5.2.1 Characteristics for stochastic processes 5.2.2 Covariance and correlation functions 5.3 Linear processes 5.3.1 Processes in discrete time 5.3.2 Processes in continuous time 5.4 Stationary processes in the frequency domain 5.5 Commonly used linear processes 5.5.1 The MA process 5.5.2 The AR process 5.5.3 The ARMA process 5.6 Non-stationary models 5.6.1 The ARIMA process 5.6.2 Seasonal models 5.6.3 Models with covariates 5.6.4 Models with time-varying mean values 5.6.5 Models with time-varying coefficients 5.7 Optimal prediction of stochastic processes 5.7.1 Prediction in the ARIMA process 5.8 Problems 97 97 97 99 103 107 107 111 113 117 117 119 125 130 130 132 134 134 135 135 137 140 “C5967 Final 1st Edition” — 2007/10/24 — 14:13 — page ix — #3 Identification, estimation, and model checking 6.1 Introduction 6.2 Estimation of covariance and correlation functions 6.2.1 Autocovariance and autocorrelation functions 6.2.2 Cross-covariance and cross-correlation functions 6.3 Identification 6.3.1 Identification of the degree of differencing 6.3.2 Identification of the ARMA part 6.3.3 Cointegration 6.4 Estimation of parameters in standard models 6.4.1 Moment estimates 6.4.2 The LS estimator for linear dynamic models 6.4.3 The prediction error method 6.4.4 The ML method for dynamic models 6.5 Selection of the model order 6.5.1 The autocorrelation functions 6.5.2 Testing the model 6.5.3 Information criteria 6.6 Model checking 6.6.1 Cross-validation 6.6.2 Residual analysis 6.7 Case study: Electricity consumption 6.8 Problems 145 145 146 146 150 152 153 154 156 157 157 159 163 166 170 171 171 174 174 175 175 179 182 187 187 189 190 194 195 196 200 203 206 206 209 210 Linear systems and stochastic processes 8.1 Relationship between input and output processes 8.1.1 Moment relations 215 215 216 Spectral analysis 7.1 The periodogram 7.1.1 Harmonic analysis 7.1.2 Properties of the periodogram 7.2 Consistent estimates of the spectrum 7.2.1 The truncated periodogram 7.2.2 Lag- and spectral windows 7.2.3 Approximative distributions for spectral estimates 7.3 The cross-spectrum 7.3.1 The co-spectrum and the quadrature spectrum 7.3.2 Cross-amplitude spectrum, phase spectrum, coherence spectrum, gain spectrum 7.4 Estimation of the cross-spectrum 7.5 Problems “C5967 Final 1st Edition” — 2007/10/24 — 14:13 — page 359 — #369 Partial autocorrelations 359 where Bi is obtained by substituting the i’th column in A with B Thus, it is seen from (B.7) that φk,k is determined from the equation set:    ρ1 φk,1     Pk   =    (B.8) ρk φk,k which is seen to be the Yule-Walker equations for an AR(k) process, i.e., φk,k can be found as the moment estimate of the last coefficient in an AR(k) model Based on (B.8) we can provide a recursive method to determine φk,k We introduce φk = (φk,1 , , φk,k ), φk = (φk,k , , φk,1 ) φk,k−1 = (φk,1 , , φk,k−1 ), φk,k−1 = (φk,k−1 , , φk,1 ) Writing (B.8) for k + yields Pk ρk ρTk φk+1,k φk+1,k+1 = ρTk ρk+1 Pk φTk+1,k + ρTk φk+1,k+1 = ρTk ⇔ ⇔ ρk φTk+1,k + φk+1,k+1 = ρk+1 φTk+1,k = Pk−1 (ρTk − ρTk φk+1,k+1 ) ρk Pk−1 (ρTk − ρTk φk+1,k+1 ) + φk+1,k+1 = ρk+1  φTk+1,k = φTk − φTk φk+1,k+1   T ρk+1 − ρk φk  φk+1,k+1 =  − ρk φTk ⇔ We get the following recursion formulas for calculation of φk,k for k = 1, 2, : φk+1,j = φk,j − φk+1,k+1 φk,k+1−j , ρk+1 − φk+1,k+1 = k j=1 1− with the initial value φ11 = ρ1 φk,j ρk+1−j k j=1 φk,j ρj j = 1, , k “C5967 Final 1st Edition” — 2007/10/24 — 14:13 — page 360 — #370 “C5967 Final 1st Edition” — 2007/10/24 — 14:13 — page 361 — #371 APPENDIX C Some results from trigonometry The following identity holds: m e−iωk = k=0 − e−iω(m+1) − e−iω = e−iωm/2 (C.1) eiω(m+1)/2 − e−iω(m+1)/2 eiω/2 − e−iω/2 (C.2) The first equality is obtained since − e−iω m e−iωk = − e−iω(m+1) k=0 i.e., the in-between terms vanish The last equality is seen directly Correspondingly we have: m eiωk = eiωm/2 k=0 eiω(m+1)/2 − e−iω(m+1)/2 eiω/2 − e−iω/2 (C.3) Applying the Euler relation yields eiω = cos(ω) + i sin(ω) (C.4) and the inverse relations: cos(ω) = yield iω e + e−iω , sin(ω) = m (C.5) cos(ωk) = cos(ωm/2) sin(ω(m + 1)/2) sin(ω/2) (C.6) sin(ωk) = sin(ωm/2) sin(ω(m + 1)/2) sin(ω/2) (C.7) k=0 m k=0 iω e − e−iω 2i “C5967 Final 1st Edition” — 2007/10/24 — 14:13 — page 362 — #372 362 Some results from trigonometry For ω equal to integer values times 2π, we have: m cos(ωk) = m + , for ω = 2pπ, p ∈ Z, (C.8) k=0 m sin(ωk) = , for ω = 2pπ, p ∈ Z (C.9) k=0 From (C.1) and (C.2) as well as the Euler relations, it follows that m k=−m e−iωk = eiωm/2 + e−iωm/2 sin(ω(m + 1)/2) −1 sin(ω/2) sin (m + 21 )ω = = Dm (ω)2π sin(ω/2) (C.10) The function Dm (ω) is called the Dirichlet kernel of order m The above mentioned relations are commonly applied in spectral analysis together with the well-known trigonometry formulas for addition “C5967 Final 1st Edition” — 2007/10/24 — 14:13 — page 363 — #373 APPENDIX D List of acronyms The number following the acronym marks the page where the acronym is first described ACF AutoCorrelation Function, 103 ELS Extended Least Squares, 321 AIC Akaike’s Information Criterion, FIR Finite Impulse Response, 223 174 GLM General Linear Model, 33 AR AutoRegressive, IACF Inverse AutoCorrelation FuncARI Integrated AutoRegressive, 131 tion, 129 ARIMA Integrated AutoRegressive- IMA Integrated Moving Average, MovingAverage, 131 ARMA AutoRegressive-MovingAverage, LS Least Squares, 34 CARMA Controlled AutoRegressiveMovingAverage, 223 MLE Maximum Likelihood Estimate, 40 CCF Cross Correlation Function, 104 OE Output Error, 223 MA Moving Average, 117 ARMAX AutoRegressiveMoving Average with eXogenous input, MARIMA Multivariate AutoRegressive Integrated Moving 223 Average, 249 ARX AutoRegressive with eXogenous input, 223 MIMO Multiple-Input, MultipleOutput, 215 BIC Bayesian Information Criteria, 174 MISO Multiple-Input, SingleOutput, 215 BLUE Best Linear Unbiased Estimator, 37 ML Maximum Likelihood, 40 CLS Conditioned Least Squares Method, 163 OEM Output Error Method, 223 OLS Ordinary Least Squares, 35 “C5967 Final 1st Edition” — 2007/10/24 — 14:13 — page 364 — #374 364 List of acronyms PACF Partial AutoCorrelation Function, 124 SISO Single-Input, Single-Output, 215 RLS Recursive Least Squares, 315 SSE Sum of Squared Errors, 34 RML Recursive Maximum Likelihood, 324 VAR Vector AutoRegressive, 260 RPEM Recursive Prediction Error Method, 321 VARMA Vector AutoRegressiveMoving Average, 249 RPLR Recursive Pseudo Linear ReVARIMA Vector AutoRegressive gression, 319 Integrated Moving Average, 249 SBC Schwartz’s Bayesian Criterion, 174 WLS Weighted Least Squares, 35 “C5967 Final 1st Edition” — 2007/10/24 — 14:13 — page 365 — #375 APPENDIX E List of symbols The number following the symbol description marks the page where the symbol is first mentioned X FX (x) P{X1 ≤ x1 } fX (x) fS (x) FS (x) fX,Y (x,y) fY |X=x µ, E[X] σx2i , Var[Xi ] σij , Cov[Xi , Xj ] ΣX , Var[X] ρij R E[Y | X = x] Var[Y | X = x] N (µ, σ ) χ2 (n) X− θ θ L i.i.d εt λ c L Tt St δk yt , y(t) n-variate random variable, 13 Joint distribution function of X, 13 Probability of X1 ≤ x1 , 13 Joint density function of X, 13 Marginal density function, 14 Marginal distribution function, 14 Joint density function of X and Y , 14 Conditional density function for Y given X = x, 14 Expectation or mean value or first moment of X, 15 Variance or second central moment of Xi , 16 Covariance between Xi and Xj , 17 Covariance matrix of X, 18 Correlation between Xi and Xj , 18 Correlation matrix, 18 Conditional expectation of Y given X, 20 Conditional variance of Y given X, 21 Normal distribution with mean µ and variance σ , 22 Chi-squared distribution with n degrees of freedom, 23 Generalized inverse (g-inverse) of X, 24 Parameter vector, 31 Estimator of parameter vector, 34 Likelihood function, 40 Independent identically distributed, 48 White noise process, 48 Forgetting factor, 50 Normalizing constant, 50 Transition matrix, 53 Trend, 60 Seasonal effect, 60 Kronecker’s delta sequence, impulse function, 60 Signal in time domain, 69 “C5967 Final 1st Edition” — 2007/10/24 — 14:13 — page 366 — #376 366 ∗ h(k), h(u) δ(t) Sk , S(t) Y (ω) H(ω) B, z −1 F, z H(z) ∇ S H(s) γXX , γXX (k) ρ(k), ACF f (ω) φkk , PACF ρi(k), IACF ∇s Yt+k|t I(θ) I(ω) W (θ) b(ω) Γk cXY (ω) qXY (ω) αXY (ω) φXY (ω) wXY (ω) GXY (ω) Kt Xt|t I List of symbols Convolution operator, 70 Impulse response function, 70 Dirac delta function, impulse function, 70 Step response function, 72 Signal in frequency domain, 73 Frequency response function, 73 Backward shift operator, 82 Forward shift operator, 82 Transfer function, 83 Difference operator, 87 Summation operator, 87 Transfer function, 92 Autocovariance function, 99 Autocorrelation function, 103 Spectrum, power spectrum, 114 Partial autocorrelation function, 124 Inverse autocorrelation function, 129 s-season difference operator, 132 k-step predictor, 138 Fisher information matrix, 168 Periodogram, 187 Spectral window, 195 Skewness, 197 Covariance matrix at lag k, 205 Co-spectrum, 206 Quadrature spectrum, 206 Cross-amplitude spectrum, 206 Phase spectrum, 206 Complex coherency, 206 Gain spectrum, 207 Kalman gain, 287 Filter estimator, reconstruction of Xt , 290 Identity matrix, 309 “C5967 Final 1st Edition” — 2007/10/24 — 14:13 — page 367 — #377 Bibliography Abraham, B., and J Ledolter (1983) Statistical Methods for Forecasting New York: Wiley Andersen, K., H Madsen, and L Hansen (2000) “Modelling the heat dynamics of a building using stochastic differential equations.” In: Energy and Buildings 31 13–34 Åström, K J (1970) Introduction to Stochastic Control Theory New York: Academic Press Åström, K J., and P Eykhoff (1971) “System identification – a survey.” In: Automatica 123–162 Baadsgaard, M., J N Nielsen, and H Madsen (2000) “Estimating Multivariate Exponentail-Affine Term Structure Models from Coupon Bound Prices using Nonlinear Filtering.” In: Econometric Journal 1–20 Bard, Y (1974) Non-linear Parameter Estimation London: Academic Press Bartlett, M S (1946) “On the theoretical specification of sampling properties of an autocorrelated process.” In: Journal Royal Statistical Society B 27–41 Bechmann, H., M K Nielsen, N K Poulsen, and H M Madsen (2002) “Greybox modelling of aeration tank settling.” In: Water Research 36 1887–1895 Box, G E P., and D R Cox (1964) “An Analysis of Transformations.” In: Journal of Royal Statistical Society, Series B 26 211–246 Box, G E P., and G M Jenkins (1970/1976) Time Series Analysis, Forecasting and Control San Francisco: Holden-Day Box, G E P., and G C Tiao (1975) “Intervention Analysis with Applications to Economic and Environmental Problems.” In: Journal of the American Statistical Association 70 70–79 Brillinger, D R (1981) Time Series; Data Analysis and Theory San Francisco: Holden-Day Brockwell, P J., and R A Davis (1987) Time Series; Theory and Methods New York: Springer-Verlag Brown, R G (1963) Smoothing, Forecasting and Prediction Englewood Cliffs, NJ: Prentice Hall Carstensen, N J (1990) Adaptive Monitoring of Air Quality Lyngby, Denmark: IMM/DTU “C5967 Final 1st Edition” — 2007/10/24 — 14:13 — page 368 — #378 368 Bibliography Chatfield, C (2003) The Analysis of Time Series: An Introduction 6th ed New York: Chapman, Hall Christensen, R (2002) Plane Answers to Complex Questions; The Theory of Linear Models New York: Springer Conradsen, K (1984) En introduktion til Statistik, Bind 5th ed Lyngby, Denmark: IMSOR Cox, D R., and H D Miller (1968) The Theory of Stochastic Processes London: Chapman, Hall Daniells, H E (1956) “The approximate distribution of serial correlation coefficients.” In: Biometrika 43 169–185 Davis, M H A., and R B Vinter (1985) Stochastic Modelling and Control London: Chapman, Hall Doob, J L (1953) Stochastic Processes New York: Wiley Dunsmuir, W., and P M Robinson (1981) “Estimation of time series models in the presence of missing data.” In: Journal American Statistical Association 76 560–568 ENFOR (2007) Prediction related to energy systems 2007 url: http://www enfor.eu Engle, R., and B S You (1985) “Forecasting and testing in cointegrated systems.” In: Journal of Econometrics 35 143–159 Funch, B O., and G K Hansen (1987) “Varianskomponenter i tidsrække- og interventionsanalyse.” MA thesis DTU, Lyngby, Denmark: IMSOR Goodwin, C G., and R L Payne (1977) Dynamic System Identification: Experimental Design and Data Analysis New York: Academic Press Grimmit, G R., and D R Stirzaker (1992) Probability and Random Processes Oxford: Clarendon Press Haahr, A M., H Madsen, J Smedegaard, W L P Bredie, L H Stanhnke, and H H F Refsgaard (2003) “Release Measurement by Atmospheric Pressure Chemical Ionization Ion Trap Mass Spectrometry, Construction of Interface and Mathematical Modeling of Release Profiles.” In: Analytical Chemistry 75 655–662 Hald, A (1952) Statistical Theory with Engineering Applications New York: Wiley Hannan, E J (1970) Multiple Time Series New York: Wiley Harvey, A C (1996) Forecasting, Structural Time Series Models and the Kalman Filter Cambridge: Cambridge University Press ——— (1981) Time Series Models New York: Halsted Press Harvey, A C., and R G Pierse (1984) “Estimating missing observations in economic time series.” In: Journal of the American Statistical Association 79 125–131 Hastie, T., R Tibshirani, and J Friedman (2001) The Elements of Statistical Learning; Data Mining, Inference and Prediction New York: Springer Holst, J (1977) Adaptive Prediction and Recursive Estimation Lund, Sweden: Department of Automatic Control, Lund Institute of Technology “C5967 Final 1st Edition” — 2007/10/24 — 14:13 — page 369 — #379 Bibliography 369 Holst, J., and N K Poulsen (1988) Noter til Stokastisk Adaptiv Regulering Lyngby, Denmark: IMM, DTU Holt, C C (1957) Forecasting trends and seasonals by exponentially weighted moving averaverages 52 ONR Memorandum Pittsburgh: Carnegie Institute of Technology Huber, P T (1964) Robust Statistics New York: Wiley Jazwinski, A H (1970) Stochastic Processes and Filtering Theory New York: Academic Press Jenkins, G M (1979) Practical Experiences with Modelling and Forecasting Time Series Jersey: Gwilym Jenkins, Partners Ltd ——— (1975) “The interaction between the muskrat and mink cycles in North Canada.” In: 8th International Biometric Conference Constanta, Romania 55–71 Jenkins, G M., and A Alavi (1981) “Some aspects of modelling and forecasting multivariate time series.” In: Journal of Time Series Analysis 1–47 Jenkins, G.M., and D G Watts (1968) Spectral Analysis and Its Applications San Francisco: Holden-Day Jensen, H E., T Høholdt, and F Nielsen (1992) Lineære systemer Lyngby, Denmark: Matematisk Institut, DTU Johansen, S (2001) “A Course in Time Series Analysis.” In: New York: Wiley Chap Cointegration in the VAR model Johansson, R (1993) System Modelling and Identification Englewood Cliffs, New Jersey: Prentice Hall Johnson, R A., and D W Wichern (2002) Applied Multivariate Statistical Analysis 5th ed London: Prentice Hall Jónsdóttir, H., J L Jacobsen, and H Madsen (2001) “A grey-box model describing the hydraulics in a creek.” In: Environmetrics 37 347–356 Kailath, T (1980) Linear Systems New York: Prentice Hall Kalman, R E (1960) “A new approach to linear filtering and prediction problems.” In: Trans ASME Journal of Basic Engineering 82 35–45 Kendall, M G., and A Stuart (1983) The Advanced Theory of Statistics 4th ed London: Griffin Kristensen, N R., H Madsen, and S B Jørgensen (2004) “Parameter estimation in stochastic grey-box models.” In: Automatica 40 225–237 Ljung, L (1976) System Identification; Advances and Case Studies New York: Academic Press Chap On the consistency of prediction error identification methods ——— (1987) System Identification: Theory for the User New York: Prentice Hall Ljung, L., and T Söderström (1983) Theory and Practice of Recursive Identification Cambridge MA: MIT Press Lütkepohl, H (1991) Introduction to Multiple Time Series Analysis Berlin: Springer “C5967 Final 1st Edition” — 2007/10/24 — 14:13 — page 370 — #380 370 Bibliography Madsen, H (1992) Projection and Separation in Hilbert Spaces Lyngby, Denmark: IMM, DTU ——— (1985) Statistically Determined Dynamical Models for Climate Processes Lyngby, Denmark: IMSOR Madsen, H., and J Holst (1995) “Estimation of continuous-time models for the heat dynamics of a building.” In: Energy and Buildings 22 67–79 ——— (1989) “Identification of a time varying transfer function model for the variations of air temperature.” In: Fourth International Meeting on Statistical Climatology Rotorua 243–247 Madsen, H., J Holst, and E Lindström (2007) Modelling Non-linear and Non-stationary Time Series Lyngby, Denmark: IMM, DTU Madsen, H., H A Nielsen, and T S Nielsen (2005) “A tool for predicting the wind power production of off-shore wind plants.” In: Proceedings of the Copenhagen Offshore Wind Conference & Exhibition Copenhagen Madsen, H., H Spliid, and P Thyregod (1985) “Markov models in discrete and continuous time for hourly observations of cloud cover.” In: Journal of Climate and Applied Meteorology 24 629–639 Madsen, H., and P Thyregod (1988) “Modelling the Time Correlation in Hourly Observations of Direct Radiation in Clear Skies.” In: Energy and Buildings 11 201–211 Madsen, H., P Pinson, G Kariniotaktis, H A Nielsen, and T S Nielsen (2005) “Standardizing the performance evaluation of short-term wind prediction models.” In: Wind Engineering 29 475–489 Madsen, H., J N Nielsen, E Lindström, M Baadsgaard, and J Holst (2004) Statistics in Finance Lund, Sweden: University of Lund, KF Sigma Makridakis, S., and S Wheelwright (1978) Interactive Forecasting San Francisco: Holden-Day Martin, R D., and V Yohai (1985) “Robustness in Time Series and Estimating ARMA Models.” In: Vol Handbook of Statistics Amsterdam: Elsevier Science 119–155 Maybeck, P S (1982) Stochastic Models, Estimation and Control; Vol 1,2,3 New York: Academic Press McLeod, A I., and W K Li (1983) “Diagnostic checking ARMA time series models using squared-residual autocorrelations.” In: Journal of Time Series Analysis 269–273 Meinhold, R J., and N D Singpurwalla (1983) “Understanding the Kalman filter.” In: American Statistician 70 123–127 Milhøj, A (1986) Tidsrækkeanalyse for Økonomer Copenhagen: Akademisk Forlag Nielsen, H., H Madsen, and T S Nielsen (2006) “Using quantile regression to extend an existing wind power forecasting system with probabilistic forecasts.” In: Wind Energy 95–108 “C5967 Final 1st Edition” — 2007/10/24 — 14:13 — page 371 — #381 Bibliography 371 Nielsen, J N., M Vestergaard, and H Madsen (2000) “Estimation in Continuous-time Stochastic Volatility Models Using Nonlinear Filters.” In: International Journal of Theoretical and Applied Finance 279–308 Nielsen, T S., A Joensen, H Madsen, L Landberg, and G Giebel (1999) “A New Reference for Predicting Wind Power.” In: Wind Energy 29–34 Øksendal, B (1995) Stochastic Differential Equations 4th ed Berlin: Springer Olbjer, L., U Holst, and J Holst (2005) Tidsserieanalys Lund, Sweden: Centre for Mathematical Sciences, Lund University, KFS AB Olsson, U (2002) Generalized Linear Models Lund, Sweden: Studentlitteratur Palsson, O P., H Madsen, and H T Søgaard (1994) “Generalized Predictive Control for Non-Stationary System.” In: Automatica 30 1991–1997 Paltridge, G W., and C M R Platt (1976) Radiative Processes in Meteorology and Climatology Amsterdam: Elsevier Papoulis, A (1983) Probability, Random Variables and Stochastic Processes New York: McGraw-Hill ——— (1984) Signal Analysis New York: McGraw-Hill Parzen, E (1963) “On spectral analysis with missing observations and amplitude modulation.” In: Sankhya, Series A 25 383–392 ——— (1962) Stochastic Processes San Francisco: Holden-Day Pawitan, Y (2001) In All Likelihood, Statistical Modelling and Inference Using Likelihood New York: Oxford University Press Porsholt, T (1989) Tidsvarierende Økonomiske Modeller Lyngby, Denmark: IMM, DTU Priestley, M B (1981) Spectral Analysis and Time Series London: Academic Press Rao, C R (1973) Linear Statistical Inference and Its Applications 2nd ed New York: Wiley Shao, Y E (1997) “Multiple intervention analysis with application to sales promostion data.” In: Journal of Applied Statistics 24 181–192 Shumway, R H (1988) Applied Statistical Time Series Analysis Englewood Cliffs, NJ: Prentice Hall Sobczyk, K (1985) Stochastic Differential Equations for Applications Lyngby, Denmark: ABK Söderström, T (1973) An On–line Algorithm for Approximate Maximum Likelihood Identification of Linear Dynamic Systems 7308 Report Lund Institute of Technology, Lund, Sweden: Deptartment of Automatic Control Söderström, T., and P Stoica (1989) System Identification London: Prentice Hall Spliid, H (1983) “A fast estimation method for the vector autoregressive moving average model with exogenous variables.” In: Journal American Statistical Association 78 843–849 Subba Rao, T., and M M Gabr (1984) An Introduction to Bisprectral Analysis and Bilinear Time Seris Models New York: Springer “C5967 Final 1st Edition” — 2007/10/24 — 14:13 — page 372 — #382 372 Bibliography Thyregod, P., and H Madsen (2006) General and Generalized Linear Models Lyngby, Denmark: IMM, DTU ——— (1992) “On an Adaptive Acceptance Control Chart for Autocorrelated Processes.” In: Frontiers in Statistical Quality Control 138–154 Tiao, G C., and G E P Box (1981) “Modelling multivariate time series with applications.” In: Journal American Statistical Association 76 802–816 Vio, R., P Rebusco, P Andreani, H Madsen, and R V Overgaard (2006) “Stochastic modeling of kHz quasi-periodic oscillation light curves.” In: Astronomy and Astrophysics 452 383–386 Wall, K D (1987) “Identification theory for varying coefficient regression models.” In: Journal Time Series Analysis 359–371 Wei, W W S (2006) Time Series Analysis 2nd ed Addison-Wesley Wiener, N (1949) Extrapolation, Interpolation, and Smoothing of Stationary Time-Series Cambridge MA: MIT Press Winters, P R (1960) “Forecasting sales by exponentially weighted moving averages.” In: Management Science 324–342 Yaglom, A M (1962) An Introduction to the Theory Of Stationary Random Functions Englewood Cliffs, NJ: Prentice Hall Young, P C (1984) Recursive Estimation and Time Series Analysis Berlin: Springer ... basic relations 1.1 Examples of time series In this section we will show examples of time series, and at the same time indicate possible applications of time series analysis The examples contain... methods for time series analysis, as demonstrated in various chapters of this book There are a number of reasons for studying time series These include a characterization of time series (or signals),... number of muskrats In such cases both series must be included in a multivariate time series This series has been considered in many texts on time series analysis, and the purpose is to describe