Modelling Mortality with Actuarial Applications Actuaries have access to a wealth of individual data in pension and insurance portfolios, but rarely use its full potential This book will pave the way, from methods using aggregate counts to modern developments in survival analysis Based on the fundamental concept of the hazard rate, Part One shows how and why to build statistical models based on data at the level of the individual persons in a pension scheme or life insurance portfolio Extensive use is made of the R statistics package Smooth models, including regression and spline models in one and two dimensions, are covered in depth in Part Two Finally, Part Three uses multiple-state models to extend survival models beyond the simple life/death setting, and includes a brief introduction to the modern counting process approach Practising actuaries will find this book indispensable and students will find it helpful when preparing for their professional examinations a n g u s s m ac d o na l d is Professor of Actuarial Mathematics at Heriot-Watt University, Edinburgh He is an actuary with much experience of modelling mortality and other life histories, particularly in connection with genetics, and as a member of Continuous Mortality Investigation committees s t e p h e n j r i c h a r d s is an actuary and principal of Longevitas Ltd., Edinburgh, a software and consultancy firm that uses many of the models described in this book with life insurance and pension scheme clients worldwide i a i n d c u r r i e is an Honorary Research Fellow at Heriot-Watt University, Edinburgh As a statistician, he was chiefly responsible for the development of the spline models described in this book, and their application to actuarial problems I N T E R NAT I O NA L S E R I E S O N AC T UA R I A L S C I E N C E Editorial Board Christopher Daykin (Independent Consultant and Actuary) Angus Macdonald (Heriot-Watt University) The International Series on Actuarial Science, published by Cambridge University Press in conjunction with the Institute and Faculty of Actuaries, contains textbooks for students taking courses in or related to actuarial science, as well as more advanced works designed for continuing professional development or for describing and synthesising research The series is a vehicle for publishing books that reflect changes and developments in the curriculum, that encourage the introduction of courses on actuarial science in universities, and that show how actuarial science can be used in all areas where there is long-term financial risk A complete list of books in the series can be found at www.cambridge.org/statistics Recent titles include the following: Claims Reserving in General Insurance David Hindley Financial Enterprise Risk Management (2nd Edition) Paul Sweeting Insurance Risk and Ruin (2nd Edition) David C.M Dickson Predictive Modeling Applications in Actuarial Science, Volume 2: Case Studies in Insurance Edited by Edward W Frees, Richard A Derrig & Glenn Meyers Predictive Modeling Applications in Actuarial Science, Volume 1: Predictive Modeling Techniques Edited by Edward W Frees, Richard A Derrig & Glenn Meyers Computation and Modelling in Insurance and Finance Erik Bølviken MODELLING MORTALITY WITH ACTUARIAL APPLICATIONS A N G U S S M AC D O NA L D Heriot-Watt University, Edinburgh STEPHEN J RICHARDS Longevitas Ltd, Edinburgh IAIN D CURRIE Heriot-Watt University, Edinburgh University Printing House, Cambridge CB2 8BS, United Kingdom One Liberty Plaza, 20th Floor, New York, NY 10006, USA 477 Williamstown Road, Port Melbourne, VIC 3207, Australia 314–321, 3rd Floor, Plot 3, Splendor Forum, Jasola District Centre, New Delhi – 110025, India 79 Anson Road, #06–04/06, Singapore 079906 Cambridge University Press is part of the University of Cambridge It furthers the University’s mission by disseminating knowledge in the pursuit of education, learning, and research at the highest international levels of excellence www.cambridge.org Information on this title: www.cambridge.org/9781107045415 DOI: 10.1017/9781107051386 © Angus S Macdonald, Stephen J Richards and Iain D Currie 2018 This publication is in copyright Subject to statutory exception and to the provisions of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press First published 2018 Printed in the United Kingdom by Clays, St Ives plc A catalogue record for this publication is available from the British Library ISBN 978-1-107-04541-5 Hardback Cambridge University Press has no responsibility for the persistence or accuracy of URLs for external or third-party internet websites referred to in this publication and does not guarantee that any content on such websites is, or will remain, accurate or appropriate Contents Preface PART ONE page xi ANALYSING PORTFOLIO MORTALITY 1 Introduction 1.1 Survival Data 1.2 Software 1.3 Grouped Counts 1.4 What Mortality Ratio Should We Analyse? 1.5 Fitting a Model to Grouped Counts 1.6 Technical Limits for Models for Grouped Data 1.7 The Problem with Grouped Counts 1.8 Modelling Grouped Counts 1.9 Survival Modelling for Actuaries 1.10 The Case Study 1.11 Statistical Notation 3 6 11 13 15 15 17 19 Data Preparation 2.1 Introduction 2.2 Data Extraction 2.3 Field Validation 2.4 Relationship Checking 2.5 Deduplication 2.6 Bias in Rejections 2.7 Sense-Checking 2.8 Derived Fields 2.9 Preparing Data for Modelling and Analysis 2.10 Exploratory Data Plots 20 20 20 25 25 26 29 29 32 36 41 v vi Contents The Basic Mathematical Model 3.1 Introduction 3.2 Random Future Lifetimes 3.3 The Life Table 3.4 The Hazard Rate, or Force of Mortality 3.5 An Alternative Formulation 3.6 The Central Rate of Mortality 3.7 Application to Life Insurance and Annuities 45 45 47 49 50 53 54 55 Statistical Inference with Mortality Data 4.1 Introduction 4.2 Right-Censoring 4.3 Left-Truncation 4.4 Choice of Estimation Approaches 4.5 A Probabilistic Model for Complete Lifetimes 4.6 Data for Estimation of Mortality Ratios 4.7 Graduation of Mortality Ratios 4.8 Examples: the Binomial and Poisson Models 4.9 Estimating the Central Rate of Mortality? 4.10 Census Formulae for E cx 4.11 Two Approaches 56 56 58 60 61 64 67 69 71 72 73 73 Fitting a Parametric Survival Model 5.1 Introduction 5.2 Probabilities of the Observed Data 5.3 Likelihoods for Survival Data 5.4 Example: a Gompertz Model 5.5 Fitting the Gompertz Model 5.6 Data for Single Years of Age 5.7 The Likelihood for the Poisson Model 5.8 Single Ages versus Complete Lifetimes 5.9 Parametric Functions Representing the Hazard Rate 75 75 76 78 79 80 86 88 90 92 Model Comparison and Tests of Fit 6.1 Introduction 6.2 Comparing Models 6.3 Deviance 6.4 Information Criteria 6.5 Tests of Fit Based on Residuals 6.6 Statistical Tests of Fit 6.7 Financial Tests of Fit 94 94 94 95 97 100 102 109 Contents vii Modelling Features of the Portfolio 7.1 Categorical and Continuous Variables 7.2 Stratifying the Experience 7.3 Consequences of Stratifying the Data 7.4 Example: a Proportional Hazards Model 7.5 The Cox Model 7.6 Analysis of the Case Study Data 7.7 Consequences of Modelling the Data 112 112 115 120 122 124 125 129 Non-parametric Methods 8.1 Introduction 8.2 Comparison against a Reference Table 8.3 The Kaplan–Meier Estimator 8.4 The Nelson–Aalen Estimator 8.5 The Fleming–Harrington Estimator 8.6 Extensions to the Kaplan–Meier Estimator 8.7 Limitations and Applications 132 132 133 134 140 141 141 142 Regulation 9.1 Introduction 9.2 Background 9.3 Approaches to Probabilistic Reserving 9.4 Quantile Estimation 9.5 Mortality Risk 9.6 Mis-estimation Risk 9.7 Trend Risk 9.8 Number of Simulations 9.9 Idiosyncratic Risk 9.10 Aggregation 145 145 145 147 148 150 151 153 155 155 157 PART TWO REGRESSION AND PROJECTION MODELS 161 Methods of Graduation I: Regression Models 10.1 Introduction 10.2 Reading Data from the Human Mortality Database into R 10.3 Fitting the Gompertz Model with Least Squares 10.4 Poisson Regression Model 10.5 Binomial Regression Model 10.6 Exponential Family 10.7 Generalised Linear Models 163 163 165 166 172 173 177 178 10 viii Contents 10.8 Gompertz Model with Poisson Errors 10.9 Gompertz Model with Binomial Errors 10.10 Polynomial Models 179 181 182 11 Methods of Graduation II: Smooth Models 11.1 Introduction 11.2 Whittaker Smoothing 11.3 B-Splines and B-Spline Bases 11.4 B-Spline Regression 11.5 The Method of P-Splines 11.6 Effective Dimension of a Model 11.7 Deviance of a Model 11.8 Choosing the Smoothing Parameter 11.9 Overdispersion 11.10 Dealing with Overdispersion 185 185 187 189 191 193 198 199 201 203 205 12 Methods of Graduation III: Two-Dimensional Models 12.1 Introduction 12.2 The Lee–Carter Model 12.3 The Cairns–Blake–Dowd Model 12.4 A Smooth Two-Dimensional Model 12.5 Comparing Models 208 208 210 214 216 222 13 Methods of Graduation IV: Forecasting 13.1 Introduction 13.2 Time Series 13.3 Penalty Forecasting 13.4 Forecasting with the Lee–Carter Model 13.5 Simulating the Future 13.6 Forecasting with the Cairns–Blake–Dowd Model 13.7 Forecasting with the Two-Dimensional P-Spline Model 13.8 Model Risk 224 224 225 232 236 238 243 247 251 PART THREE 253 14 MULTIPLE-STATE MODELS Markov Multiple-State Models 14.1 Insurance Contracts beyond “Alive” and “Dead” 14.2 Multiple-State Models for Life Histories 14.3 Definitions 14.4 Examples 14.5 Markov Multiple-State Models 14.6 The Kolmogorov Forward Equations 255 255 256 258 260 262 264 Contents 14.7 14.8 14.9 14.10 14.11 Why Multiple-State Models and Intensities? Solving the Kolmogorov Equations Life Contingencies: Thiele’s Differential Equations Semi-Markov Models Credit Risk Models ix 269 271 274 276 278 15 Inference in the Markov Model 15.1 Introduction 15.2 Counting Processes 15.3 An Example of a Life History 15.4 Jumps and Waiting Times 15.5 Aalen’s Multiplicative Model 15.6 The Likelihood for Single Years of Age 15.7 Properties of the MLEs for Single Ages 15.8 Estimation Using Complete Life Histories 15.9 The Poisson Approximation 15.10 Semi-Markov Models 15.11 Historical Notes 279 279 280 282 284 285 286 288 289 290 292 293 16 Competing Risks Models 16.1 The Competing Risks Model 16.2 The Underlying Random Future Lifetimes 16.3 The Unidentifiability Problem 16.4 A Traditional Actuarial Approach 16.5 Are Competing Risks Models Useful? 294 294 296 298 300 304 17 Counting Process Models 17.1 Introduction 17.2 Basic Concepts and Notation for Stochastic Processes 17.3 Stochastic Integrals 17.4 Martingales 17.5 Martingales out of Counting Processes 17.6 Martingale Central Limit Theorems 17.7 A Brief Outline of the Uses of Counting Process Models 307 307 308 312 314 317 319 320 Appendix A A.1 A.2 A.3 A.4 R Commands Introduction Running R R Commands Probability Distributions Appendix B Basic Likelihood Theory B.1 Scalar Parameter Models: Theory 329 329 330 330 330 334 334 ... Frees, Richard A Derrig & Glenn Meyers Computation and Modelling in Insurance and Finance Erik Bølviken MODELLING MORTALITY WITH ACTUARIAL APPLICATIONS A N G U S S M AC D O NA L D Heriot-Watt... Professor of Actuarial Mathematics at Heriot-Watt University, Edinburgh He is an actuary with much experience of modelling mortality and other life histories, particularly in connection with genetics,.. .Modelling Mortality with Actuarial Applications Actuaries have access to a wealth of individual data in pension and