global catastrophic risks sep 2008

Global Catastrophic Risks Edited by Nick Bostrom Milan M. Cirkovic OXPORD UNIVERSITY PRESS Contents Acknowledgements 10 MartinJ.Rees.Foreword 11 Contents 15 1.NickBostromandMilanM.Cirkoviс.Introduction 23 1.1Why? 23 1.2Taxonomyandorganization 24 1.3PartI:Background 27 1.4PartII:Risksfromnature 31 1.5PartIII:Risksfromunintendedconsequences 32 PartI.Background 43 2. Fred C. Adams . Long‐termastrophysicalprocesses 43 2.1Introduction:physicaleschatology 43 2.2FateoftheEarth 43 2.3Isolationofthelocalgroup 45 2.4 Collision with Andromeda 45 2.5Theendofstellarevolution 46 2.6Theeraofdegenerateremnants 47 2.7Theeraofblackholes 48 2.8TheDarkEraandbeyond 49 2.9Lifeandinformationprocessing 50 2.10Conclusion 50 Suggestionsforfurtherreading 51 References 51 3.ChristopherWills.Evolutiontheoryandthefutureofhumanity 54 3.1Introduction 54 3.2Thecausesofevolutionarychange 54 3.3Environmentalchangesandevolutionarychanges 55 3.3.1Extremeevolutionarychanges 56 3.3.2Ongoingevolutionarychanges 57 3.3.3Changesintheculturalenvironment 59 3.4Ongoinghumanevolution 62 3.4.1Behaviouralevolution 63 3.5Futureevolutionarydirections 66 Suggestionsforfurtherreading 68 4.JamesJ.Hughes.Millennialtendenciesinresponsestoapocalypticthr eats 72 4.1Introduction 72 4.2Typesofmillennialism 72 4.3Messianismandmillenarianism 74 4.4Positiveornegativeteleologies:utopianismandapocalypticism 74 4.5Contemporarytechno‐millennialism 75 4.6Techno‐apocalypticism 77 4.7Symptomsofdysfunctionalmillennialisminassessingfuturescenarios 79 4.8Conclusions 80 Suggestionsforfurtherreading 80 5. Eliezer Yudkowsky. Cognitivebiasespotentiallyaffectingjudgement ofglobalrisks 85 5.1Introduction 85 1:Availability 85 2:Hindsightbias 86 3:BlackSwans 87 4:Theconjunctionfallacy 88 5:Confirmationbias 90 6:Anchoring,adjustment,andcontamination 92 7:Theaffectheuristic 94 8:Scopeneglect 95 9:Calibrationandoverconfidence 96 10:Bystanderapathy 98 Afinalcaution 99 Conclusion 100 6.Milan M. Cirkovic. Observationselectioneffectsandglobalcatastrophicrisks 106 6.1Introduction:anthropicreasoningandglobalrisks 106 6.3DoomsdayArgument 112 6.4Fermi'sparadox 113 6.5TheSimulationArgument 118 6.6Makingprogressinstudyingobservationselectioneffects 119 7. Yacov Y. Haimes. Systems‐basedriskanalysis 121 7.1Introduction 121  7.2Risktointerdependentinfrastructureandsectorsoftheeconomy 122 7.3Hierarchicalholographicmodellingandthetheoryofscenariostructuring 123 7.4Phantomsystemmodelsforriskmanagementofemergentmulti‐scalesystems 125 7.5Riskofextremeandcatastrophicevents 127 8. Peter Taylor. Catastrophesandinsurance 135 8.1Introduction 135  8.2Catastrophes 136 8.3Whatthebusinessworldthinks 138 8.4Insurance 138 8.5Pricingtherisk 141 8.6Catastrophelossmodels 142 8.7Whatisrisk? 143 8.8Priceandprobability 145 8.9Theageofuncertainty 146 8.10Newtechniques 148 8.11Conclusion:againstthegods? 148 9. Richard A. Posner. Publicpolicytowardscatastrophe 150 PartII.Risksfromnature 162 10. Michael R. Rampino. Super‐volcanismandothergeophysicalprocessesofcatastrophicimport 163 10.1Introduction 163 10.2Atmosphericimpactofasuper‐er uption 163 10.3Volcanicwinter 164 10.4Possibleenvironmentaleffectsofasuper‐eruption 166 10.5Super‐eruptionsandhumanpopulation 167 10.6Frequencyofsuper‐eruptions 168 10.7Effectsofasuper‐eruptionsoncivilization 168 10.8Super‐eruptionsandlifeintheuniverse 169 11. William Napier. Hazardsfromcometsandasteroids 175 11.1Somethinglikeahugemountain 175 11.2Howoftenarewestruck? 175 11.3Theeffectsofimpact 179 11.4Theroleofdust 181 11.5Groundtruth? 183 12.Arnon Dar. InfluenceofSupernovae,gamma‐raybursts,solarflares,andcosmicraysonthe terrestrialenvironment 187 12.1Introduction 187 12.2Radiationthreats 187 12.2.2Solarflares 190 12.3Cosmicraythreats 194 PARTIII.RISKSFROMUNTINTENDEDCONSEQUENSES 203 13. David Frame and Myles R. Allen. Climatechangeandglobalrisk 203 13.1Introduction 203 13.2Modellingclimatechange 204 13.3Asimplemodelofclimatechange 204 13.5Definingdangerousclimatechange 210 13.6Regionalclimateriskunderanthropogenicchange 211 13.7Climateriskandmitigationpolicy 212 13.8Discussionandconclusions 214 14. Edwin Dennis Kilbourne. Plaguesandpandemics:past,present,andfuture 218 14.1Introduction 218 14.2Thebaseline:thechronicandpersistingburden of infectious disease 218 14.3Thecausationofpandemics 219 14.4Thenatureandsourceoftheparasites 219 14.6Natureofthediseaseimpact:highmorbidity,highmortality,orboth 222 14.11Plaguesofhistoricalnote 225 14.12Contemporaryplaguesandpandemics 226 14.14Discussionandconclusions 228 15.EliezerYudkowsky.ArtificialIntelli genceasapositiveandnegativefactoringlobalrisk 232 15.1Introduction 232 1:Anthropomorphicbias 232 2:Predictionanddesign 235 3:Underestimatingthepowerofintelligence 235 4:Capabilityandmotive 237 5:FriendlyAI 239 6:Technicalfailureandphilosophicalfail ure 240 7:Ratesofintelligenceincrease 243 8:Hardware 247 9:Threatsandpromises 248 10:Localandmajoritarianstrategies 251 11:AIversushumanintelligenceenhancement 254 12:InteractionsofAIwithothertechnologies 257 13:MakingprogressonFriendlyAI 258 Conclusion 260 16.FrankWilczek.Bigtroubles,imaginedandreal 264 16.1Whylookfortrouble? 264 16.2Lookingbeforeleaping 264 16.4Wondering 273 17. Robin Hanson. Catastrophe, Social Collapse, and Human Extinction 276 SocialGrowth 277 SocialCollapse 278 TheDistributionofDisaster 279 ExistentialDisasters 280 PARTIV.Risksfromhostileacts. 287 18.JosephCirincion.Thecontinuingthreatofnuclearwar 288 18.1Introduction 288 18.2CalculatingArmageddon 291 18.3Thecurrentnuclearbalance 296 18.4Thegoodnewsaboutproliferation 299 18.5Acomprehensiveapproach 299 18.6Conclusion 301 19. Gary Ackerman and William С. Potter. Catastrophicnuclearterrorism:apreventableperil 303 19.1Introduction 303 19.2Historicalrecognitionoftheriskofnuclearterrorism 304 19.3Motivationsandcapabilitiesfornuclearterrorism 305 19.5Consequencesofnuclearterrorism 319 19.6Riskassessmentandriskreduction 323 20.AliNounandChristopherF.Chyba.Biotechnologyandbiosecurity 336 20.1Introduction 336 20.2Biologicalweaponsandrisks 337 20.3Biologicalweaponsaredistinctfromotherso‐calledweaponsofmassdestruction 338 20.4Benefitscomewithrisks 339 20.5Biotechnologyrisksgobeyondtraditionalvirology,micro‐andmolecularbiology 341 20.6Addressingbiotechnologyrisks 342 20.7Catastrophicbiologicalattacks 346 20.8Strengtheningdiseasesurveillanceandresponse 348 20.9Towardsabiologicallysecurefuture 351 21. Chris Phoenix and Mike Treder. Nanotechnologyasglobalcatastrophicrisk 357 21.2Molecularmanufacturing 358 21.3Mitigationofmolecularmanufacturingrisks 365 21.4Discussionandconclusion 367 22. Bryan Caplan. Thetotalitarianthreat 371 22.1Totalitarianism:whathappenedandwhyit(mostly)ended 371 22.2Stabletotalitarianism 372 22.3Riskfactorsforstabletotalitarianism 375 22.4Totalitarianriskmanagement 378 Authors'biographies 382  OXFORD UNIVERSITY PRESS Great Clarendon Street, Oxford 0X2 6DP Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide in Oxford New York Auckland Cape Town Dares Salaam Hong Kong Karachi Kuala Lumpur Madrid Melbourne Mexico City NairobiNew Delhi Shanghai Taipei Toronto With offices in Argentina Austria Brazil Chile Czech Republic France Greece Guatemala Hungary Italy Japan Poland Portugal Singapore South Korea Switzerland Thailand Turkey Ukraine Vietnam Oxford is a registered trade mark of Oxford University Press in the UK and in certain other countries Published in the United States by Oxford University Press Inc., New York © Oxford University Press 2008 The moral rights of the authors have been asserted Database right Oxford University Press (maker) First published 2008 All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means,without the prior permission in writing of Oxford University Press,or as expressly permitted by law, or under terms agreed with the appropriatereprographics rights organization. Enquiries concerning reproductionoutside the scope of the above should be sent to the Rights Department, Oxford University Press, at the address above You must not circulate this book in any other binding or cover and you must impose the same condition on any acquirer British Library Cataloguing in Publication Data Data available Library of Congress Cataloging in Publication Data Data available Typeset by Newgen Imaging Systems (P) Ltd., Chennai, India Printed in Great Britain on acid-free paper by CPI Antony Rowe, Chippenham, Wiltshire ISBN 978-0-19-857050-9 (Hbk) 135 79 10 8642 Acknowledgements It is our pleasure to acknowledge the many people and institutions who have in one way or another contributed to the completion of this book. Our home institutions - the Future of Humanity Institute in the James Martin 21st Century School at Oxford University and the Astronomical Observatory of Belgrade - have offered environments conducive to our cross- disciplinary undertaking. Milan wishes to acknowledge the Oxford Colleges Hospitality Scheme and the Open Society Foundation of Belgrade for a pleasant time in Oxford back in 2004 during which this book project was conceived. Nick wishes to thank especially James Martin and Lou Salkind for their visionary support. Physicist and polymath Cosma R. Shalizi gave an entire draft of the book a close, erudite and immensely helpful critical reading. We owe a great debt of gratitude to Alison Jones, Jessica Churchman and Dewi Jackson of Oxford University Press, who took so much interest in the project and helped shepherd it across a range of time scales. We are also appreciative of the scientific assistance by Peter Taylor and Rafaela Hillerbrand and for administrative support by Rachel Woodcock, Miriam Wood and Jo Armitage. We thank John Leslie for stimulating our interest in extreme risk many years ago. We thank Mathew Gaverick, Julian Savulescu, Steve Rayner, Irena Diklic, Slobodan Popovic, Tanja Beric, Ken D. Olum, Istvan Aranyosi, Max Tegmark, Vesna Milosevic-Zdjelar, Toby Ord, Anders Sandberg, Bill Joy, Maja Bulatovic, Alan Robertson, James Hughes, Robert J. Bradbury, Zoran Zivkovic, Michael Vasser, Zoran Knezevic, Ivana Dragicevic, and Susan Rogers for pleasant and useful discussions of issues relevant to this book. Despairing of producing an exhaustive acknowledgement of even our most direct and immediate intellectual debts - which extend beyond science into the humanities and even music, literature, and art - we humbly apologize to all whom we have egregiously neglected. Finally, let all the faults and shortcomings of this study be an impetus for others to do better. We thank in advance those who take up this challenge. [...]... applied to many different risks and used to improve our assessments across the board 2 Global catastrophic risks Beyond these theoretical commonalities, there are also pragmatic reasons for addressing global catastrophic risks as a single field Attention is scarce Mitigation is costly To decide how to allocate effort and resources, we must make comparative judgements If we treat risks singly, and never... quality of life) In this taxonomy, global catastrophic risks occupy the four risks classes in the high-severity upper-right corner of the figure: a global catastrophic risk is of either global or trans-generational scope, and of either endurable or terminal intensity In principle, as suggested in the figure, the axes can be extended to encompass conceptually possible risks that are even more extreme... the extent to which global catastrophic risks can be privately insured is severely limited for reasons having to do with both their scope and their type Although insurance and reinsurance companies have paid relatively scant attention to global catastrophic risks, they have accumulated plenty of experience with smaller risks Some of the concepts and methods used can be applied to risks at any scale... use is a book on global catastrophic risk? The risks under consideration seem to have little in common, so does 'global catastrophic risk' even make sense as a topic? Or is the book that you hold in your hands as ill-conceived and unfocused a project as a volume on 'Gardening, Matrix Algebra, and the History of Byzantium'? We are confident that a comprehensive treatment of global catastrophic risk... about big risks and mitigation strategies The existence of an interdisciplinary community of experts and laypeople knowledgeable about global catastrophic risks will, we believe, improve the odds that good solutions will be found and implemented to the great challenges of the twenty-first century 1.2 Taxonomy and organization Let us look more closely at what would, and would not, count as a global catastrophic. .. 115 References 115 6 Observation selection effects and global catastrophic risks 120 Milan M Cirkovic 6.1 Introduction: anthropic reasoning and global risks 120 6.2 Past-future asymmetry and risk inferences 121 6.2.1 A simplified model 122 6.2.2 Anthropic overconfidence bias 124 6.2.3 Applicability class of risks 126 6.2.4 Additional astrobiological information 128 6.3... 476 21 Nanotechnology as global catastrophic risk 481 Chris Phoenix and Mikg Treder 21.1 Nanoscale technologies 482 21.1.1 Necessary simplicity of products 482 21.1.2 Risks associated with nanoscale technologies 483 21.2 Molecular manufacturing 484 21.2.1 Products of molecular manufacturing 486 21.2.2 Nano-built weaponry 487 21.2.3 Global catastrophic risks 488 21.3 Mitigation... not spending enough on mitigation.7 Posner maintains that the world is, indeed, under-investing in mitigation of several global catastrophic risks 1.4 Part II: Risks from nature Volcanic eruptions in recent historical times have had measurable effects on global climate, causing global cooling by a few tenths of one degree, the effect lasting perhaps a year But as Michael Rampino explains in Chapter... and comets are not the only potential global catastrophic threats from space Other cosmic hazards include global climatic change from fluctuations in solar activity, and very large fluxes from radiation and cosmic rays from supernova explosions or gamma ray bursts These risks are examined in Chapter 12 by Arnon Dar The findings on these risks are favourable: the risks appear to be very small No particular... for causing global disasters The objective risk is probably zero, as believed by most experts But just how confident can we be that there is no objective risk? If we are not certain that there is no objective risk, then there is a risk at least in a subjective sense Such subjective risks can be worthy of serious consideration, and we include them in our definition of global catastrophic risks The distinction . 100 6.Milan M. Cirkovic. Observationselectioneffectsand global catastrophic risks 106 6.1Introduction:anthropicreasoningand global risks 106 6.3DoomsdayArgument 112 6.4Fermi'sparadox. References 115 6 Observation selection effects and global catastrophic risks 120 Milan M. Cirkovic 6.1 Introduction: anthropic reasoning and global risks 120 6.2 Past-future asymmetry and risk. 336 20.2Biologicalweaponsand risks 337 20.3Biologicalweaponsaredistinctfromotherso‐calledweaponsofmassdestruction 338 20.4Benefitscomewith risks 339 20.5Biotechnology risks gobeyondtraditionalvirology,micro‐andmolecularbiology