The global impact of open data

Strata The Global Impact of Open Data Key Findings from Detailed Case Studies Around the World Andrew Young and Stefaan Verhulst Supported by: The Global Impact of Open Data by Andrew Young and Stefaan Verhulst Copyright © 2016 O’Reilly Media Inc All rights reserved Printed in the United States of America Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472 O’Reilly books may be purchased for educational, business, or sales promotional use Online editions are also available for most titles (http://safaribooksonline.com) For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com Editor: Shannon Cutt Production Editor: Nicholas Adams Copyeditor: Octal Publishing, Inc Interior Designer: David Futato Cover Designer: Randy Comer Illustrator: Rebecca Demarest September 2016: First Edition Revision History for the First Edition 2016-09-13: First Release The O’Reilly logo is a registered trademark of O’Reilly Media, Inc The Global Impact of Open Data, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work Use of the information and instructions contained in this work is at your own risk If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights 978-1-491-96467-5 [LSI] Acknowledgements The authors would like to thank Omdiyar Network for supporting this project In particular, Laura Bacon, principal of policy investments, has been instrumental in developing the case studies This work would not have been possible without the tireless efforts of our colleagues at the GovLab, including the case studies research team: Ali Clare, Auralice Graft, Juliet McMurren, Christina Rogawski, David Sangokoya, and Rebecca Young; Akash Kapur who provided crucial editorial support; and the GovLab developer team: Chris Wong, Claudio Mendonca, Batu Sayici, and Mark Adkins-Hastings who created odimpact.org where this research lives online Thanks also to the GovLab’s Beth Simone Noveck, Fred DeJohn, Lauren Yu, Dinorah Cantú, Julia Root, Irene Tello-Arista, and María Hermosilla for their input and support throughout the development of this research The research contained in this volume was developed thanks to the following experts and stakeholders who generously gave their time and provided essential on-the-ground knowledge about their inspiring and transformative work: Neil Ackroyd, Kim Alexander, Robert Andrews, Jay Bhalla, Uuf Brajawidagba, Franỗois Brouard, Iain Campion, John Carpenter, Daniel Carranza, Julian Carver, Pablo Clark, Izabela Corrèa, Jeff de la Beaujardiere, Leodelma de Marilac Felix, Vivien Deparday, Patrick DuFour, Ee-Peng Lim, Carl Elmstam, Peter Elson, Felipe Estefan, Aidan Eyakuze, Stephen Ferris, Christian Fischer, Mike Flowers, Rafael García Aceves, Zachary Goldstein, Hanna Helquist, Thorhildur Jetzek, Al Kags, Jason Kim, Verena Luise Knippel, Daniel Kreiss, David Lasby, Michael Lenczner, Morten Lind, Andrew Loveless, Lindsay Marchessault, Arnold Minde, Lindsay Mollineaux, Oscar Montiel, Otávio Moreira de Castro Neves, Ainun Najib, Bitange Ndemo, Bo Overgaard, Tim Owen, Allan Parnell, Ed Parsons, Maria Patterson, Hilary Pearson, Karl Peterson, Florent Peyre, Peter Rabley, Tara Ramchandani, Ira Rubinstein, Fabrizio Scrollini, Diah Setiawaty, Rupert Simons, Singapore National Environment Agency, Gabriel Sipos, Diego Soria, Allison Soussi-Tanani, Javier Teran, Samhir Vasdev, Eva Vozárová, Neil Weston, Alyssa Wright, and Alexandra Zapata Hojel Finally, our gratitude to the following dedicated individuals who provided invaluable input during the open peer review process of this research: Karin Ahlin, Antonio Almansa Morales, Andi Argast, Jos Berens, Keitha Booth, J Albert Bowden II, Mark Cardwell, Corinne Cath, Emmy Chirchir, Rafael García Aceves, Erik Holmlund, Brendan Kenny, Alessia Lefebure, Ulrich Mans, Valerie Moye, Alina Östling, Giuseppe Reale, Fathima Rifaa, Julian Singh, Rayna Stamboliyska, Julian Tait, Jamie Van Ymeren, Mario Velasco, Niki Virani, Johanna Walker, Andrew Weller, Ian White, Raymond Yee, and Maria Zuffova Executive Summary Recent years have witnessed considerable enthusiasm over open data Several studies have documented its potential to spur economic innovation and social transformation as well as to usher in fresh forms of political and government accountability Yet for all the enthusiasm, we know little about how open data actually works and what forms of impact it is really having This report seeks to remedy that informational shortcoming Supported by Omidyar Network, the GovLab has conducted 19 detailed case studies of open-data projects around the world The case studies were selected for their sectoral and geographic representativeness They were built in part from secondary sources (“desk research”), but also from a number of first-hand interviews with important players and key stakeholders In this report, we consider some overarching lessons that we can learn from the case studies and assemble them within an analytical framework that can help us better understand what works — and what doesn’t — when it comes to open data The report begins (“I What Is Open Data?”) with an overview of open data Like many technical terms, open data is a contested and dynamic concept The GovLab has conducted a study of nine widely used definitions to arrive at the following working definition, which guides our discussion here: Open data is publicly available data that can be universally and readily accessed, used, and redistributed free of charge It is structured for usability and computability “II The Case Studies” includes a brief summary of our 19 case studies, each of which is detailed at considerably greater length, in Parts II through V Sections III through V represent the core of our analytical framework; they identify the key parameters and variables that determine the impact of open data “III What Is the Impact of Open Data on People’s Lives?” discusses what we have identified as the four most important dimensions of impact Based on the case studies, GovLab has determined that open data projects are improving government, primarily by making government more accountable and efficient; empowering citizens, by facilitating more informed decisionmaking and enabling new forms of social mobilization; creating new economic opportunities; and helping policymakers and others find solutions to big, previously intractable public problems (e.g., related to public health or global warming) These types of effects cannot be taken for granted They are evident to varying degrees across our case studies, and sometimes not at all Our research also identified four enabling conditions that allow the potential of open data to manifest (“IV What Are the Enabling Conditions that Significantly Enhance the Impact of Open Data?”) Overall, we found that open data projects work best when they are based on partnerships and collaborations among various (often intersectoral) organizations; when they emerge within what we call an “open data public infrastructure” that enables the regular release of potentially impactful data; when they are accompanied by clear open data policies, including performance metrics; and when they address or attempt to solve a well-defined problem or issue that is an obvious priority to citizens and likely beneficiaries “V What Are the Challenges to Open Data Making an Impact?” identifies the key challenges that open data projects face These include a lack of readiness, especially evident in the form of low technical and human capacity in societies or nations hosting open data initiatives; projects that are unresponsive — and thus inflexible — to user or citizen needs; projects that result in inadequate protections for privacy or security; and, finally, projects that suffer from a shortage of resources, financial and otherwise None of the 19 initiatives we studied was immune to these obstacles; the most successful ones had found ways to surmount them and build applications or platforms that were nonetheless able to tap into the potential of open data “VI Recommendations: Toward a Next Generation Open-Data Roadmap” features a set of 10 recommendations directed at policymakers, entrepreneurs, activists, and others contemplating open-data projects Each of these broad recommendations is accompanied by more specific and concrete steps for Data Privacy Usability and Citizen Engagement Data Licensing Looking Forward Usability and functionality Forward-Planning Data Collaboration with Other Partners Open Contracting and Procurement in Slovakia: Establishing Trust in Government Context and Background Impact Intended Beneficiaries Engagement and Use Corruption and Perceptions of Corruption Citizen Awareness and Oversight Regional Diffusion Challenges Data Quality Cost (and Perceptions of Cost) Enforcement and Accountability (The Human Factor) Looking Forward Cross-Sectoral Collaboration Around Data Quality Training and Identification of Best Practices Monitoring Compliance Indonesia’s Kawal Pemilu: Elections: Free, Fair, and Open Data Context and Background Corruption in Indonesia Open Data and Indonesian Elections Project Description and Inception Impact Settling the 2014 Election Results Increased Trust and Greater Public Participation Reduced Polarization Challenges Legal Changes Security Reliability of Information Lasting Political Impact Looking Forward Kawal APBD Mata Massa (“Eye of the Masses”) Kawal Pilkada Denmark’s Open Address Data Set: Consolidating and Freeing-up Address Data Context and Background Project Description and Inception Impact Direct Financial Benefits Nonfinancial Benefits Benefits to Developer Community The Danish Basic Data Program Challenges Data Quality Governance of the Data Project Looking Forward Expected Benefits of the Basic Data Program Addressing Capacity Issues Opening Canada’s T3010 Charity Information Return Data: Accountability of Charities through Open Data Context and Background Project Description and Inception Impact Benefits to Charities Enabling New Streams of Research Creating a New Business Asset Financial Benefits for Government and Identification of Tax Evasion Challenges Mismatch Between Expectations and Reality Data Quality: Transcription and Reporting Errors Definitional Challenges Shifting Government Priorities Looking Forward Improving the Usability of the Data IT Support for Nonprofits Nonprofit Data Collaboratives III Case Studies: Empowering Citizens Open Education Information in Tanzania: A Tale of Two Dashboards Context and Background Product Description and Inception Impact Engagement and use Data Quality and Diversity Impact on Other Data Projects Challenges Lack of Internet Penetration and Use Fragmentation Looking Forward Improving User Engagement through Expanded Functionality Education of Infomediaries Kenya’s Open Duka: Open Data for Transactional Transparency Context and Background Corruption in Kenya Open Data and Technology in Kenya Project Description and Inception Data Sources Impact Seeding the Notion of Openness Challenges Government Devolution Data Quality and Availability Public Trust and Expectations Looking Forward Open County Program Tracking Metrics of Success Making the Platform More Accessible Expansion to Other Countries 10 Mexico’s Mejora Tu Escuela: Empowering Citizens to Make Data-Driven Context and Background Education in Mexico Information and Open Data in Mexico Product Description and Inception Project Description Get to Know Your School and Compare Your School School Grades Impact Impact on Beneficiaries Increased Accountability Corruption Report Challenges Standardized Test Cancellation Data Accessibility Data Quality Looking Forward Connecting Needs with Actions Publishing a School Budget Developing a Report Pipeline Translating Information into Action 11 Uruguay’s A Tu Servicio: Empowering Citizens to Make DataDriven Context and Background Product Description and Inception Impact Intended Beneficiaries Use and Awareness: Data Quality: Impact on Other Open Data Projects: Challenges Timing constraints Outreach and communication Looking Forward IV Case Studies: Creating Opportunity 12 Great Britain’s Ordnance Survey: A Clash of Business Models Context and Background OS Background Importance of Geospatial Data Open Data in the United Kingdom Project Description and Inception OS Today OS OpenData Impact Economic Impact OS Minecraft GeoVation Program Other Impacts Challenges Cost Addressing Open Data Principles Private Sector Competition Measuring Impact and User Demands Looking Forward Evolving GeoVation Hub OS OpenData Additions Looking Beyond the U.K Supporting Smart Cities 13 United States’ New York City Business Atlas: Leveling the Playing Field Context and Background Project Description and Inception User-Centered Design and Partnering with Department of Small Business Services Impact Leveling the Playing Field for Market Research Enabling Business Improvement District (BID) Analysis Influencing Data Analytics Innovation in New York and Abroad Challenges Communicating Opportunity Addressing Technical Challenges Improving Granularity of Data Looking Forward Business Atlas 2.0: New Tools and Features Partnerships and Collaborations 14 United States’ NOAA: Opening Up Global Weather Data in Collaboration with Businesses: Creating a New Industry Through Access to Weather Data Context and Background Project Description and Inception NOAA Partnerships31,32,33,34 Impact Economic Benefits Challenges Scaling Data Security Understanding User Needs Data Navigation and User Experience Data Awareness Looking Forward 15 United States Opening GPS for Civilian Use: Creating a Global Public Utility Context and Background GPS in Our Lives Today Early History of Satellite Navigation: Sputnik Early History of Satellite Navigation: Transit Project Description and Inception The Arrival of GPS Who Deserves the Credit? GPS Today The Nature of GPS Data How GPS Became Open Continuously Operating Reference Station (CORS) Impact Economic Impact Precise Timing Smartphones / Mobile Navigation Public Safety and Emergencies CORS Impacts Challenges Governance Challenges Funding and U.S Policymaking Privacy Concerns GPS “Jamming” / Hacking / Security CORS Challenges Looking Forward Precision Farming Indoor Navigation Intelligent Transportation Systems and Driverless Cars Terrestrial Backups Evolving the CORS System V Case Studies: Solving Public Problems 16 Battling Ebola in Sierra Leone: Data Sharing to Improve Crisis Response Context and Background Open Data in Sierra Leone The 2014 Ebola Outbreak Project Description and Inception Using Data in Combating Ebola National Ebola Response Centre (NERC) Humanitarian Data Exchange (HDX) The Ebola GeoNode Impact Providing Evidence to Decision-makers The Impact of Maps Spurring Wider Use of Open Data in International Organizations Internal Capacity Building and Identifying Best Practices Challenges Data Management Collaborating with Government Lack of Existing Structures and Best Practice Looking Forward HDX: Enabling New Insights into the Spread of Ebola Next Steps for GeoNode The Closure of NERC 17 New Zealand’s Christchurch Earthquake Clusters: Open Data for Improving Emergency Response Context and Background Open Data in New Zealand Project Description and Inception Canterbury Recovery Map (Eq.org.nz) GIS and trusted data sharing Landcheck and MyProperty Forward Works Viewer Building Our Footprints Impact Higher-Quality Geospatial Data Facilitating Collaboration Impact on Other Data Projects Challenges Data Infrastructure Challenges Lack of a Property Framework Collaborations and Sharing Challenges Looking Forward Advancing Data-Driven Innovation in New Zealand 18 Singapore’s Dengue Cluster Map: Open Data for Public Health Context and Background Dengue Fever Dengue Fever in Singapore Open Data in Singapore Project Description and Inception Impact Average Citizens Media Developer Community Challenges Looking Forward 19 United States’ Eightmaps: The Unintended Negative Consequences Context and Background Legal Foundation of Campaign Finance Data Disclosure Laws in California Electronic Filing Privacy Concerns of Electronic Filing California’s Proposition Project Description and Inception Impact Harassment of Proposition Donors Threats to Free Speech Activism Around Gay Rights Transparency vs Privacy Challenges Looking Forward Raising the Itemized Disclosure Threshold Placing New Safeguards on Downloads of Donor Information Better Transparency About How Data Is Used by Political Campaigns and Other Actors 20 Kennedy v City of Zanesville, United States: Open Data as Evidence Context and Background Case Description and Inception Impact Challenges Security Concerns Awareness and Usability Data — the Human Factor Looking Forward A Appendix Open Data Definitions — What’s in a Name? ...Strata The Global Impact of Open Data Key Findings from Detailed Case Studies Around the World Andrew Young and Stefaan Verhulst Supported by: The Global Impact of Open Data by Andrew... framework; they identify the key parameters and variables that determine the impact of open data “III What Is the Impact of Open Data on People’s Lives?” discusses what we have identified as the four... allow the potential of open data to manifest (“IV What Are the Enabling Conditions that Significantly Enhance the Impact of Open Data? ”) Overall, we found that open data projects work best when they