Bad Data Handbook ppt

[...]... ky file formats Sure, that’s part of the picture, but Bad Data is so much more It includes data that eats up your time, causes you to stay late at the office, drives you to tear out your hair in frustration It’s data that you can’t access, data that you had and then lost, data that’s not the same today as it was yesterday… In short, Bad Data is data that gets in the way There are so many ways to get... whether there is such a thing as truly bad data, in Will the Bad Data Please Stand Up? (Chapter 7) Your data may have problems, and you wouldn’t even know it As Jonathan A Schwabish explains in Subtle Sources of Bias and Error (Chapter 10), how you collect that data determines what will hurt you In Don’t Let the Perfect Be the Enemy of the Good: Is Bad Data Really Bad? (Chap‐ ter 11), Brett J Goldstein’s... stick with this data science bit long enough, you’ll certainly encounter your fair share To that end, we decided to compile Bad Data Handbook, a rogues gallery of data trou‐ blemakers We found 19 people from all reaches of the data arena to talk about how data issues have bitten them, and how they’ve healed In particular: Guidance for Grubby, Hands-on Work You can’t assume that a new dataset is clean... Thank you all! xvi | Preface CHAPTER 1 Setting the Pace: What Is Bad Data? We all say we like data, but we don’t We like getting insight out of data That’s not quite the same as liking the data itself In fact, I dare say that I don’t quite care for data It sounds like I’m not alone It’s tough to nail down a precise definition of Bad Data. ” Some people consider it a purely hands-on, technical phenomenon:... cohort Ken Gleason on Data Quality Analysis Demystified: Knowing When Your Data Is Good Enough (Chapter 19) In this complement to Kevin Fink’s article, we explain how to assess your data s quality, and how to build a structure around a data quality effort Setting the Pace: What Is Bad Data? | 3 CHAPTER 2 Is It Just Me, or Does This Data Smell Funny? Kevin Fink You are given a dataset of unknown provenance... all Marck Vaisman uses The Dark Side of Data Science (Chap‐ ter 15) to document several worst practices that you should avoid Data Policy Sure, you know the methods you used, but do you truly understand how those final figures came to be? Reid Draper’s Data Traceability (Chapter 17) is food for thought for your data processing pipelines Data is particularly bad when it’s in the wrong place: it’s supposed... This Data Smell Funny? (Chapter 2) offers several techniques to take the data for a test drive There’s plenty of data trapped in spreadsheets, a format as prolific as it is incon‐ venient for analysis efforts In Data Intended for Human Consumption, Not Machine Consumption (Chapter 3), Paul Murrell shows off moves to help you extract that data into something more usable 1 If you’re working with text data, ... (Chap‐ ter 11), Brett J Goldstein’s career retrospective explains how dirty data will give your classical statistics training a harsh reality check Data Storage and Infrastructure How you store your data weighs heavily in how you can analyze it Bobby Norton explains how to spot a graph data structure that’s trapped in a relational database in Crouching Table, Hidden Network (Chapter 13) Cloud computing’s... demands of large-scale data analysis, but it’s not without its faults In Myths of Cloud Computing (Chapter 14), Steve Francia dissects some of those assumptions so you don’t have to find out the hard way 2 | Chapter 1: Setting the Pace: What Is Bad Data? We debate using relational databases over NoSQL products, Mongo over Couch, or one Hadoop-based storage over another Tim McNamara’s When Databases Attack:... restaurants, or enjoying good beer Pete Warden is an ex-Apple software engineer, wrote the Big Data Glossary and the Data Source Handbook for O’Reilly, created the open-source projects Data Science Toolkit and OpenHeatMap, and broke the story about Apple’s iPhone location tracking file He’s the CTO and founder of Jetpac, a data- driven social photo iPad app, with over a billion pictures analyzed from 3 million . class="bi x0 y0 w0 h1" alt="" Q. Ethan McCallum Bad Data Handbook ISBN: 978-1-449-32188-8 [LSI] Bad Data Handbook by Q. Ethan McCallum Copyright © 2013. frustration. It’s data that you can’t access, data that you had and then lost, data that’s not the same today as it was yesterday… In short, Bad Data is data that