About This E-Book EPUB is an open, industry-standard format for e-books However, support for EPUB and its many features varies across reading devices and applications Use your device or app settings to customize the presentation to your liking Settings that you can customize often include font, font size, single or double column, landscape or portrait mode, and figures that you can click or tap to enlarge For additional information about the settings and features on your reading device or app, visit the device manufacturer’s Web site Many titles include programming code or configuration examples To optimize the presentation of these elements, view the e-book in single-column, landscape mode and adjust the font size to the smallest setting In addition to presenting code and configurations in the reflowable text format, we have included images of the code that mimic the presentation found in the print book; therefore, where the reflowable format may compromise the presentation of the code listing, you will see a “Click here to view code image” link Click the link to view the print-fidelity code image To return to the previous page viewed, click the Back button on your device or app Data Visualization Toolkit Using JavaScript, Rails™, and Postgres to Present Data and Geospatial Information Barrett Clark Boston • Columbus • Indianapolis • New York • San Francisco • Amsterdam • Cape Town Dubai • London • Madrid • Milan • Munich • Paris • Montreal • Toronto • Delhi • Mexico City São Paulo • Sydney • Hong Kong • Seoul • Singapore • Taipei • Tokyo Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks Where those designations appear in this book, and the publisher was aware of a trademark claim, the designations have been printed with initial capital letters or in all capitals The author and publisher have taken care in the preparation of this book, but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions No liability is assumed for incidental or consequential damages in connection with or arising out of the use of the information or programs contained herein For information about buying this title in bulk quantities, or for special sales opportunities (which may include electronic versions; custom cover designs; and content particular to your business, training goals, marketing focus, or branding interests), please contact our corporate sales department at corpsales@pearsoned.com or (800) 382-3419 For government sales inquiries, please contact governmentsales@pearsoned.com For questions about sales outside the U.S., please contact intlcs@pearson.com Visit us on the Web: informit.com/aw Library of Congress Control Number: 2016944665 Copyright © 2017 Pearson Education, Inc All rights reserved Printed in the United States of America This publication is protected by copyright, and permission must be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system, or transmission in any form or by any means, electronic, mechanical, photocopying, recording, or likewise For information regarding permissions, request forms and the appropriate contacts within the Pearson Education Global Rights & Permissions Department, please visit www.pearsoned.com/permissions/ ISBN-13: 978-0-13-446443-5 ISBN-10: 0-13-446443-5 Text printed in the United States on recycled paper at RR Donnelley in Crawfordsville, Indiana 1 16 To my children Never stop exploring and asking questions (even when you wear me out) Contents Foreword Preface Acknowledgments About the Author Part I ActiveRecord and D3 Chapter 1 D3 and Rails Your Toolbox—A Three-Ring Circus Database Application Server Graphing Library Maryland Residential Sales App Evaluating Data Data Fields Simple Pie Chart Summary Chapter 2 Transforming Data with ActiveRecord and D3 Pie Chart Revisited Legible Labels Mouseover Effects You Can Function Bar Chart New Views, New Routes Bar Chart Controller Actions Bar Chart JavaScript Scatter Plot Scatter Plot? Scatter Plot Controller Actions Scatter Plot Views and Routes Scatter Plot JavaScript Scatter Plot Revisited Box Plot Quartiles Boxes, Whiskers, Circles, What?! Box Plot Data and Views Box Plot JavaScript Summary Chapter 3 Working with Time Series Data Historic Daily Weather Data Weather Rails App Weather Readings Model Weather Readings Import Weather Stations Model Weather Stations Import Simple Line Graph Weather Controller Fetch the Data The View Files Draw the Line Graph Tweak 1: Simple Multiline Graph Tweak 2: Add Circle to Highlight the Maximum Temperature Tweak 3: Add Circle to Highlight the Minimum Temperature Tweak 4: Add Text to Display the Temperature Change Tweak 5: Add a Line Between the Focus Circles Summary Chapter 4 Working with Large Datasets Git and Large Files The Cloud Hotlinking Benchmarking Benchmark and Compare Benchmark All the Things Querying “Big Data” Using Scopes in the Model Adding Indices When Benchmarks and Statistics Lie Summary Part II Using SQL in Rails Chapter 5 Window Functions, Subqueries, and Common Table Expression Why Use SQL? Database Portability Is a Lie Tripping Over ActiveRecord User-Defined Functions Why? Heresy! How? How to Use SQL in Rails Scatter Plot with Mortgage Payment Window Functions Window Functions Greatest Hits Lead and Lag Partitions First Value and Last Value Row Number Using Subqueries Common Table Expression CTE and the Heatmap The Query The Controller and View The JavaScript Summary Chapter 6 The Chord Diagram The Matrix Is the Truth Flight Departures Data Departures App Airports Carriers Departures Transforming the Data Fetching the Data Generating the Matrix Finalizing the Matrix Create the Views Departures Controller and Routes Departures View Departures Style Draw the Chord Diagram Disjointed City Pairs Using the Lead Window Function to Find Empty Leg Flights Optimizing Slow Queries with the Materialized View Draw the Disjointed City Pairs Chord Diagram Summary Chapter 7 Time Series Aggregates in Postgres Finding Flight Segments Creating a Series of Time Turning Data into Time Series Data Graphing the Timeline Basic Timeline Fancy Timeline Summary Chapter 8 Using a Separate Reporting Database Transactional versus Reporting Databases Worker Processes Postgres Schemas Working with Multiple Schemas in Rails Defining the Schema Connection Creating a New Schema Creating Objects in the Reporting Schema Materialized View in the Reporting Schema Tables in the Reporting Schema Summary Part III Geospatial Rails Chapter 9 Working with Geospatial Data in Rails GIS Primer It’s (Longitude, Latitude) Not (Latitude, Longitude) Decimal Degrees Degrees, Minutes, Seconds (DMS) Datum Map Projection Spatial Reference System Identifier (SRID) Three Feature Types PostGIS Postgres Contrib Modules Installing PostGIS PostGIS Functions ActiveRecord and PostGIS ActiveRecord PostGIS Adapter Rails PostGIS Configuration PostGIS Hosting Considerations Using Geospatial Data in Rails Creating Geospatial Table Fields Latitude and Longitude Simple GIS Calculation Working with Shapefiles Shapefile Import Schema Importing from a Shapefile Shapefile ETL Update Missing lonlat Data Summary Chapter 10 Making Maps with Leaflet and Rails Leaflet Map Tiles Map Layers Incorporating Leaflet into Rails to Visualize Weather Stations Using a Separate Rails Layout for the Map Map Controller Map Index Map Data GeoJSON View Mapping the Weather Stations Visualizing Airports Markers Marker Cluster Drawing Flight Paths Visualizing Zip Codes Updating the Maryland Residential Sales App for PostGIS Zip Code Geographies Importing the Zip Code Shapefile Mapping Zip Codes ... Click the link to view the print-fidelity code image To return to the previous page viewed, click the Back button on your device or app Data Visualization Toolkit Using JavaScript, Rails™, and Postgres to Present Data and. .. identify and analyze that data, and be able to quickly see what it says and I want to show you to how to do that too Before we jump into all of that, let’s first take a step back and look at the various moving parts in a... • Chapter 7: Time-Series Aggregates in Postgres In this chapter we take the flight departure data and convert it from transactional to time-series data to build a timeline diagram • Chapter 8: Using a Separate Reporting Database—This chapter discusses how to use a separate database or database schema for a reporting database