Co m pl im en ts of Full Stack Web Performance Tom Barker Secure and Accelerate Your Website Incapsula helps you take care of business by simplifying ops and protecting your web apps Our PCI-certified and SOC compliant cloud service is easy to deploy, intelligent and scalable We secure websites from top web threats like SQL injections, XSS and web scraping so your customers can go about their business with confidence Find out more about what Incapsula can for your business https://www.incapsula.com/web-application-security/ Full Stack Web Performance Tom Barker Beijing Boston Farnham Sebastopol Tokyo Full Stack Web Performance by Tom Barker Copyright © 2017 O’Reilly Media, Inc All rights reserved Printed in the United States of America Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472 O’Reilly books may be purchased for educational, business, or sales promotional use Online editions are also available for most titles (http://oreilly.com/safari) For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com Editor: Meg Foley Production Editor: Shiny Kalapurakkel Copyeditor: Octal Publishing, Inc July 2017: Interior Designer: David Futato Cover Designer: Karen Montgomery Illustrator: Rebecca Demarest First Edition Revision History for the First Edition 2017-06-16: First Release The O’Reilly logo is a registered trademark of O’Reilly Media, Inc Full Stack Web Performance, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limi‐ tation responsibility for damages resulting from the use of or reliance on this work Use of the information and instructions contained in this work is at your own risk If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsi‐ bility to ensure that your use thereof complies with such licenses and/or rights 978-1-491-98844-2 [LSI] Table of Contents Introduction v Client-Side Use a Speed Test Now Integrate into Continuous Integration Use Log Introspection for Real User Monitoring Tell People! Summary Accomplishing Web Performance Wins via Infrastructure 11 Using a CDN Edge Caching: Serving Your Application as Close as Possible to Your User Make Requests to the Fastest Possible Origin Using a Cloud Provider Summary 11 12 13 14 17 Operationalize Performance 19 Setting Up an APM Using an APM to Troubleshoot Performance Issues Summary 19 20 25 Next Steps 27 Get Synthetic Web Performance Results Trial a CDN for Free Trial an Application Performance Management Tool for Free Embrace Full-Stack Development and DevOps 27 27 28 28 iii Introduction We are in the midst of another tidal change in the software engi‐ neering and IT industries This has been going on for a number of years already, but like the frog in the pot that doesn’t notice the water slowly beginning to boil around him, some of us might not have noticed the transitions in our environment We’ve overlooked these transitions because there have been many smaller ones that we just adjusted to, that accumulated to be a big significant change Or maybe it’s just that the ideas behind these changes have been talked about for quite some time, but it’s only relatively recently that they have coalesced into actionable patterns that are easy to implement and reproduce I was reminded of this recently because some of my teams—specifi‐ cally those that are working on products that we made three or more years ago—are migrating their products from physical datacenters to cloud platforms Think about that for a minute: three or four years ago we were start‐ ing new projects first by requesting nodes—in some cases, virtual machines (VMs) on a hypervisor, and in other cases actual physical boxes—and IP blocks, waiting days, or in most cases, even weeks for the boxes to be configured Today, of course, we run a script and our cloud platform of choice spins up nodes preconfigured with the image that we want nearly instantaneously Our notions of web performance and capacity planning, too, have changed Now if we need to scale a cloud web-native application to handle spikes in usage, it’s a matter of only selecting a checkbox and paying for the scale that we need There are also new challenges in the field that we are just now discovering, and coming up with v workarounds for, like the appropriate use of availability zones (think about the Amazon Web Services outage of 2017 that brought down a significant portion of the web) or even how to use multiple cloud service providers to serve a single property Even our organizational identities are changing If someone is a web developer, why can’t they learn to request and configure VMs from their cloud provider? And if they are setting up the machines on which their code resides, and maybe even the firewall rules, why not then set up their own log consumption flow, then at that point are they still just a web developer, or are they maybe a full-stack devel‐ oper or even a DevOps engineer? These are just a few examples of how a concept like DevOps has changed the day-to-day activities of our work routines Who This Book Is For DevOps encapsulates different things to different groups Some interpret it as an integrating of traditional infrastructure or Ops roles into a development team, whereas others bundle security in there and call it DevSecOps Maybe some groups have different interpretations of what Ops means, defining it as incorporating not just infrastructure but also production support roles As such, this book is for anyone who needs to think about and deal with performance in a DevOps environment From web developer, to DevOps engineer, to engineering manager and architect, this book is intended for you This book has set out to address how web performance fits into this modern landscape of the all-encompassing cross-functional DevOps team You’ll find the topics organized into three high-level areas of focus in a product development group: Client-side This is the user-facing piece of the application It will generally run on the user’s hardware Infrastructure This consists of the facilitating pieces of your application, specifically the content delivery network (CDN) and cloud ser‐ vice vi | Introduction Operations These are the practices you put in place to monitor and alert on the health of your applications This book also presents significant but quick wins throughout That is a relative term, but the way that I have approached this is to take advantage of existing tools and libraries that are fast to integrate with but have huge payoffs Depending on your architecture and team makeup, the level of effort for each of these solutions or rec‐ ommendations should be measured only in days and weeks, not months for the most part (at least from my reckoning; your mileage may vary) Introduction | vii Figure 2-3 A basic cloud infrastructure is not so different than a clas‐ sic infrastructure You can set up all of this and run it either from a GUI or automate it via command line Automate Scaling to Accommodate Spikes in Traffic This is fantastic on its own, but most cloud providers also provide for elastic scaling capabilities With elastic scaling enabled, the cloud platform will spin up new nodes to accommodate increased load, and then spin them down when they are no longer needed Each cloud provider is different, but this is generally achieved by establishing elastic scaling groups that define the set of nodes to increase and decrease as needed The scaling functionality monitors these nodes and, based on criteria you can define, will expand and contract based on the thresholds that you set Figure 2-4 provides a visual example 16 | Chapter 2: Accomplishing Web Performance Wins via Infrastructure Figure 2-4 Availability zone As the world learned in the great AWS Outage of 2017,1 even cloud providers go down When AWS went down, sites that relied solely on the availability zone that failed experienced severe outages themselves When using a cloud provider, at a minimum, you should use several availability zones and regions to minimize impacts of downtime in any one availability zone An even better idea is to take advantage of the functionality of a GTM from your CDN to route traffic between several different cloud providers to maximize potential uptime Summary This chapter looked at infrastructural performance optimizations that you can implement We looked at some of the easy wins that you can derive from using a CDN, serving cached content at the edge, and routing to the best possible origin We talked about using a cloud service provider to create an infra‐ structure that could expand and contract as needed to avoid performance-killing bottlenecks In Chapter 3, we look at tools to help operationalize our perfor‐ mance Amazon S3 experienced a large-scale outage on February 28, 2017 that affected all users connected to the AWS US-EAST-1 Region For more information on this see, go to https://aws.amazon.com/message/41926/ Summary | 17 CHAPTER Operationalize Performance Your site is out in production; performance is where you want it; everything looks great You think But how you quantify what the actual experience is out in the wild? How your machines are performing with real users, using their own devices connected via various networks, each of varied quality? Even more important, how you identify, triage, and debug an issue in production that is affecting actual customers? You use an application performance management (APM) tool There are many such tools out in the market; some of the more popular are New Relic, AppDynamics, and Dynatrace Setting Up an APM Setting up an APM is relatively painless Generally, you just install an APM agent on the machines that you want monitored The agents capture metrics for the machines on which they are installed and communicate those to the hosted APM platform The APM platform processes the data and makes it available via dashboards Figure 3-1 presents a diagram of that architecture 19 Figure 3-1 The APM platform processes the data and makes it avail‐ able via dashboards Using an APM to Troubleshoot Performance Issues Picture this: you are sitting at your desk when you get a call from one of your stakeholders They are hearing customer complaints; users are trying to access your site but are experiencing a lot of latency Luckily, your site is already being monitored by an APM, so you just fire up your dashboard and look for the time period in question Figure 3-2 shows an example of a dashboard from New Relic Figure 3-2 Dashboard from New Relic 20 | Chapter 3: Operationalize Performance The subsections that follow discuss some of the key metrics that our APM should expose Throughput Throughput is a measurement of requests over time, usually either per second or per minute This lets you see if traffic has suddenly dropped off or spiked Even more useful, we can use this to measure throughput by node to validate traffic shaping or identify potentially unhealthy nodes In our example, if there were a widespread issue, we would probably see throughput dropping as users abandon our site Or, at a granular level, there could be nodes that are receiving more requests than others, causing requests to slow down This is where we would be able to see this Errors The APM agent will track application errors that can lead to bad HTTP responses and from those, craft an error rate for your appli‐ cation Error rate is literally the number of successful requests divi‐ ded by the number of failed requests Figure 3-3 depicts the error screen Figure 3-3 Error rate With knowledge of your error rate, you can craft alerts based on thresholds around this rate to know whether issues are ticking up in production and if you are on the verge of an incident occurring Even better, if your APM supports the language and runtime that you are using, you also will be able to dig deeper into those errors Using an APM to Troubleshoot Performance Issues | 21 and get stack traces from the actual function or classes that threw the error to help debug and fix it Figure 3-4 demonstrates a stack trace of an error in New Relic This is an important point: not assume that support for what you need is there, some of my teams had to use JRuby for quite some time because the APM that our company had a contract with did not support Ruby, only Java or Microsoft NET, so we used JRuby to at least get some metrics from the Java Virtual Machine Figure 3-4 Stack trace of an error For our example, if the latency that our users are experiencing is due to errors or is causing HTTP 504 messages, it would be evident here At the very least, we could see what requests are timing out, or ide‐ ally, if there are actual errors that we could begin to debug the stack trace Most Expensive Transactions We can look here to see what, if any, transactions are suddenly tak‐ ing much longer to respond If a backend service is having issues and our calls to them are timing out or just taking much longer to resolve, this is where this problem would become evident 22 | Chapter 3: Operationalize Performance Even if we currently are not in fire-drill mode, we can use this fea‐ ture to proactively drive to better overall performance If we know what transactions take the most time, we can focus on those and try to lower their performance cost Node Health If you have too much traffic on a single node, you will likely see cer‐ tain things happening to that node You could see your CPU usage spiking as the machine struggles to keep up You could see your memory usage running high You could see HTTP requests begin to be turned away, causing the node’s throughput to drop All of these will make your user experience come to a crawl and eventually just error out The APM agent will track the health of the machine that it is installed on and include those metrics to the dashboard Figure 3-5 shows an example of node health in a New Relic dashboard Using an APM to Troubleshoot Performance Issues | 23 Figure 3-5 Node health in the New Relic dashboard Third-Party Service-Level Agreements If you have done your due diligence but are still experiencing high page load times, more often than not the root cause of the slowdown will be slow responses from the third-party or partner services that you call Maybe there is an API from another internal team that you call to load user data, or there are parts of your site dependent on an API to process user input An APM tool allows you to track the response times of your exter‐ nal APIs so that you can sort them by most time-consuming Figure 3-6 gives an example of third-party API tracking 24 | Chapter 3: Operationalize Performance Figure 3-6 Third-party API tracking Having that kind of data allows you to follow-up with these partners and have conversations about their own performance Ideally, they would have a previously agreed upon SLA governing the perfor‐ mance of their API Summary Using an APM is critical in not just maintaining full-stack perfor‐ mance, but for debugging performance issues in production, as well Most important, between APMs and cloud providers, development teams are being empowered to take operational ownership of their products Summary | 25 CHAPTER Next Steps We’ve talked at a high level about what sorts of performance wins you can achieve when taking a holistic, full-stack view of your web applications So, what are some tactical next steps you can take? Get Synthetic Web Performance Results As we talked about in Chapter 1, run your site through a web per‐ formance test like WebPageTest It is free, gives you an idea of your current standing, and gives you steps to take to remediate the issues found Here are some other web performance tests: • YSlow • Pingdom • Google’s PageSpeed Insights Trial a CDN for Free We talked at length about some of the benefits we can get from using a CDN, but signing up with a CDN can be daunting and will involve cost An easy first step on that path is to set up a free trial account with a CDN Most of the popular CDNs have a free trial available: 27 • Akamai, available at https://www.akamai.com/us/en/campaign/ get-akamaized.jsp • Incapsula, available at https://www.incapsula.com/pricing-andplans.html • MaxCDN, available at https://www.maxcdn.com/test-account/ With your free trial set up, benchmark your site behind a CDN and compare the performance numbers to your current setup If you are really feeling daring, point some traffic to it and see what benefits you can get infrastructurally (are your machines less taxed, can you quantify how many fewer nodes you would need to maintain?) Can you quantify these benefits and use them to justify the budget request to actually make the plunge? Trial an Application Performance Management Tool for Free Just like CDNs, application performance management (APM) com‐ panies are more than happy to give you a free trial to test out their products Here are s of the notable ones: • New Relic, available at https://newrelic.com/signup • Dynatrace, available at https://www.dynatrace.com/trial/ • AppDynamics, available at https://www.appdynamics.com/freetrial/ Install an agent and check out the dashboards for your application Most APMs are so feature-rich that you’ll most likely find that the company you are trialing is happy to walk you through their feature sets Some even offer extensive training In the past, I have even had the company representative offer to help me debug a production issue as a way of a capabilities demo Embrace Full-Stack Development and DevOps The most important next step of all is to embrace the idea of fullstack development and DevOps I can still remember the days of needing to reach out to an opera‐ tions team when something would go wrong in production because 28 | Chapter 4: Next Steps only they could get me a snippet of the logs And it would be a flat file that I would need to grep through to search for things that I had learned to look for; things like specific error codes or HTTP respon‐ ses I remember needing to factor hardware into my budget in the beginning of the year for projects that had not yet been scoped or even envisioned yet, and then waiting months for machines to be ordered, shipped, and set up at the datacenter And, if I had guessed wrong, how was I going to scale up in time to meet the demand? The advances of platform and infrastructure as a service have brought the power of operations on demand, if we just embrace it with open arms Embrace Full-Stack Development and DevOps | 29 About the Author Tom Barker is a software engineer, engineering manager, professor, and author Currently, he is director of Software Engineering and Development at Comcast, and an adjunct professor at Philadelphia University ... your business https://www.incapsula.com /web- application-security/ Full Stack Web Performance Tom Barker Beijing Boston Farnham Sebastopol Tokyo Full Stack Web Performance by Tom Barker Copyright... like YSlow, or free web applications like WebPageTest, to full enterprise solutions My personal favorite web performance testing tool is WebPageTest.1 You can use the hosted site as is, or, if... is a huge body of written work around improving web perfor‐ mance, beginning with Steve Souders’ seminal book, High Perfor‐ mance Web Sites, but the landscape of client-side performance changes