Co m pl im en ts of Getting Started with Varnish Cache ACCELERATE YOUR WEB APPLICATIONS Thijs Feryn Getting Started with Varnish Cache Accelerate Your Web Applications Thijs Feryn Beijing Boston Farnham Sebastopol Tokyo Getting Started with Varnish Cache by Thijs Feryn Copyright © 2017 Thijs Feryn All rights reserved Printed in the United States of America Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472 O’Reilly books may be purchased for educational, business, or sales promotional use Online editions are also available for most titles (http://oreilly.com/safari) For more information, contact our corporate/insti‐ tutional sales department: 800-998-9938 or corporate@oreilly.com Editors: Brian Anderson and Virginia Wilson Production Editor: Melanie Yarbrough Copyeditor: Gillian McGarvey Proofreader: Eliahu Sussman February 2017: Indexer: WordCo Indexing Services Interior Designer: David Futato Cover Designer: Karen Montgomery Illustrator: Rebecca Demarest First Edition Revision History for the First Edition 2017-02-01: First Release The O’Reilly logo is a registered trademark of O’Reilly Media, Inc Getting Started with Varnish Cache, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work Use of the information and instructions contained in this work is at your own risk If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights 978-1-491-97718-7 [LSI] This book is dedicated to all the people who support me day in and day out: My lovely wife Lize, my son Lex, and my daughter Lia My mom, dad, sister, mother-inlaw, and brothers-in-law And of course my friends—you know who you are Table of Contents Preface xi What Is Varnish Cache? Why Does Web Performance Matter? Where Does Varnish Fit In? The Varnish Cache Open Source Project How Does Varnish Work? Caching Is Not a Trick Conclusion 2 Go, Go, Go and Get Started! Installing Varnish Installing Varnish Using a Package Manager Installing Varnish on Ubuntu and Debian Installing Varnish on Red Hat and CentOS Configuring Varnish The Configuration File Some Remarks on Systemd on Ubuntu and Debian Startup Options What About TLS/SSL? Conclusion 8 10 10 11 11 16 18 Varnish Speaks HTTP 19 Idempotence State Expiration The Expires Header The Cache-Control Header 20 21 22 23 23 v Expiration Precedence Conditional Requests ETag Last-Modified How Varnish Deals with Conditional Requests Cache Variations Varnish Built-In VCL Behavior When Is a Request Considered Cacheable? When Does Varnish Completely Bypass the Cache? How Does Varnish Identify an Object? When Does Varnish Cache an Object? What Happens if an Object Is Not Stored in Cache? How Long Does Varnish Cache an Object? Conclusion 24 25 25 26 28 29 31 31 31 32 32 33 33 33 The Varnish Configuration Language 35 Hooks and Subroutines Client-Side Subroutines Backend Subroutines Initialization and Cleanup Subroutines Custom Subroutines Return Statements The execution flow VCL Syntax Operators Conditionals Comments Scalar Values Regular Expressions Functions Includes Importing Varnish Modules Backends and Health Probes Access Control Lists VCL Variables Varnish’s Built-In VCL A Real-World VCL File Conclusion 36 36 37 37 38 38 39 41 42 42 43 43 45 45 49 50 50 54 54 57 62 63 Invalidating the Cache 65 Caching for Too Long Purging vi | Table of Contents 65 66 Banning Lurker-Friendly Bans More Flexibility Viewing the Ban List Banning from the Command Line Forcing a Cache Miss Cache Invalidation Is Hard Conclusion 67 68 70 71 71 72 73 74 Dealing with Backends 77 Backend Selection Backend Health Directors The Round-Robin Director The Random Director The Hash Director The Fallback Director Grace Mode Enabling Grace Mode Conclusion 77 78 80 81 82 83 84 85 86 87 Improving Your Hit Rate 89 Common Mistakes Not Knowing What Hit-for-Pass Is Returning Too Soon Purging Without Purge Logic No-Purge ACL 404 Responses Get Cached Setting an Age Header Max-age Versus s-maxage Adding Basic Authentication for Acceptance Environments Session Cookies Everywhere No Cache Variations Do You Really Want to Cache Static Assets? URL Blacklists and Whitelists Decide What Gets Cached with Cache-Control Headers There Will Always Be Cookies Admin Panel Remove Tracking Cookies Remove All But Some Cookie Variations Sanitizing 89 90 90 91 91 91 92 92 93 93 94 94 95 96 97 97 98 99 99 100 Table of Contents | vii Removing the Port Query String Sorting Removing Google Analytics URL Parameters Removing the URL Hash Removing the Trailing Question Mark Hit/Miss Marker Caching Blocks AJAX Edge Side Includes Making Varnish Parse ESI ESI versus AJAX Making Your Code Block-Cache Ready An All-in-One Code Example Conclusion 100 101 101 102 102 102 103 104 104 105 106 108 108 114 Logging, Measuring, and Debugging 115 Varnishstat Example Output Displaying Specific Metrics Output Formatting Varnishlog Example Output Filtering the Output Varnishtop Conclusion 116 116 116 117 117 117 120 121 122 What Does This Mean for Your Business? 123 To CDN or Not to CDN VCL Is Cheaper Varnish as a Building Block The Original Customer Case Varnish Plus Companies Using Varnish Today NU.nl: Investing Early Pays Off SFR: Build Your Own CDN Varnish at Wikipedia Combell: Varnish on Shared Hosting Conclusion 123 124 125 125 126 126 127 127 127 128 129 10 Taking It to the Next Level 131 What About RESTful Services? Patch Support viii | Table of Contents 131 132 ... Getting Started with Varnish Cache Accelerate Your Web Applications Thijs Feryn Beijing Boston Farnham Sebastopol Tokyo Getting Started with Varnish Cache by Thijs Feryn... up with one of your competitors Where Does Varnish Fit In? With a correctly configured Varnish, you will automatically reduce the loading times of your website without much effort Given that Varnish. .. cost-reduction initiatives and ways to bring costs and efforts in-house SFR decided to build its own CDN using Varnish The company started out with reg‐ ular Varnish, but quickly got in touch with Varnish