Peter Sbarski-Serverless Architectures on AWS. With examples using AWS Lambda-Manning Publications (2017)

Peter Sbarski FOREWORDS BY Patrick Debois Donald F Ferguson MANNING Serverless Architectures on AWS PETER SBARSKI with Forewords by Patrick Debois and Donald F Ferguson MANNING SHELTER ISLAND For online information and ordering of this and other Manning books, please visit www.manning.com The publisher offers discounts on this book when ordered in quantity For more information, please contact Special Sales Department Manning Publications Co 20 Baldwin Road PO Box 761 Shelter Island, NY 11964 Email: orders@manning.com ©2017 by Manning Publications Co All rights reserved No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine Manning Publications Co 20 Baldwin Road PO Box 761 Shelter Island, NY 11964 Development editor: Technical development editor: Project editors: Copyeditor: Proofreader: Technical proofreader: Typesetter: Cover designer: ISBN 9781617293825 Printed in the United States of America 10 – EBM – 22 21 20 19 18 17 Toni Arritola Kostas Passadis Kevin Sullivan and Janet Vail Linda Recktenwald Melody Dolab David Fombella Pombal Dottie Marsico Marija Tudor To my mum and dad, who always supported and encouraged my passion for computing contents foreword by Patrick Debois xi foreword by Dr Donald F Ferguson preface xv acknowledgments xvii about this book xix about the author xxi about the cover xxii PART 1 xii FIRST STEPS .1 Going serverless 1.1 How we got to where we are Service-oriented architecture and microservices Software design 1.2 Principles of serverless architectures Use a compute service to execute code on demand Write single-purpose stateless functions 10 Design push-based, event-driven pipelines 10 Create thicker, more powerful front ends 10 Embrace third-party services 12 ■ ■ ■ 1.3 1.4 Transitioning from a server to services Serverless pros and cons 12 Decision drivers 1.5 Summary 12 ■ 15 v When to use serverless 12 14 vi CONTENTS Architectures and patterns 2.1 Use cases 16 16 Application back end 17 Data processing and manipulation 17 Real-time analytics 18 Legacy API proxy 18 Scheduled services 18 Bots and skills 19 ■ ■ ■ ■ 2.2 ■ Architectures 19 Compute as back end 19 Legacy API proxy 24 Hybrid 25 GraphQL 27 Compute as glue 28 Real-time processing 30 ■ ■ ■ 2.3 Patterns ■ 31 Command pattern 32 Messaging pattern 33 Priority queue pattern 34 Fan-out pattern 35 Pipes and filters pattern 36 ■ ■ ■ 2.4 Summary ■ 38 Building a serverless application 3.1 24-Hour Video 39 40 General requirements 41 Amazon Web Services 42 Creating your first Lambda function 44 Naming your Lambda 46 Testing locally 46 Deploying to AWS 48 Connecting S3 to Lambda 50 Testing in AWS 51 Looking at logs 52 ■ ■ ■ ■ ■ 3.2 Configuring Simple Notification Service Connecting SNS to S3 Testing SNS 57 3.3 54 ■ Setting video permissions 54 Getting email from SNS 57 57 Creating the second function 57 Configuring and securing 58 Testing the second function 59 ■ ■ 3.4 Generating metadata 60 Creating the third function and FFprobe 3.5 3.6 3.7 Finishing touches Exercises 64 Summary 65 Setting up your cloud 4.1 60 63 66 Security model and identity management 67 Creating and managing IAM users 67 Creating groups Creating roles 73 Resources 74 Permissions and policies 74 ■ ■ ■ 70 vii CONTENTS 4.2 Logging and alerting 76 Setting up logging 77 Log retention 78 Filters, metrics, and alarms 78 Searching log data 80 S3 and logging 80 More on alarms 81 CloudTrail 85 ■ ■ ■ ■ ■ 4.3 Costs 87 Creating billing alerts 87 Monitoring and optimizing costs 88 Using the Simple Monthly Calculator 89 Calculating Lambda and API Gateway costs 90 ■ ■ 4.4 4.5 PART Exercises 92 Summary 93 CORE IDEAS 95 Authentication and authorization 5.1 Authentication in a serverless environment A serverless approach 5.2 97 98 ■ Amazon Cognito 98 100 Adding authentication to 24-Hour Video ■ Auth0 101 102 The plan 102 Invoking Lambda directly 104 24-Hour Video website 104 Auth0 configuration Adding Auth0 to the website 108 Testing Auth0 integration 113 ■ ■ 106 ■ 5.3 Integration with AWS 114 User profile Lambda 115 API Gateway 118 Mappings 121 Invoking Lambda via API Gateway 124 Custom authorizer 125 ■ ■ ■ 5.4 Delegation tokens 129 Real-world examples 5.5 5.6 ■ Provisioning delegation tokens 130 Exercises 130 Summary 131 Lambda the orchestrator 6.1 130 Inside Lambda 133 133 Event models and sources 134 Push and pull event models 135 Concurrent executions 136 Container reuse 137 Cold and warm Lambda 137 ■ ■ ■ ■ 6.2 Programming model 139 Function handler 139 Event object 139 Context object 140 Callback function 141 Logging 142 ■ ■ ■ ■ viii CONTENTS 6.3 Versioning, aliases, and environment variables Versioning 6.4 142 Using the CLI Aliases ■ 142 Environment variables 146 148 Invoking commands functions 149 6.5 144 ■ Lambda patterns 148 ■ Creating and deploying 151 Async waterfall 152 Series and parallel 157 libraries 158 Move logic to another file 161 ■ ■ Using ■ 6.6 Testing Lambda functions Testing locally AWS 164 6.7 6.8 161 Writing tests ■ 162 ■ Testing in Exercises 166 Summary 166 API Gateway 7.1 161 168 API Gateway as the interface 169 Integration with AWS services 170 Caching, throttling, and logging 170 Staging and versioning 171 Scripting 171 ■ ■ 7.2 ■ Working with the API Gateway 171 The plan 173 Creating the resource and method 174 Configuring method execution 177 The Lambda function 180 Updating the website 184 ■ ■ ■ 7.3 Optimizing the gateway Throttling 7.4 187 ■ Logging 189 Stages and versions PART ■ Caching 191 194 Creating a stage variable Versions 196 7.5 7.6 187 195 ■ Using stage variables 195 Exercises 198 Summary 198 GROWING YOUR ARCHITECTURE 199 Storage 8.1 201 Smarter storage 201 Versioning 202 Hosting a static website 204 classes 206 Object lifecycle management 207 acceleration 209 Event notifications 209 ■ ■ ■ ■ ■ Storage Transfer ix CONTENTS 8.2 Secure upload 211 Architecture 212 Upload policy Lambda 213 S3 CORS configuration 218 Uploading from the website 219 ■ ■ ■ 8.3 Restricting access to files Removing public access 8.4 8.5 ■ Generating presigned URLs 227 Introduction to Firebase Data structure 9.2 223 Exercises 225 Summary 225 Database 9.1 223 228 ■ 228 Security rules Adding Firebase to 24-Hour Video 230 230 Architecture 231 Setting up Firebase 233 Modifying Transcode Video Lambda 234 Transcode Video Firebase Update 239 Connecting Lambda 241 Website 242 End-to-end testing 248 ■ ■ ■ ■ 9.3 ■ Securing access to files 248 Signed URL Lambda 249 API Gateway settings 250 Updating the website again 251 Improving performance 251 Improving Firebase security 254 ■ ■ ■ 9.4 9.5 10 Exercises 259 Summary 259 Going the last mile 10.1 10.2 260 Deployment and frameworks 260 Toward better microservices 261 Handling errors 10.3 Step Functions 264 266 Image-processing example 10.4 10.5 appendix A appendix B appendix C appendix D appendix E 267 AWS Marketplace 272 Where from here 274 Services for your serverless architecture 277 Installation and setup 282 More about authentication and authorization Lambda insider 299 Models and mapping 305 293 224 ... application Serverless Architectures on AWS has detailed sections on both topics The material explains the concepts (for example, authentication and authorization), positions the concepts within...Serverless Architectures on AWS PETER SBARSKI with Forewords by Patrick Debois and Donald F Ferguson MANNING SHELTER ISLAND For online information and ordering of this and other... exploitation of cloud computing The detailed information about AWS within the context of a real application is priceless, and the concepts and patterns apply to any serverless solution using any