Scala Design Patterns Second Edition Design modular, clean, and scalable applications by applying proven design patterns in Scala Ivan Nikolov BIRMINGHAM - MUMBAI Scala Design Patterns Second Edition Copyright © 2018 Packt Publishing All rights reserved No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews Every effort has been made in the preparation of this book to ensure the accuracy of the information presented However, the information contained in this book is sold without warranty, either express or implied Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals However, Packt Publishing cannot guarantee the accuracy of this information Commissioning Editor: Aaron Lazar Acquisition Editor: Sandeep Mishra Content Development Editor: Akshada Iyer Technical Editor: Abhishek Sharma Copy Editor: Safis Editing Project Coordinator: Prajakta Naik Proofreader: Safis Editing Indexer: Pratik Shirodkar Graphics: Jisha Chirayil Production Coordinator: Shantanu Zagade First published: February 2016 Second edition: April 2018 Production reference: 1060418 Published by Packt Publishing Ltd Livery Place 35 Livery Street Birmingham B3 2PB, UK ISBN 978-1-78847-130-5 www.packtpub.com mapt.io Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career For more information, please visit our website Why subscribe? Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals Improve your learning with Skill Plans built especially for you Get a free eBook or video every month Mapt is fully searchable Copy and paste, print, and bookmark content PacktPub.com Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy Get in touch with us at service@packtpub.com for more details At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks Contributors About the author Ivan Nikolov is a technical architect based in London He works in the ad tech industry and uses Scala in combination with libraries and technologies such as Spark, Hadoop, RabbitMQ, Kafka, SQL and NoSQL stores, and Akka He also uses other JVM and scripting languages Some of the projects Ivan has worked on include a large-scale real-time machine learning platform, batch processing solutions, and high load APIs Ivan also likes getting involved with open source projects, whether it be to contribute or get inspiration and good ideas I would like to thank Felix and Tasia and my mother, Veronika, for their interest in this book and everything I Thanks to Becky for dealing with me working until late in the evenings and all of her support Finally, thanks to everyone involved in publishing this book—editors, technical reviewer, and people I haven't been in touch with but have invested their time in making sure that everything went smoothly trait ActorFactoryComponent { this: AppConfigComponent with DaoServiceComponent => val actorFactory: ActorFactory class ActorFactoryImpl extends ActorFactory { override def createMasterActor(): Master = new Master(appConfigService.workers, this) override def createWorkerActor(): Worker = new Worker(daoService) } } You can see how the preceding factory is passed (using the this reference) and used in the Master actor for creating Worker instances We have already seen everything we need in order to run console jobs with our scheduler Now we have to implement things in order to support database access We will skip the database code here, as it is pretty much the same as the one we saw in Chapter 11, Applying What We Have Learned, it is based on the cake design pattern We have only skipped some convenience methods that we don't need here However, the database our scheduler can query still has the same schema: CREATE TABLE people( id INT PRIMARY KEY, name VARCHAR(255) NOT NULL, age INT NOT NULL ); CREATE TABLE classes( id INT PRIMARY KEY, name VARCHAR(255) NOT NULL, ); CREATE TABLE people_classes( person_id INT NOT NULL, class_id INT NOT NULL, PRIMARY KEY(person_id, class_id), FOREIGN KEY(person_id) REFERENCES people(id) ON DELETE CASCADE ON UPDATE CASCADE, FOREIGN KEY(class_id) REFERENCES classes(id) ON DELETE CASCADE ON UPDATE CASCADE ); We have also added some extra database statements to aid us in unit testing but they are minor, and not putting them here will not affect anything Wiring it all up It seems like we already have all the components our application will be using Now we just need to wire things together and fire it up We are using the cake design pattern for dependency injection; so, as you have already seen, we can create one component registry with everything we need: package com.ivan.nikolov.scheduler.registry import import import import import com.ivan.nikolov.scheduler.actors.{ActorFactory, ActorFactoryComponent} com.ivan.nikolov.scheduler.config.app.AppConfigComponent com.ivan.nikolov.scheduler.dao._ com.ivan.nikolov.scheduler.io.IOServiceComponent com.ivan.nikolov.scheduler.services.JobConfigReaderServiceComponent object with with with with with with ComponentRegistry extends AppConfigComponent IOServiceComponent JobConfigReaderServiceComponent DatabaseServiceComponent MigrationComponent DaoServiceComponent ActorFactoryComponent { override val appConfigService: ComponentRegistry.AppConfigService = new AppConfigService override val ioService: ComponentRegistry.IOService = new IOService override val jobConfigReaderService: ComponentRegistry.JobConfigReaderService = new JobConfigReaderService override val databaseService: DatabaseService = new H2DatabaseService override val migrationService: ComponentRegistry.MigrationService = new MigrationService override val daoService: DaoService = new DaoServiceImpl override val actorFactory: ActorFactory = new ActorFactoryImpl } Now that we have a component registry, we can use it and write the main application class: package com.ivan.nikolov.scheduler import akka.actor.{Props, ActorSystem} import com.ivan.nikolov.scheduler.actors.messages.Schedule import com.typesafe.scalalogging.LazyLogging import scala.concurrent.Await import scala.concurrent.duration.Duration object Scheduler extends LazyLogging { import com.ivan.nikolov.scheduler.registry.ComponentRegistry._ def main(args: Array[String]): Unit = { logger.info("Running migrations before doing anything else.") migrationService.runMigrations() logger.info("Migrations done!") val system = ActorSystem("scheduler") val master = system.actorOf( Props(actorFactory.createMasterActor()), "scheduler-master" ) sys.addShutdownHook({ logger.info("Awaiting actor system termination.") // not great Await.result(system.terminate(), Duration.Inf) logger.info("Actor system terminated Bye!") }) master ! Schedule(jobConfigReaderService.readJobConfigs()) logger.info("Started! Use CTRL+C to exit.") } } Our application has some very simple Akka wiring up and then everything is triggered when we execute the highlighted line We send a Schedule message to the master with all the job configurations and then it will schedule them to run periodically according to their definitions The end result After all this code is written, we will end up with the following tree in our IDE: You can see that our code also contains unit tests We will spend some time on them in the next subsection Testing our application Testing is a really important part of every application Using TDD is really good, as we can write and test our applications simultaneously instead of coming back to something that is already done We used this approach while writing the application, but separated the code and the tests in favor of explaining things better and not mixing them up Unit testing As you have already seen earlier, testing applications that rely on the cake design pattern is simple We have defined the following test environment: package com.ivan.nikolov.scheduler import import import import import import import com.ivan.nikolov.scheduler.actors.{ActorFactory, ActorFactoryComponent} com.ivan.nikolov.scheduler.config.app.AppConfigComponent com.ivan.nikolov.scheduler.dao._ com.ivan.nikolov.scheduler.io.IOServiceComponent com.ivan.nikolov.scheduler.services.JobConfigReaderServiceComponent org.mockito.Mockito._ org.scalatest.mockito.MockitoSugar trait TestEnvironment extends AppConfigComponent with IOServiceComponent with JobConfigReaderServiceComponent with DatabaseServiceComponent with MigrationComponent with DaoServiceComponent with ActorFactoryComponent with MockitoSugar { // use the test configuration file override val appConfigService: AppConfigService = spy(new AppConfigService) // override the path here to use the test resources when(appConfigService.configPath).thenReturn(this.getClass.getResource("/").getPath ) override val ioService: IOService = mock[IOService] override val jobConfigReaderService: JobConfigReaderService = mock[JobConfigReaderService] override val databaseService: DatabaseService = mock[DatabaseService] override val migrationService: MigrationService = mock[MigrationService] override val daoService: DaoService = mock[DaoService] override val actorFactory: ActorFactory = mock[ActorFactory] } When we testing, we use an actual application configuration file instead of working with mocks and can use any file inside our test resources folder We have written quite extensive tests for the TimeOptions class, more specifically the part that calculates the initial delay There are tests to read job configuration files as well as database access tests They can all be seen in the projects provided with this book Application testing No doubt, the part that everyone wanted to see the most is where we actually give our application an actual spin However, because it is a scheduler, we first need to prepare some configurations We will use the following application configuration file: job-scheduler { config-path="/etc/scheduler/conf.d" config-extension="json" workers=4 db { connection-string="jdbc:h2:mem:test;DB_CLOSE_DELAY=-1" username="" password="" } } We will name our file production.conf and we will put it in /etc/scheduler/conf.d Now we need to create some actual job configurations We will have to put them where the config-path property is pointing to: // ping.json { "name": "Ping Command", "command": "ping google.com -c 10", "frequency": "Hourly", "type": "Console", "time_options": { "hours": 21, "minutes": 10 } } // ping1.json { "name": "Ping1 Command", "command": "ping facebook.com -c 10", "frequency": "Hourly", "type": "Console", "time_options": { "hours": 21, "minutes": 15 } } // query.json { "name": "See all people", "command": "SELECT * FROM people", "frequency": "Hourly", "type": "Sql", "time_options": { "hours": 21, "minutes": } } The last job should have no issues running on any operating system You may need to change the commands in the first two jobs if you are using Windows, for example Also, time options might need to be changed, depending on when the application is being run and if you don't want to wait for hours before you see proof that it actually works If we run our application now with these configurations, we will get the following output: We can leave the application and it will keep executing the tasks we had every one hour or day, depending on how the tasks are scheduled Of course, we could add more meaningful jobs, allocate more workers, change other configurations, and so on The future of our application We have used a number of techniques and concepts that we have learned throughout this book—dependency injection, factory design pattern, ADTs, and the Akka library, which can be used to implement the observer design pattern This was a complete application that was designed to be testable and extendible We can easily add more granularity to the execution schedule of the jobs, the different types of tasks, and also have tasks trigger each other, different routing mechanisms, and so on We showed that we have learned about a great number of useful concepts in this book, and now we can put everything to use in order to create great programs Summary Here, we are at the end of our journey through the Scala design patterns As you already know, design patterns exist in order to cope with a certain limitation of a language They also help us to structure our code in a way that makes it easy to change, use, test, and maintain Scala is an extremely rich language, and we focused on some of its features that make it capable of achieving things that other languages might not be able to without any extra effort and knowledge We looked at the different Gang of Four design patterns from the point of view of Scala—creational, structural, and behavioral design patterns We saw that some of them are not even applicable in functional languages and that others can be approached differently We also saw that some design patterns still remain valid and knowing them is really important for any developer We can't talk about Scala without dealing with concepts such as monoids and monads At first, they could be pretty scary and abstract and manage to put people off So, we spent some time with them and showed their value They can be used to write powerful applications in a purely functional way They can be used to abstract and reuse functionality By minimizing the dry theory and focusing on understandable examples, we hopefully made them much more approachable and usable for those of you who don't have a deep mathematical background Using the rich features of Scala opens up another large group of design patterns We spent some time on design patterns that are possible just because of the way Scala works and the different features it provides Throughout the book, we have tried to provide meaningful examples that can be used as a reference to find specific patterns and applications of the techniques we learned here In this last chapter, we even implemented a complete application On many occasions, we have tried to showcase how different design patterns can be combined Of course, in some cases, the concepts could be pretty complicated by themselves, so we simplified the examples We've given some advice about when to use certain design patterns and when to avoid them These points should be extremely helpful to you in terms of what details to focus on The use of libraries has been encouraged throughout the book Especially in the last few chapters, you have been exposed to quite a large number of interesting libraries that can be easily added to your arsenal We have hopefully also sparked an interest and a habit to always checks before trying something new Finally, all the examples found in this book can also be found online at https://github.com/nikolovivan/scala-design-patterns Other Books You May Enjoy If you enjoyed this book, you may be interested in these other books by Packt: Scala Microservices Jatin Puri, Selvam Palanimalai ISBN: 978-1-78646-934-2 Learn the essentials behind Microservices, the advantages and perils associated with them Build low latency, high throughput applications using Play and Lagom Dive deeper with being asynchronous and understand the superiority it provides Model your complex domain data for scale and simplicity with CQRS and Event Sourcing Be resilient to failures by using message passing Look at best practices of version control workflow, testing, continuous integration and deployments Understand operating system level virtualization using Linux Containers Docker is used to explain how containers work Automate your infrastructure with kubernetes Scala for Machine Learning - Second Edition Patrick R Nicolas ISBN: 978-1-78712-238-3 Build dynamic workflows for scientific computing Leverage open source libraries to extract patterns from time series Write your own classification, clustering, or evolutionary algorithm Perform relative performance tuning and evaluation of Spark Master probabilistic models for sequential data Experiment with advanced techniques such as regularization and kernelization Dive into neural networks and some deep learning architecture Apply some basic multiarm-bandit algorithms Solve big data problems with Scala parallel collections, Akka actors, and Apache Spark clusters Apply key learning strategies to a technical analysis of financial markets Leave a review - let other readers know what you think Please share your thoughts on this book with others by leaving a review on the site that you bought it from If you purchased the book from Amazon, please leave us an honest review on this book's Amazon page This is vital so that other potential readers can see and use your unbiased opinion to make purchasing decisions, we can understand what our customers think about our products, and our authors can see your feedback on the title that they have worked with Packt to create It will only take a few minutes of your time, but is valuable to other potential customers, our authors, and Packt Thank you! .. .Scala Design Patterns Second Edition Design modular, clean, and scalable applications by applying proven design patterns in Scala Ivan Nikolov BIRMINGHAM - MUMBAI Scala Design Patterns. .. Patterns Out There and Setting Up Your Environment Design patterns Scala and design patterns The need for design patterns and their benefits Design pattern categories Creational design patterns The... of inheritance hierarchies Linearization rules How linearization works Initialization Method overriding Testing traits Using a class Mixing the trait in Mixing into the test class Mixing into