Architectural transformations in network services and distributed systems

Andriy Luntovskyy and Josef Spillner Architectural Transformations in Network Services and Distributed Systems Andriy Luntovskyy Berufsakademie Sachsen, Dresden, Germany Josef Spillner Service Prototyping Lab, Zürcher Hochschule für Angewandte Wissenschaften, Winterthur, Switzerland ISBN 978-3-658-14840-9 e-ISBN 978-3-658-14842-3 DOI 10.1007/978-3-658-14842-3 Library of Congress Control Number: 2016957988 © Springer Fachmedien Wiesbaden GmbH 2017 This work is subject to copyright All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed The use of general descriptive names, registered names, trademarks, service marks, etc in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made Printed on acid-free paper This Springer Vieweg imprint is published by Springer Nature The registered company is Springer Fachmedien Wiesbaden GmbH The registered company address is: Abraham-Lincoln-Str 46, 65189 Wiesbaden, Germany To our caring wives Preface About the Book Book objectives You are reading a book which aims to cover the broad field of recent innovations in network services and distributed systems The target group of the book encompasses students of universities and technical high schools, graduated engineers as well as teaching staff If you are somebody else, not worry, the covered subjects may still be of interest to you! This book offers its readers a dual functionality: As a monograph… With the given work, we decided to help not only the readers and students, but also ourselves, as the professionals who are actively involved in the networking, telecommunications and systems communities, by understanding the trends which have developed in the recent decade in distributed systems and networking applications Important architectural transformations of modern distributed systems are examined and presented in survey style Examples of new architectural solutions for network (mobile) services and applications are discussed Among them are the Internet of Services, Clouds, Smart Grids, Parallel and Distributed Computing, Fog Computing and the Internet of Things, to mention a couple of popular concepts As a handbook… Current technologies, standards and research results for advanced (mobile) networks, connected devices and provisioned services as well as for higher-level network functions and software applications are focused within this book from a practical angle The authors highlight how these technical underpinnings to our digital communication and collaboration infrastructure are being transformed to reflect society requirements Efficient architectures, principles and systems for mobile and wireless communication, criteria for optimisation of networks and distributed systems, as well as central ideas to new system concepts are widely discussed herein Use case presentations and studies with in-depth technical descriptions along with a test exam strengthen the nature of this book as handbook to use for courses and projects Learning objectives The learning objectives targeted by the book are as follows: Readers/students should be able to combine, integrate, analyse and manage the solutions to the above-mentioned technologies (Clouds, Smart Grids, Parallel and Distributed Computing, Fog Computing, Internet of Services, Internet of Things) They should also be able to implement custom systems on the basis of an adequate conceptual grounding in practical projects As a result, readers/students become skilled to create and evaluate wellperforming, reliable and secure access aspects to data and network applications, distributed systems and mobile apps The systems and services should be usable in a data protection-compliant manner and aligned with user preferences Readers/students become educated to develop custom architectures of network services and distributed systems as well as to comment critically on the associated problems Numerous examples in the chapters, comparison tables, excursions into technological stacks, figures with structures and demonstrations are highlights of this book Every chapter has a list of keywords, complemented by actual system examples, a summary and continuing bibliographic records Furthermore, at the end, there is a whole chapter dedicated to repetition and self-controlling by offering questions and answers to many of the discussed topics along with further insight into the research behind the covered systems and services Motivation Despite the existence of a broad range of scientific and practical literature on the topics of distributed computing, cloud computing, privacy-preserving systems, green IT, Internet of Things and so forth, from our perspective as researchers and lecturers there is a distinct lack of combined monographs/handbooks with a pretence to be useful to education In particular, most of the literature describes technological snapshots as points in time Instead, we want to explicitly include historical background information and focus on the ongoing evolution and trends which are similar in many areas Furthermore, we were not satisfied with literature which merely lists positions and standards instead of allowing the reader to dive right into the technology by offering concrete implementation and use case links Especially for students in co-education (for instance Berufsakademie/Duale Hochschule , Fachhochschule and (höhere) Fachschule in Germany and Switzerland, vocational and community colleges in the U.S.), the practical links are essential to decide whether or not a certain technology should indeed be evaluated for upcoming projects The book partially continues the educational approach of a previous book called Planning and Optimisation of Computer Networks: Methods, Models, Tools for Design, Diagnosis and Management in the Lifecycle of Wired and Wireless Computer Networks , by Luntovskyy, Guetter and Melnyk, which appeared by Springer Vieweg in German language in 2011 The original title is: Andriy Luntovskyy, Dietbert Guetter, Igor Melnyk Planung und Optimierung von Rechnernetzen: Methoden, Modelle, Tools für Entwurf, Diagnose und Management im Lebenszyklus von drahtgebundenen und drahtlosen Rechnernetzen Springer Fachmedien Wiesbaden GmbH, 2011, 435 pages (ISBN 978-3-8348-1458-6), 1st edition 2011, with 245 figures und 64 tables The present book complements and extends the range of topics It addresses the evolved development from computer networks to networkintegrated and network-connected services, in particular cloud and fog services, as well as modern architectures of distributed (mobile) applications such as 5G and low-energy radio links The new book therefore presents a holistic view on transformation processes, which are nowadays often less technically motivated, but rather by the needs of the society which is subject to a higher degree of pervasive services The benefits for society are about ecology (green networks), privacy (secure clouds), comfort (always on) and economy (pay as you go) Structure of the Book This book is divided into seven chapters The first chapter offers a bird’s perspective on the history and present development of networking and service topics The second chapter presents state-of-the-art distributed systems and uses them to explain the architectural transformations which most of today’s systems are subject to In the chapters three to six, different architectures and systems will be presented, including clusters, clouds, fogs and mobile applications The seventh chapter offers a holistic view on security in networked services Finally, five appendices and one more auxiliar digital appendix complete the book Chapter – Periodisation of Network Service Development The evolution of hardware and infrastructure on one hand and of services on the other hand is divided into four phases each Chapter – Architectural Transformation in Distributed Systems Clusters and clouds, peer-to-peer architectures and distributed databases will be presented and reflected on in the context of the evolution and transformation of systems Chapter – Evolution of Clustering and Parallel Computing Clusters, grids and parallel computing will be introduced Their benefits concerning the performance of computing, but also the necessary tradeoffs with energy consumption and price, will be highlighted The management of resources and applications in these environments will also be explained Chapter – Cloud Computing, Virtualisation, RAICs and SDN This chapter will introduce contemporary cloud stacks and services, including programmable networks, virtual teleconferences and safe data backups Chapter – Smart Grid, Internet of Things and Fog Computing Beyond the software side, small connected hardware devices and the connection between computer networks and energy distribution networks will be covered in this chapter Chapter – Future Mobile Communications: From 4G to 5G, 5G Enabling Techniques Mobile communication protocols for global (phones) and local distances will be presented A special focus is on the upcoming 5G connectivity Chapter – Security in Distributed Systems This chapter will give a holistic view on what is commonly called security, by introducing into concrete protection goals and matching security layers It will also include a discussion of privacy and legal aspects with a focus on how users can protect their activities and communication in today’s and tomorrow’s distributed systems Appendices First, selected originators and designers of distributed systems will be briefly presented Then, specific research projects with recent results which contribute to the evolution and transformation will be introduced The further parts contain explanations to common acronyms in mobile and wireless technologies, a repetition and control part to track the learning progress when reading the book, and finally an example of a written exam to the discussed subjects The solutions to the exam are available as auxiliar digital appendix Andriy Luntovskyy Josef Spillner Dresden, Germany, Winterthur, Switzerland concerning the aspects of data privacy and security (f) Demarcate the definition boundaries of clusters, grids and clouds (g) Assign to each cloud usage scenario in the first column of Table D.1 the correct pattern of cloud services in the second to fourth column In some cases, a term may belong to multiple columns D.5 Virtualisation Concepts Virtualisation refers to methods which allow to combine or divide physical resources in a distributed system The primary goal is to present an abstraction view to the user to isolate him or her or any application from the actual hardware and software, including computational performance and memory availability (a) A logical layer is introduced between the applications and the resources in order to hide the physical circumstances Create a sketch corresponding to this action (b) Specify a classification of kinds of virtualisation What is hardware virtualisation and what is software virtualisation? (c) What is a VM? Comment on the concept of VMM Which kinds of VMM exist? What is server virtualisation? (d) Mention the advantages of virtualisation from an economic perspective (e) Look at the following scenario: In a company network with 30 computers, there are three different architectures How many import and export routines have to be programmed and installed to facilitate a communication between all the systems? Which changes result from adding a 31st computer with a novel system architecture into the network? Which advantages and disadvantages compared to result from the use of a server virtualisation? (f) What is software virtualisation? How does it enable EAI? (g) Describe the areas of use and the functionality of OS containers D.6 Performance Characteristics of Digital Computers, Performance Optimisation in Distributed Systems Threads, replication and caching are classic methods for improving the performance of client/server systems Describe them briefly (a) Which additional methods can be used to increase the performance? Identify them with suitable examples (b) Which seven important empirical rules about the performance optimisation of client/server systems are known to you? (c) How to organise a parallel execution with threads? Create a sketch with a sample execution of statements (d) How much time is saved, when a parallelisation with N threads is enabled for a certain application? Justify your answer How can the degree of parallelisation be increased? Name the key performance indicators of digital computers (e) Demarcate the terms of processor clock frequency, MIPS and FLOPS Introduce an example for each of the terms D.7 Distributed Computing, Parallel Computing and Acceleration Models Parallel computers have an increased working speed by their distribution of load across N available processors In order to fully utilise the complete performance of such a parallel computer, the programming technique must be adapted accordingly A number of languages, techniques and programming interfaces are available for this challenge (a) Which reference solutions for the parallel execution of applications you know? Compare the features of OpenMP and MPI in this context (b) Specify a definition for the term “Grid” Classify the different types of grids (c) What is the BOINC system? Explain the functional principles of this grid (d) Exemplify the differences betwen clusters, grids and clouds (e) Name the methods for optimising application performance (f) Name performance acceleration models for parallel application Consider the laws of Amdahl and Gustafson-Barsis (g) The Amdahl law is based on a model of the acceleration of applications through parallelisation Given are the time effort for one CPU: T  = 20, 000 h Tp  = 18, 000 h Calculate the speedup factor A ( n ) according to G M Amdahl and visualise the answer in a diagram (h) Which changes in performance calculation result from explicitly considering the communication flows between threads? (i) Compare the speedup models for parallelised applications according to Amdahl and Gustafson-Barsis Given are a number of CPUs in a cluster: N  =  100 The measured acceleration: A  = 25 Calculate the sequential part e as well as the part p  = 1 − e for the parallelised application according to the metric of Karp-Flatt D.8 Towards 5G About once per decade, the user data rate in the mobile phone networks is increasing by a factor of 10 to 100 This tendency has been observed for quite a while, since the 1990s The fifth generation of mobile networks is a reference to the next important phase of development following the notion of “Beyond 4G” and IMT Advanced Standards (a) Which differences exist between 5G and its predecessors 3G and 4G? What is IMS? (b) Elaborate on the role of MIMO and DIDO technologies (c) Describe briefly the most important network technologies for the realisation of 5G deployments in the near future (d) Characterise the interoperability between mobile networks (Beyond 4G), WLAN and 6LowPAN (e) What is IoT? How does this concept correlate with the techniques 6LowPAN and 5G? D.9 Security Aspects in NGN Data privacy protocols play an important role in the electronic business due to their ability to carry important data security mechanisms Among them are confidentiality, integrity, mutual authentication and non-repudiability (a) Define the two terms “authentication” and “authorisation” properly without overlap (b) Confidential C/S communication between an online shop and an external payment provider needs to be secured Describe the necessary procedure for a symmetric data encryption Now describe the modification of the procedure for an asymmetric data encryption Discuss the “Pro” and “Contra” of both the symmetric and the asymmetric cryptographic technique Why and how should there be a combination of both? (c) What is the protocol SSL for? Which sub-protocols are used by SSL? (d) Which important cryptographic standards are used by SSL? What is the role of the X.509 specification? (e) Describe the limits of SSL for use in web services Discuss the disadvantages (f) Discuss the differences in the cryptographic protection of data communication with SSL and with VPN/IPsec Elaborate on the role of the concept of fine-grained protection (g) Specify a holistic classification or ordering of cryptorgraphic protocols for data security in distributed systems aligned with the OSI reference model (h) Discuss the hybrid solutions to SSL/SET (i) What are causes of security vulnerabilities in distributed systems? How are those being exploited by malware? (j) Mention solutions to the previous problems as they are known to you (AV-SW, FW) Discuss these concepts concerning the suitability for distributed systems (k) Explain the differences between capability models and access control lists Compare these two concepts concerning the following: Implementation effort Suitability for large-scale systems with many clients Propagation and delegation of privileges among clients Removal of privileges (l) A user of an online shop wants to order a couple of products During the course of the ordering process, the server side wants to check and confirm the identity of the user Describe the authentication with symmetric encryption methods Now describe the same with asymmetric methods (m) A user of an online shop wants to double-check on the integrity and the non-repudiability of the order What is a digital signature? How are the public and private keys used in combination with a digital signature? How are message digests (extended checksums) created? Characterise the properties of hash functions Why is only the asymmetric encrpytion method suitable for a digital signature? How many and which keys are needed for the hybrid symmetric and asymmetric encryption when a digital signature should be generated at the same time? Which protection goals are feasible in such a scenario? What is the meaning of certificate and on which occasions are they distributed? How are hierarchic CAs built? Discuss them with the example of X.509 Compare them with the Kerberos method D.10 PGP and Steganography Jointly with the steganographic methods, cryptography offers a compelling combination for protecting data and, hence, information If the secret message is concealed and does not attract the attention as would be the case for obviously encrypted messages, the application of secure steganography cannot be prosecuted anymore Therefore, a ban on encryption would be practically void (a) Discuss the assignment of the cryptographic protocols known to you to the layers in the OSI reference model (b) Which cryptographic techniques are used? Which ones are integrated in PGP? (c) Which advantages are associated with the OpenPGP standard? Which disadvantages are implied by using PGP in the web of trust scheme? Already Herodot, one of the first writers of history, reported about 2500 years ago on the concealment of messages by use of steganography Seemingly unused wax writing pads carried the message on wood underneath the wax surface Messages were also stitched off animals which were brought by a messenger, posing as hunter (d) Give more historic examples for delivering steganographic messages (e) Explain in which areas steganographic methods are used today Why is cryptography sometimes insufficient for these cases? How is steganography an efficient alternative? (f) Classify steganographic techniques according to their carrier medium and the embedding scheme Clarify the differences to watermarks (g) Explain the principles of the combination of steganographic and cryptographic methods Appendix E: Example of a Written Exam to the Discussed Subjects This examination adds to the repetition questions and answers presented before It also serves as inspiration to lecturers who want to design a custom examination of about 45 minutes The permissible tools would be all paper documents and books, as well as a non-programmable calculator, but no mobile devices, notebooks or other electronic helpers 50 points can be achieved in total Note: The reference solution to the exam is available as complementary digital-only material from the publisher’s website Task Architectural transformations (10 points) Nowadays the significant architectural transformations in network services and distributed systems have to be considered The processing, persistency as well as application data are provided from multiple servers or peers (a) Compare please C-S and P2P-architectures Specify 2–3 differences between them Introduce your examples, at least 2–3 to each architecture type! (5 points) (b) Describe the current architectural transformations in distributed systems! What is clustering, what advantages does the method possess? Discuss to the increased complexity due to synchronisation and conflict handling as a disadvantage! (5 points) Task Parallel computing (15 points) Parallel computers increase their working speed by distributing the available computational efforts to their N processors They are commonly found in data centres and in supercomputing sites (shown: CHIC, TU Chemnitz) To determine the full capabilities of a parallel computer with N CPUs, a class of acceleration models (speedup models) can be used (a) Specify please the well-known performance factors of modern computers/clusters/grids! (2 points) (b) Gustafson-Barsis Law is a speedup models about the acceleration of the programs via their parallelisation It was given: the time span is required for a CPU T  = 4000 h; T p  = 3800 h Compute please the speedup factor A ( N ) by Gustafson-Barsis! Notice: N  = 1, 11, 41, 71 (7 points) Calculate the efficiency E N , N  = 41! Compare please the speedup models for parallelised applications proposed by Amdahl and Gustafson-Barsis! (c) It was given: the available CPU number in a cluster is N  = 100, the measured speedup factor A  = 75 Compute please the sequential part e as well as the part p  = 1 − e for parallelised application by Karp-Flatt metrics! (3 points) In order to exploit the full power of a parallel computer with N CPUs, the programming techniques must be adapted accordingly For this aim a diversity of languages, methods and programming interfaces are available (d) Specify the well-known programming techniques to parallel computing (at least 3) What distinguishing features they have in comparison to the regular source code for N  = 1 increase by a factor of p? (3 points) Task Virtualisation (15 points) Virtualisation addresses the software tools as well the methods, which enable to summarise or split some available resources of a computer or network based on the concept of VM (virtual machines) The primary objective is to provide the users with a special layer of abstraction (virtualisation layer) The virtualisation is able to isolate the users and his applications from the actual (hardware) technical circumstances (a) Specify please four other cost benefits of resources virtualisation from the technical and operational perspective: (4 points) Better load balancing for virtual machines … … … … … … … … … … … …  .  … … … … … … … … … … … …  .  … … … … … … … … … … … …  .  … … … … … … … … … … … …  .  (b) A physical server (RAM of 128 GB and HDD capacity of 0.5 PB are fully available) in an mid-range enterprise supports 20 VMs with each RAM 4 GB/HDD 8000 GB How much capacity of a physical server is left? (4 points) RAM: … … … … … … … … … … HDD: … … … … … … … … … … (c) As managed servers (also virtualised) the specific hosts are titled, which operating system and software is constantly monitored and virtualised via a service provider (see the Table E.2 ): Table E.1 Managed servers Authentication server/RADIUS Database server/ SQL Mail server (e-mail server)/SMTP, MIME, POP3, IMAP Name server/DNS File server Terminal server File transfer/FTP, SFTP DHCP server/DHCP, intranet, IP, News server/NNTP NAT, PAT und Mobile IP Chat server/IRC Game server/TCP/IP Web server/HTTP, HTTPS, SOAP Proxy server/VPN, IPsec, Streaming server/SIP/RTP, SNMP, Intranet, FW codecs Domain server/backup server Terminal server/RDP etc Time server (network time protocol server)/NTP Which server types (s the list above) can be used as (virtual) managed servers in a mid-range enterprise in your opinion! Please make complete the given placeholder lines below How much capacity/ resources should be left in this case (refer to “(b)”)? (5 points) VM: … … … … … …  RAM: … … … … … …  HDD: … … … … … …   VM: … … … … … …  RAM: … … … … … …  HDD: … … … … … …   VM: … … … … … …  RAM: … … … … … …  HDD: … … … … … …   VM: … … … … … …  RAM: … … … … … …  HDD: … … … … … …   VM: … … … … … …  RAM: … … … … … …  HDD: … … … … … …   Resources left: RAM: … … … … … …  HDD: … … … … … …   (d) SDN: What does this term mean? Which SDN protocols and reference solutions you know? (2 points) Task Cloud computing (10 points) One of the common definitions of the clouds according to Amazon address the following patterns of existing cloud services: SaaS, PaaS, IaaS (a) To which type (SaaS, PaaS, IaaS) does belong a cloud-based storage? Justify your statement! (3 points) (b) What is RAIC? Discuss the benefits of mapping of traditional RAID structures to the clouds based on the specified scenario briefly (referring to the picture below): (7 points) DP: Double Parity ………………………… ………………………… ………………………… ………………………… Index A Anonymous Computing B Backup Blockchain C Cloud Computing Cluster Computing Collaboration Communication Coordination Cryptography D Data Centre Database Distributed Computing E E-Commerce Energy Efficiency F Fog Computing G Green IT Grid Computing I Interoperability M Microcontroller Mobile Computing Modulation Multi-Threading N Network O Operation P Parallel Computing Peer-to-Peer Computing Performance Planning Protection Goal Protocol Q Quality S Satellite Secure Computing Service Platform Service-Oriented Architecture Smart Grid Social Network Storage T Trade-off Transaction V Virtualisation Volunteer Computing W Web Application Web Service ... actively involved in the networking, telecommunications and systems communities, by understanding the trends which have developed in the recent decade in distributed systems and networking applications... Architectural Transformations in Network Services and Distributed Systems, DOI 10.1007/978-3-658-14842-3_2 Architectural Transformations in Distributed Systems Andriy Luntovskyy1 and Josef Spillner2 (1)... one hand and of services on the other hand is divided into four phases each Chapter – Architectural Transformation in Distributed Systems Clusters and clouds, peer-to-peer architectures and distributed