Private cloud computing and Amazon Web services

MINISTRY OF EDUCATION AND TRAINING DUY TAN UNIVERSITY NGO DINH CUONG PRIVATE CLOUD COMPUTING AND AMAZON WEB SERVICES MASTER THESIS OF COMPUTER SCIENCE Danang, 2019 MINISTRY OF EDUCATION AND TRAINING DUY TAN UNIVERSITY NGO DINH CUONG PRIVATE CLOUD COMPUTING AND AMAZON WEB SERVICES Major: Computer science Major Code: 8480101 MASTER THESIS OF COMPUTER SCIENCE Supervisor: Dr ANAND NAYYAR Danang, 2019 ACKNOWLEDGEMENT I would like to express my special thanks of gratitude to my mentor, Dr Anand Nayyar as well as Computer Science Faculty, gave me the golden opportunity to this wonderful project on the topic ”Private Cloud Computing and Amazon Web Services”, which also helped me in doing a lot of Research and i came to know about so many new things I am really thankful to them Ngo Dinh Cuong Table of Contents INTRODUCTION 12 CHAPTER 1: INTRODUCTION TO CLOUD COMPUTING AND STORAGE AS A SERVICE (STaaS) 16 1.1 Definition of Cloud Computing 16 1.1.1 Evolution and History of Cloud Computing 16 1.1.2 Essential Characteristics of Cloud Computing 18 1.1.3 Requirements for Cloud Services 19 1.1.4 Pros and Cons of Cloud Computing 21 1.1.5 Cloud Ecosystem 23 1.1.6 Cloud Service Providers 23 1.2 Deployment Models 27 1.2.1 Public Cloud 27 1.2.2 Private Cloud 28 1.2.3 Hybrid Cloud 29 1.2.4 Community Cloud 30 1.3 Service Models 30 1.3.1 Software as a Service (SaaS) 30 1.3.2 Platform as a Service (PaaS) 32 1.3.3 Infrastructure as a Service (IaaS) 34 1.3.4 Data as a Service (DaaS) 36 1.4 Introduction to Storage as a Service 38 1.4.1 Storage cloud 38 1.4.2 Storage as a Service 39 1.4.3 Forms of STaaS 40 1.4.4 Benefits of Storage as a Service 41 CHAPTER 2: AMAZON WEB SERVICES 43 2.1 Introduction to Amazon Web Services 43 2.1.1 AWS Infrastructure 43 2.1.2 Exploring AWS services 45 2.1.3 AWS Ecosystem 46 2.1.4 Benefits of AWS 47 2.2 Technologies surrounding Amazon Web Services 48 2.2.1 Amazon Elastic Cloud Compute (EC2) 48 2.2.2 Amazon Simple Storage Service (S3) 51 2.2.3 AWS Lambda (Serverless Compute) 54 2.2.4 AWS DynamoDB 55 2.2.5 AWS API Gateway 56 CHAPTER 3: IMPLEMENTATION AND ANALYSIS 59 3.1 Building a Private Cloud as Storage Cloud with ownCloud 59 3.1.1 AZWebPlus Case Study 59 3.1.2 ownCloud Architecture 59 3.2 Implementing a Serverless Web Application on AWS 64 3.2.1 Overview 64 3.2.2 Application Architecture 64 3.2.3 Setting up static web hosting 65 3.2.4 Managing user with Amazon Cognito 67 3.2.5 Serverless Service Backend with AWS Lambda 69 3.2.6 RESTful APIs 70 3.3 Results and Analysis 71 3.3.1 Total cost of owning private cloud 71 3.3.2 AWS Serverless without considering about servers 74 CONCLUSION 85 Research summary 85 Limitation 86 Future work 86 REFERENCES 87 A Books 87 B Research Papers 87 C Online Websites 88 List of Figures Figur Name Page e 1.1 Software as a Service 31 1.2 PaaS Architecture 33 1.3 Summary of Key differences 35 1.4 Benefits of DaaS 37 1.5 Storage as a Service 38 1.6 Cloud Storage 39 2.1 Popular AWS services 45 2.2 AWS Gateway Architecture 56 3.1 Storage Cloud 60 3.2 ownCloud has a single interface from which users can access, 62 sync and share files on any device, anytime, from anywhere 3.3 ownCloud home page 63 3.4 Uploading file to ownCloud 63 3.5 Managing users and group 64 3.6 Serverless Application Architecture 65 3.7 Setting static website hosting by S3 66 3.8 Accessing website by S3 endpoint 67 3.9 Cognito user pool stores user’s info 67 3.10 Verification code is sent by Amazon 68 3.11 Auth Token Key is generated by Amazon Cognito 68 3.12 Demo Application 69 3.13 Data is stored in DynamoDB 70 3.14 RESTful API is set by API Gateway 71 3.15 Cost Comparison of Commercial Cloud providers and ownCloud 73 3.16 Lambda Configuration 75 3.17 Lambda Online editor 75 3.18 Setting test event in Lambda 76 3.19 Lambda combining CloudWatch stores logs 76 3.20 Storing source code in Amazon S3 77 3.21 Amazon S3 Properties 77 3.22 Setting server access logging in S3 78 3.23 Logging directory in S3 78 3.24 Logging detail by Json format 79 3.25 Monitoring Read/Write Capacity 80 3.26 DynamoDB metrics 80 3.27 API Gateway integrated with Lambda function 81 3.28 Published URL by API Gateway 81 3.29 API Gateway controls access by Amazon Cognito 82 3.30 Managing users and groups 82 3.31 General setting 83 3.32 Password policy by Cognito 83 3.33 Setting app client 84 List of Tables Figur Name Page e 1.1 Software as a Service 79 1.2 PaaS Architecture 79 GLOSSARY Amazon Web Services A subsidiary of Amazon that (AWS) computing platforms to provides on-demand cloud individuals, companies and governments, on a metered pay-as-you-go basis Application A set of subroutine definitions, communication protocols, Programming and tools for building software In general terms, it is a set Interface (API) of clearly defined methods of communication among various components Community Cloud A collaborative effort in which infrastructure is shared between several organizations from a specific community whether managed internally or by a third-party and hosted internally or externally Cloud Computing The demand availability of computer system resources, especially data storage and computing power, without direct active management by the user Cloud Storage A model of computer data storage in which the digital data is stored in logical pools Data as a Service (DaaS) An information provision and distribution model in which data files (including text, images, sounds, and videos) are made available to customers over a network, typically the Internet Hybrid Cloud Refers to the cloud infrastructure environment that is a mix of public and private cloud solutions Infrastructure as a Online Service (IaaS) to dereference various low-level details of underlying services that provide high-level APIs used network infrastructure like physical computing resources, location, data partitioning, scaling, security, backup etc NoSQL Provides a mechanism for storage and retrieval of data that is modeled in means other than the tabular relations used in relational databases Platform as a Provides a platform allowing customers to develop, run, Service (PaaS) and manage applications without the complexity of building and maintaining the infrastructure typically associated with developing and launching an app Private Cloud Refers to the cloud solution dedicated for use by a single organization Public Cloud Refers to the cloud computing model with which the IT services are delivered across the Internet RESTful API An application program interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data Serverless Compute A misnomer referring to a cloud-computing execution model in which the cloud provider runs the server, and dynamically manages the allocation of machine resources Software as a A Service (SaaS) which software is licensed on a subscription basis and is software licensing and delivery model centrally hosted in Figure 3.18: Setting test event in Lambda Figure 3.19: Lambda combining CloudWatch stores logs Amazon S3 is easy to use, with a simple web service interface to store and retrieve any amount of data from anywhere on the web Especially, it supports to control access logs Figure 3.20: Storing source code in S3 Figure 3.21: Amazon S3 Properties Figure 3.22: Setting server access logging in S3 Figure 3.23: Logging Directory in S3 Figure 3.24: Logging Detail by Json format Amazon DynamoDB is a fast and flexible NoSQL database service for all applications that need consistent, single-digit millisecond latency at any scale It assists users to monitor Figure 3.25: Monitoring Read/Write Capacity Figure 3.26: DynamoDB metrics Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale Figure 3.27: API Gateway integrated with Lambda function Figure 3.28: Published URL by API Gateway Figure 3.29: API Gateway controls access by Amazon Cognito Amazon Cognito allows developer add user sign-up, sign-in, manage all users and access control to your web and mobile apps quickly and easily With Amazon Cognito, developers easily set up password policy via graphic interface Figure 3.30: Managing users and groups Figure 3.31: General setting Figure 3.32: Password policy by Cognito Figure 3.33: Setting app client CONCLUSION Research summary Companies are increasingly building products that are the technology itself or heavily influenced by technology In order to be more competitive, companies must create better products, and to that, they must increase agility and innovate faster Modern application development is an approach to designing, building, and managing applications This approach increases the agility of teams and the reliability, security, and scalability of applications Through modern application development, enterprises can build better products faster so enterprises can have happier customers Serverless technologies simplify infrastructure management with serverless technologies Reducing time spent on routine environment management frees time to focus on business logic Serverless technologies let enterprises build applications without thinking about the underlying infrastructure Moving business applications to the AWS Cloud helps organizations simplify infrastructure management, deploy new services faster, provide greater availability, and lower costs Having a clear understanding of existing infrastructure and migration costs, and then projecting enterprise’s savings will help to calculate payback time, project ROI, and maximize the value of organization gains from migrating to AWS AWS delivers a mature set of services specifically designed for the unique security, compliance, privacy, and governance requirements of large organizations With a technology platform that is both broad and deep, professional services and support organizations, robust training programs, and an ecosystem that is tens-of-thousands of partners strong, AWS can help enterprises move faster and more In addition, IT department knows officer’s requirements well in advance and specify them clearly such as storing internally personal or utilities data They may end up with significantly lower total costs compared to current SaaS offerings and gain all the advantages that a private cloud solution offers Limitation The limitation of ownCloud in this project is security This solution does not include any anti virus applications that each data is uploaded to cloud storage, is not ensured clearly and scanned For instance, one user upload infected file to the cloud and others will download that file into their computers This case will push whole company in dangerous position even their clients Future work The project aims to apply Amazon EC2 and S3 in conducting cloud storage This means that ownCloud will be built on Amazon EC2 and user’s data is stored in Amazon S3 With ownCloud, Amazon EC2 and S3, the project will allow members access their data without limitations of location and storage size This target requires author must considers more about security, privacy and policy that protect effectively employees and stakeholders REFERENCES A Books [1] AT Velte, TJ Velte, RC Elsenpeter, RC Elsenpeter (2010) Cloud computing: a practical approach [2] B Furht, A Escalante (2010) Handbook of cloud computing Springer [3] Chao, L (2015) Cloud Computing Networking: Theory, Practice, and Development CRC Press [4] Chandrasekaran, K (2014) Essentials of Cloud Computing CRC Press [5] Erl T., Puttini R., Mahmood Z (2013) Cloud Computing: Concepts, Technology & Architecture Prentice Hall [6] Golden, B (2013) Amazon Web Services for Dummies John Wiley & Sons, Inc [7] Hurwits, J; Bloor, R.; Kaufman, M and Halper, F (2010) Cloud Computing for Dummies Wiley Publishing, Inc [8] Ikram Hawramani (2017) Cloud Computing for Complete Beginners: Building and Scaling High-Performance Web Servers on the Amazon Cloud Hawramani [10] Rehman, TB (2018) Cloud Computing Basics Mercury Learning and Information [11] R Buyya, J Broberg, AM Goscinski (2010) Cloud Computing: Principles and paradigms John Wiley & Sons, Inc B Research Papers [12] B Hayes (2008) Cloud computing Communications of the ACM, 2008 [13] Quick D., Martini B & Choo K (2013) Cloud Storage Forensics Elsevier Inc [14] Dillon, T., Wu, C., & Chang, E (2014) Cloud Computing: Issues and Challenges IEEE International Conference [15] Coyne, L & Dain, J (2018) IBM Private, Public, and Hybrid Cloud Storage Solutions IBM Redbooks [16] E Knorr, G Gruman (2008) What cloud computing really means InfoWorld [17] Gayatri, P., Venunath, M., V.Subhashini, & Umar, S (2018) Securities and threats of Cloud Computing and IEEE Xplore Compliant [18] MD Dikaiakos, D Katsaros, P Mehr (2009) Cloud Computing: Distributed internet computing for IT and scientific research IEEE [19] Mukherji, S., & Srivastava, S (2016) Pros and Cons of Cloud Computing Technology International Journal of Science and Research [20] N Leavitt (2009) Is Cloud Computing really ready for prime time? Computer [21] RL Grossman (2009) The case of cloud computing IEEE IT professional [22] T Dillon, C Wu, E Chang (2010) Cloud Computing: issues and challenges IEEE International Conference [23] Y Jadeja, K Modi (2012) Cloud computing - concepts, architecture and challenges IEEE International Conference C Online Websites [24] Amazon EC2 (n.d.) Retrieved 2018-09-20, from https://aws.amazon.com/ec2/ [25] Amazon S3 (n.d.) Retrieved 2018-09-20, from https://aws.amazon.com/s3/ [26] Amazon Virtual Private https://aws.amazon.com/vpc/ [27] Cloud Computing: Clash of the clouds The Economist (2009) Retrieved 2019-0103 from https://www.economist.com/briefing/2009/10/15/clash-of-the-clouds [28] Data as a Service (DaaS) Retrieved 2018-11-03 https://www.techopedia.com/definition/28560/data-as-a-service-daas [29] Data as a Service: The What, Why, How, Who, and When Retrieved 2018-11-03 from https://www.rocketsource.co/blog/data-as-a-service/ [30] Désiré Athow Hybrid cloud: is it right for your business? Retrieved 2018-09-20 from https://www.techradar.com/news/internet/cloud-services/hybrid-cloud-is-itright-for-your-business 1261343 Cloud (n.d.) Retrieved 2018-09-20, from from [31] [32] How does SaaS work? Retrieved 2018-09-25 from https://www.eztalks.com/videoconference/how-does-saas-work.html IaaS, PaaS, SaaS – IBM Cloud service models Retrieved 2018-09-20 from https://www.ibm.com/cloud/learn/iaas-paas-saas [33] Overview of Amazon Web Services AWS Whitepaper Amazon Web Services [34] Metzler, Jim; Taylor, Steve (2010) "Cloud computing: Reality vs fiction", Network World Retrieved 2018-11-20 from https://www.networkworld.com/article/2216572/cloud-computing reality-vs-fiction.html [35] Miller, Ron (2015) AWS Lambda Makes Reality TechCrunch Retrieved 2018-09-20 [36] Nextcloud vs ownCloud – The Whole Story CiviHosting Retrieved 2018-11-20 from https://civihosting.com/blog/nextcloud-vs-owncloud/ [37] Rouse, Margaret What is public cloud? Retrieved 2018-09-20 from https://searchcloudcomputing.techtarget.com/definition/public-cloud [38] The evolution of Cloud Computing Retrieved 2018-09-20 from https://cirrosolutions.co.uk/the-evolution-of-cloud-computing/ [39] SaaS vs PaaS vs IaaS: What’s the Difference and How to Choose Retrieved 201809-20 from https://www.bmc.com/blogs/saas-vs-paas-vs-iaas-whats-thedifference-and-how-to-choose/ Serverless Applications A [40] Steven J Vaughan-Nichols OwnCloud founder forks popular open-source cloud ZDNET Retrieved 2018-11-20 from https://www.zdnet.com/article/owncloud-founder-forks-popular-open-sourcecloud/ [41] Storage as a Service Retrieved 2019-05-20 https://searchstorage.techtarget.com/definition/Storage-as-a-Service-SaaS from [42] When to use Saas, PaaS and IaaS Retrieved 2018-09-20 https://www.computenext.com/blog/when-to-use-saas-paas-and-iaas/ from [43] When to use Saas, PaaS and IaaS Retrieved 2018-09-20 https://www.computenext.com/blog/when-to-use-saas-paas-and-iaas/ from [44] What is Cloud Computing? Amazon Web Services 2013-03-19 Retrieved 201809-20 [45] What is SaaS? Microsoft Azure Retrieved 2018-10-22 https://azure.microsoft.com/en-in/overview/what-is-saas/ [46] Wittig, M., & Wittig, A (2016) Amazon Web Services Manning Publications Co from ... of Cloud Computing 16 1.1.1 Evolution and History of Cloud Computing 16 1.1.2 Essential Characteristics of Cloud Computing 18 1.1.3 Requirements for Cloud Services 19 1.1.4 Pros and Cons of Cloud. .. affect and effect of Private cloud computing and Amazon Web Services OBJECTIVES The main research objective of this thesis is to “researching, implementing and managing an application on private cloud. .. importance of private cloud? Research Question 2: What exactly is Amazon Web Services based products? Research Question 3: How to implement software application on private cloud and Amazon Web Services?