IT training platform as code ebook khotailieu

As Kubernetes is getting widely adopted, enterprise challenges have progressed from deployment of Kubernetes cluster to building and managing desired platform stacks on it Learn how Platform-as-Code approach offers a common language to create Kubernetes Native platform stacks in hybrid multi-cloud environments Introduction to Platform-as-Code CloudARK Inc Last Updated on 31 July 2019 Published on 19 March 2019 www.cloudark.io Table of Contents Kubernetes Native platforms Kubernetes API extensibility with Operators Evolution of as-Code systems Platform-as-Code approach Platform-as-Code Challenges Discovery of Custom Resource functionality Interoperability between Custom Resources KubePlus approach to Platform-as-Code Consistency across CRDs/Operators KubePlus API add-on Platform-as-Code tenets Kubernetes Native Composable Declarative Platform-as-Code benefits Hybrid multi-cloud application portability Common language between Devs and Ops Conclusion | Platform-as-Code eBook www.cloudark.io Kubernetes Native platforms Cloud Native technologies act as building blocks towards developing modern, scalable containerized microservices based applications The landscape of Cloud Native technologies is quite large and it includes various types of softwares like databases, API gateways, load balancers, key management systems, storage/networking systems, etc In this Cloud Native landscape, Kubernetes has emerged as one of the fastest growing projects and has become the de-facto standard for running containerized application workloads While the broader Cloud Native trends are essential, for enterprises that are adopting Kubernetes, it is more important to understand what subset of these Cloud Native technologies are best suited to work with Kubernetes so that they can build their Kubernetes platform layers using them In this regard, it is useful to split the Cloud Native technology landscape broadly into three categories: Figure 1: Kubernetes Native as subset of Cloud Native technologies ● ● ● Kubernetes Native technologies - These are tools/systems that are primarily built or distributed only for Kubernetes They don’t support any other container orchestration system Kubernetes accommodative technologies - These are tools/systems that embrace multiple container orchestration systems, Kubernetes being one of them Non-Kubernetes technologies are Cloud Native but don’t support Kubernetes An interesting aspect of Kubernetes is the ability to extend its control plane by adding new custom APIs This is done by writing ‘Kubernetes API Extensions’ Popular terms used in the community to refer to Kubernetes API extensions are ‘Custom Resource Definitions (CRDs)’ and ‘Operators’ CRDs/Operators are Kubernetes Native by design They work with Kubernetes CLI (‘kubectl’); they can be installed on a cluster with Kubernetes’s popular package manager Helm; and they seamlessly integrate with Kubernetes features such as RBAC, Service accounts, Audit logs, etc CRD/Operator has become the preferred mechanism to bundle and distribute third party platform softwares to run on Kubernetes Today, more than 400+ GitHub repositories of Kubernetes CRDs/Operators exist for variety of platform softwares, such as API gateways, relational databases (MySQL, Postgres), non-relational databases (MongoDB, Cassandra, Elastic), Volume backup/restore functions, SSL certificate management, application workloads (Spark, TensorFlow), etc Building a Kubernetes Native platform layer involves assembling and installing Kubernetes CRDs/Operators of required platform elements in a cluster Enterprises are finding this approach of building Kubernetes Native platforms attractive as it not only offers them control over the underlying Kubernetes layer, but also provides flexibility in choosing the desired platform elements in their platform layer Kubernetes API extensibility with Operators The control plane of a Kubernetes cluster is made of built-in controllers for built-in resources/APIs The builtin resources include, among others, Pod, Deployment, Service, Secret, Ingress, etc The built-in controllers continuously reconcile the state of a cluster by reacting to CRUD (create, read, update, delete) operations on these resources/APIs The resources are represented as YAML specifications and provide a declarative way for running and managing containerized workloads on a cluster | Platform-as-Code eBook www.cloudark.io An API extension (CRD/Operator) consists of a Custom Controller that monitors CRUD operations on Custom Resources/APIs and reconciles the state of the cluster For instance, one can write a CRD/Operator to run stateful databases like Cassandra on Kubernetes in which a new Custom Resource/API named ‘Cassandra’ and its Custom Controller is added to a cluster’s control plane The Custom Controller will handle CRUD operations on ‘Cassandra’ Custom Resource instances to manage Cassandra clusters on that Kubernetes cluster Following figure shows a glimpse of this rich CRD/Operator ecosystem of Custom Resources + Custom Controllers and how it enables extending Kubernetes API set Figure 2: Categories of Kubernetes APIs As shown in the figure, Kubernetes APIs can be broadly sub-divided into three categories: ● ● ● Built-in APIs - Built-in APIs consist of Kubernetes’s built-in resources and controllers As mentioned earlier, the built-in resources include Deployment, Pod, Service, Secret, ConfigMap, StatefulSet, PersistentVolume, ResourceQuota, etc As of 1.14 release of Kubernetes there are 57 such resources natively supported by Kubernetes Standardized APIs - Standardized APIs consist of Custom Resources and Custom Controllers around which some standardization has started to happen, such as monitoring (Prometheus) and Service Mesh Interface (SMI) Bespoke APIs - Bespoke API category consists of Custom Resources and Custom Controllers for which there are no standards These include most of the community Operators The Custom Resources and Custom Controllers shown in above figure are representative, picked from the 400+ Operators that exist today on GitHub Typically, DevOps team holds the responsibility of installing required Operators in a Kubernetes cluster to create a customized platform layer Once CRDs/Operators are installed, the task of application developers is to build required platform workflows leveraging this platform layer Application developers define their platform workflows as Kubernetes YAML definitions consisting of Custom and built-in resources Evolution of as-Code systems Let’s look at how Kubernetes API extensions affect the traditional notion of ‘as-Code’ systems (Refer to figure 3) Any ‘as-Code’ system is designed to offer a common language to provision a technology stack leveraging underlying APIs and resources of a cloud First generation systems like AWS CloudFormation focused on delivering this for a single cloud (AWS) Second generation systems like Terraform focused on abstracting underlying cloud APIs across multiple clouds, offering a common language to provision technology stack in multi-cloud environments | Platform-as-Code eBook www.cloudark.io Figure 3: Evolution of as-Code systems An important aspect of the first- and second-generation systems was that the set of underlying APIs that these systems wrapped was static and known apriori In the Kubernetes world, this is no longer the case as the set of control plane APIs in a Kubernetes cluster can be extended by installing CRDs/Operators in a cluster anytime Hence, for an as-Code system in Kubernetes world, the goal is no more to abstract the underlying cloud infrastructure APIs, but it is to enhance the customized platform layer made from CRDs/Operators by simplifying creation of platform workflows consisting of Kubernetes YAMLs of built-in and Custom Resources Such a ‘Platform as Code’ system thus enables creating technology stacks declaratively for hybrid multi-cloud environments Platform-as-Code approach Application developers define their platform workflows as Kubernetes YAML definitions of Custom and builtin resources Such an ensemble of Custom and built-in resources essentially represents ‘codified’ definition of platform workflows and hence can be referred to as Platform-as-Code Following figure presents this Platformas-Code (PaC) approach as a flow diagram Figure 4: Platform-as-Code To understand the PaC approach further, let us take a real-life example of a Kubernetes Native platform layer that supports creating multiple Moodle stacks Moodle is an open source e-learning software Each Moodle stack consists of - Moodle software, MySQL DB instance, Domain Name for that stack, and SSL certificate The high-level requirements for this custom platform layer are as follows: | Platform-as-Code eBook www.cloudark.io - Create multiple Moodle stacks on a Kubernetes cluster (one stack per tenant) Support ability to install ‘plugins’ on each Moodle stack Backup & Restore a Moodle instance For storage, Moodle uses both file system and a relational database Backup both (file system and database) - Provision SSL certificate from Let’s Encrypt for each Moodle instance In order to support above workflow actions, a Kubernetes native Moodle platform layer can be created using following CRDs/Operators Moodle Operator (for Moodle instance management) MySQL Operator (for MySQL database management) Stash Operator (for Volume backup/restore) Cert-Manager Operator (for SSL certificate management) NGINX Ingress Controller (for API gateway) DevOps Engineer installs these CRDs/Operators in a Kubernetes cluster and application developer leverages Kubernetes’s built-in and Custom Resources to realize the required platform workflows The Custom Resources made available by this platform layer are - Moodle, MysqlCluster, MysqlBackup, Restic, Recovery, Certificate The built-in resources that complement these Custom Resources include - Secret, PersistentVolumeClaim, PersistentVolume, Ingress The custom workflow actions that are supported by the Moodle platform layer include - creating multiple Moodle stacks, installing/deleting Moodle plugins on each stack, taking backups of MySQL database for a stack, restoring a database from backup, backup/restore of Moodle stack Volume, obtaining and installing SSL certificate per stack These actions are realized through Kubernetes YAML definitions leveraging above mentioned Custom and built-in resources - moodle.yaml, mysqlcluster.yaml, restic.yaml, mysqlbackup.yaml, recovery.yaml, secret.yaml, persistentvolumeclaim.yaml These YAML definitions are nothing but Platform-as-Code for a Moodle platform stack As an example, in the following figure two Kubernetes Custom Resources are shown – moodle.yaml and mysqlcluster.yaml The Moodle Custom Resource provides a declarative approach for managing plugins on a Moodle instance through its spec.plugins attribute For its database needs it uses MysqlCluster Custom Resource instance by referring to its Service resource in the spec definition (spec.mySQLServiceName property) Figure 5: Kubernetes Custom Resources – Moodle and MysqlCluster Platform-as-Code Challenges When application developers are leveraging Kubernetes built-in and Custom Resources to orchestrate required platform workflows, they face following key challenges: Discovery of Custom Resource functionality Today it is not easy for application developers to understand new functionality introduced by every new Custom Resource made available through CRDs/Operators installed in their clusters Kubernetes natively provides a way to discover information about Custom Resource Spec properties through ‘kubectl explain’ command However, for effectively using Custom Resources to realize workflow actions, broader usage | Platform-as-Code eBook www.cloudark.io information is needed, such as what higher-level actions are supported by a Custom Resource, how to perform such actions, what are the assumptions made by Operator developer in handling Custom Resource actions, etc Interoperability between Custom Resources Platform-as-Code process enables creating platform stacks and orchestrating platform workflows leveraging different Kubernetes Resources (Custom and built-in) For this, various Custom and built-in resources need to be bound/tied together in specific ways This may involve passing certain names, attributes or labels from one Custom Resource to another or to in-built resources For application developers it is not easy to figure out how to create this tie-in between different resources towards realizing specific workflows KubePlus approach to Platform-as-Code In order to address above challenges and simplify Platform-as-Code experience for application developers, two things are required: Consistency across different CRDs/Operators Tooling that helps with discovery and binding of Kubernetes resources Consistency across CRDs/Operators At CloudARK, we have created detailed guidelines to enable consistency and standardization across various community CRDs/Operators These guidelines describe how to design, implement, package and document Kubernetes CRDs/Operators for enabling Platform-as-Code experience We have analyzed over 100 open source CRDs/Operators for their conformance to these guidelines We also use these guidelines while curating community Operators for our customers KubePlus API add-on We have developed KubePlus API add-on to simplify discovery and use of Kubernetes Custom Resources towards building Platform as Code workflows Once installed in the Kubernetes cluster, this add-on offers new API endpoints (e.g ‘man’, ‘composition’) that enable application developers to query static and dynamic information about various Custom Resources available in the cluster Here is how KubePlus API add-on is used when realizing Platform-as-Code workflows for the Moodle platform mentioned above (detailed steps): • Binding between Moodle and Mysql Custom Resources: In order to bind to the underlying database provisioned by the MysqlCluster Custom Resource instance, the Moodle Custom Resource YAML definition needs the name of Service resource corresponding to the MysqlCluster Custom Resource instance to be specified in its 'mysqlServiceName’ Spec attribute This information and the format of the Service name can be found using the ‘man’ endpoint for Moodle and MysqlCluster Custom resources kubectl get raw "/apis/platform-as-code/v1/man?kind=Moodle" kubectl get raw "/apis/platform-as-code/v1/man?kind=MysqlCluster" • Interdependency between Moodle, Restic and Deployment Resources: In order to take backup of Moodle instance volume, the Deployment resource corresponding to that Moodle Custom Resource instance needs to be given a label and that label needs to be used in the Restic Custom Resource label selector The ‘man’ and ‘composition’ endpoints help here Use the ‘composition’ endpoint to find underlying Deployment resource corresponding to a Moodle Custom Resource instance The output of composition command will show all the Kubernetes resources that are created by the Moodle Operator as part of handling ‘moodle1’ instance of Moodle Custom Resource in the ‘ns1’ namespace This output enables us to find the Deployment resource for ‘moodle1’ instance to which label can be applied for taking its backup kubectl get raw "/apis/platform-as-code/v1/man?kind=Restic" kubectl get raw "/apis/platform-as-code/v1/man?kind=Moodle" kubectl get raw "/apis/platform-as-code/v1/composition?kind=Moodle&instance=moodle1&namespace=ns1" KubePlus API add-on currently provides information required to build interactions between Kubernetes Custom and built-in resources manually and thus define Platform-as-Code workflows We are now developing additional automation in this add-on to enable automatic binding and resolution between Kubernetes | Platform-as-Code eBook www.cloudark.io resources to realize different Platform-as-Code workflows Here is the on-going work for this that uses mutating web-hook admission controller to perform run-time discovery and binding between Custom Resources required for a Platform-as-Code workflow Platform-as-Code tenets The Platform-as-Code approach builds upon Kubernetes’s inherent declarative nature and leverages Kubernetes YAMLs to define end-to-end platform workflows Following are the key tenets of any Platform-asCode system A platform layer that satisfies these can deliver Platform-as-Code experience to its end users Kubernetes Native Kubernetes Native is about enabling all the key functionality under Kubernetes interfaces like kubectl or Kubernetes YAMLs, instead of introducing new proprietary interfaces or abstraction wrappers Composable Composability is about bringing in a level of consistency across Operators that simplifies consumption of Operators in multi-Operator settings Declarative Declarative nature is inherent to Kubernetes and its API constructs From Platform-as-Code perspective it is important to have Kubernetes Operators be designed in such a manner that they support various operations on platform softwares as declarative actions Platform-as-Code benefits Platform-as-Code approach enables enterprises to build Kubernetes-native, composable and declarative platform stacks which in-turn enables platform engineering teams to deliver more value, faster, to their end users It also offers more flexibility to choose required platform elements compared to any pre-built platform stack offering At a broad level, Platform-as-Code approach has the potential to influence following enterprise challenges and create a level of impact that technologies like Terraform have created in pre-Kubernetes era Hybrid multi-cloud application portability Due to composable and declarative nature of Platform-as-Code approach, enterprises can easily create their platform stacks repeatably on any Kubernetes cluster This significantly simplifies transfer or recreation of workloads across hybrid and multi-cloud environments It also enables the ability to lift and shift or burst workloads between private and public clouds Figure 6: Hybrid multi-cloud application portability | Platform-as-Code eBook www.cloudark.io Common language between Devs and Ops Figure 7: Platform-as-Code bridging the gap between Devs & Ops Platform-as-Code bridges the tooling gap between Developers and DevOps Engineers with Kubernetes YAML becoming the common language between them Over the years two types of tools have evolved – PaaSes and Infrastructure-as-Code PaaSes focused on delivering end-to-end application developer workflows Infrastructure-as-Code tools focused on simplifying Ops challenges in infrastructure provisioning This created tooling gap between development and DevOps teams due to adoption of different tools Platform-as-Code has the potential to bridge this gap by standardizing on Kubernetes YAML as a common language between Devs & Ops, especially around platform level functionality For instance, as mentioned in official Kubernetes case studies, ThredUp, an ecommerce company, has managed to deprecate their 3200 Ansible scripts (Infrastructure-as-Code) in favor of Kubernetes YAMLs/helm charts Conclusion Kubernetes API Extensions (also known as CRDs/Operators), are paving the way for a new era in platformization Platform-as-Code approach brings in standardization in composing custom platform layer using multiple CRDs/Operators It builds upon Kubernetes’s inherent declarative and API driven nature and leverages Kubernetes YAMLs to define end-to-end platform workflows As a result, enterprises get a Kubernetes-native, composable and declarative way of building their platform workflows Platform-asCode approach is disrupting the traditional PaaS industry by bringing in composability and addressing the inflexibility of pre-built platforms This is especially attractive in today’s era of containerization and Kubernetes where there is a large choice for building your platform technology stack At CloudARK, we offer purpose-built Kubernetes Native platforms to our customers based on the Platform-asCode approach We also offer customized training on Kubernetes extensibility, which has already been availed by some of the fortune 500 enterprises Get in touch and we will be excited to help you in your Kubernetes platformization journey | Platform-as-Code eBook ... ‘codified’ definition of platform workflows and hence can be referred to as Platform- as- Code Following figure presents this Platformas -Code (PaC) approach as a flow diagram Figure 4: Platform- as- Code To... taking its backup kubectl get raw "/apis /platform- as- code/ v1/man?kind=Restic" kubectl get raw "/apis /platform- as- code/ v1/man?kind=Moodle" kubectl get raw "/apis /platform- as- code/ v1/composition?kind=Moodle&instance=moodle1&namespace=ns1"... for a Platform- as- Code workflow Platform- as- Code tenets The Platform- as- Code approach builds upon Kubernetes’s inherent declarative nature and leverages Kubernetes YAMLs to define end-to-end platform