Introducing microsoft azure

Microsoft Azure is Microsofts application platform for the public cloud. The goal of this article is to give you a foundation for understanding the fundamentals of Azure, even if you dont know anything about cloud computing.

http://azure.microsoft.com/enus/documentation/articles/fundamentalsintroduction-to-azure/#table-of-contents Introducing Microsoft Azure By rboucherLast updated: 10/01/2014 ContributorsEdit on GitHub In this article: Microsoft Azure is Microsoft's application platform for the public cloud The goal of this article is to give you a foundation for understanding the fundamentals of Azure, even if you don't know anything about cloud computing How to read this article Azure is growing all the time so it's easy to get overloaded The basic services are listed earlier in the paper Start with just those, then go through the additional services That doesn't mean you can't use just the additional services by themselves, but the basic services make up the core of an application running in Azure Give feedback Your feedback is important This article should give you an effective overview of Azure If it does not, tell us in the comments section at the bottom of the page Give some detail on what you expected to see and how to improve the article Table of Contents Basic Services - The Components of Azure - Management Portal - Compute - Data Management - Networking - Developer Services Additional Services - Identity and Access - Mobile - Backup - Messaging and Integration - Compute Assistance - Performance - Big Compute and Data -Media - Commerce Getting Started The Components of Azure Azure groups services into categories in the Management Portal and on various visual aids like the What Is Windows Azure Infographic The Management Portal is what you use to manage most (but not all) services in Azure This paper will use a different organization to talk about services based on similar function, and to call out important sub-services that are part of larger ones Figure: Azure provides Internet-accessible application services running in Azure datacenters Management Portal Azure has a web interface called the Management Portal that allows administrators to access and administer most, but not all Azure features Microsoft typically releases the newer UI portal in beta before retiring an older one The newer one is called the "Azure Preview Portal" There is typically a long overlap when both portals are active While core services will appear in both portals, not all functionality may be available in both Newer services may show up in the newer portal first and older services and functionality may only exist in the older one The message here is that if you don't find something in the older portal, check the newer one and vice-versa Compute One of the most basic things a cloud platform does is execute applications Azure provides these options: Virtual Machines gives you control over your own virtual machine, including the operating system Websites offers a range of applications, frameworks, and templates for you to build large, scalable web applications and presence websites quickly, and then efficiently manage development, test, and operations Cloud Services is a platform-as-a-service (PaaS) option tuned to create highly scalable and fault resistant applications, but with more flexibility than Websites Each of the Azure execution models has its own role to play You can use these technologies separately or combine them as needed to create the right foundation for your application The approach you choose depends on what problems you're trying to solve Azure Virtual Machines Figure: Azure Virtual Machines gives you full control over virtual machine instances in the cloud The ability to create a virtual machine on demand, whether from a standard image or from one you supply, can be very useful This approach, commonly known as Infrastructure as a Service (IaaS), is what Azure Virtual Machines provides Figure shows a combination of how an Virtual Machine (VM) runs and how to create one from a VHD To create a VM, you specify which VHD to use and the VM's size You then pay for the time that the VM is running You pay by the minute and only while it's running, though there is a minimal storage charge for keeping the VHD available Azure offers a gallery of stock VHDs (called "images") that contain a bootable operating system to start from These include Microsoft and partner options, such as Windows Server and Linux, SQL Server, Oracle and many more You're free to create VHDs and images, and then upload them yourself You can even upload VHDs that contain only data and then access them from your running VMs Wherever the VHD comes from, you can persistently store any changes made while a VM is running The next time you create a VM from that VHD, things pick up where you left off The VHDs that back the Virtual Machines are stored in Azure Storage blobs, which we talk about later That means you get redundancy to ensure your VMs won’t disappear due to hardware and disk failures It's also possible to copy the changed VHD out of Azure, then run it locally Your application runs within one or more Virtual Machines, depending on how you created it before or decide to create it from scratch now This quite general approach to cloud computing can be used to address many different problems Virtual Machine Scenarios Dev/Test - You might use them to create an inexpensive development and test platform that you can shut down when you've finished using it You might also create and run applications that use whatever languages and libraries you like Those applications can use any of the data management options that Azure provides, and you can also choose to use SQL Server or another DBMS running in one or more virtual machines Move Applications to Azure (Lift-and-shift) - "Lift-and-shift" refers to moving you application much like you'd use a forklift to move a large object You "lift" the VHD from your local datacenter, and "shift" it to Azure and run it there You will typically have to some work to remove dependencies on other systems If there are too many, you may choose option instead Extend your Datacenter - Use Azure VMs as an extension of your on-premises datacenter, running SharePoint or other applications To support this, it's possible to create Windows domains in the cloud by running Active Directory in Azure VMs You can use Azure Virtual Network (mentioned later) to tie your local network and your network in Azure together Websites Figure: Azure Websites runs a website application in the cloud without having to manage the underlying web server One of the most common things that people in the cloud is run websites and web applications Azure Virtual Machines allows this, but it still leaves you with the responsibility of administering one or more VMs and the underlying operating systems Cloud services web roles can this, but deploying and maintaining them still takes administrative work What if you just want a website where somebody else takes care of the administrative work for you? This is exactly what Azure Websites provides This compute model offers a managed web environment using the Azure Management portal as well as APIs You can move an existing website application into Azure Websites unchanged, or you can create a new one directly in the cloud Once a website is running, you can add or remove instances dynamically, relying on Azure Websites to load balance requests across them Azure Websites offers both a shared option, where your website runs in a virtual machine with other sites, and a standard option that allows a site to run in its own VM The standard option also lets you increase the size (computing power) of your instances if needed For development, Websites supports NET, PHP, Node.js, Java and Python along with SQL Database and MySQL (from ClearDB, a Microsoft partner) for relational storage It also provides built-in support for several popular applications, including WordPress, Joomla, and Drupal The goal is to provide a low-cost, scalable, and broadly useful platform for creating websites and web applications in the public cloud WebSite Scenarios Websites is intended to be useful for corporations, developers, and web design agencies For corporations, it's an easy-to-manage, scalable, highly secure, and highly available solution for running presence websites When you need to set up a Website, it’s best to start with Azure Websites and proceed to Cloud Services once you need a feature that’s not available in Websites See the end of the "Compute" section for more links that can help you to choose between the options Cloud Services Figure: Azure Cloud Services provides a place to run highly scalable custom code on a Platform as a Service (PaaS) environment Suppose you want to build a cloud application that can support lots of simultaneous users, doesn't require much administration, and never goes down You might be an established software vendor, for example, that's decided to embrace Software as a Service (SaaS) by building a version of one of your applications in the cloud Or you might be a start-up creating a consumer application that you expect will grow fast If you're building on Azure, which execution model should you use? Azure Websites allows creating this kind of web application, but there are some constraints You don't have administrative access, for example, which means that you can't install arbitrary software Azure Virtual Machines gives you lots of flexibility, including administrative access, and you certainly can use it to build a very scalable application, but you'll have to handle many aspects of reliability and administration yourself What you'd like is an option that gives you the control you need but also handles most of the work required for reliability and administration This is exactly what's provided by Azure Cloud Services This technology is designed expressly to support scalable, reliable, and low-admin applications, and it's an example of what's commonly called Platform as a Service (PaaS) To use it, you create an application using the technology you choose, such as C#, Java, PHP, Python, Node.js, or something else Your code then executes in virtual machines (referred to as instances) running a version of Windows Server But these VMs are distinct from the ones you create with Azure Virtual Machines For one thing, Azure itself manages them, doing things like installing operating system patches and automatically rolling out new patched images This implies that your application shouldn't maintain state in web or worker role instances; it should instead be kept in one of the Azure data management options described in the next section Azure also monitors these VMs, restarting any that fail You can set cloud services to automatically create more or fewer instances in response to demand This allows you to handle increased usage, and then scale back so you aren’t paying as much when there is less usage You have two roles to choose from when you create an instance, both based on Windows Server The main difference between the two is that an instance of a web role runs IIS, while an instance of a worker role does not Both are managed in the same way, however, and it's common for an application to use both For example, a web role instance might accept requests from users, then pass them to a worker role instance for processing To scale your application up or down, you can request that Azure create more instances of either role or shut down existing instances And similar to Azure Virtual Machines, you're charged only for the time that each web or worker role instance is running Cloud Services Scenarios Cloud Services are ideal to support massive scale out when you need more control over the platform than provided by Azure Websites but don’t need control over the underlying operating system Choosing a Compute Model The page Azure Websites, Cloud Services and Virtual Machines comparison (http://azure.microsoft.com/documentation/articles/choose-web-site-cloud-service-vm/ ) provides more detailed information on how to choose a Compute model Data Management Figure: Service Bus Topics allows multiple apps to post messages and other applications to subscribe to receive messages that meet a specific criteria Service Bus provides a Publish-and-subscribe mechanism called Topics and Subscriptions With publishsubscribe, an application can send messages to a topic, while other applications can create subscriptions to this topic This allows one-to-many communication among a set of applications, letting the same message be read by multiple recipients Service Bus Relay Scenarios Anytime your are setting up where there are many messages that are all important, but various downstream systems only need to listen to differing subsets of those communications, Service Bus Topic and Subscriptions are a good option BizTalk Services Figure:BizTalk Services provides the ability to transform XML messages formats in the cloud Sometimes you need connect systems which communicate using different messaging formats It’s common for business to have different database schemas and XML messaging formats, even when a common standard is available Rather than write a lot of custom code, you can use BizTalk Server on-premises to integrate various systems Azure BizTalk Services provides the same type of service, but in the cloud You can pay for only what you use and not worry about scale like you would have to on-premises BizTalk Services Scenarios Business-to-Business (B2B) interactions commonly require this type of translation For example, a company building airplanes needs to order parts from it's various parts suppliers It will have many parts suppliers Those orders should be automated to go directly from the airplane builders systems into the suppliers systems Neither business wants to change their core systems and message formats, and it's very unlikely that those formats are the same BizTalk Services can take messages and translate between the new formats both ways Either the airplane supplier can the work to translate or the various suppliers can, depending on who wants more control and the amount of translation needed Compute Assistance Azure provides assistance for services that not need to run all the time Scheduler Figure: Azure Scheduler provides a way to schedule jobs at a specific time for a specific duration Sometimes applications only need to run at a certain time On Azure, you can save money with this type of app instead of letting an application just keep running 24x7 waiting for data to process Azure Scheduler allows you to schedule when an application should run based on interval of time or a calendar It’s reliable and will verify that a process runs even if there are network, machine, and data center failures You use the Scheduler REST API to manage these actions When a scheduled alarm occurs, Scheduler sends HTTP or HTTPS messages to a specific endpoint or can put a message in a Storage Queue So you need to have your application either have an accessible endpoint or have it monitor a storage queue Then once it gets the message, it can perform whatever action it's programmed to Scheduler Scenarios Recurring application actions: As an example, a service may periodically get data from twitter and • gather the data into a regular feed Daily maintenance: Log processing or pruning, performing backups and other intermittently schedule • tasks • • Tasks that run at night Web applications tasks like daily pruning of logs, performing backups, and other maintenance tasks An administrator may choose to backup her database at 1AM every day for the next months, for example The Scheduler API allows you to create, update, delete, view, and manage job collections and scheduled jobs programmatically Performance Performance is always important for an application Applications tend to access the same data over and over One way to improve performance is to keep a copy of that data closer to the application, minimizing the time needed to retrieve it Azure provides different services for doing this Azure Caching Figure: An Azure application can cache data in memory and even split it up across many worker roles Accessing data stored in any of Azure's data management services-SQL Database, Tables, or Blobs-is quite fast Yet accessing data stored in memory is even faster Because of this, keeping an in-memory copy of frequently accessed data can improve application performance You can use Azure's in-memory Caching to this A Cloud Services application can store data in this cache, then retrieve it directly without needing to access persistent storage The cache can be maintained inside your application's VMs or be provided by VMs dedicated solely to caching In either case, the cache can be distributed, with the data it contains spread across multiple VMs in an Azure datacenter Azure has a number of different cache technologies that have shifted over time In the order they were introduced, there is a shared, in-role, managed and Redis cache The shared caching is an older technology and you shouldn’t create new implementations with it The Managed Cache has the same features of the In-Role cache, but as managed service outside of the Azure Management Portal The Redis Cache is in preview The Redis implementation has the largest number of features and is recommended when you write new caching code Azure Cache Scenarios An application that repeatedly reads a product catalog might benefit from using this kind of caching, for example, since the data it needs will be available more quickly The technology also supports locking, letting it be used with read/write as well as read-only data And ASP.NET applications can use the service to store session data with just a configuration change CDN Figure:Copies of a blob can be cached at sites around the world Suppose you need to store blob data that will be accessed by users around the world Maybe it's a video of the latest World Cup match, for instance, or driver updates, or a popular e-book Storing a copy of the data in multiple Azure datacenters will help, but if there are lots of users, it's probably not enough For even better performance, you can use the Azure CDN The CDN has dozens of sites around the world, each capable of storing copies of Azure blobs The first time a user in some part of the world accesses a particular blob, the information it contains is copied from an Azure datacenter into local CDN storage in that geography After this, accesses from that part of the world will use the blob copy cached in the CDN-they won't need to go all the way to the nearest Azure datacenter The result is faster access to frequently accessed data by users anywhere in the world CDN Scenarios It's common to use CDN with Media Services to deliver video worldwide Video is usually large and requires a lot of bandwidth Media Services is talked about elsewhere on this page Big Data and Big Compute HDInsight (Hadoop) Figure: HDInsight helps with the bulk processing of huge amounts of data For many years, the bulk of data analysis has been done on relational data stored in a data warehouse built with a relational DBMS This kind of business analytics is still important, and it will be for a long time to come But what if the data you want to analyze is so big that relational databases just can't handle it? And suppose the data isn't relational? It might be server logs in a datacenter, for example, or historical event data from sensors, or something else In cases like this, you have what's known as a big data problem You need another approach The dominant technology today for analyzing big data is Hadoop An Apache open source project, this technology stores data using the Hadoop Distributed File System (HDFS), then lets developers create MapReduce jobs to analyze that data HDFS spreads data across multiple servers, then runs chunks of the MapReduce job on each one, letting the big data be processed in parallel HDInsight is the name of the Azure's Apache Hadoop-based service HDInsight lets HDFS store data on the cluster and distribute it across multiple VMs It also spreads the logic of a MapReduce job across those VMs Just as with on-premises Hadoop, data is processed locally-the logic and the data it works on are in the same VM-and in parallel for better performance HDInsight can also store data in Azure Storage Vault (ASV), which uses blobs Using ASV allows you to save money because you can delete your HDInsight cluster when not in use, but still keep your data in the cloud HDinsight supports other components of the Hadoop ecosystem as well, including Hive and Pig Microsoft has also created components that make it easier to work with data produced by HDInsight using traditional BI tools, such as the HiveODBC adapter and Data Explorer that work with Excel High-Performance Computing (Big Compute) One of the most attractive ways to use a cloud platform is to run high performance computing (HPC) and other "Big Compute" applications Examples include specialized engineering applications built to use the industrystandard Message Passing Interface (MPI) as well as so-called embarrassingly parallel applications, such financial risk models The essence of Big Compute is executing code on many machines at the same time On Azure, this means running many virtual machines simultaneously, all working in parallel to solve some problem Doing this requires some way to resources and to schedule applications, i.e., to distribute their work across these instances Microsoft’s free HPC Pack and other compute cluster solutions can perform well in Azure, taking advantage of Azure compute and infrastructure services to add capacity on-demand to an on-premises compute cluster or run Big Compute applications entirely in the cloud Azure provides a range of VM instance sizes with different configurations of CPU cores, memory, disk capacity, and other characteristics to meet the requirements of different applications The recently introduced A8 and A9 instances work well for many compute intensive workloads, and parallel MPI applications in particular, because they have high speed, multicore CPUs and large amounts of memory In certain configurations the instances take advantage of a low-latency and high-throughput application network in the cloud that includes remote direct memory access (RDMA) technology for maximum efficiency of parallel MPI applications Azure also offers Big Compute application developers and partners a full set of compute capabilities, services, architecture choices, and development tools Azure supports custom Big Compute workflows involving specialized data workflows and job and task scheduling patterns that can scale to thousands of compute cores Media Figure: Media Services is a platform for applications that provide video and other media to clients around the world Video makes up a large part of Internet traffic today, and that percentage will be even larger tomorrow Yet providing video on the web isn't simple There are lots of variables, such as the encoding algorithm and the display resolution of the user's screen Video also tends to have bursts in demand, like a Saturday night spike when lots of people decide they'd like to watch an online movie Given its popularity, it's a safe bet that many new applications will be created that use video Yet all of them will need to solve some of the same problems, and making each one solve those problems on its own makes no sense A better approach is to create a platform that provides common solutions for many applications to use And building this platform in the cloud has some clear advantages It can be broadly available on a pay-as-yougo basis, and it can also handle the variability in demand that video applications often face Azure Media Services addresses this problem It provides a set of cloud components that make life easier for people creating and running applications using video and other media As the figure shows, Media Services provides a set of components for applications that work with video and other media For example, it includes a media ingest component to upload video into Media Services (where it's stored in Azure Blobs), an encoding component that supports various video and audio formats, a content protection component that provides digital rights management, a component for inserting ads into a video stream, components for streaming, and more Microsoft partners can also provide components for the platform, then have Microsoft distribute those components and bill on their behalf Applications that use this platform can run on Azure or elsewhere For example, a desktop application for a video production house might let its users upload video to Media Services, then process it in various ways Alternatively, a cloud-based content management service running on Azure might rely on Media Services to process and distribute video Wherever it runs and whatever it does, each application chooses which components it needs to use, accessing them through RESTful interfaces To distribute what it produces, an application can use the Azure CDN, another CDN, or just send bits directly to users However it gets there, video created using Media Services can be consumed by various client systems, including Windows, Macintosh, HTML 5, iOS, Android, Windows Phone, Flash, and Silverlight The goal is to make it easier to create modern media applications References For a more visual view of how Media Services works, download the Azure Media Services Poster Commerce The rise of Software as a Service is transforming how we create applications It's also transforming how we sell applications Since a SaaS application lives in the cloud, it makes sense that its potential customers should look for solutions online And this change applies to data as well as to applications Why shouldn't people look to the cloud for commercially available datasets? Microsoft addresses both of these concerns with Azure Marketplace and Azure Store Figure: Azure Marketplace and Azure Store let you find and buy Azure applications and commercial datasets and use them as part of your Azure applications The difference between the two is that Marketplace is outside of the Azure Management Portal, but the Store can be accessed from inside the portal Potential customers can search to find Azure applications that meet their needs Customers can search for commercial datasets as well, including demographic data, financial data, geographic data, and more When they find something they like, they can access it either from the vendor, directly through the Marketplace or Store web locations or in some cases from the Management Portal Applications can also use the Bing Search API through the Marketplace, giving them access to the results of web searches Commerce Scenarios SendGrid is an application in the Azure Store that allows you to send email It offers additional functionality like reliable delivery and statistics You can buy this application and related services rather then try to build such an infrastructure yourself Getting Started Now that you have the big-picture, the next step is to write your first Azure application Choose your language, get the appropriate SDK, and go for it Cloud computing is the new default get started now [...]... Azure SDK that provides basic support for any language, such as C++ These SDKs help you build, deploy, and manage Azure applications They're available either from www.microsoftazure.com or GitHub, and they can be used with Visual Studio and Eclipse Azure also offers command line tools that developers can use with any editor or development environment, including tools for deploying applications to Azure. .. the kind of scenario where Azure Tables makes sense Blobs Figure: Azure Blobs provides unstructured binary data Azure Blobs (again "Blob Storage" and just "Storage Blobs" are the same thing) is designed to store unstructured binary data Like Tables, Blobs provides inexpensive storage, and a single blob can be as large as 1TB (one terabyte) Azure applications can also use Azure drives, which let blobs... Figure: Azure Import / Export provides the ability to ship a physical hard drive to or from Azure for faster and cheaper bulk data import or export Sometimes you want to move a lot of data into Azure That would take a long time, perhaps days, and use a lot of bandwidth In these cases you can use Azure Import/Export, which allows you to ship Bitlocker-encrypted 3.5" SATA hard drives directly to Azure. .. Figure : Azure Traffic Manager allows you to route global traffic to your service based on intelligent rules If your Azure application is running in multiple datacenters, you can use Azure Traffic Manager to route requests from users intelligently across instances of the application You can also route traffic to services not running in Azure as long as they are accessible from the internet An Azure application... slow down your service communications Developer Services Azure offers a number of tools to help developers and IT Professional create and maintain applications in the cloud Azure SDK Back in 2008, the very first pre-release version of Azure supported only NET development Today, however, you can create Azure applications in pretty much any language Microsoft currently provides language-specific SDKs for... storage (which we talk about later) Azure SQL Database Figure: Azure SQL Database provides a managed relational database service in the cloud For relational storage, Azure provides the feature SQL Database Don't let the naming fool you This is different than a typical SQL Database provided by SQL Server running on top of Windows Server Formerly called SQL Azure, Azure SQL Database provides all of the... applications to Azure from Linux and Macintosh systems Along with helping you build Azure applications, these SDKs also provide client libraries that help you create software that uses Azure services For example, you might build an application that reads and writes Azure blobs, or create a tool that deploys Azure applications through the Azure management interface Visual Studio Online Visual Studio Online is an... you want to extend your datacenter into the cloud using Azure Virtual Machines and Azure Virtual Network, however, Azure Active Directory isn't the right choice Instead, you'll want to run Windows Server Active Directory in Virtual Machnes To let applications access the information it contains, Azure Active Directory provides a RESTful API called Azure Active Directory Graph This API lets applications... can use Import/Export to have Microsoft transfer large amounts of data stored in Azure Blob Storage to a storage device that you send and then have that device delivered back to a location you desire Because this will take some time, it's not a good option for disaster recovery It's best for archived data that you don't need quick access to Azure File Service Figure: Azure File Services provides SMB... kinds of applications need different kinds of data Because of this, Azure provides several different ways to store and manage data Azure provides many storage options, but all are designed for very durable storage With any of these options, there are always 3 copies of your data kept in sync across and Azure datacenter 6 if you allow Azure to use geo-redundancy to back up to another datacenter at least ... running Active Directory in Azure VMs You can use Azure Virtual Network (mentioned later) to tie your local network and your network in Azure together Websites Figure: Azure Websites runs a website... by Azure Websites but don’t need control over the underlying operating system Choosing a Compute Model The page Azure Websites, Cloud Services and Virtual Machines comparison (http:/ /azure .microsoft. com/documentation/articles/choose-web-site-cloud-service-vm/... you build, deploy, and manage Azure applications They're available either from www.microsoftazure.com or GitHub, and they can be used with Visual Studio and Eclipse Azure also offers command line