Automation in cloud age

Abstract

Any organization today will end up with more than a few virtualisation and cloud solutions when they round out their final enterprise infrastructure. This typically means a private or public clouds and other development systems. These multi-location deployments are now common setup. Cloud resources can be managed using their native interfaces and consoles. This approach won't scale. There are too many parts to consider and operations without the advantage of automation will not be effective over the long term. Other approach is to remove yourself from having to deal with these different interfaces through  use of an abstraction layer. As a result organisation can easily provision and manage manage heterogeneous information systems across many types of virtualization and cloud solutions.

Outline

  • Introduction
    • Cloud computing
    • Continuous integration
  • Cloud and virtalization management
    • AWS, OpenStack, VMware - Describtion of different cloud and virtualization platforms, their history, consoles and APIs.
    • Libcloud, libvirt, vagrant - Approches to control virtualization and clouds, how support for each platforms was implemented.
  • Configuration management
    • Puppet, Salt, Ansible - Ways to automate information system architecture configuration. How modern configuration management tools provide access to
  • Process management
    • Jenkins, Foreman - Putting the pieces tohgether in controlling automation process.
  • Automation scenarios
    • Auto-provisioning
    • Multi-cloud architectures
    • Automated backups
    • Automated recovery

Introduction

Cloud computing

Cloud Computing can be defined as "Computing in an independent or remote location with shared resources which are available on demand" (1). Cloud computing is new delivery and consumption model for IT services. It involves provision of dynamically scalable and often virtualized resources typically over the Internet. It may take form of web-based application that users access through a web browser as they would access a program installed locally on their computer. In Cloud computing environment everything becomes a Service. Primary motivation for organizations to move to Cloud computing is the reduction of cost by dynamic resource allocation. Underlying infrastructure is hosted by the cloud provider, so cloud computing consumers do not need to worry about the maintenance of that. Properties (5) like Scalability, Elasticity, Multi-tenancy, Pay-per-use make Cloud computing one of the most wanted technology today. The most obvious areas of concerns for organizations using Cloud computing are security, price and reliability, because infrastructure is managed entirely by an external entity.

It’s necessary to understand the main architectural layers (1) of Cloud Computing service providers and roles that work with different layers. Hardware infrastructure is the actual Cloud itself with various servers, storage and network devices. Theoretically a Cloud consists of all types of servers to host practically any application or service. On top of hardware infrastructure lays virtualized infrastructure which provides resources to support actual service layer of the Cloud used by the consumers. The last service layer provides final services to cloud consumers.

Continuos integration

Cloud and virtalization management

  • AWS, OpenStack, VMware - Describtion of different cloud and virtualization platforms.
  • Libcloud, libvirt, vagrant - Approches to control virtualization and clouds.

Cloud providers

First to provide Cloud Computing was Amazon.

Cloud clients/controllers

For managing  many of the popular cloud service providers using a unified API we can use libraries like LibCloud for python or haze for ruby. Using client controller library can help avoid vendor lock-in, because with standard library we can talk to many different providers using the same API. LibCloud library can provide management access to Compute, Storage LB and DNS.

Configuration management

  • Puppet, Salt, Ansible - Ways to automate information system architecture configuration.

 

Ansible
Combines multi-node deployment, ad-hoc task execution, and configuration management in one package. Manages nodes over SSH and does not require any additional remote software (except python 2.4 or later)[98] to be installed on them. Modules work over JSON and standard output and can be written in any language. Uses YAML to express reusable descriptions of systems.

Process management

  • Jenkins, Foreman - Putting the pieces tohgethet in controlling automation process.

Automation scenarios

Auto-provisioning

Quick definition of application deployment architectures, where cloud controller provisions desired servers and configuration management completes their configuration.

Multi-cloud architectures

Automatic distribution of infrastructures across availability zones, regions or cloud services. Distribution of load across infrastructure components hosted in one or more virtualization or cloud services.

Automated Backups

Snapshots every servvice involved in architecture are captured on defined schedule. Snapshots are stored in  third party cloud storage services for increased disaster avoidance. Snapshots can be encrypted for increased security.

Automated Recovery

Automatically restart or replace failed service components of your infrastructure based on policies you set.
This utilizes disaster recovery to restore data from last backup in case of a complete system failure.