Ansible

Ansible seems to be the simplest tool to centrally manage systems of any kind, Docker containers, AWS, Google Compute, Rackspace, OpenStack instances, VMware VMs, etc… There isn’t any dependencies on the managed system apart from Python 2.6. Ansible isn’t using any database or daemon and won’t install anything on the managed system, all operations are executed using SSH.

In this article, I’ll details how to install it will introduce the main concepts and terminologies. You’ll then understand why companies like Evernote, Twitter, Nasa, Rackspace or Atlassian are all using this Configuration Management, deployment and orchestration tool Compared to other solutions like Chef, Puppet or SaltStack, Ansible is designed to be minimal in nature with low learning curve.

Last but not least, Ansible seems to be inspired by Ursula K. Le Guin communication device capable of instantaneous or superluminal communication.

SSH Keys

Ansible use SSH by default to communicate with remote hosts. So if you don’t have any SSH keys at hand, generate yours now

$ ssh-keygen -t rsa -C "your_email@somedomain.com"

And copy them to the node you want to manage

$ ssh-copy-id -i ~/.ssh/id_rsa.pub root@<remote host IP addr>

Most of the cloud services can inject your public key at provisioning time, we’ll be using this mechanism instead.

Ansible doesn’t add another authentication layer (PKI) on top of the managed system, so no added complexity, which is great. It’s using what’s already there for day to day operation, nothing more, nothing less !

Installation

Ansible can easily be installed from pre-built packages, but to get the latest version and the greatest features let’s install it from source. This article assume you’re using a Debian based OS.

Python Dependencies

To install Ansible required Python packages, let’s use pip, so first make sure it’s there or install it

# apt-get install python-pip

Use pip to install the following required modules

# pip install paramiko PyYAML jinja2 httplib2 six

Installation from source

Clone the Git repositoty (19MB)

$ git clone git://github.com/ansible/ansible.git --recursive

If you prefer you can checkout a specific version using

$ git tag
$ git checkout v1.6.6

If you want to stay at the bleeding of Ansible, don’t forget to pull the devel branch. You can first check the diff.

$ git diff devel origin/devel
$ git pull

But you can also review the diff after the pull.

$ git diff HEAD@{1}

But this isn’t not a Git article, so let’s continue on.

$ cd ./ansible

run env-setup if you don’t have root access on your system to use it from the repository itself.

$ source ./hacking/env-setup

Or install it on your system [require python-dev]

$ sudo make install

If the above installation fails, install more dependencies and retry the above command

$ sudo apt-get install build-essential libssl-dev libffi-dev python-dev

Host inventory

/etc/ansible/hosts is an Hosts Inventory file that you have to create to start playing with Ansible, run the following commands as root to create this file:

# echo "127.0.0.1" >> /etc/ansible/hosts

This file will be used by Ansible to know the hosts where it can act.

If you don’t have root access on your system, you can store it in your homedir by setting ANSIBLE_HOSTS

$ export ANSIBLE_HOSTS=~/ansible_hosts

We’ll see later how to use Dynamic Inventory plugins instead of such a static INI style configuration file.

You can now run your first test which will check that Ansible can communicate with your inventoried nodes, localhost in our case.

$ ansible all -m ping --ask-pass

You should get

127.0.0.1 | success >> {
  "changed": false, 
  "ping": "pong"
}

Easy isn’t it ?

Ansible upgrade

If you want to upgrade from source:

# git pull
# git checkout v1.8.4
# git submodule update --init --recursive
# make install

Check it worked

# ansible --version

Other installation methods

You’ll find other ways to install your Ansible control node in the official documentation.

Inventory

/etc/ansible/hosts is the default path of Ansible Host inventory. It contains an INI formated configuration.

[lb]
haproxy.yet.org

[web]
wiki1.yet.org
wiki2.yet.org

[web:vars]
listen_port=80

[database]
db1.yet.org:2020 ansible_ssh_user=admin ansible_ssh_private_key_file=/home/admin/.ssh/id_rsa
db2.yet.org:2020

[yet:children]
web
database

ansible_ssh_user=admin variables can be set with hosts declaration.
:2020 is used to change SSH Port to connect to on that host.
[web:vars] declare groups variables
:children syntax declare a group of groups.

Tasks

Playbook Tasks are the smallest operation Ansible can do, it’s expressed in YAML. Tasks calls Modules, currently Ansible have over 230 of them that do anything from installing packages to restart services.

tasks:
  - name: Ensure Apache is installed
    apt: pkg=apache2 state=latest

  - name: wait user confirmation
    pause: prompt="Press ENTER if you are ready to update Apache config and restart or CTRL-C to quit."

  - name: configure apache
    copy: src=files/httpd.conf dest=/etc/httpd/conf/httpd.conf
    notify: restart apache

  - name: Wait for Apache to start
    wait_for: port=80 state=started

pause instruct the Playbook to wait for user confirmation and can also be used to wait for some seconds (seconds=30 instead of prompt)
wait_for continuously poll for the specified TCP Port and wait it respond before continuing Playbook execution.

Modules

Modules are small program that run on remote host and ensure the given host is in a particular state.

You can get the full list of supported modules with -l

$ ansible-doc -l

ansible-doc <MODULE NAME> will give you a detailled information about a module. Ansible docs is great by the way.

$ ansible-doc nova_compute

Playbooks

Playbooks are the Ansible way of expressing the configurations using human readable and machine parseable YAML file.

Playbooks combine the execution of multiple Ansible modules to perform application rollouts.

To execute a Playbook run

ansible-playbook -k -K boostrap.yml

-k (--ask-pass) ask for a password instead of assuming ssh-agent key based authentication
-K (--ask-sudo) ask for a password for sudo access

Playbooks are composed of three sections:

---
# This playbook deploys YET Blog on an OpenStack cloud. 
- name: Provision a Ubuntu 14.04 Instance on OpenStack
  hosts: localhost
  connection: local
  gather_facts: false

  roles:
     - { role: bootstrap, tags: [ 'bootstrap' ] }

- name: secure the instance and install yet.
  hosts: yet-blog
  user: "{{ ansible_ssh_user | default(root) }}"
  gather_facts: false
  sudo: yes

  roles:
    - base
    - yet

hosts: the Target section defines the hosts targeted by this Playbook. You can also define SSusername and SSH related config: sudo(sudo needed ?), user, sudo_user, connection(local/ssh), gather_fac(false, to avoid gathering facts if you’ve done it in an earlier Playbook)
vars: the Variable section defines the variable available to this Playbook.
tasks: the Task section list all modules in the order of execution.:

You don’t see all of this in our example above, we are instead leveraging Roles instead.

Roles

Roles brings a way to put together a set of configuration, templates, variables, … all in one little reusable package for different kind of things.

apache.yml
roles
  web
    defaults
    files
      apache2.conf
      httpd.conf
    handlers
      main.yml
    meta (reference to other roles)
    tasks
      main.yml
    templates
    vars
      ubuntu.yml
      centos.yml

They are stored by default in the /etc/ansible/roles directory.

For Rails developpers around, it should remind you the notion of convention over configuration. From now on, within your Playbooks you won’t need to specify where you store the different components as soon as you respect the above directory structure for your roles.

Now you can use them in your main infrastructure Playbook like this

---
- hosts: all
  roles:
    - security
    - monitoring
    - fileshare

- hosts: webservers
  roles:
    - nginx
    - php
    - drupal

- hosts: database
  roles:
    - postgres

Iterate

You can easily repeat a module several times using with_items which create a variable called item

- name: create directory structure
file: path=/var/{{ item.path }} mode={{ item.mode }} owner="www-data" group="www-data" state=directory
with_items:
- { path: 'www', mode: '0775' }
- { path: 'www/yet', mode: '0775' }

with_fileglob works the same way but iterate over a pattern matching file glob.

Conditions

a when clause condition the execution of the module to the value of it. The module will only execute if the Python expression evaluate to True.

Variables

Variables can be defined from facts or inventory variable using the following syntax:

{{ ansible_eth0.ipv4.address }}
{{ innodb_buffer_pool_size_mb|default(128) }}

Use vars_files: to load them from a file like this

vars_files:
  /conf/dc.yml
  /conf/rack.yml
  /conf/web.yml

Use prompt to wait for user input

vars_prompt:
  - name: 'password'
  prompt: 'Please type the root Password'
  private: yes

By using private, you make sure that the password won’t be printed on the screen.

Handlers

This section use the same syntax as the Tasks one, Handlers are only run when called from a task that changed something

handlers:
- name: restart dhcp
service: name=httpd state=restarted

Jinja2 templates

Jinja2 is a full featured template engine for Python which use the following syntax

{# #} encloses comments.
{% %} encloses conditional logic and other Python.
{% if %} {% endif %}
{% for %} {% endfor %}
[% %] encloses system facts according to directive from the first line of the template file.

To use templates you just have to call the template module. You aren’t supposed to use complex logic in your templates, prefer to use set_fact

set_fact: innodb_buffer_pool_size_mb="{{ ansible_memtotal_mb / 2 }}"

Using Vagrant with Ansible

Vagrant is great if you want to test your overall infrastructure on your local machine. You just have to define a provisioner, give it a Playbook and an Inventory File or it will use /etc/ansible/hosts

config.vm.provision "ansible" do |ansible|
  ansible.playbook = "playbook.yaml"
  ansible.inventory_path = "inventory-vagrant"
end

Using Ansible on Rackspace cloud

First install pyrax which is the client binding for Rackspace Cloud.

pip install pyrax

Now store your Rackspace cloud auth credentials in ~/.rackspace_cloud_credentials (other location can be specified in RAX_CREDS_FILE shell variable)

[rackspace_cloud]
username = <username>
api_key = <apikey>

Provisioning an Instance using Rax module

Here is an example of YAML file which will create a new instance on Rackspace cloud

---
- name: Launch an Instance on Rackspace Cloud
  hosts: localhost
  connection: local
  gather_facts: False

  tasks:
    - name: Server build request
      local_action:
        module: rax
        name: yet-nanoc
        group: yet-blog
        credentials: ~/.rackspace_cloud_credentials
        flavor: 2
        image: bb02b1a3-bc77-4d17-ab5b-421d89850fca
        wait: yes
        state: present
        region: DFW
        key_name: seebackupforprivate-dallas
        networks:
          - public
      register: rax

We are adding yet-blog in the group metadata to easily target this host using dynamic inventory (see below).

Dynamic Inventory

For convenient access and to potentialy run multiple inventory script at the same time, cp rax.py and all your needed scripts into an inventory folder.

mkdir ~/inventory
cp ~/ansible/plugins/inventory/rax.py ~/inventory

You can use the Inventory plugin like this

~/inventory/rax.py --list
~/rax.py --host yet-nanoc1

And you can now run a Playbook using Rackspace dynamic inventory

 ansible-playbook -i ~/inventory yet-followup.yaml

The above instance can easily be targeted using the following line in any Playbook.

Hosts: yet-blog

Vault Encrypted Files

Not all information should be stored as clear text, Ansible 1.5 brings encryption to its core with Vault. You can encrypt :

group_vars/ or host_vars/ inventory variables
variables loaded using include_vars or vars_files
Role or default variables
or passed to Ansible using -e @file.yml

To encrypt an existing file use

ansible-vault encrypt cloud_id.yml

You can then run the playbook like this, all encrypted files need to use the same password.

ansible-playbook yet.yml --ask-vault-pass

Use the following commands to edit, change the password or decrypt it

ansible-vault edit cloud_id.yml
ansible-vault rekey cloud_id.yml
ansible-vault decrypt cloud_id.yml

Note: make sure you set your EDITOR environment variable or the edit won’t work.

Summary

I’ve been playing with Chef for quite some time, but I’m always curious about new things. I’m quite happy with Ansible so far, it differentiate from any other Configuration Management tool with some unique key concepts:

It’s pretty small
It’s Agentless
No central server
No configuration database
Configurations are human readable YAML files
It’s using SSH for communicating with managed nodes.

For all these reason I think you should spend some time with Ansible, you won’t regret it.

yet.org