Provisioning CentOS images on EC2 with Ansible
What I'm going to discuss here is provisioning some CentOS images on EC2 instances with Ansible and well, do that from a CentOS machine of course.
First off, you don't need the AWS tools installed on your management system, and that saves you from having to install Java as well. We need the following components on our CentOS management system:
Ansible and its small list of dependencies. There's help in gettingstarted.
- The Euca2ools, command-line utilities for interacting with Amazon's EC2 and S3 services. I think of these as the answer to Amazon's tools, but they're written in Python. Furthermore, we'll need this and its prerequisites for Ansible as well. Euca2ools are in EPEL so installation is easy.
- A rather large collection of keys and authorization codes which you obtain from the AWS site. I won't bore you with how to do that, but I will show you a list of variables which must be correctly set for things to work.
You've set up your AWS account, and you've obtained authorization secrets to interact with EC2. For everything we do from here onwards, we need the following variables in our shell's environment:
export EC2_ACCESS_KEY="xxxxxxxxxxxxxxxxxxxx"
export EC2_SECRET_KEY="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
export EC2_URL=https://ec2.amazonaws.com
export S3_URL=https://s3.amazonaws.com:443
export AWS_ACCESS_KEY_ID=${EC2_ACCESS_KEY}
export AWS_SECRET_ACCESS_KEY=${EC2_SECRET_KEY}
1. Ansible inventory
Ansible uses an inventory in which I describe the machines I want it to speak to, the groups they belong to and specific variables I want those machines to use. The inventory file defaults to /etc/ansible/hosts, but I can override that by setting $ANSIBLE_HOSTS to a different path. An inventory file can be as short as this, and believe it or not, this is actually the inventory we're going to give to Ansible in order to launch EC2 instances:
[local] 127.0.0.1
On the other hand we need to provision EC2 instances running on the other side of the world (for me at least). How do we do that? How do we know their hostnames?
There's an aws_ec2.py inventory script for Ansible which enumerates the EC2 instances we can access. If I launch this program, I see the following JSON output because I already have an instance running. (Compare the instance ID and the public hostname to what we saw earlier.)
{
"i-df75cea0": [
"ec2-54-242-141-105.compute-1.amazonaws.com"
],
"key_jp1": [
"ec2-54-242-141-105.compute-1.amazonaws.com"
],
"security_group_default": [
"ec2-54-242-141-105.compute-1.amazonaws.com"
],
"type_m1_small": [
"ec2-54-242-141-105.compute-1.amazonaws.com"
],
"us-east-1": [
"ec2-54-242-141-105.compute-1.amazonaws.com"
],
"us-east-1a": [
"ec2-54-242-141-105.compute-1.amazonaws.com"
]
}Even though we have a single machine only, it shows up in different groups. These groups will allow us to target specific groups of instances when we use Ansible to provision them. (Note: the ec2.py program caches its output in configurable paths, so it may take a minute until the list is refreshed.) If I invoke the inventory program with a specific host, I get a list of variables particular to that instance: (I'm omitting lots of output for brevity)
ec2.py --host ec2-54-242-141-105.compute-1.amazonaws.com
{
"ec2_architecture": "x86_64",
"ec2_dns_name": "ec2-54-242-141-105.compute-1.amazonaws.com",
"ec2_hypervisor": "xen",
"ec2_id": "i-df75cea0",
"ec2_image_id": "ami-8a8932e3",
"ec2_instance_type": "m1.small",
"ec2_ip_address": "54.242.141.105",
"ec2_key_name": "jp1",
"ec2_launch_time": "2012-11-18T10:57:20.000Z",
"ec2_monitored": false,
"ec2_placement": "us-east-1a",
"ec2_root_device_type": "ebs",
"ec2_security_group_ids": "sg-29652b41",
"ec2_security_group_names": "default",
"ec2_virtualization_type": "paravirtual"
}Using the ec2.py inventory script will allow Ansible to interact with instances on EC2. This happens either by installing the file as an executable /etc/ansible/hosts or by pointing $ANSIBLE_HOSTS to that executable.
Will it "ping"?
$ export ANSIBLE_HOSTS=~/ec2.py
$ ansible -u root ec2-54-242-141-105.compute-1.amazonaws.com -m ping
ec2-54-242-141-105.compute-1.amazonaws.com | success >> {
"changed": false,
"ping": "pong"
}At this point I could leave you to it, and you could successfully use Ansible to install and configure your EC2 instances. But I won't leave you to it: let's do a bit of provisioning.
2. Ansible launches a CentOS instance on EC2
Ansible's ec2 module creates an instance on EC2 and optionally waits for that instance to become ready. (Note: ready doesn't mean booted -- that can take a few minutes.) Upon creating an instance, I specify the SSH keypair I want to use (we created a key called jp1 for that), the image name, and a few other parameters which are described in the module's documentation. One parameter I want to point out is called group. This is a so-called _security group_ which, as far as I've been able to determine, specifies e.g. firewall rules from the AWS point of view. By default only port 22 (SSH) is allowed into my instance, but I'm creating Web servers so I also want (at least) port 80.
To create an EC2 security group, I used the following commands:
$ euca-add-group -d "Web Servers" webs-a $ euca-authorize -P tcp -p 80-80 -s 0.0.0.0/0 webs-a $ euca-authorize -P tcp -p 22-22 -s 0.0.0.0/0 webs-a
The specified security group later on shows up as a group in the Ansible inventory (ec2.py).
---
- hosts:
- 127.0.0.1
connection: local
gather_facts: False
vars:
keypair: jp1
instance_type: m1.small
security_group: webs-a
image: ami-8a8932e3
mail_from: Ansible
mail_to: charlie
vars_prompt:
shortname: "What is the shortname of this host to be?"
tasks:
- name: Launch new EC2 instance
local_action: ec2
keypair=${keypair}
group=${security_group}
instance_type=${instance_type}
image=${image}
wait=true
register: ec2
#
- name: Send e-mail to admins
local_action: mail
from=${mail_from}
to=${mail_to}
subject="EC2 instance ${ec2.instances[0].id}"
body="EC2 instance ${ec2.instances[0].id} created on ${ec2.instances[0].public_ip}"The variable ec2, obtained by registering the result of the ec2 module, contains the following values, which I use in the e-mail I fire off to the admins:
{
"instances" : [
{
"public_ip" : "107.22.159.172",
"id" : "i-3d1ba042"
}
],
"changed" : true
}Let me run Ansible on this playbook. Note that I use the simple inventory containing just localhost, because these modules run on my Ansible management machine and not remotely.
$ ansible-playbook newinstances.yml What is the shortname of this host to be?: : web31 PLAY [127.0.0.1] ********************* TASK: [Launch new EC2 instance] ********************* changed: [127.0.0.1] TASK: [Send e-mail to admins] ********************* ok: [127.0.0.1] PLAY RECAP ********************* 127.0.0.1 : ok=2 changed=1 unreachable=0 failed=0
To recapitulate: so far Ansible used modules locally (i.e. on our management machine) to remotely create a CentOS instance on EC2 and to send the e-mail. I could now use some of the euca- tools to look and see what is happening.
Also: have a bit of patience: it can take a few minutes for the EC2 instance to actually come alive. And also: I have mail:
Date: Sun, 18 Nov 2012 12:51:07 +0100 (CET) From: Ansible@c6.ww.example.com Subject: EC2 instance i-3d1ba042 EC2 instance i-3d1ba042 created on 107.22.159.172
3. Ansible provisions CentOS instances
After a couple minutes of patience I use Ansible to actually provision the instance I just brought up. To illustrate, I'll just install an Apache Web server and a template, so nothing special.
---
- hosts:
- security_group_webs-a
user: root
connection: paramiko
gather_facts: false
tasks:
- name: Install | Apache
action: yum pkg=httpd state=installed
- name: Machine | Launch Apache service
action: service name=httpd state=started enabled=true
- name: Machine | Disable firewall (fixme)
action: service name=iptables state=stopped enabled=false
#
- name: Web | Install Web templates
action: template src=templates/index.j2 dest=/var/www/html/index.htmlThe hosts Ansible should act upon are specified by a group name as obtained by ec2.py, and I'm connecting as root because that's the user for which our jp1 SSH key has been injected into by EC2. After the playbook has run, I can use a Web browser to connect to the ec2-*.compute-1.amazonaws.com hostname or to its public IP address.
The instance is still waiting for us to do something with it. We tell Ansible to use a different inventory this time, i.e. the EC2 inventory, and launch the configuration playbook:
ANSIBLE_HOSTS=~/ec2.py ansible-playbook apache.yml PLAY [security_group_webs-a] ********************* TASK: [Install | Apache] ********************* changed: [ec2-107-22-159-172.compute-1.amazonaws.com] TASK: [Machine | Launch Apache service] ********************* changed: [ec2-107-22-159-172.compute-1.amazonaws.com] TASK: [Machine | Disable firewall (fixme)] ********************* changed: [ec2-107-22-159-172.compute-1.amazonaws.com] TASK: [Web | Install Web templates] ********************* changed: [ec2-107-22-159-172.compute-1.amazonaws.com] PLAY RECAP ********************* ec2-107-22-159-172.compute-1.amazonaws.com : ok=4 changed=4 unreachable=0 failed=0
Provisioning is complete, and the services are running.
4. Lessons
CentOS on EC2 is cool, and using Ansible to provision CentOS instances is also. There are many ways to accomplish this I suppose, and you might do it differently.