Automating IT-Infrastructure with Vagrant and Puppet

Setting up multiple development environments is a pain. Often, we have to deal with multiple settings, multiple library versions and overlapping dependencies for different projects. Vagrant is a tool written in Ruby for managing multiple development environments as virtual machines on your local machine. It manages all the steps from creating virtual images to launching them, syncing of local folders and port forwarding. In this blogpost I will discuss how to set up Vagrant together with the provisioning tool Puppet to create automated development environments, that can be checked in into version control. A working demo of the configuration of a simple webstack (nginx, php-fpm, mysql) can be found on Github.

What is Vagrant?

Vagrant is a brilliant tool for controlling your development virtual machines and wiring them with the host operating system. Directory syncing and port forwarding are just two of a variety of nice features, that boost your development work flow. With a simple vagrant up it creates for everyone the exact same developing environment, that can be checked in into version control together with a project and used by all the team members of the project.

Vagrant uses a provider to run the virtual machines; in our example this will be Virtualbox. For the creation of the environment Vagrant starts from a base box (e.g. a plain Ubuntu iso or a community box) and applies automated software changes via a provisioning tool on top of the base box.

Getting Started with Vagrant

To get started we need Ruby and RubyGems (Ruby’s package manager) installed.

sudo apt-get install ruby1.9 rubygems1.9

Next we install Virtualbox, that will be used as a provider for Vagrant.

sudo apt-get install virtualbox

Now we can install Vagrant. Therefore we grab the latest binaries from the Vagrant installation guide and install them.

We can create a virtual machine image with running vagrant init in a project’s folder. This will create a Vagrantfile, a file the stores the configuration of the wiring with the host operating system, provisioning methods and more configurations about the virtual machine.

mkdir my_app && cd my_app
vagrant init

To get detailed information of the configuration, please check the nice documentation of Vagrant. To start your virtual development environment run vagrant up.

vagrant up

After a few minutes, your virtual machine is ready and running. You can now ssh into it with the vagrant ssh command.

vagrant ssh

And what is Puppet?

Puppet is a provisioning tool for automating software installations and configurations written in Ruby. Puppet lets you describe the software changes (installing libraries, configuring services, manipulating files, etc.), that you want to apply to a system. This allows you to automatically make the exact same changes on multiple machines.

There exist a lot of other provisioning tools (Chef, Ansible, etc.) that can be also used with vagrant. I chose Puppet because its client for a single workstation (that’s most probably what we need when we use Vagrant) is simple in describing infrastructure while having a very clean and organized way to manage environments in the same git repository as the project. In particular, I am aiming for a project structure in small web projects like the following:

project_root
- env (Directory holds the development environments)
- web-app (Directory holds the main web application)
- email-worker (Various workers for different tasks)
- database-worker
- more-workers..

In the later example, we will use the tool Librarian-Puppet to put all the dependencies to a Puppetfile. We can then use it to install all the dependencies to the modules directory, so we do not have to include the modules directory to the git repository. The structure of the environment folder looks like this

project_root
- env
  - puppet (Directory holds all Puppet specific files)
    - modules (Puppet dependencies)
    - nodes
    - manifests
    - Puppetfile (File holds Puppet specific dependencies)
  - config.yaml (File holds all (common) configurations)
  - Vagrantfile (File holds Vagrant specific configurations)

In the nodes directory we will later store the configurations for the different services on our nodes and in the manifests directory we will place the configuration for a single machines, that defines which nodes are used on this machine.

Installing Puppet

We install Puppet and Librarian-Puppet via RubyGems.

gem install puppet
gem install librarian-puppet

We can now automatically install software on our machines. Therefore we can search on Puppet Forge or Github for existing modules, that we can add to our Puppetfile. Please check out the demo project on Github or the Puppet Documentations for more details.

The Vagrant Workflow

Now I will demonstrate the workflow with Vagrant on a sample web application. First, we need to set up the project, therefore we check out the demo application from Github.

git clone https://github.com/chaosmail/vagrant-puppet-webstack

We then navigate to the puppet directory and install the dependencies with Librarian-Puppet. This will pull the dependencies and place them into the modules directory.

cd vagrant-puppet-webstack/env/puppet
librarian-puppet install

In this example, we are using the hostmanager plugin from Vagrant, that automatically manages the configuration of our local hosts file. We can install it via Vagrant’s plugin manager.

vagrant plugin install vagrant-hostmanager

Now, we change to the env directory and run vagrant up. This will download the Ubuntu base box (when run the first time), sync the project’s directories, forward the ports and sets up and configures the webstack.

cd ..
vagrant up

That’s all. We can navigate to http://my-app.dev/ to see the webstack (and the application) running.

Use vagrant provision to apply changes of your Vagrantfile without restarting the virtual machine. Use vagrant suspend and vagrant resume the freeze and resume your virtual machine.

Use vagrant ssh to ssh into the development machine.