August 7, 2014

Automation for Continuous Delivery: from Vagrant to P4Python

Continuous Delivery

continuous delivery

In this article I describe some of the steps we have been using internally to improve our Continuous Delivery (CD) pipeline for the Server Deployment Package (SDP) which is on the way to being open sourced. There are a couple of aspects of this which are relevant:

  • Using Vagrant for virtual machines (VMs)
  • Automating the configuration of the resulting VMs and building P4Python

A future article will cover the details of the SDP and the automated tests which we were able to run as a result.

Overview

The SDP scripts run on various flavors of Unix and also Windows. They provide best practice implementations of how to set up a Perforce server in a consistent way so that multiple instances can be easily managed. They also offer standard scripts for activities such as offline checkpointing, and regular recreation of database files to improve performance and standardize backup procedures.

As part of the move to CD, we wanted to include some automated tests which would run on different flavors of Unix, in the first instance Ubuntu and CentOS.

This required us to set up appropriate VMs for each operating system and then preconfigure them with standard tools required to run the tests. In this instance the key requirement was to build and install Python 3.3 and P4Python. All this needed to be easily reproducible and automated and thus fully scripted.

Using Vagrant

The Vagrant toolset has very rapidly become a key part of many development organization’s toolkit. From the web site:

  • Vagrant provides easy to configure, reproducible, and portable work environments built on top of industry-standard technology and controlled by a single consistent workflow to help maximize the productivity and flexibility of you and your team.

It is a command line based virtual machine management tool that makes it easy to create and provision virtual machines, all driven by a simple text file. If you haven't used it before be warned: simple VM management is addictive!

VirtualBox is an open source hypervisor made by Oracle. It runs on many platforms and is the default hypervisor for Vagrant (which can also work with other providers such as VMWare). You do not need to know much about Vagrant to use these tools. We'll explain the few commands you will likely need below.

For our tests we sync a simple workspace to pull down various files, including appropriate p4d and other binaries, and from a command prompt we just run ‘vagrant up’. This tells Vagrant to look for a Vagrantfile (in the local directory) and create the VM (or multiple VMs) specified in it. It will automatically download the required VM base images (Ubuntu 12.04 64-bit and CentOS 6.4 for us) and spin up new VMs. This base image download only occurs once. As part of spinning up the VMs, it executes bootstrap shell scripts.

Vagrant is written in Ruby and the Vagrantfile itself is Ruby, which makes it quite powerful should you need to get more creative.

A simple Vagrantfile is:

VAGRANTFILE_API_VERSION = "2"
domain = "example.com"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|  
  # Set the root password to 'vagrant'
  config.vm.provision "shell", inline: "echo 'root:vagrant' | sudo chpasswd"
  config.vm.define :mycentos do |mycentos|
    mycentos.vm.box = "centos64-x64"
    mycentos.vm.box_url = "http://puppet-vagrant-boxes.puppetlabs.com/centos-64-x64-vbox4210.box"
    mycentos.vm.provision :shell, :path => "bootstrap-centos.sh"
    mycentos.vm.network :private_network, ip: "10.10.50.2"
    mycentos.vm.hostname = "mycentos" + "." + domain
    mycentos.hostmanager.aliases = %w(mycentos.localdomain mycentos)
  end
end

Running ‘vagrant up’ pulls down the base box from the referenced URL and installs it locally with the name ‘centos64-x64’ – any other vagrant commands on the same host will use the locally cached box.

Note the reference to bootstrap-centos.sh which we explore below.

Other common vagrant commands are:

  • Vagrant ssh – ssh’s you into the VM so that you can work on the machine
  • Vagrant halt – stops the VM – saving its state
  • Vagrant destroy – stops and removes the VM – the next ‘vagrant up’ will re-provision it from the base box

And it is very common for a local directory to be mapped into the VM, for example ./sdp is mounted as /sdp within the VM so that any changes within the host to files are immediately reflected in the VM. This allows you to use your favorite editor on the host and see changes.

Bootstrapping our VMs

Both of the boxes we referenced require various packages to be installed or updated in order to be useable for our purposes. This just requires standard commands appropriate to your Linux distribution to install things like compilers, and then for us to build P4Python.

The bootstrap for CentOS does:

sudo yum update -y
sudo yum groupinstall -y "Development tools"
sudo yum install -y zlib-devel bzip2-devel openssl-devel ncurses-devel 
     sqlite-devel readline-devel tk-devel gdbm-devel db4-devel libpcap-
     devel xz-devel
echo /usr/local/lib>> sudo /etc/ld.so.conf

The equivalent for Ubuntu is:

sudo apt-get update
sudo apt-get install -y build-essential

Building Python 3.3

Python 3.3 is pretty easy. For CentOS:

wget -q http://www.python.org/ftp/python/3.3.5/Python-3.3.5.tar.xz
tar xJf ./Python-3.3.5.tar.xz
cd ./Python-3.3.5
./configure --prefix=/usr/local --enable-shared LDFLAGS="-Wl,-rpath 
     /usr/local/lib"
make && sudo make altinstall

For Ubuntu:

wget -q http://www.python.org/ftp/python/3.3.5/Python-3.3.5.tar.xz
tar xJf ./Python-3.3.5.tar.xz
cd ./Python-3.3.5
./configure
make && sudo make install

Building P4Python

This step is basically the same on both distributions – and shows how easy it is to automatically build P4Python.

cd /tmp
mkdir p4python
cd p4python
wget -q ftp://ftp.perforce.com/perforce/r14.1/bin.linux26x86_64/p4api.tgz
tar xzf p4api.tgz
wget -q ftp://ftp.perforce.com/perforce/r14.1/bin.tools/p4python.tgz
tar xzf p4python.tgz
P4PYTHON_PATH=`find /tmp/p4python/ -name "p4python-*"`
cd $P4PYTHON_PATH
API_PATH=`find /tmp/p4python/ -name "p4api-*" -type d`
mv setup.cfg setup.cfg.bak
echo [p4python_config] > setup.cfg
echo p4_api=$API_PATH>> setup.cfg
sudo /usr/local/bin/python3 setup.py install

Though if you check the scripts you will note that CentOS needs the last line to refer to python3.3.

In a later article I will cover how to easily build your own Windows versions of P4Python in a similar (automated) fashion.

Summary

And that is pretty much it! The actual versions of the scripts (in the Workshop) have a few tweaks:

  • The Vagrantfile is configured via a JSON script to make it easier to have multiple local VMs (we want to have a master and replica instance on Ubuntu and the same for CentOS)
  • We use several Vagrant plugins (vagrant-hostmanager, vagrant-cachier, vagrant-vbguest) which make life rather easier
  • We create a user perforce on the VMs and give it appropriate permissions to run the SDP tests – not shown.

The automated tests use ‘vagrant ssh’ to run a few commands and then parse the output. A simple Jenkins job configuration automates those tests on both Linux distributions. This will be part of the open sourcing of the SDP.