January 31, 2017

What a Difference a Space Makes

Version Control

I love the Python package manager pip. It used to be hard work to install the right packages for Python, especially when dealing with dependencies between different packages. One of the best features of pip is that it comes with a binary package format called wheels that allows me to deliver my binary extensions to Python in a very convenient way.

Alas, binary wheels are not available on Linux (yet) - something about binary incompatibilities between different Linux distributions. On Linux, pip will go and fetch the source code wheel and try to locally compile and install the package. This requires that the user installs all the prerequisites first, that is (at least on Ubuntu):

            python-dev

            build-essential

            libssl-dev

The latter is required if you want to build P4Python with SSL support. The ability to have SSL support available with a pip install was a new feature I added in P4Python 2016.1.

But how do I actually find the correct SSL library you ask, given the fact that I need at least OpenSSL 1.0.0 if SSL support is supposed to work with the P4API libraries?

I cheat.

SSL build support

I cannot know in general terms where you installed your SSL libraries, so I make an assumption that the command ‘openssl’ has to be in the path and that the executable version matches the installed libraries. Then I run:

            openssl version

and parse the output for the version string. On my Mac, this yields:

            OpenSSL 1.0.2d 9 Jul 2015

Looks good enough, so I proceed to add “-lssl” and “-lcrypto” to the build command and add the path to the libraries for good measure.

Now let’s do this on Linux Xenial (16.04) as well. The same command yields:

            OpenSSL 1.0.2g  1 Mar 2016

Notice the extra space between the version number and the date? This broke my little regular expression, with the result that the build script assumes there is no correct version of OpenSSL around – and therefore builds without it.

Solution and Workaround

I have fixed the build script now to accommodate the different forms of the version file, but you won’t see the result yet until I publish P4Python 2016.2 later this month. For now, you can help yourself with a little trick:

            sudo pip install --install-option=”--ssl” –install-option=”/usr/lib” p4python

The extra install options are passed to the build script (setup.py). Note that you have to specify –install-option twice, the second time with the path to your SSL libraries.

Incidentally, you can use the same trick to prevent ‘pip install p4python’ going off to the Perforce ftp site for every install. The reason the command does this is to download the latest version of the P4API libraries. If you have the libraries downloaded already and want to simply reference them during the build phase you can add these options to your pip command:

           --install-options=”--apidir” –install-options=”<path to P4API library>”

One day, or so they promised us, there will be binary wheels for Linux as well; then you can join the happy family of Windows and Mac users who can enjoy them already.

For now, you can wait for P4Python 2016.2, which will add support for Python 3.6 (among other small fixes), or you can use the workaround I just explained – unless you want to simply download the sources and build it yourself.

Happy hacking!