October 31, 2016

DevOps Digest 205: Getting a Working Build-per-commit CI Process

DevOps Digest

In last week’s issue,DevOps Digest 204: Configuring for Build Automation with Jenkins, we gave you pointers to avoid the extra headaches and to streamline your Jenkins configuration.

We have gradually built momentum for this final burst to establish the pipeline to build our sample application and to set up a basic, build-per-commit continuous integration process in Jenkins.

Launch the Jenkins dashboard and click the “Create New Jobs” link on the main page. The following page lets you specify the details of what you want to create. We named the job that will build our sample project “DevOpsSample”  and selected “Pipeline” as the job type.

jenkins create new jobs screenshot

In the past, if you wanted a nice-looking, horizontal, segmented pipeline display, you really had to know Jenkins and its plugin ecosystem well. Pipelines simplify this, so we use them here.

Click the “OK” button at the bottom of the browser. Click to navigate to the following page, which should look like this:

jenkins creating pipelines screenshot

Either click the “Pipeline” tab up at the top or scroll down to reveal the pipeline script for the project. We’ve already created a very simple script, which uses Windows batch files to get things done. We could make things more elegant; for now, we’re just powering through.

Familiar with the Groovy programming language or not, the basics are pretty simple. Jenkins uses a domain-specific language (DSL) to define stages in your pipeline, which are set off with the use of ‘stage’ followed by a name. Our pipeline presently has two: one to sync from Helix and another to build with MsBuild. The contents of each of our stages simply call Windows batch files to do the dirty work.

As you can see in the image above, the batch file for the first stage changes to the root folder where we’ve been working and runs a clean command. The batch file for the second stage changes to the same folder, calls the Microsoft-supplied batch file to configure our environment, and then invokes MsBuild to build our sample application. Since we verified that those commands work at the command line, we can exit back to the main Jenkins dashboard and click the clock icon to the far right to schedule our new job for a build.

Unfortunately, our build fails. Below, the red ball indicates the failed status of the last run and the thundercloud with lightning shows a trend, though admittedly it’s not much of a trend at this point. Don’t panic. It’ll get better, I promise.

Failed Build Jenkins screenshot

To figure out what went wrong, click the highlighted name of the job and drill down to its particulars. Below, notice that the pipeline failed at the sync phase. The second stage never even had a chance.

Click one of the build links under "Permalinks" for the last build to drill down further, then click the option on the left to see the console output. As you can see in the following screen snippet, the attempt to clean our working folder failed because it couldn’t connect to a Helix server named “perforce:1666”. This may be surprising, given that we’ve been using “localhost:1666” and didn’t have any trouble with the server when we did the clean from the command line earlier.

We highlighted this failure because of the point it makes: the user account Jenkins uses to execute the build is not the user account we’ve been using to install and configure it. This throws many would-be DevOps practitioners for a loop. Because of this, Jenkins supplies various mechanisms to help, not least of which being the Helix plugin we installed earlier. For now, let’s power through this error with an ugly hack.

We’re going to add a couple of commands to the first stage of our pipeline to manually set both the P4CONFIG and P4PASSWD environment variables for each run. Look at the new pipeline script below for the two set commands added to the Windows batch file.

It’s ugly, but there’s another valuable DevOps lesson here: sometimes it’s more important to get things working than it is to fully understand why they weren’t. For now, we can save the changes to the pipeline script, re-run the job, and get this welcome change to our dashboard:

Now, the blue ball on the left indicates success and the lightning icon is gone. Even though we used a hack, our second build was successful, so we achieved what we set out to do.

With that, we reached our first significant milestone. We configured our build machine, it works, and we have a sample project that we can build from our CI system. Next, we’ll automate that process, clean up some of our workarounds, and add to our CI process – but that’s a story for a continuous integration expert. And that’s just what you’ll be after we get done with Chapter 3. Stay tuned next week for Issue 301.

You Ask, We Answer

As previously mentioned, this is your roadmap to creating a successful DevOps pipeline. Don’t understand something? Just ask. Need to dive a little deeper? Send an email to info@perforce.com with your questions. Then, stay tuned for a live Q&A webinar at the end of this series. Get DevOps Digest Sent to your Inbox

You don’t need to remember to check back with us each week. Instead, get the digest delivered directly to your inbox. Subscribe to our 25-week DevOps Digest and we’ll get you where you need to go, one email at a time.

Catch up on DevOps Digest:

DevOps Digest 204: Configuring for Build Automation with Jenkins

DevOps Digest 203: Creating a New Application in Helix

DevOps Digest 202: Setting Up Your Environment for Success

DevOps Digest 201: Building a Modern Web Application Using Version Control

See Perforce Helix in Action!

Join us for a live demo every other Tuesday and see the best of Perforce Helix in 20 minutes. Save your spot!