Multibranch pipeline setup

Multibranch Pipeline support allows Jenkins to automatically find branches or change reviews in Perforce and create Jenkins Pipeline Jobs to build them.

Perforce has many ways to represent a job or collection of related branches; in order to best support the different uses the P4 Plugin uses four implementations of SCMSource:

• Helix Branches
• Helix Streams
• Helix Swarm
• Git Connector

To create a Multibranch Pipeline click New Item from the main menu, provide a name and select Multibranch Pipeline from the list.

In the configuration you will need to add one of the Helix SCMSource implementations; under the Branch Source group select the appropriate implementation from the drop-down Add source menu.

Each SCMSource implementation will require a Credential, a set of Perforce paths specific to the implantation, an optional filter and Advance options (Populate and Repository Browsing behavior).

Source Configurations

Example configuration for each of the Multibranch SCMSource Implementations supported by the p4-plugin.

Helix Streams

A simple example configuration for Helix Streams:

Jenkins will now probe each child stream associated with the Include streams path //streams/... (effectively all the streams returned by the command p4 streams //streams/...), if the probe finds a Jenkinsfile in the stream then it creates a Pipeline Job and schedules a build.

You can provide a list of stream depots in the Include streams (separated by a new line) to include other streams in the search.

Example Include streams behaviors:

• //streams/mystream would only include mystream.
• //streams/... or //streams/* would include all of the streams in the //streams path.
• //streams/mystream... or //streams/mystream* would include //streams/mystream01, //streams/mystream02, //streams/mystream-new, and so on.

Helix Swarm

The MultiBranch configuration for Swarm requires the Perforce Credentials and the Swarm Project name, multiswarmtest is used for this example.

The P4 Plugin uses the Swarm API to query what branches are used in the Swarm Project and then probes for the Jenkinsfile to create the Pipeline build.

In addition to finding the branches in a Swarm project the P4 Plugin queries Swarm for pre and post commit reviews. The P4 Plugin will create a Pipeline Job for each open review and gather the pipelines in a Reviews tab separate to the default Branches tab.

Git Connector

Git Connector configures one or more Git repositories backed by a Helix Graph depot. Provide the Credentials and a list of one or more Graph (Git) Repository.

For example; include a Graph depot called //plugins with a Git Repository called p4-plugin.

The P4 Plugin the scans all Git Branches and open Pull Requests, building Pipeline Jobs as required. Due to the multi-multi relationship with repositories and branches the Job naming convention is {depot}.{repo}.{branch} for branches and {depot}.{repo}.{branch}.{ref} for pull requests.

Helix Branches

Helix branches are more complex to define as there is no 'project' definition to group the set of Perforce depot paths together.

In addition to the Credentials you need to specify the parent paths (separated by new lines) for the branches in the Include branches text box. The Helix Plugin for Jenkins will search one directory level deep for a Jenkinsfile (or defined item) using the last directory name for the branch name.

The folder name is used as a unique identifier so branch names must be unique. For example if you use the following mapping:

//depot/projects/projA/... 
//depot/projects/projB/...

and your directories are:

//depot/projects/projA/test
//depot/projects/projA/candidate
//depot/projects/projB/test
//depot/projects/projB/candidate

The following branches are used because test and candidate are not unique:

//depot/projects/projA/test
//depot/projects/projA/candidate

The Include filter can be used to filter the list of candidate branches found by the initial search.

For example, a depot might contain the following branches, but you only want to build the com* branches. Set the Include Filter to: .*com.* as show above:

//depot/projA/comX
//depot/projA/comY
//depot/projA/comZ
//depot/projA/resA
//depot/projA/resB

If the Jenkinsfile is found in each branch you will get the following Multi Pipeline branches:

• //depot/projA/comX = Branch comX
• //depot/projA/comY = Branch comY
• //depot/projA/comZ = Branch comZ
• //depot/projA/resA = Ignored
• //depot/projA/resB = Ignored

Build Configuration options

Extra options and customisations.

Jenkinsfile Script Path

If your Jenksinfile is located in a sub directory or uses a different name, Jenkins provide a Build Configuration option to allow customization.

Lightweight checkout vs default checkout

Lightweight checkout is automatically applied by Jenkins to Multibranch jobs. If required, you can disable the default checkout of your code and add your own checkout step in the Jenkinsfile by adding options { skipDefaultCheckout() } to the agent.

For example:

pipeline {
    agent any
    
    options { skipDefaultCheckout() }
    
    stages {
        stage('Checkout') {
            steps {
                p4sync credential: 'id', populate: forceClean(), source: streamSource('//stream/main')
            }
        }
        stage('Build') {
            steps {
                echo 'Building...'
            }
        }
        stage('Test') {
            steps {
                echo 'Testing...'
            }
        }
        stage('Deploy') {
            steps {
                echo 'Deploying...'
            }
        }
    }
}

MultiBranch trigger SCM events

Perforce can trigger Jenkins to build Multibranch jobs based on an event, such as a submitted change.

A triggered build also requires an administrator to add a Perforce trigger to the Helix Server. For information about adding a trigger, see Using triggers to customize behavior in Helix Core Server Administrator Guide. The trigger needs to POST a JSON payload to the Jenkins end-point JENKINS_URL/p4/event/.

Perforce Change JSON payload

{
  "p4port":      P4PORT,          Perforce P4PORT (must match the Jenkins Credential)
  "change":      10001,           Change number to sync files for build
  "event_type":  "UPDATED"        Event type (currently only UPDATED is supported)
}

Swarm Commit JSON payload

{
  "p4port":      P4PORT,          Perforce P4PORT (must match the Jenkins Credential)
  "project":     PROJECT,         Swarm Project name
  "branch":      BRANCH,          Swarm Branch name
  "path":        PATH,            Swarm Path name
  "change":      10001,           Change number to sync files for build
  "status":      "committed",     Change status is "committed"
  "event_type":  "UPDATED"        Event type (currently only UPDATED is supported)
}

Swarm Shelved JSON payload (draft: not yet supported)

{
  "p4port":      P4PORT,          Perforce P4PORT (must match the Jenkins Credential)
  "project":     PROJECT,         Swarm Project name
  "branch":      BRANCH,          Swarm Branch name
  "path":        PATH,            Swarm Path name
  "change":      10001,           Change number to sync files for build
  "review":      10022,           Review number to unshelve for build
  "status":      "shelved",       Change status is "shelved"
  "event_type":  "UPDATED"        Event type (currently only UPDATED is supported)
}