Multiple-Helix Server instances

A single Swarm server can be connected one or more Helix Core Server (P4D) instances. This section describes how to configure Swarm to connect to multiple Helix Servers.

Complete the following steps to connect Swarm to multiple-Helix Server instances:

• Set up the Swarm configuration file for the Helix Servers
• Set the Swarm trigger token and Swarm host variable for each Helix Server
• Configure a cron job for each Helix Server instance

Set up the Swarm configuration file for the Helix Servers

The Swarm Helix Server connection details are configured using the p4 item in the <swarm_root>/data/config.php configuration file.

When Swarm is configured for a single Helix Server instance the p4 item looks like this:

<?php
    return array(
        'p4' => array(
            'port'     => 'my-helix-core-server:1666',
            'user'     => 'admin_userid',
            'password' => 'admin user ticket or password',
            'sso'      => 'disabled', // ['disabled'|'optional'|'enabled'] default value is 'disabled' 
         ),
    );

Where:

• port is the P4PORT for your Helix Server.
• user is a userid that has admin privileges for the Helix Server.
• password is the ticket or password for the Helix Server user.
• sso if the Helix Server is configured for the Helix Authentication Service, set to one of the following:

  • enabled all users must use Helix Authentication Service to log in to Swarm.

  • optional Helix Authentication Service is available for users to log in to Swarm but is not enforced.

  • disabled Helix Authentication Service is not available to Swarm. This is the default value.

Important

From Swarm 2021.1, the sso_enabled configurable is deprecated but remains supported. It is replaced with the more flexible sso configurable. If the sso_enabled configurable and sso configurable are both present in the p4 configuration block, Swarm uses the sso configurable value.

Multiple Helix Server instances can be added to the p4 item in config.php but each Helix Server instance must have a label to identify the server instance. This enables Swarm to connect to the Helix Servers in the p4 item.

The first Helix Server in the p4 item is the primary Helix Server, this instance renders the global dashboard. If Swarm fails to connect to the primary Helix Server, Swarm cannot display any of the Helix Server instances in the global dashboard. You can still connect directly to the other P4D instances by including the Helix Server label in the URL for each instance, for example: https://swarm.company.com/serverA.

Configure the Swarm configuration file for multiple Helix Server instances:

Enter a label for each Helix Server instance and enter the server connection details under the server label.

For example:

<?php
    return array(
        'p4' => array(
            'serverA' => array(
                'port'     => 'p4d-A:1666',
                'user'     => 'admin_userid_serverA',
                'password' => 'admin user ticket or password for serverA',
                'sso'      => 'disabled', // ['disabled'|'optional'|'enabled'] default value is 'disabled' 
            ),
            'serverB' => array(
                'port'     => 'p4d-B:1666',
                'user'     => 'admin_userid_serverB',
                'password' => 'admin user ticket or password for serverB',
                'sso'      => 'disabled', // ['disabled'|'optional'|'enabled'] default value is 'disabled' 
            ),		
        );

Where:

• serverA and serverB are labels that identify the individual Helix Servers.
  • The labels must be URL-friendly labels.
  • The labels must not contain any spaces or any characters that require URL encoding, and
  • The total length of the server label and Redis namespace combined is limited to ≤ 127 characters. The default namespace is Swarm.
• The port, user, password, and sso items are specific to the Helix Server instance they are nested under.

Configure event notification for each Helix Server

Swarm needs to know about a number of Helix Server events to operate correctly, this can be done by using Helix Server Extensions (recommended) or Helix Server Triggers. Swarm installs include the Swarm Helix Server extension file and trigger scripts required for Swarm to get the events it needs from your Helix Server.

You must install and configure the Swarm Helix Server extension or Triggers to complete the Swarm configuration.

Do one of the following so that Swarm is notified about events on the Helix Servers it is connected to:

• Install and configure the Swarm Helix Server extension for each Helix Server instance, see Install and configure the Swarm Helix Server extension (recommended)

• Set the Swarm trigger token and host variable for each Helix Server instance, see Set the Swarm trigger token and Swarm host variable for each Helix Server

Install and configure the Swarm Helix Server extension (recommended)

Prerequisites

To install the Swarm Helix Server extension you need:

A compatible version of Helix Server for Helix Server Extensions:

• Linux: Helix Server 2019.2 and later. If you are using an earlier version of Helix Server, you must use triggers.

• Windows: Helix Server 2021.2 and later. If you are using an earlier version of Helix Server, you must use triggers.

You will also need:

• Extensions installed and configured on your Helix Server.

• A user with super permissions to install and configure the Swarm Helix Server extension.

Swarm Helix Server extension installation

The Swarm Helix Server extension file is included for all of the Swarm installation types. The helix-swarm.p4-extension file is located in swarm_root/p4-bin/extensions

On each Helix Server you are connecting Swarm to, run the following commands as a Helix Server user with super permissions:

1. Install the Swarm Helix Server extension on the Helix Server with:

p4 extension --yes --install helix-swarm.p4-extension

Example response:

Extension 'Perforce::helix-swarm' version '2022.1.20221215' installed successfully.
Perform the following steps to turn on the Extension:

# Create a global configuration if one doesn't already exist.
p4 extension --configure Perforce::helix-swarm

# Create an instance configuration to enable the Extension.
p4 extension --configure Perforce::helix-swarm --name Perforce::helix-swarm-instanceName

For more information, visit: https://www.perforce.com/manuals/extensions/Content/Extensions/Home-extensions.html

2. Create a global configuration if one doesn't already exist:

   p4 extension --configure Perforce::helix-swarm

   The spec file opens in your text editor, for example:

   ExtName:        helix-swarm
   ExtDescription:
           Helix Swarm Extension
    
   ExtVersion:     2022.1.DEV.20220120
    
   ExtUUID:        4532BC59-7BC8-478F-ADF6-0A563C42563D
    
   ExtRev: 1
    
   ExtMaxScriptTime:       unset
    
   ExtMaxScriptMem:        unset
    
   ExtAllowedGroups:
    
   ExtEnabled:     true
    
   ExtP4USER:      sampleExtensionsUser
    
   Name:   helix-swarm
    
   Owner:  super
    
   Update: 2022/01/21 10:43:46
    
   Description:
           The description of your config.
    
   ExtConfig:
           Debug:
                   2
           Swarm-Secure:
                   true
           Swarm-Token:
                   ... SWARM-TOKEN
           Swarm-URL:
                   http://localhost/

3. Edit the spec file to match your system:

   • ExtP4USER: Set to an existing user with super permissions.

   • ExtConfig: Check and edit the values in this block:

     • Debug:

       Debug levels 0 to 3 control the amount of debug information sent to the Helix Server Extensions log.

       Important

       A debug level of 10 and higher sends all debug information to the client. This is useful for debugging, but should not be run in a production environment.

     • Swarm-Secure:

       • true will only accept secure SSL certificates.

       • false will accept insecure SSL certificates.

     • Swarm-Token: Set to the Swarm trigger token value.

       To obtain the trigger token of your Swarm instance:

       1. Log in to Swarm as a super user.

       2. Click your userid, found at the right of the main toolbar.

       3. Select About Swarm.

       4. The About Swarm dialog is displayed and Swarm generates an API token if it doesn't already exist.

       5. Copy the trigger token value displayed at the bottom of the dialog and paste the token into the spec file.

     • Swarm-URL: Set to the URL of your Swarm instance.

4. When you have finished editing the spec file, save it in your text editor.

5. Create an instance configuration to enable the extension:

   p4 extension --configure Perforce::helix-swarm --name swarm

   The spec file opens in your text editor, for example:

   ExtName:        helix-swarm
    
   ExtDescription:
           Helix Swarm Extension
    
   ExtVersion:     2022.1.DEV.20220120
    
   ExtUUID:        4532BC59-7BC8-478F-ADF6-0A563C42563D
    
   ExtRev: 1
    
   ExtMaxScriptTime:       unset
   
   ExtMaxScriptMem:        unset
    
   ExtEnabled:     true
    
   ExtDebug:       none
    
   Name:   swarm
    
   Owner:  super
    
   Update: 2022/01/21 10:58:41
   
   Description:
           The description of your config.
   
   ExtConfig: 
           depot-path:
                   //...
           enableStrict:
                   true
           enableWorkflow:
                   true
           httpTimeout:
                   30
           ignoreErrors:
                   false
   

   We recommend that these settings are left at their default values, but the configuration needs to be run to set the default values.

   • enableStrict: Enable strict checking of reviews for workflow.

   • enableWorkflow: Enable workflow.

   • httpTimeout: Timeout (in seconds) when communicating with the Swarm server.

   • ignoreErrors: If the server returns an error (timeout, not there), then default to allowing a request. It means workflow rules can be skipped, but if the Swarm server is down it will not block submits.

6. Save the spec file in your text editor.

7. Confirm your Swarm Helix Server extension is installed and configured by running the following command on the Helix Server:

p4 extension --run swarm ping

• If the Swarm Helix Server extension is working, the Helix Server responds with OK

• If the Swarm Helix Server extension is not working, the Helix Server responds with an error message. For example, BAD (Cannot reach https://swarm-example.com/)

For more information about Helix Server Extensions, see:

• Extension overview in the Helix Core Extensions Developer Guide.

• p4 extension in the Helix Core Command-Line (P4) Reference.

8. Repeat these steps for each Helix Server.
9. When you have configured all of the Helix Servers, configure a cron job to start Swarm workers for each Helix Server instance, see Configure a cron job for each Helix Server instance.

Set the Swarm trigger token and Swarm host variable for each Helix Server

Helix Server uses a Swarm trigger token to confirm that trigger requests from Swarm are valid. Each Helix Server instance must have a valid Swarm trigger token in its swarm-trigger.conf file. The swarm-trigger.conf file also contains the Swarm host URL for the Helix Server instance.

Set the Swarm trigger token and Swarm host variable for each Helix Server instance:

1. Navigate to the Swarm URL for the instance.

For example: https://swarm.company.com/serverA

2. Log in to Swarm as a super user.
3. Click your userid, in the main toolbar and select About Swarm.

The About Swarm dialog is displayed with a Trigger Token. Swarm generates the trigger token if it doesn't already exist.

4. Copy the trigger token value from the dialog.
5. Open the swarm-trigger.conf file for the Helix Server instance.
Tip

For more information about the location of the swarm-trigger.conf file, see Set up Swarm triggers with a Windows or Linux-hosted Helix Server.

6. In the swarm-trigger.conf file:
   1. Set the SWARM_HOST URL.
   2. Set the SWARM_TOKEN by pasting the Swarm trigger token you copied in the steps above.

For example:

# SWARM_HOST (required)
# Hostname of your Swarm instance, with leading "http://" or "https://".
SWARM_HOST="https://swarm.company.com/serverA"

# SWARM_TOKEN (required)
# The token used when talking to Swarm to offer some security. To obtain the
# value, log in to Swarm as a super user and select 'About Swarm' to see the
# token value.
SWARM_TOKEN="TRIGGER-TOKEN-FOR-SERVERA"

7. Save the swarm-trigger.conf file.
8. Repeat these steps for each Helix Server.
9. When you have configured all of the Helix Servers, configure a cron job to start Swarm workers for each Helix Server instance, see Configure a cron job for each Helix Server instance.

Configure a cron job for each Helix Server instance

To automatically spawn workers for each of the servers connected to Swarm, you must manually configure a cron job for each of the servers.

Swarm package and OVA installations only

The swarm-cron-hosts.conf file specifies the connection type (HTTP or HTTPS), hostname, port number and, server label for Swarm cron jobs.

If you have installed Swarm via packages or you are running the Swarm OVA, you must specify all of the Helix Server URLs within /opt/perforce/etc/swarm-cron-hosts.conf.

1. Edit the swarm-cron-hosts.conf file so that it contains the actual Swarm hostname, ports, and server labels you have configured for Swarm.

The following format is used with one Helix Server on each line:

[http[s]://]<swarm-host>[:<port>][/<base-url>]

Default if value not specified:

• [http[s]://] http
• <swarm-host> must be specified
• [:<port>] 80
• [/<base-url>] server label, must be specified

For example, for serverA and serverB configured earlier on a Swarm host of https://swarm.company.com with a default port value of 80. The entries in the swarm-cron-hosts.conf file would be:

https://swarm.company.com/serverA
https://swarm.company.com/serverB					

2. Save the swarm-cron-hosts.conf file.
3. Swarm is now configured to connect to multiple-Helix Server instances.
Tip

To check or modify Swarm worker configuration, see Worker configuration.

Swarm Tarball installation only

If you have installed Swarm from a tarball or configured cron manually, you need create a file called helix-swarm and add all of the Helix Server instances connected to Swarm.

1. Create a file named helix-swarm in /etc/cron.d if it does not already exist.
2. Edit the helix-swarm file so that it has the following content:

#
# Cron job to start Swarm workers every minute
#
* * * * * nobody [ -x /opt/perforce/swarm/p4-bin/scripts/swarm-cron.sh ] && /opt/perforce/swarm/p4-bin/scripts/swarm-cron.sh

3. Save the helix-swarm file.
4. Edit the swarm-cron-hosts.conf file so that it contains the actual Swarm hostname, ports, and server labels you have configured for Swarm.

The following format is used with one Helix Server on each line:

[http[s]://]<swarm-host>[:<port>][/<base-url>]

Default if value not specified:

• [http[s]://] http
• <swarm-host> must be specified
• :<port> 80
• [/<base-url>] server label, must be specified

For example, for serverA and serverB configured earlier on a Swarm host of https://swarm.company.com with a default port value of 80. The entries in the swarm-cron-hosts.conf file would be:

https://swarm.company.com/serverA
https://swarm.company.com/serverB					

5. Save the swarm-cron-hosts.conf file.
6. Swarm is now configured to connect to multiple-Helix Server instances.
Tip

To check or modify Swarm worker configuration, see Worker configuration.

Further information

Users

• To visit the Global Dashboard, enter the basic Swarm URL without a Helix Server instance name, for example: https://swarm.company.com.
• To visit a specific Helix Server instance in Swarm without going via the global dashboard, include the server name in the URL, for example: https://swarm.company.com/serverA.

Once you are viewing the Helix Server in Swarm, Swarm works as a standard single Helix ServerSwarm system.

Administrators

On the web server hosting Swarm, Swarm automatically creates a data folder for each Helix Server instance in <Swarm_root>/data/servers. This is because each Helix Server request is a field.

For example:

# ls -la /opt/perforce/swarm/data/servers/serverA
total 362296			
drwx------ 6 apache apache    4096 Sep  8 17:15 .
drwx------ 6 apache apache    4096 Sep  6 15:41 ..
drwx------ 2 apache apache    4096 Sep 20 18:08 cache
drwx------ 3 apache apache    4096 Sep  7 13:25 clients
-rw-r--r-- 1 apache apache 3709543 Sep 26 14:19 log
-r-------- 1 apache apache      84 Sep  6 15:41 p4trust
drwx------ 4 apache apache    4096 Sep 20 14:27 queue
drwx------ 2 apache apache    4096 Sep 20 11:18 sessions

Developers

To get a list of projects via the Swarm API for serverA, run bash:

• If you are using wget:

  $ wget -u "apiuser:password" https://swarm.company.com/serverA/api/v11/projects

• If you are using curl:

  $ curl -u "apiuser:password" https://swarm.company.com/serverA/api/v11/projects