The Perforce Broker
What is the Broker?
The Perforce Broker (P4Broker) enables you to implement local policies in your Perforce environment by allowing you to restrict the commands that can be executed, or redirect specific commands to alternate (replica or edge) Perforce servers.
The Perforce Broker is a server process that mediates between Perforce client applications and Perforce servers, including proxy servers. For example, Perforce client applications can connect to a proxy server that connects to the broker, which then connects to a Perforce server. Or, Perforce client applications can connect to a broker configured to redirect reporting-related commands to a read-only replica server, while passing other commands through to a master server.
From the perspective of the end user, the broker is transparent: users connect to a Perforce Broker just as they would connect to any other Perforce Server.
System requirements
To use the Perforce Broker, you must have:
-
A Perforce server at release 2007.2 or higher (2012.1 or higher to use SSL).
-
Perforce applications at release 2007.2 or higher (2012.1 or higher to use SSL).
The Perforce Broker is designed to run on a host that lies close to the Perforce Server (P4D), preferably on the same machine.
Installing the Broker
To install P4Broker, do the following:
-
Download the
p4broker
executable from the Perforce website, -
Copy it to a suitable directory on the host (such as
/usr/local/bin
), and ensure that the binary is executable:chmod +x p4broker
Running the Broker
After you have created your configuration file (see “Configuring the Broker”), start the Perforce Broker from the command line by issuing the the following command:
p4broker -c config_file
Alternatively, you can set P4BROKEROPTIONS
before
launching the broker and use it to specify the broker configuration
file (or other flags) to use.
For example, on Unix:
$ export P4BROKEROPTIONS="-c /usr/perforce/broker.conf" $ p4broker -d
and on Windows:
> p4 set -s P4BROKEROPTIONS="-c c:\p4broker\broker.conf" > p4broker
The Perforce Broker reads the specified broker configuration file,
and on Unix platforms the -d
flag causes the
Perforce Broker to detach itself from the controlling terminal and
run in the background.
To configure the Perforce Broker to start automatically, create a startup
script that sets P4BROKEROPTIONS
and runs the appropriate
p4broker command.
On Windows systems, you can also set P4BROKEROPTIONS
and
run the broker as a service. This involves the following steps:
cd C:\p4broker\ copy p4broker.exe p4brokers.exe copy "C:\Program Files\Perforce\Server\svcinst.exe" svcinst.exe svcinst create -nP4Broker
-e "C:\p4broker\p4brokers.exe" -a p4 set -SP4Broker
P4BROKEROPTIONS="-c C:\p4broker\p4broker.conf" svcinst start -nP4Broker
svcinst.exe is a standard Windows program.
P4Broker
is the name given to the Windows
service. For more information, see "Installing P4Broker on Windows and
Unix systems" in the Perforce Knowledge Base:
http://answers.perforce.com/articles/KB_Article/Installing-P4Broker-on-Windows-and-Unix-systems
Enabling SSL support
To encrypt the connection between a Perforce Broker and its end users,
your broker must have a valid private key and certificate pair in the
directory specified by its P4SSLDIR
environment variable.
Certificate and key generation and management for the broker works the
same as it does for the Perforce Server.
See “Linking Perforce services together”. The users' Perforce applications
must be configured to trust the fingerprint of the broker.
To encrypt the connection between a Perforce Broker and a Perforce
Server, your broker must be configured so as to trust the
fingerprint of the Perforce Server. That is, the user that runs
p4broker (typically a service user) must create
a P4TRUST
file (using p4 trust)
that recognizes the fingerprint of the Perforce Server, and must
set P4TRUST
, specifying the path to that file
(P4TRUST
cannot be specified in the broker
configuration file).
Broker information
You can issue the p4 info to determine whether you
are connected to a broker or not. When connected to a broker, the
Broker address
and Broker version
appear in the output:
$ p4 info
User name: bruno
Client name: bruno-ws
Client host: bruno.host
Client root: /Users/bruno/Workspaces/depot
Current directory: /Users/bruno/Workspaces/depot/main/jam
Peer address: 192.168.1.40:55138
Client address: 192.168.1.114
Server address: perforce:1667
Server root: /perforce/server/root
Server date: 2014/03/13 15:46:52 -0700 PDT
Server uptime: 92:26:02
Server version: P4D/LINUX26X86_64/2014.1/773873 (2014/01/21)
ServerID: master-1666
Broker address: perforce:1666
Broker version: P4BROKER/LINUX26X86_64/2014.1/782990
Server license: 10000 users (support ends 2016/01/01)
Server license-ip: 192.168.1.40
Case Handling: sensitive
When connected to a broker, you can use the p4 broker command to see a concise report of the broker's info:
$ p4 broker Current directory: /Users/bruno/Workspaces/depot/main/jam Client address: 192.168.1.114:65463 Broker address: perforce:1666 Broker version: P4BROKER/LINUX26X86_64/2014.1/782990
Broker and protections
To apply the IP address of a broker user's workstation against the
protections table, prepend the string proxy-
to the
workstation's IP address.
For instance, consider an organization with a remote development
site with workstations on a subnet of
192.168.10.0/24
. The organization also has a
central office where local development takes place; the central
office exists on the 10.0.0.0/8
subnet. A
Perforce service resides in the 10.0.0.0/8
subnet, and a broker resides in the 192.168.10.0/24
subnet. Users at the remote site belong to the group
remotedev
, and occasionally visit the central office.
Each subnet also has a corresponding set of IPv6 addresses.
To ensure that members of the remotedev
group
use the broker while working at the remote site, but do not use the
broker when visiting the local site, add the following lines to your
protections table:
list group remotedev 192.168.10.0/24 -//... list group remotedev [2001:db8:16:81::]/48 -//... write group remotedev proxy-192.168.10.0/24 //... write group remotedev proxy-[2001:db8:16:81::]/48 //... list group remotedev proxy-10.0.0.0/8 -//... list group remotedev proxy-[2001:db8:1008::]/32 -//... write group remotedev 10.0.0.0/8 //... write group remotedev proxy-[2001:db8:1008::]/32 //...
The first line denies list
access to all users in the
remotedev
group if they attempt to access Perforce
without using the broker from their workstations in the
192.168.10.0/24
subnet. The second line denies access
in identical fashion when access is attempted from the IPV6
[2001:db8:16:81::]/48
subnet.
The third line grants write
access to all users in
the remotedev
group if they are using the broker and
are working from the 192.168.10.0/24
subnet. Users of
workstations at the remote site must use the broker. (The broker
itself does not have to be in this subnet, for example, it could be at
192.168.20.0
.) The fourth line denies access in
identical fashion when access is attempted from the IPV6
[2001:db8:16:81::]/48
subnet.
Similarly, the fifth and sixth lines deny list
access
to remotedev
users when they attempt to use the
broker from workstations on the central office's subnets
(10.0.0.0/8
and
[2001:db8:1008::]/32
). The seventh and eighth lines
grant write access to remotedev
users who access the
Perforce server directly from workstations on the central office's
subnets. When visiting the local site, users from the
remotedev
group must access the Perforce server
directly.
When the Perforce service evaluates protections table entries, the
dm.proxy.protects
configurable is also evaluated.
dm.proxy.protects
defaults to 1
,
which causes the proxy-
prefix to be prepended to all
client host addresses that connect via an intermediary (proxy, broker,
broker, or edge server), indicating that the connection is not direct.
Setting dm.proxy.protects
to 0
removes the proxy-
prefix and allows you to write a
single set of protection entries that apply both to directly-connected
clients as well as to those that connect via an intermediaty. This is
more convenient but less secure if it matters that a connection is made
using an intermediary. If you use this setting, all intermediaries must
be at release 2012.1 or higher.
P4Broker flags
Flag |
Meaning |
||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
|
Specify a configuration file. Overrides
|
||||||||||
|
Output a sample configuration file, and then exit. |
||||||||||
|
Run as a daemon (in the background). |
||||||||||
|
Run as a single-threaded (non-forking) process. |
||||||||||
|
Print help message, and then exit. |
||||||||||
|
Run quietly (no startup messages). |
||||||||||
|
Print broker version, and then exit. |
||||||||||
|
Set server trace flags. Overrides the value of the
The server command trace flags and their meanings are as follows.
For command tracing, output appears in the specified log file, showing the date, time, username, IP address, and command for each request processed by the server. |
||||||||||
|
Generate SSL credentials files for the broker: create a
private key (
Requires that |
||||||||||
|
Display the fingerprint of the broker's public key, and exit. Administrators can communicate this fingerprint to end users, who can then use the p4 trust command to determine whether or not the fingerprint (of the server to which they happen to be connecting) is accurate. |
Configuring the Broker
P4Broker is controlled by a broker configuration file. The broker configuration file is a text file that contains rules for
Specifying which commands that individual users can use
Defining commands that are to be redirected to specified replica server.
To generate a sample broker configuration file, issue the following command:
p4broker -C > p4broker.conf
You can edit the newly-created p4broker.conf
file to specify your requirements.
Format of broker configuration files
A broker configuration file contains three sections:
-
Global settings: settings that apply to all broker operations
-
Alternate server definitions: the addresses and names of replica servers to which commands can be redirected in specified circumstances
-
Command handler specifications: specify how individual commands should be handled; in the absence of a command handler for any given command, the Perforce Broker permits the execution of the command
Specifying hosts
The broker configuration requires specification of the
target
setting, which identifies the Perforce service
to which commands are to be sent, the listen
address, which identifies the address where the broker listens for
commands from Perforce client applications, and the optional
altserver
alternate server address, which identifies
a replica, proxy, or other broker connected to the Perforce service.
The host specification uses the format
,
where protocol
:host
:port
protocol
is the communications protocol
(beginning with ssl:
for SSL, or
tcp:
for plaintext), host
is the name or IP address of the machine to connect to, and
port
is the number of the port on the host.
Protocol |
Behavior |
---|---|
|
Use |
|
Use |
|
Listen on/connect to an IPv4 address/port only. |
|
Listen on/connect to an IPv6 address/port only. |
|
Attempt to listen on/connect to an IPv4 address/port. If this fails, try IPv6. |
|
Attempt to listen on/connect to an IPv6 address/port. If this fails, try IPv4. |
|
Use |
|
Listen on/connect to an IPv4 address/port only, using SSL encryption. |
|
Listen on/connect to an IPv6 address/port only, using SSL encryption. |
|
Attempt to listen on/connect to an IPv4 address/port. If this fails, try IPv6. After connecting, require SSL encryption. |
|
Attempt to listen on/connect to an IPv6 address/port. If this fails, try IPv4. After connecting, require SSL encryption. |
The host
field can be the hosts' hostname or
its IP address; both IPv4 and IPv6 addresses are supported. For the
listen
setting, you can use the *
wildcard to refer to all IP addresses, but only when you are not using
CIDR notation.
If you use the *
wildcard with an IPv6 address, you
must enclose the entire IPv6 address in square brackets. For example,
[2001:db8:1:2:*]
is equivalent to
[2001:db8:1:2::]/64
. Best practice is to use CIDR
notation, surround IPv6 addresses with square brackets, and to avoid the
*
wildcard.
Global settings
The following settings apply to all operations you specify for the broker.
Setting |
Meaning |
Example |
---|---|---|
|
The default Perforce Server (P4D) to which commands are sent unless overridden by other settings in the configuration file. |
|
|
The address on which the Perforce Broker listens for commands from Perforce client applications. |
|
|
The home directory for the Perforce Broker. Other paths specified in the broker configuration file must be relative to this location. |
|
|
Path to the Perforce Broker logfile. |
|
|
Level of debugging output to log. Overrides the value
specified by the |
|
|
The name of your Perforce Administrator. This is displayed in certain error messages. |
|
|
An email address where users can contact their Perforce Administrator. This address is displayed to users when broker coniguration problems occur. |
|
|
The telephone number of the Perforce Administrator. |
|
|
The redirection mode to use:
In
In
The default mode is |
|
|
An optional user account by which the broker authenticates itself when communicating with a target server.
The broker configuration does not include a setting for
specifying a password as this is considered insecure. Use
the p4 login -u
To provide continuous operation of the broker, the
|
|
|
An optional alternate location for |
|
|
Compress connection between broker and server. Over a slow link such as a WAN, compression can increase performance. If the broker and the server are near to each other (and especially if they reside on the same physical machine), then bandwidth is not an issue, and compression should be disabled to spare CPU cycles. |
|
|
An optional alternate server to help reduce the load on the
target server. The
Multiple |
Each |
Command handler specifications
Command handlers enable you to specify how the broker responds to different commands issued by different users from within different environments. When users run commands, the Perforce Broker searches for matching command handlers and uses the first match found. If no command handler matches the user's command, the command is forwarded to the target Perforce Server for normal processing.
The general syntax of a command handler specification is outlined
in the sample broker.conf
:
command:commandpattern
{ # Conditions for the command to meet (optional) # Note that with the exception of 'flags', these are regex patterns. flags =required-flags
; args =required-argument
s; user =required-user
; workspace =required-client-workspace
; prog =required-client-program
; version =required-version-of-client-program
; # What to do with matching commands (required) action = pass | reject | redirect | filter | respond ; # How to go about it destination =altserver
; # Required for action = redirect execute =/path/to/filter/program
; # Required for action = filter message =rejection-message
; # Required for action = reject}
The commandpattern
parameter can be a regular
expression and can include the ".*
" wildcard. For
example, a commandpattern
of
"user.*
" matches both the "p4
user" and "p4 users" commands. See
“Regular expression synopsis”
The following table describes the parameters in detail.
Parameter |
Meaning |
---|---|
|
A list of flags that must be present on the command line of the command being handled. This feature enables you to specify different handling for the same p4 command, depending on which flags the user specifies. Note that only single character flags may be specified here. Multi-character flags, and flags that take arguments should be handled by a filter program. |
|
A list of arguments that must be present on the command line of the command being handled. |
|
The name of the user who issued the command. |
|
The Perforce client workspace setting in effect when the command was issued. |
|
The Perforce client application through which the user issued the command. This feature enables you to handle commands on a per-application basis. |
|
The version of the Perforce application through which the user issued the command. |
|
Defines how the Perforce Broker handles the specified
commands. Valid values are: |
|
For redirected commands, the name of the replica to which the
commands are redirected. The destination must be the name of a
previously-defined alternate (replica) server listed in the
You can implement load-balancing by setting the destination to
the keyword |
|
The path to a filter program to be executed. For details about filter programs, see “Filter Programs”. |
|
A message to be sent to the user, typically before the command is executed; this may be used with any of the above actions. |
For example, the following command handler prevents user
joe
from invoking p4 submit
from the buildonly
client workspace.
command: submit { user = joe; workspace = buildonly; action = reject; message = "Submit failed: Please do not submit from this workspace." }
Regular expression synopsis
A regular expression, or regex, is a sequence of characters that forms a search pattern, for use in pattern matching with strings. The following is a short synopsis of the regex facility available in command handler specifications.
A regular expression is formed from zero or more
branches. Branches are separated by
|
. The regex matches any string that matches at
least one of the branches.
A branch is formed from zero or more pieces, concatenated together. A branch matches when all of its pieces match in sequence, that is, a match for the first piece, followed by a match for the second piece, etc.
A piece is an atom possibly followed by a
quantifier: *
,
+
, or ?
. An atom followed by
*
matches a sequence of 0 or more instances of the
atom. An atom followed by +
matches a sequence of 1
or more instances of the atom. An atom followed by ?
matches a sequence of 0 or 1 instances of the atom.
An atom is:
-
a subordinate regular expression in parentheses - matches that subordinate regular expression
-
a range (see below),
-
.
- matches any single character, -
^
- matches the beginning of the string, -
$
- matches the end of the string, -
a
\
followed by a single character - matches that character, -
or a single character with no other significance - matches that character.
A range is a sequence of characters enclosed in square brackets
([]
), and normally matches any single character from
the sequence. If the sequence begins with ^
, it
matches any single character that is not in the
sequence. If two characters in the sequence are separated by
-
, this is shorthand for the full list of ASCII
characters between them (e.g. [0-9]
matches any
decimal digit, [a-z]
matches any lowercase
alphabetical character). To include a literal ]
in
the sequence, make it the first character (following a possible
^
). To include a literal -
, make
it the first or last character.
Filter Programs
When the action
for a command handler
is filter
, the Perforce Broker executes the
program or script specified by the execute
parameter and performs the action returned by the program. Filter
programs enable you to enforce policies beyond the capabilities
provided by the broker configuration file.
The Perforce Broker invokes the filter program by passing command details to the program's standard input in the following format:
Command detail |
Definition |
---|---|
|
user command |
|
port on which the broker is listening |
|
port on which the target server is listening |
|
client application program |
|
version of client application program |
|
level of client protocol |
|
level of api protocol |
|
maximum lock time (in ms) to lock tables before aborting |
|
maximum number of rows of data scanned by a command |
|
maximum number of rows of result data to be returned |
|
name of client workspace |
|
name of requesting user |
|
IP address of client |
|
IP address of proxy (if any) |
|
Client's working directory |
|
number of arguments to command |
|
first argument (if any) |
|
second argument (if any) |
|
Hostname of the client |
|
The internal version level of the broker. |
|
The internal version level of the proxy (if any). |
Non-printable characters in command arguments are sent to filter
programs as a percent sign followed by a pair of hex characters
representing the ASCII code for the non-printable character in
question. For example, the tab character is encoded as
%09
.
Your filter program must read this data from stdin before performing any additional processing, regardless of whether the script requires the data. If the filter script does not read the data from stdin, "broken pipe" errors can occur, and the broker rejects the user's command.
Your filter program must respond to the Broker on standard output (stdout) with data in one of the four following formats:
action: PASS message: a message for the user (optional)
action: REJECT message: a message for the user (required)
action: REDIRECT altserver: (an alternate server name) message: a message for the user (optional)
action: RESPOND message: a message for the user (required)
Note
The values for the action
are
case-sensitive.
The action
keyword is always required and
tells the Broker how to respond to the user's request. The
available action
s are:
Action |
Definition |
---|---|
|
Run the user's command unchanged. A
|
|
Reject the user's command; return an error message. A
|
|
Redirect the command to a different (alternate) replica
server. An |
|
Do not run the command; return an informational message. A
|
If the filter program returns any response other than something complying with the four message formats above, the user's command is rejected. If errors occur during the execution of your filter script code cause the broker to reject the user's command, the broker returns an error message.
Alternate server definitions
The Perforce Broker can direct user requests to an alternate server to reduce the load on the target server. These alternate servers must be replicas (or brokers, or proxies) connected to the intended target server.
To set up and configure a replica server, see Perforce Replication. The broker works with both metadata-only replicas and with replicas that have access to both metadata and versioned files.
There is no limit to the number of alternate replica servers you can define in a broker configuration file.
The syntax for specifying an alternate server is as follows:
altserver
name
{ target= [protocol
:]host
:port
}
The name assigned to the alternate server is used in command handler specifications. See “Command handler specifications”.
Configuring alternate servers to work with central authorization servers
Alternate servers require users to authenticate themselves when
they run commands. For this reason, the Perforce Broker must be
used in conjunction with a central authorization server
(P4AUTH
) and Perforce Servers at version 2007.2 or
later. For more information about setting up a central
authorization server, see “Centralized authorization server (P4AUTH)”.
When used with a central authorization server, a single p4 login request can create a ticket that is valid for the user across all servers in the Perforce Broker's configuration, enabling the user to log in once. The Perforce Broker assumes that a ticket granted by the target server is valid across all alternate servers.
If the target server in the broker configuration file is a
central authorization server, the value assigned to the
target
parameter must precisely match the
setting of P4AUTH
on the alternate server
machine(s). Similarly, if an alternate sever defined in the
broker configuration file is used as the central authorization
server, the value assigned to the target
parameter for the alternate server must match the setting of
P4AUTH
on the other server machine(s).
Configuring centralized authorization and changelist servers
Most organizations use only one Perforce Server for the entire organization, but over time, as organizations grow and merge, or as different groups within an organization need to share data, an organization can find itself with multiple Perforce Servers across a common user base.
Centralized authorization server (P4AUTH)
If you are running multiple Perforce servers, you can configure them to retrieve protections and licensing data from a centralized authorization server. By using a centralized server, you are freed from the necessity of ensuring that all your servers contain the same users and protections entries.
Note
When using a centralized authentication server, all outer servers must be at the same (or newer) release level as the central server.
If a user does not exist on the central authorization server, that user does not appear to exist on the outer server. If a user exists on both the central authorization server and the outer server, the most permissive protections of the two lines of the protections table are assigned to the user.
You can use any existing Perforce Server in your organization as your
central authorization server. The license file for the central
authorization server must be valid, as it governs the number of licensed
users that are permitted to exist on outer servers. To configure a
Perforce Server to use a central authorization server, set
P4AUTH
before starting the server, or specify it on the
command line when you start the server.
If your server is making use of a centralized authorization server, the following line will appear in the output of p4 info:
... Authorization Server: [protocol
:]host
:port
Where
[
refer to the protocol, host, and port number of the central
authorization server. See “Specifying hosts”.
protocol
:]host
:port
In the following example, an outer server (named
server2
) is configured to use a central
authorization server (named central
). The outer
server listens for user requests on port 1999 and relies on the
central server's data for user, group, protection, review, and
licensing information. It also joins the protection table from the
server at central:1666
to its own protections
table.
For example:
p4d -In server2 -a central:1666 -p 1999
Windows
On Windows, configure the outer server with p4 set -S as follows:
p4 set -S "Outer Server" P4NAME=server2 p4 set -S "Outer Server" P4AUTH=central:1666 p4 set -S "Outer Server" P4PORT=1999
When you configure a central authorization server, outer servers forward the following commands to the central server for processing:
Command |
Forwarded to auth server? |
Notes |
---|---|---|
p4 group |
Yes |
Local group data is derived from the central server. |
p4 groups |
Yes |
Local group data is derived from the central server. |
p4 license |
Yes |
License limits are derived from the central server. License updates are forwarded to the central server. |
p4 passwd |
Yes |
Password settings are stored on, and must meet the security level requirements of, the central server. |
p4 review |
No |
Service user (or |
p4 reviews |
No |
Service user (or |
p4 user |
Yes |
Local user data is derived from the central server. |
p4 users |
Yes |
Local user data is derived from the central server. |
p4 protect |
No |
The local server's protections table is displayed if the user is authorized (as defined by the combined protection tables) to edit it. |
p4 protects |
Yes |
Protections are derived from the central server's protection table as appended to the outer server's protection table. |
p4 login |
Yes |
Command is forwarded to the central server for ticket generation. |
p4 logout |
Yes |
Command is forwarded to the central server for ticket invalidation. |
Limitations and notes
-
All servers that use
P4AUTH
must have the same unicode setting as the central authorization server. -
Setting
P4AUTH
by means of a p4 configure set P4AUTH=[protocol
:server
:port
command requires a restart of the outer server. -
To ensure that p4 review and p4 reviews work correctly, you must enable remote depot access for the service user (or, if no service user is specified, for a user named
remote
) on the central server. -
To ensure that the authentication server correctly distinguishes forwarded commands from commands issued by trusted, directly-connected users, you must define any IP-based protection entries in the Perforce service by prepending the string "
proxy-
" to the[
definition.protocol
:host
-
Protections for non-forwarded commands are enforced by the outer server and use the plain client IP address, even if the protections are derived from lines in the central server's protections table.
Centralized changelist server (P4CHANGE)
By default, Perforce servers do not coordinate the numbering of changelists. Each Perforce Server numbers its changelists independently. If you are running multiple servers, you can configure your servers to refer to a centralized changelist server from which to obtain changelist numbers. Doing so ensures that changelist numbers are unique across your organization, regardless of the server to which they are submitted.
Note
When using a centralized changelist server, all outer servers must be at the same (or newer) release level as the central server.
To configure a Perforce Server to use a central changelist server,
set P4CHANGE
before starting the second server, or
specify it on the p4d command line with the
-g
flag:
p4d -In server2 -g central:1666 -p 1999
Windows
On Windows, configure the outer server with p4 set -S as follows:
p4 set -S "Outer Server" P4NAME=server2 p4 set -S "Outer Server" P4CHANGE=central:1666 p4 set -S "Outer Server" P4PORT=1999
In this example, the outer server (named
server2
) is configured to use a central
changelist server (named central
). Whenever a
user of the outer server must assign a changelist number (that is,
when a user creates a pending changelist or submits one), the
central server's next available changelist number is used instead.
There is no limit on the number of servers that can refer to a central changelist server. This configuration has no effect on the output of the p4 changes command; p4 changes lists only changelists from the currently connected server, regardless of whether it generates its own changelist numbers or relies on a central server.
If your server is making use of a centralized changelist server, the following line will appear in the output of p4 info:
... Changelist Server: [protocol
:]host
:port
Where
[
refers to the protocol, host, and port number of the central changelist server.
protocol
:]host
:port