What’s new in the Helix Core Server Administrator Guide

This section provides a summary with links to topics in this Helix Core Server Administrator Guide. For a complete list of what's new in this release, see the 2023.2 Release Notes.

2023.2 release

Support for Amazon Web Services S3 storage for archive depots as well as S3-compatible APIs. Some organizations find these to be economical locations for files that are accessed only on rare occasions.

Support for updating an already existing license file without restarting the server. For details, see Adding or updating the license file.
To aid in planning for performance, structured logging indicates the size and number of files synced. See NetworkPerformance, ClientStats, and CacheStats. For the equivalent in unstructured logs, see Logging files sent to the server or proxy.
Support for automating the deletion of inactive files from the proxy. See p4p --cache-purge in Proxy options.
Support for macOS ARM-based hardware.

2023.1 release

Monitoring System resources can aid for Performance tuning
Distribution Server is a solution for pushing selected content from a source server to a target server where authorized users can retrieve some or all of that content.
Parallel checkpointing, dumping and recovery, including The m option might improve performance for large tables.
If your organization is using central authentication (P4AUTH), the Central server must be at version 2023.1 or later

2022.2 release

Support for Parallel checkpointing, dumping and recovery.
Server log files now include an estimate of the memory used by a command. See "command-end" in Structured Log Events.
Support for failback triggers and extensions. See Triggering on failed-back.
A unique command identifier. See %commandIdent% in Trigger script variables (and Server extension session variables in Helix Core Extensions Developer Guide).

Server triggers can use %serverprocesspid% for the process to complement %serverpid% for the thread. See Trigger script variables.
Triggers can use the %maxOpenFiles% variable. See Trigger script variables (and Server extension session variables in Helix Core Extensions Developer Guide).
A command group identifier that clients can use to tie multiple events to a single action for improved traceability. Requires using version 55 of the structured log schema. See Protocol for server and clients by release and Using structured logging.
Server log support for tracking network stats for upstream server-to-server communication as well as with the client or downstream server. These stats are recorded as additional 'NetworkPerformance' events in the structured logs at version 55 (8.55). See track-rpc under Structured Log Events and the examples and explanations at server-to-server network usage on replicas.
On Windows Server 2016 and newer, journal rotation can be performed using an atomic rename instead of a copy and truncate operation. To enable this feature, see the filesys.atomic.rename configurable in Helix Core Command-Line (P4) Reference.
The -jr recovery command to p4d supports recalculating the net storage reference count change from the revision records. See -S under Journal restore options and the associated "Important" note.

For additional new features, see What’s new in the Helix Core Command-Line (P4) Reference and What’s new in Helix Core Command-Line (P4) Guide.

2022.1 release

Failback after failover
Real-time monitoring to enhance administration: set the rt.monitorfile configurable and adjust the monitor level while commands are running. To identify commands that have been holding locks on tables for long periods and the commands that are waiting for those locks to be released, set the monitor configurable to 10 or 25.
Option to set the server.id of the proxy server. See General options for P4P > -xD [serverID] in Helix Proxy
p4 group has the optional Description: field, which might be useful if your company has a large number of group
Checkpoint and journal options enhancements in Helix Core Server (p4d) Reference:
- Checkpoint production is easier with the -P server-id and -R service options. See p4d [-z] [-R service | -P server-id] -jd file
- Checkpoint recovery is easier with the -R service option. See p4d [-z] -R service -jr file
P4V administration:
- You can enable P4V to attempt to reconnect to the server and specify how many seconds between such attempts. See the P4V.Performance.ReconnectInterval property.
- You can prevent the user from editing any stream except the current stream opened for edit in the active workspace. See P4V.Features.AllowDirectStreamEdit

For additional new features, see What’s new in the Helix Core Command-Line (P4) Reference.

2021.2 release

Support for one-to-many "ditto" mappings of paths in Stream Specs. See p4 stream in Helix Core Command-Line (P4) Reference, which explains the import& type under Paths: and provides an example of ditto mapping for stream specs.
It is now possible to get a list of all the streams associated with a given depot path. See --viewmatch in the p4 streams topic in Helix Core Command-Line (P4) Reference.
The Helix Core administrator can get a list of all the Helix Core services that are connected to the server from which the command is initiated. See the Technical Preview p4 topology command in the Helix Core Command-Line (P4) Reference.
To improve performance, consider making the server use a buffer to minimize writes to the structured log file. See the serverlog.bufsz.N configurable in the Helix Core Command-Line (P4) Reference.
A forwarding replica can now offload read-only commands from an edge server. See Forwarding replica.
Librarian The librarian subsystem of the server stores, manages, and provides the archive files to other subsystems of the server. tracking metrics that are above the applicable tracking threshold are reported in the server log, and in the structured logs. See LibrarianUsage in Using structured logging.
The P4TRUST environment variable is no longer required for SSL connections where the server provides a certificate that is not self-signed and can be verified by the client. For details, see SSL connections that do not require P4TRUST in the Helix Core Command-Line (P4) Reference.

2021.1 release

You can enable the display of real-time monitoring values. See --show-realtime under General options in Helix Core Server (p4d) Reference.
You can verify a subset of database tables. This is faster than verifying all of the database tables. See -xv under Server options in Helix Core Server (p4d) Reference.
SSL and TLS Protocol Versions have configurables on the client-side.

Note

If you are licensed for Helix Core version 2021.1 patch 1 or greater, Git Connector licensing is included at no extra charge.

2020.2 release

During upgrades to a new server version, the upgrade steps now execute in the background (applies to 2019.2 and later upgrade steps). This can improve server availability and replication performance during upgrades.
- While you are upgrading, you might want to get the status of the upgrade steps. Consider using the the new p4 upgrades command, which is documented in the Helix Core Command-Line (P4) Reference.
Any new file being shelved that has the same content as an existing shelved file now refers to the existing archive file instead of creating a duplicate archive file.
- To avoid overwriting the content of shelves that share archives, the archives of the new shelved files now have an additional numerical suffix. For example, 1.1.1.gz instead of 1.1.gz.
The Helix Proxy service can now be configured to use a different path for databases by either setting the P4PROOT environment variable or by using the -R proxy option. By default P4PROOT is the same as P4PCACHE, and both databases and archives will reside on the same path.
Enhancement of Background archive transfer for edge server submits: if the administrator has not enabled background submit, the -b option of p4 submit is ignored and standard submit behavior occurs.
Failover from a mandatory standby server when the master is not participating used to require specifying -s <serverID>. Now, failover for this scenario includes checking the ReplicatingFrom field of the standby server spec for the master's serverID when -s is not specified on the command line.
- Part of the failover process involves stopping the journalcopy and pull threads. If the failover process fails, those threads needed to be restarted manually. Now any pull -L, pull -u, or journalcopy threads that were configured using startup.N configurables will automatically be restarted if the failover process did not succeed.
For your end-users of P4V 20.3 and later, you can set the maximum number of files to load into the Depot tree at any one time. See P4V.Performance.DirFetchSize in Performance-related P4V properties.

2020.1 release

Support for Helix Authentication Service.
The storage upgrade process is now visible through the p4 monitor command. See p4 monitor in Helix Core Command-Line (P4) Reference.
A heartbeat-related trigger or extension can be part of a solution to monitor whether a server is responsive. See Triggering on heartbeat (server responsiveness)
TLS 1.3 is now supported, but TLS 1.2 remains the default. See ssl.tls.version.max in the Helix Core Command-Line (P4) Reference.
A unique command identifier is now visible through structured logging for schema version 50. See Logging and the Support Knowledgebase article on Structured Server Logs.
Global labels can now be updated from edge servers using either p4 tag -g or p4 labelsync -g. See p4 tag and p4 labelsync in Helix Core Command-Line (P4) Reference.
The host field in the protections table now allows multiple IP addresses or CIDR matchers to be specified on a single line with a comma-separated list. See p4 protect in Helix Core Command-Line (P4) Reference.
The sequence numbers used in db.protect are no longer contiguous. This allows new lines to be added without rewriting the whole table.
Standard users can now view the storage record table.
A new configurable has been added to suppress the generation of digests during a storage upgrade. See lbr.storage.skipkeyed in Helix Core Command-Line (P4) Reference.
The scan of the protections table at command start is now lockless. This means that updates to the protections table on busy servers will experience less lock contention.

Documentation-related

Helix Core Server Administrator Guide is now a single volume instead of being split between "Fundamentals" and "Multi-site Deployment". What was previously "Helix Core Server Administrator Guide: Multi-Site Deployment" is included here under Deployment architecture.
For clarity, "multi-server environment" means what was formerly called "distributed environment" and "distributed" is primarily associated with Using Helix Core Server for Distributed Versioning (DVCS).
This manual now explains how to get the latest patch release. See Patch the server.

2019.2 release

Upgrading

The 2019.2 upgrade steps are significantly different from any prior release. See Upgrading the server.

Improvements to structured logging

Structured logging has a new format so it can be more helpful for the analysis of performance. See Logging and structured log files.

failed-over trigger

A new type of trigger, failed-over, can run when a standby server becomes the new master. See Triggering on failed-over.

2019.1 release

Carefully follow the steps in the chapter on Upgrading the server because they are different from those of upgrades to versions prior to 2019.1.
Extensions, a new technology that is similar to Triggers, but with advantages and additional capabilities. See Triggers and extensions.
When the server is busy with the maximum number of commands and therefore blocking standard users,the super and operator user types can still issue a subset of commands. See Limit simultaneous connections > Commands that super and operator users can always run.
You can save disk space when creating an archive depot by using the option that includes lazy copies, which are small references to the location of potentially large files. A new database table, db.storage, replaces the db.archmap table to provide a link count for archive files on the server. This tracking reduces the complexity of identifying lazy-copies, allowing +Sn files to be lazy copied by reference instead of being duplicated with their full contents. See p4 archive -z inHelix Core Command-Line (P4) Reference.
Faster verification of archives (depot files) with p4 verify
- such verification can also be done with a new command, p4 storage -v
Display, verify, or update physical archive storage with the new command, p4 storage
Faster p4 obliterate
The procedure for setting up a high availability server for Failover has changed. See A high availability standby within an existing installation should not be initially deployed as mandatory.
Note that Helix Core Command-Line (P4) Reference indicates that the net.autotune configurable is on by default.
(Doc-only change: The How protections are implemented topic has been expanded.)
End-users can benefit from the Background archive transfer for edge server submits.
Edge-to-edge chaining: an edge server can be configured to connect to another edge server without needing to sync from a remote commit server. See also "Commit-edge" in Deployment architecture and Filtering metadata during replication or edge-to-edge chaining

2018.2 release

Failover to a new master server is now an easier process
Installation support for SUSE Linux Enterprise Server - see Linux package-based installation
Clarification on when trigger-based authentication can fall back to a password request: Single sign-on and auth-check-sso triggers
If you want to write a trigger that requires users to log in with additional security, see Triggering for multi-factor authentication (MFA)
Multi-factor authentication (MFA) is the current name for a feature that was originally introduced as second-factor authentication (2fa)
Helix SAML is a new feature for authentication

2018.1 patch

If you want to write a trigger that requires users to log in with additional security, see Triggering for multi-factor authentication (MFA)

Installation support for SUSE Linux Enterprise Server 11 and 12 - see Linux package-based installation

2018.1 release

You no longer need to use the -z option to restore a compressed checkpoint or journal. This allows the chaining of files for the restore. For example:

p4d -r . -jr checkpoint.42.gz journal.42 journal.43 journal

See the topic named "Database corruption, versioned files unaffected", which has a Note about Version 2018.1

See graph-push-reference triggers at Triggering with depots of type graph

A new structured log, ldapsync.csv, has been added to record the activity of p4 ldapsync. See Enable and configure structured logging.

2017.2 release

Triggers for external file transfer

See Triggers for external file transfer.

Server background tasks

See p4 bgtask in the Command Reference.

Parallel threads

p4 shelve now accepts the --parallel flag to specify that multiple files should be transferred in parallel, using independent network connections from automatically-invoked child processes. In addition, new configurables net.parallel.shelve.* allow p4 shelve to automatically use parallel threads to transfer files. Please see p4 help shelve and p4 help configurables for complete information.

The net.parallel.sync.svrthreads configurable reduces the number of parallel transmit threads used by sync commands when the total number of "user-transmit" threads (from all commands) running concurrently in the server would exceed the value of this configurable. Server monitoring must be enabled for this new configurable to take effect.

Complete replication for graph depot archives

Edge servers support syncing file content from graph depots. Replication supports graph depots that contain pack files, loose files, or a mixture of the pack files and loose files.

New content can be pushed by using the Git Connector or committed with p4 submit or p4 merge.

For information about depots of type graph, see: