Release Notes for P4, the Helix Core Command Line client, P4D, the Helix Core Server P4P, the Helix Core Proxy Server and P4Broker, the Helix Core Broker Version 2023.2 Introduction This document lists all user-visible changes to: * Helix Core Server (P4D) * Helix Core Command Line client (P4) * Helix Core Proxy Server (P4P) * Helix Core Broker (P4Broker) in the 2023.2 release. Release notes for other Perforce products are at https://www.perforce.com/support/release-notes which includes a link to the developer notes for the Helix Core API for C/C++ at https://www.perforce.com/perforce/doc.current/user/p4devnotes.txt Perforce numbers releases YYYY.R/CCCCC, e.g. 2002.1/30547. YYYY is the year; R is the release of that year; CCCCC is the bugfix change level. Each bugfix in these release notes is marked by its change number. Any build includes (1) all bugfixes of all previous releases and (2) all bugfixes of the current release up to the bugfix change level. Both 'p4' and 'p4d' will report their version information by passing the '-V' flag. Additionally, 'p4' can report the server's information with the 'p4 info' command. Version information includes a release version and a build change number. The build change number can be compared with change numbers and patch information mentioned in the matching version release notes file to determine if a particular build of a version includes the change mentioned. Supported Platforms for p4d, p4, p4p and p4broker Linux kernel 2.6+ (glibc 2.6+) for Intel(x86_64) Windows 10 for Intel(x64) Windows 11 for Intel(x64) Windows 2012 for Intel(x64) Windows 2016 for Intel(x64) Windows 2019 for Intel(x64) Windows 2022 for Intel(x64) Apple OS X 10.12+ for Intel(x86_64) MacOS 12+ for Apple silicon (arm64) -------------------------------------------------------------------------- Important Platform End-of-Life Notice This notice identifies the platforms that will be placed in End-of-Life (EOL) status in November 2023. This applies to p4, p4d, p4p, and p4broker components of the server release. The following 64-bit platforms are now in the Discontinued phase: Windows 8.1 for Intel(x64) Windows 2012 for Intel(x64) The following 32-bit platforms are now in the Discontinued phase: Linux kernel 2.6+ (glibc 2.6+) for Intel(x86) OBSOLESCENT Phase: Starting with this release, these platforms will no longer be supported. Customers will receive full support and patches for the deprecated platforms during this phase for older releases. The platforms will remain in this phase until November 2023 after which they will transition into the discontinued phase. DISCONTINUED (EOL) Phase: Patches will no longer be issued for these platforms. We will no longer provide technical support to customers with servers on these platforms -------------------------------------------------------------------------- Important Changes to Helix Core Licenses Helix Core licenses have been modified to include an 'ExtraCapabilities' field. This field encapsulates details about the license for this server as well as additional applications that extend this Helix Core Server. Please note, a comprehensive list of possible values is not provided as the list will evolve over time and, as with other fields in the license, any modifications to the 'ExtraCapabilities' field will invalidate the license. This new license format requires an updated version of the Helix Core Server to be able to read and apply this new license. License files that use this new field require an updated version of the Helix Core Server and include the following comment specifying the exact versions required: # This license works with 2019.2 patch 14, 2020.1 patch 8, 2020.2 patch 6, 2021.1 patch 1, 2021.2 or later servers. -------------------------------------------------------------------------- Important limitation for Windows servers prior to 2021.2 Support for Helix Core Extensions was disabled for Windows servers prior to 2021.2 due to unrelated performance concerns on that platform requiring a conflicting toolchain change. Extensions remained fully supported on Linux and Mac. This limitation has been resolved in 2021.2 and extensions are now fully supported on all supported platforms. -------------------------------------------------------------------------- Important security note This release links OpenSSL version 3.0.12 -------------------------------------------------------------------------- Important export note This product is subject to U.S. export control laws and regulations including, but not limited to, the U.S. Export Administration Regulations, the International Traffic in Arms Regulation requirements, and all applicable end-use, end-user and destination restrictions. Licensee shall not permit, directly or indirectly, use of any Perforce technology in or by any U.S. embargoed country or otherwise in violation of any U.S. export control laws and regulations. -------------------------------------------------------------------------- Upgrading the Server ** IMPORTANT UPGRADE NOTES ** CHANGES TO DISTRIBUTED UPGRADE RECOMMENDATIONS For 2019.1 or later of the Helix Core Server (P4D), significant schema upgrades performed by 'p4d -xu' will only be performed on servers where those upgrades are needed. This means that metadata changes made by upgrades must be replicated and so must be journaled. To help prevent mistakes, upgrades will not run if journaling is disabled. If offline databases are in use, upgrades should not be performed on them, as the upgrade performed on the live database will be journaled and applied during normal journal replay. When upgrading from 2019.1 or later of the Helix Core Server (P4D), replicas may be upgraded safely prior to the server they are replicating from. This upgrade pattern, referred to as "outside-in", allows a multi-server environment be upgraded server-by-server starting with the outermost replica, without requiring the entire set of servers to be taken offline at once. When upgrading from a version prior to 2019.1 to 2019.1 or later, we strongly recommend that all servers are upgraded in a single upgrade session. More information can be found in the Helix Core Server Administrator Guide at https://www.perforce.com/manuals/p4sag/Content/P4SAG/chapter.upgrade.html ------------------------------------------------------------------- CHANGES TO UNLICENSED SERVER For v16.1 onwards of the Helix Core Server (P4D) the number of free users allowed with an unlicensed server has changed to 5, the number of clients remains at 20. Previous versions of the server will continue to support the 20/20 usage allowance. ------------------------------------------------------------------- BEFORE UPGRADING To upgrade your Helix Core Server (P4D), your license file must be current. Expired licenses do not work with upgraded servers. In addition to your usual checkpointing scheme, always checkpoint your server immediately before undertaking an upgrade. For any Helix4Git instances that are currently polling repos we recommend upgrading your Helix4Git instance before upgrading your Helix Core Server. Otherwise, any repo polling would cease. Read-only archive directories can interfere with the upgrade process. This may be the case for archive paths migrated from a different version control system, such as CVS, or in cases where 3rd party imported content has been made read only by a system admin. Make sure all such archive paths are writeable before attempting the upgrade. DOWNGRADING A 2023.2 SERVER IS NOT POSSIBLE UPGRADING PROCEDURE IS DIFFERENT DEPENDING ON WHAT YOUR CURRENT PERFORCE VERSION IS: UPDATE DATABASE SCHEMA vs UPDATE DATABASE FORMAT ------------------------------------------------------------------- ** UPDATE DATABASE SCHEMA If your current server version is 2013.3 or above, then this upgrade is a UPDATE DATABASE SCHEMA upgrade: Unix Platforms 1. Issue the 'p4d -r -J -xu' command. This command performs the significant schema upgrades, then exits. 2. Restart your server with your site's usual parameters. Windows Platforms 1. Issue the 'p4d -r -J -xu' command from a command prompt window. This command performs the significant schema upgrades, then exits. 2. Restart your Helix Core Server (P4D) service. Note: Personal DVCS servers will automatically perform any database schema upgrades. Although the upgrade steps described above are unnecessary for these servers, they are safe to run. ------------------------------------------------------------------- ** UPDATE DATABASE FORMAT If your current server version is 2013.2 or earlier, then this upgrade is an UPDATE DATABASE FORMAT upgrade: Due to the btree format introduced in 2013.3 this is not an UPDATE DATABASE SCHEMA upgrade. A checkpoint using your old server and then a restore using the 2021.1 server is required before the internal upgrades 'p4d -xu' (if required) can be performed. Note: If you have made use of the (undocumented) '+T' modifier (storing file content in the tiny.db database) this data is not checkpointed (and therefore not restored). See 'p4 help undoc' for backup/restore procedures for this table. Once the database has been restored with the 2021.1 version, update your database schema: Unix Platforms 1. Issue the 'p4d -r -J -xu' command. This command performs the schema upgrades, then exits. 2. Restart your server with your site's usual parameters. Windows Platforms 1. Issue the 'p4d -r -J -xu' command from a command prompt window. This command performs the schema upgrades, then exits. 2. Restart your Helix Core Server (P4D) service. -------------------------------------------------------------------------- Interoperating With Previous Releases 1. Unless stated otherwise, you can intermix any release of Helix Command Line (P4) with any release of the Helix Versioning Engine (P4D), but any functionality new to this release requires you to upgrade the client and/or P4D. See marks in the notes below. * -- requires new p4 client program including all client applications and derived APIs ** -- requires new p4d server program *** -- requires new p4p proxy program **** -- requires new p4broker broker program Any replica servers must be at least the same release level as the master server and any functionality that requires an upgrade for the master requires an upgrade for the replica. A replica can be running a later release than the master if the master is running at least 2019.1. However in this case some new functionality might not be available on the upgraded replica until the master has been upgraded. 2. Beginning with 99.2, remote depots will interoperate between UNIX and NT. Beginning with 98.2, remote depots will interoperate across (98.2 and higher) releases. In 98.1 and before, remote depots will only operate with another server of the same release. 3. As of 2005.1, remote depot support for 98.2 and 99.1 servers has been dropped. Attempts by 98.2 and 99.1 servers to contact 2005.1 servers still works, but the depots will appear devoid of files. -------------------------------------------------------------------------- Technology Preview features: Features offered in Technology Preview are experimental and not guaranteed to always work as expected. If you have feedback and functionality suggestions, email techpreview@perforce.com. -------------------------------------------------------------------------- Important note: Please refer to: http://www.perforce.com/perforce/r23.2/user/relnotes.txt to get up to date GA and post-GA information about this release. -------------------------------------------------------------------------- Major new functionality in 2023.2 (2023.2/2519561) (2023/11/14) #2489411 (Job #116768) ** Initial support S3 backed depots This is initially limited to depots of type archive, but will be extended to support other depot types in the future. #2486446 (Job #115674) ** New attribute values may now be optionally stored in a traits depot rather than in the db.traits table when the size of the value meets or exceeds a configured size (in bytes) as set by the new configurable 'trait.storagedepot.min'. The 'p4 fstat -Oa' option will fetch and display the value from its storage location, which will be either the trait depot or the db.traits table. A new command, 'p4 fstat -On' will display the attribute names with their storage locations rather than their trait values. The 'p4 attribute' command now has options to override the configurable under certain circumstances. Please see 'p4 help attribute' and 'p4 help fstat' for further details. Minor new functionality in 2023.2 (2023.2/2519561) (2023/11/14) #2499053 (Job #112875, #117469) ** Validate IP/MAC address or FQDN against valid addresses on the server host before applying a new license with 'p4 license -i'. No server server restart is needed for any license update. #2495109 (Job #97048) *** Added a new p4p option, --cache-purge, for clearing proxy cache files that have not been accessed recently. The threshold for access time is set by the new configurable 'proxy.clearcachethresh'. #2491355 (Job #116606) ** Added a new comment field to config history records: long option --comment can be used to set the comment for the last history record for a given configurable, or used with set/unset. See 'p4 help configure' for usage details. #2487108 (Job #117327, #113690) ** The new journalPrefixBackup Commit Server configurable makes it possible to manually move rotated journal files to a secondary storage location and still support edge server activity. See the Command Reference for details. #2486366 (Job #116822) ** New command 'p4 configure help' outputs all documented configurables. Shows their value, description, value type, min value (if available), max value (if available), default value, recommended value (if available), restart requirements, support level, category and URL of the online help. #2485308 (Job #110741) ** The performance of 'p4 reload' has been improved by optimising its memory usage. #2483722 (Job #112874) ** Use 'p4 license -L' to obtain a list of valid IP or MAC addresses from your server host. #2483011 (Job #095924) * ** *** Log the number of files sent/received by the server/proxy/client and size in bytes for more visibility. Unstructured logs will show separate logs for server/client/proxy as '--- filetotals (svr) send/recv files+bytes 0+0mb/0+0mb' '--- filetotals (client) send/recv files+bytes 0+0mb/0+0mb' '--- filetotals (cache) send/recv files+bytes 0+0mb/0+0mb' The following changes have been made for structured logs: 1. The existing DMG_LOG_TRACK_RPC event now has 4 new fields: f_sendDirectFiles f_recvDirectFiles f_sendDirectBytes f_recvDirectBytes 2. New event DMG_LOG_CLIENT_STATS (event number 25) with 4 fields: f_sendClientFiles f_recvClientFiles f_sendClientBytes f_recvClientBytes 3. New event DMG_LOG_PROXY_STATS (event number 26) with 4 fields: f_sendCacheFiles f_recvCacheFiles f_sendCacheBytes f_recvCacheBytes CURRENT LIMITATIONS: 1. Only proxies will forward the client stats to the commit server, and any cached delivers, file total logging disabled on broker, and forwarding-replica will log locally instead of forwarding the stats to the upstream commit server. 2. Neither forwarding replica or proxy will log filetotals cached during transfer, from either commit->proxy/replica or from client->proxy/replica. #2482034 (Job #117182) ** 'p4 streams --viewmatch' now accepts a revision specifier. #2470778 (Job #116585) * ** *** **** P4PORT transport prefixes (e.g., 'tcp', 'tcp4', 'tcp46, 'tcp6', 'tcp64', and their SSL equivalents) are no longer required to be lowercase. The alphabetic characters in a transport prefix may be uppercase, lowercase, or mixed case. #2442141 (Job #053773, #034662) * ** The 'zlib.compression.level' configurable has been added to allow for specifying the amount of compression attempted for various activities. The configurable has a range from -1 to 9, with 9 being the slowest and highest compression level. The -1 setting corresponds to the default setting of the zlib library, which is currently level 6. This allows admins to trade compression time for file size. The difference in file size can be smaller than the gains had by spending less time compressing files, so allowing for this choice offers a useful setting to expose since spending less time waiting for 'p4 submit' is cheaper than using a bit more storage. E.g. a 'p4 submit' of a binary file with the default compression level of 6 takes 4.75s and results in a 49MB archive file. Changing the level to 2 takes 2.98s and results in a 54MB archive file - that's a 37% faster commit and 10% bigger binary, and ~54% faster than the 22.2 release. The setting is primarily applied server-side, intended for archive file activities, but can also be used client-side, e.g. for the client spec 'compress' option. Bugs fixed in 2023.2 Patch 3 (2023.2/2563409) (2024/02/27) #2558646 (Jobs #118375, #112333) ** On Windows the atomic rename api is limited in scope by the OS. With this change the atomic rename is first attempted, if the atomic rename fails the operation falls back to a copy truncate. #2557887 (Job #119417) ** Fixed a crash when 'p4 unshelve -b/-S' encountered an error. #2556700 (Job #102151) ** This change improves 'p4 sync' performance with complex views by increasing the lookback limit from the previous value of 8 to a value set by the new configurable maxLookBack with a default value of 1000. #2555959 (Job #116293) ** 'p4 search' could get stuck when putting a read-lock on the ixtext table. This change fixes the problem by switching to lockess access for the table. #2555497 (Job #116539) ** It was possible for the table dump command 'p4d -jd' to create the MD5 checksum file even when the command had failed. This has been fixed. #2553624 (Job #119254) ** A parallel recovery would leak memory on note entries in the replay file(s). This has been fixed. #2554082 (Job #119336) ** OpenLDAP has been upgraded 2.4.17 #2552714 (Job #118004) ** On-demand fetching of archives from a replica will now cause the commit server to perform a fetch on-demand if the archive has not yet been transferred from an edge server as part of a background submit. The client requesting the file content will no longer receive a librarian error in this situation. #2551998 (Job #118593) ** An Extension implementing a 'pressure-pause' hook no longer prevents updates to the trigger table. #2548434 (Job #118559) * ** *** **** Clients now call shutdown() before closing a network connection in order to process all pending packets and ensure a clean connection shutdown. In particular, pending packets will not be lost when a client is terminated via control-c (^C). #2548082 (Jobs #107457, #117470, #118998, #117719) * ** *** **** Improve SSL connection establishment under heavy load. In particular, this eliminates the "wrong version number" error when the client is slow to initiate the SSL handshake after establishing the TCP connection. Improve SSL error detection and handling to prevent reporting non-existent SSL errors. Previously this was theoretically possible but had not been observed. Ignore spurious SSL errors detected during SSL client and server initialization to prevent reporting non-existent SSL errors. Previously this was theoretically possible but had not been observed. Make SSL error handling more robust and provide more detailed and user-friendly SSL error messages. #2547699 (Job #118566) ** Parallel recoveries did not detect a character set mismatch between the server and recovery files. This has been fixed. #2546176 (Job #118002) ** 'p4 obliterate' can no longer write archive delete journal notes when a journal rotation is in progress. #2544323 (Job #118360) ** Replacing a trait value that had been saved in the trait depot with a value that is saved in db.traits would cause the trait depot depotfile to not be removed. This has been fixed. #2541675 (Job #115632) ** Fixed auto-shelve of modified content on distribution server when switching between streams. #2538285 (Job #118756) ** Attempts to assign a negative value to a date field are now blocked. #2538275 (Job #118549) ** The server does not handle date fields containing negative values correctly. This has been fixed. Bugs fixed in 2023.2 Patch 2 (2023.2/2535420) (2023/12/19) #2534278 (Job #118635) ** A severe performance regression for 'p4 jobs -e Date>=yyyy/mm/dd' and similar queries was introduced in the initial 2023.2 release, causing all numeric db.ixtext entries to be scanned in the course of retrieving dates matching the specified range. This is fixed. #2533190 (Job #114420) ** 'p4 renameuser' no longer takes long write locks on db.monitor. #2528293 (Job #118342) ** The monitor level stored by 'rt.monitorfile' is no longer reset to 0 when each new command starts. #2532094 (Job #116191) ** Fixed DVCS fetch, push and zip commands incorrectly reporting a split move across changelists. #2531016 (Job #118395) ** Set digest/filesize on text+k files for 'p4 change -t' command. Bugs fixed in 2023.2 Patch 1 (2023.2/2523307) (2023/11/22) #2523153 (Job #118298) ** Back out change 2490421 as it has not had the desired effect in customer environments. #2521944 (Job #118198) ** 'p4 license -L' no longer reports the MAC addresses of network interfaces that do not have IPv4 or IPv6 addresses. Bugs fixed in 2023.2 (2023.2/2519561) (2023/11/14) #2516988 (Job #116628) ** Revisions without an archive that were retyped using the force flag could cause an error when replicated to an edge. These revisions are now marked as 'missing' and ignored for replication. 'p4 fstat' in tagged mode will report the 'isMissing' tag for these revisions. #2512340 (Job #117646) ** 'p4 unshelve' with multiple files as parameters, if this file-list is not sorted, could erroneously report 'can't unshelve (already opened for edit using a different client view path)'. This change fixes the problem. #2508491 (Job #116102) * ** *** **** OpenSSL has been upgraded to 3.0.12 #2505273 (Job #112432) ** 'p4 fetch' and 'p4 push' commands now update LastFetch/LastPush when user provided filespec argument does not limit remote view. #2504803 (Job #117741) * ** cURL (for extensions) has been upgraded 8.4.0 to address multiple cURL CVEs: CVE-2023-38545 and CVE-2023-38546 #2499611 (Job #90618, #117153) * ** *** **** Security fix. Addressed CVE-2023-5759. #2496083 (Job #117185) ** The 'db.checkpoint.threads' configurable override was not supported for non-parallel checkpoint and dump requests started directly with 'p4d' via the '-N threads' option and requests started by the 'p4 admin checkpoint' command via the '-n threads' option. This has been fixed. #2495972 (Job #114613) ** If 'p4 unlock -f' is run for a bunch of individual files, it could result unnecessary scanning of db.domain and holding lock on this table for a long period of time, blocking out other commands. This change fixes the problem. #2494767 (Job #117156) ** Security fix. Addressed CVE-2023-35767. #2493556 (Job #117155) ** Security fix. Addressed CVE-2023-45849. #2491456 (Job #107898) ** 'p4 annotate' unnecessarily scanned additional db.rev records when following integration history through files with protection exclusions. This has been fixed. #2491397 (Job #117154) ** Security fix. Addressed CVE-2023-45319. #2490421 (Job #107457) * ** *** **** Failed SSL connection attempts now produce more detailed and accurate error messages, making it easier to diagnose incorrect SSL configurations. In particular, if a server is experiencing more simultaneous SSL connection attempts than it can handle then it will produce a message to that effect rather than producing a generic SSL failure message. Additionally, transient connection errors are retried more robustly, so some connection attempts that would previously have failed may now succeed. #2489148 (Job #117392) ** There was a data race between threads in the 'p4 storage -U' command. This has been fixed. #2488264 (Job #117222) ** Ensures that 'p4d -jr' command to replay the checkpoint in a non empty directory does not update the existing database files. #2488070 (Job #117270) ** Fixed a component stream validation error when reparenting a stream. #2487821 (Job #114933) ** When a lazy copy file is sent using 'p4 unzip', 'p4 push' or 'p4 fetch' and the source is not part of the package, it will be imported as a non-lazy copy file. #2484430 (Job #113319) ** Using 'p4 unzip' with -I option to exclude integration records no longer causes a crash when the source of a lazy copy is not included in the zip. #2482627 (Job #106491) ** 'p4 zip' correctly includes archives for lazy copies when used with a revision range. #2482523 (Jobs #117098, #117137) ** For a parallel checkpoint operation, the start time and end time in the checkpoint history record were the same and the 'checkpoint' and 'journal' fields in the 'rotate-journal' trigger were not set. This has been fixed. #2481875 (Job #117071) ** For 'changes -m -u' it may produce less than m lines, missing valid records in the output. For 'p4 changes' with -i closure option, it may produce result that does not include changes for the files in the source of integration. #2479809 (Job #117093) ** The orphan scanner, 'p4 storage -l', does not work with space in the depotdirectory argument. This has been fixed. #2475698 (Job #114250) ** Concurrent 'p4 unlock' and 'p4 lock' commands can result a deadlock and system hang. This change fixes the problem. #2474301 (Job #116821) ** Some commands that take file paths with changelists specifiers will now position in db.rev to get to the next file instead of scanning. #2469524 (Job #116678) ** 'p4 submit' will no longer clear the commandGroup identifier during the submit process. #2468060 (Job #111780) ** For 'p4 shelve' with a lot of content that dedups to existing content, it produces lots of shelf archives to be deleted. In these cases database tables stay locked for extended periods of time, resulting performance degradation for other commands. This change improves on this kind of database lock time. #2467384 (Job #115204) ** For 'p4 changes -m @', this change improves performance by quickly skipping over changes that have a non-matching path. #2464103 (Job #115529) ** Many 'p4 diff2' commands run simultaneously can hold lock on db.revsh blocking other commands for long periods of time. This change improves performance by making db.revsh access lockless. #2463813 (Job #115585) ** 'p4 topology --explain' now displays the long-form syntax for all the topology options. #2463373 (Job #116078) ** A chained edge uses the server protocol version of the upstream edge rather than the server protocol version of the commit server. This has been fixed. #2463256 (Job #115584) ** 'p4 topology -m' command now results in topology usage error when used with the options '-a' or '-d'. #2462781 (Job #116121) ** P4ALTSYNC will now generate a digest on the fly for files that do not have checksums stored in the database. #2462372 (Job #116376) ** The precedence of overlapping ChangeView entries, including those generated for streams with multiple "import @change" Paths, was not honored correctly. This has been fixed. #2462343 (Job #116382) ** 'p4 fstat -e CHANGE' would unnecessarily scan db.rev records for files open in other changelists, making it very inefficient when there were many such files. This has been fixed. #2461940 (Job #116409) ** The stdout is now flushed after the starting checkpoint message is generated. This is to help scripts determine the current checkpoint number in use. #2461702 (Job #116306) ** Multifile checkpoints and dumps could contain duplicate records. This has been fixed. #2461701 (Job #116373) ** The p4d help message now includes details on the -xj option to display db.ckphist records. #2460758 (Job #115558) ** 'p4 unshelve' is no longer allowed for altsync enabled clients when P4ALTSYNC is not configured correctly. #2459572 (Job #116187) ** P4ALTSYNC now works as expected with forwarding replicas. #2459572 (Job #116348) ** Command group logging information is now correctly passed upstream from forwarding-replicas. #2459453 (Job #116334) ** Tracking metrics for 'rdb.lbr' activity is now reported for scheduling the transfers of missing and bad depot files during a 'p4 verify -t' command. #2457982 (Job #116141, #116277) ** Fixed problems with upgrade when a stream's view could not be generated. #2456741 (Job #116101) * ** cURL (for extensions) has been upgraded 8.1.2 to address multiple cURL CVEs: CVE-2023-28319 and CVE-2023-28322 #2455354 (Job #115964) ** 'p4 streams -F' would automatically escape the '&' query operator in the special case where it immediately followed the string 'import'; this would cause a query parse error when combining two queries with '&' if the first query ended with 'import'. The automatic escaping behavior has been removed and 'p4 help streams' has been clarified to explain how to escape query operators that are part of a search expression. #2454956 (Job #115981) ** 'p4 sync' of a symlink type file with P4ALTSYNC no longer causes an infinite loop. #2453580 (Job #116086) ** 'p4 fstat -m1 -S //...` once again performs a correct count of all files. #2452968 (Job #114741, #115631, #115812) ** 'p4 streams --viewmatch' has been significantly reworked in order to more accurately return the exact set of streams whose views match a given depot file and to eliminate unnecessary scans of db.streamviewx. #2452668 (Job #116033) ** 23.1 or later edge servers will now omit the altsync option if the commit server has not been upgraded to at least 23.1. #2450824 (Job #114820) ** 'p4 reload -p' will now correctly report errors to the users and if a journalwait is required, the correct journal offset is now used. #2449796 (Job #116005) ** 'p4 add' of a file in a client overlay path could unnecessarily scan all mapped db.revhx records. Fixed. #2448510 (Job #114845) ** Parallel checkpoint and dump operations now create an aggregate MD5 file to indicate success. #2445284 (Job #114041) ** For 'p4 submit' of a graph depot repository if the repository includes submodule the submit would corrupt the repository by replacing the submodule with the directory of the submodule. This change fixes the problem. #2442805 (Job #115714) ** 'p4 fstat -Oz' did not display lazy copies of purged or deleted revisions. This has been fixed. #2442681 (Job #115619) ** Fixed a 'p4 zip' bug where one half of a move/delete pair was wrongly declared to be missing. #2440369 (Job #115687) ** This ensures that the respective intermediary server log is not spammed with the topology server warning related to skipping the capture of the server records in the db.topology table in case of any upstream server that is not upgraded to the version >= 2021.2. #2437067 (Job #110437) * ** *** **** The network autotune algorithm has been corrected to address the rare cases where large numbers of files being submitted to a Linux server from a Windows client could stall or hang if autotune was enabled (net.autotune=1). #2434383 (Job #115391) ** Fixed stream view generation at a specific change causing an endless loop. #2433630 (Job #115113) ** Missing archives no longer cause 'p4 renameuser' to crash. #2433577 (Job #115504) ** For the graph depot command 'p4 receive-pack' if the pack-file includes merge commits, it is possible for the process to take apparently infinite time and never finish. This can show up as a git-connector mirroring update that never completes. This change fixes the problem. #2427917 (Job #114334) ** If a stream has one or more components, and a stream client is set to have a StreamAtChange for a change which did not involve the stream spec, then the client view might not include the component stream views, or they might be wrong. This has been fixed. #2427515 (Job #114761) ** A 'p4 retype -l' request on a revision with a shelved integration generates incorrect records in the db.storage and db.storagesh tables. This has been fixed. #2426423 (Job #100988) ** 'p4 undo' previously gave a empty error message when there were conflicts between different open actions for the same file. Now it correctly reports the problematic file, the two conflicting actions, and relevant changelist numbers that can be used to break the undo up into non-conflicting ranges. #2424554 (Job #87367) * P4IGNORE now defaults to '.p4ignore', 'p4ignore.txt', '$HOME/.p4ignore' and '$HOME/p4ignore.txt'. #2423690 (Job #115128) ** This change fixes a bug in 'p4 changes' when it is used with options '-m @'. If the '-m' value is bigger than 10% of the number of changes, and the filepath does not match the first file in a change but matches other files in the change, then the change would be skipped. This change fixes the problem. #2423392 (Jobs #114708, #114712) ** The default fields to the display checkpoint history command, 'p4d -xj', now includes the 'end' date stamp of the operation. The journal date field is now set to the current time for a parallel checkpoint or dump request. #2422126 (Job #114848) ** *** p4 pull -l' command could put shared lock on rdb.lbr for the whole duration of the command running, resulting overall performance degradation for edge or proxy servers. This improves the performance by changing 'pull -l' to quickly read all the records it needs and release the lock. -------------------------------------------------------------------------- Major new functionality in 2023.1 (2023.1/2442900) (2023/05/15) #2413980 (Job #104786) * ** *** Support for alternative sync agents The client spec has a new '[no]altsync' option which specifies if the client application should use an alternative sync agent (specified by the P4ALTSYNC environment variable) to perform file operations. The alternative sync agent will be invoked once per connection and will be provided with file metadata instead of the client application writing file content directly; the primary use-case for this functionality is to update the state of virtual file service providers. See: P4ALTSYNC in Helix Core Command-Line (P4) Reference #2350943, #2339529, #2328309 (Jobs #112184, #112183, #111439) ** Parallel checkpoint and dump operations accept an 'm' sub-option, that allows each individual database table to be split into multiple checkpoint or dump files. This should allow the checkpoint/dump workload to be spread more evenly between the available threads. The recovery operation supports the 'p' sub-option to recover a parallel checkpoint directory. See: "Parallel checkpointing, dumping and recovery" in the Helix Core Server Administrator Guide Technology Preview features in 2023.1 (2023.1/2442900) (2023/05/15) #2419976 (Job #5077, #67496, #72878, #108158, #108325, #110287) ** Helix Core Server System Resource Monitoring The server can now meter the amount of incoming work it accepts, pausing or rejecting client requests if the amount of free operating system memory or CPU availability is past a configurable threshold. This feature is platform-specific. CPU pressure responsiveness is only available on Linux systems with cgroups v2 configured. Memory awareness is available on all Linux versions as well as Windows, but not on MacOS. Contact Technical Support for guidance when enabling this feature in a production environment. See 'p4 help server-resources' for further information. Minor new functionality in 2023.1 (2023.1/2442900) (2023/05/15) #2413707 (Job #19809) * ** 'p4 sizes' has a new '-C' flag which causes the filesizes reported to be based on the size that the file would be if written with the current client connection; this accounts for line-ending conversion, charset conversion for unicode/utf8/utf16 filetypes (including the value of 'filesys.utf8bom') and keyword expansion for +k type files. These variables may be overridden with the following additional new flags: -B utf8bom - Override 'filesys.utf8bom' for utf8 type files -K - Collapse keywords for +k type files -L line-ending - Override the client's line ending setting -Q charset - Override P4CHARSET for unicode type files #2408924 (Job #114224) ** Logs now include the amount of time a command has spent in the paused state (e.g. via 'p4 monitor pause'). Tracking output in the P4LOG log file will show lines like '--- paused $seconds'. Version 56 of the 'CommandEnd' event (1.56) in the structured logs has a new field that represents the same data. The 'serverlog.version.N' configurable can be used to pin a structured log file to a specific server version's format. To retain the prior structured log events format, set the 'serverlog.version.N' configurable to 55. #2399143 (Job #95529) ** The new 'p4 integrated -m ' option reports the latest integration records. #2397436 (Job #112553) ** The server on Linux can now protect itself from the OOM killer during database writes. This happens automatically when the necessary capabilities have been granted to the server executable, e.g. "sudo setcap 'CAP_SYS_RESOURCE=+ep' p4d". A new server startup log entry is now written with details about the detected level of support for features like this. #2396097 (Job #112520) ** 'p4 topology -m' now allows the user to mark the server record as moved. Please see 'p4 help topology' for details. #2394721 (Job #112311) ** New server type 'distribution-server', can be used for content distribution, requires new specific license. #2380883 (Job #113299) ** The 'p4 admin checkpoint' command now supports the parallel checkpoint and multifile options (-p, -m, -N threads). #2373917 (Job #112371) ** Long-running server processes now release more memory back to the OS, resulting in up to half as much memory usage in some situations. #2368014 (Job #112446) * ** Compression-related tasks are now ~38% faster on 64-bit X86. E.g. 'p4 submit' on 'binary' filetypes. #2367890 (Job #110101) *** **** The default Perforce mimalloc memory manager configurables are now used in the Proxy and Broker. #2359921 (Job #111565) ** New flag --preserve-change-numbers added to 'p4 push' and 'p4 fetch' may be used to preserve the pushed/fetched change numbers on the target server, command will fail if there are any conflicting changelist numbers. Configurable 'dm.fetch.preservechangenumbers' disabled by default, can be enabled (set to 1) on the target server for the same effect, server restart not required. #2358093 (Job #112666) ** 'p4 topology -D' now accepts the destination server address and the server id to delete the record. Please see 'p4 help topology' for details. Bugs fixed in 2023.1 Patch 4 (2023.1/2560797) (2024/02/22) #2557622 (Job #119417) ** Fixed a crash when 'p4 unshelve -b/-S' encountered an error. #2557496 (Job #116539) ** It was possible for the table dump command 'p4d -jd' to create the MD5 checksum file even when the command had failed. This has been fixed. #2557184 (Job #116293) ** 'p4 search' could get stuck when putting a read-lock on the ixtext table. This change fixes the problem by switching to lockess access for the table. #2557097 (Job #102151) ** This change improves 'p4 sync' performance with complex views by increasing the lookback limit from the previous value of 8 to a value set by the new configurable maxLookBack with a default value of 1000. #2553623 (Job #119254) ** A parallel recovery would leak memory on note entries in the replay file(s). This has been fixed. #2552713 (Job #118004) ** On-demand fetching of archives from a replica will now cause the commit server to perform a fetch on-demand if the archive has not yet been transferred from an edge server as part of a background submit. The client requesting the file content will no longer receive a librarian error in this situation. #2552001 (118593) ** An Extension implementing a 'pressure-pause' hook no longer prevents updates to the trigger table. #2547700 (Job #118566) ** Parallel recoveries did not detect a character set mismatch between the server and recovery files. This has been fixed. #2546175 (Job #118002) ** 'p4 obliterate' can no longer write archive delete journal notes when a journal rotation is in progress. #2538969 (Job #115632) ** Fixed auto-shelve of modified content on distribution server when switching between streams. Bugs fixed in 2023.1 Patch 3 (2023.1/2534247) (2023/12/15) #2531279 (Job #116191) ** Fixed DVCS fetch, push and zip commands incorrectly reporting a split move across changelists. #2531018 (Job #118342) ** Fixed segmentation violation with real-time monitoring enabled. #2531015 (Job #118395) ** Set digest/filesize on text+k files for 'p4 change -t' command. Bugs fixed in 2023.1 Patch 2 (2023.1/2513900) (2023/11/03) #2512341 (Job #117646) ** 'p4 unshelve' with multiple files as parameters, if this file-list is not sorted, could erroneously report 'can't unshelve (already opened for edit using a different client view path)'. This change fixes the problem. #2510947 (Job #117156) ** Security fix. Addressed CVE-2023-35767. #2510257 (Job #117185) ** The 'db.checkpoint.threads' configurable override was not supported for non-parallel checkpoint and dump requests started directly with 'p4d' via the '-N threads' option and requests started by the 'p4 admin checkpoint' command via the '-n threads' option. This has been fixed. #2510009 (Job #112432) ** 'p4 fetch' and 'p4 push' commands now update LastFetch/LastPush when user provided filespec argument does not limit remote view. #2510008 (Job #117155) ** Security fix. Addressed CVE-2023-45849. #2510007 (Job #90618, #117153) * ** *** **** Security fix. Addressed CVE-2023-5759. #2509225 (Job #111780) ** For 'p4 shelve' with a lot of content that dedups to existing content, it produces lots of shelf archives to be deleted. In these cases database tables stay locked for extended periods of time, resulting performance degradation for other commands. This change improves on this kind of database lock time. #2508491 (Job #116102) * ** *** **** OpenSSL has been upgraded to 3.0.12 #2508379 (Job #117154) ** Security fix. Addressed CVE-2023-45319. #2507034 (Job #117098) ** For a parallel checkpoint operation, the start time and end time in the checkpoint history record were the same and the 'checkpoint' and 'journal' fields in the 'rotate-journal' trigger were not set. This has been fixed. #2505991 (Job #117741) * ** cURL (for extensions) has been upgraded 8.4.0 to address multiple cURL CVEs: CVE-2023-38545 and CVE-2023-38546 #2505638 (Job #117392) ** There was a data race between threads in the 'p4 storage -U' command. This has been fixed. #2498961 (Job #117071) ** For 'changes -m -u' it may produce less than m lines, missing valid records in the output. For 'p4 changes' with -i closure option, it may produce result that does not include changes for the files in the source of integration. #2498788 (Job #115204) ** For 'p4 changes -m @', this change improves performance by quickly skipping over changes that have a non-matching path. #2497916 (Job #114933) ** When a lazy copy file is sent using 'p4 unzip', 'p4 push' or 'p4 fetch' and the source is not part of the package, it will be imported as a non-lazy copy file. #2494346 (Job #117292) ** Parallel checkpoint and dump requests do not honor the 'db.checkpoint.bufsize' configurable. This has been fixed. Note, with this fix, 'filesys.bufsize' in no longer used to determine the checkpoint write size. #2494008 (Job #117270) ** Fixed a component stream validation error when reparenting a stream. #2489953 (Job #117222) ** Ensures that 'p4d -jr' command to replay the checkpoint in a non empty directory does not update the existing database files. #2489812 (Job #113319) ** Using 'p4 unzip' with -I option to exclude integration records no longer causes a crash when the source of a lazy copy is not included in the zip. #2489147 (Job #106491) ** 'p4 zip' correctly includes archives for lazy copies when used with a revision range. #2480069 (Job #117741, #117093) ** The orphan scanner, 'p4 storage -l', does not work with space in the depotdirectory argument. This has been fixed. #2478674 (Job #114250) ** Concurrent 'p4 unlock' and 'p4 lock' commands can result a deadlock and system hang. This change fixes the problem. #2473031 (Job #115529) ** Many 'p4 diff2' commands run simultaneously can hold lock on db.revsh blocking other commands for long periods of time. This change improves performance by making db.revsh access lockless. #2468926 (Job #115585) ** 'p4 topology --explain' now displays the long-form syntax for all the topology options. #2468905 (Job #115584) ** 'p4 topology -m' command now results in topology usage error when used with the options '-a' or '-d'. #2468481 (Job #110437) * ** *** **** The network autotune algorithm has been corrected to address the rare cases where large numbers of files being submitted to a Linux server from a Windows client could stall or hang if autotune was enabled (net.autotune=1). Bugs fixed in 2023.1 Patch 1 (2023.1/2468153) (2023/07/24) #2462781 (Job #116121) ** P4ALTSYNC will now generate a digest on the fly for files that do not have checksums stored in the database. #2462349 (Job #116376) ** The precedence of overlapping ChangeView entries, including those generated for streams with multiple "import @change" Paths, was not honored correctly. This has been fixed. #2462206 (Job #116382) ** 'p4 fstat -e CHANGE' would unnecessarily scan db.rev records for files open in other changelists, making it very inefficient when there were many such files. This has been fixed. #2461940 (Job #116409) ** The stdout is now flushed after the starting checkpoint message is generated. This is to help scripts determine the current checkpoint number in use. #2461701 (Job #116373) ** The p4d help message now includes details on the -xj option to display db.ckphist records. #2460758 (Job #115558) ** 'p4 unshelve' is no longer allowed for altsync enabled clients when P4ALTSYNC is not configured correctly. #2459572 (Job #116348) ** Command group logging information is now correctly passed upstream from forwarding-replicas. #2459572 (Job #116187) ** P4ALTSYNC now works as expected with forwarding replicas. #2459439 (Job #114905) ** 'p4 sync' is now more tolerant of out-of-memory conditions. Note that the extra error handling included in this fix makes 'p4 sync' honor MaxResults for files synced to the workspace or opened, where it previously did not. #2459434 (Job #114906) ** 'p4 filelog' is now more tolerant of out-of-memory conditions. #2459094 (Job #114845) ** Parallel checkpoint and dump operations now create an aggregate MD5 file to indicate success. #2459030 (Job #116306) ** Multifile checkpoints and dumps could contain duplicate records. This has been fixed. #2458373 (Job #116101) * ** cURL (for extensions) has been upgraded 8.1.2 to address multiple cURL CVEs: CVE-2023-28319 and CVE-2023-28322 #2458299 (Job #116141, #116277) ** Fixed problems with upgrade when a stream's view could not be generated. #2455092 (Job #114741) ** Fixed a stream viewmatch bug where an exclude path might obscure a share path in the generated streamviews. #2454956 (Job #115981) ** 'p4 sync' of a symlink type file with P4ALTSYNC no longer causes an infinite loop. #2453231 (Job #114820) ** 'p4 reload -p' will now correctly report errors to the users and if a journalwait is required, the correct journal offset is now used. #2453069 (Job #116086) ** 'p4 fstat -m1 -S //...` once again performs a correct count of all files. #2452668 (Job #116033) ** 23.1 or later edge servers will now omit the altsync option if the commit server has not been upgraded to at least 23.1. #2449758 (Job #116005) ** 'p4 add' of a file in a client overlay path could unnecessarily scan all mapped db.revhx records. Fixed. #2448511 (Jobs #114708, #114712) ** The default fields to the display checkpoint history command, 'p4d -xj', now includes the 'end' date stamp of the operation. The journal date field is now set to the current time for a parallel checkpoint or dump request. #2447644 (Job #115980) * ** *** **** OpenSSL has been upgraded to 1.1.1u #2446515 (Job #114761) ** A 'p4 retype -l' request on a revision with a shelved integration generates incorrect records in the db.storage and db.storagesh tables. This has been fixed. #2445284 (Job #114041) ** For 'p4 submit' of a graph depot repository if the repository includes submodule the submit would corrupt the repository by replacing the submodule with the directory of the submodule. This change fixes the problem. #2444620 (Job #115391) ** Fixed stream view generation at a specific change causing an endless loop. #2444614 (Job #115619) ** Fixed a 'p4 zip' bug where one half of a move/delete pair was wrongly declared to be missing. #2443869 (Job #115687) ** This ensures that the respective intermediary server log is not spammed with the topology server warning related to skipping the capture of the server records in the db.topology table in case of any upstream server that is not upgraded to the version >= 2021.2. #2442805 (Job #115714) ** 'p4 fstat -Oz' did not display lazy copies of purged or deleted revisions. This has been fixed. Bugs fixed in 2023.1 (2023.1/2442900) (2023/05/15) #2440503 (Job #115503) * ** cURL (for extensions) has been upgraded 8.0.1 to address multiple cURL CVEs: CVE-2023-27537, CVE-2023-23916, CVE-2023-23915 and CVE-2023-23914 #2439308 (Job #112863, #115676 ) ** The 'p4 failover' command could be run from a server that has been prepared for the 'p4 failback' command by running 'p4d -Fm'. Now 'p4 failover' will fail in this situation with an error message instructing the user to run 'p4 failback' instead. #2434101 (Job #115504) ** For the graph depot command 'p4 receive-pack' if the pack-file includes merge commits, it is possible for the process to take apparently infinite time and never finish. This can show up as a git-connector mirroring update that never completes. This change fixes the problem. #2429038 (Job #114515) ** Running 'p4 reopen' with multiple file arguments that include moved files could cause a member of a moved pair to fail to be reopened. This has been fixed. #2423940 (Job #115128) ** This change fixes a bug in 'p4 changes' when it is used with options '-m @'. If the '-m' value is bigger than 10% of the number of changes, and the filepath does not match the first file in a change but matches other files in the change, then the change would be skipped. This change fixes the problem. #2419973 (Job #111678) ** Starting with release 2023.1, servers using central authentication (P4AUTH) require the central server's release version to be at least 2023.1. The central server must have a valid license, and must be running when its outer servers are starting up or being upgraded. #2419808 (Job #113805) ** Queries against a client path with an overlay view would previously be applied to all matching depot paths, meaning that a command executed against a single file could return two (or more) results. This behavior has been changed so that as a general rule, commands which are expected to return a single result per file will only return one depot result per client file when a client path is used, according to the precedence specified in the client view. Queries against depot paths, or queries which permit multiple revisions to be returned per matching file, are not affected by this change. #2417619 (Job #113622) ** StreamQWorker is now launched at server startup to finish any leftover processing. #2417343 (Job #112669) The Helix Core Proxy installers (.exe and .msi) use the property P4LOG instead of PRXYLOG to specify the name or path and name of the Helix Core Proxy log file. Note that when upgrading from an old version you will need to use the new property name. #2415891 (Job #67312) ** The Remapped field in stream specifications did not allow a target path of "...", i.e., remapping into the root. This has been fixed. #2415279 (Job #94950) ** Improved locking mechanism for the 'p4 unload' operation. #2412791 (Job #113923) ** When using the 'filesys.atomic.rename' configuration, an incorrect journal size might be reported. This has been fixed. #2411873 (Job #114552) ** The performance of 'p4 populate' has been improved by optimising the inserts of integration records. #2409959 (Job #112599) ** An integration from a stream client might fail with an error message "The target stream of a stream spec integration must be the stream associated with the current client ...". This error should only occur when using a '-S' parameter with the integration command, but did sometimes occur when not using '-S' when the parent of the associated stream has a noinherit ParentView setting. This has been fixed. #2409758 (Job #113006) ** The 'p4 client -df' command run on an edge server could result in pending changes that are deleted on the edge but not on the commit server. This would cause the db.change table contents on the commit server to differ from the contents on the edge server. This has been fixed. #2409493 (Job #112877) ** Replication of large transactions is now more efficient. Previously if the upstream server had yet to complete a transaction, replicas could get to the end of the running journal and then restart the replication from the last consistency point; the repeated records would be retained in memory on the replica. Replicas now continue replication of transactions across requests to the upstream server, preventing both repeated network transfers and reducing memory use. #2407879 (Job #113804) ** Depot paths that are overlaid in the client view (i.e. lower in precedence than a +-mapping for the same client path) are now considered read-only, similarly to &-mapped paths, in order to avoid confusion around adding or branching files within ambiguously mapped client paths. #2407742 (Job #113467) ** When a file was opened for 'add' with the exclusive open filetype in a distributed environment and was submitted with the 'reopen' option, the exclusive open type would not be set. This has been fixed. #2405801 (Job #113970) ** Changes to the configuration 'filesys.atomic.rename' now only take effect upon Server startup. After changing this configurable, a 'p4 admin restart' is required. #2405137 (Job #113841) ** When using the SDP to recover a rotated / numbered journal from an active Server, a sharing violation will no longer occur. #2404949 (Job #114063 ) ** If 'p4 license -i' fails due to an IP address change in a license file, the error message now shows both the previous and expected IP addresses. #2403655 (Job #114243) ** Intermediary server (Replica, Proxy, Broker, etc) login no longer leads to the server crash if the commit server starts with an unspecified host address. #2403229 (Jobs #113829, #113840) ** Parallel checkpoint, dump and recovery operations now create a db.ckphist record. p4d now supports the '-xj' option to display db.ckphist records. #2399402 (Job #113830) ** The configurable, 'db.checkpoint.nofiles', that specifies the default split parameter for parallel multifile checkpoint and dump operations has been renamed to 'db.checkpoint.numfiles'. The help page and usage message for 'p4 dbstat' have been updated to reflect this name change. #2398674 (Job #112505) ** The 'p4 group' on a replica working with an older commit server would fail with error: "Unkown field name 'Description'". This change fixes the problem. #2394917 (Job #102021) ** Replication on edge servers after large obliterates suffered from poor performance due to excessive scanning on the db.domain table. The performance issue caused by excessive scanning of the db.domain table has been fixed. #2393897 (Job #113904) ** 'p4 shelve -f -c' no longer leads to the deletion of shared shelve archive files on the replica server. #2393497 (Job #114030) **** Broker now handles MaxMemory when provided by the Helix Core Command Line client (P4). #2393485 (Job #113985) ** Using the '-p' option to 'p4 dbstat' with a worklevel of 1 could cause a server crash. This has been fixed. #2392837 (Job #113967) ** Under certain configuration limitations 'p4 fstat' could throw an exception. This has been fixed. #2392735 (Job #113267) ** For 'p4 sync -E @