Developer Notes for P4API, the Helix Application Programming Interface for C/C++ P4, the Helix Command line client P4P, the Helix Proxy P4Broker, the Helix Broker Version 2023.2 Introduction This document is contains developer notes for: * Helix C/C++ API and other derived APIs and * Helix Client (P4) * Helix Proxy (P4P) and * Helix Broker (P4Broker) For details about installing and using the Helix C/C++ API, refer to the API User's Guide, posted on the Perforce Web site at: http://www.perforce.com/perforce/technical.html Supported Platforms for p4api Linux kernel 2.6+ for Intel(x86,x86_64) Windows for Intel(ntx86, ntx64) Apple OS X 10.12+ for Intel(x86_64) MacOS 12+ for Apple silicon (arm64) Important Platform End-of-Life Notice This notice identifies the platforms that will be placed in End-of-Life (EOL) status in November 2023. This applies to the p4api component of the server release. The following 32-bit platforms are now in the Obsolescent phase: Linux kernel 2.6+ for Intel(x86) OBSOLESCENT Phase: Starting with this release,these platforms will no longer be supported. Customers will receive full support and patches for the deprecated platforms during this phase for older releases. The platforms will remain in this phase until November 2023 after which they will transition into the discontinued phase. DISCONTINUED (EOL) Phase: Patches will no longer be issued for these platforms. We will no longer provide technical support to customers with servers on these platforms SSL Support Beginning with the 2017.1 release of the Helix C/C++ API, the dependency on OpenSSL is now enforced and the SSL stub library has been removed. Executables linked against the P4API libraries must also be linked against real OpenSSL libraries: the latest 3.0.x or 1.1.1 patch is recommended. ------------------------------------------------------------------------- See marks in the notes below: * -- requires new p4 client program including all client applications and derived APIs ** -- requires new p4d server program *** -- requires new p4p proxy program **** -- requires new p4broker program -------------------------------------------------------------------------- Technology Preview features: Technology Preview features are currently unsupported, might not be functionally complete, and are not suitable for deployment in production. These features are provided to the customer to solicit interest and feedback, with the goal of full support in future releases. Customers are encouraged to provide feedback and functionality suggestions for Technology Preview features before they become fully supported. ------------------------------------------------------------------------- Major new functionality in 2023.2 (2023.2/2519561) (2023/11/14) Minor new functionality in 2023.2 (2023.2/2519561) (2023/11/14) #2479379 (Job #117100) ** The stream spec's Type is now a 'select' field. Bugs fixed in 2023.2 Patch 3 #2558850 (Job #119500) * Reuse of a ClientApi object across connections no longer causes "Unknown command" errors after client statistics should have been reported to the server. Bugs fixed in 2023.2 Patch 2 (2023.2/2535420) (2023/12/19) #2512914 (Job #90618, #117153) * ** *** **** Security fix. Addressed CVE-2023-5759. Bugs fixed in 2023.2 (2023.2/2519561) (2023/11/14) ------------------------------------------------------------------------- Major new functionality in 2023.1 #2413980 (Job #104786) * ** *** Support for alternative sync agents The client spec has a new '[no]altsync' option which specifies if the client application should use an alternative sync agent (specified by the P4ALTSYNC environment variable) to perform file operations. The alternative sync agent will be invoked once per connection and will be provided with file metadata instead of the client application writing file content directly; the primary use-case for this functionality is for update the state of virtual file service providers. For more information, see: P4CMDREF-TBD Minor new functionality in 2023.1 #2341549 (Job #111111) ** The ClientUser class has two new thread-safe setters (SetEnviro/SetVarList). #2383088 (Job #108604) * Support for OpenSSL 3.0.x has been added to the P4API. Bugs fixed in 2023.1 Patch 3 #2510007 (Job #90618, #117153) * ** *** **** Security fix. Addressed CVE-2023-5759. Bugs fixed in 2023.1 Patch 1 #2458373 (Job #116101) * cURL (for extensions) has been upgraded 8.1.2 to address multiple cURL CVEs: CVE-2023-28319 and CVE-2023-28322 Bugs fixed in 2023.1 #2440503 (Job #115503) * cURL (for extensions) has been upgraded 8.0.1 to address multiple cURL CVEs: CVE-2023-27537, CVE-2023-23916, CVE-2023-23915 and CVE-2023-23914 #2411652 (Job #114507) * The libp4script_cstub library has been added to the P4API distribution for consumers not needing client side extension support, substituting libp4script_c, libp4script, libp4script_curl and libp4script_sqlite. ------------------------------------------------------------------------- Major new functionality in 2022.2 Minor new functionality in 2022.2 #2315124 (Job #111351) ** The new tag 'commandGroup' can be set on the protocol or per-command to populate the new 'f_cmdgroup' field in structured logging. If the identifier is longer than the new 'log.cmdgrp.maxlength' configurable, the string is truncated. Bugs fixed in 2022.2 Patch 3 #2518890 (Job #90618, #117153) * ** *** **** Security fix. Addressed CVE-2023-5759. Bugs fixed in 2022.2 Patch 1 (2022.2/2407422) (2023/02/14) #2394679 (Job #108604) * Support for OpenSSL 3.0.x has been added to the P4API. #2393903 (Job #113200) * cURL (for extensions) has been upgraded 7.86.0 Lua-cURLv3 (for extensions) has been upgraded 0.3.13 ------------------------------------------------------------------------- Major new functionality in 2022.1 Minor new functionality in 2022.1 #2211477 (Job #108644) * The new Reset() and virtual VReset() functions of the StrDict class can be used to free underlying storage in subclasses of the StrDict class using, for example, StrBuf::Reset(). #2190761 (Job #108026) A new ClientUser::CreateProgress(int type, P4INT64 filesize) method has been added, allowing the client to choose whether to ignore the file based on filesize rather than all files smaller than 1K being ignored. Bugs fixed in 2022.1 Patch 6 #2531693 (Job #90618, #117153) * ** *** **** Security fix. Addressed CVE-2023-5759. Bugs fixed in 2022.1 Patch 3 (2022.1/2361553) (2022/10/28) #2351538 (Job #111229) * Client side decompression of unicode,+C utf8+C and utf16+C type files now correctly completes before the final charset conversion, avoiding partial character translation errors. Bugs fixed in 2022.1 #2224867 (Job #108934) * The SSL handshake timeout configurable 'ssl.client.timeout' is no longer overridden by shorter 'net.maxwait' configurable value. #2210165 (Job #108624) * The headers for all Msg* classes are now included in the P4API. #2196290 (Job #108260) * Leftover state that could block a 'p4 login' is now cleared. #2190311 (Job #108028) * Clients that receive invalid encoded file types from a 22.1 or greater server will now report an error. ------------------------------------------------------------------------- Major new functionality in 2021.2 #2142991 (Job #107010) * P4API builds for Apple Silicon are now supported using OS=MACOSX, OSVER=1101 (or greater) and OSPLAT=ARM64. Minor new functionality in 2021.2 #2154547 (Job #71723, #96278, #104780) * SSL certificate validation on MacOS requires that P4API be linked with the Security framework. This may require the addition of the build option: '-framework Security'. Bugs fixed in 2021.2 Patch 10 #2535383 (Job #90618, #117153) * ** *** **** Security fix. Addressed CVE-2023-5759. Bugs fixed in 2021.2 Patch 2 #2226020 (Job #108260) * Leftover state that could block a 'p4 login' is now cleared. #2224926 (Job #108934) * The SSL handshake timeout configurable 'ssl.client.timeout' is no longer overridden by shorter 'net.maxwait' configurable value. Bugs fixed in 2021.2 #2178786 * Building against Windows SDK 10.0.19041.0 will no longer result in symbol resolution errors. ------------------------------------------------------------------------- Major new functionality in 2021.1 Minor new functionality in 2021.1 Bugs fixed in 2021.1 #2013898 The Enviro class now has a copy constructor. ------------------------------------------------------------------------- Major new functionality in 2020.2 Minor new functionality in 2020.2 Bugs fixed in 2020.2 Patch 4 #2094760 (Job #98950) * P4Lua has been moved under a namespace to prevent class name collisions when building other derived APIs. Bugs fixed in 2020.2 Patch 3 #2079570 (Bug #105445) * Deprecated OpenSSL methods have been removed from the OpenSSL 1.1.1 builds of the P4API. Bugs fixed in 2020.2 ------------------------------------------------------------------------- Major new functionality in 2020.1 Minor new functionality in 2020.1 #1919703 The P4Libraries class now has thread initialization and cleanup routines. These must be called when using the P4API from within threads. Bugs fixed in 2020.1 Patch 6 #2093951 (Job #98950) * P4Lua has been moved under a namespace to prevent class name collisions when building other derived APIs. #2081046 (Job #105445) * Deprecated OpenSSL methods have been removed from the OpenSSL 1.1.1 builds of the P4API. Bugs fixed in 2020.1 Patch 1 #1989083 (Job #103202) * The 'Navigate to URL' message reported by 'p4 login' when using the Helix Authentication Service was emitted on stderr despite being an info message. This has been fixed. #1988112 (Job #103165) ** The GetArchiveFileInfo() method called within the Archive() extension callback now returns the archive revision as a string rather than an integer. Bugs fixed in 2020.1 ------------------------------------------------------------------------- Major new functionality in 2019.2 Minor new functionality in 2019.2 #1833237 Add a P4Libraries class to perform static library initializations. Programs using the P4API must use this class to safely initialize the P4API before use. #1844499 (Bug #99450) A memory leak in the Signaler class can now be avoided by either disabling the Signaler before program exit, or by calling P4Libraries::Shutdown(). #1835586 (Bug #99404) * Options class support a new "$" opts parse flag which indicates to stop parsing the command line after encountering the given associated keywords. #1833849 (Bug #98934) * The new StrOps::AddIndex() function adds an index to the %var% parameters in a buffer (i.e. %var% to %vari%). This can useful when an Error instance contains multiple occurrences of the same message with placeholders in the message's format. Bugs fixed in 2019.2 #1835637 (Bug #job099433) * Error::Merge() no longer SIGSEGVs if the 'ErrorPrivate *ep' is NULL in the source Error instance. #1813604 (Bug #98933) * ErrorPrivate::operator =() now always snaps the formats for self-assignment. ------------------------------------------------------------------------- Major new functionality in 2019.1 #1760437 * The P4API may now be build against OpenSSL 1.1.1 Note that the P4API now comes in OpenSSL 1.0.2 and 1.1.1 variants and that each may only be linked against its respective OpenSSL version. When including the P4API in a product that links against OpenSSL, the OpenSSL versions must match. #1749825 (Bug #97547) * ** A new ClientUser::HandleUrl() method has been added. This method will be called when the server requests that the client opens a URL. The default implementation uses 'xdg-open' and 'open' on Linux and Mac to launch the default browser. On Windows, the ShellApi is used to achieve the same result; note that this means that the P4API now requires shell32.lib to be linked in. To disable the default implementation from the environment, use 'p4 set' or P4CONFIG files to set 'P4USEBROWSER=false'. #1743677 * Autotune has been enabled by default. This functionality enables the TCP stack to manage the size of the network send and receive buffers, allowing more efficient use of the network, especially over slow high latency connections. This behaviour can be disabled in clients, proxies, brokers and the server by setting the 'net.autotune' configurable to 0; the default is now 1 (enabled). Clients can set this via "p4 set" or by adding the configurable to P4CONFIG files and servers can set this via 'p4 configure'. On Windows based platforms, send buffer sizes are not autotuned but are still manually configurable with the 'net.tcpsize' configurable. Technology Preview features in 2019.1 #1750665 (Bug #21301) * The Helix Core Client Extension mechanism is a feature that allows the client to run scripts written in the Lua programming language, packaged with metadata and other resources. The client runs these programs natively, providing a portable runtime with a programmatic API. See 'p4 help extension' for further info. Minor new functionality in 2019.1 #1735885 (Bug #373) * The verbose message output from 'p4 -s' now displays the actual error level: warning, error or fatal. Both error and fatal levels still cause 'p4' to return 1. For the case where existing scripts depend on the previous behaviour, the API level can be set with '-Zapi=85' to cause all non-info output to be tagged with error. Bugs fixed in 2019.1 #1742085 (Bug #97307) * Error::Merge() following an Error::Clear() on the target of the merge without an intervening Error::Set() now correctly returns only the ErrorPrivate ep content of the source. ------------------------------------------------------------------------- Minor new functionality in 2018.2 #1711178 (Bug #95847) * 'P4INT64' and 'long' data types for the 'value' argument of StrDict::SetVar() are now supported. #1663100 (Bug #95231) * A new ClientSSO class has been added so that clients may implement alternative login when P4LOGINSSO would normally be called. #1707917 * The FileSys/PathSys classes now have some Create* functions that return std::unique_ptr instead of a bare pointer. Bugs fixed in 2018.2 #1697165 ** The second factor authentication functionality has been renamed to multi factor authentication. This only affects documentation and messages sent to the client; configuration and tagged output have not been changed. #1671476 (Bug #95082) * The Rpc::Close() P4API function no longer does a blocking select(). Use the 'net.maxclosewait' configurable to control the timeout (defaults to 1 second). ------------------------------------------------------------------------- Minor new functionality in 2018.1 #1627525 * The FOM_UWRITE file open mode has been added. With this file open mode, the O_BINARY bit is added to the flags when opening FST_ATEXT files on platforms that support the O_BINARY bit. This can be useful when untranslated writes are needed on Windows. #1614386 (Bug #05109) * A new 'z' spec string option has been added. This causes the spec parser to ensure that the field name is present in the spec, even if no value is provided. This is primarily intended for specs that contain only a list, where the presence of the list (even if empty) is required to ensure the spec is valid. #1593778 * Exposing the case sensitivity setter in MapApi. This adds the method MapApi::SetCaseSensitivity( caseMode ) to the P4API. The mode may be set to either MapCase::Sensitive or MapCase::Insensitive. If not called maps use the platform's case sensitivity as normal. Bugs fixed in 2018.1 Patch 1 #1648316 (Bug #94969) Correcting an issue where the zlib within P4API may cause a linker error if a duplicate symbol _z_errmsg is exposed in another library being linked. ------------------------------------------------------------------------- Minor new functionality in 2017.2 #1557881 The Error::CheckIdi() and Error::CheckIds() methods have been added. These methods can be used to check for a particular code in error items other than the first. #1521323 * ** Two new ClientUser::Prompt() methods now have been added to support prompt messages being sent as Error objects. These will allow applications to identify the prompt by message code rather than needing to perform string comparisons. When overloading these methods, it is recommended to call the string handling equivalent method so that any existing overloaded logic can be reused: calling the wrong Prompt method could skip overloaded prompt methods and revert to the default stdio based prompt logic in ClientUser. #1520950 ** The 'p4 login' command now reports tagged output on success. If your application relies on the previous output either continue to use the 2017.1 (or older) client API, or set the protocol variable "api" to 81 (2017.1). For the command line 'p4' executable, reverting to the previous output can be achieved by using the -Zapi=81 global argument. Bugs fixed in 2017.2 #1568065 ** 'p4 client -o' did not generate Backup or Type fields in tagged output if the fields were set to their default values. Also, tagged output for 'p4 clients' did not output the Backup field. Now these tagged fields are always generated for these cases. ------------------------------------------------------------------------- Major new functionality in 2017.1 #1478710 * ** The client must set the 'enableGraph' protocol to indicate that it supports the additional output formats reported by the graph data model. #1454418 * The KeepAlive class which works with the Client::SetBreak method can now specify a polling interval time in milliseconds #1434004 * ** TCP protocol changes to improve performance of long latency connections. Significantly more data may be transported to or from servers and timely processing of results by clients is important. Bugs fixed in 2017.1 #1483947 (Bug #89040) * P4CLIENTPATH and the dvcs related P4INITROOT no longer restrict files from being written into the system temporary directory or the directory given by TEMP or TMP. To support this, there is a new Client::GetTempPath call.