April 2, 2014

Boost Productivity with Parallel Sync & Concurrency Optimizations


Perforce continues to push the bar on performance. In release 2013.3, Perforce introduced concurrency optimizations (lockless reads) that dramatically reduced the latency in executing read commands against the Perforce versioning engine (P4D).

We have seen as much as a 24x reduction in wait times at some customer sites for executing day-to-day commands on their busiest Perforce servers. Some of these servers execute more than 6 million commands per day.

In release 2014.1, we’re introducing Parallel Sync—a unique capability that transfers multiple files simultaneously (instead of serially) as part of the sync command, thereby greatly reducing the overall sync time.

Users can leverage parallel sync by using the following command:

‘P4 sync –-parallel 

Parallel Sync accelerates creation of new workspaces and refresh of existing ones, and as a result, speeds up everything in your continuous delivery pipeline, from development to continuous build and integration.

To analyze the performance of Parallel Sync, we ran tests in our lab with P4 sync fetching 1,000 files in parallel on a Perforce server connected to the client over a LAN, and in a separate test, between client and server simulated to be in the United States and India, with 300 milliseconds WAN latency in-between.

Results of the two tests are shown in the graph below:

The results in red show Parallel Sync lapse times over the LAN compared to the results in blue over the WAN.

Over the LAN, simply running two parallel sync threads instead of a single thread reduced the sync transfer time by 31% (from 86 to 59 seconds). Results are even more impressive over the WAN, where running two parallel sync threads reduced the transfer time by 39% (from 368 to 224 seconds).

Running more parallel sync threads will further improve performance, however, the curve flattens out at some point when lots of parallel sync threads (16 threads in our test) fully utilize the available resources on the server, network, and/or client.

Key Benefits of Parallel Sync

Parallel Sync enables developers to be more productive. Perforce enables developers to fetch files from the versioning service faster than they did before, reducing the time it takes developers to populate their workspace, and in return, enabling them to complete their tasks faster—whether it’s a new feature they are working on or a bug fix.

Parallel Sync speeds up your continuous delivery pipeline. Build automation and continuous integration processes must sync their workspaces to the target revision that is to be tested. These processes can be made much faster by syncing files in parallel. The gains are particularly large especially when projects span hundreds of thousands of files. As a result, DevOps teams can ship quality products faster.