May 7, 2015

Now Faster Than Ever: Syncs and Submits


Last year I published a blog article Lightning Fast Syncs where I described how to improve the performance of p4 sync when there was network latency involved. The abbreviated version is that we enabled compression, increased the network buffer sizes, and fetched files in parallel. The tests included a mix of text and binary files totaling 135MB with 140ms latency. I think it served as a very good reason to upgrade.

With the release of 2015.1 I reran those tests again:

p4 configure set net.parallel.max=5 # enable parallelismp4 sync ... # standard syncp4 sync --parallel "threads=4,min=1,minsize=1" ... # parallel sync

Not surprisingly, the end results remain the same as the previous optimized results, with the standard sync completing in 44 seconds and parallel sync completing in 15 seconds. One new aspect is that since the default windows size has changed to 512KB, I did not have to add that to the configuration or tests.

The “Parallel Sync” feature was introduced in 2014.1 and improves the performance of delivering files from P4D to a client over networks with latency.

With the 2015.1 release of Helix Versioning Engine (P4D) the same parallelism is extended from the client to P4D with the “Parallel Submit.” Testing with the same data I observed the same ratio of improvement:

p4 submit -d “add or edit of 134MB” # standard submitp4 submit --parallel "threads=4,min=1" -d “add or edit of 134MB” # parallel submit

The standard submit completes in 65 seconds and parallel submit completes in 21 seconds. And while we can't match the 2000+% performance improvements we saw when comparing 2014.1 to prior versions, the 300+% improvement we see here with 2015.1 is still quite a productivity boost.

In summary, the Parallel Sync feature was a good justification to upgrade last year. And while there are lots of new features in 2015.1 (see DVCS), the Parallel Submit is one that may justify an upgrade all by itself. To take it for a test drive and let us know what you think.

P.S. As shown previously I created a custom tool to achieve the same performance with P4V.

Parallel Submit – submit files in parallel