January 13, 2016

P4V Goes Parallel

Flexible Workflows

How many times have you heard "Gotta sync the project, COFFEE BREAK!". No matter how fast your source control is, if you have to retrieve a ton of files from your server, you're gonna be moving at the speed of the internet. So how do you speed up the process? There's a couple ways to do it. One, improve your bandwidth, or two, move the files in parallel.

The 2014.1 Helix Versioning Engine added the ability to sync files in parallel, and the 2015.1 added the ability to submit files in parallel. The biggie feature in P4V's 2015.2 release is integrating in the ability to take advantage of this feature. Thinking about how it works now, when you sync or submit, one by one, each file in that list (either updating your files, or submitting them) gets sent across the internet. Picture a set of files where you have 1,000 little tiny text files and 1 gigantic 10GB file. In the traditional sync/submit mode, you'd happily chug along until you hit that giant file. Now you're stuck waiting on that one 10GB file to finish before the rest of your files. If the rest of the files are small text files? No big deal! They'll finish quickly. But what if you had another monster 10GB file hiding in there? Now you're waiting on that next file once your already big file finished! No bueno.

Enter, parallel functionality. Now, you can specify how many parallel 'threads' the client should use to sync or submit! Given the above scenario, each file goes into a thread to move across the pipe and as a thread becomes free, it grabs the next file in the list. This means, if you hit that first 10GB file, the other threads happily keep grabbing new files and sending them across the pipe simultaneously! Right away you've eliminated the problem where a couple big files can really hold up the rest of the operation. On its own, this dramatically improves performance for up/down from the Helix Versioning Engine, but combine it with tunables to customize the tcp send/receive buffer and it's a whole new ballgame.

Of note, this option is OFF by default (can turn it on in P4V preferences) because the server must be configured to allow parallel sync/submit. Server administrators can also force turn this on for any users against their servers. Check out the P4V release notes for the server property specifics on forcing the feature on or off."