December 6, 2010

p4 integ -3: exciting things afoot!


Those who followed the proceedings of the 2010 Perforce User Conference, or who keep a close eye on "p4 help undoc", are probably already aware of the work we've been doing on the new integration engine that we've been referring to informally as "integ -3". Over the past few releases we've been steadily polishing the new engine up to provide generally better results and better performance than the currently supported engine.

As of the 2010.2 beta release, the new engine (still "undoc" for now, but stay tuned) sports some entirely new functionality, which I am thrilled to finally be able to share with the world.

The new "help undoc" entry for "integ -3" briefly outlines the features that are new for this release.

   p4 integrate -3 -Mbdo -Rs
       The '-3' engine is designed to produce more accurate results than the
       current engine, and is intended to replace it in an upcoming release.
       By default, 'integrate -3' will detect target files that have been
       previously moved with 'p4 move', and automatically 'retarget' itself
       to follow those move operations.  The '-M' flag modifies this:
           -Mb : only retarget within branch/toFile view
           -Md : retarget when source is deleted
           -Mo : disable retargeting (old behavior)
       The '-3' engine also provides an alternative to the default behavior
       of scheduling a single resolve per file:
           -Rs : split resolves around previously-integrated revisions

Yes, you read that right -- "p4 integ" is going to notice when files have been renamed, and automatically merge into the new file instead of trying to re-branch the old one! This behavior is keyed off the special integration records created by the 2009.1 "p4 move" command, so if you've still been using the old "integ"/"delete" method, now is a good time to make the switch. (If you've been reluctant because of the lack of lazy copy behavior, that's changed in 2010.2 as well; see change 271044 in the release notes.)

Making this change involved revising our notion of what it means for two files to be "related" such that integrating between them makes sense, and developing logic within the new integration engine that notices when two files that are mapped together by the branch view don't seem to have any content in common, such as when one has been deleted or overwritten by a copy from an unrelated file. If the integration engine notices that the two files it's trying to merge aren't related, it will backtrack through the history, try to find a point where the two files were related, and then see if the target file was "moved" somewhere around that point so that it can retry the merge with the target of that move operation. The upshot of this is that the new engine should be able to correctly match the source up with the right renamed target even if the original target has been re-added or re-branched from other sources a number of times. It may also be a little bit more prone to throwing "baseless integrate" warnings in cases where it's pretty sure the files aren't related but it can't find any better options by following "moved" records.

Integrating into a moved file
Integrating into a moved file.


The other piece of new functionality is the "-Rs" flag, which allows integrate to "split" resolve records around credited revisions. In most cases, performing a single merge is easier than performing multiple merges, so historically integrate has always scheduled only one merge to account for all unintegrated changes, even if that one merge would then need to include changes that did not themselves require integration.

With this new flag, integrating a single file may schedule multiple merges if deemed necessary to avoid re-integrating changes. In some cases this may cut down on merge conflicts, and in others it may just force you to deal with the same conflicts more than once. Where it's most useful is in cases where you've "cherry-picked" and "ignored" a specific change, and you want to be able to do a real merge of all the changes before and after (but not including) the ignored change. A more widely visible benefit is that the "-3" version of "p4 interchanges" uses this logic automatically, which means that it will no longer report cherry-picked changes as outstanding.

Integrating around an ignored revision
Integrating around an ignored revision.


In the time that we've been maintaining "integ -3" in its undoc state, we've received a good amount of user feedback on things it's been doing right (and a number of things it's been doing wrong, which we've been fixing as we go). Remember that you can set up a two-user server for testing without even asking us for a duplicate license, and that you can use "p4 integ -3 -n" to see what the new engine would do but without actually scheduling a merge.Now that we're branching out into having integrate do entirely new things, I'm hoping our interested customers will continue to take advantage of this early availability to test it out and let us know how it works for them.