July 18, 2016

Helix DVCS - Push Like a Pro

Media & Entertainment

Helix DVCS – Push Like a Pro

This post was a long time coming, but we are a bit busy at the moment. Nevertheless: if you read my previous posts on DVCS, you are probably curious how to push your changes to a remote server.

 

Push

You should all have successfully created local servers by now, either through an init or a clone. It is time to share your magnum opus with the rest of the world.

            p4 push

There! That wasn’t that hard, was it?

Yes, I admit, there is a lot more magic going on in the background to make sure this command works as simple as that. So let’s review the settings to guarantee success.

 

Prerequisites

First of all, your local server must have matching case sensitivity and Unicode settings to be able to push to another Helix server. If you cloned from that server you are fine, but if you initialised your independent copy first, you should use the “-p” option to extract the settings from the target or specify them by hand. Reread the “initialise like a pro” post if you are unsure.

You also have to have permission to push your changes and the target has to be configured to accept your changes (“administer like a pro” post).

 

Remote spec

You need to have a remote spec with the name “origin” defined that maps your local files to the correct location on the target server. If you cloned your repository, this remote will have been created for you automatically, otherwise you need to set up this remote.

If you remote is not called “origin”, or if you want to push to a different server, you need to specify the remote explicitly.

            p4 push –r <remote-name>

 

Remote user

You need to be logged into the remote server, that is, have a valid ticket. If you remember the discussion about users in remote servers from the “initialise” post, you know that your local user and your remote user do not have to coincide. Since my original post, the remote spec has picked up a new field: remoteUser.

In my case, if I want to push anything into the workshop, I tend to do that under the user “sven_erik_knop”, while my local user is “sknop”. Now I simply update my remote spec (“p4 remote origin” or “p4 remote <remote spec>”) and set “RemoteUser: sven_erik_knop” and login to the remote server with

p4 login –r <remote-spec>

If you are using 2016.1 P4 and P4D, you may have noticed a new feature: if you do not have a valid ticket, then P4 will prompt you for a password automatically when you issue a command; after successful authentication, P4 will proceed immediately.  This is very handy when pushing to a remote server – no need for a separate login.

A push can fail if you have changes waiting for you in the target codeline on the remote server. In that case, you will need to fetch and potentially merge those changes first; but this is a topic for the next post.

Finally, your push is not permitted if you have unsubmitted changes in your server. This can happen because you forgot to submit your local changes – or because you need to run a resubmit because of a previous unsubmit – also a topic for a separate post.

If all these prerequisites are met, the push command will (finally) succeed as planned.

 

Details on push

What is it we are actually pushing? How do we know which changes need to be pushed and what happens to the change numbers themselves? And what about integrations, attributes, labels and fixes?

Let’s look at these questions a little more in detail.

 

What is it we are actually pushing?

When you run ‘p4 push’, you will push changes, revisions and archives, together with potential integration records, attributes and job fixes. As part of a change, the user name, server id (recorded as the client workspace name), timestamps and of course the description are pushed to the remote server.

 

How do we know what to push?

Have a look at your remote spec. It contains the fields “LastFetch” and “LastPush”, both of which either contain the word “default” or a change number. LastFetch contains the last change you pulled down from the remote server (more about this in the next post), LastPush is the last local change you have pushed to that remote server.

Determining what needs to be pushed is then simply a matter of comparing the latest change number with LastPush, and transmitting all changes not previously pushed.

If you run ‘p4 help push’ you can see that there a couple of interesting options for this command. You can push individual streams (remember that local DVCS servers are always stream based, even if the remote server is not). You can push individual changes and even individual files.

In both cases, the LastPush field does not get updated. Instead, push will determine which changes have already made it to the remote and ignore these changes. Only when all changes are pushed up does the local server update the field as well.

If you want to know more details about which changes are pushed and which are ignored, run the push command with the verbose (-v) option. This option will also give you details about the resources consumed for the push:

Change 1840 was already present in the target repository.

Change 1841 imported as change 101354.

Resource usage: qry/zip/db/arch/fin/tot=4ms/0ms/7ms+1.1K/13ms+22B/8ms/34ms

 

What happens to the change numbers?

The output for the change 1841 was interesting: the change was imported as change 101354. What is going on here?

The point about DVCS is that you are normally not connected to the remote server, sitting in a plane or on a train instead. So local change numbers and change numbers on the remote server will diverge (remember that in Perforce Helix change numbers are strictly monotonic increasing).

When I push a change from my local server to the remote server, the change will usually be renumbered. This is similar to Mercurial (which locally uses strictly monotonic increasing changes as well) but quite different from Git, which insists on SHA hash numbers as a globally unique identifier of a change. This does not make sense for Perforce Helix – since I can push part of a change or individual files, the SHA has no meaning.

Nonetheless, there is a way to link a remote change to a local change, by setting a local configurable: submit.identity.

This configurable can be set to one of three different strategies:

 

UUID

A unique identifier created as a random number (it is a type 4 UUID). The UUID is created using a small, fast pseudo-random number generator (PRNG), which guarantees uniqueness to a high degree but is not necessarily sufficient for cryptography.

Checksum

This is an MD5 checksum hashed over the details of the change, including depot path, timestamp, content and revision of each file in the change, so two users submitting identical files with identical content will still get different checksums.

Serverid

This is a concatenation of your local server.id (that is, typically your local workspace name) and your local change number.

The change identity is stored in your local change in the field Identity, which will be pushed to the remote server.

You can use the command “p4 describe –I <identity>” to find a change with a given identity.

Which strategy you use is up to you. Why don’t you let us know which option is most useful to you?

 

What happens to integrations?

Just like in the ‘p4 clone’ case, integrations within the view of a remote spec are pushed. To preserve integrations between streams, make sure to map source and target stream in your remote spec.

If one-half of the integration is not mapped, the integration will be downgraded to add/edit/delete in the remote server upon push.

Most importantly, renames within a stream are preserved when pushed.

 

How about attributes?

Since 2015.2 attributes set against a file revision on your local server will be pushed to the remote server, but only if that server is at least at 2015.2 as well (otherwise attributes will be silently ignored).

 

And fixes?

Yes, you can push fixes (links between jobs and changes) as well, but you need to copy the jobspec and the individual jobs across to the local server before you can do so. This is an exercise best left to the advanced user (and maybe subject to another post if you so wish).

 

What about labels?

Err. No.

Keep in mind that Perforce Helix is different to your usual DVCS setup. The central usually has many projects in different paths to your local server, so translation of static labels is difficult. Automatic labels can be easily recreated on the remote server, though.

 

Triggers and push

There are triggers on the remote site that can fire when changes are pushed in, and these triggers will have to be used instead of the usual submit triggers to enforce policy. I’ll cover the details in a separate post.

 

Conclusion

A simple push is very easy, but the devil is, as usual, in the details. I hope this post gives you an idea of the preparation stages and the result of a push.

As usual, tweet feedback to @p4sven, please.

 

Happy hacking!