Full Video Transcript
Hi, my name is Ryan Maffesoli, and I'm with Perforce Software. In today's Helix Core admin video, we're going to talk about file types and the P4 typemap to control how your server stores files and make sure that they've got the attributes set necessary to interact within your specific pipeline as expected.
So to talk about our typemap and how our server is going to kind of handle files as they enter into our server initially, we need to talk about what those types are and the different attributes we can assign first. For that, I'm actually going to go just into P4V, because we can edit these file types after the fact through the interface.
If I were to select a single file let's go with this FBX, I can go ahead and change file type. Now, at the top here, we've got our base file type, and the main two you're going to encounter most often are going to be text and binary. What this does is it changes how the file is stored on the server, as well as how conflicts are handled with the P4Diff tool.
So if anything is set to text what it will do is it will be storing the deltas between your changes. So if I upload an initial version, and then I go and I add a few more lines of code, it's going to be storing only those few lines of code and not the entire document to try and save space.
Also the text format, when comparing using the P4Diff tool can give you more options about how you're going to actually merge your code together and allow you to merge your files. That's because those lines are going to be individually discernible by a human and you can go line by line to see where things have changed, and when it comes down to any conflicts, you will have the option to either pick your server revision, your local revision, or to open up P4 merge and compare line by line to see how things will go back together and solve that merge conflict in a way that suits you best.
Now, if something is saved in a binary format, you can't really go through and edit things line by line like that. So things are stored compressed on the server. And once you do pull those files back, they will be then uncompressed when they land on your local machine. One of the note there is that because you are not able to open up P4 merge and kind of merge those files line by line when you get into a conflict state, you will have two choices of would you like to accept the server version, or would you like to replace the server version with your workspace version.
Underneath that, we do have the option to identify files as symlinks, so that way it can handle things a little different and know that it's pulling from somewhere else. And as well as if you're using a Unicode server and need to define specific files as Unicode you can do so with either the Unicode file type, or the UTF 8 or UTF 16 file types here as well. This Apple and Resource, those are actually more legacy, you shouldn't really be using those anymore. you should pretty much just stick to your text or binary as it will handle the majority of your work as you're coming along.
Beyond our basic file types, there are also attributes you can set on your files to change how they operate within your workspace and on the server as stored. So any of the attributes that are set with a lowercase letter relate to how things are operated on in your workspace, and anything with a capital letter is relating to how things are stored on the server and when things are submitted back.
So, taking a look at our workspace related attributes here, we've got the lowercase m, which will preserve our local modification times. So that way, when you push them up to the server, that will be the time that's recorded on the file. And also, when you sync back down, that time will then be restored back to that file.
Next, we have our lowercase w. What this does is that... Typically in a Helix Core setup, and when you are pulling things down into your workspace, everything is initially put into a read only mode. This is done because that file checkout is what should actually be flipping that file from a read only format to a read write, typically. And it's a good way to make sure that people are checking out their files correctly, and communicating back and forth with what they're working on. However, some processes... may be writing to specific files that are more cache related or you're expecting these to be updated very quickly. In that case, it can be useful to have certain files that are always in a writable state and never kind of go to that read only format. And that's what the lowercase w file attribute can do is when things get copied back down to your workspace, they'll always be in a writable state.
The lowercase x will set that executable bit when files are pulled down. So this is something that you may want to use on shell scripts or executable files. By putting that in place if you're on a Linux system or OSX you'll know that your file will be executable directly and have that bit flipped.
The next two are related to keyword expansion for your revision control system. And if you mark that, it will make sure to expand out all possible keywords that are marked with the dollar signs on each side . Then the KO will kind of limit that scope and only expand out a few keywords here, particularly the IDs and the headers.
Now we've gotten to the one that's typically most used, and that is the exclusive checkout flag, which is a lowercase l. And what this lowercase l will do is that essentially when a file is checked out, it will automatically put a lock on it to make sure that no one else can check that file out at the same time. This checkout will also prevent anyone else from uploading or submitting a change while that file is checked out. So a lot of times, particularly when it comes to binary files. You will typically see that binary+l mode put onto a lot of files to just make sure that we're avoiding the situations where we could cause file conflicts or lose any work.
Now that we talked about all of our workspace related attributes, we can take some time and look at our server side attributes as well. These are really going to relate to how things are stored on the server, and we can even limit how many revisions of a particular file stay on the server.
So, going from the top here, we've got our capital C, which will say that this file, when put to the server, should always be stored in a compressed format. By default, all binary files will be stored with this compressed style. If we go with our capital D, what this is going to do is store our revisions as deltas in RCS format. So by default, this is how text files will operate, and they will always kind of store with that revision control delta syntax.
And if you'd prefer not to do either, if you do not want to compress and you do not want to store that delta format, that capital F will just say, please just store the full file as it stands on the server.
And our final option here actually works in conjunction with the other ones, and can be on at the same time, but that is our capital S. And what this will let us do is limit the amount of revisions we're storing of a certain file on the server.
So, by default, any one file that you upload into HelixCore will keep all revisions that you upload. However, if a file is particularly large, or if it's something like a derived build that's executable, or just generally derived data, perhaps you only want to keep the latest version for quick access, and there's not really a value to storing every revision that's ever existed. In that scenario, the capital S alone will store a single revision, which will always be your head revision. If you'd like to store at least a few options, perhaps you do want to keep some history, but you just don't want to fill your server with perhaps a very large file, you can always add a integer afterwards, which will define the number of revisions that will be saved for this particular file.
So now that we've gone through all of our file types and file attributes, we can step over to our typemap and start to edit values there. So the typemap itself is going to be the default state of how HelixCore is going to handle files that match various paths and patterns. So to open up our typemap and edit it, we can jump over to our terminal that has our P4 CLI installed and I can type p4 typemap. This will then open up in our, associated text editor. And so here I just have it opening up a notepad and you can see we've got just our initial set of comments as well as our actual typemap listed below. So the way that this works is you first define your type. Or how things are going to be set for that type, followed by a pattern to match and here if we follow the syntax that is just standard to helix core, the first 2 slashes mean our root of our depot and we're followed by our 3 dots here, which are in helix core syntax, a recursive wild card, followed by our file extension. So if we read this, this is saying that anywhere on the server at any depth, if you find a file that has the extension ASP, it will be text type and no additional modifiers here. So if we go a little further down into my existing typemap, you'll see that here we have a few files, log files, ini files, pdm files that are actually set to be text files as well as having that +w so that those are always going to be writable whenever they're copied down into a workspace they will not be set to read only.
A little bit lower here. We have some binary files set up where again, we're defining our initial base type of binary saying plus capital F or store that full file uncompressed on the server. And then a lowercase l to say that this is an exclusive checkout file. And anytime someone checks out a dot zip file, make sure it's locked so no one else can edit it at the same time. So, as mentioned, that lock, that +l is likely going to be the one you use most often to just enforce that exclusive checkout style.
Also, at any point, you can always come in here and edit this typemap to change how files are going to be received on your server.
One important note with your typemap is that you can always come in here and edit and change these values as you need to. So if you need to add additional lines, you can always add those at the bottom. If you need to add different demarcations or file attributes, you can make those changes or edits and save them and they will affect all files that are coming into your server from that point on.
Um, the typemap is a server wide configurable, but if I needed to do something specific for a certain project. I could always come in here and let's just go ahead and copy this last line. And let's say we needed to store all versions of this build data for you assets on a specific project and a specific depot.
So we could always come in here and define deeper paths to match to, and that will then set for that type within a specific depot. And again, we still have our recursive wildcard here, but as we have our specific depot listed prior, this full pattern has to match for it to evaluate and assign that file type to anything that falls underneath it
One note is that if you come in here and edit your file types, that will not change how the file types are defined for files that are on your server already for that, you're going to want to use the P4 retype command.
So once we're done editing our typemap we can just go ahead and save this temp file and close. And you'll notice that here, since I haven't changed my typemap. It will just tell me that TypeMap has not changed. If I had made edits, those would then tell me that my TypeMap has been saved and that it is now available for use on our server.
And with that, we now know enough about file types and file attributes in HelixCore to set up our typemap to be ready to handle any data that our users throw at it in an expected manner that we prefer.
So I hope this helps and I'll see you next time.