Reclaim disk space by archiving files

Over time, Helix Server can accumulate many revisions of files. Some files might be from projects that are no longer active. Therefore, you might want to reclaim the disk space. One way is by transferring those versioned files into an archive depot A special depot into which versioned files (also known as archive files) can be transferred.. Archive depots free up disk space by moving files that are rarely accessed to bulk storage.

Compare the advantages of using an archive depot and p4 archive to the p4 delete and p4 obliterate commands:

p4 delete	p4 archive	p4 obliterate
no disk space reclaimed file history is preserved and "deleted" files can be restored merely marks files as deleted at the head revision	disk space reclaimed on the storage used for your local depot files moved to the archive storage file history is preserved and archived files can be restored	disk space reclaimed obliterated files cannot be restored permanently removes file and its history.

Transferring version files to an archive depot is not allowed if the files are in a remote depot or are already in an archive depot.

In a commit-edge environment, the archive depot is not replicated between the commit and edge servers.

Transfer files into an archive depot

1. Create a new depot by using the p4 depot command.

   p4 depot -t archive ArchiveDepot
   

   where ArchiveDepot is the name for your new depot of type archive.

2. Set the archive depot’s Map field to an absolute path where the archived files will be stored. The path can be locally mounted network storage, detachable storage, or a mounted block storage device for the Cloud. If you are using S3 storage, see S3 storage for archive depots.
3. To transfer all unbranched binary files to the archive depot from your existing depot:

   p4 archive -D ArchiveDepot //LocalDepot/...

   where LocalDepot is the name of your standard depot of type local.

   Binary files, compressed and uncompressed, are the default file types that p4 archive transfers. You might want to adjust the behavior of the command, so consider its Options, such as -t to make p4 archive transfer text files. To learn more, see File type modifiers in Helix Core Command-Line (P4) Reference.

4. (Optional) Unmount the volume to which the LocalDepot archive files were written, and store it in a secure location.

Restore files from an archive depot

1. Mount the volume on which the archive depot’s files are stored.
2. Make sure that the path matches the Map field of the archive depot.
3. Use the p4 verify -A command to verify files before you restore them.

   For example, p4 verify -A //ArchiveDepotName/... checks for corrupt or missing files.
   Files archived into this depot have their original path but with the depot name.
   For example, //ArchiveDepotName/LocalDepot/Mainline/... includes the LocalDepot from which the files were transferred to the archive depot.

4. Use p4 restore to transfer the files from the archive depot to the original local depot.

   p4 restore -D ArchiveDepotName //LocalDepot/...

5. (Optional) unmount the volume from which the archive files were restored.

Purge files from an archive depot

1. Mount the volume on which the archive depot’s files are stored.

2. If you want Helix Core Server to retain the metadata The data stored by Helix Core Server that describes the files in the depot, the current state of client workspaces, protections, users, labels, and branches. Metadata is stored in the server database and is separate from the archive files that users submit. of the files you are going to purge, see the -p option of the p4 archive command.

3. Use p4 archive -p to purge files.

4. (Optional) unmount the volume from which the archive files were purged.

S3 storage for archive depots

Your organization might find it economical to store archive depots by using:

• Amazon S3, the Simple Storage Service of Amazon Web Services

• An S3-compatible API, such as that of MinIO

• A cloud provider's storage offering, such as DigitalOcean's Space Object Storage, Tencent Cloud Object Storage, Google Cloud Storage, or S3Proxy for Microsoft Azure Blog Storage

You can create an S3 data source as a place to store archive files for a Helix Core Server depot. Create an archive depot specification that sets the Address field to map to the S3 datasource. The location of the S3 bucket and the credentials are stored in the depot spec on a per-depot basis using the Address field in the following format:

Address: s3,region:us-east-1,bucket:p4d-depot,accessKey:,secretKey:******

where:

• s3 must be at the start of the address

• The settings are comma-separated, with a colon to separate the key and value.

The S3 data source requires the following AWS Identity and Access Management (IAM) permissions of the Amazon S3 API Reference: s3:GetObject, s3:PutObject, and s3:DeleteObject.

Supported keys are:

Key	Note
url	https://bucketname.s3.amazonaws.com Some S3 implementations might not require this key.
region	Some S3 implementations might not require this key.
bucket	Required
accessKey	Required
secretKey	Required
token	Depending on bucket security, this key might be optional.

Create an S3-backed archive depot

1. Create a new depot by using the p4 depot command, and set the depot’s Type to archive.

   p4 depot -t archive s3ArchiveDepotName

2. In the depot specification,
   

   • Do not change the value of the Map field because it will not be used.

   • Create a new field for the address, and enter the s3 information and credentials as a comma-separated list of key:value pairs according to the requirements of your S3 storage provider.

Examples

These examples illustrate that different storage providers might have different requirements. For the most complete, correct, and current requirements, check with your storage provider.

AWS:

Address: s3,region:us-west-2,bucket:my-archivebucket,accessKey:******,secretKey:******

Digital Ocean:

Address: s3,url:https://my-archivebucket.sfo3.digitaloceanspaces.com,bucket:my-archivebucket,accessKey:******,secretKey:******

MinIO:

Address: s3,url:https://server:port/my-archivebucket,bucket:my-archivebucket,accessKey:******,secretKey:******