December 18, 2014

Git Beyond the Basics: Using Shallow Clones

Git at Scale

This is part 6 of a 6-part series on Git commands.

git beyond the basics using shallow clones

One problem commonly encountered when using Git in the context of Continuous Integration (CI) and Continuous Delivery (CD) is server load. Because Git’s design includes everything in each copy of a repository, every clone gets not only the files but every revision of every file ever committed. It isn’t hard to imagine how that can quickly become a bottleneck with expanding numbers of build, test and deployment pipelines.

Fortunately, Git supports the notion of a “shallow clone”, which is a more succinctly meaningful way of describing a local repository with history truncated to a particular depth during the clone operation. By providing an argument of --depth 1 to the clone command, the process will copy only the latest revision of everything in the repository. This can be a lifesaver for Git servers that might otherwise be overwhelmed by CI/CD automation. For example:

git clone --depth 1 jquery

The above command clones only the current HEAD of the jQuery repository on GitHub. An unscientific comparison shows that it completes in a couple of seconds compared to the roughly thirty-plus seconds it takes to clone the entire repository without the depth switch, a savings of more than 90%. That one trick can relieve a great deal of load from servers that are pulled frequently via automation.

Users should also be aware that shallow clones were quite limited in their functionality prior to Git v1.9. With v1.9 and later, however, Git has been improved to support pull and push operations even with shallow clones. This allows changes to be made and committed/pushed back to the source repository if needed as part of automated build operations.

Generally speaking, it’s probably best to avoid such a workflow, but at least it is now possible. Those seeking more details should check out the Git Book documentation.

Keep reading

Looking for more information on Git commands? Here are other Git Beyond the Basics posts that you might find helpful: