image-blog-vcs-git-lfs
August 27, 2019

How Git LFS Works: Overview of Git Large File Storage

Git at Scale
Version Control

Development teams around the world use Git to manage source code. And many use Git LFS — Git Large File Storage — to manage and store large files. Here, we break down what Git LFS does and when to use it.

Can Git Handle Large Files?

Git cannot handle large files on its own. That's why many Git teams add Git LFS to deal with large files in Git. 

What Is Git LFS (Git Large File Storage)?

Git LFS is a Git extension used to manage large files and binary files in a separate Git repository.

 

Most projects today have both code and binary assets. And storing large binary files in Git repositories can be a bottleneck for Git users.

That’s why some Git users add Git Large File Storage (LFS).

How Does Git LFS Work?

Git LFS uses pointers instead of the actual files or binary large objects (blobs).

So, instead of writing large files/blobs to a Git repository, you write a pointer file. And the files/blobs themselves are written to a separate server. You can even use multiple servers in Git LFS.

Getting started is fairly straightforward. You download the extension and configure your file types.

Using Git LFS makes it possible to version large files (and manage blobs) while freeing up space in Git repositories. And Git LFS is often a fix for pushing large files to GitHub.

Git Problems Beyond LFS 

Git has problems beyond Git LFS. And security is a big one. Native Git lacks security, and the add-on options to secure Git aren't great. Find out what you should do about Git security in our white paper — How to Lock Down Git.

Lock Down Git

 

Should I Use Git LFS? 

You should use Git LFS if you have large files or binary files to store in Git repositories.

That’s because Git is decentralized. So, every developer has the full change history on their computer. And changes in large binary files cause Git repositories to grow by the size of that file every time the file is changed (and that change is committed). That means it will take ages to get the files. And if you do, it will be difficult to version and merge the binaries.

So, every time the files grow, the Git repository grows. And when Git users need to retrieve and clone a repository, this creates problems.

Git LFS was created to solve these problems. But it has problems of its own…

When Git Large File Storage Doesn't Work

Git LFS works. But teams constantly tell us that it’s hard to manage. So, even though Git itself is free, it can rack up productivity costs.

Installing Git LFS on every server and workstation (and/or repo) takes time. It also burdens admins. Once it’s installed, there’s no visibility and little control over it. And if some developers don’t have the Git LFS extension? It breaks down.

It takes extra steps to maintain Git Large File Storage with build runners, such as Jenkins. That leads to extra time — and extra complexity.

All of this can lead to performance issues.

Alternatives to Git LFS

Git LFS isn’t the only way to manage large files in Git. There are alternatives.

This includes other open source or third party fixes, such as:

  • git-annex
  • git-bigfiles
  • git-fat
  • git-media
  • git-bigstore
  • git-sym

But, just like Git LFS, these Git large file storage options can create problems. There’s a better way to manage large files and binary files.

The Best Version Control for Large Binary Files: Helix Core

Today’s projects are bigger, with more files and mixed assets than ever before. Git and Git LFS alone can’t manage that. But Helix Core can.

Helix Core — version control software from Perforce — is the best option for managing large binary files. That’s because large file storage is a native capability, not an add-on. And it’s bulletproof.

In Helix Core, you can store binaries alongside your source code. In fact, all of your largest files — binary files, source code, art files, video files, images, libraries, and build artifacts — can live together in a single repository. Without slowing down large, distributed teams.

See for yourself why Helix Core is the tool of choice to manage large binary files.

Manage Large Files in Helix Core

By the Way, Perforce Has Git, Too

Still have teams who need to manage large files in Git? You can bring them into your build pipeline with Perforce Git tools — Helix4Git and Helix TeamHub.

Helix4Git is a Git server inside a Perforce server. Helix TeamHub is a Git code hosting solution. These tools allow Git teams to leverage the speed and performance of Helix Core — while working in Git.

So, you can use Helix4Git alongside Helix Core to bring your Git repository into your build pipeline. Helix4Git lets your developers still use their native Git tools. But they get their files a lot faster.

You get:

  • Faster CI/CD build performance.
  • Support for multi-repo projects, artifact repositories, and even a Docker container registry.
  • The ability to gradually add nodes.
  • Replication and guaranteed up-to-date content with no lag, instead of copying.

This unique combo provides development teams with the Git workflows they want. And you get faster performance and a single source of truth. Plus, it’s compatible with your existing tools.

contact us about GIT

 

Related Content: