Git LFS: The Pocketbook Explanation
Claudia Fernandes
·
Last updated on July 8, 2024
Opinions on Git LFS (Large File Storage) range from “it’s the ultimate solution” to “it’s bolted on and problematic”. The perspective often depends on the user’s relationship with large binary files: the size, number and regularity of use. Git LFS fans are more likely to deal with smaller and fewer binary files, while Git LFS detractors are often artists and devs who regularly create or handle large and numerous non-text files.
We’ll dive into these perspectives below, but first let’s take a moment to revisit what Git LFS actually is and how it functions.
Git LFS (Large File Storage) is an open source Git extension designed to improve how Git handles large files. While Git is great for tracking changes in text files, it struggles to handle and version large binary files like videos, graphics, or big datasets. Git LFS solves this by storing large files separately from the main repository and replacing them with small pointers in text format, keeping the central repository lightweight and fast.
While the Large File System solves the primary issue of uploading large files to a remote repository, it has its limitations.
By now it’s clear that teams relying on Git LFS to manage many large binary assets—like game studios or virtual production companies—can very easily run into performance, workflow and storage issues. So, what are the Git LFS alternatives? Within the Git environment, the primary options are Git-Annex and Helix4Git. Outside of Git, there are centralized version control systems that handle large binary files natively and effortlessly, such as Helix Core, Unity Version Control (PlasticSCM) and SVN.
Below is an overview of these alternatives to help you determine what could be best suited for your workflow.
Compared to Git LFS, Git-Annex is more complex to set up and manage and has a steeper learning curve, so it’s no wonder Git LFS has been more widely adopted even though it was released several years later. Even so, it’s worth discussing its merits.
Git-annex is a software tool for managing large files in Git repositories without storing the file contents directly in Git. Instead, it uses symlinks, which are references or pointers to the actual file locations. By only storing symlinks and keeping the actual large files stored elsewhere, the repository remains small and performant.
But where Git-annex stands out is that it supports a diverse range of storage backends, including local disks, networked filesystems, or cloud services. You can also specify storage locations for each file and automate file distribution. Additionally, Git-annex offers offline capabilities. By not relying on a central remote server for file retrieval, users can continue their work with locally available files, making it a good choice for distributed teams or teams working with very large files.
Helix4Git, provided by Perforce, is a version control solution that integrates with Git to manage large-scale projects with many repositories and large files. Helix4Git is designed to handle large repositories and numerous files efficiently, reducing the load on Git servers by offloading large file storage and retrieval to Perforce’s Helix Core server. This integration enhances performance, making operations like cloning and fetching faster and more reliable, particularly for large projects.
In terms of scalability, Helix4Git excels where Git LFS often struggles. Git LFS can experience slow performance during cloning and pulling operations in large repositories, and its storage needs can escalate quickly due to all revisions of the same asset counting towards the remote storage limit. Helix4Git, on the other hand, can handle larger projects and a higher number of users without significant performance degradation, thanks to its integration with Helix Core. This ensures a single source of truth for all assets, simplifying the management of large files and reducing the need for additional storage solutions like Nexus or Artifactory.
Additionally, Helix4Git leverages Helix Core’s advanced security and access control features, providing more granular control over file access and modifications compared to Git LFS. While Git LFS relies on Git’s security model, which may not meet all enterprise needs, Helix4Git offers enhanced security for larger, more complex development environments. This makes Helix4Git a robust alternative to Git LFS, particularly for large teams and projects that demand high performance, scalability, and integration with existing Perforce infrastructure.
Perforce Helix Core is a centralized version control system designed to manage large-scale projects and substantial data efficiently. It offers strong performance, scalability, and features like detailed access controls and robust security. Helix Core is designed for enterprises and large development teams, providing reliable handling of large files and repositories, ensuring all digital assets are managed in one place.
While Git LFS helps manage large binary files by storing them outside the main Git repository, it often experiences slow cloning and pulling operations in large repositories. Helix Core is optimized for handling large files and numerous assets without these performance issues. It includes features such as file locking and real-time collaboration, making it an efficient solution for teams managing extensive codebases and digital assets.
The Helix Plugin for Graphical Tools (P4GT) further enhances Perforce Helix Core. This plugin integrates version control directly into graphical tools like Photoshop and Maya, allowing users to manage versions and assets within their familiar environment. In contrast, Git LFS often requires users to switch between their development and design tools and the command line, which can disrupt workflows and isn’t user-friendly for non-programmers.
Unity Version Control (formerly PlasticSCM) is a version control system designed specifically for managing large projects and complex development workflows. It excels in handling large binary files and extensive codebases, offering advanced branching and merging capabilities, high performance, and a user-friendly interface. Not surprisingly, it is particularly popular in industries like game development, virtual reality, and other fields that rely heavily on large binary assets.
Compared to Git LFS, Unity Version Control is optimized to handle large files and complex branching structures without performance degradation. Additionally, Unity Version Control integrates seamlessly with various development environments and tools, providing a smoother workflow for developers and designers and reducing the need to switch between tools and the command line, which can be a common challenge with Git LFS.
Subversion (SVN) is a centralized version control system that has been widely used for many years to manage code and digital assets. It was designed to handle large files and extensive codebases efficiently, with a central server storing all the versioned files. SVN provides strong performance for both small and large projects, with reliable tools for branching, merging, and managing access controls. It is particularly effective for teams that prefer a centralized approach to version control and need robust support for binary files and large repositories.
Compared to Git LFS, SVN offers a more straightforward and reliable solution for handling large files and extensive repositories. Users often encounter slow cloning and pulling operations with Git LFS, particularly in large repositories. SVN avoids these performance bottlenecks by maintaining all versioned files on a central server, ensuring faster and more consistent access to large files.
Additionally, SVN’s centralized approach simplifies configuration and management, making it easier to enforce access controls and maintain a single source of truth. While Git LFS requires individual setup and configuration for each repository and user, SVN’s central server setup provides a unified environment that is easier to manage, especially for large teams and projects. This makes SVN a compelling alternative for organizations that need efficient, scalable, and reliable version control for their substantial codebases and digital assets.
