All Tools
L
DataFreemiumOpen Source
LAKEFS
Git-like version control for data lakes
Apache-2.0
ABOUT
Data engineering and ML teams working with data lakes face a fundamental problem — they have no version control for their raw and transformed data. When a data pipeline breaks or produces bad output, rolling back is expensive (reprocessing terabytes) or impossible (overwritten objects). LakeFS solves this by adding Git-like branching and atomic commits to object storage, so teams can create data branches for experiments, run pipelines in isolation, validate results, and roll back damaged datasets with a single merge operation — all without copying underlying data.
INSTALL
curl -fsSL https://github.com/treeverse/lakeFS/releases/latest/download/lakefsctl-linux-amd64.tar.gz | tar xzINTEGRATION GUIDE
1. Create isolated data branches for ML experimentation without duplicating large datasets
2. Roll back accidental data corruption or bad pipeline output to a known-good snapshot in seconds
3. Run CI/CD validation on data changes before merging into production data branches
4. Enable data scientists to version, diff, and collaborate on feature engineering and transformation
5. Maintain reproducible audit trails for datasets used in regulated AI and analytics workflows
TAGS
data-versioningdata-lakegitobject-storages3data-engineeringdevops