All Tools
L
DataFreeOpen Source
LMDB
Embedded memory-mapped key-value store for ML systems
OLDAP-2.8
ABOUT
ML pipelines need a fast, crash-safe embedded store for samples, caches, and indexes without running a separate database server. LMDB memory-maps a B+tree on disk so readers see zero-copy access, writers stay ACID-compliant, and tools such as Caffe, PyTorch data loaders, and feature caches can persist large binary datasets with minimal overhead.
INSTALL
pip install lmdbINTEGRATION GUIDE
1. Store image, text, or feature samples for high-throughput ML data loaders
2. Persist embedding caches and lookup tables inside training or inference processes
3. Use as an embedded storage engine for tools that need concurrent readers and ACID writes
4. Replace heavier local databases when an ML job only needs a fast key-value map
TAGS
key-value-storeembedded-databasememory-mappedstorage-engineml-datasets