All Tools
M
Fine-tuningFreeOpen Source
MERGEKIT
Toolkit for merging pre-trained language models
LGPL-3.0
ABOUT
Combining the strengths of multiple pre-trained LLMs — one strong at code, another at reasoning, a third at instruction following — traditionally requires expensive retraining or large GPU clusters. MergeKit solves this with an out-of-core merging approach that lets practitioners compose models on CPU or modest GPUs, democratizing access to advanced model composition techniques like TIES, DARE, and evolutionary optimization.
INSTALL
pip install mergekitINTEGRATION GUIDE
1. Combine multiple domain-specialized fine-tunes (code, math, instruction) into a single model without retraining
2. Merge top-performing open-source checkpoints from leaderboards to create state-of-the-art models
3. Replace an ensemble of models with a single merged model at equivalent quality but lower inference cost
4. Automatically discover optimal merge recipes via evolutionary optimization (mergekit-evolve)
TAGS
llmmodel-mergingfine-tuningpytorchtransformersslerptiesdare