Fine-tuningFreeOpen Source

ALIGNMENT HANDBOOK

Production-ready recipes for LLM alignment and post-training

Apache-2.0

ABOUT

Aligning language models with human preferences through RLHF, DPO, and related techniques is notoriously difficult — requiring complex reward modeling, policy optimization, careful hyperparameter tuning, and orchestration across multiple training stages. Most teams lack the infrastructure and expertise to reproduce results from papers like InstructGPT or Llama 2. The Alignment Handbook provides complete, battle-tested training recipes that span the entire post-training pipeline: supervised fine-tuning (SFT), reward modeling, and reinforcement learning from human or AI feedback (RLHF/DPO). It ships with configs, datasets, and evaluation harnesses that work out of the box with popular open-source model families.

INSTALL

pip install alignment-handbook

INTEGRATION GUIDE

1. Fine-tune a Llama 3 model with DPO to reduce refusals and improve helpfulness 2. Reproduce the InstructGPT alignment pipeline on an open-source 7B model 3. Train a reward model on preference data for use in RLHF-based alignment 4. Run the complete SmolLM3 post-training recipe including supervised and preference stages 5. Benchmark alignment techniques (DPO vs PPO vs KTO) on the same base model architecture

ALIGNMENT HANDBOOK

ABOUT

INTEGRATION GUIDE

TAGS