All Tools
R
LLMFreeOpen Source
ROUTELLM
Route prompts to the right LLM to cut cost
Apache-2.0
ABOUT
Sending every prompt to the strongest model wastes money on easy tasks. RouteLLM serves and evaluates routers that send simple queries to cheaper models and hard ones to frontier models, so teams cut LLM spend without a large quality drop.
INSTALL
pip install routellmINTEGRATION GUIDE
1. Route chat traffic between a cheap model and a frontier model
2. Evaluate LLM routers on quality versus cost before production
3. Serve a router in front of multiple OpenAI-compatible endpoints
TAGS
pythonllmroutingcostinferenceserving