All Tools
F
LLMFreeOpen Source
FASTDEPLOY
High-performance LLM and VLM serving on PaddlePaddle
Apache-2.0
ABOUT
Serving PaddlePaddle LLMs and VLMs needs a stack that is not vLLM-only. FastDeploy adds online serving, speculative decoding, prefix cache, and disaggregated prefill so ERNIE-class models run on NVIDIA and domestic accelerators without a custom server.
INTEGRATION GUIDE
1. Serve ERNIE-4.5 or ERNIE-VL with an OpenAI-compatible API
2. Run speculative decoding and prefix caching for higher throughput
3. Deploy disaggregated prefill and decode workers at scale
4. Benchmark LLM serving on NVIDIA, XPU, MLU, or Gaudi hardware
TAGS
pythonllminferenceservingpaddlepaddlevlmernie