All Tools
F
LLMFreeOpen Source
FLASHMLA
Fast Multi-head Latent Attention kernels
MIT
ABOUT
Multi-head Latent Attention is cheaper in KV cache than standard MHA, but only if the kernel is fused. FlashMLA implements that decode path on CUDA so DeepSeek-style models can serve tokens without a slow PyTorch attention fallback.
INTEGRATION GUIDE
1. Speed up MLA decoding in a DeepSeek or MLA-based serving stack
2. Benchmark fused MLA kernels against PyTorch attention on Hopper GPUs
3. Integrate FlashMLA into a custom inference engine
4. Build CUDA wheels for MLA when packaging a local DeepSeek runtime
TAGS
cudaattentionmladeepseekinferencegpukernels