All Tools
M
Dev ToolsFreeOpen Source
MNN
Alibaba's lightweight inference engine for on-device LLMs and edge AI
Apache-2.0
ABOUT
Running large models on edge devices — smartphones, IoT, embedded systems — requires an inference engine that balances speed against the strict memory and power budgets of mobile hardware. MNN is battle-tested across hundreds of millions of Alibaba devices, supporting on-device LLM inference, stable diffusion, and vision models. It converts models from TensorFlow, PyTorch, and ONNX, and dispatches to the optimal backend (ARM NEON, x86 AVX, Vulkan, CUDA, Metal, OpenCL) without manual configuration.
INSTALL
pip install MNNINTEGRATION GUIDE
1. Run a 7B LLM on-device using MNN's transformer runtime for private AI inference
2. Deploy stable diffusion image generation on a mobile phone with Metal GPU acceleration
3. Convert a TensorFlow detection model and run it at 60 FPS on an edge device
4. Power a cross-platform mobile app with real-time ML inference on Android and iOS
5. Benchmark and optimize an ONNX model across ARM, x86, and GPU backends in one pipeline
TAGS
inferencemobileon-devicedeep-learningcross-platformarmllmoptimization