MLC LLM
Apache-2.0 compiler and deployment engine for running LLMs natively on NVIDIA, AMD, Apple, Intel, web, iOS and Android.
Tags
Description
Established open project (repo mlc-ai/mlc-llm; not archived; created 2023-04-29; last pushed 2026-11-13; not presented as a 2026 launch). Root LICENSE is Apache-2.0. Site https://llm.mlc.ai/ (HTTP 200). Docs https://llm.mlc.ai/docs/ were HTTP 200; https://llm.mlc.ai/docs 301s there and is not used. The install page title still reads “mlc-llm 0.1.0 documentation”. The GitHub Releases API had no “latest” release (404) on 2026-11-14. There is no PyPI project named mlc-llm (404). PyPI mlc-ai 0.1 (“Machine learning for systems”, homepage github.com/mlc-ai) is a different package and is not the install path for this project. Install docs (https://llm.mlc.ai/docs/install/mlc_llm, HTTP 200) describe nightly prebuilt pip wheels from https://mlc.ai/wheels/ (200; the URL without a trailing slash 301s and is not used), for example: python -m pip install --pre -U -f https://mlc.ai/wheels mlc-llm-nightly-cpu mlc-ai-nightly-cpu, plus platform-specific wheels (cu128, cu130, rocm61, rocm62 and others on that page). A conda environment is recommended. Build-from-source instructions are also documented. None of those commands were executed. Quick start shows MLCEngine and mlc_llm serve with Hugging Face MLC model IDs. Model weights and any GPU or host resources are licensed and billed separately.
Agent access
Related
- MLX LM: MIT Python package for text generation and fine-tuning of large language models on Apple silicon with MLX.
- vLLM: Apache-2.0 high-throughput LLM inference and serving engine with an OpenAI-compatible API, installable via uv or pip.
- SGLang: Apache-2.0 inference framework for LLM, vision-language and diffusion models, optimized for agentic workloads and large-scale serving.
- llama.cpp: MIT C/C++ library and server for local LLM inference, widely used as a local OpenAI-compatible backend for coding agents.
- LocalAI: MIT self-hosted OpenAI-compatible engine that runs LLM, vision, voice and image backends locally, with Docker images as the documented install.
Sources
- https://llm.mlc.ai/
- https://llm.mlc.ai/docs/
- https://llm.mlc.ai/docs/install/mlc_llm
- https://mlc.ai/wheels/
- https://github.com/mlc-ai/mlc-llm
- https://github.com/mlc-ai/mlc-llm/blob/main/LICENSE
- https://github.com/mlc-ai/mlc-llm/blob/main/README.md
Machine-readable
- JSON
- Markdown
- Source file on GitHub (edit via pull request)