NativePort
Sign in Get a key
← How-to

Can You Run Kimi K3 Locally? Hardware Requirements and Hosted API Alternatives

What Moonshot AI actually publishes about Kimi K3's size and license, why full-precision local serving is out of reach for normal hardware, the current state of community GGUF quantizations, and how to reach the model through a hosted API instead.

Short answer: not on a workstation, and not on a single high-end GPU. Kimi K3 is a 2.8-trillion-parameter model, and Moonshot AI’s own deployment guidance is a multi-accelerator server, not a desktop. This article separates what Moonshot has actually published from what the community has estimated, and covers the hosted-API path most people end up using instead.

What Moonshot publishes about the model

From the official moonshotai/Kimi-K3 model card and repository:

  • 2.8T total parameters, 104B active per token: a mixture-of-experts model routing each token to 16 of 896 experts, plus 2 shared experts that are always active.
  • 93 layers: 69 using Kimi Delta Attention (KDA), Moonshot’s linear-attention mechanism, alongside 24 gated latent-attention layers.
  • Native vision through a MoonViT-V2 encoder (401M parameters). The model takes text and images natively, not through a bolted-on adapter.
  • 1,048,576-token context window.
  • Weights shipped as MXFP4 (activations MXFP8), produced via quantization-aware training rather than post-hoc quantization of a higher-precision checkpoint. This is the native release format, not a community compression.
  • Repository size: 1.56 TB, across 96 safetensors shards.
  • License: the “Kimi K3 License,” MIT-like in structure, with a Model-as-a-Service revenue gate. If you (with affiliates) run a MaaS business built on it exceeding $20M in aggregate revenue, the license requires negotiating separate terms with Moonshot before continuing. Separately, commercial products/services above 100M monthly active users or $20M/month in revenue must display “Kimi K3” prominently in their UI. Read the license text yourself before any commercial self-hosting; this article isn’t legal advice.

Why “run it locally” doesn’t mean what it does for smaller models

1.56 TB of native-precision weights already rules out any single consumer GPU or Mac by a wide margin. That figure is the repository’s actual size, not an estimate. Moonshot’s own recommended hardware for serving Kimi K3 is a “supernode configuration with 64 or more accelerators.” The company hasn’t published a minimum GPU count, specific GPU model, or aggregate-memory figure below that. No single current-generation datacenter GPU holds the full model; serving it at native precision requires a distributed multi-node cluster, which is a different category of problem than “which GPU do I buy.”

Community quantizations exist, and are early

At least one community GGUF conversion is public: GrEarl/Kimi-K3-GGUF, using a hybrid scheme (Q2_K for the routed-expert tensors, Q4_K for dense layers, with F32/Q8_0/F16 for norms, output layer and attention biases respectively). That brings the total download to roughly 864.81 GiB across 94 parts, an effective 2.673 bits per weight. Two things are worth knowing before treating this as a turnkey path:

  • llama.cpp support exists only on an unmerged pull request (PR #26185 at the time of writing), not in a released version. You’d be building from that branch, not brew install llama.cpp.
  • The converter’s own listing describes the conversion as unverified: they state they don’t have the hardware to fully run and validate it themselves. One user is reported to have loaded it with hybrid GPU+CPU offload (-ngl 99 plus -ot layer-placement flags), but no throughput, latency, or minimum-RAM numbers have been independently confirmed, so none are repeated here as if they were.

If you want to experiment, treat this as bleeding-edge and expect to debug the build yourself; it is not the same maturity level as, say, a Llama or Qwen GGUF release.

The practical path: a hosted API

For nearly everyone, the useful answer to “how do I use Kimi K3” isn’t local inference at all. NativePort has Kimi K3 cataloged on its Hugging Face route as huggingface/moonshotai/Kimi-K3, pinned to the fireworks-ai serving backend, reachable through the same OpenAI-compatible endpoint as any other model on the gateway:

export NATIVEPORT_API_KEY="np_..."
curl -X POST "https://api.nativeport.ai/inference/v1/chat/completions" \
  -H "Authorization: Bearer $NATIVEPORT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "huggingface/moonshotai/Kimi-K3",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Usage is metered at $3.00 per 1M input tokens and $15.00 per 1M output tokens: the real, pass-through rate, no markup. Sign-up seeds $5 of credit automatically; adding more is the only place a fee applies (5.5%, on top-ups of $10 to $5,000). For the full request/response shape, vision, tool calls, and error handling, see Use Kimi K3 through an OpenAI-compatible API in Python, or wire it into a coding agent with OpenCode or Pi.

Moonshot also serves Kimi K3 directly through its own hosted API (api.moonshot.ai, model id kimi-k3) if you’d rather hold a separate Moonshot account and key. The tradeoff against NativePort is the usual one: a dedicated Moonshot account and its own billing versus one key and one balance shared across every model and provider already on the gateway.

Where to go next