Short answer: not at full precision on consumer hardware, and only barely (with real tradeoffs) at reduced precision. GLM-5.2 is a 744-billion-parameter model, and Z.ai’s own repository doesn’t publish a minimum GPU count or node configuration for serving it at native precision at all. This article separates what Z.ai has actually published from what the community has built around it, and covers the hosted-API path most people end up using instead.
What Z.ai publishes about the model
From the official zai-org/GLM-5.2 model card and repository, and Z.ai’s release announcement (June 17, 2026):
- 744B total parameters, 40B active per token: a mixture-of-experts model, described by Z.ai as its flagship for long-horizon tasks, with an architecture change called IndexShare that reuses the same sparse-attention indexer across every four layers to cut per-token compute at long context.
- 1,048,576-token context window, plus multiple thinking-effort levels to trade reasoning depth for latency. This is a control surface Z.ai exposes directly, separate from whatever any given API route in front of the model chooses to expose (see the caveat in the Python API tutorial).
- Text-generation model: the catalog entry NativePort serves it under reports no vision capability; treat any vision claim about this specific route with suspicion.
- Repository size: 1,506,693,036,946 bytes, roughly 1.51 TB at native BF16 precision, per the repository’s own file listing.
- License: MIT, with no regional-access or revenue-based gating clause published in the model card as of this writing. Read the license text yourself before any commercial self-hosting; this article isn’t legal advice.
- Official deployment frameworks, per the model card: SGLang v0.5.13.post1+, vLLM v0.23.0+, Transformers v0.5.12+, KTransformers v0.5.12+, and Unsloth v0.1.47-beta+. Z.ai documents that these frameworks can serve the model; it does not publish a recommended GPU count, node topology, or tensor-parallel size to go with them.
- Z.ai also publishes an official FP8 checkpoint,
zai-org/GLM-5.2-FP8, roughly half the storage and VRAM footprint of the BF16 release, still solidly datacenter-scale hardware, not a workstation path.
Why “run it locally” doesn’t mean what it does for smaller models
1.51 TB of native-precision weights already rules out any single consumer GPU or Mac by a wide margin. That figure is the repository’s actual size, not an estimate. Unlike some model releases, Z.ai’s own documentation doesn’t state a minimum accelerator count or cluster shape for full BF16 serving at all; it names the frameworks (SGLang, vLLM, Transformers, KTransformers) that can serve the model and leaves the hardware sizing to whoever runs it. No single current-generation datacenter GPU holds the full BF16 model in memory: serving it at native precision means a multi-GPU, likely multi-node, deployment, which is a different category of problem than “which GPU do I buy.” The official FP8 checkpoint roughly halves that footprint but is still built for the same class of hardware, not a desktop.
Community quantizations: real, and llama.cpp support is fresh
Unlike full-precision serving, running a reduced-precision GGUF of GLM-5.2 on your own hardware is a genuinely live option, published by Unsloth at unsloth/GLM-5.2-GGUF under the same MIT license as the base model. Unsloth’s own documented memory requirements per quantization level:
- 1-bit: 223 GB total memory
- 2-bit: 245 GB total memory
- 3-bit: 290 to 360 GB total memory
- 4-bit: 372 to 475 GB total memory
- 5-bit: 570 GB total memory
- 8-bit: 810 GB total memory
The specific 2-bit dynamic quant Unsloth ships (UD-IQ2_M) is a 239 GB download and needs at least 245 GB of memory to run. The file size and the memory floor to actually load it aren’t the same number, and Unsloth’s own table is explicit about the gap. None of this is small: even the 1-bit floor (223 GB) is well beyond a typical high-end workstation, and it’s a community compression of the official weights, not something Z.ai ships or supports.
llama.cpp support is newer than it looks. GLM-5.2 needs indexer-tensor handling that mainline llama.cpp didn’t originally have; PR #25407, “GLM 5.2 Indexer support,” merged into ggml-org/llama.cpp on 2026-07-24, four days before this article was written, fixed loading and generation. That’s a materially more mature state than an unmerged branch, but it’s still days old: use a build from on or after that merge, not whatever a package manager happens to have cached, and expect the open tracking issue to keep collecting edge cases for a while yet. Unsloth’s documented invocation lets llama-cli pull the quant straight from Hugging Face by repo and quant name, with no separate download step and no local path to get wrong:
./llama.cpp/llama-cli \
-hf unsloth/GLM-5.2-GGUF:UD-IQ2_M \
--temp 1.0 \
--top-p 0.95 \
--min-p 0.01
If you’ve already fetched the shards yourself (e.g. with hf download unsloth/GLM-5.2-GGUF --local-dir unsloth/GLM-5.2-GGUF --include "*UD-IQ2_M*"), point --model at the first shard in that local directory instead, and llama.cpp picks up the rest of the split automatically by filename:
./llama.cpp/llama-cli \
--model unsloth/GLM-5.2-GGUF/UD-IQ2_M/GLM-5.2-UD-IQ2_M-00001-of-00006.gguf \
--temp 1.0 \
--top-p 0.95 \
--min-p 0.01
That path only resolves if the hf download step above actually ran first with a matching --local-dir; --model takes a real file on disk, not a Hugging Face repo path. Only -hf can reach the repo directly.
Colibri: an experimental low-RAM path, not a substitute for GPU serving
Colibri is a separate, community, experimental project: a small C inference engine that keeps only GLM-5.2’s dense components (attention layers, shared experts, embeddings, roughly 17B parameters, quantized to int4) resident in RAM, and streams the model’s routed experts from NVMe disk on demand instead of holding them all in memory. Its headline claim is running the 744B model with around 25 GB of RAM resident. That number is real in the narrow sense that it describes what’s kept in memory, but it comes with tradeoffs a GPU-served or fully-RAM-resident setup doesn’t have:
- It needs roughly 370 GB of fast disk space for the streamed experts. The memory requirement moves to disk, it doesn’t disappear.
- Cold-token throughput is reported in the low tenths of a token per second, since an unseen expert can mean tens of gigabytes read from disk before that token completes; a warm LRU cache of frequently-used experts is what makes the model usable at all after the first few requests.
- It’s young, single-maintainer, and specific to this disk-streaming architecture, not a maturity level comparable to llama.cpp or vLLM.
Treat Colibri as a legitimate way to load GLM-5.2 on hardware that couldn’t otherwise hold it, not as an equivalent to full-quality GPU serving on throughput, latency, or maturity.
What about Ollama?
Ollama’s library does list glm-5.2, but as of this writing the only published tag is glm-5.2:cloud, which runs on Ollama’s own cloud infrastructure, not on your machine. There is no locally-downloadable weight tag (no q4, q8, or similar) published under that listing. If you’re searching for “how to run GLM-5.2 on Ollama” expecting a local ollama pull command, that command doesn’t currently exist for this model. ollama run glm-5.2:cloud reaches Ollama’s hosted service, which is a separate product and a separate account/billing relationship from both NativePort and Z.ai’s own API. For an actual on-disk GGUF you control yourself, the Unsloth release above (loadable through llama.cpp, and through Ollama’s own ollama create workflow if you build a Modelfile around that GGUF yourself) is the closer match to what most people mean by “run it locally.”
The practical path: a hosted API
For nearly everyone, the useful answer to “how do I use GLM-5.2” isn’t local inference at all. NativePort has GLM-5.2 cataloged on its Hugging Face route as huggingface/zai-org/GLM-5.2, pinned by default to the novita serving backend, with deepinfra admitted as a cheaper alternative, reachable through the same OpenAI-compatible endpoint as any other model on the gateway:
export NATIVEPORT_API_KEY="np_..."
curl -X POST "https://api.nativeport.ai/inference/v1/chat/completions" \
-H "Authorization: Bearer $NATIVEPORT_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "huggingface/zai-org/GLM-5.2",
"messages": [{"role": "user", "content": "Hello"}],
"max_tokens": 300
}'
Set max_tokens explicitly: GLM-5.2 reasons by default before answering, and a cap too small can end generation inside that reasoning step with an empty reply; see the Python API tutorial for the numbers behind that. Usage is metered at $1.40 per 1M input tokens / $4.40 per 1M output tokens on the default novita pin, or $0.93 / $3.00 on deepinfra if you pin it explicitly: real, pass-through rates, no markup. Sign-up seeds $5 of credit automatically; adding more is the only place a fee applies (5.5%, on top-ups of $10 to $5,000). For the full request/response shape, streaming, provider pinning, tool calls, and error handling, see Use GLM-5.2 through an OpenAI-compatible API in Python, or wire it into a coding agent with OpenCode or Pi.
Z.ai also serves GLM-5.2 directly through its own hosted API (api.z.ai, model id glm-5.2) if you’d rather hold a separate Z.ai account and key. The tradeoff against NativePort is the usual one: a dedicated Z.ai account and its own billing versus one key and one balance shared across every model and provider already on the gateway.