What it takes to run locally

• Weights scale with all 594B parameters. The sparse MoE design routes each token to 8 of 384 experts, but that reduces compute per token — not memory: every expert must still be resident.
• KV cache assumes 48 layers, 8 KV heads and a 128-wide head at a 32K context, which is an estimate: the model card does not publish head counts. Longer contexts scale this term linearly.
• Total memory adds 12% over raw weight size for activation buffers, the runtime and allocator fragmentation.
• Active parameters are 17B, which is what drives generation speed; the 594B total drives how much memory you need.

Generated Nov 6, 2026 from the specs on the specifications page. If the vendor revises the parameter or layer count, these figures change on the next build automatically.

The active-parameter figure describes compute, not storage. Each token touches only 8 of 384 experts, so the arithmetic for a given token is comparable to a 17B dense model — which is why it generates at roughly the speed of one. But which experts a token needs varies token to token, so the runtime must hold all 594B parameters in memory and move them through it.

This is the single most common misconception about sparse models, and it is why the 380.7 GB 4-bit figure, not the 17B parameter count, is the number to plan around.

Third parties measure the model at 135 output tokens per second on hosted infrastructure — consistent with a model whose compute per token resembles something far smaller than its footprint.

The only way to use Inkling 2 Small today is over the API. Fine-tuning on Tinker is available now, and adapters are small enough to store locally, but the base weights stay on the vendor's side. See the pricing page for what that costs, or the weights tracker for what would have to change.