Somewhere in Beijing, a lab named after a very ambitious space pun looked at the "closed AI lab" business model, shrugged, and open-sourced a model with more parameters than most countries have citizens. Kimi K3 is here, it's enormous, and you can download the weights right now like it's a Tuesday.
2.8 Trillion Reasons to Pay Attention
Moonshot AI released the full weights and technical report for Kimi K3 on July 16, a 2.8-trillion-parameter Mixture-of-Experts model that only activates 16 of its 896 experts per token, keeping inference costs sane despite the astronomical headline number. It ships with native visual understanding and a full 1-million-token context window, which is enough room to feed it your entire codebase and still have space left for the group chat.
The real story isn't the parameter count, though — it's the architecture. Two new tricks, Kimi Delta Attention and Attention Residuals, deliver up to 6.3x faster decoding at million-token context lengths and roughly 25% higher training efficiency, which Moonshot is billing as 2.5x more "intelligence per unit of compute" than its predecessor. Translation: they didn't just build a bigger boat, they redesigned the hull.
Why This Isn't Just a Bigger Number Flex
Open-weight models catching up to proprietary frontier systems changes the calculus for every startup that's been paying per-token rent to a closed API. Kimi K3 leads on benchmarks like Program Bench and SWE Marathon, and while it still trails top-tier proprietary models like Claude Fable 5 and GPT 5.6 Sol on the hardest coding tasks, the gap is closing fast — and this one you can self-host.
Moonshot also opened up the infrastructure behind K3: attention kernels, its MoE communication library, and the agent-environment tooling used to train it. That's the part serious AI labs will actually care about, because it means the next wave of open models built on this stack won't be starting from scratch.
The frontier AI race used to be a two-horse affair. Increasingly, it looks like a very crowded stable — and several of the horses are handing out their training manuals for free.
Source: MarkTechPost