October 2026 Open-Weight AI Models Compared: Vendor Claims, Independent Evidence and What to Verify Before You Deploy

October 2026 brought several large open-weight AI models. "Open-weight" means the trained numbers that define the model (its weights) can be downloaded. It does not always mean the code, training data or license are fully open. This article looks at four releases and shows how to read their announcements: which statements are documented facts, which are vendor claims, and what you should test yourself.

All facts below come from the vendors' own pages, which I opened for this article. Most benchmark numbers on those pages were measured by the vendors, so treat them as claims until someone else reproduces them. Nothing here is investment advice, and I did not run any of these models myself.

In this article

• What was released, and when
• How these models are built: mixture of experts
• Vendor claims versus independent evidence
• Licenses and hardware
• Security and privacy implications
• A worked example
• Common mistakes
• Recap
• Research Sources

What was released, and when

Dates matter, because news sites often report a release days after it happens. The dates below come from the vendors' own pages, not from later roundups. Today is November 1, 2026.

• Kolibri-1 from Aleph Alpha: dated October 3, 2026 on its Hugging Face model card. It has 78 billion total parameters and handles German and English.
• Muse Spark Open from Meta: dated October 15, 2026 in both Meta's research post and its model card. It has 816 billion total parameters.
• Nemotron 3.5 Bolt 74B-A9B from NVIDIA: listed as October 15, 2026 on its model card.
• Mistral Large 4 open weights: Mistral's announcement says the weights followed a public preview "earlier this month". Neither that page nor the model card gives an exact date, so I do not state one.

A parameter is one learned number inside the model. More parameters usually means more memory needed, though not always better answers.

How these models are built: mixture of experts

All four are "mixture of experts" (MoE) models. Imagine a hospital with many specialists. For each patient, a router sends the case to only a few of them. In an MoE model, a router picks a few small sub-networks, called experts, for each token. A token is a small chunk of text, such as a word or part of a word.

This creates two sizes. "Total parameters" is everything you must store in memory. "Active parameters" is what runs for each token, which drives speed and compute cost. Meta's card lists 816B total and 42B active, with 256 experts and 8 used per token. Kolibri-1 has 78B total and about 3.46B active, with 384 routed experts plus one shared expert and 6 routed experts chosen per token.

Mistral's card describes about 1 trillion total parameters, with 128 routed experts, 4 active per token and one shared expert. NVIDIA's model is a hybrid of Mamba-2 (a different kind of sequence layer) and Transformer layers. Its card says it was compressed from a larger 120B model, reducing the number of experts used per token from 22 to 9.

The context window is how much text the model can read at once, including your prompt and its answer. Meta and Mistral list 1 million tokens. NVIDIA lists up to 256K tokens, shared by prompt, images, history, tool results and output. Aleph Alpha allows up to about 1 million but recommends 262,144 or fewer for efficiency and complex tasks.

Only active parameters affect per-token compute. Memory still has to hold the total. That is why a model with few active parameters can be fast yet still need a large server.

Vendor claims versus independent evidence

A benchmark is a fixed set of test tasks with a scoring rule. "Self-reported" means the model's maker ran the test. That can be honest and still not comparable. Prompts, tool setups and retries all change scores.

Here is how the four releases stand:

• Meta: the model card says its benchmark results are self-reported by Meta Superintelligence Labs, for example 89.0 on GPQA Diamond and 76.4 on Terminal-Bench 2.1. Meta's post calls the model "the most capable US open-weight model to date". That is a vendor claim, and the post itself describes only Meta's own evaluations.
• NVIDIA: all results are reported by NVIDIA. The card is unusually open about trade-offs. It trails a rival on some measures, and it says NVIDIA measured one competitor's MCP-Atlas score at 51.2 rather than the 62.8 that competitor reported.
• Aleph Alpha: the card shows scores such as 84.3 on GPQA Diamond and 66.4 on SWE-bench Verified. The developer publishes them, so independent checks would still be needed.
• Mistral: its post mixes self-run tests, results it attributes to third parties such as Artificial Analysis, and one private benchmark. It also quotes partner companies, which are testimonials, not measurements.

The NVIDIA example is worth remembering. The same test gave two different numbers depending on who ran it. Always ask who ran the test, with what setup, and whether you can repeat it.

Independent evaluators help. Artificial Analysis runs a public leaderboard of over 250 models. On the page I opened, the same model can appear several times at different "reasoning effort" settings, and I could not see an update date or full methodology there. Mistral's card cites an Artificial Analysis Intelligence Index score of 26 and calls it the highest among Western open-weight models. That scope is narrow, and I did not compare it with other scores because I could not confirm they use the same index version.

Cards can also disagree with themselves. Meta's card lists 816B parameters, while its file listing shows 822B. NVIDIA's card says 74B, while its file metadata shows 22B. NVIDIA's deployment field lists several GPU families, while its integration section lists only one. These are small, but they remind you to check the numbers that matter to you.

Licenses and hardware

"Open-weight" is not the same as "open-source". A license sets what you may do with the weights. The terms below come from the vendors' pages, and this is not legal advice.

• Meta's Muse Model License requires the license and usage policy to travel with copies. Products must display "Built with Muse Spark Open". Providers offering it as a service to third parties must notify Meta and sign a separate agreement above $20 million in gross revenue over any 12 months, according to the model card.
• Mistral's license is a modified MIT license. Businesses running a model-as-a-service product need a separate commercial license.
• Kolibri-1 uses Apache 2.0, but only for the published weights and configuration files. Code, training methods and parameter settings are excluded.
• Nemotron uses OpenMDW-1.1, and the card says it is ready for commercial or non-commercial use.

Hardware varies greatly. Quantization means storing weights with fewer bits to save memory, at some risk to quality. Meta's card says self-hosting needs roughly 700GB of VRAM (graphics memory) with NVFP4 or 2TB with BF16. Meta's post separately says the model can run on two Apple M5 Ultra Mac Studios with MLX. I could not reconcile these two figures from the pages, and the Mac claim is Meta's own demonstration.

Kolibri-1 needs roughly 78 GB of memory in FP8. NVIDIA's packed model is about 24GB and targets vLLM, a model-serving tool. Mistral's card shows examples using 8-GPU tensor parallelism, which splits one model across eight GPUs.

Security and privacy implications

Self-hosting can keep prompts and documents inside your own systems. But the same weights are also sold through many API providers. Mistral's post lists several access routes, each with its own terms. Check where your data goes before you send anything sensitive.

Safety training can be undone. Meta says the open weights have extra safety post-training, but downstream modifications could degrade safety. Its card reports a 99.2% refusal rate on an internal cyber jailbreak test, which is Meta's own measurement. A jailbreak is a prompt crafted to make a model ignore its rules.

Meta suggests running the model inside a wider system with guardrails. It also suggests human confirmation for irreversible agent actions. Mistral reports 97.1% resistance on a B3 agent security benchmark, but it also says it sees "a potential for uplift to conduct real attacks". It limits a cybersecurity variant to vetted partners.

Two documented points matter for data handling. NVIDIA's card tells users to make sure they have rights to any images or video they input. Aleph Alpha's card says that in decision support the model should only advise, not decide, and that its built-in knowledge ends June 18, 2026.

From general practice, not from these pages: if a model reads outside documents or web pages, hidden text in them can try to give it instructions. This is called prompt injection. Limit what tools the model can call, and keep a person in the loop for actions that cannot be undone.

A worked example

Illustrative scenario, not a test I ran. A small firm wants to summarize German and English contracts on its own servers. The firm owns a single server with 96 GB of GPU memory. Here is how it could use the information above.

1. Shortlist by language and size. Input: the requirement for German and English, plus 96 GB of GPU memory. What happens: the firm compares the cards. Kolibri-1 targets German and English and needs about 78 GB in FP8, per its card. Result: Kolibri-1 is a plausible candidate on paper. Why it matters: a model that does not fit your hardware or language is out before any benchmark is read.
2. Read the license. Input: Kolibri-1's Apache 2.0 notice. What happens: the firm notes the license covers only weights and configuration files. Result: internal use looks simple, but legal staff should confirm. Why it matters: licenses decide whether you can ship the result in a product.
3. Label each claim. Input: the GPQA Diamond and SWE-bench Verified scores. What happens: the firm records them as developer-reported. Result: they count as hints, not proof. Why it matters: coding scores say little about contract summaries.
4. Build a private test set. Input: 50 past contracts with human-written summaries. What happens: the firm runs the candidate and a second model, then has a lawyer score the summaries for missed clauses. Result: the firm gets its own evidence. Why it matters: your documents are the only benchmark that matches your work.
5. Add controls. Input: the card's advice to keep a person reviewing outputs. What happens: summaries go to a reviewer before use, and logs stay on the firm's servers. Result: errors are caught before they matter. Why it matters: the card warns of hallucinations and bias.
6. Recheck before launch. Input: new card versions and any license changes. What happens: the firm repeats steps 1 to 4 on the exact checkpoint it will use. Result: the decision rests on current facts. Why it matters: cards and weights get updated.

Common mistakes

• Treating a leaderboard score as proof for your task. Fix: use it to shortlist, then test on your own data.
• Copying parameter counts from blog roundups. Fix: use the vendor card, and note where the card disagrees with itself.
• Assuming "open-weight" means free for any use. Fix: read the license, especially revenue limits and service-provider terms.
• Sizing hardware by active parameters. Fix: plan memory for total parameters, and for the context length you actually use.
• Using a release date from a news article. Fix: use the vendor's page, and say so when no date is given.
• Letting a model act on untrusted text with broad permissions. Fix: limit tools, filter inputs and require approval for irreversible actions.

Recap

• Kolibri-1 (October 3), Muse Spark Open and Nemotron 3.5 Bolt (both October 15) have dates on vendor pages. Mistral Large 4's open-weight date is not given on its pages.
• All four are mixture-of-experts models: total parameters set memory needs, active parameters set per-token compute.
• Almost every benchmark number I read was self-reported. Independent leaderboards help, but check their method and version.
• Licenses differ widely. Check revenue thresholds, service-provider terms and what the license actually covers.
• Safety numbers are vendor measurements. Add your own guardrails, keep humans in the loop and decide where your data flows.
• Your own test set on your own documents is the strongest evidence you can collect.

Research Sources

• Muse Spark Open model card (Meta, Hugging Face)
• Muse Spark Open announcement (Meta Research)
• NVIDIA Nemotron 3.5 Bolt 74B-A9B model card
• Kolibri-1 model card (Aleph Alpha)
• Mistral Large 4 open-weights announcement (Mistral AI)
• Mistral Large 4 model card (Hugging Face)
• Artificial Analysis LLM leaderboard