What we are building, and what it cost
The gap we care about is between a model that answers and a model that works. Everything below is how far along that we actually are, including the parts that did not go well.
The problem we picked
Reasoning is the part of a model that is hardest to buy with parameters. A larger model recalls more; it does not necessarily notice that step four contradicted step two. So the research question here is not how large a model we can afford to train but how much of that noticing fits into one small enough to give away.
So the S1 line is deliberately small. A 0.8B model that can be given away and a 4B model that answers in a second are more useful to the people we are building for than a model that would top a leaderboard and cost too much to offer. Later generations will be more capable. The direction does not change.
One capability got its own training corpus, because very few models have it and none of the public data was usable for it. Roman-script Hinglish is not Hindi and not English; it is code-switching, and a model trained on either half alone treats a sentence that switches mid-clause as a malformed sentence. Building for it meant rejecting more rows than we kept — thirteen thousand for carrying too little Hindi to count as mixed, eight thousand for not mixing at all.
Two campaigns
Both ran in August 2026. Only the second produced the weights anyone can download, and saying so is the point — a research page that blurs the two is claiming credit for a run that did not work.
From scratch
Did not shipA mixture-of-experts model trained from nothing — 121M active parameters of 333M total, 2.38 billion tokens, on two RTX 5090s under a hard twenty-GPU-hour ceiling. The dispatch path reached roughly 52% model-FLOPs utilisation, which is a good number. It is also the campaign in which three separate runs completed while producing nothing at all.
Fine-tuning Qwen3.5
Shipped as S1The models you can download. Low-rank fine-tunes of Qwen3.5 at 0.8B, 2B and 4B, taught Hinglish, tool use, structured output and professional writing, while preserving the thinking behaviour, vision, context length and output ceiling the base already had. Three of its failures were silent too — one of them wrong by a factor of fifteen.
What shipped is a fine-tune of Qwen3.5 (Alibaba Cloud), and the licence and statement of changes are on the open-source page.
Six failures worth writing down
Forty-five were recorded. These are the ones that were silent — where the run finished, printed a plausible number, and was wrong. A crash tells you immediately; a clean run that did the wrong thing tells you nothing until you go and check.
- 01
A fine-tune that was quietly training from scratch
From scratchWeights were saved from a compiled model, which prefixes every key with `_orig_mod.`, and loaded into an uncompiled one with `strict=False`. Nothing matched. Nothing raised. The loader reported success and the model began learning from random initialisation, in a run whose loss curve looked entirely ordinary.
What it cost. A full training run, and the confidence in every run before it, until the checkpoint was diffed against the base.
- 02
Deduplication deleted the identity data
Fine-tuning Qwen3.5Near-duplicate detection is correct for a maths corpus, where the same problem appears a thousand times in different clothes. Applied to the small hand-written set that teaches the model what it is, it removed almost all of it — those rows are supposed to resemble each other.
What it cost. A model that could reason but could not say whose it was. Found by asking it.
- 03
Long context is not a longer array
From scratchRotary position embeddings do not extrapolate. Extending the context window without rescaling the frequencies produces a model that is fluent to its training length and incoherent one token past it — and the failure is at the far end of the input, where nobody looks first.
What it cost. Two rounds of debugging what looked like a data problem.
- 04
A benchmark that under-reported throughput fifteenfold
Fine-tuning Qwen3.5Step time was measured on the first step. The first step pays for kernel autotuning, memory-pool growth and cache warm-up, none of which recur. The number it produced said the run would take weeks.
What it cost. A rewrite of the training plan that was not needed, and nearly a rented machine that was not either.
- 05
A reward model grading a format nobody taught
From scratchThe reinforcement stage scored answers on a structure the supervised stage had never shown the model. Every sample scored badly, the gradient pointed at formatting rather than at reasoning, and the run converged neatly on the wrong thing.
What it cost. The lesson that a reward function is part of the dataset, not part of the optimiser.
- 06
An optional slash cost the corpus its entire maths half
Fine-tuning Qwen3.5One character in one pattern, in a filter that ran silently over hundreds of thousands of rows and reported only how many it kept. The count looked plausible because there is no such thing as an implausible count when you do not know what to expect.
What it cost. Every filter in the pipeline now prints what it rejected and why. The rejection tables on the open-source page are that change.
What changed because of them
Every one of those has a guard now, and the guards are the actual output of the work. Checkpoints are diffed against their base rather than trusted to load. Every filter in the data pipeline prints what it rejected and why, which is why there are rejection counts on the open-source page at all. Throughput is never measured on a first step. Reward functions are treated as part of the dataset rather than as part of the optimiser.
None of this is novel research. It is the unglamorous half of building a model, and it is published because a company that reports only its results is asking to be taken on faith — and because the next person to load a compiled checkpoint into an uncompiled model deserves to lose an afternoon rather than a week.