Open weights, open data
Weights alone let you run a model without letting you check it. The training data is published for the same reason the failures are — so that what is claimed here can be verified by somebody who does not take our word for it.
What is published
Weights
Two formats for each model. Safetensors for anything that loads a transformer; GGUF for running on a laptop with no GPU and no account.
SoreQen S1 Mini
0.8BSoreQen S1 Mega
4BWhich quantisation
Q4_K_M— The default. Best size-to-quality trade; runs on a modest laptop.Q8_0— Near-lossless. Take it if you have the RAM.
Quantisation costs accuracy. Q4_K_M is a good trade, not a free one — if an answer matters, check it against Q8_0 or the safetensors build.
Datasets
Three corpora, curated for these models and published as they were used. As far as we know the Hinglish set is the only curated public instruction corpus of its kind.
SoreQen Hinglish
36,326 train · 741 validationRoman-script Hinglish conversation, with an English minority slice, for training assistants that answer Indian users in the register they actually write in.
31,507 Hinglish and 5,560 English rows, drawn from licensed conversation data, synthetic generation, and a calibration set.
13,657 rows rejected for too little Hindi. A row written almost entirely in English is not code-mixing, however Indian the prompt was. Another 8,518 went for no code-mixing at all.
SoreQen Reasoning
540,175 train · 11,023 validationMaths and deductive reasoning with step-by-step traces, plus an English general-reasoning slice.
MetaMathQA, OpenR1-Math, NuminaMath-CoT, Orca-Math, GSM8K, OpenHermes 2.5, SlimOrca, ProofWriter, and a synthetic reasoning set.
6,293,545 near-duplicate rows removed. Public maths corpora are largely the same problems re-templated. More than nine rows in ten were dropped before a single one was kept.
SoreQen Writing
23,728 train · 484 validationProfessional English writing — cold and internal emails, blog posts, rewriting tasks — most with the brief that produced them kept as a system prompt.
Licensed email and blog corpora, plus a small hand-written long-form set.
17,162 truncated rows discarded. An answer that stops mid-sentence teaches a model to stop mid-sentence. Exact duplicates took another 35,410.
The rejection counts are on this page deliberately. A row count says how large a corpus is; a rejection count says whether anyone looked at it. They exist because a single optional character in a single regular expression once removed the entire maths half of a corpus without anything appearing to go wrong, so every filter in the pipeline now reports what it dropped and why.
Licences and attribution
What you may do with these, and whose work they are built on.
| Artefact | Licence |
|---|---|
| SoreQen S1 model weights (all three) | Apache 2.0 |
| SoreQen datasets | Mixed — see each dataset card |
Statement of changes. The S1 generation is derived from Qwen3.5, developed by Alibaba Cloud and released under the Apache License 2.0. SoreQen S1 Mini, S1 and S1 Mega are low-rank fine-tunes of the 0.8B, 2B and 4B checkpoints respectively, merged back into full weights. The changes are to behaviour, not to architecture: Hinglish code-switching, tool calling, structured output and professional writing were trained in, while the thinking behaviour, vision tower, context length and output ceiling of the base were preserved. No architectural modification was made.
The datasets carry mixed terms because they carry mixed provenance — licensed corpora, public instruction sets and synthetic generation, each under its own terms. Every dataset card on HuggingFace lists its sources; check the one you intend to use rather than assuming a single licence covers all three.
Everything is published under one account, which is the account the weights actually live under rather than a mirror.