Teutonic-I 10B: How a Decentralized Battle for the Throne Outperformed 72B Model
10 billion parameters. 70 days of open competition. 2163 completed model duels - everything powered by Const and Dendrite. A result that surpassed not only other models trained through decentralized efforts, but also models several times larger.
Sounds like another race to see who can throw the most GPUs at the problem?
Not this time.
Teutonic-I 10B was not built in a single lab, by a single team, following a predefined training recipe. Instead, the model evolved on Bittensor Subnet 3 through a “king of the hill” system.
Independent miners downloaded the current best checkpoint - the king, trained their own challengers, and attempted to take the crown. The network did not care who spent the most on compute or who made the loudest claims about their results. What mattered was reproducible performance: lower loss on the same randomly selected evaluation set, with an advantage large enough to pass a transparent statistical evaluation.
The result?
Teutonic-I 10B achieved an average score of 62.28% across 11 shared benchmarks, outperforming models with 72B, 40B, and 18B parameters. For comparison, Covenant-72B scored 57.55%, Quasar-Preview 18B scored 59.78%, INTELLECT-1 10B reached 45.38%, and Psyche Consilience 40B achieved 39.67%.
Teutonic - Subnet 3, is a project focused on developing AI models through a decentralized training system built within the Bittensor ecosystem.
The competition discussed in this article focused on pretraining Teutonic-I 10B (based on the Quasar-10B architecture), with a context window fixed at 2,048 tokens due to the requirements of the evaluation system.
For now, Teutonic-I 10B is a relatively small model. However, the broader project is moving toward training much larger models - exceeding 100B parameters with the goal of competing directly with leading open-source models.
In it’s current form, Teutonic does not prescribe how miners must train their models. It does not enforce specific training scripts or pipelines, dictate which datasets must be used, impose a fixed training duration, or require any particular training infrastructure.
The network defines only the required model architecture, the rules governing the data used for evaluation, and the threshold that a challenger must exceed to become the new king. How a challenger is trained and on what hardware is entirely up to the miner.
Teutonic is therefore not a conventional form of distributed training in which many machines synchronously compute gradients for the same optimization step and aggregate them across the network. Here, miners work independently, and progress emerges through the selection of successive winning checkpoints.
What is decentralized is not only access to the competition, but also the search for better training methods.
King-of-the-hill
The checkpoint uses the same architecture as the current king.
How to think about it
- The system works like evolution under a strict selection rule.
- Training effort alone earns no credit.
- Only measurable improvement matters.
Strict selection criterion
No reward for effort invested.
No reward for merely training a model.
The system is built around a “king of the hill” mechanism. At any given moment, there is one official best checkpoint - the king. Any miner can prepare a model with the same architecture and challenge it.
One way to think about the process is as evolution under an extremely strict selection criterion: a new model receives no credit simply for the effort invested in training it. It must actually outperform the best known version.
For this comparison, we use the average score across 11 benchmarks shared by the evaluated models: MMLU, ARC-C, ARC-E, PIQA, HellaSwag, OpenBookQA, BBH, TruthfulQA, WinoGrande, GPQA, MATH-500.
| Benchmark | Teutonic-I 10B | Quasar 10B | Quasar-Preview 18B | INTELLECT-1 10B | Psyche Cons. 40B | Covenant 72B |
|---|---|---|---|---|---|---|
| MMLU | 75.29 | 49.63 | 60.87 | 32.69 | 24.23 | 67.11 |
| ARC-C | 63.82 | 41.89 | 63.40 | 44.80 | 31.14 | 56.83 |
| ARC-E | 84.97 | 62.96 | 82.45 | 71.76 | 55.77 | 80.93 |
| PIQA | 82.81 | 69.91 | 83.30 | 77.73 | 76.12 | 81.56 |
| HellaSwag | 79.42 | 62.37 | 73.07 | 70.26 | 63.67 | 80.61 |
| OpenBookQA | 49.00 | 35.40 | 46.40 | 43.80 | 35.20 | 44.00 |
| BBH | 49.51 | 31.26 | 38.10 | 32.93 | 30.50 | 45.96 |
| TruthfulQA | 49.58 | 41.01 | 41.70 | 35.45 | 37.90 | 49.41 |
| WinoGrande | 77.35 | 56.27 | 67.56 | 63.30 | 56.99 | 75.85 |
| GPQA | 33.98 | 25.84 | 29.28 | 25.84 | 24.66 | 30.03 |
| MATH-500 | 39.40 | 0.00 | 71.40 | 1.00 | 0.20 | 20.80 |
| Average | 62.28 | 43.32 | 59.78 | 45.38 | 39.67 | 57.55 |
Despite having only 10 billion parameters, Teutonic-I achieved the best result on 8 of the 11 shared benchmarks.
Its strongest result is on MMLU, where it scored 75.29% more than eight percentage points above Covenant-72B. Teutonic-I also leads on BBH and GPQA, as well as both ARC variants.
Quasar-Preview retains a small advantage on PIQA, while Covenant leads on HellaSwag. MATH-500 remains Teutonic’s clearest weakness relative to Quasar-Preview, although Teutonic's result is still higher than that of roughly 80% of the models included in the comparison.
On the more difficult MMLU-Pro benchmark, Teutonic-I scored 39.84%, compared with 33.20% for Quasar-Preview.
The core mechanism for selecting the best model in Teutonic is a direct duel between the current king and a challenger.
Both models are evaluated on the same set of sequences, each 2,048 tokens long. This ensures that the comparison is not distorted by differences in evaluation data.
For each sequence, the evaluator calculates the average cross-entropy loss and then computes the difference:
- di
- the challenger’s advantage on sequence i
- Liking
- the king’s average loss on sequence i
- Lichallenger
- the challenger’s average loss on sequence i
If di > 0, the challenger achieved a lower loss and predicted the next tokens more accurately for that sequence.
However, a better average result alone is not sufficient to take the throne. First, the mean loss difference is calculated:
- N
- the number of evaluated sequences
- μˆ
- the challenger’s mean advantage over the king
To estimate the uncertainty of this result, Teutonic uses a percentile bootstrap. A total of B = 10,000 samples are drawn with replacement, and for each bootstrap replicate the mean is calculated:
- B
- the number of bootstrap replicates, set to 10,000
- ij
- the index sampled with replacement at position j
- μˆ*(b)
- the mean advantage in bootstrap replicate b
The resulting distribution of means is then used to calculate the key metric used by the system: the Lower Confidence Bound (LCB).
- Qα
- the quantile at significance level α
- α
- the significance level, set to 0.001 by default
- μˆ*
- the bootstrap distribution of mean advantages
- δ
- minimum required advantage, set to 0.0015
The LCB provides a conservative estimate of the challenger’s advantage. In other words, it is designed to provide very strong statistical confidence that the challenger’s true mean advantage is not below the calculated bound.
As a result, a new model cannot take the throne simply by achieving a slightly better average score. Its advantage must be sufficiently large and statistically robust.
Another important part of the process is the deterministic selection of evaluation data. The set of evaluation sequences is not chosen by the miner. Instead, it is determined algorithmically from the hash of the submission block and the current mixture of data sources. This means that the king and challenger are always evaluated on identical examples, preventing either model from gaining an advantage through favorable selection of evaluation data.
The entire pretraining competition can be viewed as one global training process. Its overall direction could be managed by steering miners toward higher-quality results through global hyperparameters such as delta, the number of evaluation samples, and dataset weights.
Over the course of the competition, 13 hyperparameter updates were introduced. Dataset configuration was changed 12 times, delta was changed once, and the number of evaluation samples was changed twice.
The chart shows benchmark results at the points where hyperparameters were changed. Each score was calculated for the final king preceding a given configuration update. It also includes the score of the model used to start the competition initialized with random weights as well as the highest-quality model, selected according to its average benchmark score.
During the first seven days, the average benchmark score jumped from 30.56 to 56.48. That is an increase of 25.92 percentage points in a single week, accounting for more than 80% of the total improvement achieved before the later peak.
In the second half of June, the training focus shifted more heavily toward mathematical problems and reasoning. By July, the model had entered a plateau around 61–62 points.
The best result was achieved by king #191, which reached an average benchmark score of 62.28% on August 1.
Across the entire competition, a total of 12 different datasets were used. How long each dataset remained part of the training objective depended on the performance of models across the network.
During evaluation, samples are randomly drawn from the entire dataset, which contains more than 4T tokens.
The decision to introduce, rebalance, or remove a particular dataset depended on the desired direction of the global pretraining process.
The competition lasted 70 days, beginning on June 2 and ending with the final configuration point on August 10, 2026.
The best recorded checkpoint, with an average benchmark score of 62.28%, appeared on August 1 as king #191, 60 days after the competition began.
During the competition, 2163 model duels were completed, resulting in 203 successful coronations of a new king.
Throne-based rewards
All five kings are registered. Each receives an equal 20% share.
Why reward dethroned models?
- Strong checkpoints remain valuable after they are dethroned.
- A miner keeps earning through the next four dethronements.
- This softens winner-takes-all without removing pressure to improve.
What the subnet pays for
Not claimed GPU time
Not tokens processed
The reward mechanism is directly tied to the throne.
The validator assigns subnet weights to the current king and up to four previous kings, provided that their hotkeys remain registered on the network.
Each receives an equal 20% share of the incentive. If one of the historical kings is no longer present in the metagraph, the remaining shares are normalized accordingly.
Why reward models that have already been dethroned?
A strong checkpoint does not stop being a valuable contribution the moment someone improves it by another small step.
A miner who takes the throne continues to receive a share of emissions through the next four dethronements. This softens the winner-takes-all nature of the competition without removing the pressure to make progress: to enter the five-model reward window, a miner must first become king.
In practice, the subnet does not pay miners for claimed GPU time or for the number of tokens they processed.
It pays for verified model improvement.
The next major challenge and likely the next competition on the subnet will be high-quality pretraining of significantly larger models, potentially reaching approximately 100B parameters.
The development of the subnet itself has also revealed many problems worth exploring in greater detail, including various attempts to copy models.
At the same time, several mechanisms have been developed that deserve their own explanations, including methods for creating high-quality synthetic datasets and autonomously selecting pretraining hyperparameters.
There is therefore much more to come from the work being done on the Teutonic subnet.
- Teutonic github
- https://github.com/unarbos/teutonic
- Teutonic
- https://teutonic.ai/