Group-relative RL only learns from groups whose rewards differ. A group in which every completion succeeds carries no gradient — and under uniform sampling that outcome becomes more common as the policy improves, because the model has grown past the prompt. A reactive filter discovers this only after generating the whole group, so the compute is already spent.
This report describes the mechanism Reliquary uses instead — independent operators choose which prompts to generate on, commit their own compute before the outcome is known, and are paid only for groups a validator recomputes and finds to carry reward variance — and then reports what happened when it trained a 4B model on math and code for sixteen days.
The run
Training ran from Qwen3-4B-Base for 12,852 optimizer updates. Independent operators supplied approximately 6.6 million rollouts; the trainer generated none of them. On the exact-content-disjoint evaluation subset the released checkpoint, ReliquaryForge/Reliquary-4B, gains 31.1 points of math pass@1 (95% CI 28.5–33.8) and 20.1 points of code pass@1 (18.6–21.6) over the base model.
Roughly 94% of the math gain and 72% of the code gain were acquired in the first ~800 updates. The remainder is a long marginal climb, and on math it stopped: flat from about update 4,600, +0.7 points over the last 236 checkpoints, while code continued.
Transfer
GSM8K and MATH-500 are near-domain — the math corpus is synthesized from their train splits — so the transfer claim rests on AMC23, AIME, HumanEval+ and MBPP+. Both models are evaluated under one decoding contract: vLLM, eight samples per problem at T = 1.0, top_p = 1.0, top_k = 0, an 8,192-token cap, plus a separate greedy pass.
| benchmark | N | Qwen3-4B-Base | Reliquary-4B | Δ (paired t) |
|---|---|---|---|---|
| GSM8Knear-domain | 1,319 | 55.9 (72.5) | 90.3 (91.9) | +34.5 (53.9) |
| MATH-500near-domain | 500 | 37.2 (52.8) | 72.6 (77.8) | +35.4 (28.8) |
| AMC23transfer | 40 | 15.6 (25.0) | 47.5 (55.0) | +31.9 (6.6) |
| AIME 2024transfer | 30 | 2.5 (3.3) | 10.8 (10.0) | +8.3 (2.1) |
| AIME 2025transfer | 30 | 0.8 (0.0) | 9.2 (13.3) | +8.3 (2.4) |
| HumanEval+transfer | 164 | 52.1 (78.7) | 72.9 (76.8) | +20.9 (11.4) |
| MBPP+transfer | 378 | 43.7 (68.0) | 67.6 (70.6) | +23.9 (19.6) |
What this run could not measure
The report is explicit about the gap between demonstrating that the mechanism operates and demonstrating that it is efficient. The second was not tested here.
unmeasured
Generation cost and predictive selection
The intended efficiency gain is fewer unusable groups generated, through operators predicting in-zone prompts before they spend compute. This run neither controlled emissions nor observed every local attempt, so it cannot measure prediction accuracy, avoided generation, or cost per usable group.
unmeasured
Comparative learning efficiency
No matched centralized run was made — same base, corpus, decoding contract and update count. Whether the selected batch learns faster or slower per unit of total compute than a uniform dynamic sample is unmeasured.
unmeasured
Supply elasticity
The run held four admitted groups per slot throughout, so a tighter gate would have absorbed slack rather than tested whether operators can follow moving demand. The demand curve is a capability, not a result.
unmeasured
The held-out suite
It was contaminated by content. The exact-content-disjoint subset excludes prompts whose content hashes match trained prompts, but it does not exclude near-duplicates. The next run excludes by content hash on the validator and audits the archive afterwards.
The run also paid operators the full emission the subnet allocated to it — no reserve price, no bid, no cost control — so no per-rollout figure is reported, and none should be inferred. A single run is not a replication.
Read the report
The full document opens as a typeset paper on a light substrate: sixty pages with figures, appendices, the reproduction manifest and the reference list. It prints to PDF directly from the browser.
a market mechanism for rollout selection in group-relative RL
Code is at github.com/ReliquaryForge; the evaluated weights are published as ReliquaryForge/Reliquary-4B at revision 01d004d8.
Citing
@techreport{reliquary2026rollout,
author = {Fanucci, Romain and El, Malik},
title = {A Market Mechanism for Rollout Selection in Group-Relative RL},
institution = {Reliquary},
year = {2026},
month = {9},
type = {Technical report},
note = {Not peer reviewed},
url = {https://reliqua.ai/research/rollout-selection-mechanism/paper}
}