skip to content

research · article

A Market Mechanism for Rollout Selection in Group-Relative RL

Independent operators choose the prompts, commit their own compute before the outcome is known, and are paid only for groups a validator recomputes and finds to carry reward variance.

Romain Fanucci · Malik El · 2026-09-21

Group-relative RL only learns from groups whose rewards differ. A group in which every completion succeeds carries no gradient — and under uniform sampling that outcome becomes more common as the policy improves, because the model has grown past the prompt. A reactive filter discovers this only after generating the whole group, so the compute is already spent.

This report describes the mechanism Reliquary uses instead — independent operators choose which prompts to generate on, commit their own compute before the outcome is known, and are paid only for groups a validator recomputes and finds to carry reward variance — and then reports what happened when it trained a 4B model on math and code for sixteen days.

12,852optimizer updates · Qwen3-4B-Base
≈6.6Mtraining rollouts supplied by operators
+31.1ppmath pass@1 · exact-content-disjoint
+20.1ppcode pass@1 · exact-content-disjoint

The run

Training ran from Qwen3-4B-Base for 12,852 optimizer updates. Independent operators supplied approximately 6.6 million rollouts; the trainer generated none of them. On the exact-content-disjoint evaluation subset the released checkpoint, ReliquaryForge/Reliquary-4B, gains 31.1 points of math pass@1 (95% CI 28.5–33.8) and 20.1 points of code pass@1 (18.6–21.6) over the base model.

Roughly 94% of the math gain and 72% of the code gain were acquired in the first ~800 updates. The remainder is a long marginal climb, and on math it stopped: flat from about update 4,600, +0.7 points over the last 236 checkpoints, while code continued.

Transfer

GSM8K and MATH-500 are near-domain — the math corpus is synthesized from their train splits — so the transfer claim rests on AMC23, AIME, HumanEval+ and MBPP+. Both models are evaluated under one decoding contract: vLLM, eight samples per problem at T = 1.0, top_p = 1.0, top_k = 0, an 8,192-token cap, plus a separate greedy pass.

Sampled pass@1 with greedy pass@1 in parentheses, for Qwen3-4B-Base and the released Reliquary-4B checkpoint, across seven benchmarks. Δ is the mean per-problem paired difference in percentage points, followed by its t-statistic.
benchmarkNQwen3-4B-BaseReliquary-4BΔ (paired t)
GSM8Knear-domain1,31955.9 (72.5)90.3 (91.9)+34.5 (53.9)
MATH-500near-domain50037.2 (52.8)72.6 (77.8)+35.4 (28.8)
AMC23transfer4015.6 (25.0)47.5 (55.0)+31.9 (6.6)
AIME 2024transfer302.5 (3.3)10.8 (10.0)+8.3 (2.1)
AIME 2025transfer300.8 (0.0)9.2 (13.3)+8.3 (2.4)
HumanEval+transfer16452.1 (78.7)72.9 (76.8)+20.9 (11.4)
MBPP+transfer37843.7 (68.0)67.6 (70.6)+23.9 (19.6)
Scores are percentages: sampled pass@1, with greedy pass@1 in parentheses. Δ is the mean per-problem paired difference in percentage points followed by its t-statistic — not a confidence interval. Each AIME set holds only 30 problems, and eight samples per problem do not make them 240 independent ones, so those two rows should be read cautiously. These are comparisons under one decoding contract, not against a matched centralized training control.

What this run could not measure

The report is explicit about the gap between demonstrating that the mechanism operates and demonstrating that it is efficient. The second was not tested here.

unmeasured

Generation cost and predictive selection

The intended efficiency gain is fewer unusable groups generated, through operators predicting in-zone prompts before they spend compute. This run neither controlled emissions nor observed every local attempt, so it cannot measure prediction accuracy, avoided generation, or cost per usable group.

unmeasured

Comparative learning efficiency

No matched centralized run was made — same base, corpus, decoding contract and update count. Whether the selected batch learns faster or slower per unit of total compute than a uniform dynamic sample is unmeasured.

unmeasured

Supply elasticity

The run held four admitted groups per slot throughout, so a tighter gate would have absorbed slack rather than tested whether operators can follow moving demand. The demand curve is a capability, not a result.

unmeasured

The held-out suite

It was contaminated by content. The exact-content-disjoint subset excludes prompts whose content hashes match trained prompts, but it does not exclude near-duplicates. The next run excludes by content hash on the validator and audits the archive afterwards.

The run also paid operators the full emission the subnet allocated to it — no reserve price, no bid, no cost control — so no per-rollout figure is reported, and none should be inferred. A single run is not a replication.

Read the report

The full document opens as a typeset paper on a light substrate: sixty pages with figures, appendices, the reproduction manifest and the reference list. It prints to PDF directly from the browser.

a market mechanism for rollout selection in group-relative RL

Code is at github.com/ReliquaryForge; the evaluated weights are published as ReliquaryForge/Reliquary-4B at revision 01d004d8.

Citing

@techreport{reliquary2026rollout,
  author      = {Fanucci, Romain and El, Malik},
  title       = {A Market Mechanism for Rollout Selection in Group-Relative RL},
  institution = {Reliquary},
  year        = {2026},
  month       = {9},
  type        = {Technical report},
  note        = {Not peer reviewed},
  url         = {https://reliqua.ai/research/rollout-selection-mechanism/paper}
}