skip to content
Reliquary
RELIQUARY
BlogDocsOpen App Open Console

THE DECENTRALIZED POST-TRAINING PLATFORM

Take intelligence further.

Generate training data for your tasks, evaluate your models, and train them with reinforcement learning. Powered by independent GPUs and a network that discovers what models can still learn.

Open App Explore the research

Network

Reliquary distributes RL rollout generation across independent GPUs, from consumer hardware to datacenter machines. Each contributor chooses which prompts to test and runs the same model at their own pace.

Every contribution is verified against the published checkpoint before selected groups can be used for training.

Contributing takes an NVIDIA GPU with 24 GB of memory or more. Requirements

Read the network guide See live network activity
  • 8 attempts at one prompt, filled when solved
  • Selected for training
  • How often a GPU sends a group

Learning signal discovery

Generating more responses isn’t enough. The challenge is finding prompts that still help a model learn.

In GRPO, when all attempts succeed or all fail, there’s no reward contrast to learn from. The useful prompts are those where some attempts succeed and others fail.

Reliquary’s contributors compete to find these prompts, directing compute toward useful training signals rather than simply generating more data.

Prompt difficulty for the current model

Learning signal. 3 of 8 attempts solved, so the update can favor the better ones.

How GRPO scores this group
Reward
100101001 if solved, 0 if not
Group mean
3 / 8 = 0.375the group-average line
Advantage
1 − 0.375 = +0.6253 solved, above the line
0 − 0.375 = −0.3755 failed, below the line

Advantage = reward − group mean. GRPO then divides each advantage by the spread of the group, which changes neither the signs nor the zeros.

Learnable: 3 of 8 attempts solved, group mean 0.375. The update makes the 3 solved attempts more likely (advantage +0.625) and the 5 failed ones less likely (advantage −0.375). Learning signal.

Training

Generate task-specific training data, evaluate what changes between checkpoints, and improve your models through reinforcement learning.

Early access

Batch Generation

Turn your tasks into a private training dataset you can inspect, export and train on.

  • Code
  • Math (selected)
  • Instruction following
  • Logic
  1. What is the remainder when 210 is divided by 7?
  2. Solve for x: 3x + 5 = 20.
  3. How many positive divisors does 36 have?
  4. What is the greatest common divisor of 84 and 126?

Each answer ends in \boxed{} and is checked against the reference.

2,000prompts×4samples=8,000examples

One pinned model revision, up to 4,096 output tokens per answer.

  • Passed the checks
  • Rejected
  • Not generated yet

Each square is one example. Most generated examples passed the checks, a few were rejected, and the rest of the batch is still being generated.

dataset.jsonl8,000 examples, one per line

#PromptAnswerCheck
1What is the remainder when 2^10 is divided by 7?2Passed
2Solve for x: 3x + 5 = 20.5Passed
3How many positive divisors does 36 have?9Passed
4What is the greatest common divisor of 84 and 126?42Passed

+ 7,996 more lines

{"prompt": "What is the remainder when 2^10 is divided by 7?", "response": "1024 = 7 × 146 + 2. \\boxed{2}", "meta": {"env": "math", "sample": 1, "max_tokens": 4096}}
Example data
Request early access Service details
Coming soon

Evaluations

Evaluate your models on the tasks you care about, compare checkpoints, and inspect the scores, answers, and grading behind each result.

A rising average can hide regressions on individual tasks.

Request early access Service details
Example data: pass@1 of a baseline and a new checkpoint on three sample tasks. The average improves while one task regresses.
Single-attempt success rate

Average of the three tasks: pass@1 from 55% to 61%, up 6 points

Select a task to check one of its answers

Same tasks, graders and sampling for both checkpoints, 8 samples per prompt, temperature 1.0

Task
JSON outputOne answer from the new checkpoint
Prompt
List the action items in this ticket as a JSON array of strings.
New checkpoint’s answer
Action items:
- Add a guard for empty files
- Add a regression test
Verdict
FailExpected a JSON array, received a bulleted list

Coming soon

RL Training

Train your model on your own tasks, with every update built from attempts it can still learn from.

You bring
Your model, your task environment and a way to grade each attempt.
You get
Each new checkpoint with its training setup and evaluation evidence.
Join the RL waitlist Service details

  1. Task & reward

    Your prompts, and a reward that marks each attempt solved or failed .

  2. Rollouts

    The current checkpoint makes 8 attempts at each prompt.

  3. Selection & verification

    Only groups with mixed results carry learning signal , and each one is verified before training.

  4. Training / checkpoint

    The selected groups train the model. Each new checkpoint is saved and inspectable.

The new checkpoint generates the next attempts.

Concept: the planned RL Training loop.

Research

We study how to post-train models on rollouts from independent GPUs: which prompts carry learning signal, how to check work we do not run, and what the resulting models show. Each result is published with its setup and its limits.

Released model · Sep 2026

Reliquary-4B

Qwen3-4B-Base, trained with reinforcement learning on about 6.6 million rollouts, all generated by independent contributors. The trainer and validator are centralized.

Compared with its base model. Sampled pass@1, 8 samples per problem, T = 1.0.

Read the paper Model weights
Limits. One run, no matched centralized control, not peer reviewed. Under greedy decoding the gains are smaller, and HumanEval+ falls from 78.7 to 76.8. MATH-500 is near-domain: the math training corpus was synthesized from its train split.
Data table
Sampled pass@1 (%), 8 samples per problem, T = 1.0. The change is the mean paired difference per problem, in points, computed before rounding.
BenchmarkQwen3-4B-BaseReliquary-4BChange
MATH-500Near-domain, 500 problems37.272.6+35.4
HumanEval+Transfer, 164 problems52.172.9+20.9
AMC23Transfer, 40 problems15.647.5+31.9

Also measured

  • 88% to 41%

    Under uniform sampling, the share of code prompts that still produced learning signal fell from 88% to 41% over the Reliquary-4B run (math: 90% → 53%).

    500 index-held-out prompts × 16 rollouts, not content-disjoint. One run.

    See the analysis

  • plus 14 percentage points of pass@1

    With learning-frontier selection, pass@1 ended 14 points above vanilla GRPO at the same step count.

    Qwen3-4B-Instruct, 300 steps, held-out math, 0.47 → 0.61. Single run, not replicated. June 2026; the baseline samples prompts uniformly.

    Read the article

Selected publications

  1. technical reportThe Reliquary-4B training report
  2. field noteReliquary 2B: training done
  3. field noteEight rollouts are not eight votes
All research Blog