Completed studies and their outputs
These studies show what a coding agent returned after working from a research brief with Allagma. Each package includes its protocol, code, raw results, report and critique. Start with a result below, then follow the links to the full record and reproduction instructions.
For a small example you can run without a model account, use the offline toy tutorial or preview its figure and report. That example supplies the scientific code; the native studies below were developed and executed by coding agents.
Weight EMA: a figure and a qualified finding
Section titled “Weight EMA: a figure and a qualified finding”The agent investigated how training duration and learning-rate policy affect the benefit of averaging tiny diffusion model weights. It completed 32 cells and retained 96 raw/EMA model states.
The relative benefit of EMA was larger under constant learning rate than at completed cosine endpoints. With four independent seeds per dataset, duration and interaction estimates remain inconclusive. A smaller relative benefit does not imply worse absolute EMA quality.
Read the result and its limits · Open the full-size figure · Read the agent’s report
This preview is retained r07 work. Raw-data reanalysis and saved-state checks passed; a second full training replay is not claimed. The AI Scientist source attribution and license apply to the adapted study materials.
CORE CULP: reproduced answers and an explained source defect
Section titled “CORE CULP: reproduced answers and an explained source defect”The agent reproduced a graph-classification capsule. All six requested answers matched the original scorer. Inspection also found that the Iris and Zoo outputs printed CN/AA labels while actually calling the CS predictor. The result table makes that distinction explicit.
| Output | What a reader can inspect |
|---|---|
| Six numerical answers | Printed label, actual predictor and accuracy for each dataset |
| Source and graph checks | The upstream label defect and the agent’s own corrected checker |
| Reproduction instructions | A full run in a fresh environment and a separate raw-result recomputation |
Read the CORE CULP result and full replay evidence.
The fixed transductive splits do not provide a cross-split or population uncertainty estimate. A clean-checkout replay of this r01 package passed, including environment setup and full execution.
Modular addition: improved accuracy without observed grokking
Section titled “Modular addition: improved accuracy without observed grokking”The agent compared weight decay 0 and 1 in four paired seeds of a small MLP. All eight training trajectories completed 100,000 updates and memorized the training set. Final held-out accuracy improved with weight decay, but no trajectory met the sustained 95% accuracy threshold for grokking.
The package retains training curves, checkpoints, paired estimates and a report that distinguishes the endpoint result from the unobserved transition. The mean paired accuracy difference was 67.46 percentage points, with a 95% t interval of 50.31 to 84.61 points. Four seeds make those small-sample assumptions fragile.
Read the paired results and qualifications.
The r04 package passed retained-data reanalysis and checkpoint verification. A second full training replay is not claimed, and this MLP study does not replicate the source paper’s transformer experiments.
How these examples were selected
Section titled “How these examples were selected”Each native example is the first complete Allagma package for its task in frozen run order. Selection did not depend on favorable scientific outcomes. These are workflow examples, not evidence of general research superiority.
The twelve-session comparison includes every assigned run. All twelve completed the required science. Allagma produced six complete packages out of six; plain Codex produced four out of six. The two baseline packages contain critiques but lack explicit binding to the reviewed revision. This small local comparison does not establish a general time, token or quality advantage. Model critique and controller checks do not replace independent scientific peer review.
Full archives are optional. Artifact distribution explains the downloads, and third-party notices identify the licenses. Allagma’s core is MIT licensed; study materials retain their own terms.
Canonical source · Generated from the maintained repository document at this build’s commit.