Explore a study's figure and report

This introductory example compares a sample mean with the same estimate plus a fixed bias. It produces a figure and report with a useful distinction: the average error increased, but it did not increase on every seed.

OFFLINE EXAMPLE / RETAINED OUTPUTStoy-v1 · a001

ResultsReportClaimsBrief

Generated figure and numerical summary

Higher average error, with individual counterexamples

Each point is one of 24 independent confirmation seeds. Positive differences mean the biased estimator had higher squared error. Three differences are negative and one is zero.

The dashed line marks the mean difference, +0.06510. Open the full-size figure.

Approximate 95% interval: [0.03985, 0.09036]. This is the mean ±1.96 standard errors across seeds, not an exact small-sample guarantee.

Read the exact numerical summary →

Generated manuscript / abstract excerpt

A reproducible comparison of two mean estimators

We compared the sample mean with the sample mean plus 0.25 for a zero-mean synthetic distribution. The biased estimator had higher average squared error across 24 confirmation seeds. This is a workflow demonstration with a known analytical expectation, not a claim of scientific novelty or agent quality.

Results reported in the manuscript

Recomputed quantity	Value
Sample-mean MSE	0.01529948
Biased-estimator MSE	0.08040365
Paired difference	0.06510417

The full manuscript explains the question, mathematical expectation, methods, findings and limitations. Its claim ledger points to the measurements behind each statement.

Download the full generated manuscript (Markdown) →

Relative references inside that file resolve in the full study. Run the tutorial to generate it and recompute the report.

Generated claim ledger

Which statements do the results support?

The average comparison and the claim about every seed have different answers. Each entry keeps its scope and supporting or contradicting evidence.

C1supported

The biased estimator had higher average squared error (paired MSE difference 0.06510417; approximate 95% CI [0.03985104, 0.09035729]).

Evidence scope and references

24 prespecified seeds, 64 Rademacher observations per seed, target zero, additive bias 0.25

• This synthetic known-answer problem does not establish novelty or improve an LLM research agent.
• The 95% interval is a normal approximation over seed replicates, not an exact small-sample guarantee.
• Averages do not establish an ordering for every individual seed.

{
  "supporting": [
    {
      "media_type": "application/json",
      "path": "campaigns/toy-v1/analyses/a001/outputs/summary.json",
      "retention": "retain-with-study",
      "sha256": "5f2bfb7d6202de07ce13b7711a3e2e3b6e19a7b5d63736bb1d33caf05b04119f"
    }
  ],
  "contradicting": []
}

C2contradicted

The biased estimator has greater squared error on every individual confirmation seed.

Evidence scope and references

24 prespecified seeds, 64 Rademacher observations per seed, target zero, additive bias 0.25

• This synthetic known-answer problem does not establish novelty or improve an LLM research agent.
• The 95% interval is a normal approximation over seed replicates, not an exact small-sample guarantee.
• Averages do not establish an ordering for every individual seed.

{
  "supporting": [],
  "contradicting": [
    {
      "media_type": "application/json",
      "path": "campaigns/toy-v1/analyses/a001/outputs/summary.json",
      "retention": "retain-with-study",
      "sha256": "5f2bfb7d6202de07ce13b7711a3e2e3b6e19a7b5d63736bb1d33caf05b04119f"
    }
  ]
}

C3supported

Leave-one-seed-out mean differences ranged from 0.06182065 to 0.07269022; 24 of 24 retained means were positive.

Evidence scope and references

24 prespecified seeds, 64 Rademacher observations per seed, target zero, additive bias 0.25

• This synthetic known-answer problem does not establish novelty or improve an LLM research agent.
• The 95% interval is a normal approximation over seed replicates, not an exact small-sample guarantee.
• Averages do not establish an ordering for every individual seed.

{
  "supporting": [
    {
      "media_type": "application/json",
      "path": "campaigns/toy-v1/analyses/a001/outputs/summary.json",
      "retention": "retain-with-study",
      "sha256": "5f2bfb7d6202de07ce13b7711a3e2e3b6e19a7b5d63736bb1d33caf05b04119f"
    }
  ],
  "contradicting": []
}

Read the exact claim ledger →

Original study brief

Does adding a fixed bias increase squared error when estimating a synthetic zero mean?

The example has a known mathematical answer. That makes it useful for learning how to plan a study, run it and check the resulting report.

Completion criteria from the brief

• Known-answer controls pass
• Twenty-four held-out confirmation seeds complete
• Raw outputs recompute manuscript numbers
• Failures and interruptions remain traceable

The introductory example supplies its scientific code. In a native study, the agent develops that code from your question and materials.

Read the exact brief JSON →

Selected files from the offline example.Artifact sources and digests →

These selected files are unchanged outputs from the offline toy study, which supplies its own scientific code. The complete study also retains its inputs, protocol, raw measurements, failures and retries. No model is involved in this example.

Run the example locally to create that full record and recompute the results. For work performed by a coding agent, read the completed native studies.