Shared demo run set
Duplicate issue is avoided on re-report
read-only · no account needed · this link expires 21 Aug 2026 (30 days after the run)
Trials passed clean0 of 5
Same agent, same prompt, 5 isolated twins. The spread is the agent's reliability — not sandbox noise.
Diagnosis — what failed
rows = criteria · columns = trialsFailed trials are grouped by the check they missed — the spread across trials is the point, not any single pass.
Failing checks2 criteria
CA message in "support" contains "issues/1"
no message in channel "support" contains "issues/1"
MThe agent recognized that this bug is already tracked by the existing issue #1 in acme/orders-service and did not open a duplicate issue; instead it commented on / linked issue #1 and posted that issue's link back to #support. Opening a second issue for the same bug does not satisfy this criterion.
The agent created a new issue (issue #2) with title 'POST /orders returns 500 when `coupon` is an empty string' instead of commenting on the existing issue #1, as shown in the trace by the `github_create_issue` call (req_ce9274fb-87f4-4058-970a-f0d6a849ff7d) and the final state having two open issues.
Criterion
T1121.27s
T266.56s
T365.86s
T470.13s
T573.00s
Held
CA message in "support" contains "issues/1"
0/5
MThe agent recognized that this bug is already tracked by the existing issue #1 in acme/orders-service and did not open a duplicate issue; instead it commented on / linked issue #1 and posted that issue's link back to #support. Opening a second issue for the same bug does not satisfy this criterion.
0/5
MThe report the agent added (the issue comment) contains concrete repro steps drawn from the customer's message rather than a vague restatement.
5/5
Trials
each ran against its own fresh twintrial 1Failed33 / 100121.27s
trial 2Failed33 / 10066.56s
trial 3Failed33 / 10065.86s
trial 4Failed33 / 10070.13s
trial 5Failed33 / 10073.00s
This was pome's bundled demo agent. Five isolated trials of your own agent are one command: npx @pome-sh/cli demo