0xAlpha vs Codex xhigh vs GLM-5.3
One empty folder. One identical prompt. One simulated trading-systems challenge. Then an independent judge tried to break all three.
Three agents received the same standalone specification and built TradeRace from an empty folder. All three looked strong at first.
Every submission passed its own tests, go vet, its untouched race suite, and all 14/14 common judge behaviors. The ranking changed only when the independent judge added architecture-specific adversarial probes.
Codex xhigh won with 94/100 because it was the only submission with no critical score cap, no proven production race, bounded recovery, and an atomic venue-side idempotency commit.
A timeout after execution is more dangerous than a clean failure
TradeRace is a fictional, isolated Go trading engine created only for this benchmark. It is not Algonney production code and it never touches a real exchange.
The engine submits a logical trade such as BUY 0.10 BTCUSDT. The simulated exchange can execute it successfully and then lose the response. Retry and reconciliation may also race each other.
The core invariant was fixed before any model ran:
One logical trade intent must produce at most one economic execution — while two genuinely separate trade intents must still execute independently.
The agents were not allowed to “solve” the problem by disabling retries, removing reconciliation, globally serializing the application, blocking all repeated BTC trades, swallowing errors, or hard-coding the visible scenario.
Same task, three independently built systems
0xAlpha
Anonymous submission A in the judge report.
Codex xhigh
Anonymous submission B in the judge report.
GLM-5.3
Anonymous submission C in the judge report.
The judge did not trust self-reported PASS
The three submissions were evaluated anonymously as Candidate A, B, and C, and the original folders were treated as immutable.
The judge made work copies, ran each candidate’s own suite, go vet ./..., go test -race ./... -count=1, shuffled tests, then added equivalent judge-only behavioral tests adapted to each API.
After evaluation, every original file was SHA-256 compared with its work-copy counterpart. All three reported ALL_ORIGINAL_FILES_BYTE_IDENTICAL_TO_WORK_COPY.
- Same benchmark specification and empty-folder start.
- 14/14 common judge behaviors passed by all three.
- 96-way same-intent recovery storms tested.
- 250-round high-contention scenario repeated 100 times per candidate.
- J12 alone covered 25,000 logical orders and 400,000 recovery calls per candidate.
- Retry/race/replay scenarios repeated 1,000 times per candidate.
- Contention groups repeated under Go’s race detector.
- Architecture-specific probes were added only to test real specification-relevant weaknesses.
The final rubric used by the independent judge
The published score below uses the rubric that was actually applied after the runs — not the earlier draft rubric.
Core correctness
Exactly-once economic execution under the shared contract, including legitimate independent trades.
Independent adversarial tests
Judge-created scenarios rather than candidate-authored tests.
Concurrency safety
Race freedom, retry/reconciliation coordination, and high-contention correctness.
Failure / recovery design
Bounded retries, uncertainty handling, useful errors, and honest recovery states.
Architecture / code quality
Clarity, scope, robustness, and unnecessary complexity.
Candidate regression tests
Coverage and quality of the tests each agent wrote itself.
Resource / lifecycle safety
No deadlocks, runaway retries, uncontrolled goroutines, or lifecycle leaks.
Simplicity
Reward for the smallest architecture that still satisfies the contract.
Codex xhigh wins 94 / 100
Raw score 79. Capped at 75 after a targeted race probe proved that Store.Get exposed store-owned mutable Fill memory.
No critical cap, no proven production race, atomic venue idempotency, bounded recovery, detached store snapshots, and the smallest robust implementation.
Raw score 77. A production race would cap it at 75; the controlling 70 cap came from automatic-ID collisions that merged separate legitimate trades.
| Category | Max | 0xAlpha | Codex xhigh | GLM-5.3 |
|---|---|---|---|---|
| Core correctness | 30 | 29 | 30 | 23 |
| Independent adversarial tests | 25 | 19 | 23 | 19 |
| Concurrency safety | 15 | 11 | 15 | 11 |
| Failure / recovery design | 10 | 5 | 8 | 9 |
| Architecture / code quality | 8 | 6 | 7 | 5 |
| Candidate regression tests | 5 | 5 | 4 | 5 |
| Resource / lifecycle safety | 4 | 2 | 4 | 4 |
| Simplicity / surgical design | 3 | 2 | 3 | 1 |
| Raw total | 100 | 79 | 94 | 77 |
| Final after cap | 100 | 75 | 94 | 70 |
The bugs that the candidate-authored tests missed
The most interesting part of this benchmark is that every candidate passed its own race suite before the judge attacked its architecture.
The table below contains every proven bug in the final report. Design risks are listed separately and are not presented as reproduced invariant failures.
| ID | Agent | Severity | Location | Proven failure |
|---|---|---|---|---|
| A-01 | 0xAlpha | HIGH | internal/store/store.go:17-22,60-67,108-122 | Shallow record copies expose store-owned mutable Fill; caller mutation corrupts the authoritative record and a legal concurrent snapshot probe races. |
| A-02 | 0xAlpha | HIGH | internal/engine/engine.go:113-119,149-189,195-208 | Recover recontacts the venue for an already-confirmed order and can turn durable success into ErrExhausted with a nil execution. |
| A-03 | 0xAlpha | HIGH | internal/engine/engine.go:195-208 | Reconciliation can prove execution while the racing submit still returns CONFIRMED + nil execution + ErrExhausted. |
| A-04 | 0xAlpha | MEDIUM | internal/engine/engine.go:123-128 | A same-ID single-flight follower ignores its own context cancellation and remains blocked behind the leader. |
| A-05 | 0xAlpha | MEDIUM | internal/exchange/faults.go:68-72 | A configured 10ms attempt timeout took about 300.7ms because the simulated transport slept unconditionally before checking context. |
| B-01 | Codex xhigh | MEDIUM | engine.go:71-78,172-175; store.go:88-98 | A plain context.DeadlineExceeded after a real venue commit is recorded as PENDING/non-uncertain unless wrapped in the candidate-specific DeliveryError. |
| C-01 | GLM-5.3 | CRITICAL | engine/engine.go:89-96,106-109 | Only 16 bits identify an engine instance. In the captured run, two independent trades collided; 447 submissions produced 446 executions. |
| C-02 | GLM-5.3 | HIGH | engine/engine.go:239-247 | NaN and positive infinity quantities pass validation, return success, and economically execute. |
| C-03 | GLM-5.3 | HIGH | store/store.go:239-277 | The order becomes visibly EXECUTED before its execution log and position accounting are published. |
| C-04 | GLM-5.3 | HIGH | store/store.go:132-154,217-231 | Store.Create publishes an entry, unlocks, then reads entry-owned state without the entry lock; the targeted race detector reported production frames. |
Three bugs worth studying
GLM-5.3 — 16-bit “fresh” identity. Randomness alone is not enough if the namespace is tiny. Every engine restarted its suffix at 000001, while the engine prefix contained only 65,536 possible values.
var b [2]byte
_, _ = crand.Read(b[:])
prefix: hex.EncodeToString(b[:])
id = fmt.Sprintf("%s-%06d", e.prefix, e.seq.Add(1))
0xAlpha — a “copy” that still owns the store’s pointer. Copying the outer struct did not copy the pointed-to Fill. The judge could mutate the supposedly detached snapshot and change the authoritative record.
type Record struct {
...
Fill *model.Fill
}
return *r, true
Codex xhigh — timeout semantics coupled to one wrapper. Its economic execution remained safe, but the state machine could say “not uncertain” after an order had actually executed.
func isUncertain(err error) bool {
var delivery *DeliveryError
return errors.As(err, &delivery) && delivery.Uncertain
}
Design risks the judge kept separate from reproduced failures
0xAlpha
Positive reconciliation worker counts are not capped; reconciliation discards individual engine recovery errors; exact keyed replay returns “already exists” instead of the known execution.
Codex xhigh
Recreating a strategy with the same caller namespace resets its counter and can repeat IDs. The judge treated this as an integration precondition because the namespace is explicitly caller-supplied.
GLM-5.3
A query error can trigger another submit without confirmed NOT_FOUND; direct exchange key replay does not validate payload equality; Engine.New accepts nil dependencies and may panic later.
Classification rule
PROVEN BUG means a judge test or race probe reproduced a concrete incorrect result. DESIGN RISK means a weakness was identified without enough evidence to score it as a separate invariant violation.
Why the shared test suite alone was not enough
All three passed the common matrix — including lost responses, repeated uncertainty, retry/reconciliation races, a 96-way recovery storm, legitimate identical-looking trades with distinct IDs, independent orders, bounded retries, goroutine stability, and persistent failure integrity.
The judge then looked for assumptions unique to each architecture. That is where 0xAlpha’s escaped snapshot, Codex’s timeout-state defect, and GLM-5.3’s identity/accounting/store defects appeared.
| Evidence | 0xAlpha | Codex xhigh | GLM-5.3 |
|---|---|---|---|
| Own tests | 14 · PASS | 12 · PASS | 17 · PASS |
go vet ./... | PASS | PASS | PASS |
Untouched go test -race | PASS | PASS | PASS |
| Common judge behaviors | 14/14 | 14/14 | 14/14 |
| Common contention / stress | PASS | PASS | PASS |
| Architecture-specific race probe | FAIL | PASS | FAIL |
| Proven bugs after targeted probes | 5 | 1 | 4 |
| Original files modified by judge | NO | NO | NO |
A narrow correctness boundary beat a more complicated protocol
Codex xhigh’s core economic commit was one atomic LoadOrStore keyed by an immutable logical intent ID.
That mattered because concurrent retry and reconciliation did not need to be prevented. They could race, and the venue’s atomic idempotency boundary still allowed only one economic value to win. Same-ID reuse with a different payload was explicitly rejected.
The store used per-order locking and deep-copied returned execution records. Retry attempts were bounded from 1–64; reconciliation workers were bounded from 1–32 and joined. No background loop, deadlock, runaway retry, leaked worker, or production race was proven.
Its one proven defect was real: a plain context.DeadlineExceeded after execution could be mislabeled PENDING/non-uncertain until reconciliation. The judge deducted points for that, but the venue ledger still remained at exactly one execution and reconciliation repaired the state.
The winning system was not perfect. It was the submission with the smallest robust design and the least dangerous remaining defect.
What 94 / 75 / 70 actually means
TradeRace is one controlled engineering challenge. It is not a universal ranking of every capability these agents or underlying models have.
The result should be read as: under this exact empty-folder Go trading-systems task, this exact judge rubric, and this exact verification process, Codex xhigh produced the strongest submission.
Model performance can change with harnesses, tool access, prompt structure, model versions, reasoning settings, runtime limits, and task domain.
How these three agent configurations performed on this exact TradeRace challenge.
A universal leaderboard for all coding, reasoning, or software-engineering tasks.
Standalone benchmark — not Algonney production code
TradeRace was designed and run as an independent Algonney benchmark.
It is not an official benchmark of the participating model providers. Product names identify the tested agent configurations only.
No real exchange accounts, credentials, customer funds, or live trading infrastructure were used.
The evidence bundle retained for this result includes TRADE_RACE_FINAL_FULL_REPORT.md, TRADE_RACE_JUDGE_REPORT.md, benchmark_results.json, and judge-only work-copy tests. The reports state that all original candidate files remained byte-identical after evaluation.
Codex xhigh wins TradeRace
- 94 / 100 · no critical cap.
- 14 / 14 common judge behaviors passed.
- Race-clean across own, common, and contention suites.
- Atomic venue idempotency and bounded recovery.
0xAlpha: 5 · Codex xhigh: 1 · GLM-5.3: 4. Common behavior passed by all; targeted probes separated the submissions.