Most security investigations start with a log. Ours started with a question typed into a chat box: "have there been any dangerous things flagged in these conversations?"
The system under review is Halyard, a live automated-trading platform. Almost none of it was written by a human at a keyboard. It was built across hundreds of sessions by two different coding agents — OpenAI's Codex and Anthropic's Claude — talking to a production box over SSH, committing directly to main, sometimes not committing at all. There is no tidy trail of pull requests, and no CI pipeline gating what ships. The record of how this software came to exist lived, until recently, inside ephemeral agent transcripts scattered across two vendors and two machines.
AgentPM had captured those transcripts in one searchable history. We used that record to reconstruct the incident and check what happened after the initial finding.
01 / Why it mattered
When agents ship to production, the code alone does not explain what happened
When coding agents can run ssh dev@prod-209 and edit a live trading system, the failure modes include leaked credentials, a "test" order reaching a funded account, or an auth change that ships to the server but never lands in the repository. These are the questions a compliance auditor may ask weeks later, after the operator's memory has faded and the agent session has ended.
Traditional tooling answers "what is the code now." It does not answer which agent changed the auth model, when, why, on whose instruction, and whether the fix actually reached both the server and the repo. In this case, those details were preserved in the conversation history.
02 / The search
One question, every conversation that touched credentials
The investigation is itself preserved in AgentPM as a conversation — goc_63101905…, "AgentPM review of thalos work, auth pivot, and PR 63." The reviewer loaded the agentpm-conversations skill and asked a deliberately broad question, then let the corpus answer. No grepping a codebase — a search across the shared history:
agent-pm-skill-lookup — conv search
$ agentpm conv search "secret credential api key" --org northwind 50 matching turns across 11 conversations goc_b6109e16… Broker-secret encryption implemented and deployed codex 773t goc_71ec51d0… Halyard P0 UX guardrails shipped and deployed codex 7212t goc_43591a3b… Execution-issue report published after log review claude 369t … $ agentpm conv find goc_63101905… "dangerous flagged" strict turn hit → turn 68 (CLAUDE, concluded) 3 red-flag findings, each with source conversation + turn range
The search returned a ranked answer with citations. The most serious finding was not in the conversation being reviewed; it appeared in a 7,212-turn Codex session from days earlier. Because the search covered both agents and both machines, the reviewer could open the cited conversation and inspect the relevant turns directly.
03 / Discovery
The god token in the browser bundle
The cited answer pointed to conversation goc_71ec51d0…, turns 646–689. We went straight there and read the exact turns where the agent named the problem.
agent-pm-skill-lookup — conv show
$ agentpm conv show --turn 655 goc_71ec51d033ff81398744ae6a CODEX · turn 655 · concluded the compiled JS literally contains the god token — a leaked public env token in the browser bundle. NEXT_PUBLIC_AUTH_TOKEN inlines the value into the browser bundle, so every visitor effectively carries the god API key.
The production front end carried NEXT_PUBLIC_AUTH_TOKEN in its .env.local. Because the NEXT_PUBLIC_ prefix inlines a value into the browser bundle, the admin token shipped to every visitor. Anyone who loaded the site received the bearer credential used for administrative API access to the live trading backend.
04 / Verify
Trust nothing — verify the fix against later turns
Was the remediation real, or overwritten later in a 7,000-turn thread? We didn't read 7,000 turns. We asked AgentPM to jump to the tail and to the mutating git operations only.
agent-pm-skill-lookup — verify
$ agentpm conv show --around 999999 --limit 10 goc_71ec51d0… # thread has grown to ~3597; fix commits land at turns 3288, 3445 # hard reset + redeploy at turn 3471 $ agentpm conv git goc_71ec51d0… --mutating-only …commit / push events, most recent first… $ agentpm conv find goc_71ec51d0… "verified bundle no token admin gated" 1 passage hit — all terms within 5 turns → [3581–3586]
The record shows what was attempted and claimed through the latest turns — not the live deployed state. The local halyard checkout wasn't even a git repo (fatal: not a git repository); the real code lived on the host. So "resolved in the transcript" was flagged as not independently verified against production.
05 / The continuity beat
"Deployed" is not "committed"
A day after the hardening shipped, the operator asked: "k is it deployed and in main." The agent checked the deployed state and the repository state separately.
halyard — exec (git inspection, remote, read-only)
dev@halyard$ ssh dev@prod-209 'git fetch origin main \ && git rev-parse --short HEAD origin/main \ && git status --short -- apps/api-v2/app/auth.py ...' branch=main head=ba9a213 origin_main=ba9a213 status_auth_files: M apps/api-v2/app/auth.py M apps/api-v2/app/routers/admin.py M apps/api-v2/tests/test_broker_secret_storage.py
Then a live behavior probe against the running API:
halyard — exec (inline probe, remote)
dev@halyard$ ssh dev@prod-209 'curl -s -o /dev/null -w "%{http_code}" http://127.0.0.1:8001/ ; ...' api_root=200 admin_chat_unauth=401 session_expiry_constant=1209600 expired_session_rejected=True forged_clerk_rejected=True
"Deployed: yes. In main: no."— Codex, goc_b6109e16… · turn 750
HEAD equalled origin/main at ba9a213, so a casual glance said "in sync." But the actual security changes — encrypted broker-secret storage, the auth and ownership fixes — were sitting as uncommitted working-tree edits on the live server. The protection was running in production and existed nowhere in the repository. One git checkout . or a fresh clone would have reverted the hardening. The operator then asked the agent to commit the changes. At turn 777: "Done. It's now deployed and in main."
06 / The evidence, assembled
Five findings, every one anchored to a conversation and a turn
| Finding | Source | Status on record |
|---|---|---|
| God API token in the browser bundle NEXT_PUBLIC_AUTH_TOKEN inlined; every visitor held admin access |
goc_71ec… · t646–689 remediated in goc_b610… · t282–365 |
Closed on live host token rotated, bundle clean, browser-origin auth blocked |
| Security fixes deployed but not committed encrypted storage + auth fixes as uncommitted edits on the host |
goc_b610… · t744–750 | Committed resolved at t777 after being caught |
| Test account → $1.14M funded broker account test-account resolved to 31 open live positions |
goc_71ec… keyframe | Reset unconfirmed flagged for production re-check |
| Broker secrets stored as plaintext TEXT daily_db.py:127 — storage contradicted the "never returned" copy |
goc_b610… · t33 | Encrypted implemented + deployed t34–281 |
| Single global bearer token (root cause) one operator surface protected by one global bearer token |
goc_71ec… · t162 | Migration in progress motivated god/user role split |
07 / Why traditional tooling would have struggled
Each conventional source covered only part of the incident
What the usual tools would show
- Git / PR review: most work went straight to
mainover SSH — and the security fixes weren't even committed. The audit trail was empty exactly where it mattered. - CI/CD logs: there is no gating pipeline. Nothing to review.
- A production snapshot: would show the fixes present and declare the platform safe — missing that they existed nowhere in the repo.
- Vendor chat logs: split across Codex and Claude, two products, two machines, no shared search, pruned over time.
- The leak itself: left no error. The $1.14M mapping left no error. Logs record failures, not dangerous successes.
What AgentPM showed
- One corpus across both agents and both machines, searchable by keyword, file, PR, or branch.
- Every claim anchored to a conversation id + turn you can open and read.
- Git facets (
conv git --mutating-only) to jump to just the commits and pushes in a 7,000-turn thread. - The distinction between "deployed" and "in main" preserved as first-class evidence.
- Dangerous successes retained as narrative, not discarded as non-errors.
08 / How AgentPM made it straightforward
The investigation used five read-only commands
No new instrumentation, no log shipping, no code changes. The reviewer worked entirely against history AgentPM had already captured, using a read-only CLI over the org's own record:
the investigation, in five read-only commands
# 1 — find every conversation that touched credentials $ agentpm conv search "secret credential api key" # 2 — surface the flagged dangers, with citations $ agentpm conv find goc_63101905… "dangerous flagged" # 3 — read the exact turns where the leak was found $ agentpm conv show --turn 655 goc_71ec51d0… # 4 — jump to just the commits/pushes in a 7,212-turn thread $ agentpm conv git goc_71ec51d0… --mutating-only # 5 — confirm the hardening reached server AND repo $ agentpm conv show --around 750 --limit 20 goc_b6109e16… Deployed: yes. In main: no. → then committed at turn 777