[Jul 8] Trusted a Summarizer's Answer, and an Uncancelled Order Sat on a Live Account

Building a track to safely verify a new auto-invest feature, plus an incident from guessing at an API

This is the English version of a post originally written in Korean for my algorithmic trading system devlog(new tab).

Today's main task was building a framework to safely verify a new feature, and two incidents happened along the way.

Verifying a new auto-invest feature at zero live-trading risk

I had an idea for extending the existing signal-strength-based auto-invest feature, and before putting real money behind it, I first built a structure to verify it for a long stretch on a paper (simulated) track.

• Built a ledger completely separate from the real account, running two tracks side by side that pick the same candidate tickers but differ only in allocation method — an A/B comparison to see which one actually performs better.
• Set the judgment criteria up front: only if it beats the benchmark with statistical significance over a sufficiently long period (months' worth of trading days) does expanding real capital get considered; otherwise it's automatically discarded. Rather than "build it now, decide later," the judgment button gets defined first and runs from day one.
• While building this I found something interesting: one of the inputs the current allocation method relies on had never actually been recorded, so there was no way to backtest it against historical data. I started logging it as of today — another reminder that even a good idea is just a feeling without data to verify it.
• Also attached a dashboard that continuously measures execution slippage (the gap between the price at decision time and the price actually filled).

Incident 1: A test wrote fake data into the real paper-tracking ledger

While testing the ledger code above, I ran into a classic Python trap. A function's default save-path value gets fixed at the moment the function is defined, so what I thought was a "swap in a temp path" during testing wasn't actually taking effect. As a result, the test wrote fake data into the real paper-tracking ledger. Fortunately the live-trading ledger was never touched, so real trading data was safe — I just cleaned up the fake entries that got written in.

Incident 2: Guessed at an API, and an uncancelled order sat on a live account

This happened while building and verifying a safeguard that auto-cancels orders that stay unfilled too long. When checking what the brokerage API's "cancel order" endpoint was, a summarization tool confidently answered with a specific approach based on common REST API conventions. I trusted that and implemented it, then placed a small test order at a price safe enough that it could never fill and tried to cancel it — and got an "API not found" error. Wrong endpoint.

In the meantime, an uncancelled order sat on the live account for a few minutes. Because I'd already designed the test as "a price that can never fill, plus a same-day auto-expiring order," there was no actual damage, but trusting a summarization tool's plausible-sounding answer was a hard lesson. Only after parsing the real API spec document directly did I get the correct endpoint and a working cancel.

Lesson: For any API that touches a live account, never trust a summarization tool's word — always check the original spec directly. Same lesson as yesterday's "can't trust model documentation," but one step further this time: summarization tools can also confidently get things wrong.

Also today

• The safeguard above (auto-cancel for long-unfilled orders) was turned on for real after enough verification.
• After a recent computer restart, I found that some core background processes had been running as child processes of the dev-environment (editor) process, so they'd quietly die whenever the editor died. That day, the morning report nearly got delayed. All four core processes have now been promoted to OS-level services, so they keep running independently even if the editor dies.

Neither incident today led to real damage, but both were "saved by a safeguard already in place," which was unsettling. The verification track for the new feature has now started its day-one clock — from here it's just watching the results.