"[Jul 14] Swapping the Case Unstuck a Backlog of Deferred Work"
I finally got the second GPU properly seated, and along the way found that two of the most important tickers had been missing from data for weeks
This is the English version of a post originally written in Korean for my algorithmic trading system devlog(new tab).
Today I finished the job I'd deferred a few days ago because of a case-fit issue, and along the way found a problem that had been quietly buried for quite a while.
The temporarily-running second GPU finally got seated properly
A few days ago, when I picked up a second graphics card, the slot for two cards side by side collided with the power supply's mounting spot by a few centimeters, so I couldn't fully install it in the case. Even so, from Sunday I had both cards connected and running in a makeshift setup.
Today I swapped in a case that puts the power supply in its own separate compartment, and finally got it properly seated in place.
I ran a load test with power moderately capped. Both cards held their temperatures well (staying around 68°C under load), and ran stably with no slowdowns or errors. The machine rebooted twice during the work (one of them unintended), and the background services all came back up on their own each time. As a side note, I'd once suspected "the GPU fan sounds off" — looking again today, that turned out to be a case fan, not the GPU. Wrong suspect all along.
Two of the most important tickers had been missing from data for weeks
Keeping an eye on the old-news collection job I mentioned a few days ago, I found something fairly serious. Tickers without data yet are shown as a placeholder in the form of "[No info] CompanyName(TickerCode)," and the name-extraction logic couldn't parse this format properly.
As a result, the names for two of the most frequently mentioned, effectively most important large-cap tickers stayed blank the whole time. With the name blank, no matter how many articles mentioning that ticker got collected, the system couldn't connect them to "this is about that ticker."
Other, less important tickers piled up thousands of records fine, while the two biggest names sat at nearly zero for weeks straight. It was more unsettling that this happened to the most important tickers rather than some obscure one.
Fortunately the raw data (the article headlines themselves) had already been collected in full, so once I fixed the name-recognition bug, I could re-match against the existing collected data and recover tens of thousands of records almost instantly. Not having to start collection over from scratch was the saving grace.
A bug that mistook a normal article for being "blocked"
The collector sometimes judged "this page looks blocked" and skipped an entire day's worth of collection. Looking into it, one of the criteria for detecting a block was checking whether a certain phrase appeared on the page.
It just so happened that one IT article's body coincidentally contained the exact same phrase (something like "authentication procedure ~"), so the collector mistook a completely normal page for a block — even though the article was perfectly visible.
The fix was simple: even if that phrase is present, never treat the page as blocked if it also contains a normal-looking article list. It was a case where a coincidental word match nearly wiped out an entire otherwise-fine day.
Also today
- I found that requesting too many tickers at once from an external API caused some requests to silently fail. The large-cap tickers with the most news kept getting cut off at the tail end of the batch every time; reducing the concurrent request count eliminated the loss entirely.
- I also adjusted the start time of the nightly automated analysis. Instead of running right after market close, I pushed it back to after after-hours trading settles, so it starts on cleaner data. I also switched the price reference point to the regular-session closing price instead of the after-hours price to match.
- The ongoing comparison between the newly added model and the existing one can now continue as well, now that today's data gap is filled. It's too early to tell which is better; I'll need a few more days of data.
Today was one of those days where fixing a physical problem (the case) opened the door to a string of data problems coming into view right after. Seeing the pattern of "the most important thing quietly goes missing first" repeat itself, I think going forward I need to check not just whether totals are growing, but whether the important tickers are getting covered evenly.