How Three Models Reach Consensus in an AI Recommendation Pipeline

What matters most in an AI pipeline that recommends stocks isn't any single model's performance — it's how three models with different perspectives arrive at consensus

This is the English version of a post originally written in Korean for my algorithmic trading system devlog(new tab).

Separate from the trading bot connected to my real account, my quant bot has a pipeline that analyzes stocks with several AI models and sends out a reference report. It doesn't trade real money automatically yet — it just keeps a running record of whether the recommendations turned out to be right.

What matters most in this pipeline isn't any individual model. It's the way multiple models' judgments get bundled into consensus. Let me walk through that.

Who's sitting at the table

I'll start with something familiar to anyone who's spent time around markets — good judgment more often comes from consensus among several experts with different perspectives than from a single genius.

I seated three AI models at the table, each looking at the market through a completely different lens.

• The quant seat (LightGBM) — reads statistical regularities out of years of price, volume, and financial data.
• The trend seat (Transformer) — reads trends and regime shifts as the market moves through time.
• The research seat (TradingAgents) — reads news, disclosures, and fundamentals the way a person would, and writes a reasoned report. It's a multi-agent setup running open-source models on local GPU hardware.

For a rough sense of scale on that GPU hardware: one card, in practice, feels roughly on par with the old free tier of ChatGPT (GPT-3.5). Split across two cards, it gets closer to each company's lighter free tier — something like GPT-4o mini or Gemini Flash.

(There used to be a fourth seat that trained a trading policy directly via reinforcement learning, but it never beat the benchmark, so I retired it without much regret.)

Why combine them at all

The quant seat misses the narrative behind a sudden news event. The research seat misses subtle order-flow signals. Combine independent perspectives, and their individual errors tend to cancel out.

There's a mathematical result called Condorcet's Jury Theorem. It says that if you pool enough independent judges who are each, on average, even slightly better than random, the group's accuracy surpasses any individual judge's. The key isn't "more opinions" — it's "opinions that actually differ." I checked, and the three seats' judgments turn out to diverge fairly independently in practice.

Recently I revisited this independence question, prompted by a natural follow-up: "isn't the trend the trend-seat reads just momentum?" It's still an early check, but the conclusion held, and I wrote it up separately in this post(new tab).

How the consensus works — order, not volume

This part is surprisingly more humanities than math. I don't naively average scores. Do that, and one seat shouting "100% sure!" about a stock would dominate the whole board.

Instead, each seat only submits a ranking. What gets counted isn't confidence, it's ordering — who each seat put on top. It respects the order of judgment, not the loudest voice (the same principle behind rank aggregation in preferential voting).

• Weight is kept nearly equal across all three seats, so no single one gets over-trusted. It's the same insight well known in asset allocation — beating simple equal weighting is harder than it sounds.
• If a seat stays silent on a given stock (say, it lacks the data to form a view), that counts as a neutral abstention, not a vote against.

The design philosophy — humility

This is the part I think matters most. The consensus mechanism is deliberately kept simple.

The edge here isn't in some clever aggregation formula — it's in actually having seats with genuinely different, independent perspectives. There's always a temptation to make the aggregation logic more sophisticated, but I've repeatedly seen that temptation lead straight into overfitting historical data.

So when I want to improve things, I put the effort into diversifying the seats at the table or sharpening each seat's individual skill, rather than the consensus mechanism itself. It boils down to one line: diverse inputs, and a robust, simple consensus.

The models are still a work in progress

I've also run a formal head-to-head comparison of several local LLMs under identical conditions. More than once I had to dig straight into the raw logs to catch problems that surface-level grade distributions couldn't reveal — final-answer generation quietly failing and falling back to a default, low-probability hallucinations where a completely different stock got analyzed, and a stretch where tool calls simply weren't going through so reports came out with no real grounding at all.

That was a comparison among local models. It also raised a separate question afterward — between models running locally for free and paid external API models, which actually performs better by real trading outcomes. I built a separate simulator to answer that question, and wrote it up in its own post(new tab).

More recently I also reworked the serving engine running this framework and how material gets handed to each seat, which cut the time it takes to run through a single stock quite a bit.

Looking back, the thing I put the most effort into in this pipeline wasn't which model to use — it was how to respect and combine different opinions. If you're curious about the overall project structure, check out the project architecture post(new tab) too.