What happens when you let AI build your stock research system (Part 2)
Part 2 of five. In Part 1 I confessed that most of this system's code wasn't written by me. This is what that actually looks like day to day — what worked, what didn't, and the rules we had to invent along the way.
The division of labor: I decide, AI builds
The main developer on this project is an AI agent (Claude Code). It runs on my PC, reads and writes files, writes code, runs tests, and repairs the database.
My own job boils down to three things:
- Put what I want into words — "every day, split the decliners into healthy dips and dangerous drops"
- Decide at the forks — judgment calls like "precision or explainability, which do we take?"
- Doubt the output — check the numbers against real data before believing anything
Everything else — the implementation details — is delegated. But getting to the point where I could delegate took a few hard-won rules.
Rule 1: use two AIs
This project runs two AIs. The first designs and implements. The second does exactly one thing: hunt for bugs in the first one's diffs. It reviews every change and comes back with things like "this branch breaks at the boundary value."
This works far better than it has any right to. It's the same reason human teams do peer review: the author — human or AI — cannot see their own blind spots. Real bugs that would have shipped to paying readers have been caught this way, more than once.
There was a failure mode, though. Early on I consulted the second AI about everything — design, strategy, naming. And I noticed it kept giving me the same generic advice: "mind your data quality," "write tests." Thanks, heard that one. The cause: every consultation started from zero context. The fix: narrow its role to reviewing implementation diffs only — no design brainstorming. AIs, it turns out, have a natural grain-size of work they're good at. You only learn it by operating one.
Rule 2: exactly one writer at a time
Two AIs touching the same files invites disaster, so we made a "baton". A file at the top of the project states who currently holds write permission. No baton, read only. Check the baton before you start; hand it back when you finish.
Primitive? Absolutely. Primitive rules are the ones that don't get broken. Human organizations discovered this long ago.
Rule 3: health check before any work
Before the AI starts a task, it must run a one-command health check: did last night's jobs succeed, what failed recently, how fresh is the data. Skip this, and it will eventually mis-diagnose "a thing that broke yesterday" as "a thing my change broke today" — and sprint off fixing the wrong thing.
AI is capable, but it also jumps to conclusions at superhuman speed. The pre-work ritual matters.
What I learned about giving instructions
The biggest lesson: vague expectations produce vague deliverables. Ask for "a nice weekly report" and you'll get something nice-ish. What actually worked:
- State acceptance criteria first — "no broken numbers may ever reach paying readers; when in doubt, leave it out"
- Write down what's forbidden — "never edit production data directly", "no spec changes without tests"
- Ask for verification along with the feature — "when it's built, show me the procedure to confirm it against real data"
You may notice this is exactly how you'd brief a human colleague. The skill that improved most through AI collaboration was probably my own ability to delegate.
What remains human
After months of this, two jobs have stubbornly remained on my side of the desk.
One is noticing that something feels wrong. A while back, a generated weekly report was about to say a blue-chip manufacturer fell -91% in one week. The AI's arithmetic was correct — the data was lying (that story gets its own chapter in Part 4). But the common-sense brake — "a company that size doesn't just do that" — still lives in the human.
The other is responsibility. What gets published, what people are charged for, who apologizes when it's wrong. That can't be delegated to AI, and I don't think it should be.
Next in Part 3: the heart of the system — why the scorecard that records our misses matters more than anything that records our wins.
Tsukiyo Research publishes a weekly systematic report on Japan's uncovered small caps — every issue opens with the scorecard of our past calls. Subscribe here.
This series is a personal build-and-operate log. Nothing here is investment advice or a recommendation to buy or sell any security.
