My trading bot lost $20.83 and taught me statistics
I spent a chunk of this summer building a bot that trades 5-minute crypto prediction markets — the "will BTC be higher in five minutes" kind. It streams the order book and live prices, watches for a specific pattern near the end of each window, and buys when it appears. It is genuinely well-built software. It also lost money, and the way it lost money taught me more than any winning trade would have.
This post isn't trading advice. If anything it's the opposite — it's about how easy it is to fool yourself with a backtest, and how the fix is boring old statistics.
The seductive backtest
My first live strategy lost $20.83 over 27 trades at a 48% win rate. Fine — first attempts lose. So I did what everyone does: I downloaded historical data, replayed thousands of past market windows, and searched for a pattern that would have made money. I found one. Buying the favorite side in a certain price zone during the last minute showed +3.45 cents per trade over 167 simulated trades.
Positive expectancy! Deploy it! Except I made myself do one more calculation first, and I'm glad I did.
The standard error ruins everything
Each of these trades has roughly two outcomes: win about +28 cents, or lose about −72 cents. Outcomes that lumpy have huge variance. The standard error of the average over 167 trades works out to about ±3.4 cents per trade.
Read those two numbers together: measured edge +3.45c, standard error ±3.4c. My "edge" was one standard error above zero. In plain words: totally consistent with there being no edge at all. If you ran the same experiment on pure coin flips, you'd see a result this good quite often just by luck.
And there's a second trap stacked on the first: I had searched many price zones and time windows and kept the best-looking one. Searching a grid and keeping the winner practically guarantees you find something that looks profitable in the past. The overfitting signature is easy to check and nobody checks it: if your winning cell is real, its neighbours should look decent too. If the cells right next to it all lose, you found a lucky square, not a law of nature.
The other killer: it barely fired
Even pretending the edge was real, the bot went entire evenings without a single trigger. A 3-cent edge on $4 positions a couple of times a day is pennies. Meanwhile my exchange websocket kept dropping with keepalive timeouts, my trade ledger lived in memory and vanished on every restart, and the whole thing ran on a Windows laptop that sleeps. The unglamorous operational stuff — reconnects, persistence, an always-on server — matters more than one more clever signal.
What I actually do differently now
- Big samples before money. The plan is a backtest over ~150,000 historical windows — enough trades to tell 3 cents from zero — with the data cached locally so every future idea tests in seconds instead of weeks of paper trading.
- Pick parameters on one time period, validate on another. If the edge only exists on the data it was mined from, it isn't an edge, it's a memory.
- Count execution costs as part of the strategy. Crossing the spread to buy a ~70-cent favorite means you need to win more than 72% of the time just to break even. Resting orders instead of taking can shift breakeven by more than my entire supposed edge. The execution style was the trade, and I'd been treating it as a detail.
- Paper trade with honest fills. A simulator that assumes you always get filled at the price you saw is a flattery machine.
Why I'm still glad
$20.83 is the cheapest statistics course ever sold. The same lesson at real position sizes costs people their savings — the pattern-mine-a-backtest pipeline is exactly how most retail algo traders lose, and you can't feel how convincing the illusion is until your own dashboard shows you a beautiful green number that means nothing.
The bot still runs, in paper mode, collecting data. If the big backtest ever says the edge is real, I'll write the follow-up. If it says noise — honestly, that post might be more useful.