The Edge That Failed, Then Passed
I built a falsification engine to test crypto trading edges, then pointed it at the most promising one I could find. It killed my edge. Then more data brought it back — but not the way I expected. The anatomy of honestly validating a strategy, with the real numbers.
The setup
I built a paper-first research framework whose default verdict for any "edge" is REJECT. A strategy only becomes a candidate after it survives fees, slippage, random-baseline comparison, parameter-plateau checks, and — the one that matters most — walk-forward out-of-sample validation.
The most promising market-neutral idea was funding-rate carry: hold spot long and perpetual short, delta-neutral, and collect the funding that leveraged longs pay. Structural, low-risk, scalable. The research said it should be real.
Act 1 — it looked like a winner
On one year of real Binance funding data across ten liquid pairs, a hold-and-collect implementation netted +2.75%/year, market-neutral. Tiny, but positive and low-risk. The gauntlet marked it a CANDIDATE. I was ready to call it.
Act 2 — walk-forward killed it
Then I did the thing most "backtest" screenshots never show. I split the year into folds, tuned the parameters on each training slice, and tested them on the unseen slice that followed. The result:
The +2.75% was in-sample overfitting — I had chosen the parameters (hold length, number of names) on the very same year I then reported. On data it hadn't seen, the edge evaporated. A single-period backtest had lied to me, exactly the way it lies to everyone who trusts one.
That alone is the lesson most people pay to learn the hard way, with real money: one backtest is not evidence. Walk-forward is mandatory. My own engine caught my own overfit and stopped me.
Act 3 — more data, an honest reversal
A one-year window is thin, and 2025–2026 happened to be a low-funding bear market. So I pulled three years (2023–2026, including the 2024 bull) and re-ran the same walk-forward:
The yearly breakdown explains everything:
| Year | Annualized funding |
|---|---|
| 2023 | +7.2% |
| 2024 | +11.6% (bull) |
| 2025 | +3.1% |
| 2026 | +0.5% (bear) |
The edge is real and robust across a full cycle — but strongly regime-dependent. It harvests 5–12% when the market is bullish and longs are paying up, and roughly nothing in a flat/bear market. My one-year test had failed only because it caught the weakest regime in isolation.
The meta-lesson
Honest validation cuts both ways. It killed a false positive (the overfit +2.75%) and rescued a true edge that a single bad year had hidden. Most strategy backtests do neither — they show one flattering period and call it alpha.
The uncomfortable, useful truth about this edge for a small solo operator:
- It is real, low-risk, and market-neutral — and regime-dependent and thin (a fraction of a percent in the current regime; the good years need a bull market).
- At small capital, fees and minimum-notional drag eat most of it; it becomes worthwhile with size and maker fills.
- Honest paper track record over the recent (weak) 90 days: +1.2% annualized, 0.24% max drawdown, positive — safe, but not a salary.
Why this matters if you're paying for a bot
If someone sells you a "profitable trading bot," ask one question: show me the walk-forward, across market regimes, net of fees. If they can't — or if they only have one good backtest — you're looking at an overfit, not an edge. The discipline to kill your own best idea when the data says so is the whole job. It's also the rarest thing in this space.