Everything you need to understand Kestrel Signal: how to build and test strategies, what every metric means, and how to tell a real edge from statistical noise.
Kestrel Signal lets you backtest trading strategies and immediately see whether the results are statistically meaningful — not just profitable on historical data. Here's the typical workflow:
Free tier includes DSR and PSR — statistical validity metrics are never paywalled. You can tell if a strategy has real edge on a free account.
The DSR is Kestrel Signal's primary validity signal. It adjusts the Sharpe Ratio downward based on three sources of statistical bias:
| DSR | Interpretation | Action |
|---|---|---|
| ≥ 0.95 | Very strong evidence of a real edge | Proceed to walk-forward validation |
| 0.80 – 0.95 | Good signal; worth further investigation | Walk-forward, check out-of-sample decay |
| 0.50 – 0.80 | Marginal — may be noise or mild overfitting | Run CPCV before trusting results |
| < 0.50 | Statistically insignificant — likely noise | Redesign the strategy; do not trade live |
The DSR tier displayed on each backtest ("Strong", "Good", "Marginal", "Weak") maps to these thresholds. Published strategies show DSR prominently in the gallery.
The PSR answers a simpler question: What is the probability that the true Sharpe ratio of this strategy is greater than a benchmark Sharpe (typically 0)?
It is derived from the same statistical framework as the DSR but expressed as a probability (0–1). A PSR of 0.95 means there's a 95% probability that the strategy has a positive Sharpe in a new, unseen period — assuming stationarity.
| PSR | Interpretation |
|---|---|
| ≥ 0.95 | High confidence in a real positive edge |
| 0.80 – 0.95 | Reasonably confident; worth further testing |
| 0.50 – 0.80 | Uncertain — could be random chance |
| < 0.50 | Coin-flip territory; no meaningful evidence of edge |
Like the Sharpe Ratio, but only penalises downside volatility (returns below a target, usually 0%). Upside volatility is not penalised — which is more intuitive for strategies with positive skew.
A Sortino Ratio significantly higher than the Sharpe Ratio suggests the strategy has positively-skewed returns — good losses are rare and wins are frequent. This is typical of trend-following and momentum strategies.
Measures annualised return relative to maximum drawdown. Useful for assessing whether the strategy's gains justify its worst-case losses.
| Calmar | Quality signal |
|---|---|
| ≥ 3.0 | Excellent risk-adjusted performance |
| 1.0 – 3.0 | Good |
| 0.5 – 1.0 | Mediocre — drawdowns are too deep relative to gains |
| < 0.5 | Poor — risk not compensated |
The largest peak-to-trough decline in portfolio value during the backtest period, expressed as a percentage. Max drawdown is crucial for position sizing and psychological sustainability — can you hold through a 40% decline?
| Max Drawdown | Typical tolerance |
|---|---|
| < 10% | Very conservative; suitable for most capital |
| 10% – 20% | Moderate; sustainable for most traders |
| 20% – 35% | High; requires strong conviction and sizing discipline |
| > 35% | Extreme; most retail traders will abandon mid-drawdown |
Note: Drawdowns in live trading often exceed backtest drawdowns due to slippage, execution latency, and market regime changes.
The percentage of closed trades that are profitable. Win rate alone tells you very little — a strategy can be highly profitable with a 30% win rate (if winners are large) or be losing money despite a 70% win rate (if losers are large).
The relevant metric is profit factor (gross profit / gross loss) orexpected value per trade = (win rate × avg win) − (loss rate × avg loss). A positive expected value is what matters, not win rate alone.
Total Return is the raw percentage gain from initial capital over the entire backtest period. CAGR (Compound Annual Growth Rate) normalises this to an annualised percentage, making strategies with different test durations comparable.
High CAGR with high drawdown is common in backtests but unsustainable live. Aim for CAGR / Max Drawdown (Calmar) ≥ 1.0 as a minimum threshold.
There is no single threshold for a "good" strategy — it depends on your capital, trading frequency, and risk tolerance. But here are the minimum bars we recommend before taking a backtest seriously:
| Metric | Minimum | Target | Exceptional |
|---|---|---|---|
| DSR | ≥ 0.50 | ≥ 0.80 | ≥ 0.95 |
| PSR | ≥ 0.60 | ≥ 0.80 | ≥ 0.95 |
| Sharpe Ratio | ≥ 0.5 | ≥ 1.0 | ≥ 2.0 |
| Sortino Ratio | ≥ 0.7 | ≥ 1.5 | ≥ 3.0 |
| Calmar Ratio | ≥ 0.5 | ≥ 1.0 | ≥ 3.0 |
| Max Drawdown | ≤ 35% | ≤ 20% | ≤ 10% |
| Sample size (bars) | ≥ 500 bars | ≥ 1,000 bars | ≥ 2,500 bars |
| # Trades | ≥ 30 trades | ≥ 100 trades | ≥ 500 trades |
Small sample warning: Kestrel Signal requires at least 50 data rows for upload and flags strategies with fewer than 30 trades. Short backtests inflate all metrics — a 3-month backtest with 5 trades proves nothing.
Walk-forward validation splits your data into sequential in-sample (IS) and out-of-sample (OOS) windows to test whether a strategy's parameters generalise to unseen data.
Two modes are supported (Researcher tier and above):
The key metric is the Walk-Forward Efficiency (WFE):
| WFE | Meaning |
|---|---|
| ≥ 0.70 | OOS performance retains ≥70% of IS — very good generalisation |
| 0.50 – 0.70 | Moderate decay — investigate parameter sensitivity |
| < 0.50 | Heavy OOS decay — the strategy is likely overfit |
| Negative | Strategy loses money OOS — clear overfitting signal |
CPCV (Lab tier and above) is the gold standard for backtest validation. It generates every possible combination of training and test splits from your data, producing a full distribution of out-of-sample Sharpe ratios rather than a single number.
The most important output is the Probability of Backtest Overfitting (PBO) — the fraction of CPCV splits where the OOS Sharpe is worse than a benchmark (usually 0).
| PBO | Meaning |
|---|---|
| < 0.05 | Very low overfit probability — strong evidence of real edge |
| 0.05 – 0.15 | Low-moderate overfit — proceed cautiously |
| 0.15 – 0.40 | Moderate overfit — reduce parameters, extend data |
| ≥ 0.40 | High probability of overfitting — do not trade live |
CPCV also produces a median OOS Sharpe and a 5th percentile Sharpe — your worst-case expected performance. Size positions to survive the 5th percentile, not the median.
The JSON DSL is a declarative strategy format that defines indicators, entry conditions, and exit conditions. The visual builder generates valid DSL automatically.
A minimal DSL strategy looks like this:
Lab and Pro tier users can upload Python strategy files that run in a sandboxed Firecracker microVM. The sandbox enforces strict resource limits and static analysis (Bandit + Semgrep) before execution. Network access, filesystem writes, and subprocess calls are all blocked.
Q: Why is my DSR lower than my Sharpe Ratio?
A: The DSR deflates the Sharpe Ratio based on how many trials you ran, how short your backtest is, and how non-normal your returns are. A Sharpe of 2.0 from a 3-month backtest with 50 parameter combinations will produce a very low DSR — that's correct behaviour, not a bug.
Q: What symbol/ticker data is available?
A: We provide 5 years of synthetic OHLCV data for 50 instruments covering equities, crypto, and FX. The data is generated from a calibrated GBM model with realistic volatility clustering. Pro tier users can upload their own CSV data (max 10,000 rows per file, 5 MB).
Q: Are the backtest results deterministic?
A: Yes. Every backtest is identified by a SHA-256 hash derived from the strategy version ID, data snapshot ID, engine version, and configuration. Identical inputs always produce byte-identical results. The hash is shown on every result page.
Q: What does 'equity curve' show?
A: The equity curve is the portfolio value over time, starting at your initial capital. Each point represents the portfolio value at the close of that bar. Drawdown is the percentage decline from any peak in this curve.
Q: How is commission/slippage applied?
A: Commission is applied as a percentage of trade value on entry and exit. Slippage is applied as a percentage of price on fills. Both are applied to each trade individually. Default values are 0.1% commission and 0.05% slippage — conservative but realistic for retail traders on liquid instruments.
Q: Can I trade live from this platform?
A: No. Kestrel Signal is a research and backtesting platform only. It does not connect to any broker or exchange, and cannot execute live orders. This is intentional — we're software, not an advisory or execution service.
Q: How do I upload custom price data?
A: Pro tier users can upload CSV files via the Data page. The CSV must include columns: date (YYYY-MM-DD), open, high, low, close, volume. Prices must be positive; volume must be non-negative. OHLC ordering violations (e.g., low > high) are automatically skipped. Max 10,000 rows.
Q: What happens to my account if I cancel my subscription?
A: You drop to the free tier immediately (or at end of billing period, depending on your Paddle subscription settings). Your strategies, backtest history, and exported data are retained. Custom data uploads that exceed free tier limits will no longer be accessible for new backtests, but the data itself is not deleted.
Something unclear? Found a mistake?
Use the feedback button (bottom-right) to send us a suggestion or correction. We update this documentation regularly.