backtesting-frameworks
- Use it when:
- You're building a backtester, or checking whether a trading strategy's historical results can be trusted.
- It does:
- Names the biases that inflate backtests (look-ahead, survivorship, overfitting, cherry-picking, ignored trading costs) and how to avoid each.
- You get:
- Backtests that split history into training, validation and test sets and test walk-forward over rolling windows.
- Also has:
- Python for an event-driven and a vectorized backtester, walk-forward optimization, Monte Carlo analysis and performance metrics.
TL;DR
A reference for trading-strategy backtests. Each bias comes with its fix: point-in-time data against look-ahead, delisted securities against survivorship, out-of-sample testing against overfitting, pre-registration against cherry-picking strategies, and realistic cost models for trading costs. History is split three ways: develop on the training set, choose parameters on the validation set without peeking, and touch the test set only for the final estimate; walk-forward analysis repeats that over rolling windows. Its rules: limit parameters, never optimize on the full history, handle adjusted prices with care, use Monte Carlo analysis to understand uncertainty, and account for capacity and market impact. A separate file holds the implementations in Python with pandas and NumPy: an event-driven backtester, a faster vectorized one, walk-forward optimization, Monte Carlo analysis, and a function computing returns, volatility, Sharpe, Sortino and Calmar ratios, maximum drawdown, win rate and profit factor.
Summarizes the skill at 4bde316.
npx skills add https://github.com/wshobson/agents --skill backtesting-frameworks skills.sh pageSource on GitHubAll skills from wshobson/agents
- Total Downloads
- 17,500 on skills.sh, Nov 9, 2026
- Repo stars
- 42,334 wshobson/agents
- First seen
- Jan 20, 2026 on skills.sh
Weekly values
| Week to | Installs |
|---|---|
| Sep 21 | 239 |
| Sep 28 | 276 |
| Oct 5 | 373 |
| Oct 12 | 336 |
| Oct 19 | 336 |
| Oct 26 | 381 |
| Nov 2 | 354 |
| Nov 9 | 320 |
| Category |
Research reproduction, financial data, charts and diagrams. |
|---|---|
| Type |
Knowledge and rules the agent applies while it works: best practices, design principles, API references. |
| Effect |
Adds no side effects of its own. It shapes how the agent thinks, writes or reviews. Guidelines and routers are always effect:none, even when the task they support edits code. |
| Needs |
Instructions only. Works in any agent that supports skills. |
| Publisher |
Everyone else. Unverified vendor claims default to community. |
| Topics | |
| Stack |
SKILL.md
From wshobson/agents at 4bde316. Checked daily, last changed Nov 6, 2026.
Frontmatter
name: backtesting-frameworks
description: Build robust backtesting systems for trading strategies with proper handling of look-ahead bias, survivorship bias, and transaction costs. Use when developing trading algorithms, validating strategies, or building backtesting infrastructure.Backtesting Frameworks
Build robust, production-grade backtesting systems that avoid common pitfalls and produce reliable strategy performance estimates.
When to Use This Skill
- Developing trading strategy backtests
- Building backtesting infrastructure
- Validating strategy performance
- Avoiding common backtesting biases
- Implementing walk-forward analysis
- Comparing strategy alternatives
Core Concepts
1. Backtesting Biases
| Bias | Description | Mitigation |
|---|---|---|
| Look-ahead | Using future information | Point-in-time data |
| Survivorship | Only testing on survivors | Use delisted securities |
| Overfitting | Curve-fitting to history | Out-of-sample testing |
| Selection | Cherry-picking strategies | Pre-registration |
| Transaction | Ignoring trading costs | Realistic cost models |
2. Proper Backtest Structure
Historical Data
│
▼
┌─────────────────────────────────────────┐
│ Training Set │
│ (Strategy Development & Optimization) │
└─────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ Validation Set │
│ (Parameter Selection, No Peeking) │
└─────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ Test Set │
│ (Final Performance Evaluation) │
└─────────────────────────────────────────┘
3. Walk-Forward Analysis
Window 1: [Train──────][Test]
Window 2: [Train──────][Test]
Window 3: [Train──────][Test]
Window 4: [Train──────][Test]
─────▶ Time
Detailed worked examples and patterns
Detailed sections (starting with ## Implementation Patterns) live in references/details.md. Read that file when the navigation summary above is insufficient.
Best Practices
Do's
- Use point-in-time data - Avoid look-ahead bias
- Include transaction costs - Realistic estimates
- Test out-of-sample - Always reserve data
- Use walk-forward - Not just train/test
- Monte Carlo analysis - Understand uncertainty
Don'ts
- Don't overfit - Limit parameters
- Don't ignore survivorship - Include delisted
- Don't use adjusted data carelessly - Understand adjustments
- Don't optimize on full history - Reserve test set
- Don't ignore capacity - Market impact matters