diff --git a/README.md b/README.md index 7d98b42..08e1279 100644 --- a/README.md +++ b/README.md @@ -6,6 +6,76 @@ Find the full reference docs [here](https://jathoms.github.io/backtest-lib) +## Developer setup + +The package supports Python 3.12 and newer. CI and the manual review guide use +Python 3.14. Building the editable package also requires the stable Rust +toolchain because the project includes a small Maturin extension. + +From PowerShell at the repository root, confirm the tools and create the locked +development environment: + +```powershell +python --version +uv --version +rustc --version +just --version +uv sync --group dev --group docs --frozen +.\.venv\Scripts\python.exe --version +``` + +Run a fast end-to-end check before exploring the code: + +```powershell +uv run pytest tests/e2e/test_stockfit_demo.py -q +uv run python -m examples.stockfit.demo --output-dir artifacts/stockfit-review +``` + +Both commands are offline and use invented fixtures. A StockFit token is not +needed for development or for the manual review. + +For VS Code, open the repository folder, select +`.venv\Scripts\python.exe` with **Python: Select Interpreter**, and follow the +[manual code-review guide](docs/stockfit/learning-guide.md) to install the local +debug configurations. + +## Architecture at a glance + +```mermaid +flowchart LR + Data[Prices and signals] --> Market[MarketView] + Market --> Backtest[Backtest period loop] + Backtest --> Engine[Engine] + Engine --> Strategy[Strategy callable] + Strategy --> Decision[Decision] + Decision --> Engine + Engine --> Portfolio[Updated portfolio] + Portfolio --> Backtest + Backtest --> Results[BacktestResults] +``` + +`MarketView` time-fences the data visible at each period. `Backtest` passes that +view to the engine, which calls the strategy. The strategy returns a declarative +`Decision`; the engine turns it into an execution plan and updated portfolio. +`Backtest` coordinates the period loop and materialises the result history. + +### Repository map + +```text +src/backtest_lib/ Core market, strategy, engine, portfolio and backtest code +examples/stockfit/ Offline-first StockFit adapter and research workflow +tests/unit/ Focused behavioural tests for individual components +tests/e2e/ Complete workflow and lookahead-safety tests +docs/source/ Sphinx reference documentation +docs/stockfit/ Methodology, evidence and manual learning material +rust/ Native universe-mapping extension built by Maturin +``` + +To understand the code rather than only run it, follow the +[manual code-review guide](docs/stockfit/learning-guide.md). It traces one +synthetic fact through the StockFit adapter, point-in-time transforms, +strategy, backtest engine and derived outputs. + ## Usage ### Quickstart @@ -166,11 +236,24 @@ def aapl_momentum_with_liquidity( return target_weights(target, fill_cash=True) ``` -## Building +## Validation + +Run the same local gates used by the repository workflows: + +```powershell +uv run pytest +uv run ruff format --check +uv run ruff check +uv run pyrefly check +# Run the documentation gates in Linux CI or WSL. +just doctest +just docs +``` -- get python 3.14 -- run `pip install uv` -- run `uv run python --version` and it will create a venv for you +On native Windows, Sphinx's generated `Cash` class and `cash` function pages +collide on a case-insensitive filesystem. The Python checks above work in +PowerShell; use WSL or the repository's Linux CI for `just doctest` and +`just docs` until those API stub names are made distinct. ## Code style diff --git a/docs/stockfit/README.md b/docs/stockfit/README.md index 60c3f39..e517842 100644 --- a/docs/stockfit/README.md +++ b/docs/stockfit/README.md @@ -20,6 +20,20 @@ uv sync uv run python --version ``` +For a complete development environment, including tests, type checking and +documentation tools, use: + +```powershell +uv sync --group dev --group docs --frozen +uv run pytest tests/e2e/test_stockfit_demo.py -q +``` + +The [manual code-review guide](learning-guide.md) provides a structured route +through both the StockFit example and the underlying `backtest-lib` engine. It +includes VS Code debug configurations, architecture diagrams, breakpoints, +self-check questions and reversible exercises. The whole review is offline and +does not require a StockFit token. + Run the deterministic offline demonstration: ```powershell @@ -92,6 +106,10 @@ try { ## Design and learning material +- [`learning-guide.md`](learning-guide.md) is the main code-level review path, + from environment setup to a small test-first exercise. +- [`vscode-launch.example.json`](vscode-launch.example.json) provides local + offline demo, audit and pytest debugger configurations. - [`methodology.md`](methodology.md) defines the signal and time gates. - [`case-study-draft.md`](case-study-draft.md) is an unpublished portfolio narrative combining the verified live backtest and data-quality audit. @@ -100,5 +118,3 @@ try { equal-weight baseline. - [`data-quality.svg`](../../artifacts/stockfit-audit-live/data-quality.svg) is the supporting audit visual across the frozen twelve-company cohort. -- [`learning-guide.md`](learning-guide.md) provides a 90-minute code-tracing - session for revisiting the implementation in VS Code. diff --git a/docs/stockfit/learning-guide.md b/docs/stockfit/learning-guide.md index 95fbe7e..bca405d 100644 --- a/docs/stockfit/learning-guide.md +++ b/docs/stockfit/learning-guide.md @@ -1,89 +1,413 @@ -# 90-minute retrospective learning guide +# Manual code-review guide -Open the repository in VS Code and select -`.venv\Scripts\python.exe` as the interpreter. Use the Run and Debug view with -`tests/e2e/test_stockfit_demo.py`, or start with: +This guide is a practical route through `backtest-lib` and the StockFit +integration. It is designed for VS Code on Windows, but every command is an +ordinary PowerShell command and can also be run from another IDE terminal. + +The review is deliberately offline. It uses invented StockFit-shaped fixtures, +makes no network requests and does not need a StockFit token. + +## What you should be able to explain afterwards + +By the end, you should be able to: + +- describe how `MarketView`, a strategy, a `Decision`, the engine and a + portfolio fit together; +- trace a filing-dated fact from a fixture into a point-in-time signal; +- explain how the code prevents a later filing from leaking into an earlier + decision; +- follow one monthly decision through `Backtest.run` into new holdings; +- distinguish raw inputs from the derived outputs that are safe to retain; +- make and verify a small test-first change. + +Allow about four hours in total. Each session is independent, so stop at any +checkpoint and return later. + +## Session 0: prepare the workspace (20 minutes) + +### 1. Build the environment + +From the repository root: + +```powershell +python --version +uv --version +rustc --version +uv sync --group dev --group docs --frozen +.\.venv\Scripts\python.exe --version +uv run pytest tests/e2e/test_stockfit_demo.py -q +``` + +The Python version should be 3.12 or newer; the repository CI uses 3.14. The +focused test should pass without reading `STOCKFIT_TOKEN`. + +### 2. Install the local VS Code launch configurations + +The repository intentionally ignores `.vscode/`, so copy the reviewed example +instead of committing personal editor state: + +```powershell +New-Item -ItemType Directory -Force .vscode | Out-Null +Copy-Item docs\stockfit\vscode-launch.example.json .vscode\launch.json +code . +``` + +In VS Code, run **Python: Select Interpreter** and choose +`.venv\Scripts\python.exe`. Open **Run and Debug** with `Ctrl+Shift+D`. Four +configurations should be available: + +- `Debug: StockFit offline demo`; +- `Debug: StockFit offline audit`; +- `Debug: StockFit end-to-end test`; +- `Debug: current pytest file`. + +They use `justMyCode: false`, allowing `F11` to step from the example into +`src/backtest_lib`. The module-based configurations also avoid relying on a +hard-coded path to VS Code's internal `debugpy` launcher. + +If the debugger itself stalls, first run the equivalent terminal command. A +passing terminal command separates a debugger configuration problem from a +project problem: ```powershell uv run pytest tests/e2e/test_stockfit_demo.py -q ``` -The objective is to trace one piece of synthetic data from request-shaped input -to a portfolio decision. Use the offline path; no token is needed. +### 3. Generate the offline outputs once + +```powershell +uv run python -m examples.stockfit.demo --output-dir artifacts/stockfit-review +uv run python -m examples.stockfit.audit_cli --output-dir artifacts/stockfit-audit-review +``` -## 0–15 minutes: client boundary +Inspect the filenames, but do not try to interpret every value yet. The demo +writes a derived summary and NAV chart; the audit writes a derived summary, +company table and quality chart. -Read `examples/stockfit/client.py`. Set breakpoints at: +**Checkpoint:** You can run tests and both offline commands, and VS Code uses +the repository virtual environment. -- `StockFitClient.price_history` where query parameters are constructed; -- `StockFitClient._get` immediately before the request is opened; -- `_get` where the HTTP status is validated; -- `_get` where the decoded JSON shape is validated. +## Architecture map -Run the focused client tests and inspect `url`, `query`, `status` and the decoded -object. Notice that exception text contains context but never the bearer token -or response body. +### Components -## 15–40 minutes: time-aware transforms +```mermaid +flowchart LR + Fixtures[Invented StockFit-shaped fixtures] + Client[StockFitClient boundary] + Transforms[Point-in-time transforms] + Market[MarketView] + Strategy[Fundamental strategy] + Decision[TargetWeightsDecision] + Backtest[Backtest period loop] + Engine[Engine] + Planner[Plan generator] + Executor[Plan executor] + Portfolio[Portfolio state] + Results[BacktestResults] + Outputs[Derived JSON and SVG] -Read `examples/stockfit/transforms.py`. Set breakpoints at: + Fixtures --> Transforms + Client -. live mode only .-> Transforms + Transforms --> Market + Market --> Backtest + Backtest --> Engine + Engine --> Strategy + Strategy --> Decision + Decision --> Engine + Engine --> Planner --> Executor --> Portfolio + Portfolio --> Backtest + Backtest --> Results --> Outputs +``` + +The dashed client path is not used during this review. Offline fixtures already +have the same narrow shape consumed by the transforms. + +### One decision period + +```mermaid +sequenceDiagram + participant B as Backtest.run + participant M as MarketView + participant S as Strategy + participant E as Engine + participant P as Portfolio + + B->>M: truncated_to(current period) + B->>E: execute_strategy(...) + E->>S: inject market and universe + S-->>E: target_weights(...) + E->>E: Decision to Plan + E->>P: execute plan at current prices + P-->>B: updated holdings and cash +``` + +The strategy never mutates a portfolio directly. It returns a declarative +decision, which the engine plans and executes. + +## Session 1: understand the core library (45 minutes) + +Read these files in order: + +1. `src/backtest_lib/__init__.py` — public API exports; +2. `src/backtest_lib/market/__init__.py` — `MarketView` and `PastView`; +3. `src/backtest_lib/strategy/__init__.py` — the strategy callable contract; +4. `src/backtest_lib/engine/decision/__init__.py` — decision types and helpers; +5. `src/backtest_lib/portfolio/__init__.py` — holdings, cash and value; +6. `src/backtest_lib/backtest/__init__.py` — orchestration loop. + +Keep this mental model while reading: + +```python +def strategy(universe, market): + # Read only the time-fenced view available at this decision. + return target_weights({security: 1 / len(universe) for security in universe}) +``` + +Parameter names matter. `Engine._parse_strategy_args` inspects the callable and +injects only `universe`, `current_portfolio`, `market` and `ctx`. + +### Breakpoints + +Use `Debug: StockFit end-to-end test` and set breakpoints at: + +- `Backtest.__init__` after the decision schedule is constructed; +- `Backtest.run` at `past_market_view = self.market_view.truncated_to(i)`; +- `Engine.execute_strategy` after `used_kwargs` is created; +- `PerfectWorldPlanGenerator.generate_plan`; +- `PerfectWorldPlanExecutor.execute_plan`. + +At each stop, answer: + +1. What data can the strategy see? +2. What is the current decision date? +3. Is the strategy returning an action or mutating state? +4. Where does the returned decision become a new portfolio? + +**Checkpoint:** You can explain the difference between a strategy, decision, +plan, execution result and portfolio. + +## Session 2: trace point-in-time data (60 minutes) + +Read: + +- `examples/stockfit/client.py`; +- `examples/stockfit/transforms.py`; +- `tests/unit/examples/stockfit/test_transforms.py`. + +### Provider boundary + +`StockFitClient` is deliberately narrow. It constructs three authenticated GET +requests, validates HTTP and JSON shape, and removes the bearer token from any +provider error message. Tests replace its transport, so validation does not +need a network call. + +Run: + +```powershell +uv run pytest tests/unit/examples/stockfit/test_client.py -q +``` + +### Price alignment + +`price_frame` validates each observation, sorts by date and retains only dates +shared by every security. It does not forward-fill a missing price. + +### Filing gates and rollback + +```mermaid +flowchart TD + D[Decision date] + F{Original filing date on or before D?} + V[Start from current reported value] + S[Find later sources with numeric before values] + R[Roll back newest source to oldest] + O[Fact knowable on D] + N[Fact unavailable] + + D --> F + F -- no --> N + F -- yes --> V --> S --> R --> O +``` + +The key idea in `fact_as_of` is: + +```python +for _, before in sorted(rollbacks, reverse=True): + value = before +``` + +The loop reverses changes filed after the decision date. It applies to any +later source with a numeric `before` value, not only sources labelled as an +amendment. + +Set breakpoints at: - `fact_as_of` after `original_filing_date` is parsed; -- the reverse-sorted source rollback loop; +- the loop collecting `rollbacks`; +- the reverse-sorted rollback loop; - `_score_as_of` after `available_indexes` is built; -- `_score_as_of` where the score and eligibility flag are returned; -- `signal_frames` while iterating over an `as_of` date. +- `signal_frames` inside the `as_of` loop; +- `build_market` immediately before `btl.MarketView(...)`. -Run: +Run the focused transform tests: ```powershell uv run pytest tests/unit/examples/stockfit/test_transforms.py -q ``` -Watch how the same statement produces a different fact before and after a later -filing date. +Pause on +`test_fact_as_of_rolls_back_before_value_from_ordinary_later_filing` and +write down the value before and after each rollback. + +**Checkpoint:** You can explain why a later restatement cannot change an +earlier decision and why an incomplete fact produces ineligibility rather than +an estimate. + +## Session 3: follow strategy to result (60 minutes) + +Read: + +- `examples/stockfit/strategy.py`; +- `examples/stockfit/demo.py`; +- `tests/unit/examples/stockfit/test_strategy.py`; +- `tests/e2e/test_stockfit_demo.py`. + +The fundamental strategy reads the latest score and eligibility rows, ranks +eligible securities by descending score and uses the ticker as a deterministic +tie-break: + +```python +ranked = sorted( + (security for security in universe if eligible[security] == 1), + key=lambda security: (-score[security], security), +) +``` + +It returns equal target weights for the first `top_n` securities. If none are +eligible, it returns `hold()`. + +Select `Debug: StockFit offline demo` and set breakpoints at: + +- `run_demo` after `build_market`; +- `make_fundamental_strategy.strategy` after `ranked` is created; +- `Backtest.run` immediately before `execute_strategy`; +- `Engine.execute_strategy` after the strategy returns; +- `BacktestResults.from_weights_market_initial_capital`; +- `_metrics` in `examples/stockfit/demo.py`. + +For the first rebalance, record: + +| Question | Your observation | +| --- | --- | +| Which securities are eligible? | | +| What are their scores? | | +| Which two are selected? | | +| What decision object is returned? | | +| What are the resulting holdings and cash? | | + +Then compare the fundamental path with `equal_weight_strategy`. Both use the +same cohort, price dates, schedule and starting cash; only the decision rule +differs. + +**Checkpoint:** You can trace a score into weights and explain why the backtest +result ending behind the comparator is evidence, not a framework error. + +## Session 4: understand the audit and safe outputs (45 minutes) + +Read: + +- `examples/stockfit/audit.py`; +- `examples/stockfit/audit_cli.py`; +- `tests/unit/examples/stockfit/test_audit.py`; +- `tests/e2e/test_stockfit_audit.py`. + +```mermaid +flowchart LR + Inputs[Company, price and statement payloads] + Checks[Metadata, coverage, integrity and fact checks] + Company[CompanyAudit] + Cohort[AuditRun and cohort summary] + Safe[Derived summary.json, CSV and SVG] + + Inputs --> Checks --> Company --> Cohort --> Safe +``` + +Use `Debug: StockFit offline audit` and stop at: -## 40–55 minutes: portfolio decision +- `audit_company_metadata`; +- `audit_prices` after dates are normalised; +- `audit_statements` after required facts are counted; +- `combine_company_audit`; +- `summarise_cohort`; +- `publish_artifacts` before the temporary directory is promoted. -Read `examples/stockfit/strategy.py`. In the inner `strategy` function, stop -after `score` and `eligible` are read, then again after `ranked` is created. -Inspect the deterministic sort key and the weights returned by -`target_weights`. Run: +Confirm that status is computed from predeclared reason codes and that output +functions consume audit dataclasses rather than raw provider payloads. + +Run: ```powershell -uv run pytest tests/unit/examples/stockfit/test_strategy.py -q +uv run pytest tests/unit/examples/stockfit/test_audit.py -q +uv run pytest tests/e2e/test_stockfit_audit.py -q ``` -## 55–75 minutes: complete backtest +**Checkpoint:** You can explain why a failed company remains visible, how an +incomplete run is labelled, and why retained artefacts cannot reproduce raw +prices or filings. + +## Session 5: reversible exercises (30–60 minutes) + +Before each exercise, predict the result. Run the smallest relevant test after +the change, then use `git diff` to inspect and revert your experiment. + +1. Change one synthetic `dateFiled` value and predict the first eligible date. +2. Change a numeric `before` value and trace each rollback step. +3. Remove `operatingIncome` from one fixture and confirm ineligibility. +4. Change `make_fundamental_strategy(top_n=2)` to `top_n=1` and predict the + returned target weights. +5. Add a failing test for a malformed price observation, then make the smallest + implementation change needed to pass it. +6. Add an unknown parameter to a toy strategy and trace the error from + `Engine._parse_strategy_args`. + +Useful focused commands: + +```powershell +uv run pytest tests/unit/examples/stockfit/test_transforms.py -q +uv run pytest tests/unit/examples/stockfit/test_strategy.py -q +uv run pytest tests/e2e/test_stockfit_demo.py -q +git diff +``` -Read `examples/stockfit/demo.py`. Set breakpoints at: +Revert only your deliberate learning edits. Do not remove other work that may +already be present in the checkout. -- `run_demo` immediately after `build_market`; -- each `btl.Backtest(...)` construction; -- `Backtest.run` in `src/backtest_lib/backtest/__init__.py`; -- `_metrics` when the results are converted to the safe summary. +## Glossary -Step through one decision period. Follow `market.signals` into the strategy, -the returned decision into the engine, and the new holdings into the NAV. Then -run the offline command and open `artifacts/stockfit/nav-comparison.svg`. +| Term | Meaning in this repository | +| --- | --- | +| `MarketView` | Prices, signals and tradability indexed across the full market history | +| `PastView` | A time-aware view used to expose slices by period or security | +| Strategy | Callable that reads injected context and returns a `Decision` | +| `Decision` | Declarative request such as hold, trade or target weights | +| Plan | Engine operations generated from a decision at current prices | +| Executor | Applies plan operations to a portfolio | +| Portfolio | Holdings, cash, universe and total value at a point in the simulation | +| `BacktestResults` | Allocation and value history plus derived performance statistics | +| Point-in-time | Restricting a decision to information knowable on its historical date | -## 75–90 minutes: six exercises +## Final teach-back checklist -Use one exercise per short debugging session; rerun the smallest relevant test -after every change. +Without looking at the diagrams, try to explain: -1. Change one synthetic `dateFiled` value and predict the first eligible price - date before running the transform test. -2. Put breakpoints around a fixture's two later sources and write down each value - produced while rollback proceeds from newest to oldest. -3. Remove `operatingIncome` from one synthetic filing and confirm that the - security becomes ineligible rather than receiving a partial score. -4. Change `make_fundamental_strategy(top_n=2)` to `top_n=1`, predict the target - weights, then compare the generated summary. -5. Trace the first decision for the cash-starting fundamental strategy and the - equal-weight comparator; explain why their entry dates can differ. -6. Add one failing test for a malformed price observation or statement fact, - then make the smallest implementation change needed to pass it. +1. Why does `Backtest.run` pass a truncated `MarketView` to the engine? +2. How are strategy arguments selected? +3. What separates a `Decision` from portfolio mutation? +4. How does `fact_as_of` undo future knowledge? +5. Why does missing fundamental data make a security ineligible? +6. What makes the comparator a fairer baseline? +7. Which outputs are safe to retain, and what is intentionally excluded? -Finish by reverting experimental fixture or strategy changes unless you intend -to develop them as a separate, reviewed feature. +If any answer is unclear, return to the corresponding checkpoint and step +through one test rather than rereading the entire repository. diff --git a/docs/stockfit/vscode-launch.example.json b/docs/stockfit/vscode-launch.example.json new file mode 100644 index 0000000..341af41 --- /dev/null +++ b/docs/stockfit/vscode-launch.example.json @@ -0,0 +1,49 @@ +{ + "version": "0.2.0", + "configurations": [ + { + "name": "Debug: StockFit offline demo", + "type": "debugpy", + "request": "launch", + "module": "examples.stockfit.demo", + "args": ["--output-dir", "artifacts/stockfit-review"], + "cwd": "${workspaceFolder}", + "console": "integratedTerminal", + "justMyCode": false + }, + { + "name": "Debug: StockFit offline audit", + "type": "debugpy", + "request": "launch", + "module": "examples.stockfit.audit_cli", + "args": ["--output-dir", "artifacts/stockfit-audit-review"], + "cwd": "${workspaceFolder}", + "console": "integratedTerminal", + "justMyCode": false + }, + { + "name": "Debug: StockFit end-to-end test", + "type": "debugpy", + "request": "launch", + "module": "pytest", + "args": [ + "tests/e2e/test_stockfit_demo.py::test_offline_demo_writes_deterministic_derived_outputs", + "-q", + "-s" + ], + "cwd": "${workspaceFolder}", + "console": "integratedTerminal", + "justMyCode": false + }, + { + "name": "Debug: current pytest file", + "type": "debugpy", + "request": "launch", + "module": "pytest", + "args": ["${file}", "-q", "-s"], + "cwd": "${workspaceFolder}", + "console": "integratedTerminal", + "justMyCode": false + } + ] +}