Release v1.0.1

.gitignore

@@ -4,7 +4,21 @@ __OLD__/
 .specstory/
 .history/
 .vscode/
+*.py[cod]
+.ipynb_checkpoints/
+.pytest_cache/
+
+# Local environments
+.venv/
+venv/
+
+# Local test data and generated analysis results
+data/*
+!data/.gitkeep
+results/*
+!results/.gitkeep
+
 data
+
 cvttpy
-results/
 tmp/

AGENTS.md

@@ -105,6 +105,10 @@ If verification cannot be run, report exactly what was skipped and why.
 
 - Update `CHANGELOG.md` for every release with the release version, release
   date, Git tag, and a concise summary of notable changes.
+- Keep an `Unreleased` section at the top of `CHANGELOG.md` for changes that
+  have not been included in a tagged release yet.
+- Move relevant entries from `Unreleased` into the dated release section when
+  creating a release, and leave `Unreleased` present for future changes.
 - Use release headers in `YYYY-MM-DD vMAJOR.MINOR.PATCH` form.
 - Use version numbers in `MAJOR.MINOR.PATCH` form. Start this repository at
   `0.0.1`.
@@ -112,6 +116,9 @@ If verification cannot be run, report exactly what was skipped and why.
   exactly. For example, version `0.0.1` must be tagged as `v0.0.1`.
 - Create the Git tag only after the changelog and any release-related version
   changes are complete.
+- When the user requests creating a release, treat that as explicit permission
+  to commit the release changes, create the matching Git tag, and push both the
+  branch and tag.
 - Do not push release commits or tags unless the user explicitly requests it.
 
 ## Mandatory background review

CHANGELOG.md

@@ -2,6 +2,56 @@
 
 All notable changes to this project are documented in this file.
 
+## Unreleased
+
+No unreleased changes yet.
+
+## 2026-07-28 v1.0.1
+
+- Added the `spbt_day` notebook for interactive single-day backtest result
+  analysis, including SQLite result file selection from the local data
+  directory.
+- Added selector-pair loading and dense ranking by `mr_score.final`, preserving
+  rows with invalid score JSON for inspection.
+- Added theoretical return calculation for ranked pairs from
+  `trading_instructions`, including reusable helper functions and tests.
+- Added a Plotly histogram for visual analysis of total theoretical return by
+  pair.
+- Moved notebook support code into reusable `scripts/spbt_day.py` helpers.
+- Adjusted notebook table outputs to show all relevant rows and reduce
+  redundant intermediate displays.
+- Added an alphabetically sorted pair selector for individual pair analysis.
+- Added selected-pair theoretical execution tables and aligned TheoRet
+  calculations with target-delta trade generation.
+- Added per-asset `strength` values to selected-pair theoretical execution
+  tables.
+- Corrected theoretical execution size to use
+  `10000 * strength / reference_price`.
+- Removed `:USD` quote suffixes from displayed pair names in notebook tables,
+  chart hovers, and the pair selector dropdown while preserving full internal
+  pair keys for calculations.
+- Added `num_trades` to pair TheoRet summaries, counting asset-level theoretical
+  trades from effective `TARGET` and `CLOSE` instructions.
+- Added sortable interactive grids for the pair TheoRet and selected-pair
+  theoretical execution tables.
+- Styled interactive dataframe grids with black text on white backgrounds for
+  readability across notebook themes.
+- Added a selected-pair Plotly chart that overlays theoretical BUY/SELL
+  executions on relative 1-minute market close data for both instruments.
+- Anchored the selected-pair market chart at trading-day midnight and normalized
+  relative prices to each instrument's close at that timestamp.
+- Added a `min_pctg_change` threshold for ranked pair TheoRet calculations to
+  skip small target-strength changes after a position is acquired.
+- Added a notebook input field for the minimum TARGET strength-change threshold.
+
+## 2026-07-25 v0.0.9
+
+- Added contributing guidance and Python dependency declarations.
+- Added placeholder files for active project directories.
+- Updated ignore rules for local data, generated results, caches, and local
+  environments.
+- Documented unreleased changelog handling and release push behavior.
+
 ## 2026-07-25 v0.0.1
 
 - Established the initial repository structure and project guidance.

CONTRIBUTING.md

@@ -0,0 +1,54 @@
+# Contributing
+
+## Setup
+
+Use the shared Python 3.12 virtual environment:
+
+```bash
+source ~/.pyenv/python3.12-venv/bin/activate
+python -m pip install -r requirements.txt
+```
+
+If you install another package for repository work, add its direct dependency
+to `requirements.txt`.
+
+## Repository layout
+
+- Put notebooks in `notebooks/`.
+- Put reusable Python and Bash utilities in `scripts/`.
+- Put local input files in `data/`.
+- Put generated artifacts in `results/`.
+
+The contents of `data/` and `results/` are ignored. Do not force-add test
+databases, raw test results, generated exports, or notebook outputs.
+
+`__SAV__/` is unrelated legacy material and is outside the active project.
+
+## Working with notebooks
+
+Notebooks must execute from top to bottom in a fresh kernel. Use relative paths,
+document data assumptions, and move reusable logic into tested scripts.
+
+Before handing off a change:
+
+```bash
+python -m pytest
+python -m pytest --nbmake notebooks
+jupyter nbconvert --ClearOutputPreprocessor.enabled=True --inplace path/to/notebook.ipynb
+```
+
+Run only the checks relevant to the files present in the repository, and report
+anything that could not be run.
+
+## Review requirement
+
+Python scripts, Bash scripts, and notebook code-cell changes require review and
+approval by a separate background agent. Address material findings and rerun
+affected checks before completion. If a reviewer is unavailable, the change may
+be handed off only with the status `review pending`.
+
+Documentation, dependency declarations, and ignore rules do not require this
+background review when no Python, Bash, or notebook code cells changed.
+
+The `master` branch is not protected. That does not remove the review
+requirement or authorize an agent to commit or push without an explicit request.

notebooks/.gitkeep

@@ -0,0 +1 @@
+

notebooks/spbt_day.ipynb

@@ -0,0 +1,357 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "single-day-title",
+   "metadata": {},
+   "source": [
+    "# Single-Day Backtest Result Analysis\n",
+    "\n",
+    "This notebook analyzes the result of one single-day backtest stored in a SQLite database. Development is staged; Step 1 only selects the database file that later sections will read.\n",
+    "\n",
+    "Input assumptions for Step 1:\n",
+    "\n",
+    "- The default data directory is `data/` at the repository root.\n",
+    "- SQLite result files usually use `.db`, `.sqlite`, or `.sqlite3` extensions.\n",
+    "- The directory can be changed interactively if the result file lives elsewhere."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "imports-and-paths",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from pathlib import Path\n",
+    "import importlib\n",
+    "import sys\n",
+    "\n",
+    "from IPython.display import display\n",
+    "import ipywidgets as widgets\n",
+    "import pandas as pd\n",
+    "\n",
+    "START_DIR = Path.cwd().resolve()\n",
+    "for candidate in (START_DIR, *START_DIR.parents):\n",
+    "    if (candidate / \"scripts\" / \"spbt_day.py\").exists():\n",
+    "        if str(candidate) not in sys.path:\n",
+    "            sys.path.insert(0, str(candidate))\n",
+    "        break\n",
+    "\n",
+    "import scripts.spbt_day as spbt_day\n",
+    "\n",
+    "spbt_day = importlib.reload(spbt_day)\n",
+    "\n",
+    "add_total_pnl = spbt_day.add_total_pnl\n",
+    "calculate_pair_theo_executions = spbt_day.calculate_pair_theo_executions\n",
+    "calculate_ranked_pairs_theo_ret = spbt_day.calculate_ranked_pairs_theo_ret\n",
+    "create_database_file_selector = spbt_day.create_database_file_selector\n",
+    "create_pair_name_dropdown = spbt_day.create_pair_name_dropdown\n",
+    "create_pair_trades_market_plot = spbt_day.create_pair_trades_market_plot\n",
+    "create_total_pnl_histogram = spbt_day.create_total_pnl_histogram\n",
+    "find_repo_root = spbt_day.find_repo_root\n",
+    "format_pair_name_for_display = spbt_day.format_pair_name_for_display\n",
+    "format_pair_names_for_display = spbt_day.format_pair_names_for_display\n",
+    "infer_trading_day_start_ns = spbt_day.infer_trading_day_start_ns\n",
+    "load_selector_pair_rankings = spbt_day.load_selector_pair_rankings\n",
+    "load_pair_market_data = spbt_day.load_pair_market_data\n",
+    "load_trading_instructions = spbt_day.load_trading_instructions\n",
+    "show_interactive_dataframe = spbt_day.show_interactive_dataframe\n",
+    "\n",
+    "REPO_ROOT = find_repo_root()\n",
+    "DEFAULT_DATA_DIR = REPO_ROOT / \"data\"\n",
+    "\n",
+    "REPO_ROOT, DEFAULT_DATA_DIR"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "database-file-selector",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "db_selector = create_database_file_selector(\n",
+    "    default_data_dir=DEFAULT_DATA_DIR,\n",
+    "    repo_root=REPO_ROOT,\n",
+    ")\n",
+    "\n",
+    "display(db_selector.widget)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "selected-database-helpers",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "selected_database_path = db_selector.selected_database_path\n",
+    "connect_selected_database = db_selector.connect_selected_database\n",
+    "\n",
+    "# Later notebook sections can call selected_database_path() or connect_selected_database()."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "selector-pair-rankings-context",
+   "metadata": {},
+   "source": [
+    "## Selector Pair Rankings\n",
+    "\n",
+    "Load `selector_pairs.pair_name` and `selector_pairs.mr_score` from the selected SQLite database. The JSON field `mr_score.final` is parsed as a numeric score and ranked descending with dense ranks, so tied scores share the same rank and the next distinct score gets the next rank.\n",
+    "\n",
+    "Rows with missing, malformed, non-numeric, or non-finite `mr_score.final` values are preserved, sorted after ranked rows, and marked in `mr_score_parse_status`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "load-selector-pair-rankings",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "conn = connect_selected_database()\n",
+    "try:\n",
+    "    selector_pair_rankings = load_selector_pair_rankings(conn)\n",
+    "finally:\n",
+    "    conn.close()\n",
+    "\n",
+    "selector_pair_rankings_display = format_pair_names_for_display(\n",
+    "    selector_pair_rankings[[\"pair_rank\", \"pair_name\", \"mr_score_final\"]]\n",
+    ")\n",
+    "with pd.option_context(\"display.max_rows\", None):\n",
+    "    display(selector_pair_rankings_display)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "theoretical-return-context",
+   "metadata": {},
+   "source": [
+    "## Theoretical Return by Pair\n",
+    "\n",
+    "Load `trading_instructions` and calculate theoretical return for each ranked pair. Each pair starts from a fixed `$10,000` theoretical USD base. `TARGET` trades from the current theoretical position to the new target position, where target size is `10000 * strength / reference_price`; `CLOSE` liquidates the open position at the close row's `reference_price`; `HOLD` is ignored.\n",
+    "\n",
+    "`MIN_TARGET_STRENGTH_CHANGE_PCTG` can be raised above `0.0` to skip `TARGET` updates whose absolute percentage strength change is smaller than the threshold since the position was acquired. `num_trades` counts asset-level theoretical trades caused by effective `TARGET` and `CLOSE` rows. `realized_pnl` and `unrealized_pnl` are percentage returns relative to `$10,000`. The displayed dataframe is sorted by total return (`realized_pnl + unrealized_pnl`) ascending."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "target-change-threshold-input",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "min_target_change_input = widgets.FloatText(\n",
+    "    value=0.0,\n",
+    "    description=\"Mininal TARGET change (%)\",\n",
+    "    step=1.0,\n",
+    "    layout=widgets.Layout(width=\"420px\"),\n",
+    "    style={\"description_width\": \"190px\"},\n",
+    ")\n",
+    "display(min_target_change_input)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "load-trading-instructions",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "conn = connect_selected_database()\n",
+    "try:\n",
+    "    trading_instructions = load_trading_instructions(conn)\n",
+    "finally:\n",
+    "    conn.close()\n",
+    "\n",
+    "print(f\"Loaded {len(trading_instructions):,} trading instruction rows.\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "calculate-pair-theoretical-returns",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "MIN_TARGET_STRENGTH_CHANGE_PCTG = float(min_target_change_input.value)\n",
+    "\n",
+    "pair_theo_ret = add_total_pnl(\n",
+    "    calculate_ranked_pairs_theo_ret(\n",
+    "        selector_pair_rankings,\n",
+    "        trading_instructions,\n",
+    "        min_pctg_change=MIN_TARGET_STRENGTH_CHANGE_PCTG,\n",
+    "    )\n",
+    ").sort_values(\n",
+    "    [\"total_pnl\", \"pair_name\"],\n",
+    "    ascending=[True, True],\n",
+    "    kind=\"mergesort\",\n",
+    ").drop(columns=\"total_pnl\").reset_index(drop=True)\n",
+    "\n",
+    "pair_theo_ret_display = format_pair_names_for_display(pair_theo_ret)\n",
+    "\n",
+    "show_interactive_dataframe(\n",
+    "    pair_theo_ret_display,\n",
+    "    table_id=\"pair-theo-ret-grid\",\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "theoretical-return-histogram-context",
+   "metadata": {},
+   "source": [
+    "## Total Theoretical Return Distribution\n",
+    "\n",
+    "Plot the distribution of total theoretical return, calculated as `realized_pnl + unrealized_pnl`. Plotly chooses histogram bins automatically."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "plot-total-theoretical-return-histogram",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "total_pnl_histogram = create_total_pnl_histogram(pair_theo_ret)\n",
+    "\n",
+    "total_pnl_histogram"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "individual-pair-analysis-context",
+   "metadata": {},
+   "source": [
+    "## Individual Pair Analysis\n",
+    "\n",
+    "Choose one pair for detailed follow-up analysis. Pair names are sorted alphabetically."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "choose-individual-pair",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "pair_name_dropdown = create_pair_name_dropdown(selector_pair_rankings)\n",
+    "display(pair_name_dropdown)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "selected-individual-pair",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "selected_pair_name = pair_name_dropdown.value\n",
+    "format_pair_name_for_display(selected_pair_name)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "selected-pair-theo-executions-context",
+   "metadata": {},
+   "source": [
+    "### Selected Pair Theoretical Executions\n",
+    "\n",
+    "Create the theoretical asset-level executions used by the PnL calculation for the selected pair. `TARGET` rows trade the position difference from the current theoretical position to the new target position, where target size is `10000 * strength / reference_price`; `CLOSE` rows flatten the current theoretical position. Positive size is `BUY`; negative size is `SELL`; USD value is signed as the opposite cash movement."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "selected-pair-theo-executions",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "selected_pair_theo_executions = calculate_pair_theo_executions(\n",
+    "    selected_pair_name,\n",
+    "    trading_instructions,\n",
+    "    min_pctg_change=MIN_TARGET_STRENGTH_CHANGE_PCTG,\n",
+    ")\n",
+    "\n",
+    "selected_pair_theo_execution_columns = [\n",
+    "    \"time\",\n",
+    "    \"asset\",\n",
+    "    \"action\",\n",
+    "    \"side\",\n",
+    "    \"strength\",\n",
+    "    \"size\",\n",
+    "    \"price\",\n",
+    "    \"usd_value\",\n",
+    "]\n",
+    "selected_pair_theo_executions_display = selected_pair_theo_executions.reindex(\n",
+    "    columns=selected_pair_theo_execution_columns\n",
+    ")\n",
+    "show_interactive_dataframe(\n",
+    "    selected_pair_theo_executions_display,\n",
+    "    table_id=\"selected-pair-theo-executions-grid\",\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "selected-pair-market-trades-context",
+   "metadata": {},
+   "source": [
+    "### Selected Pair Trades on Market Data\n",
+    "\n",
+    "Load full available 1-minute market data for the selected pair's instruments from `ohlcv_1min`, starting at midnight UTC of the trading day inferred from `trading_instructions`. Close prices are shown as relative prices from each instrument's close at that midnight. Theoretical executions are overlaid at their execution `reference_price`, normalized by the same midnight close. Execution markers use execution timestamps directly and do not require a matching OHLCV row."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "selected-pair-market-trades-plot",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "trading_day_start_ns = infer_trading_day_start_ns(trading_instructions)\n",
+    "\n",
+    "conn = connect_selected_database()\n",
+    "try:\n",
+    "    selected_pair_market_data = load_pair_market_data(\n",
+    "        conn,\n",
+    "        selected_pair_name,\n",
+    "        trading_day_start_ns=trading_day_start_ns,\n",
+    "    )\n",
+    "finally:\n",
+    "    conn.close()\n",
+    "\n",
+    "selected_pair_market_trades_plot = create_pair_trades_market_plot(\n",
+    "    selected_pair_name,\n",
+    "    selected_pair_market_data,\n",
+    "    selected_pair_theo_executions,\n",
+    ")\n",
+    "\n",
+    "selected_pair_market_trades_plot"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "python3.12-venv (3.12.13.final.0)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.12.13"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

requirements.txt

@@ -0,0 +1,12 @@
+# Interactive analysis
+ipykernel>=6.29,<7
+ipywidgets>=8.1,<9
+itables>=2.2,<3
+jupyter>=1.1,<2
+nbformat>=5.10,<6
+pandas>=2.2,<3
+plotly>=5.24,<7
+
+# Verification
+nbmake>=1.5,<2
+pytest>=8,<9

results/.gitkeep

@@ -0,0 +1 @@
+

scripts/.gitkeep

@@ -0,0 +1 @@
+