{ "cells": [ { "cell_type": "markdown", "id": "single-day-title", "metadata": {}, "source": [ "# Single-Day Backtest Result Analysis\n", "\n", "This notebook analyzes the result of one single-day backtest stored in a SQLite database. Development is staged; Step 1 only selects the database file that later sections will read.\n", "\n", "Input assumptions for Step 1:\n", "\n", "- The default data directory is `data/` at the repository root.\n", "- SQLite result files usually use `.db`, `.sqlite`, or `.sqlite3` extensions.\n", "- The directory can be changed interactively if the result file lives elsewhere." ] }, { "cell_type": "code", "execution_count": null, "id": "imports-and-paths", "metadata": {}, "outputs": [], "source": [ "from pathlib import Path\n", "import importlib\n", "import sys\n", "\n", "from IPython.display import display\n", "import ipywidgets as widgets\n", "import pandas as pd\n", "\n", "START_DIR = Path.cwd().resolve()\n", "for candidate in (START_DIR, *START_DIR.parents):\n", " if (candidate / \"scripts\" / \"spbt_day.py\").exists():\n", " if str(candidate) not in sys.path:\n", " sys.path.insert(0, str(candidate))\n", " break\n", "\n", "import scripts.spbt_day as spbt_day\n", "\n", "spbt_day = importlib.reload(spbt_day)\n", "\n", "add_total_pnl = spbt_day.add_total_pnl\n", "calculate_pair_theo_executions = spbt_day.calculate_pair_theo_executions\n", "calculate_ranked_pairs_theo_ret = spbt_day.calculate_ranked_pairs_theo_ret\n", "create_database_file_selector = spbt_day.create_database_file_selector\n", "create_pair_name_dropdown = spbt_day.create_pair_name_dropdown\n", "create_pair_trades_market_plot = spbt_day.create_pair_trades_market_plot\n", "create_total_pnl_histogram = spbt_day.create_total_pnl_histogram\n", "find_repo_root = spbt_day.find_repo_root\n", "format_pair_name_for_display = spbt_day.format_pair_name_for_display\n", "format_pair_names_for_display = spbt_day.format_pair_names_for_display\n", "infer_trading_day_start_ns = spbt_day.infer_trading_day_start_ns\n", "load_selector_pair_rankings = spbt_day.load_selector_pair_rankings\n", "load_pair_market_data = spbt_day.load_pair_market_data\n", "load_trading_instructions = spbt_day.load_trading_instructions\n", "show_interactive_dataframe = spbt_day.show_interactive_dataframe\n", "\n", "REPO_ROOT = find_repo_root()\n", "DEFAULT_DATA_DIR = REPO_ROOT / \"data\"\n", "\n", "REPO_ROOT, DEFAULT_DATA_DIR" ] }, { "cell_type": "code", "execution_count": null, "id": "database-file-selector", "metadata": {}, "outputs": [], "source": [ "db_selector = create_database_file_selector(\n", " default_data_dir=DEFAULT_DATA_DIR,\n", " repo_root=REPO_ROOT,\n", ")\n", "\n", "display(db_selector.widget)" ] }, { "cell_type": "code", "execution_count": null, "id": "selected-database-helpers", "metadata": {}, "outputs": [], "source": [ "selected_database_path = db_selector.selected_database_path\n", "connect_selected_database = db_selector.connect_selected_database\n", "\n", "# Later notebook sections can call selected_database_path() or connect_selected_database()." ] }, { "cell_type": "markdown", "id": "selector-pair-rankings-context", "metadata": {}, "source": [ "## Selector Pair Rankings\n", "\n", "Load `selector_pairs.pair_name` and `selector_pairs.mr_score` from the selected SQLite database. The JSON field `mr_score.final` is parsed as a numeric score and ranked descending with dense ranks, so tied scores share the same rank and the next distinct score gets the next rank.\n", "\n", "Rows with missing, malformed, non-numeric, or non-finite `mr_score.final` values are preserved, sorted after ranked rows, and marked in `mr_score_parse_status`." ] }, { "cell_type": "code", "execution_count": null, "id": "load-selector-pair-rankings", "metadata": {}, "outputs": [], "source": [ "conn = connect_selected_database()\n", "try:\n", " selector_pair_rankings = load_selector_pair_rankings(conn)\n", "finally:\n", " conn.close()\n", "\n", "selector_pair_rankings_display = format_pair_names_for_display(\n", " selector_pair_rankings[[\"pair_rank\", \"pair_name\", \"mr_score_final\"]]\n", ")\n", "with pd.option_context(\"display.max_rows\", None):\n", " display(selector_pair_rankings_display)" ] }, { "cell_type": "markdown", "id": "theoretical-return-context", "metadata": {}, "source": [ "## Theoretical Return by Pair\n", "\n", "Load `trading_instructions` and calculate theoretical return for each ranked pair. Each pair starts from a fixed `$10,000` theoretical USD base. `TARGET` trades from the current theoretical position to the new target position, where target size is `10000 * strength / reference_price`; `CLOSE` liquidates the open position at the close row's `reference_price`; `HOLD` is ignored.\n", "\n", "`MIN_TARGET_STRENGTH_CHANGE_PCTG` can be raised above `0.0` to skip `TARGET` updates whose absolute percentage strength change is smaller than the threshold since the position was acquired. `num_trades` counts asset-level theoretical trades caused by effective `TARGET` and `CLOSE` rows. `realized_pnl` and `unrealized_pnl` are percentage returns relative to `$10,000`. The displayed dataframe is sorted by total return (`realized_pnl + unrealized_pnl`) ascending." ] }, { "cell_type": "code", "execution_count": null, "id": "target-change-threshold-input", "metadata": {}, "outputs": [], "source": [ "min_target_change_input = widgets.FloatText(\n", " value=0.0,\n", " description=\"Mininal TARGET change (%)\",\n", " step=1.0,\n", " layout=widgets.Layout(width=\"420px\"),\n", " style={\"description_width\": \"190px\"},\n", ")\n", "display(min_target_change_input)" ] }, { "cell_type": "code", "execution_count": null, "id": "load-trading-instructions", "metadata": {}, "outputs": [], "source": [ "conn = connect_selected_database()\n", "try:\n", " trading_instructions = load_trading_instructions(conn)\n", "finally:\n", " conn.close()\n", "\n", "print(f\"Loaded {len(trading_instructions):,} trading instruction rows.\")" ] }, { "cell_type": "code", "execution_count": null, "id": "calculate-pair-theoretical-returns", "metadata": {}, "outputs": [], "source": [ "MIN_TARGET_STRENGTH_CHANGE_PCTG = float(min_target_change_input.value)\n", "\n", "pair_theo_ret = add_total_pnl(\n", " calculate_ranked_pairs_theo_ret(\n", " selector_pair_rankings,\n", " trading_instructions,\n", " min_pctg_change=MIN_TARGET_STRENGTH_CHANGE_PCTG,\n", " )\n", ").sort_values(\n", " [\"total_pnl\", \"pair_name\"],\n", " ascending=[True, True],\n", " kind=\"mergesort\",\n", ").drop(columns=\"total_pnl\").reset_index(drop=True)\n", "\n", "pair_theo_ret_display = format_pair_names_for_display(pair_theo_ret)\n", "\n", "show_interactive_dataframe(\n", " pair_theo_ret_display,\n", " table_id=\"pair-theo-ret-grid\",\n", ")" ] }, { "cell_type": "markdown", "id": "theoretical-return-histogram-context", "metadata": {}, "source": [ "## Total Theoretical Return Distribution\n", "\n", "Plot the distribution of total theoretical return, calculated as `realized_pnl + unrealized_pnl`. Plotly chooses histogram bins automatically." ] }, { "cell_type": "code", "execution_count": null, "id": "plot-total-theoretical-return-histogram", "metadata": {}, "outputs": [], "source": [ "total_pnl_histogram = create_total_pnl_histogram(pair_theo_ret)\n", "\n", "total_pnl_histogram" ] }, { "cell_type": "markdown", "id": "individual-pair-analysis-context", "metadata": {}, "source": [ "## Individual Pair Analysis\n", "\n", "Choose one pair for detailed follow-up analysis. Pair names are sorted alphabetically." ] }, { "cell_type": "code", "execution_count": null, "id": "choose-individual-pair", "metadata": {}, "outputs": [], "source": [ "pair_name_dropdown = create_pair_name_dropdown(selector_pair_rankings)\n", "display(pair_name_dropdown)" ] }, { "cell_type": "code", "execution_count": null, "id": "selected-individual-pair", "metadata": {}, "outputs": [], "source": [ "selected_pair_name = pair_name_dropdown.value\n", "format_pair_name_for_display(selected_pair_name)" ] }, { "cell_type": "markdown", "id": "selected-pair-theo-executions-context", "metadata": {}, "source": [ "### Selected Pair Theoretical Executions\n", "\n", "Create the theoretical asset-level executions used by the PnL calculation for the selected pair. `TARGET` rows trade the position difference from the current theoretical position to the new target position, where target size is `10000 * strength / reference_price`; `CLOSE` rows flatten the current theoretical position. Positive size is `BUY`; negative size is `SELL`; USD value is signed as the opposite cash movement." ] }, { "cell_type": "code", "execution_count": null, "id": "selected-pair-theo-executions", "metadata": {}, "outputs": [], "source": [ "selected_pair_theo_executions = calculate_pair_theo_executions(\n", " selected_pair_name,\n", " trading_instructions,\n", " min_pctg_change=MIN_TARGET_STRENGTH_CHANGE_PCTG,\n", ")\n", "\n", "selected_pair_theo_execution_columns = [\n", " \"time\",\n", " \"asset\",\n", " \"action\",\n", " \"side\",\n", " \"strength\",\n", " \"size\",\n", " \"price\",\n", " \"usd_value\",\n", "]\n", "selected_pair_theo_executions_display = selected_pair_theo_executions.reindex(\n", " columns=selected_pair_theo_execution_columns\n", ")\n", "show_interactive_dataframe(\n", " selected_pair_theo_executions_display,\n", " table_id=\"selected-pair-theo-executions-grid\",\n", ")" ] }, { "cell_type": "markdown", "id": "selected-pair-market-trades-context", "metadata": {}, "source": [ "### Selected Pair Trades on Market Data\n", "\n", "Load full available 1-minute market data for the selected pair's instruments from `ohlcv_1min`, starting at midnight UTC of the trading day inferred from `trading_instructions`. Close prices are shown as relative prices from each instrument's close at that midnight. Theoretical executions are overlaid at their execution `reference_price`, normalized by the same midnight close. Execution markers use execution timestamps directly and do not require a matching OHLCV row." ] }, { "cell_type": "code", "execution_count": null, "id": "selected-pair-market-trades-plot", "metadata": {}, "outputs": [], "source": [ "trading_day_start_ns = infer_trading_day_start_ns(trading_instructions)\n", "\n", "conn = connect_selected_database()\n", "try:\n", " selected_pair_market_data = load_pair_market_data(\n", " conn,\n", " selected_pair_name,\n", " trading_day_start_ns=trading_day_start_ns,\n", " )\n", "finally:\n", " conn.close()\n", "\n", "selected_pair_market_trades_plot = create_pair_trades_market_plot(\n", " selected_pair_name,\n", " selected_pair_market_data,\n", " selected_pair_theo_executions,\n", ")\n", "\n", "selected_pair_market_trades_plot" ] } ], "metadata": { "kernelspec": { "display_name": "python3.12-venv (3.12.13.final.0)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.13" } }, "nbformat": 4, "nbformat_minor": 5 }