commit f598141500f34ce02e9312f6ed657aebf30c102c Author: Yasha Sheynin Date: Wed Apr 16 16:50:59 2025 -0400 initial commit for predictor module diff --git a/gru_sac_predictor/README.md b/gru_sac_predictor/README.md new file mode 100644 index 00000000..c74bb52f --- /dev/null +++ b/gru_sac_predictor/README.md @@ -0,0 +1,141 @@ +# v7 - GRU + Simplified SAC Trading Agent (V6 GRU Adaptation) + +This project implements a cryptocurrency trading system using a GRU model for price prediction and a **Simplified SAC (Soft Actor-Critic)** agent for position sizing. + +The system predicts future *price* using a GRU model adapted from the V6 architecture. It calculates the *predicted percentage return* from this price prediction and estimates prediction *uncertainty* based on the standard deviation of Monte Carlo dropout predictions. These two values (`predicted_return`, `mc_unscaled_std_dev`) form the state input to the SAC reinforcement learning agent, which determines optimal position sizing (-1 to +1). + +The system incorporates efficiency improvements by pre-computing GRU predictions and uncertainties before generating SAC experiences or running the backtest. It includes detailed backtesting, performance reporting, and visualization capabilities. + +## System Design + +The system integrates a GRU predictor and a Simplified SAC agent within a backtesting framework. + +### 1. Data Flow & Processing + +1. **Loading:** Raw 1-minute OHLCV data is loaded from the SQLite database directory specified in `main.py` (e.g., `downloaded_data/`) using `src.data_pipeline.load_data_from_db` which utilizes `src.crypto_db_fetcher.CryptoDBFetcher`. +2. **Splitting:** Data is chronologically split into training (60%), validation (20%), and test (20%) sets using `src.data_pipeline.create_data_pipeline`. +3. **GRU Training / Loading (on Train/Validation Sets):** + * If `TRAIN_GRU_MODEL` is `True`: + * *Preprocessing*: `TradingSystem._preprocess_data_for_gru_training` calculates V6 features plus basic return features (`calculate_v6_features`) on the raw train/val data. It determines the future *price* target (`prediction_horizon` steps ahead) and aligns features, targets (prices), and the *unscaled* starting close prices needed for return calculation. + * *Scaling*: Within `TradingSystem.train_gru`, a `StandardScaler` is fitted *only* on the training features. A `MinMaxScaler` is fitted *only* on the training future *price* targets. Train and validation features/targets are scaled using these fitted scalers. + * *Sequence Creation*: `src.data_pipeline.create_sequences_v2` creates input sequences `(batch, sequence_length, num_features)` and corresponding scaled target prices using the scaled features/targets and the unscaled start prices. + * *Model Training*: `CryptoGRUModel.train` builds the V6-style GRU model (if not already built) and trains it using Mean Squared Error (MSE) loss on the scaled sequences. Callbacks monitor `val_rmse` for early stopping and model checkpointing. The best model (`best_model_reg.keras`) and the fitted scalers (`feature_scaler.joblib`, `y_scaler.joblib`) are saved. + * If `LOAD_EXISTING_SYSTEM` is `True` and `TRAIN_GRU_MODEL` is `False`: Attempts to load a pre-trained GRU model and scalers. If `GRU_MODEL_LOAD_RUN_ID` is set in `main.py`, it loads from that specific run ID's directory (`v7/models/run_`); otherwise, it attempts to load from the default `MODEL_SAVE_PATH` (expecting a `gru_model` subdirectory). +4. **SAC Training (on Validation Set):** + * **Training Loop:** The training process runs for a fixed number of agent update steps (`TOTAL_TRAINING_STEPS`) instead of epochs. + * **Experience Generation** (`TradingSystem.generate_trading_experiences`): + * **Efficiency:** Pre-computes all required GRU outputs (predicted returns, uncertainties) for the entire validation set by calling `CryptoGRUModel.evaluate` *once*. + * **Initial Fill:** Generates an initial set of experiences (`experience_config['initial_experiences']`). Uses the sampling strategy. + * **Sampling (`_sample_experience_indices`):** When generating a specific number of experiences (initial fill or periodic updates), it applies **weighted sampling** (controlled by `recency_bias_strength`) and **stratified sampling** (ensuring minimum ratios `min_uncertainty_ratio`, `min_extreme_return_ratio` of high uncertainty/extreme return examples based on quantiles `high_uncertainty_quantile`, `extreme_return_quantile`) based on parameters in `experience_config`. + * **Experience Format:** Iterates through the (potentially sampled) pre-computed results. Forms the state `s_t = [predicted_return_t, uncertainty_t]`. The SAC agent (`SimplifiedSACTradingAgent.get_action`) provides a *non-deterministic* action `a_t`. The next state `s_{t+1}` is retrieved. A reward `r_t = action * actual_return` is calculated (transaction costs are currently ignored in reward calculation during generation for simplicity). The transition `(s_t, a_t, r_t, s_{t+1}, done=False)` is created. + * **Periodic Updates:** During the main training loop (controlled by `total_training_steps`), new batches of experiences (`experience_config['experiences_per_batch']`) are generated periodically (every `experience_config['batch_generation_interval']` loop steps) using the sampling strategy and added to the replay buffer. + * **Agent Training** (`SimplifiedSACTradingAgent.train`): In each step of the main training loop, the agent performs `experience_config['training_iterations_per_step']` update(s). Batches are sampled from the replay buffer. Actor and Critic networks are updated using the SAC algorithm. The agent uses a standard FIFO circular buffer for experience storage. +5. **Backtesting (on Test Set):** + * *Pre-computation* (`ExtendedBacktester.backtest`): Similar to SAC training, preprocesses the test data, scales it, creates sequences, and calls `CryptoGRUModel.evaluate` *once* to get all predicted returns and uncertainties for the test set. + * *Iteration*: Steps chronologically through the pre-computed results. + * *State Generation*: Retrieves `predicted_return` and `uncertainty_sigma` from the pre-computed arrays to form the state `s_t`. + * *Action Selection*: The trained `SimplifiedSACTradingAgent` selects a *deterministic* action `a_t`. + * *Portfolio Simulation*: Calculates PnL based on the previous position held (`current_position`), the actual return over the step, and subtracts transaction costs based on the change in position (`abs(action - current_position)`). + * *Logging*: Records detailed metrics, trade history, and timestamps. +6. **Evaluation:** + * *Performance Metrics*: `ExtendedBacktester._calculate_performance_metrics` computes overall portfolio metrics (Sharpe, Sortino, Drawdown, correlations, etc.) and Buy & Hold benchmark metrics. + * *Visualization*: `ExtendedBacktester.plot_results` generates a 3-panel plot: GRU Predictions vs Actual Price (with uncertainty), SAC Actions (Position Size), and Portfolio Value vs Buy & Hold (with trade markers). + * *Reporting*: `ExtendedBacktester.generate_performance_report` creates a detailed Markdown report. + +### 2. Core Components & Inputs/Outputs + +* **`src.crypto_db_fetcher.CryptoDBFetcher`**: Loads and resamples data from SQLite DBs. +* **`src.data_pipeline`**: Functions for DB loading, data splitting, sequence creation. +* **`src.trading_system.calculate_v6_features`**: Calculates features (TA-Lib based V6 set + past returns). +* **`src.trading_system._preprocess_data_for_gru_training`**: Prepares features, future price targets, and start prices. +* **`src.gru_predictor.CryptoGRUModel`**: (V6 Adaptation) + * `train()`: Trains the GRU price prediction model. Saves model (`.keras`) and scalers (`.joblib`). + * `evaluate()`: Performs standard prediction and MC dropout inference. Returns dict including `pred_percent_change`, `mc_unscaled_std_dev`, `predicted_unscaled_prices`, `true_unscaled_prices`. +* **`src.sac_agent_simplified.SimplifiedSACTradingAgent`**: (V7 Simplified) + * **Goal:** Learns a policy mapping state to optimal position size (-1.0 to +1.0). Optimized for faster training. + * **State Input:** 2-element array `[predicted_return, mc_unscaled_std_dev]`. + * **Action Output:** Float between -1.0 and +1.0. + * `get_action()`: Selects action (stochastic or deterministic). Adds uncertainty-scaled noise during exploration. + * `store_transition()`: Adds experience to internal NumPy buffer. + * `train()`: Updates agent using buffer samples (internally handles batch size). Uses `@tf.function` for performance. + * `save()` / `load()`: Handles Actor/Critic weights (`.weights.h5`), potentially `alpha.npy`. + * **Note:** Models and optimizers are built explicitly during `__init__` to prevent TensorFlow graph mode issues. +* **`src.trading_system.TradingSystem`**: Integrates GRU and SAC. Manages training pipelines, experience generation (including advanced sampling). +* **`src.trading_system.ExtendedBacktester`**: Performs efficient backtesting using pre-computed GRU outputs, calculates metrics, plots results, generates reports. + +### 3. Model Architectures + +* **GRU (`src.gru_predictor.CryptoGRUModel._build_model`)**: V6 Architecture. + * Input -> GRU(100) -> Dropout(0.2) -> Dense(1, linear). + * Compiled with Adam (LR=0.001), MSE loss. +* **Simplified SAC (`src.sac_agent_simplified.SimplifiedSACTradingAgent`)**: + * **Actor Network**: MLP `(state_dim=2)` -> Dense(64, relu) -> [BN] -> Dense(64, relu) -> [BN] -> [Residual] -> Dense(1, tanh). + * **Critic Network (x2)**: MLP `(state_dim=2 + action_dim=1)` -> Dense(64, relu) -> [BN] -> Dense(64, relu) -> [BN] -> [Residual] -> Dense(1, linear). + * **Algorithm**: Implements SAC with Clipped Double-Q, fixed alpha (tunable via `SAC_ALPHA`), faster learning rates, smaller networks/buffer, optional Batch Normalization / Residual connections. Uses Huber loss for critics. No distributional critics. `@tf.function` used for update steps. + +### 4. Features + +The GRU model uses the V6 feature set plus basic past returns: +* **TA-Lib Indicators & Derived Indicators:** SMA, EMA, MACD, SAR, ADX, RSI, Stochastics, WILLR, ROC, CCI, BBands, ATR, OBV, CMF, etc. (Requires TA-Lib installation). Fallback calculations for SMA, EMA, RSI if TA-Lib is unavailable. +* **Custom Crypto Features:** Parkinson Volatility, Garman-Klass Volatility, VWAP ratios, Volume Intensity, Wick Ratios. +* **Past Returns:** `return_1m`, `return_5m`, `return_15m`, `return_60m` (percentage change). +* **Scaling:** Features scaled with `StandardScaler` (fitted on train). Target variable (future price) scaled with `MinMaxScaler` (fitted on train). + +### 5. Evaluation + +* **GRU Model:** Evaluated using RMSE loss on validation set. Callbacks monitor `val_rmse`. Plots compare predicted vs actual price. +* **SAC Agent & Overall System:** Evaluated via the `ExtendedBacktester` metrics (Sharpe, Sortino, Max Drawdown, correlations, etc.), plots (Portfolio vs B&H, Actions), and a final Markdown report. + +## File Structure + +- `data/`: *Not used by default if loading from DB.* +- `downloaded_data/`: **Place your V6 SQLite database files here.** (Or update `DB_DIR` in `main.py`). +- `models/`: Trained models (GRU + scalers, SAC weights) saved here under `run_/` directories by default. +- `results/`: Backtest results (plots, reports, config) saved here under `/` directories. +- `logs/`: Log files saved here under `/` directories. +- `src/`: Core Python modules. + - `crypto_db_fetcher.py`: Class for fetching data from SQLite DBs. + - `data_pipeline.py`: DB loading function, data splitting, sequence creation. + - `gru_predictor.py`: V6-style GRU model for price regression and MC uncertainty. + - `sac_agent_simplified.py`: Simplified SAC agent implementation (V7.5+). + - `sac_agent.py`: Original SAC agent implementation (potentially outdated). + - `trading_system.py`: Integration class, feature calculation, scaling, experience generation, `ExtendedBacktester` class. +- `main.py`: Main script using DB loading, orchestrates training and backtesting. +- `requirements.txt`: Dependencies. +- `v7_instructions.txt`: Design notes for Simplified SAC. +- `experience_instructions.txt`: Design notes for experience generation. +- `README.md`: This file. + +## Setup + +1. **Data:** Place your V6 `downloaded_data` directory containing the SQLite files relative to the `v7` project root, or update the `DB_DIR` variable in `main.py` to point to the correct location. +2. **Dependencies:** Install required packages: + ```bash + pip install -r requirements.txt + ``` + *Strongly Recommended:* Install TA-Lib for the full feature set. See TA-Lib installation guides for your OS. +3. **Configuration:** Review and adjust parameters in `main.py`. Key parameters include: + * `DB_DIR`, `TICKER`, `EXCHANGE`, `START_DATE`, `END_DATE`, `INTERVAL` + * Model hyperparameters (GRU and SAC sections) + * Control Flags: `LOAD_EXISTING_SYSTEM`, `TRAIN_GRU_MODEL`, `TRAIN_SAC_AGENT`, `LOAD_SAC_AGENT` + * Loading Specific Models: `GRU_MODEL_LOAD_RUN_ID` (set to a specific run ID string like `'YYYYMMDD_HHMMSS'` to load that GRU model from `v7/models/run_/`). Note: This expects GRU and SAC files to be in the *same* directory if loading this way. + * SAC Training: `TOTAL_TRAINING_STEPS` defines the length of SAC training (number of agent `train()` calls). + * Experience Generation: `experience_config` dictionary controls initial fill, periodic updates, and sampling strategies (recency bias, stratification for uncertainty/extreme returns). + * Backtesting: `INITIAL_CAPITAL`, `TRANSACTION_COST`. +4. **Run:** Execute from the project root directory (containing the `v7` folder): + ```bash + python -m v7.main + ``` + Output files (logs, models, plots, report) will be generated in `v7/logs/`, `v7/models/`, and `v7/results/` within run-specific subdirectories. + +## Reporting + +The report generated by the `ExtendedBacktester` includes performance metrics, correlation analyses, and configuration details. Key metrics include: + +* Total/Annualized Return +* Sharpe & Sortino Ratios +* Volatility & Max Drawdown +* Buy & Hold Comparison +* Position/Prediction Accuracy +* Prediction/Position/Uncertainty Correlations +* Total Trades \ No newline at end of file diff --git a/gru_sac_predictor/__pycache__/main.cpython-312.pyc b/gru_sac_predictor/__pycache__/main.cpython-312.pyc new file mode 100644 index 00000000..04fa44fa Binary files /dev/null and b/gru_sac_predictor/__pycache__/main.cpython-312.pyc differ diff --git a/gru_sac_predictor/logs/20250416_142744/main_v7_20250416_142744.log b/gru_sac_predictor/logs/20250416_142744/main_v7_20250416_142744.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_144232/main_v7_20250416_144232.log b/gru_sac_predictor/logs/20250416_144232/main_v7_20250416_144232.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_144418/main_v7_20250416_144418.log b/gru_sac_predictor/logs/20250416_144418/main_v7_20250416_144418.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_144645/main_v7_20250416_144645.log b/gru_sac_predictor/logs/20250416_144645/main_v7_20250416_144645.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_144757/main_v7_20250416_144757.log b/gru_sac_predictor/logs/20250416_144757/main_v7_20250416_144757.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_144847/main_v7_20250416_144847.log b/gru_sac_predictor/logs/20250416_144847/main_v7_20250416_144847.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_145035/main_v7_20250416_145035.log b/gru_sac_predictor/logs/20250416_145035/main_v7_20250416_145035.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_145128/main_v7_20250416_145128.log b/gru_sac_predictor/logs/20250416_145128/main_v7_20250416_145128.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_150616/main_v7_20250416_150616.log b/gru_sac_predictor/logs/20250416_150616/main_v7_20250416_150616.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_150829/main_v7_20250416_150829.log b/gru_sac_predictor/logs/20250416_150829/main_v7_20250416_150829.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_150924/main_v7_20250416_150924.log b/gru_sac_predictor/logs/20250416_150924/main_v7_20250416_150924.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_151322/main_v7_20250416_151322.log b/gru_sac_predictor/logs/20250416_151322/main_v7_20250416_151322.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_151849/main_v7_20250416_151849.log b/gru_sac_predictor/logs/20250416_151849/main_v7_20250416_151849.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_152415/main_v7_20250416_152415.log b/gru_sac_predictor/logs/20250416_152415/main_v7_20250416_152415.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_153132/main_v7_20250416_153132.log b/gru_sac_predictor/logs/20250416_153132/main_v7_20250416_153132.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_153846/main_v7_20250416_153846.log b/gru_sac_predictor/logs/20250416_153846/main_v7_20250416_153846.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_154636/main_v7_20250416_154636.log b/gru_sac_predictor/logs/20250416_154636/main_v7_20250416_154636.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_162528/main_v7_20250416_162528.log b/gru_sac_predictor/logs/20250416_162528/main_v7_20250416_162528.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_162624/main_v7_20250416_162624.log b/gru_sac_predictor/logs/20250416_162624/main_v7_20250416_162624.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_162718/main_v7_20250416_162718.log b/gru_sac_predictor/logs/20250416_162718/main_v7_20250416_162718.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_162921/main_v7_20250416_162921.log b/gru_sac_predictor/logs/20250416_162921/main_v7_20250416_162921.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_163030/main_v7_20250416_163030.log b/gru_sac_predictor/logs/20250416_163030/main_v7_20250416_163030.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_163440/main_v7_20250416_163440.log b/gru_sac_predictor/logs/20250416_163440/main_v7_20250416_163440.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_164324/main_20250416_164324.log b/gru_sac_predictor/logs/20250416_164324/main_20250416_164324.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_164410/main_20250416_164410.log b/gru_sac_predictor/logs/20250416_164410/main_20250416_164410.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_164547/main_20250416_164547.log b/gru_sac_predictor/logs/20250416_164547/main_20250416_164547.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/20250416_164726/main_20250416_164726.log b/gru_sac_predictor/logs/20250416_164726/main_20250416_164726.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/logs/main_v7.log b/gru_sac_predictor/logs/main_v7.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/main.py b/gru_sac_predictor/main.py new file mode 100644 index 00000000..ee888cee --- /dev/null +++ b/gru_sac_predictor/main.py @@ -0,0 +1,465 @@ +import pandas as pd +import numpy as np +import matplotlib.pyplot as plt +import os +from datetime import datetime +import warnings +import logging +import sys +import json + +# --- Generate Run ID --- +run_id = datetime.now().strftime("%Y%m%d_%H%M%S") + +# Import components +# V7 Update: Import load_data_from_db +from .src.data_pipeline import create_data_pipeline, load_data_from_db +from .src.trading_system import TradingSystem, ExtendedBacktester, plot_sac_training_history +# V7.3 Fix: Add missing imports +# V7-V6 Final Update: Import CryptoGRUModel +from .src.gru_predictor import CryptoGRUModel +# V7.5 Import the simplified agent +from .src.sac_agent_simplified import SimplifiedSACTradingAgent +# GRU and SAC classes are implicitly imported via TradingSystem + +# --- Base Output Directories --- +BASE_RESULTS_DIR = "gru_sac_predictor/results" +BASE_LOGS_DIR = "gru_sac_predictor/logs" +BASE_MODELS_DIR = "gru_sac_predictor/models" + +# --- Run Specific Directories --- +RUN_RESULTS_DIR = os.path.join(BASE_RESULTS_DIR, run_id) +RUN_LOGS_DIR = os.path.join(BASE_LOGS_DIR, run_id) +RUN_MODELS_DIR = os.path.join(BASE_MODELS_DIR, f"run_{run_id}") + +# --- Logging Setup --- +log_format = '%(asctime)s - %(name)s - %(levelname)s - %(message)s' +# Ensure logs directory exists +os.makedirs(RUN_LOGS_DIR, exist_ok=True) +log_file_path = os.path.join(RUN_LOGS_DIR, f"main_{run_id}.log") # Removed _v7 +logging.basicConfig( + level=logging.INFO, + format=log_format, + handlers=[ + logging.FileHandler(log_file_path, mode='a'), # Use path variable + logging.StreamHandler(sys.stdout) + ] +) +logger = logging.getLogger(__name__) + +# --- Configuration --- +# V7 Update: Add DB parameters +DB_DIR = '../downloaded_data' # V7 Fix: Point to correct relative path for V6 data +TICKER = 'BTC-USD' # Example ticker +EXCHANGE = 'COINBASE' # Example exchange +START_DATE = '2025-03-01' # Example start date - NOTE: VERY SHORT! +END_DATE = '2025-03-10' # Example end date - NOTE: VERY SHORT! +INTERVAL = '1min' # Data interval to fetch and use + +MODEL_SAVE_PATH = RUN_MODELS_DIR # Use run-specific directory +# Updated paths to use RUN_RESULTS_DIR and include run_id +RESULTS_PLOT_PATH = os.path.join(RUN_RESULTS_DIR, f'backtest_results_{run_id}.png') # Removed _v7 +REPORT_SAVE_PATH = os.path.join(RUN_RESULTS_DIR, f'backtest_performance_report_{run_id}.md') # Removed _v7 +# GRU_PLOT_PATH = 'gru_performance_v7.png' # Not used directly in main + +# V7.6 Add specific run ID for loading GRU model +GRU_MODEL_LOAD_RUN_ID = '20250416_142744' # Set this to a specific 'YYYYMMDD_HHMMSS' string to load that GRU model + +# Data split ratios +TRAIN_RATIO = 0.6 +VALIDATION_RATIO = 0.2 + +# Model/Training Parameters (V7.3) +GRU_LOOKBACK = 60 +GRU_PREDICTION_HORIZON = 1 +GRU_EPOCHS = 20 +GRU_BATCH_SIZE = 32 # Updated default +GRU_PATIENCE = 10 # Updated default +GRU_LR_PATIENCE = 10 # Updated default +GRU_LR_FACTOR = 0.5 # Updated default +GRU_RETURN_SCALE = 0.03 # Updated default + +# SAC Parameters (V7.5 - Simplified Agent) +SAC_STATE_DIM = 5 # [pred_return, uncertainty, z, momentum_5, volatility_20] - Updated from 2 +SAC_HIDDEN_SIZE = 64 +SAC_GAMMA = 0.97 +SAC_TAU = 0.02 +# SAC_ALPHA = 0.1 # Removed - Will use automatic tuning +SAC_ACTOR_LR = 3e-4 # Lowered from 5e-4 +SAC_CRITIC_LR = 5e-4 # Lowered from 8e-4 +SAC_BATCH_SIZE = 64 +SAC_BUFFER_MAX_SIZE = 20000 +SAC_MIN_BUFFER_SIZE = 1000 +SAC_UPDATE_INTERVAL = 1 +SAC_TARGET_UPDATE_INTERVAL = 2 +SAC_GRADIENT_CLIP = 1.0 +SAC_REWARD_SCALE = 2.0 # Decreased from 10.0 +SAC_USE_BATCH_NORM = True +SAC_USE_RESIDUAL = True +SAC_MODEL_DIR = 'models/simplified_sac' # Default dir within the agent class +SAC_EPOCHS = 50 # Keep this from previous config for training loop control + +# V7.9 Experience Generation Config (Based on instructions.txt) +# TOTAL_TRAINING_STEPS = 1000 # Removed - Not used in current training loop +experience_config = { + # Basic setup + 'initial_experiences': 3000, # Start with this many experiences + 'experiences_per_batch': 64, # Generate this many in each new batch + 'batch_generation_interval': 500, # Generate a new batch every N training steps + + # Distribution control (Flags for future implementation in generate_trading_experiences) + 'balance_market_regimes': False, # Not implemented + 'recency_bias_strength': 0.5, # 0 = uniform, >0 weights recent data more + 'high_uncertainty_quantile': 0.75, # Threshold for high uncertainty + 'extreme_return_quantile': 0.1, # Threshold for extreme returns (upper/lower) + 'min_uncertainty_ratio': 0.2, # Min % of samples with high uncertainty + 'min_extreme_return_ratio': 0.1, # Min % of samples with extreme returns + + # Efficient processing + 'use_parallel_generation': False, # Not implemented + 'precompute_all_gru_outputs': True, # Already implemented + 'buffer_update_strategy': 'fifo', # Agent currently uses FIFO + + # Training optimization + 'training_iterations_per_step': 1, # Number of agent.train calls per main loop step + # Max/Min buffer size are defined by the agent itself now +} + +# Backtesting Parameters +INITIAL_CAPITAL = 10000.0 +TRANSACTION_COST = 0.0005 +# V7.12 Add Opportunity Cost Penalty Parameters +OPPORTUNITY_COST_PENALTY_FACTOR = 0.0 # How much to penalize missed high returns - Disabled (was 1.0) +HIGH_RETURN_THRESHOLD = 0.002 # Actual return magnitude threshold to trigger penalty check +ACTION_TOLERANCE = 0.3 # Action magnitude below which penalty applies if return threshold met - Lowered from 0.5 +# RISK_PENALTY_FACTOR = 0.0 # Removed as state reverted + +# Control Flags +LOAD_EXISTING_SYSTEM = True +TRAIN_GRU_MODEL = False +TRAIN_SAC_AGENT = True # V7.8 Set to True to train SAC +LOAD_SAC_AGENT = False # V7.8 Set to False to avoid loading SAC +RUN_BACKTEST = True +GENERATE_PLOTS = True +GENERATE_REPORT = True +# --- End Configuration --- + +def main(): + # Access config variables defined at module level + global LOAD_EXISTING_SYSTEM, TRAIN_GRU_MODEL, TRAIN_SAC_AGENT, LOAD_SAC_AGENT + + logger.info(f"--- Starting GRU+SAC Trading System Pipeline (Run ID: {run_id}) ---") # Removed V7 + + # Ensure results directory exists + os.makedirs(RUN_RESULTS_DIR, exist_ok=True) + # Ensure base models directory exists (RUN_MODELS_DIR created later if training) + os.makedirs(BASE_MODELS_DIR, exist_ok=True) + + # LOAD_EXISTING_SYSTEM is now declared global before use here + # --- Save Configuration --- + config_to_save = { + "run_id": run_id, + "db_dir": DB_DIR, + "ticker": TICKER, + "exchange": EXCHANGE, + "start_date": START_DATE, + "end_date": END_DATE, + "interval": INTERVAL, + "model_save_path": MODEL_SAVE_PATH, + "results_plot_path": RESULTS_PLOT_PATH, + "report_save_path": REPORT_SAVE_PATH, + "train_ratio": TRAIN_RATIO, + "validation_ratio": VALIDATION_RATIO, + "gru_lookback": GRU_LOOKBACK, + "gru_prediction_horizon": GRU_PREDICTION_HORIZON, + "gru_epochs": GRU_EPOCHS, + "gru_batch_size": GRU_BATCH_SIZE, + "gru_patience": GRU_PATIENCE, + "gru_lr_factor": GRU_LR_FACTOR, + "gru_return_scale": GRU_RETURN_SCALE, + "gru_model_load_run_id": GRU_MODEL_LOAD_RUN_ID, + "sac_state_dim": SAC_STATE_DIM, + "sac_hidden_size": SAC_HIDDEN_SIZE, + "sac_gamma": SAC_GAMMA, + "sac_tau": SAC_TAU, + "sac_actor_lr": SAC_ACTOR_LR, + "sac_critic_lr": SAC_CRITIC_LR, + "sac_batch_size": SAC_BATCH_SIZE, + "sac_buffer_max_size": SAC_BUFFER_MAX_SIZE, + "sac_min_buffer_size": SAC_MIN_BUFFER_SIZE, + "sac_update_interval": SAC_UPDATE_INTERVAL, + "sac_target_update_interval": SAC_TARGET_UPDATE_INTERVAL, + "sac_gradient_clip": SAC_GRADIENT_CLIP, + "sac_reward_scale": SAC_REWARD_SCALE, + "sac_use_batch_norm": SAC_USE_BATCH_NORM, + "sac_use_residual": SAC_USE_RESIDUAL, + "sac_model_dir": SAC_MODEL_DIR, + "sac_epochs": SAC_EPOCHS, + "experience_config": experience_config, + "initial_capital": INITIAL_CAPITAL, + "transaction_cost": TRANSACTION_COST, + # V7.12 Add new params to saved config + "opportunity_cost_penalty_factor": OPPORTUNITY_COST_PENALTY_FACTOR, + "high_return_threshold": HIGH_RETURN_THRESHOLD, + "action_tolerance": ACTION_TOLERANCE, + "load_existing_system": LOAD_EXISTING_SYSTEM, + "train_gru_model": TRAIN_GRU_MODEL, + "train_sac_agent": TRAIN_SAC_AGENT, + "load_sac_agent": LOAD_SAC_AGENT, + "run_backtest": RUN_BACKTEST, + "generate_plots": GENERATE_PLOTS, + "generate_report": GENERATE_REPORT + } + config_save_path = os.path.join(RUN_RESULTS_DIR, f'config_{run_id}.json') + try: + with open(config_save_path, 'w') as f: + json.dump(config_to_save, f, indent=4) + logger.info(f"Run configuration saved to {config_save_path}") + except Exception as e: + logger.error(f"Failed to save run configuration: {e}") + # --- End Save Configuration --- + + # 1. Load Data from Database + logger.info(f"Loading data from DB: {TICKER}/{EXCHANGE} ({START_DATE}-{END_DATE}) @ {INTERVAL}") + data = load_data_from_db( + db_dir=DB_DIR, + ticker=TICKER, + exchange=EXCHANGE, + start_date=START_DATE, + end_date=END_DATE, + interval=INTERVAL + ) + + if data.empty: + logger.error("Failed to load data from database. Please check DB_DIR and parameters. Aborting.") + return + + # --- Re-inserted Steps Start --- + # Basic Data Validation (Timestamp index assumed from load_data_from_db) + if 'close' not in data.columns: # Check essential columns + raise ValueError("Loaded data must contain 'close' column.") + logger.info(f"Data loaded: {len(data)} rows, from {data.index.min()} to {data.index.max()}") + initial_len = len(data); data.dropna(subset=['open', 'high', 'low', 'close', 'volume'], inplace=True) + if len(data) < initial_len: logger.info(f"Dropped {initial_len - len(data)} NaN rows.") + if len(data) < GRU_LOOKBACK * 3: raise ValueError(f"Insufficient data ({len(data)} rows) for lookback/splits.") + + # Add cyclical features immediately + logger.info("Calculating cyclical time features (hour_sin, hour_cos)...") + timestamp_source = None + if isinstance(data.index, pd.DatetimeIndex): + timestamp_source = data.index + logger.debug("Using index for hour features.") + elif 'timestamp' in data.columns and pd.api.types.is_datetime64_any_dtype(data['timestamp']): + timestamp_source = pd.to_datetime(data['timestamp']) + logger.debug("Using 'timestamp' column for hour features.") + elif 'date' in data.columns and pd.api.types.is_datetime64_any_dtype(data['date']): + timestamp_source = pd.to_datetime(data['date']) + logger.debug("Using 'date' column for hour features.") + + if timestamp_source is not None: + data['hour_sin'] = np.sin(2 * np.pi * timestamp_source.hour / 24) + data['hour_cos'] = np.cos(2 * np.pi * timestamp_source.hour / 24) + logger.info("Added hour_sin/hour_cos to main dataframe.") + else: + logger.warning("Could not find suitable timestamp source. Setting hour_sin/cos defaults (0.0, 1.0).") + data['hour_sin'] = 0.0 + data['hour_cos'] = 1.0 # Default to cos(0) = 1 + + # 2. Split Data Chronologically + logger.info("Splitting data...") + test_ratio = round(1.0 - TRAIN_RATIO - VALIDATION_RATIO, 2) + if test_ratio <= 0: raise ValueError("Train+Validation ratios must sum to < 1.") + train_data, val_data, test_data = create_data_pipeline(data, [TRAIN_RATIO, VALIDATION_RATIO, test_ratio]) + if len(train_data) < GRU_LOOKBACK or len(val_data) < GRU_LOOKBACK or len(test_data) < GRU_LOOKBACK: + warnings.warn(f"Splits smaller than GRU lookback ({GRU_LOOKBACK}). Backtesting might fail.") + + # 3. Initialize Trading System + logger.info("Initializing Trading System...") + trading_system = TradingSystem( + gru_model=CryptoGRUModel(), # Instantiate the correct model + sac_agent=SimplifiedSACTradingAgent( + state_dim=SAC_STATE_DIM, + hidden_size=SAC_HIDDEN_SIZE, + gamma=SAC_GAMMA, + tau=SAC_TAU, + actor_lr=SAC_ACTOR_LR, + critic_lr=SAC_CRITIC_LR, + batch_size=SAC_BATCH_SIZE, + buffer_max_size=SAC_BUFFER_MAX_SIZE, + min_buffer_size=SAC_MIN_BUFFER_SIZE, + update_interval=SAC_UPDATE_INTERVAL, + target_update_interval=SAC_TARGET_UPDATE_INTERVAL, + gradient_clip=SAC_GRADIENT_CLIP, + reward_scale=SAC_REWARD_SCALE, + use_batch_norm=SAC_USE_BATCH_NORM, + use_residual=SAC_USE_RESIDUAL, + model_dir=os.path.join(MODEL_SAVE_PATH, 'sac_agent') # Point to subfolder within run + ), # Pass the configured agent + gru_lookback=GRU_LOOKBACK + ) + + # --- Model Loading/Training --- + gru_loaded = False; sac_loaded = False + if LOAD_EXISTING_SYSTEM: + load_base_path = MODEL_SAVE_PATH + logger.info(f"Attempting to load existing system components...") + logger.info(f"Base path for loading: {load_base_path}") + + gru_model_load_dir = None + sac_model_load_dir = None + if GRU_MODEL_LOAD_RUN_ID: + gru_model_load_dir = os.path.join(BASE_MODELS_DIR, f'run_{GRU_MODEL_LOAD_RUN_ID}') + logger.info(f"Using specific GRU load path based on run ID: {gru_model_load_dir}") + if LOAD_SAC_AGENT: + sac_model_load_dir = os.path.join(BASE_MODELS_DIR, f'run_{GRU_MODEL_LOAD_RUN_ID}') + logger.info(f"Using specific SAC load path based on GRU run ID (LOAD_SAC_AGENT=True): {sac_model_load_dir}") + else: + sac_model_load_dir = os.path.join(MODEL_SAVE_PATH, 'sac_agent') + logger.info(f"Defaulting SAC path to current run (LOAD_SAC_AGENT=False): {sac_model_load_dir}") + elif os.path.exists(load_base_path): + gru_model_load_dir = os.path.join(load_base_path, 'gru_model') + sac_model_load_dir = os.path.join(load_base_path, 'sac_agent') + logger.info(f"Using GRU load path based on MODEL_SAVE_PATH: {gru_model_load_dir}") + logger.info(f"Using SAC load path based on MODEL_SAVE_PATH: {sac_model_load_dir}") + else: + logger.warning(f"LOAD_EXISTING_SYSTEM is True, but MODEL_SAVE_PATH does not exist: {load_base_path}. Cannot determine model paths.") + LOAD_EXISTING_SYSTEM = False + + if LOAD_EXISTING_SYSTEM: + try: + if gru_model_load_dir and os.path.isdir(gru_model_load_dir): + logger.info(f"Found GRU model directory: {gru_model_load_dir}. Loading...") + if trading_system.gru_model is None: trading_system.gru_model = CryptoGRUModel() + if trading_system.gru_model.load(gru_model_load_dir): + logger.info("GRU model loaded successfully.") + gru_loaded = True + trading_system.feature_scaler = trading_system.gru_model.feature_scaler + trading_system.y_scaler = trading_system.gru_model.y_scaler + logger.info("Scalers propagated from loaded GRU model.") + else: logger.warning(f"GRU model directory found, but loading failed.") + elif gru_model_load_dir: logger.warning(f"GRU model directory specified or derived, but not found at {gru_model_load_dir}. GRU model cannot be loaded.") + else: logger.warning("GRU model path could not be determined. GRU model cannot be loaded.") + + if LOAD_SAC_AGENT: + if sac_model_load_dir and os.path.isdir(sac_model_load_dir): + logger.info(f"Found SAC model directory: {sac_model_load_dir}. Loading (LOAD_SAC_AGENT=True)...") + if trading_system.sac_agent is None: + trading_system.sac_agent = SimplifiedSACTradingAgent(state_dim=SAC_STATE_DIM, model_dir=sac_model_load_dir) + if trading_system.sac_agent.load(sac_model_load_dir): + logger.info("SAC agent loaded successfully.") + sac_loaded = True + else: logger.warning(f"SAC model directory found, but loading failed.") + elif sac_model_load_dir: logger.warning(f"SAC agent model directory derived, but not found at {sac_model_load_dir}. SAC agent cannot be loaded (LOAD_SAC_AGENT=True).") + else: logger.info("Skipping SAC agent loading (LOAD_SAC_AGENT=False).") + + if gru_loaded: TRAIN_GRU_MODEL = False + if sac_loaded: TRAIN_SAC_AGENT = False; LOAD_SAC_AGENT = True + + except Exception as e: + logger.warning(f"Could not load existing system components: {e}. Proceeding based on training flags.") + gru_loaded = False; sac_loaded = False + TRAIN_GRU_MODEL = True; TRAIN_SAC_AGENT = True; LOAD_SAC_AGENT = False + + elif LOAD_EXISTING_SYSTEM: pass + else: logger.info("LOAD_EXISTING_SYSTEM=False. Proceeding with training flags.") + + # --- Sanity Check After Loading --- + if not gru_loaded and not TRAIN_GRU_MODEL: + logger.error("Critical Error: GRU model was not loaded and TRAIN_GRU_MODEL is False. Cannot proceed.") + return + if not sac_loaded and not TRAIN_SAC_AGENT: + if RUN_BACKTEST: + logger.error("Critical Error: SAC agent was not loaded and TRAIN_SAC_AGENT is False. Aborting because RUN_BACKTEST is True.") + return + else: logger.warning("Proceeding without a functional SAC agent as RUN_BACKTEST is False.") + + # Train GRU Model (if flag is set and not loaded) + if TRAIN_GRU_MODEL: + logger.info("--- Training GRU Model --- ") + gru_save_dir = MODEL_SAVE_PATH + history = trading_system.train_gru( + train_data=train_data, val_data=val_data, + prediction_horizon=GRU_PREDICTION_HORIZON, + epochs=GRU_EPOCHS, batch_size=GRU_BATCH_SIZE, + patience=GRU_PATIENCE, + model_save_dir=gru_save_dir + ) + if history is None: logger.error("GRU Training failed. Aborting."); return + logger.info("--- GRU Model Training Finished --- ") + elif not gru_loaded: logger.error("GRU Model must be trained or loaded."); return + else: logger.info("Skipping GRU training (already loaded).") + + # Train SAC Agent (if flag is set and not loaded) + if TRAIN_SAC_AGENT: + logger.info("--- Training SAC Agent --- ") + if not trading_system.gru_model or not (trading_system.gru_model.is_trained or trading_system.gru_model.is_loaded): + logger.error("Cannot train SAC: GRU model not ready."); return + + if trading_system.sac_agent is None: logger.error("SAC Agent instance is missing in the trading system before training."); return + trading_system.sac_agent.model_dir = os.path.join(MODEL_SAVE_PATH, 'sac_agent') + logger.info(f"Ensured SAC agent model save dir is set to: {trading_system.sac_agent.model_dir}") + + sac_history = trading_system.train_sac( + val_data=val_data, + epochs=SAC_EPOCHS, + batch_size=SAC_BATCH_SIZE, + transaction_cost=TRANSACTION_COST, + prediction_horizon=GRU_PREDICTION_HORIZON + ) + logger.info("Finished training SAC agent.") + + if sac_history is not None: + sac_save_dir = os.path.join(MODEL_SAVE_PATH, 'sac_agent') + logger.info(f"Saving Simplified SAC agent to {sac_save_dir}") + trading_system.sac_agent.save(sac_save_dir) + + if sac_history: + sac_plot_save_path = os.path.join(RUN_RESULTS_DIR, f'sac_training_history_{run_id}.png') + logger.info(f"Plotting SAC training history to {sac_plot_save_path}...") + try: plot_sac_training_history(sac_history, save_path=sac_plot_save_path) + except Exception as plot_e: logger.error(f"Failed to plot SAC training history: {plot_e}", exc_info=True) + else: logger.warning("SAC training finished, but no history data returned for plotting.") + + elif not sac_loaded and LOAD_SAC_AGENT: + # This block handles loading SAC if LOAD_EXISTING_SYSTEM was False but LOAD_SAC_AGENT was True (unlikely case) + if trading_system.sac_agent is None: trading_system.sac_agent = SimplifiedSACTradingAgent(state_dim=SAC_STATE_DIM) + sac_load_path = os.path.join(MODEL_SAVE_PATH, 'sac_agent') # Load from current run models + if os.path.isdir(sac_load_path): + logger.info(f"Attempting to load SAC weights from {sac_load_path} (LOAD_SAC_AGENT=True)...") + try: trading_system.sac_agent.load(sac_load_path); logger.info("SAC weights loaded."); sac_loaded = True + except Exception as e: logger.warning(f"Could not load SAC weights: {e}") + else: logger.warning(f"LOAD_SAC_AGENT=True but no weights found at {sac_load_path}.") + elif not sac_loaded: logger.warning("SAC Agent not trained or loaded.") + else: logger.info("Skipping SAC training (already loaded).") + + # 5. Backtest on Test Data + if RUN_BACKTEST: + logger.info("--- Running Extended Backtest --- ") + if not trading_system.gru_model or not (trading_system.gru_model.is_trained or trading_system.gru_model.is_loaded): + logger.error("Cannot backtest: GRU model not ready."); return + if not trading_system.sac_agent: logger.error("Cannot backtest: SAC Agent not initialized."); return + + instrument_label = f"{TICKER}/{EXCHANGE}" + backtester = ExtendedBacktester( + trading_system, + initial_capital=INITIAL_CAPITAL, + transaction_cost=TRANSACTION_COST, + instrument_label=instrument_label + ) + backtest_results = backtester.backtest(test_data, verbose=True) + + # 6. Generate Plots and Report + if GENERATE_PLOTS: + logger.info(f"Generating overall performance plot: {RESULTS_PLOT_PATH}...") + backtester.plot_results(save_path=RESULTS_PLOT_PATH) + if GENERATE_REPORT: + logger.info(f"Generating performance report: {REPORT_SAVE_PATH}...") + backtester.generate_performance_report(report_path=REPORT_SAVE_PATH) + else: + logger.info("Skipping backtesting.") + # --- Re-inserted Steps End --- + + logger.info("--- GRU+SAC Pipeline Finished --- ") + +if __name__ == "__main__": + main() \ No newline at end of file diff --git a/gru_sac_predictor/main_v7.log b/gru_sac_predictor/main_v7.log new file mode 100644 index 00000000..e69de29b diff --git a/gru_sac_predictor/models/run_20250416_142744/actor.weights.h5 b/gru_sac_predictor/models/run_20250416_142744/actor.weights.h5 new file mode 100644 index 00000000..0f80090e Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_142744/actor.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_142744/best_model_reg.keras b/gru_sac_predictor/models/run_20250416_142744/best_model_reg.keras new file mode 100644 index 00000000..c9ba7dfa Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_142744/best_model_reg.keras differ diff --git a/gru_sac_predictor/models/run_20250416_142744/critic1.weights.h5 b/gru_sac_predictor/models/run_20250416_142744/critic1.weights.h5 new file mode 100644 index 00000000..70b5a1a4 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_142744/critic1.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_142744/critic2.weights.h5 b/gru_sac_predictor/models/run_20250416_142744/critic2.weights.h5 new file mode 100644 index 00000000..f4661419 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_142744/critic2.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_142744/feature_scaler.joblib b/gru_sac_predictor/models/run_20250416_142744/feature_scaler.joblib new file mode 100644 index 00000000..65594f07 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_142744/feature_scaler.joblib differ diff --git a/gru_sac_predictor/models/run_20250416_142744/gru_training_history.png b/gru_sac_predictor/models/run_20250416_142744/gru_training_history.png new file mode 100644 index 00000000..69ac8723 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_142744/gru_training_history.png differ diff --git a/gru_sac_predictor/models/run_20250416_142744/log_alpha.npy b/gru_sac_predictor/models/run_20250416_142744/log_alpha.npy new file mode 100644 index 00000000..f6dcc78a Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_142744/log_alpha.npy differ diff --git a/gru_sac_predictor/models/run_20250416_142744/y_scaler.joblib b/gru_sac_predictor/models/run_20250416_142744/y_scaler.joblib new file mode 100644 index 00000000..815292e7 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_142744/y_scaler.joblib differ diff --git a/gru_sac_predictor/models/run_20250416_144757/sac_agent/actor.weights.h5 b/gru_sac_predictor/models/run_20250416_144757/sac_agent/actor.weights.h5 new file mode 100644 index 00000000..821c86f5 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_144757/sac_agent/actor.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_144757/sac_agent/alpha.npy b/gru_sac_predictor/models/run_20250416_144757/sac_agent/alpha.npy new file mode 100644 index 00000000..d8cc008b Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_144757/sac_agent/alpha.npy differ diff --git a/gru_sac_predictor/models/run_20250416_144757/sac_agent/critic_1.weights.h5 b/gru_sac_predictor/models/run_20250416_144757/sac_agent/critic_1.weights.h5 new file mode 100644 index 00000000..219924ae Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_144757/sac_agent/critic_1.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_144757/sac_agent/critic_2.weights.h5 b/gru_sac_predictor/models/run_20250416_144757/sac_agent/critic_2.weights.h5 new file mode 100644 index 00000000..556227b0 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_144757/sac_agent/critic_2.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_144757/sac_agent/target_critic_1.weights.h5 b/gru_sac_predictor/models/run_20250416_144757/sac_agent/target_critic_1.weights.h5 new file mode 100644 index 00000000..0d6ff386 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_144757/sac_agent/target_critic_1.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_144757/sac_agent/target_critic_2.weights.h5 b/gru_sac_predictor/models/run_20250416_144757/sac_agent/target_critic_2.weights.h5 new file mode 100644 index 00000000..2d695ae2 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_144757/sac_agent/target_critic_2.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_145128/sac_agent/actor.weights.h5 b/gru_sac_predictor/models/run_20250416_145128/sac_agent/actor.weights.h5 new file mode 100644 index 00000000..b474b03b Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_145128/sac_agent/actor.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_145128/sac_agent/alpha.npy b/gru_sac_predictor/models/run_20250416_145128/sac_agent/alpha.npy new file mode 100644 index 00000000..d8cc008b Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_145128/sac_agent/alpha.npy differ diff --git a/gru_sac_predictor/models/run_20250416_145128/sac_agent/critic_1.weights.h5 b/gru_sac_predictor/models/run_20250416_145128/sac_agent/critic_1.weights.h5 new file mode 100644 index 00000000..b8e977f8 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_145128/sac_agent/critic_1.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_145128/sac_agent/critic_2.weights.h5 b/gru_sac_predictor/models/run_20250416_145128/sac_agent/critic_2.weights.h5 new file mode 100644 index 00000000..a6643e59 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_145128/sac_agent/critic_2.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_145128/sac_agent/target_critic_1.weights.h5 b/gru_sac_predictor/models/run_20250416_145128/sac_agent/target_critic_1.weights.h5 new file mode 100644 index 00000000..5a5b2bd0 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_145128/sac_agent/target_critic_1.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_145128/sac_agent/target_critic_2.weights.h5 b/gru_sac_predictor/models/run_20250416_145128/sac_agent/target_critic_2.weights.h5 new file mode 100644 index 00000000..5b674e70 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_145128/sac_agent/target_critic_2.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_150829/sac_agent/actor.weights.h5 b/gru_sac_predictor/models/run_20250416_150829/sac_agent/actor.weights.h5 new file mode 100644 index 00000000..1d707323 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_150829/sac_agent/actor.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_150829/sac_agent/alpha.npy b/gru_sac_predictor/models/run_20250416_150829/sac_agent/alpha.npy new file mode 100644 index 00000000..d8cc008b Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_150829/sac_agent/alpha.npy differ diff --git a/gru_sac_predictor/models/run_20250416_150829/sac_agent/critic_1.weights.h5 b/gru_sac_predictor/models/run_20250416_150829/sac_agent/critic_1.weights.h5 new file mode 100644 index 00000000..5306d00d Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_150829/sac_agent/critic_1.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_150829/sac_agent/critic_2.weights.h5 b/gru_sac_predictor/models/run_20250416_150829/sac_agent/critic_2.weights.h5 new file mode 100644 index 00000000..0c91518c Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_150829/sac_agent/critic_2.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_150829/sac_agent/target_critic_1.weights.h5 b/gru_sac_predictor/models/run_20250416_150829/sac_agent/target_critic_1.weights.h5 new file mode 100644 index 00000000..566cb8a0 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_150829/sac_agent/target_critic_1.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_150829/sac_agent/target_critic_2.weights.h5 b/gru_sac_predictor/models/run_20250416_150829/sac_agent/target_critic_2.weights.h5 new file mode 100644 index 00000000..2a9289d3 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_150829/sac_agent/target_critic_2.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_150924/sac_agent/actor.weights.h5 b/gru_sac_predictor/models/run_20250416_150924/sac_agent/actor.weights.h5 new file mode 100644 index 00000000..48f87255 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_150924/sac_agent/actor.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_150924/sac_agent/alpha.npy b/gru_sac_predictor/models/run_20250416_150924/sac_agent/alpha.npy new file mode 100644 index 00000000..d8cc008b Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_150924/sac_agent/alpha.npy differ diff --git a/gru_sac_predictor/models/run_20250416_150924/sac_agent/critic_1.weights.h5 b/gru_sac_predictor/models/run_20250416_150924/sac_agent/critic_1.weights.h5 new file mode 100644 index 00000000..2b49793b Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_150924/sac_agent/critic_1.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_150924/sac_agent/critic_2.weights.h5 b/gru_sac_predictor/models/run_20250416_150924/sac_agent/critic_2.weights.h5 new file mode 100644 index 00000000..8819325c Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_150924/sac_agent/critic_2.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_150924/sac_agent/target_critic_1.weights.h5 b/gru_sac_predictor/models/run_20250416_150924/sac_agent/target_critic_1.weights.h5 new file mode 100644 index 00000000..d9d16fbe Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_150924/sac_agent/target_critic_1.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_150924/sac_agent/target_critic_2.weights.h5 b/gru_sac_predictor/models/run_20250416_150924/sac_agent/target_critic_2.weights.h5 new file mode 100644 index 00000000..2c5effde Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_150924/sac_agent/target_critic_2.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_151322/sac_agent/actor.weights.h5 b/gru_sac_predictor/models/run_20250416_151322/sac_agent/actor.weights.h5 new file mode 100644 index 00000000..07956adf Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_151322/sac_agent/actor.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_151322/sac_agent/alpha.npy b/gru_sac_predictor/models/run_20250416_151322/sac_agent/alpha.npy new file mode 100644 index 00000000..ff5c2883 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_151322/sac_agent/alpha.npy differ diff --git a/gru_sac_predictor/models/run_20250416_151322/sac_agent/critic_1.weights.h5 b/gru_sac_predictor/models/run_20250416_151322/sac_agent/critic_1.weights.h5 new file mode 100644 index 00000000..459fdc33 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_151322/sac_agent/critic_1.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_151322/sac_agent/critic_2.weights.h5 b/gru_sac_predictor/models/run_20250416_151322/sac_agent/critic_2.weights.h5 new file mode 100644 index 00000000..5bfe8264 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_151322/sac_agent/critic_2.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_151322/sac_agent/target_critic_1.weights.h5 b/gru_sac_predictor/models/run_20250416_151322/sac_agent/target_critic_1.weights.h5 new file mode 100644 index 00000000..175e4146 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_151322/sac_agent/target_critic_1.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_151322/sac_agent/target_critic_2.weights.h5 b/gru_sac_predictor/models/run_20250416_151322/sac_agent/target_critic_2.weights.h5 new file mode 100644 index 00000000..f2efe7f3 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_151322/sac_agent/target_critic_2.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_151849/sac_agent/actor.weights.h5 b/gru_sac_predictor/models/run_20250416_151849/sac_agent/actor.weights.h5 new file mode 100644 index 00000000..d49f4f57 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_151849/sac_agent/actor.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_151849/sac_agent/alpha.npy b/gru_sac_predictor/models/run_20250416_151849/sac_agent/alpha.npy new file mode 100644 index 00000000..d8cc008b Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_151849/sac_agent/alpha.npy differ diff --git a/gru_sac_predictor/models/run_20250416_151849/sac_agent/critic_1.weights.h5 b/gru_sac_predictor/models/run_20250416_151849/sac_agent/critic_1.weights.h5 new file mode 100644 index 00000000..08716159 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_151849/sac_agent/critic_1.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_151849/sac_agent/critic_2.weights.h5 b/gru_sac_predictor/models/run_20250416_151849/sac_agent/critic_2.weights.h5 new file mode 100644 index 00000000..43b82f3f Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_151849/sac_agent/critic_2.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_151849/sac_agent/target_critic_1.weights.h5 b/gru_sac_predictor/models/run_20250416_151849/sac_agent/target_critic_1.weights.h5 new file mode 100644 index 00000000..20ee67f2 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_151849/sac_agent/target_critic_1.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_151849/sac_agent/target_critic_2.weights.h5 b/gru_sac_predictor/models/run_20250416_151849/sac_agent/target_critic_2.weights.h5 new file mode 100644 index 00000000..1f58a4d9 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_151849/sac_agent/target_critic_2.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_152415/sac_agent/actor.weights.h5 b/gru_sac_predictor/models/run_20250416_152415/sac_agent/actor.weights.h5 new file mode 100644 index 00000000..d5fc1f9c Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_152415/sac_agent/actor.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_152415/sac_agent/alpha.npy b/gru_sac_predictor/models/run_20250416_152415/sac_agent/alpha.npy new file mode 100644 index 00000000..fdf08736 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_152415/sac_agent/alpha.npy differ diff --git a/gru_sac_predictor/models/run_20250416_152415/sac_agent/critic_1.weights.h5 b/gru_sac_predictor/models/run_20250416_152415/sac_agent/critic_1.weights.h5 new file mode 100644 index 00000000..ec11f9e4 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_152415/sac_agent/critic_1.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_152415/sac_agent/critic_2.weights.h5 b/gru_sac_predictor/models/run_20250416_152415/sac_agent/critic_2.weights.h5 new file mode 100644 index 00000000..78096da4 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_152415/sac_agent/critic_2.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_152415/sac_agent/target_critic_1.weights.h5 b/gru_sac_predictor/models/run_20250416_152415/sac_agent/target_critic_1.weights.h5 new file mode 100644 index 00000000..91a0b1c6 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_152415/sac_agent/target_critic_1.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_152415/sac_agent/target_critic_2.weights.h5 b/gru_sac_predictor/models/run_20250416_152415/sac_agent/target_critic_2.weights.h5 new file mode 100644 index 00000000..a41c4a39 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_152415/sac_agent/target_critic_2.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_153132/sac_agent/actor.weights.h5 b/gru_sac_predictor/models/run_20250416_153132/sac_agent/actor.weights.h5 new file mode 100644 index 00000000..76cbdefc Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_153132/sac_agent/actor.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_153132/sac_agent/alpha.npy b/gru_sac_predictor/models/run_20250416_153132/sac_agent/alpha.npy new file mode 100644 index 00000000..fdf08736 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_153132/sac_agent/alpha.npy differ diff --git a/gru_sac_predictor/models/run_20250416_153132/sac_agent/critic_1.weights.h5 b/gru_sac_predictor/models/run_20250416_153132/sac_agent/critic_1.weights.h5 new file mode 100644 index 00000000..931046cf Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_153132/sac_agent/critic_1.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_153132/sac_agent/critic_2.weights.h5 b/gru_sac_predictor/models/run_20250416_153132/sac_agent/critic_2.weights.h5 new file mode 100644 index 00000000..daaeb3ff Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_153132/sac_agent/critic_2.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_153132/sac_agent/target_critic_1.weights.h5 b/gru_sac_predictor/models/run_20250416_153132/sac_agent/target_critic_1.weights.h5 new file mode 100644 index 00000000..c2bf2d44 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_153132/sac_agent/target_critic_1.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_153132/sac_agent/target_critic_2.weights.h5 b/gru_sac_predictor/models/run_20250416_153132/sac_agent/target_critic_2.weights.h5 new file mode 100644 index 00000000..1eee4657 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_153132/sac_agent/target_critic_2.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_153846/sac_agent/actor.weights.h5 b/gru_sac_predictor/models/run_20250416_153846/sac_agent/actor.weights.h5 new file mode 100644 index 00000000..67ea1863 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_153846/sac_agent/actor.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_153846/sac_agent/alpha.npy b/gru_sac_predictor/models/run_20250416_153846/sac_agent/alpha.npy new file mode 100644 index 00000000..d8cc008b Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_153846/sac_agent/alpha.npy differ diff --git a/gru_sac_predictor/models/run_20250416_153846/sac_agent/critic_1.weights.h5 b/gru_sac_predictor/models/run_20250416_153846/sac_agent/critic_1.weights.h5 new file mode 100644 index 00000000..717ed3d9 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_153846/sac_agent/critic_1.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_153846/sac_agent/critic_2.weights.h5 b/gru_sac_predictor/models/run_20250416_153846/sac_agent/critic_2.weights.h5 new file mode 100644 index 00000000..6d3328d7 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_153846/sac_agent/critic_2.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_153846/sac_agent/target_critic_1.weights.h5 b/gru_sac_predictor/models/run_20250416_153846/sac_agent/target_critic_1.weights.h5 new file mode 100644 index 00000000..5a67aa12 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_153846/sac_agent/target_critic_1.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_153846/sac_agent/target_critic_2.weights.h5 b/gru_sac_predictor/models/run_20250416_153846/sac_agent/target_critic_2.weights.h5 new file mode 100644 index 00000000..44f93484 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_153846/sac_agent/target_critic_2.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_154636/sac_agent/actor.weights.h5 b/gru_sac_predictor/models/run_20250416_154636/sac_agent/actor.weights.h5 new file mode 100644 index 00000000..e0b1186f Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_154636/sac_agent/actor.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_154636/sac_agent/alpha.npy b/gru_sac_predictor/models/run_20250416_154636/sac_agent/alpha.npy new file mode 100644 index 00000000..d8cc008b Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_154636/sac_agent/alpha.npy differ diff --git a/gru_sac_predictor/models/run_20250416_154636/sac_agent/critic_1.weights.h5 b/gru_sac_predictor/models/run_20250416_154636/sac_agent/critic_1.weights.h5 new file mode 100644 index 00000000..eddc6ce5 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_154636/sac_agent/critic_1.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_154636/sac_agent/critic_2.weights.h5 b/gru_sac_predictor/models/run_20250416_154636/sac_agent/critic_2.weights.h5 new file mode 100644 index 00000000..9e1a8b2e Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_154636/sac_agent/critic_2.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_154636/sac_agent/target_critic_1.weights.h5 b/gru_sac_predictor/models/run_20250416_154636/sac_agent/target_critic_1.weights.h5 new file mode 100644 index 00000000..dfbd183e Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_154636/sac_agent/target_critic_1.weights.h5 differ diff --git a/gru_sac_predictor/models/run_20250416_154636/sac_agent/target_critic_2.weights.h5 b/gru_sac_predictor/models/run_20250416_154636/sac_agent/target_critic_2.weights.h5 new file mode 100644 index 00000000..1af03e89 Binary files /dev/null and b/gru_sac_predictor/models/run_20250416_154636/sac_agent/target_critic_2.weights.h5 differ diff --git a/gru_sac_predictor/requirements.txt b/gru_sac_predictor/requirements.txt new file mode 100644 index 00000000..ae8121a4 --- /dev/null +++ b/gru_sac_predictor/requirements.txt @@ -0,0 +1,10 @@ +pandas +numpy +tensorflow +tensorflow-probability +matplotlib +joblib +scikit-learn +tqdm +PyYAML +TA-Lib \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_142744/backtest_performance_report_v7_20250416_142744.md b/gru_sac_predictor/results/20250416_142744/backtest_performance_report_v7_20250416_142744.md new file mode 100644 index 00000000..f338fc28 --- /dev/null +++ b/gru_sac_predictor/results/20250416_142744/backtest_performance_report_v7_20250416_142744.md @@ -0,0 +1,42 @@ +# GRU+SAC Backtesting Performance Report + +Report generated on: 2025-04-16 14:29:57.872322 +Data range: 2025-03-06 15:23:00+00:00 to 2025-03-07 23:57:00+00:00 +Total duration: 1 days 08:34:00 + +## Strategy Performance Metrics + +* **Initial capital:** $10,000.00 +* **Final portfolio value:** $10,320.55 +* **Total return:** 3.21% +* **Annualized return:** 506709.93% +* **Sharpe ratio (annualized):** 10.5985 +* **Sortino ratio (annualized):** 16.0926 +* **Volatility (annualized):** 83.80% +* **Maximum drawdown:** 5.18% +* **Total trades:** 1 + +## Buy and Hold Benchmark + +* **Final value (B&H):** $9,658.75 +* **Total return (B&H):** -3.41% + +## Position & Prediction Analysis + +* **Average absolute position size:** 0.7616 +* **Position sign accuracy vs return:** 50.93% +* **Prediction sign accuracy vs return:** 48.92% +* **Prediction RMSE (on returns):** 0.004036 + +## Correlations + +* **Prediction-Return correlation:** -0.0042 +* **Prediction-Position correlation:** nan +* **Uncertainty-Position Size correlation:** nan + +## Notes + +* Transaction cost used: 0.0500% per position change value. +* GRU lookback period: 60 minutes. +* V6 features + return features used. +* Uncertainty estimated via MC Dropout standard deviation. diff --git a/gru_sac_predictor/results/20250416_142744/backtest_results_v7_20250416_142744.png b/gru_sac_predictor/results/20250416_142744/backtest_results_v7_20250416_142744.png new file mode 100644 index 00000000..a41fa974 Binary files /dev/null and b/gru_sac_predictor/results/20250416_142744/backtest_results_v7_20250416_142744.png differ diff --git a/gru_sac_predictor/results/20250416_142744/config_20250416_142744.json b/gru_sac_predictor/results/20250416_142744/config_20250416_142744.json new file mode 100644 index 00000000..d1c64343 --- /dev/null +++ b/gru_sac_predictor/results/20250416_142744/config_20250416_142744.json @@ -0,0 +1,45 @@ +{ + "run_id": "20250416_142744", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/crypto_trading_system_v7_20250416_142744", + "results_plot_path": "v7/results/20250416_142744/backtest_results_v7_20250416_142744.png", + "report_save_path": "v7/results/20250416_142744/backtest_performance_report_v7_20250416_142744.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "sac_state_dim": 2, + "sac_initial_lr": 0.0003, + "sac_end_lr": 5e-06, + "sac_decay_steps": 100000, + "sac_lr_decay_rate": 0.96, + "sac_gamma": 0.99, + "sac_tau": 0.005, + "sac_alpha_initial": 0.2, + "sac_alpha_auto_tune": true, + "sac_target_entropy": -1.0, + "sac_ou_noise_stddev": 0.2, + "sac_ou_noise_theta": 0.15, + "sac_ou_noise_dt": 0.01, + "sac_buffer_capacity": 100000, + "sac_batch_size": 256, + "sac_min_buffer_size": 1000, + "sac_epochs": 50, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "load_existing_system": false, + "train_gru_model": true, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_144232/config_20250416_144232.json b/gru_sac_predictor/results/20250416_144232/config_20250416_144232.json new file mode 100644 index 00000000..7f6c6560 --- /dev/null +++ b/gru_sac_predictor/results/20250416_144232/config_20250416_144232.json @@ -0,0 +1,49 @@ +{ + "run_id": "20250416_144232", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/crypto_trading_system_v7_20250416_144232", + "results_plot_path": "v7/results/20250416_144232/backtest_results_v7_20250416_144232.png", + "report_save_path": "v7/results/20250416_144232/backtest_performance_report_v7_20250416_144232.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 2, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_alpha": 0.1, + "sac_actor_lr": 0.0005, + "sac_critic_lr": 0.0008, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 1.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_144418/config_20250416_144418.json b/gru_sac_predictor/results/20250416_144418/config_20250416_144418.json new file mode 100644 index 00000000..3a592099 --- /dev/null +++ b/gru_sac_predictor/results/20250416_144418/config_20250416_144418.json @@ -0,0 +1,49 @@ +{ + "run_id": "20250416_144418", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/run_20250416_144418", + "results_plot_path": "v7/results/20250416_144418/backtest_results_v7_20250416_144418.png", + "report_save_path": "v7/results/20250416_144418/backtest_performance_report_v7_20250416_144418.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 2, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_alpha": 0.1, + "sac_actor_lr": 0.0005, + "sac_critic_lr": 0.0008, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 1.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_144645/config_20250416_144645.json b/gru_sac_predictor/results/20250416_144645/config_20250416_144645.json new file mode 100644 index 00000000..ffee3555 --- /dev/null +++ b/gru_sac_predictor/results/20250416_144645/config_20250416_144645.json @@ -0,0 +1,49 @@ +{ + "run_id": "20250416_144645", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/run_20250416_144645", + "results_plot_path": "v7/results/20250416_144645/backtest_results_v7_20250416_144645.png", + "report_save_path": "v7/results/20250416_144645/backtest_performance_report_v7_20250416_144645.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 2, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_alpha": 0.1, + "sac_actor_lr": 0.0005, + "sac_critic_lr": 0.0008, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 1.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_144757/config_20250416_144757.json b/gru_sac_predictor/results/20250416_144757/config_20250416_144757.json new file mode 100644 index 00000000..7d3bbdb2 --- /dev/null +++ b/gru_sac_predictor/results/20250416_144757/config_20250416_144757.json @@ -0,0 +1,49 @@ +{ + "run_id": "20250416_144757", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/run_20250416_144757", + "results_plot_path": "v7/results/20250416_144757/backtest_results_v7_20250416_144757.png", + "report_save_path": "v7/results/20250416_144757/backtest_performance_report_v7_20250416_144757.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 2, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_alpha": 0.1, + "sac_actor_lr": 0.0005, + "sac_critic_lr": 0.0008, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 1.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_144847/config_20250416_144847.json b/gru_sac_predictor/results/20250416_144847/config_20250416_144847.json new file mode 100644 index 00000000..29633eb2 --- /dev/null +++ b/gru_sac_predictor/results/20250416_144847/config_20250416_144847.json @@ -0,0 +1,49 @@ +{ + "run_id": "20250416_144847", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/run_20250416_144847", + "results_plot_path": "v7/results/20250416_144847/backtest_results_v7_20250416_144847.png", + "report_save_path": "v7/results/20250416_144847/backtest_performance_report_v7_20250416_144847.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 2, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_alpha": 0.1, + "sac_actor_lr": 0.0005, + "sac_critic_lr": 0.0008, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 1.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_145035/config_20250416_145035.json b/gru_sac_predictor/results/20250416_145035/config_20250416_145035.json new file mode 100644 index 00000000..9af3cbf4 --- /dev/null +++ b/gru_sac_predictor/results/20250416_145035/config_20250416_145035.json @@ -0,0 +1,49 @@ +{ + "run_id": "20250416_145035", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/run_20250416_145035", + "results_plot_path": "v7/results/20250416_145035/backtest_results_v7_20250416_145035.png", + "report_save_path": "v7/results/20250416_145035/backtest_performance_report_v7_20250416_145035.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 2, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_alpha": 0.1, + "sac_actor_lr": 0.0005, + "sac_critic_lr": 0.0008, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 1.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_145128/backtest_performance_report_v7_20250416_145128.md b/gru_sac_predictor/results/20250416_145128/backtest_performance_report_v7_20250416_145128.md new file mode 100644 index 00000000..b7ab9c01 --- /dev/null +++ b/gru_sac_predictor/results/20250416_145128/backtest_performance_report_v7_20250416_145128.md @@ -0,0 +1,42 @@ +# GRU+SAC Backtesting Performance Report + +Report generated on: 2025-04-16 14:54:21.018426 +Data range: 2025-03-06 15:23:00+00:00 to 2025-03-07 23:57:00+00:00 +Total duration: 1 days 08:34:00 + +## Strategy Performance Metrics + +* **Initial capital:** $10,000.00 +* **Final portfolio value:** $9,839.98 +* **Total return:** -1.60% +* **Annualized return:** -98.72% +* **Sharpe ratio (annualized):** -11.7108 +* **Sortino ratio (annualized):** -17.6542 +* **Volatility (annualized):** 36.67% +* **Maximum drawdown:** 3.81% +* **Total trades:** 1622 + +## Buy and Hold Benchmark + +* **Final value (B&H):** $9,658.75 +* **Total return (B&H):** -3.41% + +## Position & Prediction Analysis + +* **Average absolute position size:** 0.2889 +* **Position sign accuracy vs return:** 50.93% +* **Prediction sign accuracy vs return:** 48.92% +* **Prediction RMSE (on returns):** 0.004036 + +## Correlations + +* **Prediction-Return correlation:** -0.0042 +* **Prediction-Position correlation:** 0.3861 +* **Uncertainty-Position Size correlation:** 0.9980 + +## Notes + +* Transaction cost used: 0.0500% per position change value. +* GRU lookback period: 60 minutes. +* V6 features + return features used. +* Uncertainty estimated via MC Dropout standard deviation. diff --git a/gru_sac_predictor/results/20250416_145128/backtest_results_v7_20250416_145128.png b/gru_sac_predictor/results/20250416_145128/backtest_results_v7_20250416_145128.png new file mode 100644 index 00000000..43831d21 Binary files /dev/null and b/gru_sac_predictor/results/20250416_145128/backtest_results_v7_20250416_145128.png differ diff --git a/gru_sac_predictor/results/20250416_145128/config_20250416_145128.json b/gru_sac_predictor/results/20250416_145128/config_20250416_145128.json new file mode 100644 index 00000000..bb5486f8 --- /dev/null +++ b/gru_sac_predictor/results/20250416_145128/config_20250416_145128.json @@ -0,0 +1,49 @@ +{ + "run_id": "20250416_145128", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/run_20250416_145128", + "results_plot_path": "v7/results/20250416_145128/backtest_results_v7_20250416_145128.png", + "report_save_path": "v7/results/20250416_145128/backtest_performance_report_v7_20250416_145128.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 2, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_alpha": 0.1, + "sac_actor_lr": 0.0005, + "sac_critic_lr": 0.0008, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 1.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_150616/config_20250416_150616.json b/gru_sac_predictor/results/20250416_150616/config_20250416_150616.json new file mode 100644 index 00000000..e399afbc --- /dev/null +++ b/gru_sac_predictor/results/20250416_150616/config_20250416_150616.json @@ -0,0 +1,65 @@ +{ + "run_id": "20250416_150616", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/run_20250416_150616", + "results_plot_path": "v7/results/20250416_150616/backtest_results_v7_20250416_150616.png", + "report_save_path": "v7/results/20250416_150616/backtest_performance_report_v7_20250416_150616.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 2, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_alpha": 0.1, + "sac_actor_lr": 0.0005, + "sac_critic_lr": 0.0008, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 1.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "total_training_steps": 100000, + "experience_config": { + "initial_experiences": 3000, + "experiences_per_batch": 64, + "batch_generation_interval": 500, + "balance_market_regimes": false, + "recency_bias_strength": 0.5, + "high_uncertainty_quantile": 0.75, + "extreme_return_quantile": 0.1, + "min_uncertainty_ratio": 0.2, + "min_extreme_return_ratio": 0.1, + "use_parallel_generation": false, + "precompute_all_gru_outputs": true, + "buffer_update_strategy": "fifo", + "training_iterations_per_step": 1 + }, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_150829/backtest_performance_report_v7_20250416_150829.md b/gru_sac_predictor/results/20250416_150829/backtest_performance_report_v7_20250416_150829.md new file mode 100644 index 00000000..a1413ccb --- /dev/null +++ b/gru_sac_predictor/results/20250416_150829/backtest_performance_report_v7_20250416_150829.md @@ -0,0 +1,42 @@ +# GRU+SAC Backtesting Performance Report + +Report generated on: 2025-04-16 15:09:06.744482 +Data range: 2025-03-06 15:23:00+00:00 to 2025-03-07 23:57:00+00:00 +Total duration: 1 days 08:34:00 + +## Strategy Performance Metrics + +* **Initial capital:** $10,000.00 +* **Final portfolio value:** $9,811.94 +* **Total return:** -1.88% +* **Annualized return:** -99.41% +* **Sharpe ratio (annualized):** -7.9546 +* **Sortino ratio (annualized):** -11.8533 +* **Volatility (annualized):** 62.11% +* **Maximum drawdown:** 6.00% +* **Total trades:** 1756 + +## Buy and Hold Benchmark + +* **Final value (B&H):** $9,658.75 +* **Total return (B&H):** -3.41% + +## Position & Prediction Analysis + +* **Average absolute position size:** 0.5121 +* **Position sign accuracy vs return:** 50.93% +* **Prediction sign accuracy vs return:** 48.92% +* **Prediction RMSE (on returns):** 0.004036 + +## Correlations + +* **Prediction-Return correlation:** -0.0042 +* **Prediction-Position correlation:** 0.3196 +* **Uncertainty-Position Size correlation:** 0.9811 + +## Notes + +* Transaction cost used: 0.0500% per position change value. +* GRU lookback period: 60 minutes. +* V6 features + return features used. +* Uncertainty estimated via MC Dropout standard deviation. diff --git a/gru_sac_predictor/results/20250416_150829/backtest_results_v7_20250416_150829.png b/gru_sac_predictor/results/20250416_150829/backtest_results_v7_20250416_150829.png new file mode 100644 index 00000000..454c57d3 Binary files /dev/null and b/gru_sac_predictor/results/20250416_150829/backtest_results_v7_20250416_150829.png differ diff --git a/gru_sac_predictor/results/20250416_150829/config_20250416_150829.json b/gru_sac_predictor/results/20250416_150829/config_20250416_150829.json new file mode 100644 index 00000000..167b6ab9 --- /dev/null +++ b/gru_sac_predictor/results/20250416_150829/config_20250416_150829.json @@ -0,0 +1,65 @@ +{ + "run_id": "20250416_150829", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/run_20250416_150829", + "results_plot_path": "v7/results/20250416_150829/backtest_results_v7_20250416_150829.png", + "report_save_path": "v7/results/20250416_150829/backtest_performance_report_v7_20250416_150829.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 2, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_alpha": 0.1, + "sac_actor_lr": 0.0005, + "sac_critic_lr": 0.0008, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 1.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "total_training_steps": 100, + "experience_config": { + "initial_experiences": 3000, + "experiences_per_batch": 64, + "batch_generation_interval": 500, + "balance_market_regimes": false, + "recency_bias_strength": 0.5, + "high_uncertainty_quantile": 0.75, + "extreme_return_quantile": 0.1, + "min_uncertainty_ratio": 0.2, + "min_extreme_return_ratio": 0.1, + "use_parallel_generation": false, + "precompute_all_gru_outputs": true, + "buffer_update_strategy": "fifo", + "training_iterations_per_step": 1 + }, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_150924/backtest_performance_report_v7_20250416_150924.md b/gru_sac_predictor/results/20250416_150924/backtest_performance_report_v7_20250416_150924.md new file mode 100644 index 00000000..a912284f --- /dev/null +++ b/gru_sac_predictor/results/20250416_150924/backtest_performance_report_v7_20250416_150924.md @@ -0,0 +1,42 @@ +# GRU+SAC Backtesting Performance Report + +Report generated on: 2025-04-16 15:11:02.339105 +Data range: 2025-03-06 15:23:00+00:00 to 2025-03-07 23:57:00+00:00 +Total duration: 1 days 08:34:00 + +## Strategy Performance Metrics + +* **Initial capital:** $10,000.00 +* **Final portfolio value:** $9,946.65 +* **Total return:** -0.53% +* **Annualized return:** -76.46% +* **Sharpe ratio (annualized):** -11.2012 +* **Sortino ratio (annualized):** -17.0343 +* **Volatility (annualized):** 12.84% +* **Maximum drawdown:** 1.32% +* **Total trades:** 1128 + +## Buy and Hold Benchmark + +* **Final value (B&H):** $9,658.75 +* **Total return (B&H):** -3.41% + +## Position & Prediction Analysis + +* **Average absolute position size:** 0.1015 +* **Position sign accuracy vs return:** 50.93% +* **Prediction sign accuracy vs return:** 48.92% +* **Prediction RMSE (on returns):** 0.004036 + +## Correlations + +* **Prediction-Return correlation:** -0.0042 +* **Prediction-Position correlation:** 0.4118 +* **Uncertainty-Position Size correlation:** 1.0000 + +## Notes + +* Transaction cost used: 0.0500% per position change value. +* GRU lookback period: 60 minutes. +* V6 features + return features used. +* Uncertainty estimated via MC Dropout standard deviation. diff --git a/gru_sac_predictor/results/20250416_150924/backtest_results_v7_20250416_150924.png b/gru_sac_predictor/results/20250416_150924/backtest_results_v7_20250416_150924.png new file mode 100644 index 00000000..4817f646 Binary files /dev/null and b/gru_sac_predictor/results/20250416_150924/backtest_results_v7_20250416_150924.png differ diff --git a/gru_sac_predictor/results/20250416_150924/config_20250416_150924.json b/gru_sac_predictor/results/20250416_150924/config_20250416_150924.json new file mode 100644 index 00000000..a075cbfe --- /dev/null +++ b/gru_sac_predictor/results/20250416_150924/config_20250416_150924.json @@ -0,0 +1,65 @@ +{ + "run_id": "20250416_150924", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/run_20250416_150924", + "results_plot_path": "v7/results/20250416_150924/backtest_results_v7_20250416_150924.png", + "report_save_path": "v7/results/20250416_150924/backtest_performance_report_v7_20250416_150924.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 2, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_alpha": 0.1, + "sac_actor_lr": 0.0005, + "sac_critic_lr": 0.0008, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 1.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "total_training_steps": 1000, + "experience_config": { + "initial_experiences": 3000, + "experiences_per_batch": 64, + "batch_generation_interval": 500, + "balance_market_regimes": false, + "recency_bias_strength": 0.5, + "high_uncertainty_quantile": 0.75, + "extreme_return_quantile": 0.1, + "min_uncertainty_ratio": 0.2, + "min_extreme_return_ratio": 0.1, + "use_parallel_generation": false, + "precompute_all_gru_outputs": true, + "buffer_update_strategy": "fifo", + "training_iterations_per_step": 1 + }, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_151322/backtest_performance_report_v7_20250416_151322.md b/gru_sac_predictor/results/20250416_151322/backtest_performance_report_v7_20250416_151322.md new file mode 100644 index 00000000..45b13917 --- /dev/null +++ b/gru_sac_predictor/results/20250416_151322/backtest_performance_report_v7_20250416_151322.md @@ -0,0 +1,42 @@ +# GRU+SAC Backtesting Performance Report + +Report generated on: 2025-04-16 15:15:17.184796 +Data range: 2025-03-06 15:23:00+00:00 to 2025-03-07 23:57:00+00:00 +Total duration: 1 days 08:34:00 + +## Strategy Performance Metrics + +* **Initial capital:** $10,000.00 +* **Final portfolio value:** $9,857.64 +* **Total return:** -1.42% +* **Annualized return:** -97.93% +* **Sharpe ratio (annualized):** -7.9260 +* **Sortino ratio (annualized):** -11.9087 +* **Volatility (annualized):** 47.49% +* **Maximum drawdown:** 4.68% +* **Total trades:** 1702 + +## Buy and Hold Benchmark + +* **Final value (B&H):** $9,658.75 +* **Total return (B&H):** -3.41% + +## Position & Prediction Analysis + +* **Average absolute position size:** 0.3745 +* **Position sign accuracy vs return:** 50.93% +* **Prediction sign accuracy vs return:** 48.92% +* **Prediction RMSE (on returns):** 0.004036 + +## Correlations + +* **Prediction-Return correlation:** -0.0042 +* **Prediction-Position correlation:** 0.3604 +* **Uncertainty-Position Size correlation:** 0.9947 + +## Notes + +* Transaction cost used: 0.0500% per position change value. +* GRU lookback period: 60 minutes. +* V6 features + return features used. +* Uncertainty estimated via MC Dropout standard deviation. diff --git a/gru_sac_predictor/results/20250416_151322/backtest_results_v7_20250416_151322.png b/gru_sac_predictor/results/20250416_151322/backtest_results_v7_20250416_151322.png new file mode 100644 index 00000000..dedc9b9d Binary files /dev/null and b/gru_sac_predictor/results/20250416_151322/backtest_results_v7_20250416_151322.png differ diff --git a/gru_sac_predictor/results/20250416_151322/config_20250416_151322.json b/gru_sac_predictor/results/20250416_151322/config_20250416_151322.json new file mode 100644 index 00000000..f0a10f9f --- /dev/null +++ b/gru_sac_predictor/results/20250416_151322/config_20250416_151322.json @@ -0,0 +1,65 @@ +{ + "run_id": "20250416_151322", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/run_20250416_151322", + "results_plot_path": "v7/results/20250416_151322/backtest_results_v7_20250416_151322.png", + "report_save_path": "v7/results/20250416_151322/backtest_performance_report_v7_20250416_151322.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 2, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_alpha": 0.2, + "sac_actor_lr": 0.0005, + "sac_critic_lr": 0.0008, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 1.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "total_training_steps": 1000, + "experience_config": { + "initial_experiences": 3000, + "experiences_per_batch": 64, + "batch_generation_interval": 500, + "balance_market_regimes": false, + "recency_bias_strength": 0.5, + "high_uncertainty_quantile": 0.75, + "extreme_return_quantile": 0.1, + "min_uncertainty_ratio": 0.2, + "min_extreme_return_ratio": 0.1, + "use_parallel_generation": false, + "precompute_all_gru_outputs": true, + "buffer_update_strategy": "fifo", + "training_iterations_per_step": 1 + }, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_151849/backtest_performance_report_v7_20250416_151849.md b/gru_sac_predictor/results/20250416_151849/backtest_performance_report_v7_20250416_151849.md new file mode 100644 index 00000000..099d70ec --- /dev/null +++ b/gru_sac_predictor/results/20250416_151849/backtest_performance_report_v7_20250416_151849.md @@ -0,0 +1,42 @@ +# GRU+SAC Backtesting Performance Report + +Report generated on: 2025-04-16 15:20:34.953163 +Data range: 2025-03-06 15:23:00+00:00 to 2025-03-07 23:57:00+00:00 +Total duration: 1 days 08:34:00 + +## Strategy Performance Metrics + +* **Initial capital:** $10,000.00 +* **Final portfolio value:** $9,863.85 +* **Total return:** -1.36% +* **Annualized return:** -97.54% +* **Sharpe ratio (annualized):** -9.5883 +* **Sortino ratio (annualized):** -14.1873 +* **Volatility (annualized):** 37.91% +* **Maximum drawdown:** 3.68% +* **Total trades:** 1638 + +## Buy and Hold Benchmark + +* **Final value (B&H):** $9,658.75 +* **Total return (B&H):** -3.41% + +## Position & Prediction Analysis + +* **Average absolute position size:** 0.2993 +* **Position sign accuracy vs return:** 50.93% +* **Prediction sign accuracy vs return:** 48.92% +* **Prediction RMSE (on returns):** 0.004036 + +## Correlations + +* **Prediction-Return correlation:** -0.0042 +* **Prediction-Position correlation:** 0.3821 +* **Uncertainty-Position Size correlation:** 0.9978 + +## Notes + +* Transaction cost used: 0.0500% per position change value. +* GRU lookback period: 60 minutes. +* V6 features + return features used. +* Uncertainty estimated via MC Dropout standard deviation. diff --git a/gru_sac_predictor/results/20250416_151849/backtest_results_v7_20250416_151849.png b/gru_sac_predictor/results/20250416_151849/backtest_results_v7_20250416_151849.png new file mode 100644 index 00000000..db77a50d Binary files /dev/null and b/gru_sac_predictor/results/20250416_151849/backtest_results_v7_20250416_151849.png differ diff --git a/gru_sac_predictor/results/20250416_151849/config_20250416_151849.json b/gru_sac_predictor/results/20250416_151849/config_20250416_151849.json new file mode 100644 index 00000000..6ecf41d0 --- /dev/null +++ b/gru_sac_predictor/results/20250416_151849/config_20250416_151849.json @@ -0,0 +1,65 @@ +{ + "run_id": "20250416_151849", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/run_20250416_151849", + "results_plot_path": "v7/results/20250416_151849/backtest_results_v7_20250416_151849.png", + "report_save_path": "v7/results/20250416_151849/backtest_performance_report_v7_20250416_151849.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 2, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_alpha": 0.1, + "sac_actor_lr": 0.0005, + "sac_critic_lr": 0.0008, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 1.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "total_training_steps": 1000, + "experience_config": { + "initial_experiences": 3000, + "experiences_per_batch": 64, + "batch_generation_interval": 500, + "balance_market_regimes": false, + "recency_bias_strength": 0.5, + "high_uncertainty_quantile": 0.75, + "extreme_return_quantile": 0.1, + "min_uncertainty_ratio": 0.2, + "min_extreme_return_ratio": 0.1, + "use_parallel_generation": false, + "precompute_all_gru_outputs": true, + "buffer_update_strategy": "fifo", + "training_iterations_per_step": 1 + }, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_151849/sac_training_history_20250416_151849.png b/gru_sac_predictor/results/20250416_151849/sac_training_history_20250416_151849.png new file mode 100644 index 00000000..35a87b67 Binary files /dev/null and b/gru_sac_predictor/results/20250416_151849/sac_training_history_20250416_151849.png differ diff --git a/gru_sac_predictor/results/20250416_152415/backtest_performance_report_v7_20250416_152415.md b/gru_sac_predictor/results/20250416_152415/backtest_performance_report_v7_20250416_152415.md new file mode 100644 index 00000000..ca4b6ae3 --- /dev/null +++ b/gru_sac_predictor/results/20250416_152415/backtest_performance_report_v7_20250416_152415.md @@ -0,0 +1,42 @@ +# GRU+SAC Backtesting Performance Report + +Report generated on: 2025-04-16 15:26:31.107123 +Data range: 2025-03-06 15:23:00+00:00 to 2025-03-07 23:57:00+00:00 +Total duration: 1 days 08:34:00 + +## Strategy Performance Metrics + +* **Initial capital:** $10,000.00 +* **Final portfolio value:** $9,828.39 +* **Total return:** -1.72% +* **Annualized return:** -99.07% +* **Sharpe ratio (annualized):** -7.6460 +* **Sortino ratio (annualized):** -11.7314 +* **Volatility (annualized):** 58.94% +* **Maximum drawdown:** 5.78% +* **Total trades:** 1737 + +## Buy and Hold Benchmark + +* **Final value (B&H):** $9,658.75 +* **Total return (B&H):** -3.41% + +## Position & Prediction Analysis + +* **Average absolute position size:** 0.4765 +* **Position sign accuracy vs return:** 50.93% +* **Prediction sign accuracy vs return:** 48.92% +* **Prediction RMSE (on returns):** 0.004036 + +## Correlations + +* **Prediction-Return correlation:** -0.0042 +* **Prediction-Position correlation:** 0.3427 +* **Uncertainty-Position Size correlation:** 0.9854 + +## Notes + +* Transaction cost used: 0.0500% per position change value. +* GRU lookback period: 60 minutes. +* V6 features + return features used. +* Uncertainty estimated via MC Dropout standard deviation. diff --git a/gru_sac_predictor/results/20250416_152415/backtest_results_v7_20250416_152415.png b/gru_sac_predictor/results/20250416_152415/backtest_results_v7_20250416_152415.png new file mode 100644 index 00000000..0b1ef320 Binary files /dev/null and b/gru_sac_predictor/results/20250416_152415/backtest_results_v7_20250416_152415.png differ diff --git a/gru_sac_predictor/results/20250416_152415/config_20250416_152415.json b/gru_sac_predictor/results/20250416_152415/config_20250416_152415.json new file mode 100644 index 00000000..a6122e67 --- /dev/null +++ b/gru_sac_predictor/results/20250416_152415/config_20250416_152415.json @@ -0,0 +1,65 @@ +{ + "run_id": "20250416_152415", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/run_20250416_152415", + "results_plot_path": "v7/results/20250416_152415/backtest_results_v7_20250416_152415.png", + "report_save_path": "v7/results/20250416_152415/backtest_performance_report_v7_20250416_152415.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 2, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_alpha": 0.05, + "sac_actor_lr": 0.0005, + "sac_critic_lr": 0.0008, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 10.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "total_training_steps": 1000, + "experience_config": { + "initial_experiences": 3000, + "experiences_per_batch": 64, + "batch_generation_interval": 500, + "balance_market_regimes": false, + "recency_bias_strength": 0.5, + "high_uncertainty_quantile": 0.75, + "extreme_return_quantile": 0.1, + "min_uncertainty_ratio": 0.2, + "min_extreme_return_ratio": 0.1, + "use_parallel_generation": false, + "precompute_all_gru_outputs": true, + "buffer_update_strategy": "fifo", + "training_iterations_per_step": 1 + }, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_152415/sac_training_history_20250416_152415.png b/gru_sac_predictor/results/20250416_152415/sac_training_history_20250416_152415.png new file mode 100644 index 00000000..bc479347 Binary files /dev/null and b/gru_sac_predictor/results/20250416_152415/sac_training_history_20250416_152415.png differ diff --git a/gru_sac_predictor/results/20250416_153132/backtest_performance_report_v7_20250416_153132.md b/gru_sac_predictor/results/20250416_153132/backtest_performance_report_v7_20250416_153132.md new file mode 100644 index 00000000..25402c96 --- /dev/null +++ b/gru_sac_predictor/results/20250416_153132/backtest_performance_report_v7_20250416_153132.md @@ -0,0 +1,42 @@ +# GRU+SAC Backtesting Performance Report + +Report generated on: 2025-04-16 15:33:21.447644 +Data range: 2025-03-06 15:23:00+00:00 to 2025-03-07 23:57:00+00:00 +Total duration: 1 days 08:34:00 + +## Strategy Performance Metrics + +* **Initial capital:** $10,000.00 +* **Final portfolio value:** $8,969.88 +* **Total return:** -10.30% +* **Annualized return:** -100.00% +* **Sharpe ratio (annualized):** -42.8779 +* **Sortino ratio (annualized):** -49.6834 +* **Volatility (annualized):** 68.01% +* **Maximum drawdown:** 10.92% +* **Total trades:** 1771 + +## Buy and Hold Benchmark + +* **Final value (B&H):** $9,658.75 +* **Total return (B&H):** -3.41% + +## Position & Prediction Analysis + +* **Average absolute position size:** 0.5605 +* **Position sign accuracy vs return:** 49.07% +* **Prediction sign accuracy vs return:** 48.92% +* **Prediction RMSE (on returns):** 0.004036 + +## Correlations + +* **Prediction-Return correlation:** -0.0042 +* **Prediction-Position correlation:** -0.3021 +* **Uncertainty-Position Size correlation:** 0.9760 + +## Notes + +* Transaction cost used: 0.0500% per position change value. +* GRU lookback period: 60 minutes. +* V6 features + return features used. +* Uncertainty estimated via MC Dropout standard deviation. diff --git a/gru_sac_predictor/results/20250416_153132/backtest_results_v7_20250416_153132.png b/gru_sac_predictor/results/20250416_153132/backtest_results_v7_20250416_153132.png new file mode 100644 index 00000000..f1c6803f Binary files /dev/null and b/gru_sac_predictor/results/20250416_153132/backtest_results_v7_20250416_153132.png differ diff --git a/gru_sac_predictor/results/20250416_153132/config_20250416_153132.json b/gru_sac_predictor/results/20250416_153132/config_20250416_153132.json new file mode 100644 index 00000000..a0ca8a4c --- /dev/null +++ b/gru_sac_predictor/results/20250416_153132/config_20250416_153132.json @@ -0,0 +1,68 @@ +{ + "run_id": "20250416_153132", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/run_20250416_153132", + "results_plot_path": "v7/results/20250416_153132/backtest_results_v7_20250416_153132.png", + "report_save_path": "v7/results/20250416_153132/backtest_performance_report_v7_20250416_153132.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 2, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_alpha": 0.05, + "sac_actor_lr": 0.0005, + "sac_critic_lr": 0.0008, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 10.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "total_training_steps": 1000, + "experience_config": { + "initial_experiences": 3000, + "experiences_per_batch": 64, + "batch_generation_interval": 500, + "balance_market_regimes": false, + "recency_bias_strength": 0.5, + "high_uncertainty_quantile": 0.75, + "extreme_return_quantile": 0.1, + "min_uncertainty_ratio": 0.2, + "min_extreme_return_ratio": 0.1, + "use_parallel_generation": false, + "precompute_all_gru_outputs": true, + "buffer_update_strategy": "fifo", + "training_iterations_per_step": 1 + }, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "opportunity_cost_penalty_factor": 1.0, + "high_return_threshold": 0.002, + "action_tolerance": 0.5, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_153132/sac_training_history_20250416_153132.png b/gru_sac_predictor/results/20250416_153132/sac_training_history_20250416_153132.png new file mode 100644 index 00000000..1778d69b Binary files /dev/null and b/gru_sac_predictor/results/20250416_153132/sac_training_history_20250416_153132.png differ diff --git a/gru_sac_predictor/results/20250416_153846/backtest_performance_report_v7_20250416_153846.md b/gru_sac_predictor/results/20250416_153846/backtest_performance_report_v7_20250416_153846.md new file mode 100644 index 00000000..5ed4bdd3 --- /dev/null +++ b/gru_sac_predictor/results/20250416_153846/backtest_performance_report_v7_20250416_153846.md @@ -0,0 +1,42 @@ +# GRU+SAC Backtesting Performance Report + +Report generated on: 2025-04-16 15:45:06.190054 +Data range: 2025-03-06 15:23:00+00:00 to 2025-03-07 23:57:00+00:00 +Total duration: 1 days 08:34:00 + +## Strategy Performance Metrics + +* **Initial capital:** $10,000.00 +* **Final portfolio value:** $9,859.25 +* **Total return:** -1.41% +* **Annualized return:** -97.83% +* **Sharpe ratio (annualized):** -45.8767 +* **Sortino ratio (annualized):** -51.2465 +* **Volatility (annualized):** 8.35% +* **Maximum drawdown:** 1.50% +* **Total trades:** 623 + +## Buy and Hold Benchmark + +* **Final value (B&H):** $9,658.75 +* **Total return (B&H):** -3.41% + +## Position & Prediction Analysis + +* **Average absolute position size:** 0.0662 +* **Position sign accuracy vs return:** 49.07% +* **Prediction sign accuracy vs return:** 48.92% +* **Prediction RMSE (on returns):** 0.004036 + +## Correlations + +* **Prediction-Return correlation:** -0.0042 +* **Prediction-Position correlation:** -0.4139 +* **Uncertainty-Position Size correlation:** 1.0000 + +## Notes + +* Transaction cost used: 0.0500% per position change value. +* GRU lookback period: 60 minutes. +* V6 features + return features used. +* Uncertainty estimated via MC Dropout standard deviation. diff --git a/gru_sac_predictor/results/20250416_153846/backtest_results_v7_20250416_153846.png b/gru_sac_predictor/results/20250416_153846/backtest_results_v7_20250416_153846.png new file mode 100644 index 00000000..cde0cc84 Binary files /dev/null and b/gru_sac_predictor/results/20250416_153846/backtest_results_v7_20250416_153846.png differ diff --git a/gru_sac_predictor/results/20250416_153846/config_20250416_153846.json b/gru_sac_predictor/results/20250416_153846/config_20250416_153846.json new file mode 100644 index 00000000..b32e374b --- /dev/null +++ b/gru_sac_predictor/results/20250416_153846/config_20250416_153846.json @@ -0,0 +1,68 @@ +{ + "run_id": "20250416_153846", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/run_20250416_153846", + "results_plot_path": "v7/results/20250416_153846/backtest_results_v7_20250416_153846.png", + "report_save_path": "v7/results/20250416_153846/backtest_performance_report_v7_20250416_153846.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 2, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_alpha": 0.1, + "sac_actor_lr": 0.0003, + "sac_critic_lr": 0.0005, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 2.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "total_training_steps": 5000, + "experience_config": { + "initial_experiences": 3000, + "experiences_per_batch": 64, + "batch_generation_interval": 500, + "balance_market_regimes": false, + "recency_bias_strength": 0.5, + "high_uncertainty_quantile": 0.75, + "extreme_return_quantile": 0.1, + "min_uncertainty_ratio": 0.2, + "min_extreme_return_ratio": 0.1, + "use_parallel_generation": false, + "precompute_all_gru_outputs": true, + "buffer_update_strategy": "fifo", + "training_iterations_per_step": 1 + }, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "opportunity_cost_penalty_factor": 1.0, + "high_return_threshold": 0.002, + "action_tolerance": 0.3, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_153846/sac_training_history_20250416_153846.png b/gru_sac_predictor/results/20250416_153846/sac_training_history_20250416_153846.png new file mode 100644 index 00000000..f5736ca7 Binary files /dev/null and b/gru_sac_predictor/results/20250416_153846/sac_training_history_20250416_153846.png differ diff --git a/gru_sac_predictor/results/20250416_154636/backtest_performance_report_v7_20250416_154636.md b/gru_sac_predictor/results/20250416_154636/backtest_performance_report_v7_20250416_154636.md new file mode 100644 index 00000000..8b38c34c --- /dev/null +++ b/gru_sac_predictor/results/20250416_154636/backtest_performance_report_v7_20250416_154636.md @@ -0,0 +1,42 @@ +# GRU+SAC Backtesting Performance Report + +Report generated on: 2025-04-16 16:07:30.131377 +Data range: 2025-03-06 15:23:00+00:00 to 2025-03-07 23:57:00+00:00 +Total duration: 1 days 08:34:00 + +## Strategy Performance Metrics + +* **Initial capital:** $10,000.00 +* **Final portfolio value:** $9,808.33 +* **Total return:** -1.92% +* **Annualized return:** -99.47% +* **Sharpe ratio (annualized):** -8.4100 +* **Sortino ratio (annualized):** -12.3689 +* **Volatility (annualized):** 60.07% +* **Maximum drawdown:** 5.96% +* **Total trades:** 1745 + +## Buy and Hold Benchmark + +* **Final value (B&H):** $9,658.75 +* **Total return (B&H):** -3.41% + +## Position & Prediction Analysis + +* **Average absolute position size:** 0.4862 +* **Position sign accuracy vs return:** 50.93% +* **Prediction sign accuracy vs return:** 48.92% +* **Prediction RMSE (on returns):** 0.004036 + +## Correlations + +* **Prediction-Return correlation:** -0.0042 +* **Prediction-Position correlation:** 0.3155 +* **Uncertainty-Position Size correlation:** 0.9861 + +## Notes + +* Transaction cost used: 0.0500% per position change value. +* GRU lookback period: 60 minutes. +* V6 features + return features used. +* Uncertainty estimated via MC Dropout standard deviation. diff --git a/gru_sac_predictor/results/20250416_154636/backtest_results_v7_20250416_154636.png b/gru_sac_predictor/results/20250416_154636/backtest_results_v7_20250416_154636.png new file mode 100644 index 00000000..e9d0556c Binary files /dev/null and b/gru_sac_predictor/results/20250416_154636/backtest_results_v7_20250416_154636.png differ diff --git a/gru_sac_predictor/results/20250416_154636/config_20250416_154636.json b/gru_sac_predictor/results/20250416_154636/config_20250416_154636.json new file mode 100644 index 00000000..a5ead9ac --- /dev/null +++ b/gru_sac_predictor/results/20250416_154636/config_20250416_154636.json @@ -0,0 +1,68 @@ +{ + "run_id": "20250416_154636", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/run_20250416_154636", + "results_plot_path": "v7/results/20250416_154636/backtest_results_v7_20250416_154636.png", + "report_save_path": "v7/results/20250416_154636/backtest_performance_report_v7_20250416_154636.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 2, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_alpha": 0.1, + "sac_actor_lr": 0.0003, + "sac_critic_lr": 0.0005, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 2.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "total_training_steps": 5000, + "experience_config": { + "initial_experiences": 3000, + "experiences_per_batch": 64, + "batch_generation_interval": 500, + "balance_market_regimes": false, + "recency_bias_strength": 0.5, + "high_uncertainty_quantile": 0.75, + "extreme_return_quantile": 0.1, + "min_uncertainty_ratio": 0.2, + "min_extreme_return_ratio": 0.1, + "use_parallel_generation": false, + "precompute_all_gru_outputs": true, + "buffer_update_strategy": "fifo", + "training_iterations_per_step": 1 + }, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "opportunity_cost_penalty_factor": 0.0, + "high_return_threshold": 0.002, + "action_tolerance": 0.3, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_154636/sac_training_history_20250416_154636.png b/gru_sac_predictor/results/20250416_154636/sac_training_history_20250416_154636.png new file mode 100644 index 00000000..44d8b43e Binary files /dev/null and b/gru_sac_predictor/results/20250416_154636/sac_training_history_20250416_154636.png differ diff --git a/gru_sac_predictor/results/20250416_162528/config_20250416_162528.json b/gru_sac_predictor/results/20250416_162528/config_20250416_162528.json new file mode 100644 index 00000000..f101a0d7 --- /dev/null +++ b/gru_sac_predictor/results/20250416_162528/config_20250416_162528.json @@ -0,0 +1,67 @@ +{ + "run_id": "20250416_162528", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/run_20250416_162528", + "results_plot_path": "v7/results/20250416_162528/backtest_results_v7_20250416_162528.png", + "report_save_path": "v7/results/20250416_162528/backtest_performance_report_v7_20250416_162528.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 5, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_actor_lr": 0.0003, + "sac_critic_lr": 0.0005, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 2.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "total_training_steps": 1000, + "experience_config": { + "initial_experiences": 3000, + "experiences_per_batch": 64, + "batch_generation_interval": 500, + "balance_market_regimes": false, + "recency_bias_strength": 0.5, + "high_uncertainty_quantile": 0.75, + "extreme_return_quantile": 0.1, + "min_uncertainty_ratio": 0.2, + "min_extreme_return_ratio": 0.1, + "use_parallel_generation": false, + "precompute_all_gru_outputs": true, + "buffer_update_strategy": "fifo", + "training_iterations_per_step": 1 + }, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "opportunity_cost_penalty_factor": 0.0, + "high_return_threshold": 0.002, + "action_tolerance": 0.3, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_162624/config_20250416_162624.json b/gru_sac_predictor/results/20250416_162624/config_20250416_162624.json new file mode 100644 index 00000000..c2cb0588 --- /dev/null +++ b/gru_sac_predictor/results/20250416_162624/config_20250416_162624.json @@ -0,0 +1,67 @@ +{ + "run_id": "20250416_162624", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/run_20250416_162624", + "results_plot_path": "v7/results/20250416_162624/backtest_results_v7_20250416_162624.png", + "report_save_path": "v7/results/20250416_162624/backtest_performance_report_v7_20250416_162624.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 5, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_actor_lr": 0.0003, + "sac_critic_lr": 0.0005, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 2.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "total_training_steps": 1000, + "experience_config": { + "initial_experiences": 3000, + "experiences_per_batch": 64, + "batch_generation_interval": 500, + "balance_market_regimes": false, + "recency_bias_strength": 0.5, + "high_uncertainty_quantile": 0.75, + "extreme_return_quantile": 0.1, + "min_uncertainty_ratio": 0.2, + "min_extreme_return_ratio": 0.1, + "use_parallel_generation": false, + "precompute_all_gru_outputs": true, + "buffer_update_strategy": "fifo", + "training_iterations_per_step": 1 + }, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "opportunity_cost_penalty_factor": 0.0, + "high_return_threshold": 0.002, + "action_tolerance": 0.3, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_162718/config_20250416_162718.json b/gru_sac_predictor/results/20250416_162718/config_20250416_162718.json new file mode 100644 index 00000000..9c12e4ec --- /dev/null +++ b/gru_sac_predictor/results/20250416_162718/config_20250416_162718.json @@ -0,0 +1,67 @@ +{ + "run_id": "20250416_162718", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/run_20250416_162718", + "results_plot_path": "v7/results/20250416_162718/backtest_results_v7_20250416_162718.png", + "report_save_path": "v7/results/20250416_162718/backtest_performance_report_v7_20250416_162718.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 5, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_actor_lr": 0.0003, + "sac_critic_lr": 0.0005, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 2.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "total_training_steps": 1000, + "experience_config": { + "initial_experiences": 3000, + "experiences_per_batch": 64, + "batch_generation_interval": 500, + "balance_market_regimes": false, + "recency_bias_strength": 0.5, + "high_uncertainty_quantile": 0.75, + "extreme_return_quantile": 0.1, + "min_uncertainty_ratio": 0.2, + "min_extreme_return_ratio": 0.1, + "use_parallel_generation": false, + "precompute_all_gru_outputs": true, + "buffer_update_strategy": "fifo", + "training_iterations_per_step": 1 + }, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "opportunity_cost_penalty_factor": 0.0, + "high_return_threshold": 0.002, + "action_tolerance": 0.3, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_162921/config_20250416_162921.json b/gru_sac_predictor/results/20250416_162921/config_20250416_162921.json new file mode 100644 index 00000000..e40eb606 --- /dev/null +++ b/gru_sac_predictor/results/20250416_162921/config_20250416_162921.json @@ -0,0 +1,67 @@ +{ + "run_id": "20250416_162921", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/run_20250416_162921", + "results_plot_path": "v7/results/20250416_162921/backtest_results_v7_20250416_162921.png", + "report_save_path": "v7/results/20250416_162921/backtest_performance_report_v7_20250416_162921.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 5, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_actor_lr": 0.0003, + "sac_critic_lr": 0.0005, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 2.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "total_training_steps": 1000, + "experience_config": { + "initial_experiences": 3000, + "experiences_per_batch": 64, + "batch_generation_interval": 500, + "balance_market_regimes": false, + "recency_bias_strength": 0.5, + "high_uncertainty_quantile": 0.75, + "extreme_return_quantile": 0.1, + "min_uncertainty_ratio": 0.2, + "min_extreme_return_ratio": 0.1, + "use_parallel_generation": false, + "precompute_all_gru_outputs": true, + "buffer_update_strategy": "fifo", + "training_iterations_per_step": 1 + }, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "opportunity_cost_penalty_factor": 0.0, + "high_return_threshold": 0.002, + "action_tolerance": 0.3, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_163030/config_20250416_163030.json b/gru_sac_predictor/results/20250416_163030/config_20250416_163030.json new file mode 100644 index 00000000..4f9ac35e --- /dev/null +++ b/gru_sac_predictor/results/20250416_163030/config_20250416_163030.json @@ -0,0 +1,67 @@ +{ + "run_id": "20250416_163030", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/run_20250416_163030", + "results_plot_path": "v7/results/20250416_163030/backtest_results_v7_20250416_163030.png", + "report_save_path": "v7/results/20250416_163030/backtest_performance_report_v7_20250416_163030.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 5, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_actor_lr": 0.0003, + "sac_critic_lr": 0.0005, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 2.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "total_training_steps": 1000, + "experience_config": { + "initial_experiences": 3000, + "experiences_per_batch": 64, + "batch_generation_interval": 500, + "balance_market_regimes": false, + "recency_bias_strength": 0.5, + "high_uncertainty_quantile": 0.75, + "extreme_return_quantile": 0.1, + "min_uncertainty_ratio": 0.2, + "min_extreme_return_ratio": 0.1, + "use_parallel_generation": false, + "precompute_all_gru_outputs": true, + "buffer_update_strategy": "fifo", + "training_iterations_per_step": 1 + }, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "opportunity_cost_penalty_factor": 0.0, + "high_return_threshold": 0.002, + "action_tolerance": 0.3, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_163440/config_20250416_163440.json b/gru_sac_predictor/results/20250416_163440/config_20250416_163440.json new file mode 100644 index 00000000..b2194903 --- /dev/null +++ b/gru_sac_predictor/results/20250416_163440/config_20250416_163440.json @@ -0,0 +1,67 @@ +{ + "run_id": "20250416_163440", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "v7/models/run_20250416_163440", + "results_plot_path": "v7/results/20250416_163440/backtest_results_v7_20250416_163440.png", + "report_save_path": "v7/results/20250416_163440/backtest_performance_report_v7_20250416_163440.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 5, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_actor_lr": 0.0003, + "sac_critic_lr": 0.0005, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 2.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "total_training_steps": 1000, + "experience_config": { + "initial_experiences": 3000, + "experiences_per_batch": 64, + "batch_generation_interval": 500, + "balance_market_regimes": false, + "recency_bias_strength": 0.5, + "high_uncertainty_quantile": 0.75, + "extreme_return_quantile": 0.1, + "min_uncertainty_ratio": 0.2, + "min_extreme_return_ratio": 0.1, + "use_parallel_generation": false, + "precompute_all_gru_outputs": true, + "buffer_update_strategy": "fifo", + "training_iterations_per_step": 1 + }, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "opportunity_cost_penalty_factor": 0.0, + "high_return_threshold": 0.002, + "action_tolerance": 0.3, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_164410/config_20250416_164410.json b/gru_sac_predictor/results/20250416_164410/config_20250416_164410.json new file mode 100644 index 00000000..126f80fc --- /dev/null +++ b/gru_sac_predictor/results/20250416_164410/config_20250416_164410.json @@ -0,0 +1,67 @@ +{ + "run_id": "20250416_164410", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "gru_sac_predictor/models/run_20250416_164410", + "results_plot_path": "gru_sac_predictor/results/20250416_164410/backtest_results_20250416_164410.png", + "report_save_path": "gru_sac_predictor/results/20250416_164410/backtest_performance_report_20250416_164410.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 5, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_actor_lr": 0.0003, + "sac_critic_lr": 0.0005, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 2.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "total_training_steps": 1000, + "experience_config": { + "initial_experiences": 3000, + "experiences_per_batch": 64, + "batch_generation_interval": 500, + "balance_market_regimes": false, + "recency_bias_strength": 0.5, + "high_uncertainty_quantile": 0.75, + "extreme_return_quantile": 0.1, + "min_uncertainty_ratio": 0.2, + "min_extreme_return_ratio": 0.1, + "use_parallel_generation": false, + "precompute_all_gru_outputs": true, + "buffer_update_strategy": "fifo", + "training_iterations_per_step": 1 + }, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "opportunity_cost_penalty_factor": 0.0, + "high_return_threshold": 0.002, + "action_tolerance": 0.3, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_164547/config_20250416_164547.json b/gru_sac_predictor/results/20250416_164547/config_20250416_164547.json new file mode 100644 index 00000000..bfc9cfa0 --- /dev/null +++ b/gru_sac_predictor/results/20250416_164547/config_20250416_164547.json @@ -0,0 +1,67 @@ +{ + "run_id": "20250416_164547", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "gru_sac_predictor/models/run_20250416_164547", + "results_plot_path": "gru_sac_predictor/results/20250416_164547/backtest_results_20250416_164547.png", + "report_save_path": "gru_sac_predictor/results/20250416_164547/backtest_performance_report_20250416_164547.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 5, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_actor_lr": 0.0003, + "sac_critic_lr": 0.0005, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 2.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "total_training_steps": 1000, + "experience_config": { + "initial_experiences": 3000, + "experiences_per_batch": 64, + "batch_generation_interval": 500, + "balance_market_regimes": false, + "recency_bias_strength": 0.5, + "high_uncertainty_quantile": 0.75, + "extreme_return_quantile": 0.1, + "min_uncertainty_ratio": 0.2, + "min_extreme_return_ratio": 0.1, + "use_parallel_generation": false, + "precompute_all_gru_outputs": true, + "buffer_update_strategy": "fifo", + "training_iterations_per_step": 1 + }, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "opportunity_cost_penalty_factor": 0.0, + "high_return_threshold": 0.002, + "action_tolerance": 0.3, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/results/20250416_164726/config_20250416_164726.json b/gru_sac_predictor/results/20250416_164726/config_20250416_164726.json new file mode 100644 index 00000000..d421cc57 --- /dev/null +++ b/gru_sac_predictor/results/20250416_164726/config_20250416_164726.json @@ -0,0 +1,67 @@ +{ + "run_id": "20250416_164726", + "db_dir": "../downloaded_data", + "ticker": "BTC-USD", + "exchange": "COINBASE", + "start_date": "2025-03-01", + "end_date": "2025-03-10", + "interval": "1min", + "model_save_path": "gru_sac_predictor/models/run_20250416_164726", + "results_plot_path": "gru_sac_predictor/results/20250416_164726/backtest_results_20250416_164726.png", + "report_save_path": "gru_sac_predictor/results/20250416_164726/backtest_performance_report_20250416_164726.md", + "train_ratio": 0.6, + "validation_ratio": 0.2, + "gru_lookback": 60, + "gru_prediction_horizon": 1, + "gru_epochs": 20, + "gru_batch_size": 32, + "gru_patience": 10, + "gru_lr_factor": 0.5, + "gru_return_scale": 0.03, + "gru_model_load_run_id": "20250416_142744", + "sac_state_dim": 5, + "sac_hidden_size": 64, + "sac_gamma": 0.97, + "sac_tau": 0.02, + "sac_actor_lr": 0.0003, + "sac_critic_lr": 0.0005, + "sac_batch_size": 64, + "sac_buffer_max_size": 20000, + "sac_min_buffer_size": 1000, + "sac_update_interval": 1, + "sac_target_update_interval": 2, + "sac_gradient_clip": 1.0, + "sac_reward_scale": 2.0, + "sac_use_batch_norm": true, + "sac_use_residual": true, + "sac_model_dir": "models/simplified_sac", + "sac_epochs": 50, + "total_training_steps": 1000, + "experience_config": { + "initial_experiences": 3000, + "experiences_per_batch": 64, + "batch_generation_interval": 500, + "balance_market_regimes": false, + "recency_bias_strength": 0.5, + "high_uncertainty_quantile": 0.75, + "extreme_return_quantile": 0.1, + "min_uncertainty_ratio": 0.2, + "min_extreme_return_ratio": 0.1, + "use_parallel_generation": false, + "precompute_all_gru_outputs": true, + "buffer_update_strategy": "fifo", + "training_iterations_per_step": 1 + }, + "initial_capital": 10000.0, + "transaction_cost": 0.0005, + "opportunity_cost_penalty_factor": 0.0, + "high_return_threshold": 0.002, + "action_tolerance": 0.3, + "load_existing_system": true, + "train_gru_model": false, + "train_sac_agent": true, + "load_sac_agent": false, + "run_backtest": true, + "generate_plots": true, + "generate_report": true +} \ No newline at end of file diff --git a/gru_sac_predictor/src/__init__.py b/gru_sac_predictor/src/__init__.py new file mode 100644 index 00000000..0519ecba --- /dev/null +++ b/gru_sac_predictor/src/__init__.py @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/gru_sac_predictor/src/__pycache__/__init__.cpython-312.pyc b/gru_sac_predictor/src/__pycache__/__init__.cpython-312.pyc new file mode 100644 index 00000000..0d1fbc17 Binary files /dev/null and b/gru_sac_predictor/src/__pycache__/__init__.cpython-312.pyc differ diff --git a/gru_sac_predictor/src/__pycache__/crypto_db_fetcher.cpython-312.pyc b/gru_sac_predictor/src/__pycache__/crypto_db_fetcher.cpython-312.pyc new file mode 100644 index 00000000..ceb82e15 Binary files /dev/null and b/gru_sac_predictor/src/__pycache__/crypto_db_fetcher.cpython-312.pyc differ diff --git a/gru_sac_predictor/src/__pycache__/data_pipeline.cpython-312.pyc b/gru_sac_predictor/src/__pycache__/data_pipeline.cpython-312.pyc new file mode 100644 index 00000000..e2827e01 Binary files /dev/null and b/gru_sac_predictor/src/__pycache__/data_pipeline.cpython-312.pyc differ diff --git a/gru_sac_predictor/src/__pycache__/gru_predictor.cpython-312.pyc b/gru_sac_predictor/src/__pycache__/gru_predictor.cpython-312.pyc new file mode 100644 index 00000000..ec868d2f Binary files /dev/null and b/gru_sac_predictor/src/__pycache__/gru_predictor.cpython-312.pyc differ diff --git a/gru_sac_predictor/src/__pycache__/sac_agent.cpython-312.pyc b/gru_sac_predictor/src/__pycache__/sac_agent.cpython-312.pyc new file mode 100644 index 00000000..eed5f91d Binary files /dev/null and b/gru_sac_predictor/src/__pycache__/sac_agent.cpython-312.pyc differ diff --git a/gru_sac_predictor/src/__pycache__/sac_agent_simplified.cpython-312.pyc b/gru_sac_predictor/src/__pycache__/sac_agent_simplified.cpython-312.pyc new file mode 100644 index 00000000..2bffe33b Binary files /dev/null and b/gru_sac_predictor/src/__pycache__/sac_agent_simplified.cpython-312.pyc differ diff --git a/gru_sac_predictor/src/__pycache__/trading_system.cpython-312.pyc b/gru_sac_predictor/src/__pycache__/trading_system.cpython-312.pyc new file mode 100644 index 00000000..bbe4ec5c Binary files /dev/null and b/gru_sac_predictor/src/__pycache__/trading_system.cpython-312.pyc differ diff --git a/gru_sac_predictor/src/crypto_db_fetcher.py b/gru_sac_predictor/src/crypto_db_fetcher.py new file mode 100644 index 00000000..4346e099 --- /dev/null +++ b/gru_sac_predictor/src/crypto_db_fetcher.py @@ -0,0 +1,471 @@ +""" +Market data fetcher module for cryptocurrency market data from SQLite databases. + +This module provides classes for fetching historical cryptocurrency market data +from SQLite databases downloaded from the crypto_md service. The raw data is always +stored in 1-minute intervals, and can be resampled to other intervals as needed. +""" + +import os +import logging +import pandas as pd +import sqlite3 +import glob +from datetime import datetime, timedelta +from typing import Dict, List, Tuple, Optional, Union +import re +import sys # Added + +# V7: Update logger setup +# Configure logging with explicit console output +logging.basicConfig( + level=logging.INFO, + format='%(asctime)s - %(name)s - %(levelname)s - %(message)s', + handlers=[ + # logging.FileHandler("db_fetcher.log", mode='a'), # Optional file logging + logging.StreamHandler(sys.stdout) + ] +) +logger = logging.getLogger(__name__) +logger.setLevel(logging.INFO) # Set level for this specific logger + +class CryptoDBFetcher: + """ + Fetches historical cryptocurrency market data from SQLite databases. + + The raw data in the SQLite databases is always stored in 1-minute intervals. + This class can resample the data to other intervals (e.g., 5min, 1h, 1d) as needed. + """ + + # V7 Update: Adjusted defaults slightly, cache dir relative to project + def __init__(self, db_dir: str = "downloaded_data", cache_dir: str = "data/cache", use_cache: bool = False): + """ + Initialize the crypto database fetcher. + + Args: + db_dir: Directory where SQLite database files are stored + cache_dir: Directory to store cached data (relative to project root) + use_cache: Whether to use cached data when available (default False for V7) + """ + # V7 Update: Make db_dir potentially relative to workspace + if not os.path.isabs(db_dir): + # This assumes the script is run from the project root (e.g., v7/) + # Adjust if running from elsewhere + base_path = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) # Go up two levels from src + self.db_dir = os.path.join(base_path, db_dir) + else: + self.db_dir = db_dir + + self.cache_dir = cache_dir + self.use_cache = use_cache + + if self.use_cache: + os.makedirs(cache_dir, exist_ok=True) + + # Map of exchanges and pairs - discovered lazily now + self._available_exchanges = None + self._available_pairs = None + self._db_files = None # Cache discovered DB files + + logger.info(f"Initialized CryptoDBFetcher with db_dir={self.db_dir}") + + @property + def available_exchanges(self) -> List[str]: + if self._available_exchanges is None: + self._available_exchanges = self._discover_available_exchanges() + logger.info(f"Discovered exchanges: {', '.join(self._available_exchanges)}") + return self._available_exchanges + + @property + def available_pairs(self) -> List[str]: + if self._available_pairs is None: + self._available_pairs = self._discover_available_pairs() + logger.info(f"Discovered pairs: {', '.join(self._available_pairs)}") + return self._available_pairs + + def _discover_available_exchanges(self) -> List[str]: + """Discover available exchanges from database files.""" + exchanges = set() + try: + db_files = self._get_db_files() + if not db_files: return [] + + # Use the first DB file to discover exchanges + with sqlite3.connect(db_files[0]) as conn: + tables = conn.execute("SELECT name FROM sqlite_master WHERE type='table' AND name LIKE '%_ohlcv_%'").fetchall() + for table in tables: + table_name = table[0] + try: + # Attempt 1: Extract from table name (e.g., coinbase_ohlcv_1min) + match = re.match(r'([a-z0-9]+)_ohlcv_', table_name, re.IGNORECASE) + if match: + exchanges.add(match.group(1).upper()) + # Attempt 2: Query distinct exchange_id if column exists + elif 'exchange_id' in [c[1].lower() for c in conn.execute(f"PRAGMA table_info({table_name})").fetchall()]: + query = f"SELECT DISTINCT exchange_id FROM {table_name}" + for row in conn.execute(query).fetchall(): exchanges.add(row[0].upper()) + except sqlite3.Error as e: logger.debug(f"Could not query table {table_name}: {e}") + + return sorted(list(exchanges)) + except Exception as e: logger.error(f"Error discovering exchanges: {e}"); return [] + + def _discover_available_pairs(self) -> List[str]: + """Discover available trading pairs from database files.""" + pairs = set() + try: + db_files = self._get_db_files() + if not db_files: return [] + + with sqlite3.connect(db_files[0]) as conn: + tables = conn.execute("SELECT name FROM sqlite_master WHERE type='table' AND name LIKE '%_ohlcv_%'").fetchall() + for table in tables: + table_name = table[0] + try: + # Check if instrument_id column exists + if 'instrument_id' in [c[1].lower() for c in conn.execute(f"PRAGMA table_info({table_name})").fetchall()]: + query = f"SELECT DISTINCT instrument_id FROM {table_name}" + for row in conn.execute(query).fetchall(): + instr_id = row[0] + if instr_id.startswith("PAIR-"): instr_id = instr_id[5:] + pairs.add(instr_id) + else: + # Attempt to parse from table name if instrument_id col missing + # e.g., coinbase_ohlcv_1min_btc_usdt + match = re.search(r'_([A-Z0-9]+_[A-Z0-9]+)$|_([A-Z0-9]+-[A-Z0-9]+)$', table_name, re.IGNORECASE) + if match: + pair = match.group(1) or match.group(2) + if pair: pairs.add(pair.replace('_','-').upper()) + except sqlite3.Error as e: logger.debug(f"Could not query table {table_name}: {e}") + return sorted(list(pairs)) + except Exception as e: logger.error(f"Error discovering pairs: {e}"); return [] + + def _get_db_files(self) -> List[str]: + """Get available database files, sorted by date desc (cached).""" + if self._db_files is not None: + return self._db_files + + logger.info(f"Scanning for DB files in: {self.db_dir}") + if not os.path.exists(self.db_dir): + logger.error(f"Database directory {self.db_dir} does not exist") + self._db_files = [] + return [] + + patterns = ["*.mktdata.ohlcv.db", "*.db", "*.sqlite", "*.sqlite3"] + db_files = [] + for pattern in patterns: + files = glob.glob(os.path.join(self.db_dir, pattern)) + if files: logger.debug(f"Found {len(files)} files with pattern {pattern}") + db_files.extend(files) + + if not db_files: + logger.warning(f"No database files found in {self.db_dir} matching patterns: {patterns}") + self._db_files = [] + return [] + + db_files = list(set(db_files)) # Remove duplicates + + # Sort by date (newest first) if possible + date_pattern = re.compile(r'(\d{8})') + file_dates = [] + for file in db_files: + basename = os.path.basename(file) + match = date_pattern.search(basename) + date_obj = None + if match: + try: date_obj = pd.to_datetime(match.group(1), format='%Y%m%d') + except ValueError: pass + # Fallback: try modification time + if date_obj is None: + try: date_obj = pd.to_datetime(os.path.getmtime(file), unit='s') + except Exception: date_obj = pd.Timestamp.min # Default to oldest if error + file_dates.append((date_obj, file)) + + file_dates.sort(key=lambda x: x[0], reverse=True) # Sort by date object, newest first + self._db_files = [file for _, file in file_dates] + + logger.info(f"Found {len(self._db_files)} DB files. Using newest: {os.path.basename(self._db_files[0]) if self._db_files else 'None'}") + return self._db_files + + # V7 Update: Simplified date finding - use files covering the range + def _get_relevant_db_files(self, start_dt: pd.Timestamp, end_dt: pd.Timestamp) -> List[str]: + """Find DB files potentially containing data for the date range.""" + all_files = self._get_db_files() + relevant_files = set() + date_pattern = re.compile(r'(\d{8})') + + for file in all_files: + basename = os.path.basename(file) + match = date_pattern.search(basename) + if match: + try: + file_date = pd.to_datetime(match.group(1), format='%Y%m%d') + # Check if the file's date is within or overlaps the target range + # (Assume file contains data for that single day) + if start_dt.date() <= file_date.date() <= end_dt.date(): + relevant_files.add(file) + except ValueError: + pass # Ignore files with unparseable dates + else: + # If no date in filename, conservatively include recent files + # based on modification time (might be less accurate) + try: + mod_time = pd.to_datetime(os.path.getmtime(file), unit='s') + # Include if modified within or shortly after the requested range + if start_dt <= mod_time <= (end_dt + timedelta(days=1)): + relevant_files.add(file) + except Exception: + pass # Ignore files with mod time errors + + # If no files found based on date, return the most recent one as a fallback + if not relevant_files and all_files: + logger.warning(f"No DB files found matching date range {start_dt.date()} - {end_dt.date()}. Using most recent file.") + return [all_files[0]] + elif not relevant_files: + logger.error("No relevant DB files found and no fallback files available.") + return [] + + # Sort the relevant files chronologically (oldest first for processing) + return sorted(list(relevant_files), key=lambda f: os.path.basename(f)) + + def _convert_ticker_to_instrument_id(self, ticker: str) -> str: + if not ticker.startswith("PAIR-"): return f"PAIR-{ticker}" + return ticker + + def _convert_interval(self, interval: str) -> Optional[str]: + interval = interval.lower() + interval_map = {"1m": "1min", "5m": "5min", "15m": "15min", "30m": "30min", + "1h": "1hour", "4h": "4hour", "1d": "1day", "1D": "1day"} + if interval in interval_map: return interval_map[interval] + if interval.endswith(('min', 'hour', 'day')): return interval + logger.warning(f"Unsupported interval format: {interval}") + return None # Return None for unsupported intervals + + def _get_table_name(self, conn: sqlite3.Connection, exchange: str, interval: str = "1min") -> Optional[str]: + """Find the correct table name, trying variations.""" + base_table = f"{exchange.lower()}_ohlcv_{interval}" + cursor = conn.cursor() + cursor.execute("SELECT name FROM sqlite_master WHERE type='table'") + tables = [row[0] for row in cursor.fetchall()] + + if base_table in tables: return base_table + + # Try variations (case, interval format) + variations = [ + f"{exchange.upper()}_ohlcv_{interval}", + f"{exchange.lower()}_ohlcv_1m", # Always check 1min as source + f"{exchange.upper()}_ohlcv_1m", + ] + for var in variations: + if var in tables: + logger.debug(f"Found table using variation: {var}") + return var + + # Check if any OHLCV table exists for the exchange + for t in tables: + if t.lower().startswith(f"{exchange.lower()}_ohlcv_"): + logger.warning(f"Using first available OHLCV table for exchange: {t}") + return t + + logger.warning(f"Table for {exchange} interval {interval} not found.") + return None + + def _query_data_from_db(self, db_file: str, ticker: str, start_timestamp: int, end_timestamp: int, + interval: str = "1min", exchange: str = "COINBASE") -> pd.DataFrame: + """ + Query market data from a database file. + """ + instrument_id = self._convert_ticker_to_instrument_id(ticker) + # Always query 1min interval from DB as it's the source resolution + query_interval = "1min" + + try: + # logger.debug(f"Querying DB {db_file} for {instrument_id}...") + with sqlite3.connect(db_file) as conn: + table_name = self._get_table_name(conn, exchange, query_interval) + if not table_name: + return pd.DataFrame() + + cursor = conn.cursor() + cursor.execute(f"PRAGMA table_info({table_name})") + columns_info = cursor.fetchall() + column_names = [col[1].lower() for col in columns_info] + # logger.debug(f"Columns in {table_name}: {column_names}") + + # Build query dynamically + select_cols = ["tstamp", "open", "high", "low", "close", "volume"] + select_str = ", ".join([c for c in select_cols if c in column_names]) + if not all(c in column_names for c in ["tstamp", "open", "high", "low", "close", "volume"]): + logger.warning(f"Table {table_name} in {db_file} missing standard OHLCV columns.") + # Attempt to map common variations if needed - skipped for simplicity + # For now, return empty if standard columns are missing + return pd.DataFrame() + + where_clauses = ["tstamp >= ?", "tstamp <= ?"] + params = [start_timestamp, end_timestamp] + + if 'instrument_id' in column_names: + where_clauses.append("instrument_id = ?") + params.append(instrument_id) + if 'exchange_id' in column_names: + # Normalize exchange name from DB if needed + cursor.execute(f"SELECT DISTINCT exchange_id FROM {table_name} WHERE exchange_id LIKE ? LIMIT 1", (f'%{exchange}%',)) + db_exchange = cursor.fetchone() + if db_exchange: + params.append(db_exchange[0]) + where_clauses.append("exchange_id = ?") + else: # If exchange not found, query might return empty + params.append(exchange) + where_clauses.append("exchange_id = ?") + + query = f"SELECT {select_str} FROM {table_name} WHERE {' AND '.join(where_clauses)} ORDER BY tstamp" + # logger.debug(f"Executing query: {query} with params {params[:2]}...{params[-1]}") + + df = pd.read_sql_query(query, conn, params=params) + if df.empty: return pd.DataFrame() + + # Convert timestamp and set index + df['date'] = pd.to_datetime(df['tstamp'], unit='ns', utc=True) + df = df.set_index('date').drop(columns=['tstamp']) # Drop original tstamp + # logger.debug(f"Query returned {len(df)} rows.") + return df + + except Exception as e: + logger.error(f"Error querying {db_file} table {table_name if 'table_name' in locals() else 'N/A'}: {e}", exc_info=False) + return pd.DataFrame() + + def _resample_data(self, df: pd.DataFrame, interval: str) -> pd.DataFrame: + """Resample 1-minute data to a different interval.""" + if df.empty or not isinstance(df.index, pd.DatetimeIndex): return df + + # V7 Update: More robust interval conversion + try: + freq = pd.tseries.frequencies.to_offset(interval) + if freq is None: raise ValueError("Invalid frequency") + except ValueError: + # Try manual mapping for common cases + interval_map = {"1min": "1min", "5min": "5min", "15min": "15min", "30min": "30min", + "1hour": "1h", "4hour": "4h", "1day": "1d"} + if interval in interval_map: freq = interval_map[interval] + else: logger.error(f"Unsupported interval for resampling: {interval}"); return df + + logger.info(f"Resampling data to {freq}...") + try: + agg_dict = {'open': 'first', 'high': 'max', 'low': 'min', 'close': 'last'} + # Only include volume if present + if 'volume' in df.columns: agg_dict['volume'] = 'sum' + + # Check for required columns + missing_cols = [c for c in ['open','high','low','close'] if c not in df.columns] + if missing_cols: + logger.error(f"Cannot resample, missing required columns: {missing_cols}") + return pd.DataFrame() # Return empty if essential cols missing + + resampled = df.resample(freq).agg(agg_dict) + resampled = resampled.dropna(subset=['open', 'high', 'low', 'close']) # Drop rows where OHLC couldn't be computed + logger.info(f"Resampling complete. New shape: {resampled.shape}") + + except Exception as e: + logger.error(f"Error during resampling to {freq}: {e}", exc_info=True) + return pd.DataFrame() # Return empty on error + + return resampled + + def fetch_data(self, ticker: str, start_date: str = None, end_date: str = None, interval: str = "1min", + exchange: str = "COINBASE") -> pd.DataFrame: + """ + Fetch cryptocurrency market data for a given ticker and date range. + Always sources 1-minute data from DB and resamples if needed. + """ + logger.info(f"Fetching {ticker} data from {start_date} to {end_date} at {interval} (Exchange: {exchange})...") + + # V7 Update: Stricter date handling + try: + start_dt = pd.to_datetime(start_date, utc=True) if start_date else pd.Timestamp.now(tz='utc') - timedelta(days=30) + end_dt = pd.to_datetime(end_date, utc=True) if end_date else pd.Timestamp.now(tz='utc') + if start_dt >= end_dt: raise ValueError("Start date must be before end date") + except Exception as e: logger.error(f"Invalid date format/range: {e}"); return pd.DataFrame() + logger.info(f"Querying date range: {start_dt.date()} to {end_dt.date()}") + + # V7 Update: Check supported interval format + target_interval = interval # Store requested interval + resample_freq_pd = None + try: resample_freq_pd = pd.tseries.frequencies.to_offset(target_interval) + except ValueError: + interval_map = {"1min": "1T", "5min": "5T", "15min": "15T", "30min": "30T", + "1hour": "1H", "4hour": "4H", "1day": "1D"} + if target_interval in interval_map: resample_freq_pd = interval_map[target_interval] + else: logger.error(f"Unsupported interval: {target_interval}"); return pd.DataFrame() + + # V7 Update: Cache key includes exchange + cache_key = f"{exchange}_{ticker}_{start_dt.strftime('%Y%m%d')}_{end_dt.strftime('%Y%m%d')}_{target_interval}".replace("-","_") + cache_path = os.path.join(self.cache_dir, f"{cache_key}.parquet") # Use parquet for better type handling + + if self.use_cache and os.path.exists(cache_path): + logger.info(f"Loading data from cache: {cache_path}") + try: + data = pd.read_parquet(cache_path) + # Ensure index is datetime and UTC (Parquet often preserves this) + if not isinstance(data.index, pd.DatetimeIndex): data.index = pd.to_datetime(data.index) + if data.index.tz is None: data.index = data.index.tz_localize('utc') + elif data.index.tz != 'UTC': data.index = data.index.tz_convert('utc') + logger.info(f"Loaded {len(data)} rows from cache.") + return data + except Exception as e: logger.warning(f"Error loading cache: {e}. Fetching fresh data.") + + # Convert timestamps for DB query + start_timestamp_ns = int(start_dt.timestamp() * 1e9) + end_timestamp_ns = int(end_dt.timestamp() * 1e9) + + # V7 Update: Use _get_relevant_db_files + db_files_to_query = self._get_relevant_db_files(start_dt, end_dt) + if not db_files_to_query: + logger.error("No relevant database files found for the specified date range.") + return pd.DataFrame() + logger.info(f"Querying {len(db_files_to_query)} DB files: {[os.path.basename(f) for f in db_files_to_query]}") + + all_data = [] + for db_file in db_files_to_query: + # Always query 1min data + df = self._query_data_from_db(db_file, ticker, start_timestamp_ns, end_timestamp_ns, "1min", exchange) + if not df.empty: all_data.append(df) + + if not all_data: logger.warning(f"No data found in DBs for {ticker}..."); return pd.DataFrame() + + combined_df = pd.concat(all_data) + combined_df = combined_df[~combined_df.index.duplicated(keep='first')].sort_index() + # Filter exact date range AFTER combining, before resampling + combined_df = combined_df[(combined_df.index >= start_dt) & (combined_df.index <= end_dt)] + logger.info(f"Combined data shape before resampling: {combined_df.shape}") + + # Resample if target interval is not 1 minute + final_df = combined_df + if target_interval != "1min": + final_df = self._resample_data(combined_df, target_interval) + if final_df.empty: + logger.error("Resampling resulted in empty DataFrame.") + return pd.DataFrame() + + # Cache the result + if self.use_cache: + try: + os.makedirs(os.path.dirname(cache_path), exist_ok=True) + final_df.to_parquet(cache_path) + logger.info(f"Saved {len(final_df)} rows to cache: {cache_path}") + except Exception as e: logger.warning(f"Failed to save cache: {e}") + + logger.info(f"Fetch data complete. Returning {len(final_df)} rows.") + return final_df + + # ... (keep _extract_date_from_filename) ... + def _extract_date_from_filename(self, filename): + if not filename: return None + base_name = os.path.basename(filename) + match = re.match(r"(\d{8})\.mktdata", base_name) + if match: + try: return datetime.strptime(match.group(1), "%Y%m%d").date() + except ValueError: return None + return None + +# --- Removed fetch_intraday_data, fetch_daily_data, batch_fetch_data, DataFetcherFactory --- +# These methods are not directly used by the V7 workflow as defined so far. +# They can be added back if needed. \ No newline at end of file diff --git a/gru_sac_predictor/src/data_pipeline.py b/gru_sac_predictor/src/data_pipeline.py new file mode 100644 index 00000000..cac5b638 --- /dev/null +++ b/gru_sac_predictor/src/data_pipeline.py @@ -0,0 +1,290 @@ +import pandas as pd +import numpy as np +import logging +import os +import sys + +# V7 Update: Import the fetcher +from .crypto_db_fetcher import CryptoDBFetcher + +data_pipeline_logger = logging.getLogger(__name__) + +# V7 Update: Add load_data_from_db function from V6 +def load_data_from_db( + db_dir: str, + ticker: str, + exchange: str, + start_date: str, + end_date: str, + interval: str = "1min" +) -> pd.DataFrame: + """ + Loads cryptocurrency OHLCV data from the local SQLite database using CryptoDBFetcher. + Adapted from V6. + + Args: + db_dir: Directory containing the SQLite database files. + ticker: The trading pair symbol (e.g., 'BTC-USDT'). + exchange: The exchange name (e.g., 'COINBASE'). + start_date: Start date string (YYYY-MM-DD). + end_date: End date string (YYYY-MM-DD). + interval: The desired data interval (e.g., '1min', '5min', '1h'). + + Returns: + A Pandas DataFrame containing the OHLCV data, indexed by timestamp. + Returns an empty DataFrame if data loading fails. + """ + data_pipeline_logger.info(f"Loading data via DB: {ticker} from {exchange} ({start_date} to {end_date}, interval: {interval})") + try: + # Initialize fetcher (db_dir path is handled within fetcher now) + fetcher = CryptoDBFetcher(db_dir=db_dir) + df = fetcher.fetch_data( + ticker=ticker, + start_date=start_date, + end_date=end_date, + interval=interval, + exchange=exchange + ) + if df.empty: + data_pipeline_logger.warning(f"No data found for {ticker} in the specified DB range.") + else: + # Ensure index is datetime and timezone-aware (UTC) + if not isinstance(df.index, pd.DatetimeIndex): + try: + df.index = pd.to_datetime(df.index, errors='coerce', utc=True) + if df.index.isnull().any(): + data_pipeline_logger.warning("Dropping rows with invalid datetime index after conversion.") + df = df.dropna(subset=[df.index.name]) + except Exception as idx_e: + data_pipeline_logger.error(f"Failed to convert index to DatetimeIndex: {idx_e}") + return pd.DataFrame() + elif df.index.tz is None: # Ensure timezone if index is already datetime + df.index = df.index.tz_localize('utc') + elif df.index.tz != 'UTC': # Convert to UTC if different timezone + df.index = df.index.tz_convert('utc') + + df = df.sort_index() + data_pipeline_logger.info(f"Successfully loaded {len(df)} rows from DB.") + + return df + except Exception as e: + data_pipeline_logger.error(f"Error loading data from database via fetcher: {e}", exc_info=True) + return pd.DataFrame() + +def create_data_pipeline(historical_data, split_ratios=[0.6, 0.2, 0.2]): + """ + Prepare data pipeline for training both GRU and SAC models using chronological split. + + Args: + historical_data: DataFrame with OHLCV data, sorted chronologically. + Must have a DatetimeIndex. + split_ratios: Train/validation/test split ratios based on time. + + Returns: + Tuple of (train_data, validation_data, test_data) DataFrames. + """ + if historical_data is None or historical_data.empty: + logging.error("Input data is empty, cannot create pipeline.") + return None, None, None + + # Ensure data is sorted by time + historical_data.sort_index(inplace=True) + + # V7 Change: Use index for duration calculation + try: + if not isinstance(historical_data.index, pd.DatetimeIndex): + raise TypeError("Data index must be a DatetimeIndex for splitting.") + total_duration = historical_data.index[-1] - historical_data.index[0] + except IndexError: + logging.error("Cannot calculate duration: Data has insufficient rows.") + return None, None, None + except TypeError as e: + logging.error(f"Error calculating duration: {e}") + return None, None, None + + train_ratio, val_ratio, test_ratio = split_ratios + + # Calculate split points based on time duration + train_end_time = historical_data.index[0] + total_duration * train_ratio + val_end_time = train_end_time + total_duration * val_ratio + + # Perform the split using time index + # Ensure the split points are valid timestamps within the index range + # Use searchsorted to find the index locations closest to the calculated times + train_end_idx = historical_data.index.searchsorted(train_end_time) + val_end_idx = historical_data.index.searchsorted(val_end_time) + + train_data = historical_data.iloc[:train_end_idx] + val_data = historical_data.iloc[train_end_idx:val_end_idx] + test_data = historical_data.iloc[val_end_idx:] + + logging.info(f"Data split complete:") + logging.info(f" Train: {len(train_data)} rows ({train_data.index.min()} to {train_data.index.max()})") + logging.info(f" Validation: {len(val_data)} rows ({val_data.index.min()} to {val_data.index.max()})") + logging.info(f" Test: {len(test_data)} rows ({test_data.index.min()} to {test_data.index.max()})") + + return train_data, val_data, test_data + +def create_sequences_v2(features_scaled, targets_scaled, start_price_unscaled, seq_length=60): + """ + Create sequences for GRU training, handling potential mismatches in indices. + + Args: + features_scaled: Scaled feature DataFrame + targets_scaled: Scaled target Series + start_price_unscaled: Unscaled starting price Series + seq_length: Sequence length for GRU + + Returns: + Tuple of (X sequences, y targets, starting prices) or (None, None, None) if creation fails + """ + data_pipeline_logger.info(f"Creating sequences (v2) with length {seq_length}...") + + try: + # Type checking and conversion for features + if features_scaled is None: + data_pipeline_logger.error("features_scaled is None") + return None, None, None + + if not isinstance(features_scaled, pd.DataFrame): + data_pipeline_logger.warning(f"features_scaled is not DataFrame but {type(features_scaled)}") + try: + if isinstance(features_scaled, pd.Series): + # Try to convert Series to DataFrame + features_scaled = pd.DataFrame(features_scaled) + else: + # Try to convert numpy array to DataFrame + features_scaled = pd.DataFrame(features_scaled) + except Exception as e: + data_pipeline_logger.error(f"Failed to convert features_scaled to DataFrame: {e}") + return None, None, None + + # Type checking and conversion for targets + if targets_scaled is None: + data_pipeline_logger.error("targets_scaled is None") + return None, None, None + + if not isinstance(targets_scaled, pd.Series): + data_pipeline_logger.warning(f"targets_scaled is not Series but {type(targets_scaled)}") + try: + if isinstance(targets_scaled, pd.DataFrame) and targets_scaled.shape[1] == 1: + # Convert single-column DataFrame to Series + targets_scaled = targets_scaled.iloc[:, 0] + elif isinstance(targets_scaled, np.ndarray) and targets_scaled.ndim == 1: + # Convert 1D array to Series + targets_scaled = pd.Series(targets_scaled) + else: + data_pipeline_logger.error(f"targets_scaled shape is not compatible: {getattr(targets_scaled, 'shape', 'unknown')}") + return None, None, None + except Exception as e: + data_pipeline_logger.error(f"Failed to convert targets_scaled to Series: {e}") + return None, None, None + + # Type checking and conversion for prices + if start_price_unscaled is None: + data_pipeline_logger.error("start_price_unscaled is None") + return None, None, None + + if not isinstance(start_price_unscaled, pd.Series): + data_pipeline_logger.warning(f"start_price_unscaled is not Series but {type(start_price_unscaled)}") + try: + if isinstance(start_price_unscaled, pd.DataFrame) and start_price_unscaled.shape[1] == 1: + # Convert single-column DataFrame to Series + start_price_unscaled = start_price_unscaled.iloc[:, 0] + elif isinstance(start_price_unscaled, np.ndarray) and start_price_unscaled.ndim == 1: + # Convert 1D array to Series + start_price_unscaled = pd.Series(start_price_unscaled) + else: + data_pipeline_logger.error(f"start_price_unscaled shape is not compatible: {getattr(start_price_unscaled, 'shape', 'unknown')}") + return None, None, None + except Exception as e: + data_pipeline_logger.error(f"Failed to convert start_price_unscaled to Series: {e}") + return None, None, None + + # Log input info + data_pipeline_logger.info(f"Features index type: {type(features_scaled.index)}, length: {len(features_scaled)}") + data_pipeline_logger.info(f"Targets index type: {type(targets_scaled.index)}, length: {len(targets_scaled)}") + data_pipeline_logger.info(f"Start price index type: {type(start_price_unscaled.index)}, length: {len(start_price_unscaled)}") + + # Check for index compatibility + if (len(features_scaled) != len(targets_scaled) or + len(features_scaled) != len(start_price_unscaled)): + data_pipeline_logger.warning(f"Input lengths don't match! Features: {len(features_scaled)}, Targets: {len(targets_scaled)}, Prices: {len(start_price_unscaled)}") + + # Try to align on common index if all have DatetimeIndex + if (isinstance(features_scaled.index, pd.DatetimeIndex) and + isinstance(targets_scaled.index, pd.DatetimeIndex) and + isinstance(start_price_unscaled.index, pd.DatetimeIndex)): + + # Find common dates + common_index = features_scaled.index.intersection( + targets_scaled.index.intersection(start_price_unscaled.index) + ) + + # Check if we have any overlap + if len(common_index) < seq_length: + data_pipeline_logger.error(f"Not enough common indices ({len(common_index)}) for sequence length ({seq_length})") + + # If we don't have enough common indices, create synthetic indices + # First find the shortest length + min_len = min(len(features_scaled), len(targets_scaled), len(start_price_unscaled)) + + # If we have enough data for at least one sequence + if min_len >= seq_length: + data_pipeline_logger.warning(f"Using the shortest dataset length ({min_len})") + + # Convert features to numpy + features_np = features_scaled.values[:min_len] + + # Try to keep targets a Series and preserve its index + if len(targets_scaled) > min_len: + targets_scaled = targets_scaled.iloc[:min_len] + + # Try to keep prices a Series and preserve its index + if len(start_price_unscaled) > min_len: + start_price_unscaled = start_price_unscaled.iloc[:min_len] + + # Create sequences from numpy arrays + X_sequences, y_targets, starting_prices = [], [], [] + for i in range(len(features_np) - seq_length): + X_sequences.append(features_np[i:i+seq_length]) + y_targets.append(targets_scaled.iloc[i+seq_length]) + starting_prices.append(start_price_unscaled.iloc[i+seq_length]) + + if len(X_sequences) == 0: + data_pipeline_logger.error("No sequences created") + return None, None, None + + return np.array(X_sequences), np.array(y_targets), np.array(starting_prices) + else: + data_pipeline_logger.error(f"Cannot create sequences: Not enough data for sequence length {seq_length}") + return None, None, None + + # Align on common index + data_pipeline_logger.info(f"Aligning on {len(common_index)} common indices") + features_scaled = features_scaled.loc[common_index] + targets_scaled = targets_scaled.loc[common_index] + start_price_unscaled = start_price_unscaled.loc[common_index] + else: + data_pipeline_logger.error("Cannot create sequences v2: No common index found.") + return None, None, None + + # Convert features to numpy array for sequence creation + features_np = features_scaled.values + + # Create sequences + X_sequences, y_targets, starting_prices = [], [], [] + for i in range(len(features_np) - seq_length): + X_sequences.append(features_np[i:i+seq_length]) + y_targets.append(targets_scaled.iloc[i+seq_length]) + starting_prices.append(start_price_unscaled.iloc[i+seq_length]) + + if len(X_sequences) == 0: + data_pipeline_logger.error("No sequences created, check sequence length vs data length") + return None, None, None + + data_pipeline_logger.info(f"Created {len(X_sequences)} sequences of length {seq_length}") + return np.array(X_sequences), np.array(y_targets), np.array(starting_prices) + except Exception as e: + data_pipeline_logger.error(f"Error creating sequences: {e}", exc_info=True) + return None, None, None \ No newline at end of file diff --git a/gru_sac_predictor/src/gru_predictor.py b/gru_sac_predictor/src/gru_predictor.py new file mode 100644 index 00000000..09e3edd0 --- /dev/null +++ b/gru_sac_predictor/src/gru_predictor.py @@ -0,0 +1,646 @@ +""" +V7 - GRU Model for Cryptocurrency Price Prediction (Adapted from V6) + +This module implements a GRU-based neural network model for predicting +cryptocurrency prices directly (regression), calculating uncertainty via +Monte Carlo dropout, and deriving predicted returns. +""" +import numpy as np +import tensorflow as tf +import os +import joblib +import logging +import time +import sys +from sklearn.preprocessing import MinMaxScaler, StandardScaler # Keep both options +from sklearn.metrics import mean_absolute_error, mean_squared_error, accuracy_score, classification_report, confusion_matrix +from tensorflow.keras.models import Sequential, load_model, Model # Keep Model if needed +from tensorflow.keras.layers import GRU, Dropout, Dense, Input # Keep Input if needed +from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint, ReduceLROnPlateau +from tensorflow.keras.optimizers import Adam +import matplotlib.pyplot as plt + +# Configure logging (ensure it doesn't conflict with other loggers) +gru_logger = logging.getLogger(__name__) +# Avoid adding handlers if they already exist from a root config +if not gru_logger.hasHandlers(): + gru_logger.setLevel(logging.INFO) + # Check if root logger has handlers to avoid duplicate console output + root_logger = logging.getLogger() + if not root_logger.hasHandlers() or all(isinstance(h, logging.FileHandler) for h in root_logger.handlers): + console_handler = logging.StreamHandler(sys.stdout) + console_handler.setLevel(logging.INFO) + formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') + console_handler.setFormatter(formatter) + gru_logger.addHandler(console_handler) + gru_logger.propagate = False # Prevent propagation to root if we added a handler + else: + gru_logger.propagate = True # Propagate to root if it has handlers + +# Test log message +gru_logger.info("GRU predictor V7 (V6 Adaptation) logger initialized.") + +# --- Helper: Predict in Batches --- +def predict_in_batches(model, X, batch_size=1024): + """Process predictions in batches to avoid OOM errors""" + n_samples = X.shape[0] + n_batches = (n_samples + batch_size - 1) // batch_size + predictions = [] + for i in range(n_batches): + start_idx = i * batch_size + end_idx = min((i + 1) * batch_size, n_samples) + # Use training=False for standard prediction + batch_predictions = model(X[start_idx:end_idx], training=False).numpy() + predictions.append(batch_predictions) + if n_batches > 10 and (i+1) % max(1, n_batches//10) == 0: + gru_logger.info(f"Prediction batch progress: {i+1}/{n_batches}") + return np.vstack(predictions) + + +class CryptoGRUModel: + """ + GRU-based model for cryptocurrency price prediction, adapted from V6. + Predicts price directly and calculates MC uncertainty. + """ + + def __init__(self, model_dir=None): + """ + Initialize the GRU regression model. + + Args: + model_dir (str, optional): Directory to load pre-trained model and scalers. + """ + gru_logger.info("Initializing V7 CryptoGRUModel (V6 Adaptation)...") + self.model: tf.keras.Model = None + self.feature_scaler: StandardScaler | MinMaxScaler = None # Allow either + self.y_scaler: MinMaxScaler = None # V6 used MinMaxScaler for target + self.model_dir = model_dir # Store model dir for saving/loading convenience + self.is_trained = False + self.is_loaded = False + + if model_dir: + self.load(model_dir) + + def _build_model(self, input_data): + """ + Build the GRU model architecture dynamically from input data dimensions, + exactly matching the V6 implementation. + + Args: + input_data (np.array): Training data, used to determine input shape. + + Returns: + tf.keras.Model: Compiled GRU model matching V6. + """ + gru_logger.info("Building V6 GRU model architecture...") + # Determine input shape from data + input_shape = (input_data.shape[1], input_data.shape[2]) + + # Create Sequential model to match V6 implementation + model = Sequential(name="V6_GRU_Regression") + + # Add explicit Input layer as recommended + model.add(Input(shape=input_shape, name="input_layer")) + + # Add single GRU layer (100 units) - remove input_shape argument + model.add(GRU(100, name="gru_100")) + + # Add Dropout layer (V6 used 0.2) + model.add(Dropout(0.2, name="dropout_0.2")) + + # Output layer with linear activation for price prediction + model.add(Dense(1, activation='linear', name="output_price")) # output_dim = 1 + + # Compile the model using MSE loss as in V6 + model.compile( + optimizer=Adam(learning_rate=0.001), # V6 used 0.001 LR + loss='mse', # V6 used MSE + metrics=['mae', 'mape', tf.keras.metrics.RootMeanSquaredError(name='rmse')] # V6 metrics + ) + + gru_logger.info(f"V6 Model built: 1 GRU layer (100 units), Dropout (0.2), Linear Output.") + gru_logger.info(f"Input shape: {(input_data.shape[1], input_data.shape[2])}") + model.summary(print_fn=gru_logger.info) + + return model + + def train(self, X_train, y_train_scaled, X_val, y_val_scaled, + feature_scaler, y_scaler, # Pass fitted scalers + batch_size=32, epochs=20, patience=10, + model_save_dir='models/gru_predictor_trained'): + """ + Train the GRU regression model using V6 parameters. + Assumes sequences (X) and scaled targets (y) are provided. + Fitted scalers must also be provided for saving. + + Args: + X_train (np.array): Training sequences. + y_train_scaled (np.array): Scaled training target prices (shape: [samples, 1]). + X_val (np.array): Validation sequences. + y_val_scaled (np.array): Scaled validation target prices (shape: [samples, 1]). + feature_scaler: Fitted feature scaler (StandardScaler or MinMaxScaler). + y_scaler: Fitted target scaler (MinMaxScaler). + batch_size (int): Batch size for training (default: 32 as per V6). + epochs (int): Maximum number of epochs (default: 20 as per V6). + patience (int): Patience for early stopping (default: 10). + model_save_dir (str): Directory to save the best model and scalers. + + Returns: + dict: Training history. + """ + gru_logger.info("--- Starting GRU Model Training (V6 Adaptation) ---") + + # Store scalers + if feature_scaler is None or y_scaler is None: + gru_logger.error("Fitted scalers must be provided for training.") + return None + self.feature_scaler = feature_scaler + self.y_scaler = y_scaler + + # Build model dynamically based on input data shape if not already loaded/built + if self.model is None: + gru_logger.info(f"Building model dynamically from input data with shape {X_train.shape}") + self.model = self._build_model(X_train) + + # Set up callbacks (monitoring val_rmse as in V6) + os.makedirs(model_save_dir, exist_ok=True) + best_model_path = os.path.join(model_save_dir, "best_model_reg.keras") + + gru_logger.info("Monitoring val_rmse for EarlyStopping and ModelCheckpoint.") + callbacks = [ + EarlyStopping( + monitor='val_rmse', patience=patience, restore_best_weights=True, + verbose=1, mode='min' + ), + ModelCheckpoint( + filepath=best_model_path, monitor='val_rmse', save_best_only=True, + verbose=1, mode='min' + ), + ReduceLROnPlateau( + monitor='val_rmse', factor=0.5, patience=patience // 2, # Reduce LR faster + min_lr=1e-6, verbose=1, mode='min' + ) + ] + + # Record start time + start_time = time.time() + gru_logger.info(f"Starting training: epochs={epochs}, batch_size={batch_size}, patience={patience}") + + # --- Log first input sequence and target price --- + if len(X_train) > 0 and len(y_train_scaled) > 0: + gru_logger.info(f"First training sequence (X_train[0]) shape: {X_train[0].shape}") + log_steps = min(5, X_train.shape[1]) + gru_logger.info(f"First {log_steps} steps of first training sequence (scaled):\n{X_train[0][:log_steps]}") + gru_logger.info(f"Target scaled price for first sequence (y_train_scaled[0]): {y_train_scaled[0]:.6f}") + else: + gru_logger.warning("X_train or y_train_scaled is empty, cannot log first sequence.") + + # Train the model + history = self.model.fit( + X_train, y_train_scaled, + validation_data=(X_val, y_val_scaled), + batch_size=batch_size, + epochs=epochs, + callbacks=callbacks, + verbose=1, + ) + + # Calculate training duration + training_duration = time.time() - start_time + hours, remainder = divmod(training_duration, 3600) + minutes, seconds = divmod(remainder, 60) + gru_logger.info(f"Training completed in {int(hours)}h {int(minutes)}m {int(seconds)}s") + + # Load the best model saved by ModelCheckpoint + if os.path.exists(best_model_path): + gru_logger.info(f"Loading best regression model from {best_model_path}") + # No custom objects needed for standard MSE loss and metrics + self.model = load_model(best_model_path) + self.is_trained = True + self.is_loaded = False # Trained in this session + # Save scalers alongside the best model + self.save(model_save_dir) # Save model and scalers + else: + gru_logger.warning(f"Best model file not found at {best_model_path}. Using the final state.") + self.is_trained = True + self.is_loaded = False + # Save the final state model and scalers + self.save(model_save_dir) + + return history.history + + def predict_scaled_price(self, X): + """ + Make scaled price predictions with the trained regression model. + + Args: + X (np.array): Input sequences. + + Returns: + np.array: Predicted scaled prices (shape: [samples, 1]). + """ + if self.model is None: + gru_logger.error("Model not loaded or trained. Cannot predict.") + raise ValueError("Model is not available for prediction.") + + gru_logger.info(f"Predicting scaled prices on data with shape {X.shape}") + return self.model.predict(X) + # Consider using predict_in_batches for large X + + + def evaluate(self, X_test, y_test_scaled, y_start_price_test, n_mc_samples=30): + """ + Evaluate the regression model on test data, calculate uncertainty, + and derive predicted returns and confidence. Mirrors V6 evaluation logic. + + Args: + X_test (np.array): Test sequences. + y_test_scaled (np.array): Scaled true target prices (shape: [samples, 1]). + y_start_price_test (np.array): Unscaled price at the start of each test target window (shape: [samples] or [samples, 1]). + n_mc_samples (int): Number of Monte Carlo samples for uncertainty estimation. + + Returns: + dict: Dictionary containing evaluation results: + 'pred_percent_change': Predicted % change based on unscaled price predictions. + 'raw_confidence_score': Confidence score (1 - normalized MC std dev). + 'predicted_unscaled_prices': Unscaled price predictions. + 'mae': Mean Absolute Error (unscaled). + 'rmse': Root Mean Squared Error (unscaled). + 'mape': Mean Absolute Percentage Error (unscaled). + 'mc_unscaled_std_dev': Unscaled standard deviation from MC dropout. + 'misc_metrics': Dict with scaled metrics if needed. + """ + gru_logger.info("--- Starting Model Evaluation (V6 Adaptation) ---") + if self.model is None or self.y_scaler is None: + gru_logger.error("Model or y_scaler not available for evaluation.") + return None + if X_test is None or y_test_scaled is None or y_start_price_test is None: + gru_logger.error("Missing data for evaluation (X_test, y_test_scaled, or y_start_price_test).") + return None + + # --- 1. Standard Prediction --- + gru_logger.info(f"Predicting on Test set (Standard) with shape {X_test.shape}") + y_pred_test_standard_scaled = predict_in_batches(self.model, X_test) # Use batch predictor + + # --- 2. Monte Carlo Dropout Prediction --- + gru_logger.info(f"Running Monte Carlo dropout inference ({n_mc_samples} samples)...") + mc_preds_test_list = [] + + # Define the prediction step with training=True inside the loop + @tf.function + def mc_predict_step_test(batch): + return self.model(batch, training=True) # Enable dropout + + for i in range(n_mc_samples): + mc_preds = [] + # Batch processing within MC loop + for j in range(0, len(X_test), 1024): + batch = X_test[j:j+1024] + batch_preds = mc_predict_step_test(tf.constant(batch, dtype=tf.float32)).numpy() + mc_preds.append(batch_preds) + mc_preds_test_list.append(np.vstack(mc_preds)) + if n_mc_samples > 1 and (i+1) % max(1, n_mc_samples//5) == 0: + gru_logger.info(f" MC progress: {i+1}/{n_mc_samples}") + + mc_preds_stack = np.stack(mc_preds_test_list) + y_pred_test_mc_std_scaled = np.std(mc_preds_stack, axis=0) + gru_logger.info(f"MC dropout completed. Scaled Std Dev: Min={np.min(y_pred_test_mc_std_scaled):.6f}, Max={np.max(y_pred_test_mc_std_scaled):.6f}, Mean={np.mean(y_pred_test_mc_std_scaled):.6f}") + + # --- 3. Inverse Transform --- + gru_logger.info("Inverse transforming predictions and true values...") + try: + y_pred_test_standard_unscaled = self.y_scaler.inverse_transform(y_pred_test_standard_scaled).flatten() + # Reshape y_test_scaled to 2D before inverse_transform + if y_test_scaled.ndim == 1: + y_test_scaled_2d = y_test_scaled.reshape(-1, 1) + else: + y_test_scaled_2d = y_test_scaled # Assume it's already 2D if not 1D + y_test_true_unscaled = self.y_scaler.inverse_transform(y_test_scaled_2d).flatten() + except Exception as e: + gru_logger.error(f"Error during inverse transform: {e}", exc_info=True) + return None + + # --- 4. Unscale MC Standard Deviation --- + gru_logger.info("Unscaling MC standard deviation...") + y_pred_test_mc_std_unscaled_flat = np.zeros_like(y_pred_test_mc_std_scaled.flatten()) # Default + try: + # Use scaler's data range if available (more robust for MinMaxScaler) + if hasattr(self.y_scaler, 'data_min_') and hasattr(self.y_scaler, 'data_max_'): + data_range = self.y_scaler.data_max_[0] - self.y_scaler.data_min_[0] + if data_range > 1e-9: + y_pred_test_mc_std_unscaled_flat = y_pred_test_mc_std_scaled.flatten() * data_range + gru_logger.info(f"Unscaled MC std dev (Test): Mean={np.mean(y_pred_test_mc_std_unscaled_flat):.6f}") + else: + gru_logger.warning("Scaler data range is near zero. Cannot reliably unscale MC std dev.") + else: + gru_logger.warning("y_scaler missing data_min_/data_max_. Cannot unscale MC std dev accurately.") + except Exception as e: + gru_logger.error(f"Error unscaling MC std dev: {e}", exc_info=True) + # Keep the default of zeros or potentially use scaled std dev as fallback? + + # --- 5. Calculate Raw Confidence Score --- + gru_logger.info("Calculating raw confidence score (1 - normalized std dev)...") + test_raw_confidence_score = np.ones_like(y_pred_test_mc_std_unscaled_flat) * 0.5 # Default if std dev is constant + epsilon = 1e-9 + max_std_dev_test = np.max(y_pred_test_mc_std_unscaled_flat) + epsilon + min_std_dev_test = np.min(y_pred_test_mc_std_unscaled_flat) + + if max_std_dev_test > min_std_dev_test: + # Normalize the UNscaled std dev between 0 and 1 + normalized_std_dev = (y_pred_test_mc_std_unscaled_flat - min_std_dev_test) / (max_std_dev_test - min_std_dev_test) + # Confidence is inverse of normalized uncertainty + test_raw_confidence_score = 1.0 - normalized_std_dev + else: + gru_logger.warning("MC standard deviation is constant or near-constant. Setting raw confidence to 0.5.") + + test_raw_confidence_score = np.clip(test_raw_confidence_score, 0.0, 1.0) # Ensure bounds + gru_logger.info(f"Raw Confidence score: Min={np.min(test_raw_confidence_score):.4f}, Max={np.max(test_raw_confidence_score):.4f}, Mean={np.mean(test_raw_confidence_score):.4f}") + + # --- 6. Calculate Predicted Percentage Change --- + gru_logger.info("Calculating predicted percentage change...") + # Ensure y_start_price_test is flattened and aligned + y_start_price_flat = y_start_price_test.flatten() + if len(y_pred_test_standard_unscaled) != len(y_start_price_flat): + gru_logger.error(f"Length mismatch: Pred Price ({len(y_pred_test_standard_unscaled)}) vs Start Price ({len(y_start_price_flat)})") + # Attempt to align if possible (e.g., maybe y_start_price_test has extra initial values) + if len(y_start_price_flat) > len(y_pred_test_standard_unscaled): + diff = len(y_start_price_flat) - len(y_pred_test_standard_unscaled) + gru_logger.warning(f"Attempting alignment by trimming {diff} elements from start_price array.") + y_start_price_flat = y_start_price_flat[diff:] # Trim from the beginning? Or end? Needs context. Assume end. + # y_start_price_flat = y_start_price_flat[:-diff] # Trim from end + + if len(y_pred_test_standard_unscaled) != len(y_start_price_flat): + gru_logger.error("Alignment failed. Cannot calculate predicted change.") + pred_percent_change = np.zeros_like(y_pred_test_standard_unscaled) # Fallback + else: + gru_logger.info("Alignment successful.") + pred_percent_change = np.where( + np.abs(y_start_price_flat) > epsilon, + (y_pred_test_standard_unscaled / y_start_price_flat) - 1, + 0 # Assign 0 change if start price is near zero + ) + else: + pred_percent_change = np.where( + np.abs(y_start_price_flat) > epsilon, + (y_pred_test_standard_unscaled / y_start_price_flat) - 1, + 0 # Assign 0 change if start price is near zero + ) + gru_logger.info(f"Predicted Percent Change: Min={np.min(pred_percent_change):.4f}, Max={np.max(pred_percent_change):.4f}, Mean={np.mean(pred_percent_change):.4f}") + + # --- 7. Calculate Regression Metrics --- + gru_logger.info("Calculating regression metrics (on unscaled data)...") + eval_mae = mean_absolute_error(y_test_true_unscaled, y_pred_test_standard_unscaled) + eval_mse = mean_squared_error(y_test_true_unscaled, y_pred_test_standard_unscaled) + eval_rmse = np.sqrt(eval_mse) + mask = y_test_true_unscaled != 0 + eval_mape = np.mean(np.abs((y_test_true_unscaled[mask] - y_pred_test_standard_unscaled[mask]) / y_test_true_unscaled[mask])) * 100 if np.any(mask) else 0.0 + + gru_logger.info(f"Test MAE (Unscaled): {eval_mae:.4f}") + gru_logger.info(f"Test RMSE (Unscaled): {eval_rmse:.4f}") + gru_logger.info(f"Test MAPE (Unscaled): {eval_mape:.4f}%") + + # --- 8. Return Results --- + results = { + 'pred_percent_change': pred_percent_change, + 'raw_confidence_score': test_raw_confidence_score, + 'predicted_unscaled_prices': y_pred_test_standard_unscaled, + 'true_unscaled_prices': y_test_true_unscaled, # Include true values for plotting/analysis + 'mae': eval_mae, + 'rmse': eval_rmse, + 'mape': eval_mape, + 'mc_unscaled_std_dev': y_pred_test_mc_std_unscaled_flat, + # Add derived direction accuracy for comparison? + # 'derived_direction_accuracy': accuracy_score(np.sign(y_test_true_unscaled - y_start_price_flat), np.sign(y_pred_test_standard_unscaled - y_start_price_flat)) + } + gru_logger.info("--- Evaluation Completed ---") + return results + + + def save(self, model_dir): + """ + Save the trained regression model and scalers. + + Args: + model_dir (str): Directory to save artifacts. + """ + if not (self.is_trained or self.is_loaded): + gru_logger.error("Cannot save, model not trained/loaded.") + return + if self.model is None or self.feature_scaler is None or self.y_scaler is None: + gru_logger.error("Cannot save, model or scalers missing.") + return + + os.makedirs(model_dir, exist_ok=True) + model_path = os.path.join(model_dir, "best_model_reg.keras") # V6 name + feature_scaler_path = os.path.join(model_dir, "feature_scaler.joblib") + y_scaler_path = os.path.join(model_dir, "y_scaler.joblib") + + try: + self.model.save(model_path) + gru_logger.info(f"Keras model saved to {model_path}") + joblib.dump(self.feature_scaler, feature_scaler_path) + gru_logger.info(f"Feature scaler saved to {feature_scaler_path}") + joblib.dump(self.y_scaler, y_scaler_path) + gru_logger.info(f"Target scaler (MinMaxScaler) saved to {y_scaler_path}") + except Exception as e: + gru_logger.error(f"Error saving model/scalers to {model_dir}: {e}", exc_info=True) + + def load(self, model_dir): + """ + Load a previously trained regression model and its scalers. + + Args: + model_dir (str): Directory containing model (.keras) and scalers (.joblib). + """ + gru_logger.info(f"Attempting to load V6-style GRU regression model and scalers from: {model_dir}") + self.model_dir = model_dir + model_path = os.path.join(model_dir, 'best_model_reg.keras') # V7.20 Fix: Load correct filename + scaler_feature_path = os.path.join(model_dir, 'feature_scaler.joblib') + scaler_y_path = os.path.join(model_dir, 'y_scaler.joblib') + + # Check if all required files exist + files_exist = all(os.path.exists(p) for p in [model_path, scaler_feature_path, scaler_y_path]) + + if not files_exist: + gru_logger.warning(f"Cannot load model. Required files missing in {model_dir}.") + gru_logger.warning(f" Missing: {[p for p in [model_path, scaler_feature_path, scaler_y_path] if not os.path.exists(p)]}") + self.is_loaded = False + return False + + try: + # Load Keras model + gru_logger.info(f"Loading GRU model from: {model_path}") + # Load without compiling if optimizer state is not needed or causes issues + self.model = load_model(model_path, compile=False) + + # Load scalers + gru_logger.info(f"Loading feature scaler from: {scaler_feature_path}") + self.feature_scaler = joblib.load(scaler_feature_path) + gru_logger.info(f"Loading target scaler from: {scaler_y_path}") + self.y_scaler = joblib.load(scaler_y_path) + + self.is_loaded = True + self.is_trained = False # Loaded, not trained in this session + gru_logger.info(f"CryptoGRUModel loaded successfully from {model_dir}.") + + # V7.9 Explicitly build after loading to be safe + if self.model and hasattr(self.model, 'input_shape'): + gru_logger.info("Triggering model build after loading...") + dummy_input = tf.zeros((1,) + self.model.input_shape[1:], dtype=tf.float32) + _ = self.model(dummy_input) + gru_logger.info("Model built after loading.") + else: + gru_logger.warning("Could not get input shape to explicitly build model after loading.") + + return True + except Exception as e: + gru_logger.error(f"Error loading model/scalers from {model_dir}: {e}", exc_info=True) + self.model = None; self.feature_scaler = None; self.y_scaler = None + self.is_trained = False; self.is_loaded = False + return False + + def plot_training_history(self, history, save_path=None): + """ + Plot training history (Loss, MAE, RMSE). Adapted from V6. + + Args: + history (dict): Training history from model.fit(). + save_path (str, optional): Path to save the plot. + """ + if not history: + gru_logger.warning("No history data provided to plot.") + return + + plt.figure(figsize=(18, 5)) + + # Plot training & validation loss (MSE) + plt.subplot(1, 3, 1) + plt.plot(history.get('loss', []), label='Train Loss (MSE)') + plt.plot(history.get('val_loss', []), label='Val Loss (MSE)') + plt.title('Model Loss (MSE)') + plt.ylabel('Loss') + plt.xlabel('Epoch') + plt.legend(loc='upper right') + plt.grid(True) + + # Plot training & validation MAE + plt.subplot(1, 3, 2) + plt.plot(history.get('mae', []), label='Train MAE') + plt.plot(history.get('val_mae', []), label='Val MAE') + plt.title('Model Mean Absolute Error') + plt.ylabel('MAE') + plt.xlabel('Epoch') + plt.legend(loc='upper right') + plt.grid(True) + + # Plot training & validation RMSE + plt.subplot(1, 3, 3) + plt.plot(history.get('rmse', []), label='Train RMSE') + plt.plot(history.get('val_rmse', []), label='Val RMSE') + plt.title('Model Root Mean Squared Error') + plt.ylabel('RMSE') + plt.xlabel('Epoch') + plt.legend(loc='upper right') + plt.grid(True) + + plt.tight_layout() + + if save_path: + try: + plt.savefig(save_path) + gru_logger.info(f"Training history plot saved to {save_path}") + except Exception as e: + gru_logger.error(f"Error saving training history plot: {e}") + else: + plt.show() # Display plot if not saving + + plt.close() + + def plot_evaluation_results(self, eval_results, save_path=None): + """ + Plot evaluation results for the REGRESSION model using data from evaluate(). + Adapted from V6. + + Args: + eval_results (dict): Dictionary returned by the evaluate() method. + save_path (str, optional): Path to save the plots. + """ + if not eval_results: + gru_logger.warning("No evaluation results provided to plot.") + return + + y_true = eval_results.get('true_unscaled_prices') + y_pred = eval_results.get('predicted_unscaled_prices') + mae = eval_results.get('mae', -1) + mape = eval_results.get('mape', -1) + confidence = eval_results.get('raw_confidence_score') + uncertainty_std = eval_results.get('mc_unscaled_std_dev') + + if y_true is None or y_pred is None: + gru_logger.error("Missing true or predicted prices in evaluation results for plotting.") + return + + plt.figure(figsize=(15, 12)) # Adjusted size + + # Plot 1: True vs. Predicted Prices with Uncertainty Bands + plt.subplot(3, 1, 1) # Changed to 3 rows + plt.plot(y_true, label='True Prices', alpha=0.7) + plt.plot(y_pred, label=f'Predicted Prices (MAE: {mae:.2f})', alpha=0.7) + if uncertainty_std is not None: + plt.fill_between( + range(len(y_pred)), + y_pred - uncertainty_std, + y_pred + uncertainty_std, + color='orange', alpha=0.2, label='Uncertainty (MC Std Dev)' + ) + plt.title(f"True vs. Predicted Prices (MAPE: {mape:.2f}%)") + plt.ylabel('Price') + plt.legend() + plt.grid(True) + + # Plot 2: Prediction Errors (Residuals) + plt.subplot(3, 1, 2) + errors = y_true - y_pred + plt.hist(errors, bins=50, alpha=0.7) + plt.title(f"Prediction Errors (Residuals) - MAE: {mae:.4f}") + plt.xlabel('Error (True - Predicted)') + plt.ylabel('Frequency') + plt.grid(True) + + # Plot 3: Confidence Scores + plt.subplot(3, 1, 3) + if confidence is not None: + plt.plot(confidence, label='Raw Confidence Score', color='green') + plt.title(f"Confidence Score (Mean: {np.mean(confidence):.3f})") + plt.ylabel("Confidence (0-1)") + plt.ylim(0, 1.05) + plt.legend() + else: + plt.title("Confidence Score (Not Available)") + plt.xlabel('Time Step (Test Set)') + plt.grid(True) + + + plt.tight_layout() + + if save_path: + try: + plt.savefig(save_path) + gru_logger.info(f"Regression evaluation plots saved to {save_path}") + except Exception as e: + gru_logger.error(f"Error saving evaluation plot: {e}") + else: + plt.show() + + plt.close() + +# Example usage placeholder (won't run directly here) +if __name__ == "__main__": + gru_logger.info("CryptoGRUModel (V6 Adaptation) module loaded.") + # Example: + # model = CryptoGRUModel() + # # Need to load data, preprocess, scale, create sequences first... + # # history = model.train(X_train, y_train_scaled, X_val, y_val_scaled, feature_scaler, y_scaler) + # # eval_results = model.evaluate(X_test, y_test_scaled, y_start_price_test) + # # model.plot_training_history(history, save_path='training_hist.png') + # # model.plot_evaluation_results(eval_results, save_path='evaluation.png') diff --git a/gru_sac_predictor/src/sac_agent.py b/gru_sac_predictor/src/sac_agent.py new file mode 100644 index 00000000..bb5e0085 --- /dev/null +++ b/gru_sac_predictor/src/sac_agent.py @@ -0,0 +1,357 @@ +import numpy as np +import tensorflow as tf +from tensorflow.keras import layers, Model +import tensorflow_probability as tfp +from tensorflow.keras.optimizers.schedules import ExponentialDecay +import logging +import os + +sac_logger = logging.getLogger(__name__) +sac_logger.setLevel(logging.INFO) + +class OrnsteinUhlenbeckActionNoise: + def __init__(self, mean, std_deviation, theta=0.15, dt=0.01, x_initial=None): + self.theta = theta + self.mean = mean + self.std_dev = std_deviation + self.dt = dt + self.x_initial = x_initial + self.x_prev = self.x_initial if self.x_initial is not None else np.zeros_like(self.mean) + + def __call__(self): + x = ( + self.x_prev + + self.theta * (self.mean - self.x_prev) * self.dt + + self.std_dev * np.sqrt(self.dt) * np.random.normal(size=self.mean.shape) + ) + self.x_prev = x + return x + + def reset(self): + self.x_prev = self.x_initial if self.x_initial is not None else np.zeros_like(self.mean) + +class ReplayBuffer: + """Standard Experience replay buffer for SAC agent""" + + def __init__(self, capacity=100000, state_dim=2, action_dim=1): + self.capacity = capacity + self.counter = 0 + + # Initialize buffer arrays + self.states = np.zeros((capacity, state_dim), dtype=np.float32) + self.actions = np.zeros((capacity, action_dim), dtype=np.float32) + self.rewards = np.zeros((capacity, 1), dtype=np.float32) + self.next_states = np.zeros((capacity, state_dim), dtype=np.float32) + self.dones = np.zeros((capacity, 1), dtype=np.float32) + + def add(self, state, action, reward, next_state, done): + """Add experience to buffer""" + idx = self.counter % self.capacity + + # Ensure inputs are correctly shaped numpy arrays + state = np.array(state, dtype=np.float32).flatten() + action = np.array(action, dtype=np.float32).flatten() + reward = np.array([reward], dtype=np.float32) + next_state = np.array(next_state, dtype=np.float32).flatten() + done = np.array([done], dtype=np.float32) + + if state.shape[0] != self.states.shape[1]: + sac_logger.error(f"State shape mismatch: {state.shape} vs {self.states.shape[1]}") + return + if next_state.shape[0] != self.next_states.shape[1]: + sac_logger.error(f"Next State shape mismatch: {next_state.shape} vs {self.next_states.shape[1]}") + return + if action.shape[0] != self.actions.shape[1]: + sac_logger.error(f"Action shape mismatch: {action.shape} vs {self.actions.shape[1]}") + return + + self.states[idx] = state + self.actions[idx] = action + self.rewards[idx] = reward + self.next_states[idx] = next_state + self.dones[idx] = done + + self.counter += 1 + + def sample(self, batch_size): + """Sample batch of experiences from buffer""" + max_idx = min(self.counter, self.capacity) + if max_idx < batch_size: + print(f"Warning: Trying to sample {batch_size} elements, but buffer only has {max_idx}. Sampling with replacement.") + indices = np.random.choice(max_idx, batch_size, replace=True) + else: + indices = np.random.choice(max_idx, batch_size, replace=False) + + states = tf.convert_to_tensor(self.states[indices], dtype=tf.float32) + actions = tf.convert_to_tensor(self.actions[indices], dtype=tf.float32) + rewards = tf.convert_to_tensor(self.rewards[indices], dtype=tf.float32) + next_states = tf.convert_to_tensor(self.next_states[indices], dtype=tf.float32) + dones = tf.convert_to_tensor(self.dones[indices], dtype=tf.float32) + + return states, actions, rewards, next_states, dones + + def __len__(self): + """Get current size of buffer""" + return min(self.counter, self.capacity) + +class SACTradingAgent: + """V7.3 Enhanced: SAC agent with updated params and architecture fixes.""" + + def __init__(self, + state_dim=2, # Standard [pred_ret, uncert] + action_dim=1, + gamma=0.99, + tau=0.005, + initial_lr=3e-4, + decay_steps=100000, + end_lr=5e-6, # Note: End LR not directly used by ExponentialDecay + lr_decay_rate=0.96, + buffer_capacity=100000, + ou_noise_stddev=0.2, + ou_noise_theta=0.15, + ou_noise_dt=0.01, + alpha=0.2, + alpha_auto_tune=True, + target_entropy=-1.0, + min_buffer_size=1000): + """ + Initialize the SAC agent with enhancements. + """ + self.state_dim = state_dim + self.action_dim = action_dim + self.gamma = gamma + self.tau = tau + self.min_buffer_size = min_buffer_size + self.target_entropy = tf.constant(target_entropy, dtype=tf.float32) + self.alpha_auto_tune = alpha_auto_tune + + if self.alpha_auto_tune: + self.log_alpha = tf.Variable(tf.math.log(alpha), trainable=True, name='log_alpha') + self.alpha = tfp.util.DeferredTensor(self.log_alpha, tf.exp) + self.alpha_optimizer = tf.keras.optimizers.Adam(learning_rate=initial_lr) + else: + self.alpha = tf.constant(alpha, dtype=tf.float32) + + self.ou_noise = OrnsteinUhlenbeckActionNoise( + mean=np.zeros(action_dim), + std_deviation=float(ou_noise_stddev) * np.ones(action_dim), + theta=ou_noise_theta, dt=ou_noise_dt) + + self.lr_schedule = ExponentialDecay( + initial_learning_rate=initial_lr, decay_steps=decay_steps, + decay_rate=lr_decay_rate, staircase=False) + sac_logger.info(f"Using ExponentialDecay LR: init={initial_lr}, steps={decay_steps}, rate={lr_decay_rate}") + self.actor_optimizer = tf.keras.optimizers.Adam(learning_rate=self.lr_schedule) + self.critic1_optimizer = tf.keras.optimizers.Adam(learning_rate=self.lr_schedule) + self.critic2_optimizer = tf.keras.optimizers.Adam(learning_rate=self.lr_schedule) + + # Initialize networks + self.actor = self._build_actor() + self.critic1 = self._build_critic() # Outputs [Q_mean, Q_log_std] + self.critic2 = self._build_critic() + self.target_critic1 = self._build_critic() + self.target_critic2 = self._build_critic() + + self.update_target_networks(tau=1.0) + self.buffer = ReplayBuffer(capacity=buffer_capacity, state_dim=state_dim, action_dim=action_dim) + + sac_logger.info("Enhanced SAC Agent Initialized (V7.3).") + sac_logger.info(f" State Dim: {state_dim}, Action Dim: {action_dim}") + sac_logger.info(f" Hyperparams: gamma={gamma}, tau={tau}, alpha={'auto' if alpha_auto_tune else alpha}, target_entropy={target_entropy}") + sac_logger.info(f" LR Schedule: Exponential {initial_lr} -> ? (decay_rate={lr_decay_rate})") + sac_logger.info(f" Buffer: {buffer_capacity}, Min Size: {min_buffer_size}, Batch Size: Default 256 (in train)") + sac_logger.info(f" OU Noise: std={ou_noise_stddev}, theta={ou_noise_theta}, dt={ou_noise_dt}") + sac_logger.info(f" PER Note: Standard buffer used (PER={False})") + + def _build_actor(self): + inputs = layers.Input(shape=(self.state_dim,)) + x1 = layers.Dense(128, activation='relu')(inputs); x1_norm = layers.BatchNormalization()(x1) + x2 = layers.Dense(64, activation='relu')(x1_norm); x2_norm = layers.BatchNormalization()(x2) + x1_proj = layers.Dense(64)(x1_norm); x_res = layers.add([x1_proj, x2_norm]) + mean = layers.Dense(self.action_dim, activation='tanh')(x_res) + log_std = layers.Dense(self.action_dim, activation='linear')(x_res) + # V7.3 Fix: Use Lambda layer for clipping + log_std_clipped = layers.Lambda(lambda x: tf.clip_by_value(x, -5, 2), name='log_std_clip')(log_std) + model = Model(inputs=inputs, outputs=[mean, log_std_clipped]); return model + + def _build_critic(self): + """ + Build distributional critic network. + Outputs: [Q_mean, Q_log_std] + """ + state_inputs = layers.Input(shape=(self.state_dim,)) + action_inputs = layers.Input(shape=(self.action_dim,)) + concat_inputs = layers.Concatenate()([state_inputs, action_inputs]) + + x1 = layers.Dense(128, activation='relu')(concat_inputs); x1_norm = layers.BatchNormalization()(x1) + x2 = layers.Dense(64, activation='relu')(x1_norm); x2_norm = layers.BatchNormalization()(x2) + x1_proj = layers.Dense(64)(x1_norm); x_res = layers.add([x1_proj, x2_norm]) + + q_mean = layers.Dense(1, name='q_mean')(x_res) + q_log_std = layers.Dense(1, name='q_log_std')(x_res) + # V7.3 Fix: Use Lambda layer for clipping + q_log_std_clipped = layers.Lambda(lambda x: tf.clip_by_value(x, -5, 2), name='q_log_std_clip')(q_log_std) + + # V7.3 Revert: Remove auxiliary outputs + model = Model(inputs=[state_inputs, action_inputs], + outputs=[q_mean, q_log_std_clipped]) + return model + + @tf.function + def _get_action_distribution(self, state): + mean, log_std = self.actor(state); std = tf.exp(log_std) + distribution = tfp.distributions.Normal(loc=mean, scale=std) + squashed_distribution = tfp.distributions.TransformedDistribution(distribution=distribution, bijector=tfp.bijectors.Tanh()) + return squashed_distribution + + def get_action(self, state, deterministic=False): + state_tensor = tf.convert_to_tensor([state], dtype=tf.float32) + mean, log_std = self.actor(state_tensor) + if deterministic: + action = tf.tanh(mean) + else: + std = tf.exp(log_std); distribution = tfp.distributions.Normal(loc=mean, scale=std) + raw_action = distribution.sample(); action = tf.tanh(raw_action) + noise = self.ou_noise(); action = action + noise + action = tf.clip_by_value(action, -1.0, 1.0) + return action.numpy()[0] + + def update_target_networks(self, tau=None): + if tau is None: tau = self.tau + for target_weight, weight in zip(self.target_critic1.weights, self.critic1.weights): target_weight.assign(tau * weight + (1.0 - tau) * target_weight) + for target_weight, weight in zip(self.target_critic2.weights, self.critic2.weights): target_weight.assign(tau * weight + (1.0 - tau) * target_weight) + + def _update_critics(self, states, actions, rewards, next_states, dones): + next_means, next_log_stds = self.actor(next_states); next_stds = tf.exp(next_log_stds) + next_distributions = tfp.distributions.Normal(loc=next_means, scale=next_stds) + next_actions_raw = next_distributions.sample(); next_actions_tanh = tf.tanh(next_actions_raw) + next_log_probs = next_distributions.log_prob(next_actions_raw) + log_correction = tf.math.log(1.0 - tf.square(next_actions_tanh) + 1e-6) + next_log_probs = tf.reduce_sum(next_log_probs - log_correction, axis=1, keepdims=True) + + target_q1_mean, target_q1_log_std = self.target_critic1([next_states, next_actions_tanh]) + target_q2_mean, target_q2_log_std = self.target_critic2([next_states, next_actions_tanh]) + target_q_min_mean = tf.minimum(target_q1_mean, target_q2_mean) + target_q = target_q_min_mean - self.alpha * next_log_probs + target_q_values = rewards + (1.0 - dones) * self.gamma * target_q + + # Explicitly get trainable variables before the tape + critic1_vars = self.critic1.trainable_variables + critic2_vars = self.critic2.trainable_variables + + with tf.GradientTape(persistent=True) as tape: + # Ensure the tape watches the correct variables if needed, though default should be fine + # tape.watch(critic1_vars) + # tape.watch(critic2_vars) + + current_q1_mean, current_q1_log_std = self.critic1([states, actions]) + current_q2_mean, current_q2_log_std = self.critic2([states, actions]) + + pred_dist1 = tfp.distributions.Normal(loc=current_q1_mean, scale=tf.exp(current_q1_log_std)) + pred_dist2 = tfp.distributions.Normal(loc=current_q2_mean, scale=tf.exp(current_q2_log_std)) + nll_loss1 = -pred_dist1.log_prob(tf.stop_gradient(target_q_values)) + nll_loss2 = -pred_dist2.log_prob(tf.stop_gradient(target_q_values)) + critic1_loss = tf.reduce_mean(nll_loss1) + critic2_loss = tf.reduce_mean(nll_loss2) + + # Calculate gradients w.r.t the specific variable lists + critic1_gradients = tape.gradient(critic1_loss, critic1_vars) + critic2_gradients = tape.gradient(critic2_loss, critic2_vars) + del tape + + # Apply gradients paired with the specific variable lists using separate optimizers + self.critic1_optimizer.apply_gradients(zip(critic1_gradients, critic1_vars)) + self.critic2_optimizer.apply_gradients(zip(critic2_gradients, critic2_vars)) + + return critic1_loss, critic2_loss + + def _update_actor(self, states): + # Explicitly get trainable variables before the tape + actor_vars = self.actor.trainable_variables + + with tf.GradientTape() as tape: + # tape.watch(actor_vars) + means, log_stds = self.actor(states); stds = tf.exp(log_stds) + distributions = tfp.distributions.Normal(loc=means, scale=stds) + actions_raw = distributions.sample(); actions_tanh = tf.tanh(actions_raw) + log_probs = distributions.log_prob(actions_raw) + log_correction = tf.math.log(1.0 - tf.square(actions_tanh) + 1e-6) + log_probs = tf.reduce_sum(log_probs - log_correction, axis=1, keepdims=True) + + q1_mean, _ = self.critic1([states, actions_tanh]) + q2_mean, _ = self.critic2([states, actions_tanh]) + q_min_mean = tf.minimum(q1_mean, q2_mean) + + actor_loss = tf.reduce_mean(self.alpha * log_probs - q_min_mean) + + # Calculate gradients w.r.t the specific variable list + actor_gradients = tape.gradient(actor_loss, actor_vars) + # Apply gradients paired with the specific variable list + self.actor_optimizer.apply_gradients(zip(actor_gradients, actor_vars)) + return actor_loss, log_probs + + def _update_alpha(self, log_probs): + with tf.GradientTape() as tape: + alpha_loss = -tf.reduce_mean(self.log_alpha * tf.stop_gradient(log_probs + self.target_entropy)) + + alpha_gradients = tape.gradient(alpha_loss, [self.log_alpha]) + self.alpha_optimizer.apply_gradients(zip(alpha_gradients, [self.log_alpha])) + return alpha_loss + + def train(self, batch_size=256): + """ + Train the enhanced SAC agent. + Includes alpha auto-tuning. + Reverted aux tasks and state dim. + """ + if len(self.buffer) < self.min_buffer_size: + return {} + + states, actions, rewards, next_states, dones = self.buffer.sample(batch_size) + + critic1_loss, critic2_loss = self._update_critics( + states, actions, rewards, next_states, dones + ) + + actor_loss, log_probs = self._update_actor(states) + + alpha_loss = None + if self.alpha_auto_tune: + alpha_loss = self._update_alpha(log_probs) + + self.update_target_networks() + + metrics = { + "critic1_loss": float(critic1_loss), + "critic2_loss": float(critic2_loss), + "actor_loss": float(actor_loss), + "learning_rate": float(self.lr_schedule(self.actor_optimizer.iterations)), + "alpha": float(tf.exp(self.log_alpha)) if self.alpha_auto_tune else float(self.alpha) + } + if alpha_loss is not None: + metrics["alpha_loss"] = float(alpha_loss) + + return metrics + + def save(self, path): + try: + self.actor.save_weights(f"{path}/actor.weights.h5"); self.critic1.save_weights(f"{path}/critic1.weights.h5") + self.critic2.save_weights(f"{path}/critic2.weights.h5") + if self.alpha_auto_tune and hasattr(self, 'log_alpha'): np.save(f"{path}/log_alpha.npy", self.log_alpha.numpy()) + sac_logger.info(f"Enhanced SAC Agent weights saved to {path}/") + except Exception as e: sac_logger.error(f"Error saving SAC weights: {e}") + + def load(self, path): + try: + if not self.actor.built: self.actor.build((None, self.state_dim)) + if not self.critic1.built: self.critic1.build([(None, self.state_dim), (None, self.action_dim)]) + if not self.critic2.built: self.critic2.build([(None, self.state_dim), (None, self.action_dim)]) + if not self.target_critic1.built: self.target_critic1.build([(None, self.state_dim), (None, self.action_dim)]) + if not self.target_critic2.built: self.target_critic2.build([(None, self.state_dim), (None, self.action_dim)]) + self.actor.load_weights(f"{path}/actor.weights.h5"); self.critic1.load_weights(f"{path}/critic1.weights.h5") + self.critic2.load_weights(f"{path}/critic2.weights.h5"); self.target_critic1.load_weights(f"{path}/critic1.weights.h5") + self.target_critic2.load_weights(f"{path}/critic2.weights.h5") + log_alpha_path = f"{path}/log_alpha.npy" + if self.alpha_auto_tune and os.path.exists(log_alpha_path): self.log_alpha.assign(np.load(log_alpha_path)); sac_logger.info(f"Loaded log_alpha value") + sac_logger.info(f"Enhanced SAC Agent weights loaded from {path}/") + except Exception as e: sac_logger.error(f"Error loading SAC weights from {path}: {e}. Ensure files exist/shapes match.") \ No newline at end of file diff --git a/gru_sac_predictor/src/sac_agent_simplified.py b/gru_sac_predictor/src/sac_agent_simplified.py new file mode 100644 index 00000000..0c8863fd --- /dev/null +++ b/gru_sac_predictor/src/sac_agent_simplified.py @@ -0,0 +1,503 @@ +import os +import numpy as np +import tensorflow as tf +from tensorflow.keras.models import Model +from tensorflow.keras.layers import Input, Dense, Concatenate, BatchNormalization, Add +from tensorflow.keras.optimizers import Adam +import tensorflow.keras.backend as K +import tensorflow_probability as tfp # V7.13 Import TFP + +tfd = tfp.distributions # V7.13 TFP distribution alias + +LOG_STD_MIN = -20 # V7.13 Min log std dev for numerical stability +LOG_STD_MAX = 2 # V7.13 Max log std dev for numerical stability + +class SimplifiedSACTradingAgent: + """ + Simplified SAC Trading Agent optimized for GRU-predicted returns and uncertainty + with guarantees for performance on M1 chips and smaller datasets. + V7.13: Updated for 5D state and automatic alpha tuning. + """ + def __init__( + self, + state_dim=5, # V7.13 Updated state: [pred_ret, unc, z, mom, vol] + action_dim=1, # Position size between -1 and 1 + hidden_size=64, # Reduced network size for faster training + gamma=0.97, # Discount factor for faster adaptation + tau=0.02, # Target network update rate + # alpha=0.1, # V7.13 Removed: Use automatic alpha tuning + actor_lr=3e-4, # V7.13 Default updated + critic_lr=5e-4, # V7.13 Default updated + alpha_lr=3e-4, # V7.13 Learning rate for alpha tuning + batch_size=64, # Smaller batch size for faster updates + buffer_max_size=20000, # Smaller buffer for recency bias + min_buffer_size=1000, # Start learning after this many experiences + update_interval=1, # Update actor every step + target_update_interval=2, # Update target networks every 2 steps + gradient_clip=1.0, # Clip gradients for stability + reward_scale=2.0, # V7.13 Default updated + use_batch_norm=True, # Use batch normalization + use_residual=True, # Use residual connections + target_entropy=None, # V7.13 Target entropy for alpha tuning + model_dir='models/simplified_sac', + ): + self.state_dim = state_dim + self.action_dim = action_dim + self.hidden_size = hidden_size + self.gamma = gamma + self.tau = tau + # self.alpha = alpha # V7.13 Removed + self.actor_lr = actor_lr + self.critic_lr = critic_lr + self.alpha_lr = alpha_lr # V7.13 Store alpha LR + self.batch_size = batch_size + self.buffer_max_size = buffer_max_size + self.min_buffer_size = min_buffer_size + self.update_interval = update_interval + self.target_update_interval = target_update_interval + self.gradient_clip = gradient_clip + self.reward_scale = reward_scale + self.use_batch_norm = use_batch_norm + self.use_residual = use_residual + self.model_dir = model_dir + + # V7.13 Alpha tuning setup + if target_entropy is None: + # Default target entropy heuristic: -dim(A)/2 suggested in instructions + self.target_entropy = -np.prod(self.action_dim) / 2.0 + else: + self.target_entropy = target_entropy + # Initialize log_alpha (trainable variable) - start near log(0.1) ~ -2.3 + self.log_alpha = tf.Variable(np.log(0.1), dtype=tf.float32, name='log_alpha') + self.alpha = tfp.util.DeferredTensor(self.log_alpha, tf.exp) # Exponentiated alpha + self.alpha_optimizer = Adam(learning_rate=self.alpha_lr, name='alpha_optimizer') + print(f"Initialized SAC with automatic alpha tuning. Target Entropy: {self.target_entropy:.2f}") + + # Experience replay buffer (simple numpy arrays instead of deque for performance) + self.buffer_counter = 0 + self.buffer_capacity = buffer_max_size + self.state_buffer = np.zeros((self.buffer_capacity, self.state_dim)) + self.action_buffer = np.zeros((self.buffer_capacity, self.action_dim)) + self.reward_buffer = np.zeros((self.buffer_capacity, 1)) + self.next_state_buffer = np.zeros((self.buffer_capacity, self.state_dim)) + self.done_buffer = np.zeros((self.buffer_capacity, 1)) + + # Step counter + self.train_step_counter = 0 + + # Create actor and critic networks + self.actor = self._build_actor() + self.critic_1 = self._build_critic() + self.critic_2 = self._build_critic() + self.target_critic_1 = self._build_critic() + self.target_critic_2 = self._build_critic() + + # Initialize target networks with actor and critic's weights + self.target_critic_1.set_weights(self.critic_1.get_weights()) + self.target_critic_2.set_weights(self.critic_2.get_weights()) + + # Create optimizers with gradient clipping + self.actor_optimizer = Adam(learning_rate=self.actor_lr, clipnorm=self.gradient_clip) + self.critic_optimizer = Adam(learning_rate=self.critic_lr, clipnorm=self.gradient_clip) + # V7.13 Alpha optimizer already created above + + # Loss tracking + self.actor_loss_history = [] + self.critic_loss_history = [] + + # Ensure model directory exists + os.makedirs(self.model_dir, exist_ok=True) + + # V7.9: Explicitly build models to prevent graph mode errors with optimizers + self._build_models_with_dummy_input() + + def _build_models_with_dummy_input(self): + """Builds actor and critic models with dummy inputs to initialize weights and optimizers.""" + try: + # Dummy state and action tensors + dummy_state = tf.zeros((1, self.state_dim), dtype=tf.float32) + dummy_action = tf.zeros((1, self.action_dim), dtype=tf.float32) + + # Build actor + self.actor(dummy_state) + # Build critics + self.critic_1([dummy_state, dummy_action]) + self.critic_2([dummy_state, dummy_action]) + self.target_critic_1([dummy_state, dummy_action]) + self.target_critic_2([dummy_state, dummy_action]) + + # Optionally, build optimizers (Adam usually builds on first apply_gradients) + # V7.10: Explicitly build optimizers too + if hasattr(self.actor_optimizer, 'build'): # Check if build method exists (newer TF) + self.actor_optimizer.build(self.actor.trainable_variables) + if hasattr(self.critic_optimizer, 'build'): + self.critic_optimizer.build(self.critic_1.trainable_variables + self.critic_2.trainable_variables) + # V7.13 Build alpha optimizer + if hasattr(self.alpha_optimizer, 'build'): + self.alpha_optimizer.build([self.log_alpha]) + print("Simplified SAC Agent models and optimizers built explicitly.") # Log success + except Exception as e: + print(f"Warning: Failed to explicitly build SAC models/optimizers: {e}") + + def _build_actor(self): + """Build a simplified actor network with optional batch norm and residual connections + V7.13: Outputs mean and log_std for a Gaussian policy. + """ + # Input layer + state_input = Input(shape=(self.state_dim,)) + + # First hidden layer + x = Dense(self.hidden_size, activation='relu')(state_input) + if self.use_batch_norm: + x = BatchNormalization()(x) + + # Second hidden layer + y = Dense(self.hidden_size, activation='relu')(x) + if self.use_batch_norm: + y = BatchNormalization()(y) + + # Optional residual connection + if self.use_residual and self.hidden_size == self.hidden_size: + z = Add()([x, y]) + else: + z = y + + # V7.13 Output layer(s) for mean and log_std + mu = Dense(self.action_dim, activation=None, name='mu')(z) + log_std = Dense(self.action_dim, activation=None, name='log_std')(z) + + # V7.13 Clip log_std for stability + log_std = tf.keras.ops.clip(log_std, LOG_STD_MIN, LOG_STD_MAX) + + # V7.13 Create model - outputs mean and log_std + model = Model(inputs=state_input, outputs=[mu, log_std]) + return model + + def _build_critic(self): + """Build a simplified critic network for Q-value estimation""" + # Input layers + state_input = Input(shape=(self.state_dim,)) + action_input = Input(shape=(self.action_dim,)) + + # Concatenate state and action + concat = Concatenate()([state_input, action_input]) + + # First hidden layer + x = Dense(self.hidden_size, activation='relu')(concat) + if self.use_batch_norm: + x = BatchNormalization()(x) + + # Second hidden layer + y = Dense(self.hidden_size, activation='relu')(x) + if self.use_batch_norm: + y = BatchNormalization()(y) + + # Optional residual connection + if self.use_residual and self.hidden_size == self.hidden_size: + z = Add()([x, y]) + else: + z = y + + # Output layer (linear activation for Q-value) + outputs = Dense( + 1, + activation=None, + kernel_initializer=tf.keras.initializers.RandomUniform(minval=-0.003, maxval=0.003) + )(z) + + # Create model + model = Model(inputs=[state_input, action_input], outputs=outputs) + return model + + def get_action(self, state, deterministic=False): + """Select trading action (-1 to 1) based on current state + V7.13: Samples from Gaussian policy, calculates log_prob. + """ + state_tensor = tf.convert_to_tensor([state], dtype=tf.float32) + # V7.13 Get mean and log_std from actor + mu, log_std = self.actor(state_tensor) + std = tf.exp(log_std) + + # Create policy distribution + policy_dist = tfd.Normal(mu, std) + + if deterministic: + # Use mean for deterministic action + action = mu + else: + # Sample using reparameterization trick + action = policy_dist.sample() + + # Calculate log probability of the sampled/deterministic action + log_prob = policy_dist.log_prob(action) + log_prob = tf.reduce_sum(log_prob, axis=1, keepdims=True) # Sum across action dim if > 1 + + # Apply tanh squashing + # Action is sampled from Normal, then squashed by tanh + squashed_action = tf.tanh(action) + + # Adjust log_prob for tanh squashing (important!) + # log π(a|s) = log ρ(u|s) - Σ log(1 - tanh(u)^2) + # where u is the pre-tanh action sampled from Normal + # See Appendix C in SAC paper: https://arxiv.org/abs/1801.01290 + log_prob -= tf.reduce_sum(tf.math.log(1.0 - squashed_action**2 + 1e-6), axis=1, keepdims=True) + + # Return the SQUASHED action and its log_prob + # Return numpy arrays for consistency with buffer etc. + return squashed_action[0].numpy(), log_prob[0].numpy() + + def store_transition(self, state, action, reward, next_state, done): + """Store experience in replay buffer with efficient circular indexing""" + # Scale reward + scaled_reward = reward * self.reward_scale + + # Get the index to store experience + index = self.buffer_counter % self.buffer_capacity + + # Store experience + self.state_buffer[index] = state + self.action_buffer[index] = action + self.reward_buffer[index] = scaled_reward + self.next_state_buffer[index] = next_state + self.done_buffer[index] = done + + # Increment counter + self.buffer_counter += 1 + + def sample_batch(self): + """Sample a batch of experiences from replay buffer""" + # Get the valid buffer size (min of counter and capacity) + buffer_size = min(self.buffer_counter, self.buffer_capacity) + + # Sample random indices + batch_indices = np.random.choice(buffer_size, self.batch_size) + + # Get batch + state_batch = self.state_buffer[batch_indices] + action_batch = self.action_buffer[batch_indices] + reward_batch = self.reward_buffer[batch_indices] + next_state_batch = self.next_state_buffer[batch_indices] + done_batch = self.done_buffer[batch_indices] + + return state_batch, action_batch, reward_batch, next_state_batch, done_batch + + def update_target_networks(self): + """Update target critic networks using Polyak averaging""" + # Update target critic 1 + target_weights = self.target_critic_1.get_weights() + critic_weights = self.critic_1.get_weights() + new_weights = [] + for i in range(len(target_weights)): + new_weights.append(self.tau * critic_weights[i] + (1 - self.tau) * target_weights[i]) + self.target_critic_1.set_weights(new_weights) + + # Update target critic 2 + target_weights = self.target_critic_2.get_weights() + critic_weights = self.critic_2.get_weights() + new_weights = [] + for i in range(len(target_weights)): + new_weights.append(self.tau * critic_weights[i] + (1 - self.tau) * target_weights[i]) + self.target_critic_2.set_weights(new_weights) + + @tf.function + def _update_critics(self, states, actions, rewards, next_states, dones): + """Update critic networks with TensorFlow graph execution""" + with tf.GradientTape(persistent=True) as tape: + # V7.18 START: Get next action and log_prob for target Q calculation + next_mu, next_log_std = self.actor(next_states) + next_std = tf.exp(next_log_std) + next_policy_dist = tfd.Normal(next_mu, next_std) + + # Sample action for policy evaluation + next_action_presquash = next_policy_dist.sample() + next_action = tf.tanh(next_action_presquash) # Apply squashing + + # Calculate log_prob, adjusting for tanh squashing + next_log_prob = next_policy_dist.log_prob(next_action_presquash) + next_log_prob = tf.reduce_sum(next_log_prob, axis=1, keepdims=True) + next_log_prob -= tf.reduce_sum(tf.math.log(1.0 - next_action**2 + 1e-6), axis=1, keepdims=True) + # V7.18 END: Get next action and log_prob + + # Get target Q values using the SQUASHED next_action + target_q1 = self.target_critic_1([next_states, next_action]) + target_q2 = self.target_critic_2([next_states, next_action]) + + # Use minimum Q value (Double Q learning) + min_target_q = tf.minimum(target_q1, target_q2) + + # V7.18 Calculate target Q including entropy term + target_q_entropy_adjusted = min_target_q - self.alpha * next_log_prob + + # Calculate target values: r + γ(1-done) * (min(Q′(s′,a′)) - α log π(a′|s′)) + target_values = rewards + self.gamma * (1 - dones) * target_q_entropy_adjusted + + # Stop gradient flow through target values + target_values = tf.stop_gradient(target_values) + + # Get current Q estimates + current_q1 = self.critic_1([states, actions]) + current_q2 = self.critic_2([states, actions]) + + # Calculate critic losses (Huber loss for robustness) + critic1_loss = tf.reduce_mean(tf.keras.losses.huber(target_values, current_q1, delta=1.0)) + critic2_loss = tf.reduce_mean(tf.keras.losses.huber(target_values, current_q2, delta=1.0)) + critic_loss = critic1_loss + critic2_loss + + # Get critic gradients + critic1_gradients = tape.gradient(critic1_loss, self.critic_1.trainable_variables) + critic2_gradients = tape.gradient(critic2_loss, self.critic_2.trainable_variables) + + # Apply critic gradients + self.critic_optimizer.apply_gradients(zip(critic1_gradients, self.critic_1.trainable_variables)) + self.critic_optimizer.apply_gradients(zip(critic2_gradients, self.critic_2.trainable_variables)) + + del tape + return critic_loss + + @tf.function + def _update_actor_and_alpha(self, states): + """Update actor network and alpha temperature with TensorFlow graph execution.""" + with tf.GradientTape(persistent=True) as tape: + # Get policy distribution parameters + mu, log_std = self.actor(states) + std = tf.exp(log_std) + policy_dist = tfd.Normal(mu, std) + + # Sample action using reparameterization trick + actions_presquash = policy_dist.sample() + log_prob_presquash = policy_dist.log_prob(actions_presquash) + log_prob_presquash = tf.reduce_sum(log_prob_presquash, axis=1, keepdims=True) + + # Apply squashing + squashed_actions = tf.tanh(actions_presquash) + + # Adjust log_prob for squashing + log_prob = log_prob_presquash - tf.reduce_sum(tf.math.log(1.0 - squashed_actions**2 + 1e-6), axis=1, keepdims=True) + + # Get Q values from critics using squashed actions + q1_values = self.critic_1([states, squashed_actions]) + q2_values = self.critic_2([states, squashed_actions]) + min_q_values = tf.minimum(q1_values, q2_values) + + # Calculate actor loss: E[alpha * log_prob - Q] + actor_loss = tf.reduce_mean(self.alpha * log_prob - min_q_values) + + # Calculate alpha loss: E[-alpha * (log_prob + target_entropy)] + # Note: We use log_alpha and optimize that variable + alpha_loss = -tf.reduce_mean(self.log_alpha * tf.stop_gradient(log_prob + self.target_entropy)) + + # --- Actor Gradients --- + actor_gradients = tape.gradient(actor_loss, self.actor.trainable_variables) + self.actor_optimizer.apply_gradients(zip(actor_gradients, self.actor.trainable_variables)) + + # --- Alpha Gradients --- + alpha_gradients = tape.gradient(alpha_loss, [self.log_alpha]) + self.alpha_optimizer.apply_gradients(zip(alpha_gradients, [self.log_alpha])) + + del tape # Release tape resources + + return actor_loss, alpha_loss # Return losses for tracking + + def train(self, num_iterations=1): + """Train the agent for a specified number of iterations""" + # Don't train if buffer doesn't have enough experiences + if self.buffer_counter < self.min_buffer_size: + return None # V7.17 Return None if not training + + total_actor_loss = 0 + total_critic_loss = 0 + total_alpha_loss = 0 # V7.19 Track alpha loss + + iterations_run = 0 + for _ in range(num_iterations): + # Sample a batch of experiences + states, actions, rewards, next_states, dones = self.sample_batch() + + # Convert to tensors + states = tf.convert_to_tensor(states, dtype=tf.float32) + actions = tf.convert_to_tensor(actions, dtype=tf.float32) + rewards = tf.convert_to_tensor(rewards, dtype=tf.float32) + next_states = tf.convert_to_tensor(next_states, dtype=tf.float32) + dones = tf.convert_to_tensor(dones, dtype=tf.float32) + + # Update critics + critic_loss = self._update_critics(states, actions, rewards, next_states, dones) + total_critic_loss += critic_loss + + # Update actor and alpha less frequently for stability? + # Original implementation updates actor every step, let's stick to that for now + # if self.train_step_counter % self.update_interval == 0: + actor_loss, alpha_loss = self._update_actor_and_alpha(states) + total_actor_loss += actor_loss + total_alpha_loss += alpha_loss # V7.19 Accumulate alpha loss + + # Track losses (consider appending outside the loop if num_iterations > 1) + self.actor_loss_history.append(actor_loss.numpy()) + self.critic_loss_history.append(critic_loss.numpy()) + + # Update target networks less frequently + if self.train_step_counter % self.target_update_interval == 0: + self.update_target_networks() + + # Increment step counter + self.train_step_counter += 1 + iterations_run += 1 + + # Return average losses for the iterations run in this call + avg_actor_loss = total_actor_loss / iterations_run if iterations_run > 0 else tf.constant(0.0) + avg_critic_loss = total_critic_loss / iterations_run if iterations_run > 0 else tf.constant(0.0) + avg_alpha_loss = total_alpha_loss / iterations_run if iterations_run > 0 else tf.constant(0.0) + + # V7.17 Return tuple (actor_loss, critic_loss) - Needs update for alpha + # V7.19 Return losses including alpha loss (or maybe just actor/critic for main history) + return avg_actor_loss, avg_critic_loss # Keep original return for compatibility with main loop plotting for now + + def save(self, checkpoint_dir=None): + """Save the model weights""" + if checkpoint_dir is None: + checkpoint_dir = self.model_dir + + os.makedirs(checkpoint_dir, exist_ok=True) + + # Save actor weights + self.actor.save_weights(os.path.join(checkpoint_dir, 'actor.weights.h5')) + + # Save critic weights + self.critic_1.save_weights(os.path.join(checkpoint_dir, 'critic_1.weights.h5')) + self.critic_2.save_weights(os.path.join(checkpoint_dir, 'critic_2.weights.h5')) + + # Save target critic weights + self.target_critic_1.save_weights(os.path.join(checkpoint_dir, 'target_critic_1.weights.h5')) + self.target_critic_2.save_weights(os.path.join(checkpoint_dir, 'target_critic_2.weights.h5')) + + # Save alpha + np.save(os.path.join(checkpoint_dir, 'alpha.npy'), self.alpha) + + print(f"Model saved to {checkpoint_dir}") + + def load(self, checkpoint_dir=None): + """Load the model weights""" + if checkpoint_dir is None: + checkpoint_dir = self.model_dir + + try: + # Load actor weights + self.actor.load_weights(os.path.join(checkpoint_dir, 'actor.weights.h5')) + + # Load critic weights + self.critic_1.load_weights(os.path.join(checkpoint_dir, 'critic_1.weights.h5')) + self.critic_2.load_weights(os.path.join(checkpoint_dir, 'critic_2.weights.h5')) + + # Load target critic weights + self.target_critic_1.load_weights(os.path.join(checkpoint_dir, 'target_critic_1.weights.h5')) + self.target_critic_2.load_weights(os.path.join(checkpoint_dir, 'target_critic_2.weights.h5')) + + # Load alpha + if os.path.exists(os.path.join(checkpoint_dir, 'alpha.npy')): + self.alpha = float(np.load(os.path.join(checkpoint_dir, 'alpha.npy'))) + + print(f"Model loaded from {checkpoint_dir}") + return True + except Exception as e: + print(f"Error loading model: {e}") + return False \ No newline at end of file diff --git a/gru_sac_predictor/src/trading_system.py b/gru_sac_predictor/src/trading_system.py new file mode 100644 index 00000000..1bd725a3 --- /dev/null +++ b/gru_sac_predictor/src/trading_system.py @@ -0,0 +1,1783 @@ +import numpy as np +import pandas as pd +import os +from tqdm import tqdm # Added for progress tracking +import logging # Added for feature functions +from scipy import stats # Added for feature functions +from typing import Tuple +import tensorflow as tf +import warnings +import sys + +# Optional: Import TA-Lib if available (used by V6 features) +try: + import talib + TALIB_AVAILABLE = True + # logging.info("TA-Lib found. Using TA-Lib for V6 features.") # Reduce noise +except ImportError: + TALIB_AVAILABLE = False + logging.warning("TA-Lib not found. V6 feature calculation will be limited.") + +# Import other components from the src directory +from .gru_predictor import CryptoGRUModel +from .sac_agent_simplified import SimplifiedSACTradingAgent +# Remove direct import of prepare_features as logic is integrated here +# from .data_pipeline import prepare_features + +# Optional: Matplotlib for plotting +try: + import matplotlib.pyplot as plt +except ImportError: + plt = None + +# --- V7.12 START: Add SAC Training History Plotting Function --- +def plot_sac_training_history(history, save_path=None, run_id=None): + """ + Plots the actor and critic loss from the SAC training history. + + Args: + history (list[dict]): A list of dictionaries, where each dict contains + metrics like 'actor_loss' and 'critic_loss'. + save_path (str, optional): Path to save the plot. If None, tries to show plot. + run_id (str, optional): Run ID to include in the plot title. + """ + if plt is None: + print("Matplotlib not available, skipping SAC training plot.") + return + + if not history: + print("No SAC training history data provided, skipping plot.") + return + + try: + actor_losses = [item['actor_loss'] for item in history if 'actor_loss' in item and item['actor_loss'] is not None] + critic_losses = [item['critic_loss'] for item in history if 'critic_loss' in item and item['critic_loss'] is not None] + steps = range(1, len(history) + 1) # Assuming one entry per training step/batch + + if not actor_losses or not critic_losses: + print("Warning: Could not extract valid actor or critic losses from history. Skipping plot.") + return + + # Determine common length if lists differ (shouldn't usually happen if logged together) + min_len = min(len(actor_losses), len(critic_losses)) + if min_len < len(history): + print(f"Warning: Plotting only {min_len} steps due to missing data.") + steps = range(1, min_len + 1) + actor_losses = actor_losses[:min_len] + critic_losses = critic_losses[:min_len] + + fig, axs = plt.subplots(2, 1, figsize=(12, 10), sharex=True) + fig.suptitle(f'SAC Training History {"(Run: " + run_id + ")" if run_id else ""}', fontsize=16) + + # Actor Loss Plot + axs[0].plot(steps, actor_losses, label='Actor Loss', color='tab:blue') + axs[0].set_ylabel('Loss') + axs[0].set_title('Actor Loss over Training Steps') + axs[0].legend() + axs[0].grid(True) + + # Critic Loss Plot + axs[1].plot(steps, critic_losses, label='Critic Loss', color='tab:orange') + axs[1].set_xlabel('Agent Training Step') + axs[1].set_ylabel('Loss') + axs[1].set_title('Critic Loss over Training Steps') + axs[1].legend() + axs[1].grid(True) + + plt.tight_layout(rect=[0, 0.03, 1, 0.95]) # Adjust layout to prevent title overlap + + if save_path: + try: + plt.savefig(save_path) + print(f"SAC training history plot saved to {save_path}") + except Exception as e: + print(f"Error saving SAC training plot: {e}") + else: + try: + print("Displaying SAC training plot...") + plt.show() + except Exception as e: + print(f"Error displaying SAC training plot: {e}") + + except KeyError as e: + print(f"KeyError accessing loss data in history: {e}. Skipping plot.") + except Exception as e: + print(f"An unexpected error occurred during SAC plot generation: {e}") + finally: + # Ensure plot is closed to free memory, especially if not shown interactively + if plt: + plt.close(fig) +# --- V7.12 END: Add SAC Training History Plotting Function --- + + +# --- V6 Feature Calculation Logic (Copied & Adapted) --- +# Configure logging for feature calculation functions +feature_logger = logging.getLogger('TradingSystemFeatures') +# Set level to WARNING to avoid excessive logs during backtest +feature_logger.setLevel(logging.WARNING) + +# V7 Update: Add Scalers and sequence creation +from sklearn.preprocessing import MinMaxScaler, StandardScaler +from .data_pipeline import create_sequences_v2 + +# Copied from v6/src/cryptofeatures.py +def calculate_vwap(df: pd.DataFrame, period: int = None) -> pd.Series: + """ + Calculate Volume Weighted Average Price (VWAP). + Adapted from V6. + """ + try: + typical_price = (df['high'] + df['low'] + df['close']) / 3 + tp_volume = typical_price * df['volume'] + volume_sum = df['volume'].rolling(window=period, min_periods=1).sum() + tp_volume_sum = tp_volume.rolling(window=period, min_periods=1).sum() + volume_sum_safe = volume_sum.replace(0, np.nan) + vwap = tp_volume_sum / volume_sum_safe + vwap = vwap.fillna(method='ffill').fillna(method='bfill').fillna(df['close']) # More robust fill + return vwap + except Exception as e: + feature_logger.error(f"Failed to calculate VWAP (period={period}): {e}") + return df['close'] # Fallback + +# Copied from v6/src/cryptofeatures.py and adapted +def add_crypto_features(df: pd.DataFrame) -> pd.DataFrame: + """Add cryptocurrency-specific features. Adapted from V6.""" + df_features = df.copy() + try: + # Parkinson Volatility - Add better safeguards + high_safe = df_features['high'].replace(0, np.nan) + low_safe = df_features['low'].replace(0, np.nan).fillna(high_safe * 0.999) # Avoid division by zero + high_to_low = np.clip(high_safe / low_safe, 1.0, 10.0) # Clip extreme ratios + log_hl_sq = np.log(high_to_low)**2 + df_features['parkinson_vol_14'] = np.sqrt((1 / (4 * np.log(2))) * log_hl_sq.rolling(window=14).mean()) + + # Garman-Klass Volatility - Add better safeguards + close_safe = df_features['close'].replace(0, np.nan) + open_safe = df_features['open'].replace(0, np.nan).fillna(close_safe * 0.999) # Avoid division by zero + close_to_open = np.clip(close_safe / open_safe, 0.5, 2.0) # Clip extreme ratios + log_co_sq = np.log(close_to_open)**2 + gk_vol = (0.5 * log_hl_sq - (2 * np.log(2) - 1) * log_co_sq).rolling(window=14).mean() + # Ensure non-negative values before sqrt + gk_vol = np.maximum(gk_vol, 0) + df_features['garman_klass_vol_14'] = np.sqrt(gk_vol) + + # VWAP Features - Improve robustness + for period in [30, 60, 120]: + df_features[f'vwap_{period}'] = calculate_vwap(df_features, period=period) + vwap_safe = df_features[f'vwap_{period}'].replace(0, np.nan).fillna(close_safe) + # Clip the ratio to a reasonable range to avoid extreme values + df_features[f'close_to_vwap_{period}'] = np.clip((close_safe / vwap_safe) - 1, -0.5, 0.5) + + # V7 Fix: Cyclical features are now calculated earlier in main.py + # # Cyclical Features (Simplified) + # if isinstance(df_features.index, pd.DatetimeIndex): + # df_features['hour_sin'] = np.sin(2 * np.pi * df_features.index.hour / 24) + # df_features['hour_cos'] = np.cos(2 * np.pi * df_features.index.hour / 24) + + # Volume Intensity - Improve stability + vol = df_features['volume'].replace(0, np.nan) + vol_mean_30 = vol.rolling(window=30).mean() + vol_mean_30_safe = vol_mean_30.replace(0, np.nan).fillna(vol.mean()) + df_features['vol_intensity'] = np.clip(vol / vol_mean_30_safe, 0, 10) # Clip to reasonable range + + # Price Pattern Features - Improved stability + body = abs(close_safe - open_safe) + # Use a meaningful minimum body size to avoid extreme ratios + min_body_size = df_features['close'].mean() * 0.0001 # Small percentage of avg price + body_safe = np.maximum(body, min_body_size) + + # Calculate wicks with better handling of extreme cases + upper_wick = df_features['high'] - df_features[['open', 'close']].max(axis=1) + lower_wick = df_features[['open', 'close']].min(axis=1) - df_features['low'] + # Clip the ratios to reasonable ranges + df_features['upper_wick_ratio'] = np.clip(upper_wick / body_safe, 0, 5) + df_features['lower_wick_ratio'] = np.clip(lower_wick / body_safe, 0, 5) + + # Fill NaNs introduced here + cols_to_fill = ['parkinson_vol_14', 'garman_klass_vol_14', 'vol_intensity', + 'upper_wick_ratio', 'lower_wick_ratio'] # Removed hour_sin/cos + cols_to_fill.extend([f'close_to_vwap_{p}' for p in [30, 60, 120]]) + for col in cols_to_fill: + if col in df_features.columns: + # Fill NaNs with median values rather than zeros to maintain distribution + median_val = df_features[col].median() + df_features[col].fillna(median_val if not np.isnan(median_val) else 0, inplace=True) + + except Exception as e: + feature_logger.error(f"Error calculating crypto-specific features: {e}", exc_info=True) + return df_features + +# Adapted from v6/src/data_preprocessing.py::calculate_technical_indicators +def calculate_v6_features(df: pd.DataFrame) -> pd.DataFrame: + """Calculates V6 technical indicators + basic return features.""" + df_features = df.copy() + required_cols = ['open', 'high', 'low', 'close', 'volume'] + if not all(col in df_features.columns for col in required_cols): + missing = [col for col in required_cols if col not in df_features.columns] + feature_logger.error(f"Missing required V6 columns: {missing}") + return df_features + for col in required_cols: + df_features[col] = pd.to_numeric(df_features[col], errors='coerce') + if df_features[col].isnull().any(): + df_features[col] = df_features[col].ffill().bfill() + if df_features[col].isnull().any(): df_features[col] = df_features[col].fillna(0) + + # --- V7.2 Add Past Return Features --- + for lag in [1, 5, 15, 60]: # 1m, 5m, 15m, 1h returns + # Use pct_change for robustness to price levels + df_features[f'return_{lag}m'] = df_features['close'].pct_change(periods=lag) + # --- End Return Features --- + + if TALIB_AVAILABLE: + try: + close = df_features['close'].values; open_price = df_features['open'].values + high = df_features['high'].values; low = df_features['low'].values; volume = df_features['volume'].values + for period in [5, 10, 20, 30, 50, 100]: + df_features[f'SMA_{period}'] = talib.SMA(close, timeperiod=period) + df_features[f'EMA_{period}'] = talib.EMA(close, timeperiod=period) + macd, macdsignal, macdhist = talib.MACD(close, fastperiod=12, slowperiod=26, signalperiod=9) + df_features['MACD'] = macd; df_features['MACD_signal'] = macdsignal; df_features['MACD_hist'] = macdhist + df_features['SAR'] = talib.SAR(high, low, acceleration=0.02, maximum=0.2) + df_features['ADX_14'] = talib.ADX(high, low, close, timeperiod=14) + for period in [9, 14, 21]: df_features[f'RSI_{period}'] = talib.RSI(close, timeperiod=period) + slowk, slowd = talib.STOCH(high, low, close, fastk_period=14, slowk_period=3, slowk_matype=0, slowd_period=3, slowd_matype=0) + df_features['STOCH_K'] = slowk; df_features['STOCH_D'] = slowd + df_features['WILLR_14'] = talib.WILLR(high, low, close, timeperiod=14) + for period in [5, 10, 20]: df_features[f'ROC_{period}'] = talib.ROC(close, timeperiod=period) + df_features['CCI_14'] = talib.CCI(high, low, close, timeperiod=14) + upper20, middle20, lower20 = talib.BBANDS(close, timeperiod=20, nbdevup=2, nbdevdn=2, matype=0) + df_features['BB_upper_20'] = upper20; df_features['BB_middle_20'] = middle20; df_features['BB_lower_20'] = lower20 + middle20_safe = np.where(middle20 == 0, np.nan, middle20) + bb_width = (upper20 - lower20) / middle20_safe + df_features['BB_width_20'] = np.nan_to_num(bb_width, nan=0.0) + for period in [7, 14, 21]: df_features[f'ATR_{period}'] = talib.ATR(high, low, close, timeperiod=period) + df_features['OBV'] = talib.OBV(close, volume) + df_features['CMF_20'] = talib.ADOSC(high, low, close, volume, fastperiod=3, slowperiod=10) + vol_sma_20 = talib.SMA(volume, timeperiod=20) + vol_sma_20_safe = np.where(vol_sma_20 == 0, np.nan, vol_sma_20) + df_features['volume_SMA_20'] = vol_sma_20 + df_features['volume_ratio'] = np.nan_to_num(volume / vol_sma_20_safe, nan=1.0) + df_features['DOJI'] = talib.CDLDOJI(open_price, high, low, close) / 100 + df_features['ENGULFING'] = talib.CDLENGULFING(open_price, high, low, close) / 100 + df_features['HAMMER'] = talib.CDLHAMMER(open_price, high, low, close) / 100 + df_features['SHOOTING_STAR'] = talib.CDLSHOOTINGSTAR(open_price, high, low, close) / 100 + except Exception as e: feature_logger.error(f"Error calculating TA-Lib indicators: {e}", exc_info=True) + + if not TALIB_AVAILABLE: + feature_logger.warning("Calculating subset of features manually (TA-Lib unavailable)") + for period in [5, 10, 20]: + df_features[f'SMA_{period}'] = df_features['close'].rolling(window=period).mean() + df_features[f'EMA_{period}'] = df_features['close'].ewm(span=period, adjust=False).mean() + # df_features[f'ROC_{period}'] = df_features['close'].pct_change(periods=period) * 100 # Replaced by return_xm + delta = df_features['close'].diff(); gain = (delta.where(delta > 0, 0)).rolling(window=14).mean() + loss = (-delta.where(delta < 0, 0)).rolling(window=14).mean().replace(0, 1e-10); rs = gain / loss + df_features['RSI_14'] = 100 - (100 / (1 + rs)) + + # Derived Features (always calculated) + for period in [14, 30]: + roll_max = df_features['close'].rolling(window=period).max() + roll_min = df_features['close'].rolling(window=period).min() + # Ensure the range isn't too small to avoid extreme ratios + min_range = df_features['close'].mean() * 0.001 # 0.1% of mean price as minimum range + denominator = np.maximum(roll_max - roll_min, min_range) + # Relative position is a ratio - apply Fisher transform to make more normal + rel_pos = np.clip((df_features['close'] - roll_min) / denominator, 0.01, 0.99) + # Apply logit transform (inverse of sigmoid) to normalize the distribution + df_features[f'rel_position_{period}'] = np.log(rel_pos / (1 - rel_pos)) + + for period in [20, 50]: + if f'SMA_{period}' in df_features.columns: + sma = df_features[f'SMA_{period}'] + close = df_features['close'] + # Apply log transform to price-to-SMA ratio to make distribution more normal + ratio = np.clip(close / np.maximum(sma, close.mean() * 0.01), 0.5, 2.0) + df_features[f'price_dist_SMA_{period}'] = np.log(ratio) + + # Intraday return - with log transform for better distribution + open_prices = df_features['open'].replace(0, np.nan) + close_prices = df_features['close'] + # Use a reasonable minimum price to avoid extreme ratios + min_open = close_prices.mean() * 0.01 # 1% of mean close + open_safe = np.maximum(open_prices.fillna(close_prices), min_open) + # Apply log to price ratio for better distribution + ratio = np.clip(close_prices / open_safe, 0.5, 2.0) + df_features['intraday_return'] = np.log(ratio) + + # Log return with improved handling + prev_close = df_features['close'].shift(1).replace(0, np.nan) + current_close = df_features['close'] + # Avoid division by very small values + ratio = np.clip(current_close / np.maximum(prev_close.fillna(current_close), current_close.mean() * 0.01), 0.5, 2.0) + df_features['log_return'] = np.log(ratio) + + # Volatility with robust handling - apply log transform to make more normal + log_return_clipped = np.clip(df_features['log_return'], -0.2, 0.2) + vol = log_return_clipped.rolling(window=14).std().fillna(0) + # Log transform volatility for more normal distribution + df_features['volatility_14d'] = np.log1p(vol * 100) # log1p avoids issues with zero values + + # Add Crypto Features + df_features = add_crypto_features(df_features) + + # Final NaN fill (important after all calculations) + # Fill with 0 as done in V6 for most features + cols_to_fill = df_features.columns.difference(required_cols) + for col in cols_to_fill: + if df_features[col].isnull().any(): + df_features[col].fillna(0, inplace=True) + + # Drop original OHLCV + volume columns? No, keep them for potential analysis/state + # df_features = df_features.drop(columns=required_cols) + + return df_features +# --- End V6 Feature Logic --- + + +class TradingSystem: + """ + V7 Trading System: Integrates GRU Predictor and SAC Agent. + Generates features, uses GRU for price/uncertainty prediction, + and SAC for trading decisions. + """ + + # V7.2 Add system logger + _logger = logging.getLogger(__name__) + if not _logger.hasHandlers(): + _logger.setLevel(logging.INFO) + handler = logging.StreamHandler(sys.stdout) + formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') + handler.setFormatter(formatter) + _logger.addHandler(handler) + _logger.propagate = False + + def __init__(self, gru_model: CryptoGRUModel = None, sac_agent: SimplifiedSACTradingAgent = None, gru_lookback=60): + """ + Initialize the TradingSystem. + + Args: + gru_model (CryptoGRUModel, optional): Pre-initialized GRU model. Defaults to None. + sac_agent (SACTradingAgent, optional): Pre-initialized SAC agent. Defaults to None. + If None, a default agent with state_dim=2 is created. + gru_lookback (int): Number of time steps for GRU input sequence. + """ + self._logger.info("Initializing V7 Trading System...") + self.gru_model = gru_model or CryptoGRUModel() # Use provided or create new + self.sac_agent = sac_agent or SimplifiedSACTradingAgent(state_dim=2) # V7 state: [pred_return, uncertainty] + self.gru_lookback = gru_lookback + self.feature_scaler = None + self.y_scaler = None + self.last_prediction = None + self.experiences = [] # Store (state, action, reward, next_state, done) + # Initialize scalers from GRU model if available and loaded + if self.gru_model and self.gru_model.is_loaded: + self.feature_scaler = self.gru_model.feature_scaler + self.y_scaler = self.gru_model.y_scaler + self._logger.info("Scalers initialized from pre-loaded GRU model.") + + # V7-V6 Update: Extract features and predict return/uncertainty for SAC state + def _extract_features_and_predict(self, data_df_full, current_idx): + """ + Extracts feature sequence and uses the trained GRU model to predict + the next period's return and associated confidence/uncertainty. + + Args: + data_df_full: Full DataFrame with OHLCV and potentially pre-calculated features. + current_idx: The current index in the DataFrame. + + Returns: + Tuple(np.array, float, float) or (None, None, None): + - features_sequence_scaled: Scaled NumPy array (sequence_length, num_features) + - predicted_return: GRU predicted return for the next period. + - uncertainty_sigma: Uncertainty estimate for the next period. + """ + # --- Step 1: Extract Raw Feature Sequence --- + # Determine the slice needed for feature calculation (V6 style) + # Needs enough history for longest V6 feature calc + sequence length + # Assume V6 features require ~100 periods + sequence length + required_feature_history = 100 + self.gru_lookback # Conservative estimate + if current_idx < required_feature_history - 1: + # feature_logger.debug(f"Not enough history at idx {current_idx} for feature calc + lookback ({required_feature_history})") + return None, None, None + + start_idx_calc = max(0, current_idx - required_feature_history + 1) + end_idx_calc = current_idx + 1 # Include current index + data_slice_for_calc = data_df_full.iloc[start_idx_calc:end_idx_calc] + + # --- Step 2: Calculate All V6 Features --- + try: + df_with_features = calculate_v6_features(data_slice_for_calc) + if df_with_features.empty or len(df_with_features) < self.gru_lookback: + feature_logger.warning(f"Feature calculation failed or insufficient length at idx {current_idx}") + return None, None, None + except Exception as e: + feature_logger.error(f"Error calculating V6 features at idx {current_idx}: {e}", exc_info=False) + return None, None, None + + # --- Step 3: Select Final Sequence & Scale --- + # The sequence ends at the *current* index (current_idx) + feature_sequence_unscaled_df = df_with_features.iloc[-self.gru_lookback:] + + # Check if feature scaler exists (should after GRU training/loading) + if self.gru_model is None or self.feature_scaler is None: + feature_logger.error(f"GRU model or feature scaler not available at idx {current_idx}. Cannot scale features.") + return None, None, None + + try: + # Ensure columns match scaler's expectations + expected_features = self.feature_scaler.feature_names_in_ + # Select and reorder columns + feature_sequence_aligned_df = feature_sequence_unscaled_df[expected_features] + + # Scale the sequence - Pass the DataFrame directly + # V7 Warning Fix: Pass DataFrame to transform, not .values + features_sequence_scaled_array = self.feature_scaler.transform(feature_sequence_aligned_df) + + # Ensure output is numpy array for reshaping + if isinstance(features_sequence_scaled_array, pd.DataFrame): + features_sequence_scaled = features_sequence_scaled_array.values + else: + features_sequence_scaled = features_sequence_scaled_array # Assume it's already numpy + + if features_sequence_scaled.shape[0] != self.gru_lookback: + feature_logger.error(f"Scaled sequence shape incorrect at idx {current_idx}: {features_sequence_scaled.shape}") + return None, None, None + except KeyError as e: + feature_logger.error(f"Missing expected feature columns for scaler at idx {current_idx}: {e}") + feature_logger.error(f" Available columns: {feature_sequence_unscaled_df.columns.tolist()}") + feature_logger.error(f" Expected columns: {expected_features}") + return None, None, None + except Exception as e: + feature_logger.error(f"Error scaling feature sequence at idx {current_idx}: {e}", exc_info=False) + return None, None, None + + # --- Step 4: Predict Return and Uncertainty using GRU --- + # Perform efficient single-step prediction and MC dropout here + try: + # Reshape for model prediction (1, seq_len, n_features) + model_input = features_sequence_scaled.reshape(1, self.gru_lookback, features_sequence_scaled.shape[1]) + + # 4a. Get standard prediction (scaled price) + pred_scaled = self.gru_model.model.predict(model_input, verbose=0)[0, 0] + + # 4b. Perform MC Dropout for uncertainty + n_mc_samples = 30 # Use a reasonable number of samples + mc_preds_scaled_list = [] + + # Define the prediction step with training=True + @tf.function + def mc_predict_step_single(batch): + return self.gru_model.model(batch, training=True) # Enable dropout + + for _ in range(n_mc_samples): + mc_pred_scaled = mc_predict_step_single(tf.constant(model_input, dtype=tf.float32)).numpy() + mc_preds_scaled_list.append(mc_pred_scaled[0, 0]) # Get the scalar prediction + + mc_preds_scaled_array = np.array(mc_preds_scaled_list) + mc_std_scaled = np.std(mc_preds_scaled_array) + + # 4c. Unscale prediction and uncertainty + if self.y_scaler is None: + feature_logger.error(f"Y-scaler not available at idx {current_idx}, cannot unscale prediction/uncertainty.") + return None, None, None + + pred_unscaled = self.y_scaler.inverse_transform([[pred_scaled]])[0, 0] + mc_unscaled_std_dev = 0.0 # Default + try: + # Use scaler's data range + if hasattr(self.y_scaler, 'data_min_') and hasattr(self.y_scaler, 'data_max_'): + data_range = self.y_scaler.data_max_[0] - self.y_scaler.data_min_[0] + if data_range > 1e-9: + mc_unscaled_std_dev = mc_std_scaled * data_range + else: feature_logger.warning("Scaler data range is near zero for unscaling std dev.") + else: feature_logger.warning("y_scaler missing data_min_/data_max_ for unscaling std dev.") + except Exception as e: + feature_logger.error(f"Error unscaling MC std dev at idx {current_idx}: {e}") + + # 4d. Calculate predicted return + # Get the *actual* close price at the *current* index to calculate return + # Use the unscaled features dataframe for this + start_price = feature_sequence_unscaled_df.iloc[-1]['close'] + epsilon = 1e-9 + if abs(start_price) > epsilon: + predicted_return = (pred_unscaled / start_price) - 1.0 + else: + predicted_return = 0.0 + feature_logger.warning(f"Start price is near zero at idx {current_idx}, cannot calculate return.") + + # Return the SCALED features, predicted return, and UNCERTAINTY SIGMA + return features_sequence_scaled, predicted_return, mc_unscaled_std_dev + + except Exception as e: + feature_logger.error(f"Error during GRU prediction/evaluation step at idx {current_idx}: {e}", exc_info=False) + return None, None, None + + # V7-V6 Update: Rename and modify to reflect price target + def _preprocess_data_for_gru_training(self, df_data: pd.DataFrame, prediction_horizon: int) -> Tuple[pd.DataFrame, pd.Series, pd.Series]: + """ + Preprocess data specifically for GRU model training (V6 Style - Price Target). + Calculates V6 features, defines future price target, and aligns data. + Scaling and sequence creation will happen *within* train_gru. + + Args: + df_data: Raw OHLCV DataFrame + prediction_horizon: How many steps ahead to predict the price. + + Returns: + Tuple of (features DataFrame, future_price_target Series, start_price Series) + Returns (None, None, None) if processing fails. + """ + if df_data is None or df_data.empty: + logging.error("Empty/None dataframe provided to _preprocess_data_for_gru_training") + return None, None, None + + logging.info(f"Preprocessing data for GRU training (Target: Future Price, Horizon: {prediction_horizon})...") + + # --- Calculate V6 Features --- + logging.debug(f"Calculating V6 features for data shape {df_data.shape}") + try: + features_df = calculate_v6_features(df_data.copy()) # Use a copy + if features_df is None or features_df.empty: + logging.error("Feature calculation returned empty DataFrame.") + return None, None, None + except Exception as e: + logging.error(f"Error calculating V6 features: {e}", exc_info=True) + return None, None, None + + # --- Define Target (Future Price) & Start Price --- + try: + # Target is the closing price `prediction_horizon` steps into the future + future_price_target_ser = features_df['close'].shift(-prediction_horizon) + + # Start price is the current closing price (aligned with features) + start_price_ser = features_df['close'] + + # --- Align Features, Target, and Start Price --- + # Drop rows where the future target is NaN (typically at the end) + common_index = future_price_target_ser.dropna().index + + # Ensure indices exist in all structures before slicing + common_index = common_index.intersection(features_df.index).intersection(start_price_ser.index) + + if common_index.empty: + logging.error("No common index found after aligning features and future price target.") + return None, None, None + + features_aligned_df = features_df.loc[common_index] + target_aligned_ser = future_price_target_ser.loc[common_index] + start_price_aligned_ser = start_price_ser.loc[common_index] + + except KeyError as e: + logging.error(f"Missing 'close' column for target/start price calculation: {e}") + return None, None, None + except Exception as e: + logging.error(f"Error defining/aligning target and start price: {e}", exc_info=True) + return None, None, None + + # Final check for NaN values introduced during alignment or feature calc + if features_aligned_df.isna().any().any(): + nan_cols = features_aligned_df.columns[features_aligned_df.isna().any()].tolist() + logging.warning(f"NaN values detected in final aligned feature columns: {nan_cols}. Dropping rows.") + features_aligned_df = features_aligned_df.dropna() + # Realign target and start price + common_index = features_aligned_df.index + target_aligned_ser = target_aligned_ser.loc[common_index] + start_price_aligned_ser = start_price_aligned_ser.loc[common_index] + + # --- Final Shape Check --- + if not (len(features_aligned_df) == len(target_aligned_ser) == len(start_price_aligned_ser)): + logging.error(f"Final shape mismatch after NaN handling: Features={len(features_aligned_df)}, Target={len(target_aligned_ser)}, StartPrice={len(start_price_aligned_ser)}") + return None, None, None + if features_aligned_df.empty: + logging.error("Preprocessing resulted in empty features DataFrame after alignment.") + return None, None, None + + logging.info(f"Preprocessing complete. Features: {features_aligned_df.shape}, Target: {target_aligned_ser.shape}, Start Price: {start_price_aligned_ser.shape}") + return features_aligned_df, target_aligned_ser, start_price_aligned_ser + + def generate_trading_experiences(self, val_data, transaction_cost=0.00001, prediction_horizon=1): + """ + V7-V6 Update: Use predicted return and uncertainty sigma for SAC state. + V7 Efficiency Update: Pre-compute GRU predictions/uncertainty for the whole validation set. + """ + print(f"Generating trading experiences for SAC agent on {len(val_data)} validation data points...") + + # --- Pre-computation Step --- + if self.gru_model is None or not (self.gru_model.is_trained or self.gru_model.is_loaded): + logging.error("Cannot generate experiences: GRU model not ready.") + return [] + if self.feature_scaler is None or self.y_scaler is None: + logging.error("Cannot generate experiences: Scalers not loaded/trained.") + return [] + + logging.info("Preprocessing validation data for experience generation...") + val_features_df, val_target_price_ser, val_start_price_ser = self._preprocess_data_for_gru_training(val_data, prediction_horizon) + if val_features_df is None: + logging.error("Failed to preprocess validation data for experiences.") + return [] + + logging.info("Scaling validation features and targets...") + try: + val_features_scaled = self.feature_scaler.transform(val_features_df.select_dtypes(include=np.number).fillna(0)) + val_target_scaled = self.y_scaler.transform(val_target_price_ser.fillna(0).values.reshape(-1, 1)) + except Exception as e: + logging.error(f"Error scaling validation data for experiences: {e}", exc_info=True) + return [] + + logging.info("Creating validation sequences...") + try: + val_start_price_aligned_ser = val_start_price_ser.loc[val_features_df.index] # Align before sequencing + X_val, y_val_scaled_seq, y_start_price_val_seq = create_sequences_v2( + pd.DataFrame(val_features_scaled, index=val_features_df.index), + pd.Series(val_target_scaled.flatten(), index=val_features_df.index), + val_start_price_aligned_ser, + self.gru_lookback + ) + if X_val is None or y_val_scaled_seq is None or y_start_price_val_seq is None: + logging.error("Sequence creation failed for validation data.") + return [] + except Exception as e: + logging.error(f"Error creating validation sequences: {e}", exc_info=True) + return [] + + logging.info(f"Pre-computing GRU predictions/uncertainty for {len(X_val)} validation sequences...") + try: + eval_results = self.gru_model.evaluate(X_val, y_val_scaled_seq, y_start_price_val_seq, n_mc_samples=30) + if eval_results is None: + logging.error("GRU evaluate failed on validation sequences.") + return [] + all_pred_returns = eval_results['pred_percent_change'] + all_uncertainties = eval_results['mc_unscaled_std_dev'] + except Exception as e: + logging.error(f"Error during pre-computation evaluate call: {e}", exc_info=True) + return [] + + # --- V7.15 START: Extract Momentum and Volatility for Experience Generation --- + # Similar to backtest logic, extract features aligned with sequences + num_sequences = len(all_pred_returns) # Use num_sequences from evaluate results + all_momentum_5 = np.zeros(num_sequences) + all_volatility_20 = np.zeros(num_sequences) + try: + # Check required columns in the *unscaled* preprocessed features df + required_state_cols = ['return_5m', 'volatility_14d'] + if not all(col in val_features_df.columns for col in required_state_cols): + missing_cols = [col for col in required_state_cols if col not in val_features_df.columns] + logging.error(f"Missing required state columns in val_features_df: {missing_cols}") + return [] + + # Align features with the sequences + if len(val_features_df) >= self.gru_lookback - 1 + num_sequences: + aligned_feature_indices = val_features_df.index[self.gru_lookback - 1 : self.gru_lookback - 1 + num_sequences] + aligned_features_for_state = val_features_df.loc[aligned_feature_indices] + + all_momentum_5 = aligned_features_for_state['return_5m'].values + all_volatility_20 = aligned_features_for_state['volatility_14d'].values + + if len(all_momentum_5) != num_sequences or len(all_volatility_20) != num_sequences: + logging.error(f"Exp Gen: Length mismatch extracting state features: Mom5({len(all_momentum_5)}), Vol20({len(all_volatility_20)}) vs NumSeq({num_sequences})") + return [] + else: + logging.error("Exp Gen: Length mismatch: val_features_df too short to extract aligned state features.") + return [] + except KeyError as e: + logging.error(f"Exp Gen: KeyError extracting state features: {e}") + return [] + except Exception as e: + logging.error(f"Exp Gen: Error extracting state features: {e}", exc_info=True) + return [] + # --- V7.15 END: Extract Momentum and Volatility --- + + logging.info("GRU pre-computation finished. Generating experiences loop...") + # --- End Pre-computation --- + + experiences = [] + current_position = 0.0 # Position *before* taking action at step i + + # Loop through the *results* (length = num_sequences) + # num_sequences = len(all_pred_returns) # Already defined + if num_sequences <= 1: + logging.warning("Not enough sequences generated from validation data to create experiences.") + return [] + + # Need original close prices aligned with the sequences for reward calculation + # The i-th sequence corresponds to the original data ending at index i + gru_lookback - 1 + # The prediction is for the step *after* this sequence + # The relevant close prices for reward are at original indices [i + gru_lookback] and [i + gru_lookback + 1] + original_close_prices = val_data['close'] # Use original validation data + + for i in tqdm(range(num_sequences - 1), desc="Generating Experiences"): # Iterate up to second-to-last sequence result + # --- V7.15 START: Construct 5D state s_t --- + pred_return_t = all_pred_returns[i] + uncertainty_t = all_uncertainties[i] + momentum_5_t = all_momentum_5[i] + volatility_20_t = all_volatility_20[i] + # Calculate z_proxy using position *before* action (current_position) + z_proxy_t = current_position * volatility_20_t + state = np.array([pred_return_t, uncertainty_t, z_proxy_t, momentum_5_t, volatility_20_t], dtype=np.float32) + # Handle potential NaNs/Infs + if np.any(np.isnan(state)) or np.any(np.isinf(state)): + logging.warning(f"NaN/Inf in state at step {i}. Replacing with 0. State: {state}") + state = np.nan_to_num(state, nan=0.0, posinf=0.0, neginf=0.0) + # --- V7.15 END: Construct 5D state s_t --- + + # Get action a_t (unpack action and log_prob) + action_tuple = self.sac_agent.get_action(state, deterministic=False) + if isinstance(action_tuple, (tuple, list)) and len(action_tuple) > 0: + action = action_tuple[0] # Action is the first element + # Ensure action is scalar if action_dim is 1 + if self.sac_agent.action_dim == 1 and isinstance(action, (np.ndarray, list)): + action = action[0] + else: + logging.error(f"SAC agent get_action did not return expected tuple at step {i}. Got: {action_tuple}. Skipping experience.") + continue # Skip this experience if action format is wrong + + # --- V7.15 START: Construct 5D next_state s_{t+1} --- + pred_return_t1 = all_pred_returns[i+1] + uncertainty_t1 = all_uncertainties[i+1] + momentum_5_t1 = all_momentum_5[i+1] + volatility_20_t1 = all_volatility_20[i+1] + # Calculate z_proxy using the *action* taken (action is position for next step) + z_proxy_t1 = action * volatility_20_t1 # Use action, not current_position + next_state = np.array([pred_return_t1, uncertainty_t1, z_proxy_t1, momentum_5_t1, volatility_20_t1], dtype=np.float32) + # Handle potential NaNs/Infs + if np.any(np.isnan(next_state)) or np.any(np.isinf(next_state)): + logging.warning(f"NaN/Inf in next_state at step {i}. Replacing with 0. State: {next_state}") + next_state = np.nan_to_num(next_state, nan=0.0, posinf=0.0, neginf=0.0) + # --- V7.15 END: Construct 5D next_state s_{t+1} --- + + # Calculate actual return for reward r_t + # Map sequence index 'i' back to the original dataframe index + original_idx_t = i + self.gru_lookback # Index of the *last* element of the sequence i + original_idx_t_plus_1 = original_idx_t + 1 # Index for the next closing price + + try: + # Use original_close_prices Series which should have the same index as val_data + close_t = original_close_prices.iloc[original_idx_t] + close_t1 = original_close_prices.iloc[original_idx_t_plus_1] + + if close_t != 0: + actual_return = (close_t1 / close_t) - 1.0 + else: + actual_return = 0.0 # Assign 0 return if start price is zero + except IndexError: + logging.warning(f"IndexError accessing original close prices for reward at step {i} (original indices {original_idx_t}, {original_idx_t_plus_1}). Skipping experience.") + continue + except Exception as e: + logging.error(f"Error accessing original close prices for reward at step {i}: {e}") + continue + + # Reward calculation (unchanged, uses action) + reward = action * actual_return - transaction_cost * abs(action - current_position) + + # Done flag (only True for the very last possible transition) + done = (i == num_sequences - 2) + # Store the experience (state, action, reward, next_state, done) + experiences.append((state, [action], reward, next_state, float(done))) # Ensure action is in a list for consistency if needed by buffer + + # Update current_position for the *next* iteration's z_proxy_t calculation + current_position = action + + print(f"Generated {len(experiences)} experiences efficiently.") + return experiences + + def train_sac(self, val_data, epochs=100, batch_size=256, transaction_cost=0.00001, + generate_new_experiences_on_epoch=False, # Default is now False + prediction_horizon=1): # Need prediction horizon for experience gen + """ + Train SAC agent using experiences generated from historical data. + (Code largely unchanged, relies on updated generate_trading_experiences) + """ + print(f"Starting SAC training for {epochs} epochs...") + if not generate_new_experiences_on_epoch: + print("Generating initial experiences..."); experiences = self.generate_trading_experiences(val_data, transaction_cost) + print(f"Adding {len(experiences)} experiences to replay buffer...") + for state, action, reward, next_state, done in tqdm(experiences, desc="Filling Buffer"): + action_to_add = action[0] if isinstance(action, (list, np.ndarray)) and len(action) == 1 else action + # V7.16 Fix: Use agent's store_transition method + # self.sac_agent.buffer.add(state, action_to_add, reward, next_state, done) + self.sac_agent.store_transition(state, action_to_add, reward, next_state, done) + print("Initial buffer fill complete.") + metrics_history = [] + for epoch in tqdm(range(epochs), desc="SAC Training Epochs"): + if generate_new_experiences_on_epoch: + experiences = self.generate_trading_experiences(val_data, transaction_cost, prediction_horizon) # Pass prediction_horizon + for state, action, reward, next_state, done in experiences: + action_to_add = action[0] if isinstance(action, (list, np.ndarray)) and len(action) == 1 else action + self.sac_agent.buffer.add(state, action_to_add, reward, next_state, done) + # V7.2 Revert: Use SAC agent default min buffer size + # V7.17 Fix: Check buffer size using agent's buffer_counter + # if len(self.sac_agent.buffer) >= self.sac_agent.min_buffer_size if hasattr(self.sac_agent, 'min_buffer_size') else batch_size: + if self.sac_agent.buffer_counter >= (self.sac_agent.min_buffer_size if hasattr(self.sac_agent, 'min_buffer_size') else batch_size): + # V7.17 Fix: train returns tuple (actor_loss, critic_loss) or None + loss_tuple = self.sac_agent.train(batch_size) + if loss_tuple: + actor_loss, critic_loss = loss_tuple + metrics = {'actor_loss': actor_loss, 'critic_loss': critic_loss} + metrics_history.append(metrics) + # Print metrics every 10 epochs instead of 100 + if metrics and epoch % 10 == 0: print(f"Epoch {epoch}/{epochs}: {metrics}") + else: + # Handle case where train() returns None (e.g., not enough data yet) + pass # Or log a warning if needed + elif epoch == 0: + min_buff = self.sac_agent.min_buffer_size if hasattr(self.sac_agent, 'min_buffer_size') else batch_size + # V7.16 Fix: Use buffer_counter in log message + print(f"Epoch {epoch}: Buffer size ({self.sac_agent.buffer_counter}) < min ({min_buff}). Skipping.") + print("SAC training complete."); return metrics_history + + def backtest_simple(self, data, transaction_cost=0.00001): + """ + V7-V6 Update: Use predicted return and confidence for SAC state. + """ + print(f"Starting simple backtest on {len(data)} data points...") + portfolio_value = 1.0; current_position = 0.0 + portfolio_values = [portfolio_value]; positions_history = [current_position] + predictions = []; uncertainties = [] # Store pred_return and confidence + + for i in tqdm(range(len(data) - 1), desc="Simple Backtest"): + # V7-V6 Update: Use _extract_features_and_predict + _, pred_return, uncertainty_sigma = self._extract_features_and_predict(data, i) + # V7-V6 Sigma Update: Check uncertainty_sigma + if pred_return is None or uncertainty_sigma is None: + continue + + predictions.append(pred_return) + # V7-V6 Sigma Update: Store uncertainty_sigma + uncertainties.append(uncertainty_sigma) + + # State for SAC: [predicted_return, mc_unscaled_std_dev] + # V7-V6 Sigma Update: Use uncertainty_sigma + state = np.array([pred_return, uncertainty_sigma], dtype=np.float32) + + action = self.sac_agent.get_action(state, deterministic=True)[0] + + tx_cost_fraction = transaction_cost * abs(action - current_position) + + # Use actual close prices for backtest return calc + if data.iloc[i]['close'] != 0: + price_return = (data.iloc[i+1]['close'] / data.iloc[i]['close']) - 1.0 + else: + price_return = 0 + + position_return = current_position * price_return + portfolio_value *= (1.0 + position_return) * (1.0 - tx_cost_fraction) + positions_history.append(action); portfolio_values.append(portfolio_value) + current_position = action + + # Calculate metrics + portfolio_values = np.array(portfolio_values) + returns = np.diff(portfolio_values) / portfolio_values[:-1] # Calculate portfolio returns + returns = np.nan_to_num(returns) + + sharpe_ratio = 0.0 + if np.std(returns) > 1e-9: + # Assume daily data for annualization? Needs correct frequency. + # Let's assume minutes, T = minutes in a year + T = 365 * 24 * 60 + sharpe_ratio = (np.mean(returns) / np.std(returns)) * np.sqrt(T) + + max_drawdown = self._calculate_max_drawdown(portfolio_values) + results = { + 'final_value': portfolio_values[-1], + 'total_return': portfolio_values[-1] - 1.0, + 'sharpe_ratio': sharpe_ratio, + 'max_drawdown': max_drawdown, + 'positions': positions_history, + 'portfolio_values': portfolio_values, + 'predictions': predictions, # Stored predicted returns + 'uncertainties': uncertainties # Stored confidence scores + } + print("Simple backtest complete.") + print(f" Final Value: {results['final_value']:.4f}, Sharpe: {results['sharpe_ratio']:.4f}, Max Drawdown: {results['max_drawdown']:.4f}") + return results + + def _calculate_max_drawdown(self, portfolio_values): + """Calculate maximum drawdown from portfolio values""" + if len(portfolio_values) < 2: + return 0.0 + values = np.array(portfolio_values) + running_max = np.maximum.accumulate(values) + running_max[running_max == 0] = 1.0 # Avoid division by zero + drawdown = (running_max - values) / running_max + return np.max(drawdown) + + def save(self, path): + """Save the integrated trading system (GRU model/scalers and SAC agent).""" + print(f"Saving trading system to {path}...") + os.makedirs(path, exist_ok=True) + gru_path = os.path.join(path, "gru_model") + sac_path = os.path.join(path, "sac_agent") + os.makedirs(gru_path, exist_ok=True) + os.makedirs(sac_path, exist_ok=True) + + if self.gru_model: + # V7-V6 Update: Use CryptoGRUModel save + self.gru_model.save(gru_path) # CryptoGRUModel.save handles model+scalers + if self.sac_agent: + self.sac_agent.save(sac_path) + print("Trading system saved.") + + def load(self, path): + """Load the integrated trading system (GRU model/scalers and SAC agent).""" + print(f"Loading trading system from {path}...") + gru_path = os.path.join(path, "gru_model") + sac_path = os.path.join(path, "sac_agent") + models_loaded = True + try: + if os.path.isdir(gru_path): + # V7-V6 Update: Instantiate and load CryptoGRUModel + self.gru_model = CryptoGRUModel() + if not self.gru_model.load(gru_path): print("Warning: Failed to load GRU model/scalers.") + # Store loaded scalers in TradingSystem as well for convenience? + self.feature_scaler = self.gru_model.feature_scaler + self.y_scaler = self.gru_model.y_scaler + else: print(f"Warning: GRU model directory not found: {gru_path}") + if os.path.isdir(sac_path): + # State dim is 2: [pred_return, confidence] + self.sac_agent = SimplifiedSACTradingAgent(state_dim=2) + self.sac_agent.load(sac_path) + else: print(f"Warning: SAC agent directory not found: {sac_path}") + print(f"Trading system loading {'successful' if models_loaded else 'failed (partially?) '}.") + except Exception as e: print(f"An error occurred during loading: {e}") + + # V7-V6 Update: Adapt GRU training call for price regression + def train_gru(self, train_data: pd.DataFrame, val_data: pd.DataFrame, + prediction_horizon: int, epochs=20, batch_size=32, + patience=10, # Revert to V6 defaults + model_save_dir='models/gru_predictor_trained'): + """V7 Adaptation: Train GRU model (V6 style - price prediction). """ + print(f"--- Starting GRU Training Pipeline (V6 Adaptation) ---") + print(f"Preprocessing training data (Target: Future Price, Horizon: {prediction_horizon})...") + # V7-V6 Update: Use renamed preprocessing function + train_features_df, train_target_price_ser, train_start_price_ser = self._preprocess_data_for_gru_training(train_data, prediction_horizon) + if train_features_df is None: + print("ERROR: Failed to preprocess training data for GRU.") + return None + + print(f"Preprocessing validation data (Target: Future Price, Horizon: {prediction_horizon})...") + # V7-V6 Update: Use renamed preprocessing function + val_features_df, val_target_price_ser, val_start_price_ser = self._preprocess_data_for_gru_training(val_data, prediction_horizon) + if val_features_df is None: + print("ERROR: Failed to preprocess validation data for GRU.") + return None + + # --- V7-V6 Update: Add Scaling Step --- + logging.info("Scaling features and target price...") + try: + # Feature Scaling (Use StandardScaler as in V6 preproc example, but make flexible) + # Consider making scaler type a parameter if needed + feature_scaler = StandardScaler() + # Ensure features are numeric and handle potential NaNs before scaling + train_features_numeric = train_features_df.select_dtypes(include=np.number).fillna(0) + val_features_numeric = val_features_df.select_dtypes(include=np.number).fillna(0) + + # Log columns used for scaling + logging.info(f"Feature columns for scaling: {train_features_numeric.columns.tolist()}") + + train_features_scaled = feature_scaler.fit_transform(train_features_numeric) + val_features_scaled = feature_scaler.transform(val_features_numeric) + self.feature_scaler = feature_scaler # Store fitted scaler + logging.info(f"Feature scaling complete. Scaler type: {type(feature_scaler)}") + + # Target Scaling (Use MinMaxScaler as in V6) + y_scaler = MinMaxScaler() # Default range (0, 1) is fine for prices + # Handle potential NaNs in target before scaling + train_target_numeric = train_target_price_ser.fillna(0).values.reshape(-1, 1) + val_target_numeric = val_target_price_ser.fillna(0).values.reshape(-1, 1) + + train_target_scaled = y_scaler.fit_transform(train_target_numeric) + val_target_scaled = y_scaler.transform(val_target_numeric) + self.y_scaler = y_scaler # Store fitted scaler + logging.info(f"Target price scaling complete. Scaler type: {type(y_scaler)}") + + except Exception as e: + logging.error(f"Error during scaling: {e}", exc_info=True) + return None # Corrected indentation + # --- End Scaling Step --- + + # --- V7-V6 Update: Add Sequence Creation Step --- + logging.info("Creating sequences...") + try: # Added missing except block below + # Pass scaled features as DataFrame to preserve column info if needed by create_sequences_v2? + # Or just pass the numpy array? Let's assume numpy is fine for V6 logic. + # Also need to pass the UNscaled start price series correctly aligned. + + # Align start price series with the scaled data length before sequencing + train_start_price_aligned_ser = train_start_price_ser.loc[train_features_df.index] # Index from before scaling + val_start_price_aligned_ser = val_start_price_ser.loc[val_features_df.index] + + X_train, y_train_scaled, y_start_price_train = create_sequences_v2( + pd.DataFrame(train_features_scaled, index=train_features_df.index), # Pass features with index + pd.Series(train_target_scaled.flatten(), index=train_features_df.index), # Pass target with index + train_start_price_aligned_ser, # Pass aligned start price Series + self.gru_lookback + ) + X_val, y_val_scaled, y_start_price_val = create_sequences_v2( + pd.DataFrame(val_features_scaled, index=val_features_df.index), + pd.Series(val_target_scaled.flatten(), index=val_features_df.index), + val_start_price_aligned_ser, + self.gru_lookback + ) + + if X_train is None or X_val is None or y_train_scaled is None or y_val_scaled is None: + logging.error("Sequence creation failed. Returned None.") + return None + + except Exception as e: # Added missing except block + logging.error(f"Error creating sequences: {e}", exc_info=True) # Corrected indentation + return None # Corrected indentation + # --- End Sequence Creation --- + + # Debug info about the final data shapes for training + print(f"Data ready for GRU training:") + print(f" X_train shape: {X_train.shape}") + print(f" y_train_scaled shape: {y_train_scaled.shape}") + print(f" X_val shape: {X_val.shape}") + print(f" y_val_scaled shape: {y_val_scaled.shape}") + + # Initialize GRU model if needed + if self.gru_model is None: + # V7-V6 Update: Use CryptoGRUModel + self.gru_model = CryptoGRUModel() + + # --- V7-V6 Update: Call CryptoGRUModel.train --- + print(f"Training GRU model (V6 Adaptation) for {epochs} epochs...") + history = self.gru_model.train( + X_train, y_train_scaled, # Pass sequences and scaled targets + X_val, y_val_scaled, + feature_scaler=self.feature_scaler, # Pass fitted scalers + y_scaler=self.y_scaler, + epochs=epochs, + batch_size=batch_size, + patience=patience, + model_save_dir=model_save_dir # Pass save dir + # Removed LR args, handled within model train + ) + # --- End Updated Call --- + + if history is not None: + print(f"GRU model training complete. Model and scalers saved to {model_save_dir}.") + # Store scalers in TradingSystem instance after successful training + self.feature_scaler = self.gru_model.feature_scaler + self.y_scaler = self.gru_model.y_scaler + + # Optional: Plot training history + try: + history_plot_path = os.path.join(model_save_dir, 'gru_training_history.png') + self.gru_model.plot_training_history(history, save_path=history_plot_path) + except Exception as plot_e: + logging.warning(f"Could not plot GRU training history: {plot_e}") + + return history + else: + print("GRU model training failed.") + return None + + +class ExtendedBacktester: + """ + Enhanced backtesting framework for GRU+SAC integration. + V7 Efficiency Update: Pre-computes GRU outputs for faster backtesting. + """ + def __init__(self, trading_system: TradingSystem, initial_capital=10000.0, transaction_cost=0.0001, instrument_label="Unknown Instrument"): + print("Initializing ExtendedBacktester...") + self.trading_system = trading_system + self.initial_capital = initial_capital + self.transaction_cost = transaction_cost + self.instrument_label = instrument_label # Store instrument label + self.portfolio_values = [] + self.positions = [] + self.predictions = [] # Will store predicted returns used for state + self.uncertainties_used_in_state = [] # Will store uncertainty sigmas used for state + self.all_precomputed_uncertainties = None # Stores the full array from evaluate() + self.actions = [] # Actions taken by SAC + self.actual_returns = [] # Actual price returns observed + self.trade_counts = 0 + self.trade_history = [] + self.timestamps = [] # Timestamps for portfolio/action logging + self.gru_lookback = self.trading_system.gru_lookback # Get from TradingSystem + self.buy_hold_values = None # Initialize B&H value storage + # Add storage for GRU prediction plot data + self.prediction_timestamps = None + self.predicted_prices = None + self.true_prices_for_pred = None + # self.uncertainties already exists for uncertainty sigma + print("ExtendedBacktester initialized.") + + def backtest(self, test_data, verbose=True, prediction_horizon=1): + """V7 Efficiency Update: Pre-compute GRU outputs.""" + if not isinstance(test_data, pd.DataFrame) or 'close' not in test_data.columns: + raise ValueError("test_data must be a pandas DataFrame with a 'close' column.") + if self.trading_system.gru_model is None or not (self.trading_system.gru_model.is_trained or self.trading_system.gru_model.is_loaded): + raise ValueError("Cannot backtest: GRU model not ready.") + if self.trading_system.feature_scaler is None or self.trading_system.y_scaler is None: + raise ValueError("Cannot backtest: Scalers not loaded/trained.") + if not self.trading_system.sac_agent: + raise ValueError("Cannot backtest: SAC Agent not initialized.") + + print("--- Pre-computing GRU outputs for Test Set ---") + + # 1. Preprocess test data (Features, Target Price, Start Price) + feature_logger.info("Preprocessing test data...") # Use feature_logger + test_features_df, test_target_price_ser, test_start_price_ser = self.trading_system._preprocess_data_for_gru_training(test_data, prediction_horizon) + if test_features_df is None: + feature_logger.error("Failed to preprocess test data.") # Use feature_logger + return None + + # 2. Scale Features and Target + feature_logger.info("Scaling test features and targets...") # Use feature_logger + try: + test_features_scaled = self.trading_system.feature_scaler.transform(test_features_df.select_dtypes(include=np.number).fillna(0)) + test_target_scaled = self.trading_system.y_scaler.transform(test_target_price_ser.fillna(0).values.reshape(-1, 1)) + except Exception as e: + feature_logger.error(f"Error scaling test data: {e}", exc_info=True) # Use feature_logger + return None + + # 3. Create Sequences + feature_logger.info("Creating test sequences...") # Use feature_logger + try: + test_start_price_aligned_ser = test_start_price_ser.loc[test_features_df.index] # Align before sequencing + X_test, y_test_scaled_seq, y_start_price_test_seq = create_sequences_v2( + pd.DataFrame(test_features_scaled, index=test_features_df.index), + pd.Series(test_target_scaled.flatten(), index=test_features_df.index), + test_start_price_aligned_ser, + self.gru_lookback + ) + if X_test is None or y_test_scaled_seq is None or y_start_price_test_seq is None: + feature_logger.error("Sequence creation failed for test data.") # Use feature_logger + return None + except Exception as e: + feature_logger.error(f"Error creating test sequences: {e}", exc_info=True) # Use feature_logger + return None + + # 4. Run GRU Evaluate Once + feature_logger.info(f"Pre-computing GRU predictions/uncertainty for {len(X_test)} test sequences...") # Use feature_logger + try: + eval_results = self.trading_system.gru_model.evaluate(X_test, y_test_scaled_seq, y_start_price_test_seq, n_mc_samples=30) + if eval_results is None: + feature_logger.error("GRU evaluate failed on test sequences.") # Use feature_logger + return None + all_pred_returns = eval_results['pred_percent_change'] + all_uncertainties = eval_results['mc_unscaled_std_dev'] + # Store data needed for GRU prediction plot + self.predicted_prices = eval_results.get('predicted_unscaled_prices') + self.true_prices_for_pred = eval_results.get('true_unscaled_prices') + # V7 Fix: Store full uncertainty array separately + self.all_precomputed_uncertainties = all_uncertainties # Store full array for plotting + # self.uncertainties will store the ones used in the loop state below + self.uncertainties_used_in_state = [] # Initialize loop list + + # Store the corresponding timestamps + # The prediction results align with the *end* of the sequences + # The index of test_start_price_aligned_ser aligns with the start of the window + # So, the timestamp for the k-th prediction is at index k + gru_lookback - 1 + num_sequences_eval = len(eval_results['predicted_unscaled_prices']) # Get length from results + if len(test_start_price_aligned_ser) >= self.gru_lookback - 1 + num_sequences_eval: + prediction_indices = test_start_price_aligned_ser.index[self.gru_lookback - 1 : self.gru_lookback - 1 + num_sequences_eval] + self.prediction_timestamps = prediction_indices.to_list() + else: + feature_logger.error("Cannot extract prediction timestamps due to index length mismatch.") + self.prediction_timestamps = [] # Assign empty list on error + + # Validate stored prediction data + if self.predicted_prices is None or self.true_prices_for_pred is None or self.uncertainties_used_in_state is None: + feature_logger.error("Missing price/uncertainty data from GRU evaluate results.") + return None + if len(self.prediction_timestamps) != len(self.predicted_prices): + feature_logger.error("Prediction timestamp length mismatch.") + return None + + except Exception as e: + feature_logger.error(f"Error during pre-computation evaluate call on test data: {e}", exc_info=True) # Use feature_logger + return None + + # --- V7.13 START: Extract Momentum and Volatility Features --- + # These features should be aligned with the *sequences* used for GRU prediction + # Use test_features_df which is aligned with the sequences' start times + try: + # Check if required columns exist in the preprocessed features + required_state_cols = ['return_5m', 'volatility_14d'] # Example columns used in state + if not all(col in test_features_df.columns for col in required_state_cols): + missing_cols = [col for col in required_state_cols if col not in test_features_df.columns] + feature_logger.error(f"Missing required state columns in test_features_df: {missing_cols}") + return None + + # Select the features corresponding to the *end* of each sequence + # The evaluate results (all_pred_returns) align with the *output* of the sequences + # We need features from the *last timestep* of the input sequence (index k + gru_lookback - 1) + # Slice test_features_df to match the number of sequences evaluated + if len(test_features_df) >= self.gru_lookback - 1 + num_sequences: + aligned_feature_indices = test_features_df.index[self.gru_lookback - 1 : self.gru_lookback - 1 + num_sequences] + aligned_features_for_state = test_features_df.loc[aligned_feature_indices] + + # V7.14 Use return_5m for momentum_5, volatility_14d for volatility_20 (adapt names if needed) + all_momentum_5 = aligned_features_for_state['return_5m'].values + all_volatility_20 = aligned_features_for_state['volatility_14d'].values # Assuming 'volatility_14d' is the correct column + + if len(all_momentum_5) != num_sequences or len(all_volatility_20) != num_sequences: + feature_logger.error(f"Length mismatch extracting state features: Mom5({len(all_momentum_5)}), Vol20({len(all_volatility_20)}) vs NumSeq({num_sequences})") + return None + else: + feature_logger.error("Length mismatch: test_features_df too short to extract aligned state features.") + return None + + except KeyError as e: + feature_logger.error(f"KeyError extracting state features (momentum/volatility): {e}") + return None + except Exception as e: + feature_logger.error(f"Error extracting state features (momentum/volatility): {e}", exc_info=True) + return None + # --- V7.13 END: Extract Momentum and Volatility Features --- + + num_sequences = len(all_pred_returns) + if num_sequences <= 1: + feature_logger.warning("Not enough sequences generated from test data to perform backtest.") # Use feature_logger + return None + + feature_logger.info("--- GRU Pre-computation Complete. Starting Backtest Simulation ---") # Use feature_logger + + # --- Refined Timestamp Handling START (from original test_data before preprocessing) --- + time_col_name = None + initial_timestamp_for_loop = None + + # Use a fresh copy for timestamp handling to avoid modifying the original input df + test_data_copy = test_data.copy() + + if 'timestamp' in test_data_copy.columns: + time_col_name = 'timestamp' + test_data_copy[time_col_name] = pd.to_datetime(test_data_copy[time_col_name]) + # Get timestamp corresponding to the *end* of the first sequence + if len(test_data_copy) > self.gru_lookback: + initial_timestamp_for_loop = test_data_copy.iloc[self.gru_lookback][time_col_name] + else: + raise ValueError("Test data too short for lookback after timestamp check.") + print(f"Using '{time_col_name}' column for time.") + + elif isinstance(test_data_copy.index, pd.DatetimeIndex): + feature_logger.info("Using DatetimeIndex for time.") # Changed from print + original_index_name = test_data_copy.index.name + if len(test_data_copy) > self.gru_lookback: + initial_timestamp_for_loop = test_data_copy.index[self.gru_lookback] + else: + raise ValueError("Test data index too short for lookback after timestamp check.") + + test_data_copy = test_data_copy.reset_index() + + # Find the name of the column created from the index + if original_index_name and original_index_name in test_data_copy.columns: + time_col_name = original_index_name + elif 'index' in test_data_copy.columns and pd.api.types.is_datetime64_any_dtype(test_data_copy['index']): + time_col_name = 'index' + else: + found_col = None + for col in test_data_copy.columns: + if pd.api.types.is_datetime64_any_dtype(test_data_copy[col]): + found_col = col; break + if found_col: time_col_name = found_col + else: raise ValueError("Could not identify the reset index column containing timestamps.") + print(f"Identified reset index time column as: '{time_col_name}'") + else: + raise ValueError("Could not find 'timestamp' column or DatetimeIndex.") + # --- Refined Timestamp Handling END --- + + # --- Backtest Loop Initialization --- + self.portfolio_values = [self.initial_capital] + self.positions = [0.0] # Position held *before* action at step i + self.predictions = [] + self.uncertainties_used_in_state = [] + self.actions = [] + self.actual_returns = [] + self.trade_history = [] + self.trade_counts = 0 + # Initialize timestamp list with the timestamp for the END of the first sequence period + self.timestamps = [initial_timestamp_for_loop] + + portfolio_value = self.initial_capital + current_position = 0.0 # Position held *before* taking action at step i + + # --- Main Backtest Loop (Iterates over pre-computed results) --- + # Use original close prices from the time-handled copy + original_close_prices = test_data_copy['close'] + + for i in tqdm(range(num_sequences - 1), desc="Backtest Simulation", disable=not verbose): + # 1. Get state from pre-computed results + pred_return = all_pred_returns[i] + uncertainty_sigma = all_uncertainties[i] + # V7.13 Get momentum and volatility for the current step + momentum_5 = all_momentum_5[i] + volatility_20 = all_volatility_20[i] + + # V7.13 Calculate z-proxy (Position as proxy for risk aversion) + # Use position *before* action, scaled by volatility + z_proxy = current_position * volatility_20 # Simpler proxy for now + + # V7.13 Construct 5D state: [pred_return, uncertainty, z_proxy, momentum_5, volatility_20] + state = np.array([ + pred_return, + uncertainty_sigma, + z_proxy, + momentum_5, + volatility_20 + ], dtype=np.float32) + + # V7.13 Handle NaNs/Infs in state (replace with 0 for simplicity) + if np.any(np.isnan(state)) or np.any(np.isinf(state)): + feature_logger.warning(f"NaN/Inf detected in state at step {i}. Replacing with 0. State: {state}") + state = np.nan_to_num(state, nan=0.0, posinf=0.0, neginf=0.0) + + # 2. Get deterministic action from SAC agent + # V7.13: get_action now returns (action, log_prob), unpack needed + action_tuple = self.trading_system.sac_agent.get_action(state, deterministic=True) + # V7.14 Ensure unpacking handles potential tuple vs single value return if agent API changes + if isinstance(action_tuple, (tuple, list)) and len(action_tuple) > 0: + raw_action = action_tuple[0] # Take the first element (the action) + else: + # Fallback if the API returns just the action (for robustness) + raw_action = action_tuple + feature_logger.warning("SAC agent get_action did not return a tuple as expected. Assuming single value is action.") + + # V7.14 Ensure raw_action is a scalar if action_dim is 1 + if self.trading_system.sac_agent.action_dim == 1 and isinstance(raw_action, (np.ndarray, list)): + raw_action = raw_action[0] + + # 3. Calculate PnL and portfolio value + position_change = raw_action - current_position + + # Map sequence index 'i' back to original dataframe index for price lookup + # Use the index from the time-handled copy (test_data_copy) + original_idx_t = i + self.gru_lookback # End of sequence i / Start of action period + original_idx_t_plus_1 = original_idx_t + 1 # End of action period + + try: + # Use prices from the time-handled copy + current_price = original_close_prices.iloc[original_idx_t] + next_price = original_close_prices.iloc[original_idx_t_plus_1] + if current_price != 0: + price_return = (next_price / current_price) - 1.0 + else: + price_return = 0.0 + except IndexError: + feature_logger.warning(f"IndexError accessing original close prices for PnL at step {i} (original indices {original_idx_t}, {original_idx_t_plus_1}). Ending backtest early.") + break + except Exception as e: + feature_logger.error(f"Error accessing original close prices for PnL at step {i}: {e}") + break + + tx_cost = abs(position_change) * portfolio_value * self.transaction_cost + # PnL based on position held *during* the period (current_position) + position_pnl = current_position * price_return * portfolio_value + new_portfolio_value = portfolio_value + position_pnl - tx_cost + + # 4. Store results & update state for next iteration + self.predictions.append(pred_return) + # V7 Fix: Append uncertainty used in state to the correct list + self.uncertainties_used_in_state.append(uncertainty_sigma) + self.actions.append(raw_action) + self.actual_returns.append(price_return) + self.portfolio_values.append(new_portfolio_value) + self.positions.append(raw_action) # Store position *after* action + try: + # Use .loc with the identified time_col_name from the time-handled copy + self.timestamps.append(test_data_copy.loc[original_idx_t_plus_1, time_col_name]) + except KeyError: + feature_logger.warning(f"Could not find timestamp for index {original_idx_t_plus_1}. Appending last known timestamp.") + self.timestamps.append(self.timestamps[-1]) + + if abs(position_change) > 0.01: + trade_timestamp = test_data_copy.loc[original_idx_t, time_col_name] # Timestamp when action is decided + trade = {'timestamp': trade_timestamp, 'price': current_price, 'old_position': current_position, + 'new_position': raw_action, 'prediction': pred_return, 'uncertainty': uncertainty_sigma, + 'portfolio_value_before': portfolio_value, 'portfolio_value_after': new_portfolio_value, 'transaction_cost': tx_cost} + self.trade_history.append(trade) + self.trade_counts += 1 + + portfolio_value = new_portfolio_value + current_position = raw_action # Position for the *next* period + + if portfolio_value <= 0: + print(f"Portfolio zeroed at step {i}. Stopping.") + # Adjust fill length based on remaining sequences + fill_len = (num_sequences - 1) - (i + 1) + self.portfolio_values.extend([0.0] * fill_len) + self.positions.extend([current_position] * fill_len) # Fill with last position + self.predictions.extend([0.0] * fill_len) + # V7 Fix: Extend the correct list + self.uncertainties_used_in_state.extend([0.0] * fill_len) + self.actions.extend([current_position] * fill_len) + self.actual_returns.extend([0.0] * fill_len) + # Extend timestamps carefully using the time-handled copy + last_valid_orig_idx = original_idx_t_plus_1 + if last_valid_orig_idx + fill_len <= len(test_data_copy) - 1: + remaining_timestamps = test_data_copy.loc[last_valid_orig_idx + 1 : last_valid_orig_idx + fill_len, time_col_name].tolist() + self.timestamps.extend(remaining_timestamps) + else: + # If fill_len exceeds available data, add NaT or last timestamp + available_count = len(test_data_copy) - 1 - last_valid_orig_idx + if available_count > 0: + remaining_timestamps = test_data_copy.loc[last_valid_orig_idx + 1 : last_valid_orig_idx + available_count, time_col_name].tolist() + self.timestamps.extend(remaining_timestamps) + if fill_len - available_count > 0: + self.timestamps.extend([pd.NaT] * (fill_len - available_count)) + break # Corrected indentation again + # --- End Backtest Loop --- + + # --- Calculate Buy and Hold Benchmark --- + feature_logger.info("Calculating Buy and Hold benchmark...") + try: + # Use the original test_data before reset_index if applicable + start_price_bh = test_data['close'].iloc[0] + # Use the close price corresponding to the *last timestamp* used in the strategy backtest + # The last timestamp added was for original_idx_t_plus_1 where i = num_sequences - 2 + last_strategy_idx = (num_sequences - 2) + self.gru_lookback + 1 + end_price_bh = test_data_copy['close'].iloc[last_strategy_idx] + + if pd.isna(start_price_bh) or start_price_bh <= 1e-9: + feature_logger.warning("Could not calculate Buy & Hold: Invalid start price.") + self.buy_hold_values = None + else: + initial_assets_bh = self.initial_capital / start_price_bh + # Calculate B&H value for the same period as the strategy + # Use close prices from the time-handled copy aligned with strategy timestamps + relevant_close_prices = test_data_copy['close'].iloc[self.gru_lookback : last_strategy_idx + 1] + self.buy_hold_values = initial_assets_bh * relevant_close_prices + # Ensure the length matches the strategy portfolio values + if len(self.buy_hold_values) != len(self.portfolio_values): + feature_logger.warning(f"Buy & Hold length mismatch ({len(self.buy_hold_values)}) vs Strategy ({len(self.portfolio_values)}). Adjusting B&H series.") + # Pad or truncate B&H to match strategy length (more robust approach needed if indices differ significantly) + bh_series = pd.Series(self.buy_hold_values, index=self.timestamps) # Use strategy timestamps + self.buy_hold_values = bh_series.reindex(self.timestamps).fillna(method='ffill').values + else: + self.buy_hold_values = self.buy_hold_values.values # Convert to numpy + + bh_final_value = self.buy_hold_values[-1] + feature_logger.info(f"Buy & Hold Final Value: ${bh_final_value:.2f}") + except Exception as e: + feature_logger.error(f"Error calculating Buy & Hold benchmark: {e}", exc_info=True) + self.buy_hold_values = None + # --- End Buy and Hold Calculation --- + + # Calculate & print metrics + metrics = self._calculate_performance_metrics() # Now calculates B&H metrics internally + metrics['trade_history'] = self.trade_history + metrics['timestamps'] = self.timestamps # Already populated + if verbose: + print("\n--- Extended Backtest Complete ---") + print(f"Final portfolio value: {metrics.get('final_value', 0):.2f}") + # ... (print other metrics) ... + print("---------------------------------") + return metrics + + def _calculate_performance_metrics(self): + if len(self.portfolio_values) < 2: + return {'final_value': 0.0, 'initial_value': 0.0, 'total_return': 0.0, 'annual_return': 0.0, 'sharpe_ratio': 0.0, 'sortino_ratio': 0.0, 'volatility': 0.0, 'max_drawdown': 0.0, 'avg_position': 0.0, 'position_accuracy': 0.0, 'pred_accuracy': 0.0, 'prediction_rmse': 0.0, 'pred_return_corr': 0.0, 'pred_pos_corr': 0.0, 'unc_pos_corr': 0.0, 'total_trades': 0} + portfolio_values = np.array(self.portfolio_values) + positions = np.array(self.positions[1:]) + predictions = np.array(self.predictions) + actual_returns = np.array(self.actual_returns) + # V7 Fix: Use the uncertainties actually used in the state loop + uncertainties_used = np.array(self.uncertainties_used_in_state) + safe_portfolio_values = portfolio_values[:-1].copy() + safe_portfolio_values[safe_portfolio_values == 0] = 1e-9 + period_returns = np.diff(portfolio_values) / safe_portfolio_values + total_minutes = len(period_returns) + minutes_per_year = 365 * 24 * 60 + if total_minutes == 0: + # Return default metrics if no periods + return {'final_value': 0.0, 'initial_value': 0.0, 'total_return': 0.0, 'annual_return': 0.0, 'sharpe_ratio': 0.0, 'sortino_ratio': 0.0, 'volatility': 0.0, 'max_drawdown': 0.0, 'avg_position': 0.0, 'position_accuracy': 0.0, 'pred_accuracy': 0.0, 'prediction_rmse': 0.0, 'pred_return_corr': 0.0, 'pred_pos_corr': 0.0, 'unc_pos_corr': 0.0, 'total_trades': 0} + + final_value = portfolio_values[-1] + initial_value = portfolio_values[0] + total_return = (final_value / initial_value) - 1.0 if initial_value != 0 else 0.0 + annual_return = ((1.0 + total_return) ** (minutes_per_year / total_minutes)) - 1.0 if total_return > -1 else -1.0 + volatility = np.std(period_returns) * np.sqrt(minutes_per_year) + mean_return = np.mean(period_returns) + std_return = np.std(period_returns) + sharpe_ratio = (mean_return / std_return) * np.sqrt(minutes_per_year) if std_return > 1e-9 else 0.0 + negative_returns = period_returns[period_returns < 0] + downside_std = np.std(negative_returns) if len(negative_returns) > 0 else 0.0 + sortino_ratio = (mean_return / downside_std) * np.sqrt(minutes_per_year) if downside_std > 1e-9 else 0.0 + max_drawdown = self.trading_system._calculate_max_drawdown(portfolio_values) + # V7 Fix: Use uncertainties_used here + common_length = min(len(predictions), len(actual_returns), len(positions), len(uncertainties_used)) + if common_length > 0: + aligned_predictions = predictions[:common_length] + aligned_actual_returns = actual_returns[:common_length] + aligned_positions = positions[:common_length] + # V7 Fix: Use uncertainties_used here + aligned_uncertainties = uncertainties_used[:common_length] + avg_position = np.mean(np.abs(aligned_positions)) + non_zero_mask = aligned_positions != 0 + # Use nan_to_num on signs before comparison for robustness + sign_pos = np.sign(np.nan_to_num(aligned_positions[non_zero_mask])) + sign_ret = np.sign(np.nan_to_num(aligned_actual_returns[non_zero_mask])) + position_accuracy = np.mean(sign_pos == sign_ret) if np.any(non_zero_mask) else 0.0 + pred_accuracy = np.mean(np.sign(aligned_predictions) == np.sign(aligned_actual_returns)) + prediction_rmse = np.sqrt(np.mean((aligned_predictions - aligned_actual_returns)**2)) + + # Robust Correlation Calculation with warnings suppressed: + with warnings.catch_warnings(): + # Ignore the specific RuntimeWarning from np.corrcoef with zero variance + warnings.filterwarnings('ignore', r'invalid value encountered in divide', RuntimeWarning) + + pred_safe = np.nan_to_num(aligned_predictions) + ret_safe = np.nan_to_num(aligned_actual_returns) + pos_safe = np.nan_to_num(aligned_positions) + unc_safe = np.nan_to_num(aligned_uncertainties) + abs_pos_safe = np.nan_to_num(np.abs(aligned_positions)) + + pred_return_corr = np.corrcoef(pred_safe, ret_safe)[0, 1] if np.std(pred_safe) > 1e-9 and np.std(ret_safe) > 1e-9 else 0.0 + pred_pos_corr = np.corrcoef(pred_safe, pos_safe)[0, 1] if np.std(pred_safe) > 1e-9 and np.std(pos_safe) > 1e-9 else 0.0 + unc_pos_corr = np.corrcoef(unc_safe, abs_pos_safe)[0, 1] if np.std(unc_safe) > 1e-9 and np.std(abs_pos_safe) > 1e-9 else 0.0 + + # Explicitly handle potential NaNs from corrcoef if std dev check wasn't perfect + if np.isnan(pred_return_corr): pred_return_corr = 0.0 + if np.isnan(pred_pos_corr): pred_pos_corr = 0.0 + if np.isnan(unc_pos_corr): unc_pos_corr = 0.0 + else: + avg_position, position_accuracy, pred_accuracy, prediction_rmse, pred_return_corr, pred_pos_corr, unc_pos_corr = (0.0,) * 7 + + metrics = { + 'final_value': final_value, 'initial_value': initial_value, 'total_return': total_return, + 'annual_return': annual_return, 'sharpe_ratio': sharpe_ratio, 'sortino_ratio': sortino_ratio, + 'volatility': volatility, 'max_drawdown': max_drawdown, 'avg_position': avg_position, + 'position_accuracy': position_accuracy, 'pred_accuracy': pred_accuracy, + 'prediction_rmse': prediction_rmse, 'pred_return_corr': pred_return_corr, + 'pred_pos_corr': pred_pos_corr, 'unc_pos_corr': unc_pos_corr, + 'total_trades': self.trade_counts + } + + # Add Buy & Hold metrics if available + if self.buy_hold_values is not None and len(self.buy_hold_values) > 0: + bh_final = self.buy_hold_values[-1] + bh_initial = self.buy_hold_values[0] # Should correspond to initial capital approx. + metrics['buy_hold_final_value'] = bh_final + metrics['buy_hold_total_return'] = (bh_final / self.initial_capital) - 1.0 if self.initial_capital > 0 else 0.0 + else: + metrics['buy_hold_final_value'] = None + metrics['buy_hold_total_return'] = None + + for key, value in metrics.items(): + # Avoid rounding None + if isinstance(value, (float, np.float_)): + metrics[key] = round(value, 6) + return metrics + + # V7 Plot Update: Rewrite plot_results for 3 specific subplots + def plot_results(self, save_path='backtest_results_v7.png', n_std_dev=1.5): + """Generate and save a 3-panel plot: GRU Preds, SAC Actions, Portfolio Perf.""" + if plt is None: + print("Matplotlib not installed. Skipping plotting.") + return None + + # --- Log Timestamp Range Used for Plotting --- + if self.timestamps: + feature_logger.info(f"Plotting Results - Timestamp range: {min(self.timestamps)} to {max(self.timestamps)}") + else: + feature_logger.warning("Plotting Results - Timestamps list is empty.") + + # --- Construct Title Suffix --- + title_suffix = f" ({self.instrument_label})" + if self.timestamps: + try: + start_ts_str = pd.to_datetime(min(self.timestamps)).strftime('%Y-%m-%d %H:%M') + end_ts_str = pd.to_datetime(max(self.timestamps)).strftime('%Y-%m-%d %H:%M') + title_suffix = f" ({self.instrument_label} | {start_ts_str} to {end_ts_str})" + except Exception as e: + feature_logger.warning(f"Could not format timestamps for plot title: {e}") + else: + feature_logger.warning("Timestamps missing for plot title.") + + # --- Data Validation for Plotting --- + valid_portfolio = self.portfolio_values is not None and len(self.portfolio_values) > 1 + valid_bh = self.buy_hold_values is not None and len(self.buy_hold_values) == len(self.portfolio_values) + valid_actions = self.actions is not None and len(self.actions) == len(self.portfolio_values) -1 # Actions map to periods + # V7 Fix: Check all_precomputed_uncertainties for plotting + valid_gru_preds = (self.predicted_prices is not None and + self.true_prices_for_pred is not None and + self.all_precomputed_uncertainties is not None and # Check the correct variable + self.prediction_timestamps is not None and + len(self.prediction_timestamps) == len(self.predicted_prices)) + + # Timestamps for portfolio/action plots (start from the first action's effect) + portfolio_time_axis = self.timestamps # Should align with portfolio_values + + # Timestamps for GRU predictions (align with the prediction time) + gru_time_axis = pd.to_datetime(self.prediction_timestamps) if valid_gru_preds else None + + # Determine the number of plots needed + n_plots = sum([valid_gru_preds, valid_actions, valid_portfolio]) + if n_plots == 0: + print("Warning: No valid data available for any plots.") + return None + + fig, axs = plt.subplots(n_plots, 1, figsize=(15, 6 * n_plots), sharex=True) + # Ensure axs is always an array, even if n_plots is 1 + if n_plots == 1: + axs = [axs] + plot_idx = 0 + + # --- Subplot 1: GRU Price Prediction vs Actual --- + if valid_gru_preds: + ax = axs[plot_idx] + # Align true prices with prediction timestamps + true_prices = pd.Series(self.true_prices_for_pred, index=gru_time_axis) + pred_prices = pd.Series(self.predicted_prices, index=gru_time_axis) + # V7 Fix: Use all_precomputed_uncertainties for plotting + if len(self.all_precomputed_uncertainties) != len(gru_time_axis): + print(f"Warning: Uncertainty length ({len(self.all_precomputed_uncertainties)}) doesn't match time axis ({len(gru_time_axis)}) for plotting.") + # Attempt to align or skip plotting uncertainty + uncertainty = None # Skip fill_between if lengths mismatch + else: + uncertainty = pd.Series(self.all_precomputed_uncertainties, index=gru_time_axis) + + ax.plot(gru_time_axis, true_prices, label='Actual Price', color='#2ca02c', alpha=0.8, linewidth=1.5) + ax.plot(gru_time_axis, pred_prices, label='Predicted Price (GRU)', color='#1f77b4', alpha=0.8, linewidth=1.5) + + # Only plot uncertainty band if uncertainty Series was created successfully + if uncertainty is not None: + lower_bound = pred_prices - n_std_dev * uncertainty + upper_bound = pred_prices + n_std_dev * uncertainty + ax.fill_between(gru_time_axis, lower_bound, upper_bound, + color='#aec7e8', alpha=0.3, + label=f'Prediction +/- {n_std_dev} Std Dev') + + ax.set_title('GRU Price Prediction vs Actual' + title_suffix) + ax.set_ylabel('Price') + ax.legend() + ax.grid(True) + ax.ticklabel_format(style='plain', axis='y') + plot_idx += 1 + else: + print("Skipping GRU Prediction plot due to missing data.") + + # --- Subplot 2: SAC Agent Actions --- + if valid_actions: + ax = axs[plot_idx] + # Align actions with the correct timestamps (action taken at T determines position for T to T+1) + action_time_axis = portfolio_time_axis[:-1] # Actions align with the start of the period + if len(action_time_axis) != len(self.actions): + feature_logger.warning(f"Action timestamp length mismatch ({len(action_time_axis)}) vs actions ({len(self.actions)}). Skipping SAC plot.") + else: + ax.plot(action_time_axis, self.actions, label='SAC Position Size', drawstyle='steps-post', color='#ff7f0e') + ax.set_title('SAC Agent Position Size (-1 to 1)' + title_suffix) + ax.set_ylabel('Position') + ax.set_ylim(-1.1, 1.1) + ax.grid(True) + ax.legend() + plot_idx += 1 + else: + print("Skipping SAC Action plot due to missing data.") + + # --- Subplot 3: Portfolio Performance vs Buy & Hold --- + if valid_portfolio: + ax = axs[plot_idx] + ax.plot(portfolio_time_axis, self.portfolio_values, label=f'Strategy Value: ${self.portfolio_values[-1]:,.2f}', color='blue') + + if valid_bh: + ax.plot(portfolio_time_axis, self.buy_hold_values, label=f'Buy & Hold Value: ${self.buy_hold_values[-1]:,.2f}', color='orange', linestyle='--') + elif self.buy_hold_values is not None: + feature_logger.warning("Could not plot Buy & Hold line due to length mismatch.") + + # Add trade markers + if self.trade_history: + buys = [(pd.to_datetime(t['timestamp']), t['price']) for t in self.trade_history if t['new_position'] > t['old_position']] + sells = [(pd.to_datetime(t['timestamp']), t['price']) for t in self.trade_history if t['new_position'] < t['old_position']] + if buys: + buy_times, buy_prices = zip(*buys) + # Find corresponding portfolio values at buy times for marker placement + portfolio_series = pd.Series(self.portfolio_values, index=pd.to_datetime(portfolio_time_axis)) + buy_portfolio_values = portfolio_series.reindex(buy_times, method='ffill') + ax.plot(buy_times, buy_portfolio_values, '^', color='green', markersize=8, label='Buy Signal') + if sells: + sell_times, sell_prices = zip(*sells) + portfolio_series = pd.Series(self.portfolio_values, index=pd.to_datetime(portfolio_time_axis)) + sell_portfolio_values = portfolio_series.reindex(sell_times, method='ffill') + ax.plot(sell_times, sell_portfolio_values, 'v', color='red', markersize=8, label='Sell Signal') + + ax.set_title('Strategy Portfolio Value vs. Buy & Hold' + title_suffix) + ax.set_ylabel('Portfolio Value ($)') + ax.grid(True); ax.legend(); ax.ticklabel_format(style='plain', axis='y') + plot_idx += 1 + else: + print("Skipping Portfolio Performance plot due to missing data.") + + # Final adjustments + axs[-1].set_xlabel('Time') # Add xlabel only to the bottom-most plot + fig.autofmt_xdate() + plt.tight_layout() + + if save_path: + try: # Added missing except block below + plt.savefig(save_path); + print(f"Combined performance plot saved to {save_path}") + except Exception as e: # Added missing except block + print(f"Error saving combined plot: {e}") # Corrected indentation + # plt.show() # Optional: uncomment to display plot interactively + plt.close(); + return fig + + def generate_performance_report(self, report_path="backtest_performance_report.md"): + # ... (implementation remains the same, note update) + metrics = self._calculate_performance_metrics() + if not metrics: + feature_logger.error("Cannot generate report: No metrics calculated.") + return None + + report = f"# GRU+SAC Backtesting Performance Report\n\n" + report += f"Report generated on: {pd.Timestamp.now()}\n" + # Ensure timestamps list is not empty before accessing + if self.timestamps: + # --- Log Timestamp Range Used for Report Header --- + report_start_ts = self.timestamps[0] + report_end_ts = self.timestamps[-1] + feature_logger.info(f"Generating Report - Timestamp range: {report_start_ts} to {report_end_ts}") + report += f"Data range: {self.timestamps[0]} to {self.timestamps[-1]}\n" + report += f"Total duration: {self.timestamps[-1] - self.timestamps[0]}\n\n" + else: + feature_logger.warning("Generating Report - Timestamps list is empty.") + report += "Data range: N/A\n" + report += "Total duration: N/A\n\n" + + report += "## Strategy Performance Metrics\n\n" + report += f"* **Initial capital:** ${metrics.get('initial_value', 0):,.2f}\n" + report += f"* **Final portfolio value:** ${metrics.get('final_value', 0):,.2f}\n" + report += f"* **Total return:** {metrics.get('total_return', 0)*100:.2f}%\n" + report += f"* **Annualized return:** {metrics.get('annual_return', 0)*100:.2f}%\n" + report += f"* **Sharpe ratio (annualized):** {metrics.get('sharpe_ratio', 0):.4f}\n" + report += f"* **Sortino ratio (annualized):** {metrics.get('sortino_ratio', 0):.4f}\n" + report += f"* **Volatility (annualized):** {metrics.get('volatility', 0)*100:.2f}%\n" + report += f"* **Maximum drawdown:** {metrics.get('max_drawdown', 0)*100:.2f}%\n" + report += f"* **Total trades:** {metrics.get('total_trades', 0)}\n" + + # Add Buy and Hold Performance section + report += "\n## Buy and Hold Benchmark\n\n" + bh_final_value = metrics.get('buy_hold_final_value') + bh_total_return = metrics.get('buy_hold_total_return') + if bh_final_value is not None and bh_total_return is not None: + report += f"* **Final value (B&H):** ${bh_final_value:,.2f}\n" + report += f"* **Total return (B&H):** {bh_total_return*100:.2f}%\n" + else: + report += "* *Buy and Hold benchmark could not be calculated.*\n" + + report += "\n## Position & Prediction Analysis\n\n" + report += f"* **Average absolute position size:** {metrics.get('avg_position', 0):.4f}\n" + report += f"* **Position sign accuracy vs return:** {metrics.get('position_accuracy', 0)*100:.2f}%\n" + report += f"* **Prediction sign accuracy vs return:** {metrics.get('pred_accuracy', 0)*100:.2f}%\n" + report += f"* **Prediction RMSE (on returns):** {metrics.get('prediction_rmse', 0):.6f}\n" + report += "\n## Correlations\n\n" + report += f"* **Prediction-Return correlation:** {metrics.get('pred_return_corr', 0):.4f}\n" + report += f"* **Prediction-Position correlation:** {metrics.get('pred_pos_corr', 0):.4f}\n" + report += f"* **Uncertainty-Position Size correlation:** {metrics.get('unc_pos_corr', 0):.4f}\n" + report += "\n## Notes\n\n" + report += f"* Transaction cost used: {self.transaction_cost * 100:.4f}% per position change value.\n" + report += f"* GRU lookback period: {self.gru_lookback} minutes.\n" + report += "* V6 features + return features used.\n" + report += "* Uncertainty estimated via MC Dropout standard deviation.\n" + # report += "* Uncertainty estimated directly by GRU second head.\n" + if report_path: + try: + with open(report_path, "w") as f: + f.write(report) + print(f"Performance report saved to {report_path}") + except Exception as e: + print(f"Error saving performance report: {e}") + return report \ No newline at end of file