141 lines
18 KiB
Plaintext
141 lines
18 KiB
Plaintext
2025-04-18 03:31:11,814 - root - INFO - Using Base Models Directory: /home/yasha/develop/gru_sac_predictor/models
|
|
2025-04-18 03:31:11,814 - root - INFO - Using results directory: /home/yasha/develop/gru_sac_predictor/results/20250418_033111
|
|
2025-04-18 03:31:11,814 - root - INFO - Using logs directory: /home/yasha/develop/gru_sac_predictor/logs/20250418_033111
|
|
2025-04-18 03:31:11,814 - root - INFO - Using models directory: /home/yasha/develop/gru_sac_predictor/models/20250418_033111
|
|
2025-04-18 03:31:11,814 - root - INFO - Logging setup complete. Log file: /home/yasha/develop/gru_sac_predictor/logs/20250418_033111/pipeline_20250418_033111.log
|
|
2025-04-18 03:31:11,814 - root - INFO - --- Starting Pipeline Run: 20250418_033111 ---
|
|
2025-04-18 03:31:11,814 - root - INFO - Using config: /home/yasha/develop/gru_sac_predictor/gru_sac_predictor/config.yaml
|
|
2025-04-18 03:31:11,814 - root - INFO - Resolved relative db_dir '../data/crypto_market_data' to absolute path: /home/yasha/develop/data/crypto_market_data
|
|
2025-04-18 03:31:11,814 - gru_sac_predictor.src.data_loader - INFO - Initialized DataLoader with db_dir='/home/yasha/develop/data/crypto_market_data'
|
|
2025-04-18 03:31:11,815 - gru_sac_predictor.src.feature_engineer - INFO - FeatureEngineer initialized with minimal whitelist: ['return_1m', 'return_15m', 'return_60m', 'ATR_14', 'volatility_14d', 'chaikin_AD_10', 'svi_10', 'EMA_10', 'EMA_50', 'MACD', 'MACD_signal', 'hour_sin', 'hour_cos']
|
|
2025-04-18 03:31:11,815 - gru_sac_predictor.src.gru_model_handler - INFO - GRUModelHandler initialized for run 20250418_033111 in /home/yasha/develop/gru_sac_predictor/models/20250418_033111
|
|
2025-04-18 03:31:11,815 - gru_sac_predictor.src.calibrator - INFO - Calibrator initialized with edge threshold: 0.55
|
|
2025-04-18 03:31:11,815 - gru_sac_predictor.src.backtester - INFO - Backtester initialized.
|
|
2025-04-18 03:31:11,815 - gru_sac_predictor.src.backtester - INFO - Initial Capital: 10000.00
|
|
2025-04-18 03:31:11,815 - gru_sac_predictor.src.backtester - INFO - Transaction Cost: 0.0500%
|
|
2025-04-18 03:31:11,815 - gru_sac_predictor.src.backtester - INFO - Edge Threshold: 0.550
|
|
2025-04-18 03:31:11,819 - root - INFO - Saved run configuration to /home/yasha/develop/gru_sac_predictor/results/20250418_033111/run_config.yaml
|
|
2025-04-18 03:31:11,819 - root - INFO - === Starting Pipeline Execution ===
|
|
2025-04-18 03:31:11,819 - root - INFO - --- Stage: Loading and Preprocessing Data ---
|
|
2025-04-18 03:31:11,819 - gru_sac_predictor.src.data_loader - INFO - Loading data for SOL-USDT (bnbspot) from 2025-03-01 to 2025-03-10, interval 1min
|
|
2025-04-18 03:31:11,822 - gru_sac_predictor.src.data_loader - INFO - Scanning for DB files recursively in: /home/yasha/develop/data/crypto_market_data
|
|
2025-04-18 03:31:11,869 - gru_sac_predictor.src.data_loader - INFO - Found 316 DB files. Using newest: 20250416.mktdata.ohlcv.db
|
|
2025-04-18 03:31:11,973 - gru_sac_predictor.src.data_loader - INFO - Identified 8 potential DB files: ['20250228.mktdata.ohlcv.db', '20250301.mktdata.ohlcv.db', '20250302.mktdata.ohlcv.db', '20250303.mktdata.ohlcv.db', '20250304.mktdata.ohlcv.db', '20250305.mktdata.ohlcv.db', '20250306.mktdata.ohlcv.db', '20250307.mktdata.ohlcv.db']
|
|
2025-04-18 03:31:12,037 - gru_sac_predictor.src.data_loader - INFO - Combined data shape before final filtering/resampling: (9827, 5)
|
|
2025-04-18 03:31:12,038 - gru_sac_predictor.src.data_loader - INFO - Shape after final date filtering: (9827, 5)
|
|
2025-04-18 03:31:12,039 - gru_sac_predictor.src.data_loader - INFO - Successfully loaded and processed data for SOL-USDT. Final shape: (9827, 5)
|
|
2025-04-18 03:31:12,040 - root - INFO - Raw data loaded successfully: 9827 rows from 2025-03-01 00:00:00+00:00 to 2025-03-07 23:59:00+00:00
|
|
2025-04-18 03:31:12,040 - root - INFO - --- Stage: Engineering Features ---
|
|
2025-04-18 03:31:12,040 - gru_sac_predictor.src.feature_engineer - INFO - --- Adding Base Features ---
|
|
2025-04-18 03:31:12,040 - gru_sac_predictor.src.feature_engineer - INFO - Adding cyclical hour features (sin/cos)...
|
|
2025-04-18 03:31:12,042 - gru_sac_predictor.src.feature_engineer - INFO - Adding imbalance features...
|
|
2025-04-18 03:31:12,051 - gru_sac_predictor.src.feature_engineer - INFO - Successfully added imbalance features.
|
|
2025-04-18 03:31:12,051 - gru_sac_predictor.src.feature_engineer - INFO - Adding TA features...
|
|
2025-04-18 03:31:12,140 - gru_sac_predictor.src.feature_engineer - INFO - Successfully added TA features.
|
|
2025-04-18 03:31:12,141 - gru_sac_predictor.src.feature_engineer - INFO - Base feature engineering complete. DataFrame shape: (9827, 20)
|
|
2025-04-18 03:31:12,143 - root - INFO - Feature engineering complete. Shape: (9827, 20)
|
|
2025-04-18 03:31:12,143 - root - INFO - --- Stage: Defining Labels and Aligning ---
|
|
2025-04-18 03:31:12,146 - root - INFO - Dropped 5 rows due to NaN targets (horizon=5).
|
|
2025-04-18 03:31:12,147 - root - INFO - Labels (horizon=5) defined and aligned. Features shape: (9822, 20), Targets shape: (9822, 2)
|
|
2025-04-18 03:31:12,147 - root - INFO - --- Stage: Splitting Data ---
|
|
2025-04-18 03:31:12,147 - __main__ - INFO - Using split ratios: Train=0.60, Val=0.20, Test=0.20
|
|
2025-04-18 03:31:12,148 - root - INFO - Data split complete:
|
|
2025-04-18 03:31:12,148 - root - INFO - Train: X=(5893, 20), y=(5893, 2) (2025-03-01 00:00:00+00:00 to 2025-03-05 06:01:00+00:00)
|
|
2025-04-18 03:31:12,148 - root - INFO - Val: X=(1964, 20), y=(1964, 2) (2025-03-05 06:02:00+00:00 to 2025-03-06 14:55:00+00:00)
|
|
2025-04-18 03:31:12,148 - root - INFO - Test: X=(1965, 20), y=(1965, 2) (2025-03-06 14:56:00+00:00 to 2025-03-07 23:53:00+00:00)
|
|
2025-04-18 03:31:12,148 - root - INFO - --- Stage: Selecting and Pruning Features ---
|
|
2025-04-18 03:31:12,148 - gru_sac_predictor.src.feature_engineer - INFO - --- Selecting Features (LogReg L1 + VIF) ---
|
|
2025-04-18 03:31:12,148 - gru_sac_predictor.src.feature_engineer - INFO - Starting selection from 20 raw features.
|
|
2025-04-18 03:31:12,149 - gru_sac_predictor.src.feature_engineer - INFO - Performing Logistic Regression (L1, C=0.1) selection...
|
|
2025-04-18 03:31:13,975 - gru_sac_predictor.src.feature_engineer - INFO - Features selected by LogReg L1: ['open', 'high', 'low', 'close', 'volume', 'hour_sin', 'hour_cos', 'chaikin_AD_10', 'svi_10', 'gap_imbalance', 'return_1m', 'return_15m', 'return_60m', 'ATR_14', 'volatility_14d', 'EMA_10', 'EMA_50', 'MACD', 'MACD_signal', 'RSI_14']
|
|
2025-04-18 03:31:13,976 - gru_sac_predictor.src.feature_engineer - INFO - Candidate whitelist after LogReg (20 features): ['ATR_14', 'EMA_10', 'EMA_50', 'MACD', 'MACD_signal', 'RSI_14', 'chaikin_AD_10', 'close', 'gap_imbalance', 'high', 'hour_cos', 'hour_sin', 'low', 'open', 'return_15m', 'return_1m', 'return_60m', 'svi_10', 'volatility_14d', 'volume']
|
|
2025-04-18 03:31:13,976 - gru_sac_predictor.src.feature_engineer - INFO - Performing VIF filtering (threshold=10.0) on candidate features...
|
|
2025-04-18 03:31:14,095 - gru_sac_predictor.src.feature_engineer - INFO - Removing feature 'low' due to high VIF (13101.36 > 10.0)...
|
|
2025-04-18 03:31:14,206 - gru_sac_predictor.src.feature_engineer - INFO - Removing feature 'high' due to high VIF (12437.61 > 10.0)...
|
|
2025-04-18 03:31:14,308 - gru_sac_predictor.src.feature_engineer - INFO - Removing feature 'EMA_10' due to high VIF (8173.76 > 10.0)...
|
|
2025-04-18 03:31:14,401 - gru_sac_predictor.src.feature_engineer - INFO - Removing feature 'open' due to high VIF (2292.25 > 10.0)...
|
|
2025-04-18 03:31:14,483 - gru_sac_predictor.src.feature_engineer - INFO - Removing feature 'close' due to high VIF (482.83 > 10.0)...
|
|
2025-04-18 03:31:14,556 - gru_sac_predictor.src.feature_engineer - INFO - Removing feature 'MACD' due to high VIF (71.12 > 10.0)...
|
|
2025-04-18 03:31:14,623 - gru_sac_predictor.src.feature_engineer - INFO - VIF filtering complete. Max VIF = 3.97 <= 10.0.
|
|
2025-04-18 03:31:14,624 - gru_sac_predictor.src.feature_engineer - INFO - Final whitelist after VIF filtering (14 features): ['ATR_14', 'EMA_50', 'MACD_signal', 'RSI_14', 'chaikin_AD_10', 'gap_imbalance', 'hour_cos', 'hour_sin', 'return_15m', 'return_1m', 'return_60m', 'svi_10', 'volatility_14d', 'volume']
|
|
2025-04-18 03:31:14,625 - root - INFO - Saved final feature whitelist (14 features) to /home/yasha/develop/gru_sac_predictor/models/20250418_033111/final_whitelist_20250418_033111.json
|
|
2025-04-18 03:31:14,626 - root - INFO - Pruning feature sets using final whitelist: ['ATR_14', 'EMA_50', 'MACD_signal', 'RSI_14', 'chaikin_AD_10', 'gap_imbalance', 'hour_cos', 'hour_sin', 'return_15m', 'return_1m', 'return_60m', 'svi_10', 'volatility_14d', 'volume']
|
|
2025-04-18 03:31:14,631 - root - INFO - Feature shapes after pruning: Train=(5893, 14), Val=(1964, 14), Test=(1965, 14)
|
|
2025-04-18 03:31:14,632 - root - INFO - --- Stage: Scaling Features ---
|
|
2025-04-18 03:31:14,633 - root - INFO - Fitting StandardScaler on training data (numeric columns only)...
|
|
2025-04-18 03:31:14,640 - root - INFO - Feature scaler saved to /home/yasha/develop/gru_sac_predictor/models/20250418_033111/feature_scaler_20250418_033111.joblib
|
|
2025-04-18 03:31:14,660 - root - INFO - Features scaled successfully.
|
|
2025-04-18 03:31:14,660 - root - INFO - --- Stage: Creating Sequences ---
|
|
2025-04-18 03:31:14,661 - root - INFO - Creating sequences with lookback=60
|
|
2025-04-18 03:31:14,949 - root - INFO - Sequence shapes created:
|
|
2025-04-18 03:31:14,949 - root - INFO - Train: X=(5833, 60, 14), y_ret=(5833,), y_dir=(5833,)
|
|
2025-04-18 03:31:14,949 - root - INFO - Val: X=(1904, 60, 14), y_ret=(1904,), y_dir=(1904,)
|
|
2025-04-18 03:31:14,950 - root - INFO - Test: X=(1905, 60, 14), y_ret=(1905,), y_dir=(1905,)
|
|
2025-04-18 03:31:14,950 - root - INFO - --- Stage: Training or Loading GRU Model ---
|
|
2025-04-18 03:31:14,950 - root - INFO - Attempting to train a new GRU model for run 20250418_033111...
|
|
2025-04-18 03:31:14,950 - gru_sac_predictor.src.gru_model_handler - INFO - Building GRU model: lookback=60, n_features=14
|
|
2025-04-18 03:31:16,165 - gru_sac_predictor.src.gru_model_handler - INFO - Model built successfully.
|
|
2025-04-18 03:31:16,172 - gru_sac_predictor.src.gru_model_handler - INFO - Model: "crypto_gru"
|
|
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┓
|
|
┃ Layer (type) ┃ Output Shape ┃ Param # ┃ Connected to ┃
|
|
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━┩
|
|
│ input (InputLayer) │ (None, 60, 14) │ 0 │ - │
|
|
├──────────────────────────┼───────────────────────┼───────────────┼───────────────────────┤
|
|
│ gru (GRU) │ (None, 64) │ 15,360 │ input[0][0] │
|
|
├──────────────────────────┼───────────────────────┼───────────────┼───────────────────────┤
|
|
│ gauss_params (Dense) │ (None, 2) │ 130 │ gru[0][0] │
|
|
├──────────────────────────┼───────────────────────┼───────────────┼───────────────────────┤
|
|
│ ret (Lambda) │ (None, 1) │ 0 │ gauss_params[0][0] │
|
|
├──────────────────────────┼───────────────────────┼───────────────┼───────────────────────┤
|
|
│ dir (Dense) │ (None, 1) │ 65 │ gru[0][0] │
|
|
└──────────────────────────┴───────────────────────┴───────────────┴───────────────────────┘
|
|
Total params: 15,555 (60.76 KB)
|
|
Trainable params: 15,555 (60.76 KB)
|
|
Non-trainable params: 0 (0.00 B)
|
|
|
|
2025-04-18 03:31:16,173 - gru_sac_predictor.src.gru_model_handler - INFO - Starting GRU training: epochs=25, batch=256, patience=5
|
|
2025-04-18 03:31:16,173 - gru_sac_predictor.src.gru_model_handler - INFO - Train X shape: (5833, 60, 14)
|
|
2025-04-18 03:31:16,173 - gru_sac_predictor.src.gru_model_handler - INFO - Val X shape: (1904, 60, 14)
|
|
2025-04-18 03:31:16,173 - gru_sac_predictor.src.gru_model_handler - INFO - Train y keys: ['ret', 'gauss_params', 'dir']
|
|
2025-04-18 03:31:16,173 - gru_sac_predictor.src.gru_model_handler - INFO - Val y keys: ['ret', 'gauss_params', 'dir']
|
|
2025-04-18 03:31:25,608 - gru_sac_predictor.src.gru_model_handler - INFO - GRU training finished.
|
|
2025-04-18 03:31:25,608 - gru_sac_predictor.src.gru_model_handler - INFO - Best validation loss: -0.7041
|
|
2025-04-18 03:31:25,647 - gru_sac_predictor.src.gru_model_handler - INFO - GRU model saved successfully to: /home/yasha/develop/gru_sac_predictor/models/20250418_033111/gru_model_20250418_033111.keras
|
|
2025-04-18 03:31:25,647 - root - INFO - Newly trained GRU model saved to /home/yasha/develop/gru_sac_predictor/models/20250418_033111/gru_model_20250418_033111.keras
|
|
2025-04-18 03:31:25,647 - root - INFO - Using GRU model trained in current run: 20250418_033111
|
|
2025-04-18 03:31:25,647 - root - INFO - --- Stage: Baseline Checks (Placeholder) ---
|
|
2025-04-18 03:31:25,647 - root - WARNING - Baseline checks stage not implemented.
|
|
2025-04-18 03:31:25,647 - root - INFO - --- Stage: Calibrating Probabilities ---
|
|
2025-04-18 03:31:25,648 - root - INFO - No existing calibration temperature found for run 20250418_033111 at /home/yasha/develop/gru_sac_predictor/models/run_20250418_033111/calibration_temp_20250418_033111.npy.
|
|
2025-04-18 03:31:25,648 - root - INFO - Calculating optimal temperature on validation set...
|
|
2025-04-18 03:31:25,648 - gru_sac_predictor.src.gru_model_handler - INFO - Generating predictions for 1904 samples...
|
|
2025-04-18 03:31:25,916 - gru_sac_predictor.src.gru_model_handler - INFO - Predictions generated successfully.
|
|
2025-04-18 03:31:25,916 - gru_sac_predictor.src.calibrator - INFO - Optimizing calibration temperature using 1904 samples...
|
|
2025-04-18 03:31:25,921 - gru_sac_predictor.src.calibrator - INFO - Optimal temperature found: T = 10.0000
|
|
2025-04-18 03:31:25,922 - root - INFO - Saved newly calculated calibration temperature T=10.0000 to /home/yasha/develop/gru_sac_predictor/models/20250418_033111/calibration_temp_20250418_033111.npy
|
|
2025-04-18 03:31:25,922 - root - INFO - Generating validation reliability curve plot...
|
|
2025-04-18 03:31:26,138 - gru_sac_predictor.src.calibrator - INFO - Reliability curve saved to /home/yasha/develop/gru_sac_predictor/results/20250418_033111/reliability_curve_val_20250418_033111.png
|
|
2025-04-18 03:31:26,138 - root - INFO - --- Stage: Training or Loading SAC Agent ---
|
|
2025-04-18 03:31:26,138 - root - INFO - SAC training is enabled. Instantiating SACTrainer...
|
|
2025-04-18 03:31:26,138 - gru_sac_predictor.src.sac_trainer - INFO - Initializing SACTrainer with Run ID: sac_train_20250418_033126
|
|
2025-04-18 03:31:26,139 - gru_sac_predictor.src.sac_trainer - INFO - SAC Models Dir: /home/yasha/develop/gru_sac_predictor/models/sac_train_20250418_033126
|
|
2025-04-18 03:31:26,139 - gru_sac_predictor.src.sac_trainer - INFO - SAC Logs Dir: /home/yasha/develop/gru_sac_predictor/logs/20250418_033111/sac_train_20250418_033126
|
|
2025-04-18 03:31:26,139 - gru_sac_predictor.src.sac_trainer - INFO - SAC Results Dir:/home/yasha/develop/gru_sac_predictor/results/20250418_033111/sac_train_20250418_033126
|
|
2025-04-18 03:31:26,139 - gru_sac_predictor.src.sac_trainer - INFO - SAC TB Dir: /home/yasha/develop/gru_sac_predictor/logs/20250418_033111/sac_train_20250418_033126/tensorboard
|
|
2025-04-18 03:31:26,139 - gru_sac_predictor.src.sac_trainer - INFO - === Starting SAC Training Process (SAC Run ID: sac_train_20250418_033126) ===
|
|
2025-04-18 03:31:26,139 - gru_sac_predictor.src.sac_trainer - INFO - Using artifacts from GRU Run ID: 20250418_033111
|
|
2025-04-18 03:31:26,139 - gru_sac_predictor.src.sac_trainer - INFO - --- Loading Dependencies from GRU Run ID: 20250418_033111 ---
|
|
2025-04-18 03:31:26,139 - gru_sac_predictor.src.sac_trainer - ERROR - Models directory for GRU run 20250418_033111 not found at: /home/yasha/develop/gru_sac_predictor/models/run_20250418_033111
|
|
2025-04-18 03:31:26,139 - gru_sac_predictor.src.sac_trainer - ERROR - Failed to load GRU dependencies. Aborting SAC training.
|
|
2025-04-18 03:31:26,139 - __main__ - ERROR - SAC training failed. Proceeding without a newly trained agent.
|
|
2025-04-18 03:31:26,139 - __main__ - ERROR - SAC training failed and no load path specified in config. Cannot proceed with backtesting.
|
|
2025-04-18 03:31:26,139 - root - INFO - --- Stage: Running Backtest ---
|
|
2025-04-18 03:31:26,139 - root - ERROR - Pipeline execution failed: 'NoneType' object has no attribute 'columns'
|
|
Traceback (most recent call last):
|
|
File "/home/yasha/develop/gru_sac_predictor/gru_sac_predictor/src/trading_pipeline.py", line 1053, in execute
|
|
self.run_backtest()
|
|
File "/home/yasha/develop/gru_sac_predictor/gru_sac_predictor/src/trading_pipeline.py", line 910, in run_backtest
|
|
if 'close' in self.df_test.columns: # df_test has original columns before feature selection
|
|
AttributeError: 'NoneType' object has no attribute 'columns'
|
|
2025-04-18 03:31:26,140 - root - ERROR - === Pipeline Execution Terminated Due to Error ===
|