Skip to content
TheChatArchiveThe Chat Archive
Claudeclaude-opus-4-1-20250805-thinking-16k· AnthropicHugging Face Arena Expert 5K

"Act as a lead quantitative researcher. Your task is to design and build a…

  1. User

    "Act as a lead quantitative researcher. Your task is to design and build a state-of-the-art, end-to-end Python script for a sophisticated ensemble AI model. The model's objective is short-term price movement forecasting for a highly liquid US stock. Your solution must be robust, production-ready, and adhere to rigorous financial modeling standards. I am giving you the liberty to choose the specific models and features, but you must justify your architectural decisions. Core Objective & Constraints: Forecasting Task: Predict the directional movement (up or down) and the closing price 5 trading days into the future. Asset: Choose a volatile, high-volume US stock that presents a suitable challenge (e.g., NVDA, TSLA, AMD). Justify your choice briefly. Data Sourcing: Use a reliable, free API like yfinance to fetch the necessary daily data. Critical Methodological Constraint: You must implement a time-series-aware validation strategy. Walk-forward validation or time-series cross-validation is mandatory to prevent lookahead bias and accurately simulate a live trading environment. A simple random train-test split is unacceptable. Data Leakage Prevention: All preprocessing steps (like feature scaling) must be fitted only on the training data for each fold. Areas for Your AI Expertise & Liberty: Feature Engineering: Engineer a comprehensive and powerful feature set. I leave the selection to you, but it should be well-reasoned. Create features that capture different market dynamics (e.g., trend, momentum, volatility, volume). In your comments, explain the categories of indicators you chose and why they are relevant for this forecasting task. Ensemble Architecture: Design a heterogeneous ensemble model. Your goal is to combine models with different strengths to create a more robust system. You have the freedom to choose your base learners and your ensembling method (e.g., stacking, blending, weighted averaging). Justify your choices. For example, explain why you might combine a deep learning model for sequence analysis (like a Transformer, GRU, or TCN) with a gradient boosting model for its performance on tabular data (like LightGBM or CatBoost). Explain why your chosen ensembling technique is suitable. Final Deliverable: A single, fully runnable, and well-documented Python script. Use modern, mainstream libraries (e.g., PyTorch/TensorFlow, scikit-learn, LightGBM/XGBoost, pandas). Extensive Markdown Comments: Your code should read like a research notebook. Explain the 'why' behind your code—your architectural decisions, feature choices, and the implementation of the walk-forward validation. Performance Evaluation: Conclude by evaluating the model on the final test set(s). Report key metrics appropriate for a financial forecast: For Price Prediction: Root Mean Squared Error (RMSE) & Mean Absolute Error (MAE). For Directional Prediction: Classification Accuracy, F1-Score, and a Confusion Matrix.

  2. Claude Assistant

    Sophisticated Ensemble AI Model for Short-Term Stock Price Forecasting

    """
    Production-Ready Ensemble Model for 5-Day Stock Price Forecasting
    ================================================================
    Author: Lead Quantitative Researcher
    Objective: Predict directional movement and closing price 5 days ahead
    Target Asset: NVIDIA Corporation (NVDA)
    Methodology: Heterogeneous ensemble with walk-forward validation
    """
    
    import numpy as np
    import pandas as pd
    import warnings
    warnings.filterwarnings('ignore')
    
    # Data acquisition and manipulation
    import yfinance as yf
    from datetime import datetime, timedelta
    
    # Feature engineering
    from ta import add_all_ta_features
    from ta.utils import dropna
    from scipy import stats
    
    # Machine Learning models
    from sklearn.preprocessing import StandardScaler, RobustScaler
    from sklearn.ensemble import RandomForestRegressor, RandomForestClassifier
    from sklearn.metrics import (mean_squared_error, mean_absolute_error, 
                               accuracy_score, f1_score, confusion_matrix, 
                               classification_report)
    import lightgbm as lgb
    import xgboost as xgb
    
    # Deep Learning
    import torch
    import torch.nn as nn
    import torch.optim as optim
    from torch.utils.data import DataLoader, TensorDataset
    from torch.nn.utils import clip_grad_norm_
    
    # Visualization
    import matplotlib.pyplot as plt
    import seaborn as sns
    
    # Set random seeds for reproducibility
    np.random.seed(42)
    torch.manual_seed(42)
    
    # ================================================================================
    # SECTION 1: ASSET SELECTION AND DATA ACQUISITION
    # ================================================================================
    
    """
    Asset Selection Rationale: NVIDIA (NVDA)
    -----------------------------------------
    1. High Liquidity: One of the most traded stocks in the US market
    2. Volatility: Tech sector with exposure to AI/GPU markets creates price swings
    3. Data Richness: Strong reaction to market events, earnings, and sector news
    4. Forecasting Challenge: Complex price dynamics suitable for ensemble modeling
    """
    
    class DataAcquisition:
        """
        Handles data fetching and initial preprocessing
        """
        def __init__(self, ticker='NVDA', start_date='2018-01-01', end_date=None):
            self.ticker = ticker
            self.start_date = start_date
            self.end_date = end_date or datetime.now().strftime('%Y-%m-%d')
            
        def fetch_data(self):
            """
            Fetch OHLCV data using yfinance
            """
            print(f"📊 Fetching data for {self.ticker} from {self.start_date} to {self.end_date}")
            
            # Download data with auto-adjust for splits/dividends
            data = yf.download(self.ticker, 
                              start=self.start_date, 
                              end=self.end_date,
                              auto_adjust=True,
                              progress=False)
            
            # Ensure we have sufficient data
            if len(data) < 500:
                raise ValueError("Insufficient data points. Need at least 500 trading days.")
            
            print(f"✅ Successfully fetched {len(data)} trading days of data")
            return data
    
    # ================================================================================
    # SECTION 2: FEATURE ENGINEERING
    # ================================================================================
    
    class FeatureEngineer:
        """
        Comprehensive Feature Engineering Pipeline
        
        Feature Categories:
        1. Price-Based Features: Capture price dynamics and patterns
        2. Technical Indicators: Traditional TA signals (momentum, trend, volatility)
        3. Volume Features: Market participation and liquidity measures
        4. Market Microstructure: Advanced features capturing market behavior
        5. Statistical Features: Distribution and regime characteristics
        """
        
        def __init__(self, lookback_periods=[5, 10, 20, 50]):
            self.lookback_periods = lookback_periods
            
        def create_price_features(self, df):
            """
            Price-based features capturing various return dynamics
            """
            feature_df = df.copy()
            
            # Log returns - more stable for modeling
            feature_df['log_return'] = np.log(df['Close'] / df['Close'].shift(1))
            
            # Multiple timeframe returns
            for period in self.lookback_periods:
                feature_df[f'return_{period}d'] = df['Close'].pct_change(period)
                feature_df[f'log_return_{period}d'] = np.log(df['Close'] / df['Close'].shift(period))
            
            # Price relative to moving averages (momentum/trend)
            for period in self.lookback_periods:
                ma = df['Close'].rolling(window=period).mean()
                feature_df[f'price_to_ma{period}'] = df['Close'] / ma
                
            # High-Low spread (intraday volatility proxy)
            feature_df['hl_spread'] = (df['High'] - df['Low']) / df['Close']
            
            # Close position within day's range (momentum indicator)
            feature_df['close_position'] = (df['Close'] - df['Low']) / (df['High'] - df['Low'] + 1e-10)
            
            return feature_df
        
        def create_volume_features(self, df):
            """
            Volume-based features for liquidity and participation analysis
            """
            feature_df = pd.DataFrame(index=df.index)
            
            # Volume rate of change
            for period in self.lookback_periods:
                feature_df[f'volume_roc_{period}d'] = df['Volume'].pct_change(period)
            
            # Volume moving average ratio (unusual volume detector)
            for period in self.lookback_periods:
                vol_ma = df['Volume'].rolling(window=period).mean()
                feature_df[f'volume_ratio_{period}d'] = df['Volume'] / (vol_ma + 1e-10)
            
            # Price-Volume correlation (accumulation/distribution signal)
            for period in [10, 20]:
                price_change = df['Close'].pct_change()
                feature_df[f'pv_corr_{period}d'] = price_change.rolling(window=period).corr(df['Volume'])
            
            # On-Balance Volume momentum
            obv = (np.sign(df['Close'].diff()) * df['Volume']).cumsum()
            feature_df['obv_momentum'] = obv.pct_change(20)
            
            return feature_df
        
        def create_volatility_features(self, df):
            """
            Volatility measures for risk assessment
            """
            feature_df = pd.DataFrame(index=df.index)
            
            # Historical volatility (different windows)
            returns = df['Close'].pct_change()
            for period in self.lookback_periods:
                feature_df[f'volatility_{period}d'] = returns.rolling(window=period).std() * np.sqrt(252)
            
            # Parkinson volatility (using high-low range)
            for period in [5, 20]:
                hl_ratio = np.log(df['High'] / df['Low'])
                feature_df[f'parkinson_vol_{period}d'] = (
                    hl_ratio.rolling(window=period).apply(
                        lambda x: np.sqrt(np.sum(x**2) / (4 * period * np.log(2)))
                    ) * np.sqrt(252)
                )
            
            # Average True Range (ATR) - volatility measure
            high_low = df['High'] - df['Low']
            high_close = np.abs(df['High'] - df['Close'].shift())
            low_close = np.abs(df['Low'] - df['Close'].shift())
            true_range = pd.concat([high_low, high_close, low_close], axis=1).max(axis=1)
            
            for period in [10, 20]:
                feature_df[f'atr_{period}d'] = true_range.rolling(window=period).mean() / df['Close']
            
            return feature_df
        
        def create_market_microstructure_features(self, df):
            """
            Advanced market microstructure features
            """
            feature_df = pd.DataFrame(index=df.index)
            
            # Amihud illiquidity measure proxy
            returns = df['Close'].pct_change()
            dollar_volume = df['Close'] * df['Volume']
            feature_df['illiquidity'] = (np.abs(returns) / (dollar_volume + 1e-10)).rolling(window=20).mean()
            
            # Kyle's lambda (price impact) proxy
            for period in [10, 20]:
                signed_volume = np.sign(returns) * df['Volume']
                price_change = df['Close'].diff()
                
                # Rolling regression coefficient
                feature_df[f'kyle_lambda_{period}d'] = (
                    price_change.rolling(window=period).apply(
                        lambda y: np.nan if period > len(y) else 
                        stats.linregress(signed_volume[-period:], y[-period:])[0],
                        raw=False
                    )
                )
            
            # Volatility of volatility (vol clustering)
            volatility = returns.rolling(window=20).std()
            feature_df['vol_of_vol'] = volatility.rolling(window=20).std()
            
            return feature_df
        
        def create_technical_indicators(self, df):
            """
            Traditional technical analysis indicators using TA library
            """
            # Add all TA features
            ta_df = add_all_ta_features(
                df, open="Open", high="High", low="Low", 
                close="Close", volume="Volume", fillna=True
            )
            
            # Select most relevant indicators
            selected_columns = [
                'momentum_rsi',  # RSI for overbought/oversold
                'momentum_stoch_rsi',  # Stochastic RSI
                'trend_macd_signal',  # MACD signal
                'trend_cci',  # Commodity Channel Index
                'volatility_bbhi',  # Bollinger Bands indicators
                'volatility_bbli',
                'momentum_ao',  # Awesome Oscillator
                'trend_adx',  # ADX for trend strength
                'volume_adi',  # Accumulation/Distribution
                'volume_mfi'  # Money Flow Index
            ]
            
            # Filter available columns
            available_cols = [col for col in selected_columns if col in ta_df.columns]
            return ta_df[available_cols]
        
        def create_target_variables(self, df, forecast_horizon=5):
            """
            Create target variables for both regression and classification
            """
            # Future closing price (regression target)
            df['target_price'] = df['Close'].shift(-forecast_horizon)
            
            # Future return for classification
            df['future_return'] = df['Close'].shift(-forecast_horizon) / df['Close'] - 1
            
            # Directional movement (classification target)
            df['target_direction'] = (df['future_return'] > 0).astype(int)
            
            return df
        
        def engineer_all_features(self, df):
            """
            Combine all feature engineering steps
            """
            print("🔧 Engineering comprehensive feature set...")
            
            # Create different feature categories
            price_features = self.create_price_features(df)
            volume_features = self.create_volume_features(df)
            volatility_features = self.create_volatility_features(df)
            microstructure_features = self.create_market_microstructure_features(df)
            technical_features = self.create_technical_indicators(df)
            
            # Combine all features
            feature_df = pd.concat([
                price_features,
                volume_features,
                volatility_features,
                microstructure_features,
                technical_features
            ], axis=1)
            
            # Add target variables
            feature_df = self.create_target_variables(feature_df)
            
            # Remove any remaining NaN values
            feature_df = feature_df.dropna()
            
            print(f"✅ Created {len(feature_df.columns) - 3} features")  # -3 for targets and future_return
            
            return feature_df
    
    # ================================================================================
    # SECTION 3: DEEP LEARNING MODELS
    # ================================================================================
    
    class TransformerModel(nn.Module):
        """
        Transformer-based model for sequence modeling
        
        Architecture Rationale:
        - Self-attention mechanism captures long-range dependencies
        - Position encoding preserves temporal order
        - Multi-head attention learns different pattern types
        """
        def __init__(self, input_dim, d_model=64, n_heads=4, n_layers=2, dropout=0.1):
            super(TransformerModel, self).__init__()
            
            self.input_projection = nn.Linear(input_dim, d_model)
            self.positional_encoding = PositionalEncoding(d_model, dropout)
            
            encoder_layers = nn.TransformerEncoderLayer(
                d_model=d_model,
                nhead=n_heads,
                dim_feedforward=d_model * 4,
                dropout=dropout,
                batch_first=True
            )
            self.transformer_encoder = nn.TransformerEncoder(encoder_layers, num_layers=n_layers)
            
            # Dual heads for regression and classification
            self.regression_head = nn.Sequential(
                nn.Linear(d_model, d_model // 2),
                nn.ReLU(),
                nn.Dropout(dropout),
                nn.Linear(d_model // 2, 1)
            )
            
            self.classification_head = nn.Sequential(
                nn.Linear(d_model, d_model // 2),
                nn.ReLU(),
                nn.Dropout(dropout),
                nn.Linear(d_model // 2, 2)
            )
            
        def forward(self, x):
            # Project input to model dimension
            x = self.input_projection(x)
            x = self.positional_encoding(x)
            
            # Transformer encoding
            x = self.transformer_encoder(x)
            
            # Global pooling (use last timestep)
            x = x[:, -1, :]
            
            # Dual outputs
            regression_out = self.regression_head(x)
            classification_out = self.classification_head(x)
            
            return regression_out, classification_out
    
    class PositionalEncoding(nn.Module):
        """
        Add positional encoding to preserve sequence order information
        """
        def __init__(self, d_model, dropout=0.1, max_len=5000):
            super(PositionalEncoding, self).__init__()
            self.dropout = nn.Dropout(p=dropout)
            
            pe = torch.zeros(max_len, d_model)
            position = torch.arange(0, max_len, dtype=torch.float).unsqueeze(1)
            div_term = torch.exp(torch.arange(0, d_model, 2).float() * 
                               (-np.log(10000.0) / d_model))
            pe[:, 0::2] = torch.sin(position * div_term)
            pe[:, 1::2] = torch.cos(position * div_term)
            pe = pe.unsqueeze(0).transpose(0, 1)
            self.register_buffer('pe', pe)
            
        def forward(self, x):
            x = x + self.pe[:x.size(1), :].transpose(0, 1)
            return self.dropout(x)
    
    class GRUModel(nn.Module):
        """
        GRU-based model for sequence modeling
        
        Architecture Rationale:
        - GRU gates handle vanishing gradient problem
        - Bidirectional processing captures both past and future context
        - Fewer parameters than LSTM, reducing overfitting risk
        """
        def __init__(self, input_dim, hidden_dim=64, n_layers=2, dropout=0.2):
            super(GRUModel, self).__init__()
            
            self.gru = nn.GRU(
                input_dim, 
                hidden_dim, 
                n_layers, 
                batch_first=True,
                dropout=dropout if n_layers > 1 else 0,
                bidirectional=True
            )
            
            # Account for bidirectional
            final_dim = hidden_dim * 2
            
            # Dual heads
            self.regression_head = nn.Sequential(
                nn.Linear(final_dim, hidden_dim),
                nn.ReLU(),
                nn.Dropout(dropout),
                nn.Linear(hidden_dim, 1)
            )
            
            self.classification_head = nn.Sequential(
                nn.Linear(final_dim, hidden_dim),
                nn.ReLU(),
                nn.Dropout(dropout),
                nn.Linear(hidden_dim, 2)
            )
            
        def forward(self, x):
            # GRU encoding
            gru_out, _ = self.gru(x)
            
            # Use last timestep
            x = gru_out[:, -1, :]
            
            # Dual outputs
            regression_out = self.regression_head(x)
            classification_out = self.classification_head(x)
            
            return regression_out, classification_out
    
    # ================================================================================
    # SECTION 4: WALK-FORWARD VALIDATION FRAMEWORK
    # ================================================================================
    
    class WalkForwardValidator:
        """
        Time-Series Aware Walk-Forward Validation
        
        Methodology:
        - Expanding window approach: Train on all available historical data
        - Fixed test window: Evaluate on next period
        - No data leakage: Preprocessing fitted only on training data
        - Realistic simulation: Mimics real-world deployment scenario
        """
        
        def __init__(self, n_splits=5, test_size=60, gap=5):
            """
            Parameters:
            - n_splits: Number of walk-forward windows
            - test_size: Size of each test window (trading days)
            - gap: Gap between train and test to account for execution lag
            """
            self.n_splits = n_splits
            self.test_size = test_size
            self.gap = gap
            
        def split(self, X, y):
            """
            Generate train/test indices for walk-forward validation
            """
            n_samples = len(X)
            splits = []
            
            # Calculate size of each fold
            fold_size = (n_samples - self.test_size - self.gap) // self.n_splits
            
            for i in range(self.n_splits):
                # Expanding window for training
                train_end = fold_size * (i + 1)
                train_idx = np.arange(0, train_end)
                
                # Fixed window for testing (with gap)
                test_start = train_end + self.gap
                test_end = min(test_start + self.test_size, n_samples)
                test_idx = np.arange(test_start, test_end)
                
                if len(test_idx) > 0:
                    splits.append((train_idx, test_idx))
                    
            return splits
    
    # ================================================================================
    # SECTION 5: ENSEMBLE MODEL ARCHITECTURE
    # ================================================================================
    
    class HeterogeneousEnsemble:
        """
        Sophisticated Ensemble Architecture
        
        Design Philosophy:
        1. Model Diversity: Combine models with different inductive biases
           - Transformer: Long-range pattern recognition
           - GRU: Sequential dependency modeling  
           - LightGBM: Non-linear feature interactions
           - Random Forest: Robust to outliers
           
        2. Stacking Strategy: Meta-learner combines base predictions
           - Learns optimal weighting dynamically
           - Captures model complementarity
           
        3. Dual-Task Learning: Simultaneous regression and classification
           - Shared representations improve both tasks
           - Classification helps regression by identifying regimes
        """
        
        def __init__(self, input_dim, sequence_length=20, device='cpu'):
            self.input_dim = input_dim
            self.sequence_length = sequence_length
            self.device = device
            
            # Initialize base models
            self.transformer = TransformerModel(input_dim).to(device)
            self.gru = GRUModel(input_dim).to(device)
            
            # Gradient boosting models
            self.lgb_regressor = None
            self.lgb_classifier = None
            
            # Random Forest models  
            self.rf_regressor = None
            self.rf_classifier = None
            
            # Meta-learners for stacking
            self.meta_regressor = None
            self.meta_classifier = None
            
            # Scalers (will be fitted on training data only)
            self.scaler = None
            self.meta_scaler_reg = None
            self.meta_scaler_clf = None
            
        def prepare_sequences(self, X, y_reg=None, y_clf=None):
            """
            Prepare sequences for deep learning models
            """
            sequences = []
            targets_reg = [] if y_reg is not None else None
            targets_clf = [] if y_clf is not None else None
            
            for i in range(len(X) - self.sequence_length):
                sequences.append(X[i:i + self.sequence_length])
                if y_reg is not None:
                    targets_reg.append(y_reg[i + self.sequence_length])
                if y_clf is not None:
                    targets_clf.append(y_clf[i + self.sequence_length])
                    
            return np.array(sequences), np.array(targets_reg) if targets_reg else None, \
                   np.array(targets_clf) if targets_clf else None
        
        def train_deep_models(self, X_seq, y_reg, y_clf, epochs=50, batch_size=32):
            """
            Train deep learning models (Transformer and GRU)
            """
            # Convert to tensors
            X_tensor = torch.FloatTensor(X_seq).to(self.device)
            y_reg_tensor = torch.FloatTensor(y_reg.reshape(-1, 1)).to(self.device)
            y_clf_tensor = torch.LongTensor(y_clf).to(self.device)
            
            # Create data loader
            dataset = TensorDataset(X_tensor, y_reg_tensor, y_clf_tensor)
            loader = DataLoader(dataset, batch_size=batch_size, shuffle=True)
            
            # Training settings
            models = [self.transformer, self.gru]
            optimizers = [
                optim.AdamW(self.transformer.parameters(), lr=0.001, weight_decay=0.01),
                optim.AdamW(self.gru.parameters(), lr=0.001, weight_decay=0.01)
            ]
            
            criterion_reg = nn.MSELoss()
            criterion_clf = nn.CrossEntropyLoss()
            
            # Train each model
            for model, optimizer in zip(models, optimizers):
                model.train()
                
                for epoch in range(epochs):
                    epoch_loss = 0
                    for batch_X, batch_y_reg, batch_y_clf in loader:
                        optimizer.zero_grad()
                        
                        # Forward pass
                        reg_out, clf_out = model(batch_X)
                        
                        # Combined loss
                        loss_reg = criterion_reg(reg_out, batch_y_reg)
                        loss_clf = criterion_clf(clf_out, batch_y_clf)
                        loss = loss_reg + 0.5 * loss_clf  # Weight classification less
                        
                        # Backward pass
                        loss.backward()
                        clip_grad_norm_(model.parameters(), max_norm=1.0)
                        optimizer.step()
                        
                        epoch_loss += loss.item()
                        
                model.eval()
        
        def train_gbm_models(self, X, y_reg, y_clf):
            """
            Train gradient boosting models (LightGBM)
            """
            # LightGBM parameters tuned for financial data
            lgb_params = {
                'objective': 'regression',
                'metric': 'rmse',
                'boosting_type': 'gbdt',
                'num_leaves': 31,
                'learning_rate': 0.05,
                'feature_fraction': 0.8,
                'bagging_fraction': 0.7,
                'bagging_freq': 5,
                'verbose': -1,
                'random_state': 42,
                'n_estimators': 200,
                'early_stopping_rounds': 20
            }
            
            # Regression model
            self.lgb_regressor = lgb.LGBMRegressor(**lgb_params)
            self.lgb_regressor.fit(
                X, y_reg,
                eval_set=[(X[-500:], y_reg[-500:])],  # Use last 500 samples for validation
                verbose=False
            )
            
            # Classification model
            lgb_params['objective'] = 'binary'
            lgb_params['metric'] = 'binary_logloss'
            self.lgb_classifier = lgb.LGBMClassifier(**lgb_params)
            self.lgb_classifier.fit(
                X, y_clf,
                eval_set=[(X[-500:], y_clf[-500:])],
                verbose=False
            )
        
        def train_rf_models(self, X, y_reg, y_clf):
            """
            Train Random Forest models for robustness
            """
            # Random Forest parameters
            rf_params = {
                'n_estimators': 100,
                'max_depth': 10,
                'min_samples_split': 20,
                'min_samples_leaf': 10,
                'max_features': 'sqrt',
                'random_state': 42,
                'n_jobs': -1
            }
            
            self.rf_regressor = RandomForestRegressor(**rf_params)
            self.rf_regressor.fit(X, y_reg)
            
            self.rf_classifier = RandomForestClassifier(**rf_params)
            self.rf_classifier.fit(X, y_clf)
        
        def get_base_predictions(self, X, X_seq):
            """
            Get predictions from all base models
            """
            predictions = {}
            
            # Deep learning predictions
            with torch.no_grad():
                X_tensor = torch.FloatTensor(X_seq).to(self.device)
                
                # Transformer predictions
                trans_reg, trans_clf = self.transformer(X_tensor)
                predictions['transformer_reg'] = trans_reg.cpu().numpy().flatten()
                predictions['transformer_clf'] = torch.softmax(trans_clf, dim=1)[:, 1].cpu().numpy()
                
                # GRU predictions
                gru_reg, gru_clf = self.gru(X_tensor)
                predictions['gru_reg'] = gru_reg.cpu().numpy().flatten()
                predictions['gru_clf'] = torch.softmax(gru_clf, dim=1)[:, 1].cpu().numpy()
            
            # Adjust X for non-sequence models (use latest values)
            X_latest = X[self.sequence_length:]
            
            # LightGBM predictions
            if self.lgb_regressor is not None:
                predictions['lgb_reg'] = self.lgb_regressor.predict(X_latest)
                predictions['lgb_clf'] = self.lgb_classifier.predict_proba(X_latest)[:, 1]
            
            # Random Forest predictions
            if self.rf_regressor is not None:
                predictions['rf_reg'] = self.rf_regressor.predict(X_latest)
                predictions['rf_clf'] = self.rf_classifier.predict_proba(X_latest)[:, 1]
            
            return predictions
        
        def train_meta_learners(self, base_predictions, y_reg, y_clf):
            """
            Train meta-learners for stacking ensemble
            """
            # Prepare meta-features
            meta_features_reg = np.column_stack([
                base_predictions['transformer_reg'],
                base_predictions['gru_reg'],
                base_predictions['lgb_reg'],
                base_predictions['rf_reg']
            ])
            
            meta_features_clf = np.column_stack([
                base_predictions['transformer_clf'],
                base_predictions['gru_clf'],
                base_predictions['lgb_clf'],
                base_predictions['rf_clf']
            ])
            
            # Scale meta-features
            self.meta_scaler_reg = StandardScaler()
            self.meta_scaler_clf = StandardScaler()
            
            meta_features_reg = self.meta_scaler_reg.fit_transform(meta_features_reg)
            meta_features_clf = self.meta_scaler_clf.fit_transform(meta_features_clf)
            
            # Train meta-learners (using XGBoost for final ensemble)
            self.meta_regressor = xgb.XGBRegressor(
                n_estimators=50,
                max_depth=3,
                learning_rate=0.1,
                random_state=42
            )
            self.meta_regressor.fit(meta_features_reg, y_reg)
            
            self.meta_classifier = xgb.XGBClassifier(
                n_estimators=50,
                max_depth=3,
                learning_rate=0.1,
                random_state=42
            )
            self.meta_classifier.fit(meta_features_clf, y_clf)
        
        def fit(self, X, y_reg, y_clf):
            """
            Full training pipeline for ensemble
            """
            print("🚀 Training Heterogeneous Ensemble Model")
            
            # Scale features
            self.scaler = RobustScaler()
            X_scaled = self.scaler.fit_transform(X)
            
            # Prepare sequences
            X_seq, y_reg_seq, y_clf_seq = self.prepare_sequences(X_scaled, y_reg, y_clf)
            
            # Train deep learning models
            print("   📊 Training deep learning models...")
            self.train_deep_models(X_seq, y_reg_seq, y_clf_seq)
            
            # Train gradient boosting models
            print("   📊 Training gradient boosting models...")
            X_latest = X_scaled[self.sequence_length:]
            self.train_gbm_models(X_latest, y_reg_seq, y_clf_seq)
            
            # Train random forest models
            print("   📊 Training random forest models...")
            self.train_rf_models(X_latest, y_reg_seq, y_clf_seq)
            
            # Get base predictions for meta-learning
            print("   📊 Training meta-learners...")
            base_predictions = self.get_base_predictions(X_scaled, X_seq)
            self.train_meta_learners(base_predictions, y_reg_seq, y_clf_seq)
            
            print("✅ Ensemble training complete")
        
        def predict(self, X):
            """
            Make predictions using trained ensemble
            """
            # Scale features
            X_scaled = self.scaler.transform(X)
            
            # Prepare sequences
            X_seq, _, _ = self.prepare_sequences(X_scaled)
            
            # Get base predictions
            base_predictions = self.get_base_predictions(X_scaled, X_seq)
            
            # Prepare meta-features
            meta_features_reg = np.column_stack([
                base_predictions['transformer_reg'],
                base_predictions['gru_reg'],
                base_predictions['lgb_reg'],
                base_predictions['rf_reg']
            ])
            
            meta_features_clf = np.column_stack([
                base_predictions['transformer_clf'],
                base_predictions['gru_clf'],
                base_predictions['lgb_clf'],
                base_predictions['rf_clf']
            ])
            
            # Scale meta-features
            meta_features_reg = self.meta_scaler_reg.transform(meta_features_reg)
            meta_features_clf = self.meta_scaler_clf.transform(meta_features_clf)
            
            # Get final predictions
            final_reg = self.meta_regressor.predict(meta_features_reg)
            final_clf_proba = self.meta_classifier.predict_proba(meta_features_clf)[:, 1]
            final_clf = (final_clf_proba > 0.5).astype(int)
            
            return final_reg, final_clf, final_clf_proba
    
    # ================================================================================
    # SECTION 6: MODEL EVALUATION AND BACKTESTING
    # ================================================================================
    
    class ModelEvaluator:
        """
        Comprehensive evaluation framework for financial forecasting
        """
        
        @staticmethod
        def evaluate_regression(y_true, y_pred, prices_true, prices_pred):
            """
            Evaluate regression performance (price prediction)
            """
            # Standard metrics
            rmse = np.sqrt(mean_squared_error(y_true, y_pred))
            mae = mean_absolute_error(y_true, y_pred)
            
            # Financial metrics
            # Convert back to actual prices for interpretability
            price_rmse = np.sqrt(mean_squared_error(prices_true, prices_pred))
            price_mae = mean_absolute_error(prices_true, prices_pred)
            
            # Directional accuracy from regression
            direction_true = (y_true > 0).astype(int)
            direction_pred = (y_pred > 0).astype(int)
            directional_accuracy = accuracy_score(direction_true, direction_pred)
            
            return {
                'RMSE': rmse,
                'MAE': mae,
                'Price_RMSE': price_rmse,
                'Price_MAE': price_mae,
                'Directional_Accuracy': directional_accuracy
            }
        
        @staticmethod
        def evaluate_classification(y_true, y_pred, y_proba=None):
            """
            Evaluate classification performance (directional prediction)
            """
            accuracy = accuracy_score(y_true, y_pred)
            f1 = f1_score(y_true, y_pred, average='weighted')
            
            # Confusion matrix
            cm = confusion_matrix(y_true, y_pred)
            
            # Calculate precision for up moves (important for trading)
            if cm[1, 1] + cm[0, 1] > 0:
                precision_up = cm[1, 1] / (cm[1, 1] + cm[0, 1])
            else:
                precision_up = 0
                
            return {
                'Accuracy': accuracy,
                'F1_Score': f1,
                'Precision_Up': precision_up,
                'Confusion_Matrix': cm
            }
        
        @staticmethod
        def plot_results(y_true_reg, y_pred_reg, y_true_clf, y_pred_clf, dates=None):
            """
            Visualize prediction results
            """
            fig, axes = plt.subplots(2, 2, figsize=(15, 10))
            
            # Plot 1: Predicted vs Actual Returns
            axes[0, 0].scatter(y_true_reg, y_pred_reg, alpha=0.5)
            axes[0, 0].plot([y_true_reg.min(), y_true_reg.max()], 
                           [y_true_reg.min(), y_true_reg.max()], 'r--', lw=2)
            axes[0, 0].set_xlabel('Actual Returns')
            axes[0, 0].set_ylabel('Predicted Returns')
            axes[0, 0].set_title('Regression: Predicted vs Actual Returns')
            axes[0, 0].grid(True, alpha=0.3)
            
            # Plot 2: Time series of predictions
            if dates is not None:
                axes[0, 1].plot(dates, y_true_reg, label='Actual', alpha=0.7)
                axes[0, 1].plot(dates, y_pred_reg, label='Predicted', alpha=0.7)
                axes[0, 1].set_xlabel('Date')
                axes[0, 1].set_ylabel('Returns')
                axes[0, 1].set_title('Time Series: Actual vs Predicted Returns')
                axes[0, 1].legend()
                axes[0, 1].grid(True, alpha=0.3)
            
            # Plot 3: Confusion Matrix
            cm = confusion_matrix(y_true_clf, y_pred_clf)
            sns.heatmap(cm, annot=True, fmt='d', cmap='Blues', ax=axes[1, 0])
            axes[1, 0].set_xlabel('Predicted')
            axes[1, 0].set_ylabel('Actual')
            axes[1, 0].set_title('Classification: Confusion Matrix')
            
            # Plot 4: Residual distribution
            residuals = y_true_reg - y_pred_reg
            axes[1, 1].hist(residuals, bins=50, edgecolor='black', alpha=0.7)
            axes[1, 1].axvline(x=0, color='r', linestyle='--')
            axes[1, 1].set_xlabel('Residuals')
            axes[1, 1].set_ylabel('Frequency')
            axes[1, 1].set_title('Residual Distribution')
            axes[1, 1].grid(True, alpha=0.3)
            
            plt.tight_layout()
            plt.show()
    
    # ================================================================================
    # SECTION 7: MAIN EXECUTION PIPELINE
    # ================================================================================
    
    def main():
        """
        Main execution pipeline with walk-forward validation
        """
        print("=" * 80)
        print("SOPHISTICATED ENSEMBLE MODEL FOR STOCK PRICE FORECASTING")
        print("=" * 80)
        
        # Set device for deep learning
        device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
        print(f"🖥️  Using device: {device}")
        
        # Step 1: Data Acquisition
        data_fetcher = DataAcquisition(ticker='NVDA', start_date='2018-01-01')
        raw_data = data_fetcher.fetch_data()
        
        # Step 2: Feature Engineering
        feature_engineer = FeatureEngineer()
        data = feature_engineer.engineer_all_features(raw_data)
        
        # Separate features and targets
        feature_cols = [col for col in data.columns if col not in 
                       ['target_price', 'target_direction', 'future_return']]
        X = data[feature_cols].values
        y_price = data['target_price'].values
        y_direction = data['target_direction'].values
        
        # Store original prices for evaluation
        original_prices = data['Close'].values
        
        # Step 3: Walk-Forward Validation
        print("\n📈 Starting Walk-Forward Validation")
        print("-" * 40)
        
        validator = WalkForwardValidator(n_splits=5, test_size=60, gap=5)
        splits = validator.split(X, y_price)
        
        # Store results for each fold
        all_results_reg = []
        all_results_clf = []
        all_predictions_reg = []
        all_predictions_clf = []
        all_true_reg = []
        all_true_clf = []
        
        for fold, (train_idx, test_idx) in enumerate(splits):
            print(f"\n📊 Fold {fold + 1}/{len(splits)}")
            print(f"   Training samples: {len(train_idx)}")
            print(f"   Testing samples: {len(test_idx)}")
            
            # Split data
            X_train, X_test = X[train_idx], X[test_idx]
            y_train_price, y_test_price = y_price[train_idx], y_price[test_idx]
            y_train_dir, y_test_dir = y_direction[train_idx], y_direction[test_idx]
            
            # Calculate returns for evaluation
            train_prices = original_prices[train_idx]
            test_prices = original_prices[test_idx]
            
            # Adjust for sequence length in test indices
            seq_length = 20
            actual_test_idx = test_idx[seq_length:]
            
            # Train ensemble
            ensemble = HeterogeneousEnsemble(
                input_dim=X_train.shape[1],
                sequence_length=seq_length,
                device=device
            )
            ensemble.fit(X_train, y_train_price, y_train_dir)
            
            # Make predictions
            pred_price, pred_dir, pred_dir_proba = ensemble.predict(X_test)
            
            # Adjust test targets for sequence length
            y_test_price_adj = y_test_price[seq_length:]
            y_test_dir_adj = y_test_dir[seq_length:]
            test_prices_adj = test_prices[seq_length:]
            
            # Convert predictions to returns for evaluation
            test_returns_true = (y_test_price_adj - test_prices_adj) / test_prices_adj
            test_returns_pred = (pred_price - test_prices_adj) / test_prices_adj
            
            # Evaluate fold
            evaluator = ModelEvaluator()
            
            reg_metrics = evaluator.evaluate_regression(
                test_returns_true, test_returns_pred,
                y_test_price_adj, pred_price
            )
            
            clf_metrics = evaluator.evaluate_classification(
                y_test_dir_adj, pred_dir, pred_dir_proba
            )
            
            # Store results
            all_results_reg.append(reg_metrics)
            all_results_clf.append(clf_metrics)
            all_predictions_reg.extend(test_returns_pred)
            all_predictions_clf.extend(pred_dir)
            all_true_reg.extend(test_returns_true)
            all_true_clf.extend(y_test_dir_adj)
            
            # Print fold results
            print(f"\n   Regression Metrics:")
            print(f"   - RMSE: {reg_metrics['RMSE']:.4f}")
            print(f"   - MAE: {reg_metrics['MAE']:.4f}")
            print(f"   - Price RMSE: ${reg_metrics['Price_RMSE']:.2f}")
            print(f"   - Directional Accuracy: {reg_metrics['Directional_Accuracy']:.2%}")
            
            print(f"\n   Classification Metrics:")
            print(f"   - Accuracy: {clf_metrics['Accuracy']:.2%}")
            print(f"   - F1 Score: {clf_metrics['F1_Score']:.4f}")
            print(f"   - Precision (Up): {clf_metrics['Precision_Up']:.2%}")
        
        # Step 4: Aggregate Results
        print("\n" + "=" * 80)
        print("FINAL RESULTS (AGGREGATED ACROSS ALL FOLDS)")
        print("=" * 80)
        
        # Convert to numpy arrays
        all_predictions_reg = np.array(all_predictions_reg)
        all_predictions_clf = np.array(all_predictions_clf)
        all_true_reg = np.array(all_true_reg)
        all_true_clf = np.array(all_true_clf)
        
        # Calculate final metrics
        evaluator = ModelEvaluator()
        
        # Price prediction metrics
        final_rmse = np.sqrt(mean_squared_error(all_true_reg, all_predictions_reg))
        final_mae = mean_absolute_error(all_true_reg, all_predictions_reg)
        
        # Direction prediction metrics
        final_accuracy = accuracy_score(all_true_clf, all_predictions_clf)
        final_f1 = f1_score(all_true_clf, all_predictions_clf, average='weighted')
        final_cm = confusion_matrix(all_true_clf, all_predictions_clf)
        
        print("\n📊 REGRESSION PERFORMANCE (5-Day Price Prediction)")
        print("-" * 40)
        print(f"Root Mean Squared Error (RMSE): {final_rmse:.4f}")
        print(f"Mean Absolute Error (MAE): {final_mae:.4f}")
        
        # Calculate average metrics across folds
        avg_price_rmse = np.mean([r['Price_RMSE'] for r in all_results_reg])
        avg_price_mae = np.mean([r['Price_MAE'] for r in all_results_reg])
        print(f"Average Price RMSE: ${avg_price_rmse:.2f}")
        print(f"Average Price MAE: ${avg_price_mae:.2f}")
        
        print("\n📊 CLASSIFICATION PERFORMANCE (5-Day Direction Prediction)")
        print("-" * 40)
        print(f"Overall Accuracy: {final_accuracy:.2%}")
        print(f"F1 Score: {final_f1:.4f}")
        
        print("\n📊 Confusion Matrix:")
        print("-" * 40)
        print("     Predicted")
        print("     Down  Up")
        print(f"Down {final_cm[0, 0]:4d} {final_cm[0, 1]:4d}")
        print(f"Up   {final_cm[1, 0]:4d} {final_cm[1, 1]:4d}")
        
        # Calculate additional metrics
        if final_cm[1, 1] + final_cm[0, 1] > 0:
            precision_up = final_cm[1, 1] / (final_cm[1, 1] + final_cm[0, 1])
            print(f"\nPrecision for Up moves: {precision_up:.2%}")
        
        if final_cm[1, 1] + final_cm[1, 0] > 0:
            recall_up = final_cm[1, 1] / (final_cm[1, 1] + final_cm[1, 0])
            print(f"Recall for Up moves: {recall_up:.2%}")
        
        # Visualize results
        print("\n📊 Generating visualization...")
        evaluator.plot_results(all_true_reg, all_predictions_reg, 
                              all_true_clf, all_predictions_clf)
        
        print("\n" + "=" * 80)
        print("MODEL EVALUATION COMPLETE")
        print("=" * 80)
        
        return {
            'regression': {
                'rmse': final_rmse,
                'mae': final_mae,
                'predictions': all_predictions_reg,
                'true_values': all_true_reg
            },
            'classification': {
                'accuracy': final_accuracy,
                'f1_score': final_f1,
                'confusion_matrix': final_cm,
                'predictions': all_predictions_clf,
                'true_values': all_true_clf
            }
        }
    
    # ================================================================================
    # EXECUTE MAIN PIPELINE
    # ================================================================================
    
    if __name__ == "__main__":
        # Run the complete pipeline
        results = main()
        
        print("\n🎯 Model training and evaluation completed successfully!")
        print("The ensemble model combines:")
        print("  • Transformer (attention-based pattern recognition)")
        print("  • GRU (sequential dependency modeling)")
        print("  • LightGBM (gradient boosting for tabular features)")
        print("  • Random Forest (robust baseline)")
        print("\nAll integrated through meta-learning for optimal performance.")
    

    Key Architectural Decisions & Justifications

    1. Asset Selection: NVIDIA (NVDA)

    • High volatility in the AI/semiconductor sector provides rich patterns
    • Massive liquidity ensures price discovery and reduces market microstructure noise
    • News sensitivity creates exploitable momentum and mean-reversion opportunities

    2. Feature Engineering Philosophy

    • Multi-scale features: Capture patterns across different time horizons (5, 10, 20, 50 days)
    • Market microstructure: Kyle's lambda and Amihud illiquidity for execution cost awareness
    • Regime indicators: Volatility clustering and correlation features for market state detection
    • Technical indicators: Well-established signals that many market participants follow

    3. Ensemble Architecture Rationale

    • Transformer: Self-attention mechanism captures non-local dependencies and regime changes
    • GRU: Efficient sequential modeling with fewer parameters than LSTM
    • LightGBM: Superior performance on tabular data with built-in regularization
    • Random Forest: Robust baseline resistant to outliers and overfitting
    • Meta-learning stacking: Learns optimal combination weights dynamically

    4. Walk-Forward Validation

    • Expanding window: Uses all available historical data (realistic for production)
    • Fixed test window: Consistent evaluation period across folds
    • Gap parameter: Accounts for execution delay in real trading

    5. Dual-Task Learning

    • Simultaneous regression and classification improves feature representations
    • Classification helps identify market regimes that affect price prediction accuracy
    • Shared representations reduce overfitting through multi-task regularization

    This implementation represents a production-ready, institutional-grade forecasting system that combines modern deep learning with traditional quantitative finance techniques, all while maintaining strict time-series discipline to prevent lookahead bias.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.