"Act as a lead quantitative researcher. Your task is to design and build a…
- User
"Act as a lead quantitative researcher. Your task is to design and build a state-of-the-art, end-to-end Python script for a sophisticated ensemble AI model. The model's objective is short-term price movement forecasting for a highly liquid US stock. Your solution must be robust, production-ready, and adhere to rigorous financial modeling standards. I am giving you the liberty to choose the specific models and features, but you must justify your architectural decisions. Core Objective & Constraints: Forecasting Task: Predict the directional movement (up or down) and the closing price 5 trading days into the future. Asset: Choose a volatile, high-volume US stock that presents a suitable challenge (e.g., NVDA, TSLA, AMD). Justify your choice briefly. Data Sourcing: Use a reliable, free API like yfinance to fetch the necessary daily data. Critical Methodological Constraint: You must implement a time-series-aware validation strategy. Walk-forward validation or time-series cross-validation is mandatory to prevent lookahead bias and accurately simulate a live trading environment. A simple random train-test split is unacceptable. Data Leakage Prevention: All preprocessing steps (like feature scaling) must be fitted only on the training data for each fold. Areas for Your AI Expertise & Liberty: Feature Engineering: Engineer a comprehensive and powerful feature set. I leave the selection to you, but it should be well-reasoned. Create features that capture different market dynamics (e.g., trend, momentum, volatility, volume). In your comments, explain the categories of indicators you chose and why they are relevant for this forecasting task. Ensemble Architecture: Design a heterogeneous ensemble model. Your goal is to combine models with different strengths to create a more robust system. You have the freedom to choose your base learners and your ensembling method (e.g., stacking, blending, weighted averaging). Justify your choices. For example, explain why you might combine a deep learning model for sequence analysis (like a Transformer, GRU, or TCN) with a gradient boosting model for its performance on tabular data (like LightGBM or CatBoost). Explain why your chosen ensembling technique is suitable. Final Deliverable: A single, fully runnable, and well-documented Python script. Use modern, mainstream libraries (e.g., PyTorch/TensorFlow, scikit-learn, LightGBM/XGBoost, pandas). Extensive Markdown Comments: Your code should read like a research notebook. Explain the 'why' behind your code—your architectural decisions, feature choices, and the implementation of the walk-forward validation. Performance Evaluation: Conclude by evaluating the model on the final test set(s). Report key metrics appropriate for a financial forecast: For Price Prediction: Root Mean Squared Error (RMSE) & Mean Absolute Error (MAE). For Directional Prediction: Classification Accuracy, F1-Score, and a Confusion Matrix.
Assistant
I'll design a comprehensive ensemble model for short-term price forecasting. Let me structure this as a production-ready research script.
#!/usr/bin/env python3 """ State-of-the-Art Ensemble Model for Short-Term Stock Price Forecasting Lead Quantitative Researcher - Advanced AI Trading System Asset: NVIDIA Corporation (NVDA) Justification: - High volatility (beta ~1.7) provides ample signal for short-term forecasting - Massive trading volume (>50M shares daily) ensures liquidity - Strong exposure to AI/tech trends creates complex, non-linear patterns - Represents a challenging forecasting problem suitable for advanced ensemble methods Architecture: Heterogeneous Stacking Ensemble Base Learners: 1. Temporal Convolutional Network (TCN): Captures complex temporal dependencies and multi-scale patterns 2. LightGBM: Excellent for tabular feature processing and handling heterogeneous feature types 3. Transformer Encoder: Models long-range dependencies and attention mechanisms Meta Learner: Logistic Regression (for directional) + Linear Regression (for price) Validation: Strict Walk-Forward with Expanding Window """ # Core imports import numpy as np import pandas as pd import yfinance as yf from datetime import datetime, timedelta import warnings warnings.filterwarnings('ignore') # ML imports from sklearn.preprocessing import StandardScaler, RobustScaler from sklearn.linear_model import LogisticRegression, LinearRegression from sklearn.metrics import (accuracy_score, f1_score, confusion_matrix, mean_squared_error, mean_absolute_error) import lightgbm as lgb import torch import torch.nn as nn from torch.utils.data import Dataset, DataLoader # Visualization (optional but useful for research) import matplotlib.pyplot as plt import seaborn as sns # Set random seeds for reproducibility np.random.seed(42) torch.manual_seed(42) # Configuration class Config: # Stock and timeframe TICKER = "NVDA" START_DATE = "2015-01-01" END_DATE = datetime.now().strftime("%Y-%m-%d") # Forecasting parameters FORECAST_HORIZON = 5 # 5 trading days ahead LOOKBACK_WINDOW = 60 # Use 60 days of history for prediction # Walk-forward validation INITIAL_TRAIN_SIZE = 1000 # Initial training period (days) VALIDATION_STEP = 20 # Step size for walk-forward # Model parameters BATCH_SIZE = 32 EPOCHS = 100 LEARNING_RATE = 0.001 EARLY_STOPPING_PATIENCE = 10 # Data fetching and preprocessing def fetch_stock_data(ticker, start_date, end_date): """Fetch OHLCV data from Yahoo Finance with error handling""" print(f"Fetching data for {ticker} from {start_date} to {end_date}") try: data = yf.download(ticker, start=start_date, end=end_date) if data.empty: raise ValueError("No data returned from Yahoo Finance") # Clean and validate data data = data.dropna() data = data[['Open', 'High', 'Low', 'Close', 'Volume']] print(f"Successfully fetched {len(data)} trading days") return data except Exception as e: print(f"Error fetching data: {e}") return None # Feature Engineering class FeatureEngineer: """ Comprehensive feature engineering capturing multiple market dynamics: 1. Trend Features: Moving averages, MACD, parabolic SAR 2. Momentum Features: RSI, Stochastic Oscillator, Rate of Change 3. Volatility Features: ATR, Bollinger Bands, historical volatility 4. Volume Features: Volume indicators, OBV, volume moving averages 5. Pattern Features: Price action patterns, support/resistance levels """ @staticmethod def add_trend_features(df, windows=[5, 10, 20, 50]): """Add trend-following indicators""" for window in windows: df[f'MA_{window}'] = df['Close'].rolling(window=window).mean() df[f'EMA_{window}'] = df['Close'].ewm(span=window).mean() # MACD exp12 = df['Close'].ewm(span=12).mean() exp26 = df['Close'].ewm(span=26).mean() df['MACD'] = exp12 - exp26 df['MACD_Signal'] = df['MACD'].ewm(span=9).mean() df['MACD_Hist'] = df['MACD'] - df['MACD_Signal'] return df @staticmethod def add_momentum_features(df, windows=[5, 10, 14]): """Add momentum oscillators""" for window in windows: # RSI delta = df['Close'].diff() gain = (delta.where(delta > 0, 0)).rolling(window=window).mean() loss = (-delta.where(delta < 0, 0)).rolling(window=window).mean() rs = gain / loss df[f'RSI_{window}'] = 100 - (100 / (1 + rs)) # Rate of Change df[f'ROC_{window}'] = (df['Close'] / df['Close'].shift(window) - 1) * 100 return df @staticmethod def add_volatility_features(df, windows=[10, 20]): """Add volatility measures""" for window in windows: # Average True Range high_low = df['High'] - df['Low'] high_close = np.abs(df['High'] - df['Close'].shift()) low_close = np.abs(df['Low'] - df['Close'].shift()) ranges = pd.concat([high_low, high_close, low_close], axis=1) true_range = ranges.max(axis=1) df[f'ATR_{window}'] = true_range.rolling(window=window).mean() # Bollinger Bands rolling_mean = df['Close'].rolling(window=window).mean() rolling_std = df['Close'].rolling(window=window).std() df[f'BB_Upper_{window}'] = rolling_mean + (rolling_std * 2) df[f'BB_Lower_{window}'] = rolling_mean - (rolling_std * 2) df[f'BB_Width_{window}'] = (df[f'BB_Upper_{window}'] - df[f'BB_Lower_{window}']) / rolling_mean return df @staticmethod def add_volume_features(df, windows=[5, 10, 20]): """Add volume-based indicators""" # On-Balance Volume df['OBV'] = (np.sign(df['Close'].diff()) * df['Volume']).cumsum() for window in windows: df[f'Volume_MA_{window}'] = df['Volume'].rolling(window=window).mean() df[f'Volume_Ratio_{window}'] = df['Volume'] / df[f'Volume_MA_{window}'] return df @staticmethod def add_price_features(df): """Add price action features""" df['Daily_Return'] = df['Close'].pct_change() df['Log_Return'] = np.log(df['Close'] / df['Close'].shift(1)) df['High_Low_Ratio'] = df['High'] / df['Low'] df['Close_Open_Ratio'] = df['Close'] / df['Open'] # Price position within daily range df['Price_Position'] = (df['Close'] - df['Low']) / (df['High'] - df['Low']) return df @staticmethod def create_features(df): """Create comprehensive feature set""" df = df.copy() # Add all feature categories df = FeatureEngineer.add_trend_features(df) df = FeatureEngineer.add_momentum_features(df) df = FeatureEngineer.add_volatility_features(df) df = FeatureEngineer.add_volume_features(df) df = FeatureEngineer.add_price_features(df) # Drop initial rows with NaN values from rolling calculations df = df.dropna() print(f"Created {len(df.columns) - 5} features") # Subtract original 5 columns return df # Deep Learning Models class TemporalBlock(nn.Module): """Temporal Convolutional Network Block""" def __init__(self, n_inputs, n_outputs, kernel_size, stride, dilation, dropout=0.2): super().__init__() self.conv1 = nn.Conv1d(n_inputs, n_outputs, kernel_size, stride=stride, padding=(kernel_size-1)*dilation, dilation=dilation) self.relu1 = nn.ReLU() self.dropout1 = nn.Dropout(dropout) self.conv2 = nn.Conv1d(n_outputs, n_outputs, kernel_size, stride=stride, padding=(kernel_size-1)*dilation, dilation=dilation) self.relu2 = nn.ReLU() self.dropout2 = nn.Dropout(dropout) self.net = nn.Sequential(self.conv1, self.relu1, self.dropout1, self.conv2, self.relu2, self.dropout2) self.downsample = nn.Conv1d(n_inputs, n_outputs, 1) if n_inputs != n_outputs else None self.relu = nn.ReLU() def forward(self, x): out = self.net(x) res = x if self.downsample is None else self.downsample(x) return self.relu(out + res) class TCN(nn.Module): """Temporal Convolutional Network for sequence modeling""" def __init__(self, input_size, output_size, num_channels, kernel_size=2, dropout=0.2): super().__init__() layers = [] num_levels = len(num_channels) for i in range(num_levels): dilation_size = 2 ** i in_channels = input_size if i == 0 else num_channels[i-1] out_channels = num_channels[i] layers.append(TemporalBlock(in_channels, out_channels, kernel_size, stride=1, dilation=dilation_size, dropout=dropout)) self.network = nn.Sequential(*layers) self.linear = nn.Linear(num_channels[-1], output_size) def forward(self, x): x = x.transpose(1, 2) # Convert to (batch, features, seq_len) output = self.network(x) output = output[:, :, -1] # Take last time step return self.linear(output) class TransformerModel(nn.Module): """Transformer Encoder for time series""" def __init__(self, input_dim, hidden_dim, num_layers, num_heads, output_dim, dropout=0.1): super().__init__() self.input_projection = nn.Linear(input_dim, hidden_dim) self.pos_encoding = nn.Parameter(torch.randn(1, Config.LOOKBACK_WINDOW, hidden_dim)) encoder_layer = nn.TransformerEncoderLayer( d_model=hidden_dim, nhead=num_heads, dropout=dropout, batch_first=True ) self.transformer = nn.TransformerEncoder(encoder_layer, num_layers=num_layers) self.output_layer = nn.Linear(hidden_dim, output_dim) def forward(self, x): x = self.input_projection(x) x = x + self.pos_encoding[:, :x.size(1), :] x = self.transformer(x) x = x[:, -1, :] # Take last time step return self.output_layer(x) # Dataset class class StockDataset(Dataset): def __init__(self, features, price_targets, direction_targets): self.features = torch.FloatTensor(features) self.price_targets = torch.FloatTensor(price_targets) self.direction_targets = torch.LongTensor(direction_targets) def __len__(self): return len(self.features) def __getitem__(self, idx): return self.features[idx], self.price_targets[idx], self.direction_targets[idx] # Ensemble Model class HeterogeneousEnsemble: """ Heterogeneous stacking ensemble combining: 1. TCN: Captures multi-scale temporal patterns 2. LightGBM: Handles tabular features effectively 3. Transformer: Models long-range dependencies with attention Meta-learners: Logistic Regression (direction) + Linear Regression (price) """ def __init__(self, lookback_window, feature_dim): self.lookback_window = lookback_window self.feature_dim = feature_dim # Base models self.tcn = TCN(input_size=feature_dim, output_size=2, num_channels=[64, 32, 16]) self.transformer = TransformerModel(input_dim=feature_dim, hidden_dim=64, num_layers=2, num_heads=4, output_dim=2) self.lgb_model = None # Meta models self.meta_direction = LogisticRegression() self.meta_price = LinearRegression() # Scalers self.feature_scaler = RobustScaler() self.price_scaler = StandardScaler() def prepare_sequences(self, features, targets): """Convert tabular data to sequential format""" X_seq, y_seq = [], [] for i in range(self.lookback_window, len(features)): X_seq.append(features[i-self.lookback_window:i]) y_seq.append(targets[i]) return np.array(X_seq), np.array(y_seq) def train_base_models(self, X_train, y_price_train, y_dir_train): """Train the base models""" # Prepare sequential data for DL models X_seq, y_price_seq = self.prepare_sequences(X_train, y_price_train) _, y_dir_seq = self.prepare_sequences(X_train, y_dir_train) # Train TCN dataset = StockDataset(X_seq, y_price_seq, y_dir_seq) dataloader = DataLoader(dataset, batch_size=Config.BATCH_SIZE, shuffle=True) optimizer = torch.optim.Adam(self.tcn.parameters(), lr=Config.LEARNING_RATE) criterion = nn.MSELoss() self.tcn.train() for epoch in range(Config.EPOCHS): for batch_X, batch_price, batch_dir in dataloader: optimizer.zero_grad() outputs = self.tcn(batch_X) loss = criterion(outputs[:, 0], batch_price) loss.backward() optimizer.step() # Train Transformer optimizer = torch.optim.Adam(self.transformer.parameters(), lr=Config.LEARNING_RATE) self.transformer.train() for epoch in range(Config.EPOCHS): for batch_X, batch_price, batch_dir in dataloader: optimizer.zero_grad() outputs = self.transformer(batch_X) loss = criterion(outputs[:, 0], batch_price) loss.backward() optimizer.step() # Train LightGBM # Use the last feature vector of each sequence for LightGBM X_lgb = X_seq[:, -1, :] # Take the most recent features self.lgb_model = lgb.LGBMRegressor() self.lgb_model.fit(X_lgb, y_price_seq) def get_base_predictions(self, X): """Get predictions from all base models""" X_seq, _ = self.prepare_sequences(X, np.zeros(len(X))) # TCN predictions self.tcn.eval() with torch.no_grad(): tcn_preds = self.tcn(torch.FloatTensor(X_seq)).numpy() # Transformer predictions self.transformer.eval() with torch.no_grad(): transformer_preds = self.transformer(torch.FloatTensor(X_seq)).numpy() # LightGBM predictions X_lgb = X_seq[:, -1, :] lgb_preds = self.lgb_model.predict(X_lgb) # Stack predictions meta_features = np.column_stack([ tcn_preds[:, 0], transformer_preds[:, 0], lgb_preds, tcn_preds[:, 1], transformer_preds[:, 1] ]) return meta_features def fit(self, X_train, y_price_train, y_dir_train): """Train the complete ensemble""" # Scale features X_train_scaled = self.feature_scaler.fit_transform(X_train) y_price_scaled = self.price_scaler.fit_transform(y_price_train.reshape(-1, 1)).flatten() # Train base models self.train_base_models(X_train_scaled, y_price_scaled, y_dir_train) # Get base predictions for meta-training meta_X = self.get_base_predictions(X_train_scaled[self.lookback_window:]) meta_y_price = y_price_scaled[self.lookback_window:] meta_y_dir = y_dir_train[self.lookback_window:] # Train meta models self.meta_price.fit(meta_X[:, :3], meta_y_price) # First 3 for price prediction self.meta_direction.fit(meta_X[:, 3:], meta_y_dir) # Last 2 for direction def predict(self, X): """Make predictions using the ensemble""" X_scaled = self.feature_scaler.transform(X) meta_features = self.get_base_predictions(X_scaled) # Make predictions price_pred_scaled = self.meta_price.predict(meta_features[:, :3]) direction_pred = self.meta_direction.predict(meta_features[:, 3:]) # Inverse transform price predictions price_pred = self.price_scaler.inverse_transform( price_pred_scaled.reshape(-1, 1) ).flatten() return price_pred, direction_pred # Walk-forward validation def walk_forward_validation(data, config): """Implement rigorous walk-forward validation to prevent lookahead bias"""
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.