this code is meant to make a forecast for df with "ds" and 'y' column for prophet model,…
- User
this code is meant to make a forecast for df with "ds" and 'y' column for prophet model, however, this piece of code was used for daily data, now I have a grouped data on week by the mondays, how to adapt thise code work? for i in range(len(df)): if df.iloc[i]['y'] == 0: # Calculate the mean of the 3 most recent values at the same weekday recent_values = [df.iloc[i-7]['y'], df.iloc[i-14]['y'], df.iloc[i-21]['y']] # Remove any zeros from the recent values to avoid skewing the mean recent_values = [val for val in recent_values if val != 0] if recent_values: mean_value = np.mean(recent_values) df.at[i, 'y'] = mean_value
1.3) Soft-clip extreme spikes
low, high = df['y'].quantile([.02, .99]) df['y'] = df['y'].clip(lower=low, upper=high) 3) BAYESIAN OPTIMIZATION with OPTUNA for Prophet hyperparameters
HOLDOUT_DAYS = 365 train_df = df.iloc[:-HOLDOUT_DAYS] test_df = df.iloc[-HOLDOUT_DAYS:]
print(f"\n🔍 Starting Bayesian optimization with Optuna...") print(f"Training on {len(train_df)} days, testing on {HOLDOUT_DAYS} days")
3.1) Define objective function for Optuna def objective(trial): params = { 'changepoint_prior_scale': trial.suggest_loguniform('changepoint_prior_scale', 0.001, 0.5), 'seasonality_prior_scale': trial.suggest_loguniform('seasonality_prior_scale', 0.01, 10.0), 'changepoint_range': trial.suggest_uniform('changepoint_range', 0.8, 0.95), 'n_changepoints': trial.suggest_int('n_changepoints', 15, 50), 'seasonality_mode': trial.suggest_categorical('seasonality_mode', ['additive', 'multiplicative']), 'interval_width': trial.suggest_uniform('interval_width', 0.90, 0.95) }
m = Prophet( daily_seasonality=False, weekly_seasonality=True, yearly_seasonality=True, changepoint_prior_scale=params['changepoint_prior_scale'], seasonality_prior_scale=params['seasonality_prior_scale'], changepoint_range=params['changepoint_range'], n_changepoints=params['n_changepoints'], seasonality_mode=params['seasonality_mode'], interval_width=params['interval_width'] )
m.fit(train_df) future_test = m.make_future_dataframe(periods=HOLDOUT_DAYS, freq='D') fcst_test = m.predict(future_test) fcst_holdout = fcst_test[['ds','yhat','yhat_lower','yhat_upper']].tail(HOLDOUT_DAYS)
smape_score = smape(test_df['y'].values, fcst_holdout['yhat'].values) return smape_score 3.2) Run Bayesian optimization study = optuna.create_study(direction='minimize') study.optimize(objective, n_trials=50)
3.3) Get best parameters best_params = study.best_params best_smape = study.best_value
print(f"\n✅ Best SMAPE found: {best_smape:.2f}%") print(f"🎯 Best parameters:") for param, value in best_params.items(): print(f" {param}: {value}")
3.4) Train final model with best parameters and evaluate print(f"\n📊 Training final model with optimized parameters...")
m_best = Prophet( daily_seasonality=False, weekly_seasonality=True, yearly_seasonality=True, changepoint_prior_scale=best_params['changepoint_prior_scale'], seasonality_prior_scale=best_params['seasonality_prior_scale'], changepoint_range=best_params['changepoint_range'], n_changepoints=best_params['n_changepoints'], seasonality_mode=best_params['seasonality_mode'], interval_width=best_params['interval_width'] )
m_best.fit(train_df) future_test = m_best.make_future_dataframe(periods=HOLDOUT_DAYS, freq='D') fcst_test = m_best.predict(future_test) fcst_holdout = fcst_test[['ds','yhat','yhat_lower','yhat_upper']].tail(HOLDOUT_DAYS)
Final SMAPE final_smape = smape(test_df['y'].values, fcst_holdout['yhat'].values) print(f"✨ Final SMAPE with best params: {final_smape:.2f}%")
3.5) Plot comparison: Actual vs Optimized Forecast plt.figure(figsize=(15, 6))
Plot 1: Backtest results plt.subplot(1, 2, 1) plt.plot(train_df['ds'], train_df['y'], label='train', color='C0', alpha=0.7) plt.plot(test_df['ds'], test_df['y'], label='actual', color='C1', linewidth=2) plt.plot(fcst_holdout['ds'], fcst_holdout['yhat'], label='forecast', color='C3', linewidth=2, alpha=0.6) plt.fill_between(fcst_holdout['ds'], fcst_holdout['yhat_lower'], fcst_holdout['yhat_upper'], color='C3', alpha=0.2, label='confidence') plt.axvline(test_df['ds'].min(), ls='--', color='grey', alpha=0.5) plt.legend() plt.title(f"Optimized Prophet Backtest\nSMAPE = {final_smape:.2f}%") plt.xticks(rotation=45)
Plot 2: Residuals plt.subplot(1, 2, 2) residuals = test_df['y'].values - fcst_holdout['yhat'].values plt.plot(fcst_holdout['ds'], residuals, 'o-', alpha=0.7, color='C2') plt.axhline(0, ls='--', color='grey', alpha=0.5) plt.title(f"Forecast Residuals\nMean: {np.mean(residuals):.1f}, Std: {np.std(residuals):.1f}") plt.xticks(rotation=45)
plt.tight_layout() plt.show()
fcst_backtest_avg = fcst_holdout.copy()
Assistant
Adapting the Code for Weekly Data
To adapt your code for weekly data (grouped by Mondays), you'll need to make several adjustments. Here's how to modify each part of your code:
1. Handling Zero Values
The current code looks at the same weekday (every 7 days) to find recent values. For weekly data, you should look at the same week number in previous years instead.
for i in range(len(df)): if df.iloc[i]['y'] == 0: # Calculate the mean of the 3 most recent values from the same week in previous years # Assuming df is sorted by date recent_values = [] for j in range(1, 4): # Look at 3 previous years try: # Find the same week in the previous year prev_year = df.iloc[i]['ds'].year - j prev_week = df.iloc[i]['ds'].isocalendar().week prev_date = df[(df['ds'].dt.year == prev_year) & (df['ds'].dt.isocalendar().week == prev_week)]['y'].values[0] recent_values.append(prev_date) except IndexError: continue # Remove any zeros from the recent values recent_values = [val for val in recent_values if val != 0] if recent_values: mean_value = np.mean(recent_values) df.at[i, 'y'] = mean_value # Soft-clip extreme spikes low, high = df['y'].quantile([.02, .99]) df['y'] = df['y'].clip(lower=low, upper=high)2. Bayesian Optimization with Optuna
For the Prophet model, you'll need to adjust the frequency and parameters:
# 3) BAYESIAN OPTIMIZATION with OPTUNA for Prophet hyperparameters HOLDOUT_WEEKS = 52 # 1 year in weeks train_df = df.iloc[:-HOLDOUT_WEEKS] test_df = df.iloc[-HOLDOUT_WEEKS:] print(f"\n🔍 Starting Bayesian optimization with Optuna...") print(f"Training on {len(train_df)} weeks, testing on {HOLDOUT_WEEKS} weeks") # 3.1) Define objective function for Optuna def objective(trial): params = { 'changepoint_prior_scale': trial.suggest_loguniform('changepoint_prior_scale', 0.001, 0.5), 'seasonality_prior_scale': trial.suggest_loguniform('seasonality_prior_scale', 0.01, 10.0), 'changepoint_range': trial.suggest_uniform('changepoint_range', 0.8, 0.95), 'n_changepoints': trial.suggest_int('n_changepoints', 15, 50), 'seasonality_mode': trial.suggest_categorical('seasonality_mode', ['additive', 'multiplicative']), 'interval_width': trial.suggest_uniform('interval_width', 0.90, 0.95) } m = Prophet( daily_seasonality=False, weekly_seasonality=True, yearly_seasonality=True, changepoint_prior_scale=params['changepoint_prior_scale'], seasonality_prior_scale=params['seasonality_prior_scale'], changepoint_range=params['changepoint_range'], n_changepoints=params['n_changepoints'], seasonality_mode=params['seasonality_mode'], interval_width=params['interval_width'] ) m.fit(train_df) future_test = m.make_future_dataframe(periods=HOLDOUT_WEEKS, freq='W-MON') # Weekly frequency starting on Monday fcst_test = m.predict(future_test) fcst_holdout = fcst_test[['ds','yhat','yhat_lower','yhat_upper']].tail(HOLDOUT_WEEKS) smape_score = smape(test_df['y'].values, fcst_holdout['yhat'].values) return smape_score # 3.2) Run Bayesian optimization study = optuna.create_study(direction='minimize') study.optimize(objective, n_trials=50) # 3.3) Get best parameters best_params = study.best_params best_smape = study.best_value print(f"\n✅ Best SMAPE found: {best_smape:.2f}%") print(f"🎯 Best parameters:") for param, value in best_params.items(): print(f" {param}: {value}") # 3.4) Train final model with best parameters and evaluate print(f"\n📊 Training final model with optimized parameters...") m_best = Prophet( daily_seasonality=False, weekly_seasonality=True, yearly_seasonality=True, changepoint_prior_scale=best_params['changepoint_prior_scale'], seasonality_prior_scale=best_params['seasonality_prior_scale'], changepoint_range=best_params['changepoint_range'], n_changepoints=best_params['n_changepoints'], seasonality_mode=best_params['seasonality_mode'], interval_width=best_params['interval_width'] ) m_best.fit(train_df) future_test = m_best.make_future_dataframe(periods=HOLDOUT_WEEKS, freq='W-MON') fcst_test = m_best.predict(future_test) fcst_holdout = fcst_test[['ds','yhat','yhat_lower','yhat_upper']].tail(HOLDOUT_WEEKS) final_smape = smape(test_df['y'].values, fcst_holdout['yhat'].values) print(f"✨ Final SMAPE with best params: {final_smape:.2f}%") # 3.5) Plot comparison: Actual vs Optimized Forecast plt.figure(figsize=(15, 6)) # Plot 1: Backtest results plt.subplot(1, 2, 1) plt.plot(train_df['ds'], train_df['y'], label='train', color='C0', alpha=0.7) plt.plot(test_df['ds'], test_df['y'], label='actual', color='C1', linewidth=2) plt.plot(fcst_holdout['ds'], fcst_holdout['yhat'], label='forecast', color='C3', linewidth=2, alpha=0.6) plt.fill_between(fcst_holdout['ds'], fcst_holdout['yhat_lower'], fcst_holdout['yhat_upper'], color='C3', alpha=0.2, label='confidence') plt.axvline(test_df['ds'].min(), ls='--', color='grey', alpha=0.5) plt.legend() plt.title(f"Optimized Prophet Backtest\nSMAPE = {final_smape:.2f}%") plt.xticks(rotation=45) # Plot 2: Residuals plt.subplot(1, 2, 2) residuals = test_df['y'].values - fcst_holdout['yhat'].values plt.plot(fcst_holdout['ds'], residuals, 'o-', alpha=0.7, color='C2') plt.axhline(0, ls='--', color='grey', alpha=0.5) plt.title(f"Forecast Residuals\nMean: {np.mean(residuals):.1f}, Std: {np.std(residuals):.1f}") plt.xticks(rotation=45) plt.tight_layout() plt.show() fcst_backtest_avg = fcst_holdout.copy()Key Changes Made:
-
Frequency Adjustment:
- Changed
HOLDOUT_DAYStoHOLDOUT_WEEKS(52 weeks for a year) - Changed
freq='D'tofreq='W-MON'inmake_future_dataframe()
- Changed
-
Zero Value Handling:
- Modified the logic to look at the same week in previous years instead of the same weekday
-
Seasonality Settings:
- Kept
weekly_seasonality=Truesince we're working with weekly data - Kept
yearly_seasonality=Trueto capture annual patterns - Set
daily_seasonality=Falsesince we don't have daily data
- Kept
-
Plotting:
- The plotting code remains largely the same, but now it's showing weekly data
These changes should make your code work properly with weekly data grouped by Mondays. The Prophet model will now be trained and evaluated on a weekly basis rather than daily.
-
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.