This code is forecasting then number of mortgage contract, but it is grouping by weeks,…
- User
This code is forecasting then number of mortgage contract, but it is grouping by weeks, I need it to be divided proportionally on each day of week, by the proprtions according to the last year before forecast import pandas as pd import numpy as np from prophet import Prophet import optuna import matplotlib.pyplot as plt
- Load your data (ds = date, y = daily contract count)
Example data (replace with your actual data) df['ds'] = pd.to_datetime(df['ds'])
- Keep only business days (drop Sat/Sun)
df = df[df['ds'].dt.weekday < 5].reset_index(drop=True)
- Aggregate daily data into weekly sums
Add a 'week_start' column (Monday of each week) df['week_start'] = df['ds'] - pd.to_timedelta(df['ds'].dt.weekday, unit='D')
Group by week_start and sum the 'y' values weekly_df = df.groupby('week_start').agg({'y': 'sum'}).reset_index() weekly_df.columns = ['ds', 'y'] # Prophet expects 'ds' and 'y'
- Train / test split -> last 52 weeks as hold-out
HOLDOUT_WEEKS = 52 train_df = weekly_df.iloc[:-HOLDOUT_WEEKS] test_df = weekly_df.iloc[-HOLDOUT_WEEKS:]
- SMAPE helper
def smape(y_true, y_pred): y_true, y_pred = np.array(y_true), np.array(y_pred) return 100 * np.mean(2 * np.abs(y_pred - y_true) / (np.abs(y_true) + np.abs(y_pred) + 1e-8))
- Optuna objective
def objective(trial): params = { 'changepoint_prior_scale': trial.suggest_float('changepoint_prior_scale', 0.001, 0.5, log=True), 'seasonality_prior_scale': trial.suggest_float('seasonality_prior_scale', 0.01, 10, log=True), 'changepoint_range': trial.suggest_float('changepoint_range', 0.8, 0.95), 'n_changepoints': trial.suggest_int('n_changepoints', 10, 30), # Fewer changepoints for weekly data 'seasonality_mode': trial.suggest_categorical('seasonality_mode', ['additive', 'multiplicative']), 'interval_width': trial.suggest_float('interval_width', 0.90, 0.95) }
m = Prophet( daily_seasonality=False, weekly_seasonality=False, # Disable weekly seasonality (data is already weekly) yearly_seasonality=True, **params ) m.fit(train_df)
Forecast exactly the test period
future = pd.DataFrame({'ds': test_df['ds']}) fcst = m.predict(future)
Align and compute SMAPE
merged = pd.merge(test_df, fcst[['ds', 'yhat']], on='ds', how='inner') return smape(merged['y'], merged['yhat'])
- Run Bayesian optimisation
study = optuna.create_study(direction='minimize') study.optimize(objective, n_trials=30, show_progress_bar=True)
print("Best SMAPE:", round(study.best_value, 2)) print("Best params:", study.best_params)
- Final model on best parameters
best = study.best_params m_final = Prophet( daily_seasonality=False, weekly_seasonality=False, # Disable weekly seasonality (data is weekly) yearly_seasonality=True, **best ) m_final.fit(train_df)
future = pd.DataFrame({'ds': test_df['ds']}) fcst = m_final.predict(future)
merged = pd.merge(test_df, fcst[['ds', 'yhat', 'yhat_lower', 'yhat_upper']], on='ds')
fcst_backtest_CNT = merged.drop('y', axis = 1)
Assistant
To divide the forecasted weekly mortgage contracts proportionally across each day of the week, you can follow these steps. The proportions will be based on the historical daily data from the last year before the forecast period.
Step 1: Calculate Daily Proportions
First, you need to calculate the proportion of contracts for each day of the week based on the historical data. You will use the data from the year before the test period for this calculation.
# Calculate the start date of the last year before the test period last_year_start = test_df['ds'].min() - pd.DateOffset(years=1) # Filter data for the last year before the test period last_year_df = df[(df['ds'] >= last_year_start) & (df['ds'] < test_df['ds'].min())] # Calculate daily proportions for each day of the week daily_proportions = last_year_df.groupby(df['ds'].dt.weekday)['y'].sum() / last_year_df['y'].sum()Step 2: Adjust Forecast to Daily Level
Now, you will adjust the weekly forecast to the daily level using the proportions calculated above. First, you need to generate a daily date range for the forecast period.
# Generate a daily date range for the test period daily_dates = pd.date_range(start=test_df['ds'].min(), end=test_df['ds'].max() + pd.Timedelta(days=6), freq='D') # Filter out non-business days daily_dates = daily_dates[daily_dates.weekday < 5] # Create a DataFrame with these dates daily_forecast_df = pd.DataFrame({'ds': daily_dates}) # Map the weekday to the corresponding proportion daily_forecast_df['weekday'] = daily_forecast_df['ds'].dt.weekday daily_forecast_df['proportion'] = daily_forecast_df['weekday'].map(daily_proportions) # Calculate the week start for each day daily_forecast_df['week_start'] = daily_forecast_df['ds'] - pd.to_timedelta(daily_forecast_df['ds'].dt.weekday, unit='D') # Merge the forecast with the daily proportions fcst = fcst.merge(daily_forecast_df[['week_start', 'ds', 'proportion']], left_on='ds', right_on='week_start', how='left') # Adjust the forecast to daily level fcst['yhat_daily'] = fcst['yhat'] * fcst['proportion'] fcst['yhat_lower_daily'] = fcst['yhat_lower'] * fcst['proportion'] fcst['yhat_upper_daily'] = fcst['yhat_upper'] * fcst['proportion'] # Select relevant columns for daily forecast daily_forecast = fcst[['ds_y', 'yhat_daily', 'yhat_lower_daily', 'yhat_upper_daily']].rename(columns={'ds_y': 'ds'})Step 3: Review and Adjust
Review the daily forecast to ensure it aligns with your expectations. You might need to adjust the proportions or the forecasting model based on your specific requirements or observations.
Complete Code Adjustments
Here's how the complete adjusted code might look, focusing on the key adjustments:
# ... (Steps 1 through 7 remain largely the same) # Step 8: Adjustments for daily forecast best = study.best_params m_final = Prophet( daily_seasonality=False, weekly_seasonality=False, yearly_seasonality=True, **best ) m_final.fit(train_df) future = pd.DataFrame({'ds': test_df['ds']}) fcst = m_final.predict(future) # Calculate daily proportions last_year_start = test_df['ds'].min() - pd.DateOffset(years=1) last_year_df = df[(df['ds'] >= last_year_start) & (df['ds'] < test_df['ds'].min())] daily_proportions = last_year_df.groupby(df['ds'].dt.weekday)['y'].sum() / last_year_df['y'].sum() # Generate daily dates and adjust forecast daily_dates = pd.date_range(start=test_df['ds'].min(), end=test_df['ds'].max() + pd.Timedelta(days=6), freq='D') daily_dates = daily_dates[daily_dates.weekday < 5] daily_forecast_df = pd.DataFrame({'ds': daily_dates}) daily_forecast_df['weekday'] = daily_forecast_df['ds'].dt.weekday daily_forecast_df['proportion'] = daily_forecast_df['weekday'].map(daily_proportions) daily_forecast_df['week_start'] = daily_forecast_df['ds'] - pd.to_timedelta(daily_forecast_df['ds'].dt.weekday, unit='D') fcst = fcst.merge(daily_forecast_df[['week_start', 'ds', 'proportion']], left_on='ds', right_on='week_start', how='left') fcst['yhat_daily'] = fcst['yhat'] * fcst['proportion'] fcst['yhat_lower_daily'] = fcst['yhat_lower'] * fcst['proportion'] fcst['yhat_upper_daily'] = fcst['yhat_upper'] * fcst['proportion'] daily_forecast = fcst[['ds_y', 'yhat_daily', 'yhat_lower_daily', 'yhat_upper_daily']].rename(columns={'ds_y': 'ds'}) # Now, `daily_forecast` contains your daily forecast.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.