Skip to content
TheChatArchiveThe Chat Archive
gpt-4-1106-preview· openaiHugging Face Suzhen/CodeChat

read these wbsites and learn from them carefully, first one…

  1. User

    read these wbsites and learn from them carefully, first one https://github.com/Nixtla/statsforecast/blob/main/experiments/m3/src/experiment.py, second one https://nixtlaverse.nixtla.io/statsforecast/docs/models/autoces.html, third one https://nixtlaverse.nixtla.io/statsforecast/docs/models/autoets.html, fourth one https://nixtlaverse.nixtla.io/statsforecast/docs/models/dynamicoptimizedtheta.html, is my ensemble model code correct? from statsforecast import StatsForecast from statsforecast.models import AutoARIMA, AutoETS, AutoCES, DynamicOptimizedTheta from statsforecast.utils import ConformalIntervals import numpy as np import polars as pl

    Polars option to display all rows

    pl.Config.set_tbl_rows(None)

    Initialize the models

    models = [ AutoARIMA(season_length=52), AutoETS(season_length=52), AutoCES(season_length=52), DynamicOptimizedTheta(season_length=52) ]

    Initialize the StatsForecast model

    sf = StatsForecast(models=models, freq='1w', n_jobs=-1)

    Perform cross-validation with a step size of 1 to mimic an expanding window

    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True)

    Create an ensemble out-of-sample predictions by taking the median of the predictions from all models

    crossvalidation_df = crossvalidation_df.with_columns( pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOp']).mean().alias("Ensemble") )

    def wmape(y_true, y_pred): return np.abs(y_true - y_pred).sum() / np.abs(y_true).sum()

    Calculate the WMAPE for the ensemble model

    wmape_value = wmape(crossvalidation_df['y'], crossvalidation_df['Ensemble']) print('Average WMAPE for Ensemble: ', round(wmape_value, 4))

    Calculate the errors for the ensemble model

    errors = crossvalidation_df['y'] - crossvalidation_df['Ensemble']

    For an individual forecast

    individual_accuracy = 1 - (abs(crossvalidation_df['y'] - crossvalidation_df['Ensemble']) / crossvalidation_df['y']) individual_bias = (crossvalidation_df['Ensemble'] / crossvalidation_df['y']) - 1

    Add these calculations as new columns to your DataFrame

    crossvalidation_df = crossvalidation_df.with_columns([ individual_accuracy.alias("individual_accuracy"), individual_bias.alias("individual_bias") ])

    Print the individual accuracy and bias for each week

    for row in crossvalidation_df.to_dicts(): id = row['unique_id'] date = row['ds'] accuracy = row['individual_accuracy'] bias = row['individual_bias'] print(f"{id}, {date}, Individual Accuracy: {accuracy:.4f}, Individual Bias: {bias:.4f}")

    For groups of forecasts

    group_accuracy = 1 - (errors.abs().sum() / crossvalidation_df['y'].sum()) group_bias = (crossvalidation_df['Ensemble'].sum() / crossvalidation_df['y'].sum()) - 1

    Print the average group accuracy and group bias over all folds for the ensemble model

    print('Average Group Accuracy: ', round(group_accuracy, 4)) print('Average Group Bias: ', round(group_bias, 4))

    Fit the models on the entire dataset

    sf.fit(y_cl4)

    Instantiate the ConformalIntervals class

    prediction_intervals = ConformalIntervals()

    Generate 24 months forecasts

    forecasts_df = sf.forecast(h=4*24, prediction_intervals=prediction_intervals, level=[95], id_col='unique_id', sort_df=True)

    Create an ensemble forecast by taking the mean of the forecasts from all models

    forecasts_df = forecasts_df.with_columns( pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOp']).mean().alias("Ensemble") )

    forecasts_df = forecasts_df.with_columns([ pl.col("Ensemble").round().alias("Ensemble"), pl.col("Ensemble-lo-95").round().alias("Ensemble-lo-95"), pl.col("Ensemble-hi-95").round().alias("Ensemble-hi-95") ])

    Create an empty list

    forecasts_list = []

    Append each row to the list

    for row in forecasts_df.to_dicts(): forecasts_list.append(row)

    Print the list

    for forecast in forecasts_list: print(forecast)

  2. Assistant

    Your ensemble model code shows that you're trying to aggregate forecasts from multiple models using the median of their predictions and calculate accuracy and bias metrics for both individual and group forecasts using the StatsForecast library. However, while the intention is clear, there are a few potential issues and inaccuracies in the code that need to be addressed:

    1. Model names in the with_columns line: You have to ensure that the names of the columns used in pl.col() correspond to the output column names of each model prediction when creating the ensemble.

    2. Ensemble calculation: In your ensemble calculation, you're using .mean() to compute the ensemble but you mentioned taking the median in your description. You should use .median() if you want the median or update your description to accurately reflect the use of the mean in the ensemble calculation.

    3. Model Output Names Consistency: Make sure that the output forecast columns match the names of the models added to the StatsForecast instance. It's possible the names could be automatically generated based on the model classes you're using and might not directly match 'AutoARIMA', 'AutoETS', 'CES', 'DynamicOp'. You need to check the output DataFrame to confirm this.

    4. Conformal Intervals: If you use the ConformalIntervals class, you need to prepare it before using it to generate prediction intervals. You need to call fit on ConformalIntervals with the residuals (errors), which seems to be missing in your code.

    5. DataFrame Group Calculations: For calculating the group accuracy and group bias, you need to ensure that the calculations bring back a single value rather than a DataFrame series or column.

    6. Rounding Predictions: Rounding predictions and prediction intervals directly may not always be appropriate, especially if the predictions can include fractional values that are significant for the forecasting application.

    7. Polars Tables Config: You set a configuration for Polars to display all rows. This is typically not required for the functioning of the code, and it might be redundant if it's only meant to affect the display.

    Here's a slightly modified version of your code, fixing some of the issues identified:

    ...
    # Assuming y_cl4 is already defined DataFrame with 'unique_id' and 'y' columns.
    
    # Create an ensemble model with median prediction
    crossvalidation_df = crossvalidation_df.with_columns(
        pl.col(['model1', 'model2', 'model3', 'model4']).median().alias("Ensemble")
        # Change 'model1', 'model2', etc., to the actual model output column names
    )
    
    ...
    # For group calculations, ensure to aggregate to a single value, not a DataFrame
    group_accuracy = 1 - (errors.abs().sum()[0] / crossvalidation_df['y'].sum()[0])
    group_bias = (crossvalidation_df['Ensemble'].sum()[0] / crossvalidation_df['y'].sum()[0]) - 1
    # Use [0] to extract the value from the polars' Series
    
    ...
    # Check how the ConformalIntervals class should be used with your version of statsforecast
    
    ...
    # The output of the forecasts may not need to be rounded or should be treated case-by-case depending on significance
    
    ...
    # There is no need to convert the DataFrame into a list as polars already provides an efficient way to handle data
    forecasts_df.collect()  # Collect data if it's lazy
    
    # Now you can print or use forecasts_df directly
    print(forecasts_df)
    
    # Overall, be sure to follow the precise output naming conventions and API usage from the `StatsForecast` docs.
    

    Make sure to test each part of the code and confirm that the actual output of your models corresponds to the column names and behavior that you are expecting. Also, remember to review the documentation and examples provided for the library to ensure full compatibility with the library's functionality and data structures.

  3. Assistant

    It seems you haven't included a question or context in your latest input. Could you please provide more detail or clarify your question? I'm here to help!

  4. User

    error 8170 self, 8171 *exprs: IntoExpr | Iterable[IntoExpr], 8172 **named_exprs: IntoExpr, 8173 ) -> DataFrame: 8174 """ 8175 Add columns to this DataFrame. 8176 (...) 8313 └─────┴──────┴─────────────┘ 8314 """ -> 8315 return self.lazy().with_columns(*exprs, **named_exprs).collect(_eager=True)

    File ~/anaconda3/lib/python3.10/site-packages/polars/lazyframe/frame.py:1940, in LazyFrame.collect(self, type_coercion, predicate_pushdown, projection_pushdown, simplify_expression, slice_pushdown, comm_subplan_elim, comm_subexpr_elim, no_optimization, streaming, background, _eager) 1937 if background: 1938 return InProcessQuery(ldf.collect_concurrently()) -> 1940 return wrap_df(ldf.collect())

    ColumnNotFoundError: DynamicOp

    Error originated just after this operation: DF ["unique_id", "ds", "cutoff", "y"]; PROJECT */8 COLUMNS; SELECTION: "None" but print(crossvalidation_df) unique_id ┆ ds ┆ cutoff ┆ y ┆ AutoARIMA ┆ AutoETS ┆ CES ┆ DynamicOp │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ timizedTh │ │ str ┆ datetime[μ ┆ datetime[μ ┆ f32 ┆ f32 ┆ f32 ┆ f32 ┆ eta │ │ ┆ s] ┆ s] ┆ ┆ ┆ ┆ ┆ --- │ │ ┆ ┆ ┆ ┆ ┆ ┆ ┆ f32 │ ╞═══════════╪════════════╪════════════╪═════════╪════════════╪════════════╪════════════╪═══════════╡ │ 1108284 ┆ 2023-04-17 ┆ 2023-04-10 ┆ 25394.0 ┆ 27515.5644 ┆ 27464.0410 ┆ 28000.5351 ┆ 27720.164 │ │ ┆ 00:00:00 ┆ 00:00:00 ┆ ┆ 53 ┆ 16 ┆ 56 ┆ 062 │ │ 1108284 ┆ 2023-04-24 ┆ 2023-04-10 ┆ 26566.0 ┆ 27515.5644 ┆ 27464.0410 ┆ 10834.9970 ┆ 27720.164 │ │ ┆ 00:00:00 ┆ 00:00:00 ┆ ┆ 53 ┆ 16 ┆ 7 ┆ 062 │ │ 1108284 ┆ 2023-05-01 ┆ 2023-04-10 ┆ 17182.0 ┆ 27515.5644 ┆ 27464.0410 ┆ 27656.6699 ┆ 27720.164 │ │ ┆ 00:00:00 ┆ 00:00:00 ┆ ┆ 53 ┆ 16 ┆ 22 ┆ 062 │ │ 1108284 ┆ 2023-05-08 ┆ 2023-04-10 ┆ 26975.0 ┆ 27515.5644 ┆ 27464.0410 ┆ 12330.1611 ┆ 27720.164 │ │ ┆ 00:00:00 ┆ 00:00:00 ┆ ┆ 53 ┆ 16 ┆ 33 ┆ 062 │ │ 1108284 ┆ 2023-05-15 ┆ 2023-04-10 ┆ 25899.0 ┆ 27515.5644 ┆ 27464.0410 ┆ 23127.0449 ┆ 27720.164 │ │ ┆ 00:00:00 ┆ 00:00:00 ┆ ┆ 53 ┆ 16 ┆ 22 ┆ 062 │ │ … ┆ … ┆ … ┆ … ┆ … ┆ … ┆ … ┆ … │ │ 6922332 ┆ 2023-10-30 ┆ 2023-08-07 ┆ 24201.0 ┆ 20886.6171 ┆ 21016.8125 ┆ 20105.2109 ┆ 20876.681 │ │ ┆ 00:00:00 ┆ 00:00:00 ┆ ┆ 88 ┆ ┆ 38 ┆ 641 │ │ 6922332 ┆ 2023-11-06 ┆ 2023-08-07 ┆ 21469.0 ┆ 20886.6171 ┆ 21016.8125 ┆ 26514.2363 ┆ 20876.681 │

  5. Assistant

    The error ColumnNotFoundError: DynamicOp suggests that when you attempted to add or manipulate columns in the Polars DataFrame, one of the specified columns, "DynamicOp", was not found in the existing DataFrame.

    From the partial output provided, it looks like the actual column name for the forecast from the Dynamic Optimized Theta model is "DynamicOp timizedTh eta" instead of just "DynamicOp". This might be due to some formatting or column naming issue.

    To fix this, you need to ensure that you correctly reference the full name of the column. Here is the corrected line of code based on the displayed column names:

    crossvalidation_df = crossvalidation_df.with_columns(
        pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOp timizedTh eta']).median().alias("Ensemble")
    )
    

    However, if the space between "DynamicOp" and "timizedTh eta" was not intentional, and it looks like part of the column name got unexpectedly split, this could be due to the way the DataFrame is being displayed or an issue with how the DataFrame was created.

    Here are some steps to solve this:

    1. Print the column names of the DataFrame directly to check for any unexpected splitting or formatting issues.
    print(crossvalidation_df.columns)
    
    1. Use the exact column names obtained in the previous step when referencing your model output columns.

    2. Update the ensemble calculation line with the correct model output names:

    crossvalidation_df = crossvalidation_df.with_columns(
        pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).median().alias("Ensemble")
    )
    

    Replace 'DynamicOptimizedTheta' with the actual column name corresponding to the model's output.

    1. In case the space in the column name is indeed unintentional and was caused during data manipulation, you might need to sanitize or fix the column names immediately after the forecasting operation, before any further manipulation.
  6. User

    8169 def with_columns( 8170 self, 8171 *exprs: IntoExpr | Iterable[IntoExpr], 8172 **named_exprs: IntoExpr, 8173 ) -> DataFrame: 8174 """ 8175 Add columns to this DataFrame. 8176 (...) 8313 └─────┴──────┴─────────────┘ 8314 """ -> 8315 return self.lazy().with_columns(*exprs, **named_exprs).collect(_eager=True)

    File ~/anaconda3/lib/python3.10/site-packages/polars/lazyframe/frame.py:1940, in LazyFrame.collect(self, type_coercion, predicate_pushdown, projection_pushdown, simplify_expression, slice_pushdown, comm_subplan_elim, comm_subexpr_elim, no_optimization, streaming, background, _eager) 1937 if background: 1938 return InProcessQuery(ldf.collect_concurrently()) -> 1940 return wrap_df(ldf.collect())

    ComputeError: The name: 'Ensemble' passed to LazyFrame.with_columns is duplicate

    Error originated just after this operation: DF ["unique_id", "ds", "cutoff", "y"]; PROJECT */8 COLUMNS; SELECTION: "None" but print(crossvalidation_df) unique_id ┆ ds ┆ cutoff ┆ y ┆ AutoARIMA ┆ AutoETS ┆ CES ┆ DynamicOp │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ timizedTh │ │ str ┆ datetime[μ ┆ datetime[μ ┆ f32 ┆ f32 ┆ f32 ┆ f32 ┆ eta │ │ ┆ s] ┆ s] ┆ ┆ ┆ ┆ ┆ --- │ │ ┆ ┆ ┆ ┆ ┆ ┆ ┆ f32 │ ╞═══════════╪════════════╪════════════╪═════════╪════════════╪════════════╪════════════╪═══════════╡ │ 1108284 ┆ 2023-04-17 ┆ 2023-04-10 ┆ 25394.0 ┆ 27515.5644 ┆ 27464.0410 ┆ 28000.5351 ┆ 27720.164 │ │ ┆ 00:00:00 ┆ 00:00:00 ┆ ┆ 53 ┆ 16 ┆ 56 ┆ 062 │ │ 1108284 ┆ 2023-04-24 ┆ 2023-04-10 ┆ 26566.0 ┆ 27515.5644 ┆ 27464.0410 ┆ 10834.9970 ┆ 27720.164 │ │ ┆ 00:00:00 ┆ 00:00:00 ┆ ┆ 53 ┆ 16 ┆ 7 ┆ 062 │ │ 1108284 ┆ 2023-05-01 ┆ 2023-04-10 ┆ 17182.0 ┆ 27515.5644 ┆ 27464.0410 ┆ 27656.6699 ┆ 27720.164 │ │ ┆ 00:00:00 ┆ 00:00:00 ┆ ┆ 53 ┆ 16 ┆ 22 ┆ 062 │ │ 1108284 ┆ 2023-05-08 ┆ 2023-04-10 ┆ 26975.0 ┆ 27515.5644 ┆ 27464.0410 ┆ 12330.1611 ┆ 27720.164 │ │ ┆ 00:00:00 ┆ 00:00:00 ┆ ┆ 53 ┆ 16 ┆ 33 ┆ 062 │ │ 1108284 ┆ 2023-05-15 ┆ 2023-04-10 ┆ 25899.0 ┆ 27515.5644 ┆ 27464.0410 ┆ 23127.0449 ┆ 27720.164 │ │ ┆ 00:00:00 ┆ 00:00:00 ┆ ┆ 53 ┆ 16 ┆ 22 ┆ 062 │ │ … ┆ … ┆ … ┆ … ┆ … ┆ … ┆ … ┆ … │ │ 6922332 ┆ 2023-10-30 ┆ 2023-08-07 ┆ 24201.0 ┆ 20886.6171 ┆ 21016.8125 ┆ 20105.2109 ┆ 20876.681 │

  7. Assistant

    The ComputeError: The name: 'Ensemble' passed to 'LazyFrame.with_columns' is duplicate error indicates that there is an attempt to add a new column with the name 'Ensemble' to the DataFrame, but a column with that name already exists. Polars DataFrames cannot have duplicate column names.

    To resolve the error, you should do one of the following:

    • Rename the new 'Ensemble' column you are trying to add if your intention was to have a separate column.
    • If you intended to overwrite the existing 'Ensemble' column, you should first drop the existing 'Ensemble' column and then add the new one.

    Here is how you can address these situations:

    Renaming the New Column

    If you want to keep the existing 'Ensemble' column and add the results of a new calculation as a different column, give the new column a unique name. For example:

    crossvalidation_df = crossvalidation_df.with_columns(
        pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOp timizedTh eta']).median().alias("NewEnsemble")
    )
    

    Overwriting the Existing Column

    If you want to update the existing 'Ensemble' column with new values:

    # Drop the existing 'Ensemble' column before adding the new one
    crossvalidation_df = crossvalidation_df.drop('Ensemble')
    
    # Now, add a new 'Ensemble' column with updated values
    crossvalidation_df = crossvalidation_df.with_columns(
        pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOp timizedTh eta']).median().alias("Ensemble")
    )
    

    Further Validate the Column Names

    Before making these changes, ensure that you have the correct model output column names. From the given output, it looks like the column names may include spaces or other unintended formatting. To verify, you can print the actual column names:

    print(crossvalidation_df.columns)
    

    And then use the exact names as printed in the appropriate pl.col(...) constructs within your calls to with_columns. If the output still shows the names split by spaces (like "DynamicOp timizedTh eta"), you may want to consider renaming them to something more manageable:

    # Example for renaming the 'DynamicOp timizedTh eta' column
    crossvalidation_df = crossvalidation_df.rename({'DynamicOp timizedTh eta': 'DynamicOptimizedTheta'})
    

    Please perform the necessary steps according to your specific requirements.

  8. User

    print(crossvalidation_df.columns) ['unique_id', 'ds', 'cutoff', 'y', 'AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta'] so there's no ensemble column at all? this is the ensemble model and what we did is correct right? on our code

  9. Assistant

    Based on the output of print(crossvalidation_df.columns), it seems that there isn't currently a column named 'Ensemble' in your DataFrame. The column names are:

    ['unique_id', 'ds', 'cutoff', 'y', 'AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']
    

    This suggests that there might be a discrepancy between what you expected and the actual state of the DataFrame at the time the error occurred. If you are sure that 'Ensemble' should not already exist in the DataFrame at the point where you're adding it, but you're still getting an error about a duplicate column name, it could be that the error is being thrown in a different part of your code, or the DataFrame is in an unexpected state.

    Given the correct column names, let's re-visit your code for calculating the 'Ensemble' median forecasts:

    crossvalidation_df = crossvalidation_df.with_columns(
        pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).median().alias("Ensemble")
    )
    

    This code should now work correctly, as it will create a new column named 'Ensemble' with the median of the forecasts from the other models (assuming the rest of the code is correct and 'Ensemble' does not exist in crossvalidation_df when this code is executed).

    Please ensure that this piece of code is not being run more than once, or if it's part of a loop or function, ensure that 'Ensemble' is not added in a previous iteration or a previous call.

    If the error persists, review the logic flow and ensure this line is only executed once, or explicitly drop the 'Ensemble' column before attempting to add it again if it is part of an iterative process or expected to be executed multiple times.

    Additionally, since we now know the correct column names, ensure that any previous mentions of 'DynamicOp' or 'DynamicOp timizedTh eta' are replaced with 'DynamicOptimizedTheta' to maintain consistency.

  10. User

    I'm gonna restart kernel to make sure

  11. Assistant

    Restarting the kernel can often help resolve issues with the state of the notebook or your environment. It ensures that any variables, data frames, or imported modules are cleared out, giving you a fresh start. This can help eliminate any discrepancies or residual state that may be causing issues with your code execution.

    Here's what you should do now:

    1. Restart the kernel/environment: Depending on your development environment, you will find an option to restart the kernel. In Jupyter, for example, you can find it under the 'Kernel' menu.

    2. Rerun the imports: Make sure to first run the cell that contains all of your import statements to ensure that all necessary packages are loaded into memory.

    3. Rerun the setup code: This includes any data preprocessing, defining functions or variables, and any other initial setup that your code relies on.

    4. Rerun the code to add the 'Ensemble' column: Now that you have a clean start, try running the code again that adds the 'Ensemble' median forecasts column to your DataFrame:

    crossvalidation_df = crossvalidation_df.with_columns(
        pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).median().alias("Ensemble")
    )
    

    Restarting the kernel and rerunning the code will help you determine if there was a transient issue or a persistent one that needs to be debugged further. If the problem was transient and due to some sort of state within your session, it should be resolved after a restart; otherwise, you may need to debug further by carefully examining the code for logical errors or incorrect flow of execution.

  12. User

    still the same error crossvalidation_df after kernel restarted and nothing is in crossvalidation_df, I checked before running the model, lets check crossvalidation_df now to see what's in there after the error print(crossvalidation_df.columns)

    ['unique_id', 'ds', 'cutoff', 'y', 'AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta'] print(crossvalidation_df) unique_id ┆ ds ┆ cutoff ┆ y ┆ AutoARIMA ┆ AutoETS ┆ CES ┆ DynamicOp │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ timizedTh │ │ str ┆ datetime[μ ┆ datetime[μ ┆ f32 ┆ f32 ┆ f32 ┆ f32 ┆ eta │ │ ┆ s] ┆ s] ┆ ┆ ┆ ┆ ┆ --- │ │ ┆ ┆ ┆ ┆ ┆ ┆ ┆ f32 │ ╞═══════════╪════════════╪════════════╪═════════╪════════════╪════════════╪════════════╪═══════════╡ │ 1108284 ┆ 2023-04-17 ┆ 2023-04-10 ┆ 25394.0 ┆ 27515.5644 ┆ 27464.0410 ┆ 28000.5351 ┆ 27720.164 │ │ ┆ 00:00:00 ┆ 00:00:00 ┆ ┆ 53 ┆ 16 ┆ 56 ┆ 062 │ │ 1108284 ┆ 2023-04-24 ┆ 2023-04-10 ┆ 26566.0 ┆ 27515.5644 ┆ 27464.0410 ┆ 10834.9970 ┆ 27720.164 │ │ ┆ 00:00:00 ┆ 00:00:00 ┆ ┆ 53 ┆ 16 ┆ 7 ┆ 062 │ │ 1108284 ┆ 2023-05-01 ┆ 2023-04-10 ┆ 17182.0 ┆ 27515.5644 ┆ 27464.0410 ┆ 27656.6699 ┆ 27720.164 │ │ ┆ 00:00:00 ┆ 00:00:00 ┆ ┆ 53 ┆ 16 ┆ 22 ┆ 062 │ │ 1108284 ┆ 2023-05-08 ┆ 2023-04-10 ┆ 26975.0 ┆ 27515.5644 ┆ 27464.0410 ┆ 12330.1611 ┆ 27720.164 │ could it be .alias("Ensemble") not dont correctly? this is to create an ensemble model on polars

  13. Assistant

    The information you have provided indicates that the crossvalidation_df DataFrame does not currently have an "Ensemble" column before running the operation to add it, and after running the operation, you are receiving an error stating that the 'Ensemble' column already exists which is contradictory.

    The problem seems to stem from adding a new column with the .alias("Ensemble") operation, which triggers a ComputeError complaining about duplicate column names even though 'Ensemble' does not appear to exist in the DataFrame before this operation.

    Given the situation, let's try the following steps:

    1. Make sure that there are no other parts of your code that might be causing an 'Ensemble' column to be generated before or during the with_columns() call. A thorough review of your code before this point will help ensure there are no unexpected operations adding an 'Ensemble' column.

    2. Verify that the DataFrame just before the operation does not have an 'Ensemble' column already:

    print(crossvalidation_df.columns)
    

    If 'Ensemble' is indeed not in the list, then you can proceed with the operation:

    1. Attempt to add the 'Ensemble' column again:
    crossvalidation_df = crossvalidation_df.with_columns(
        pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).median().alias('Ensemble')
    )
    

    Having confirmed that 'Ensemble' doesn't exist beforehand should avoid the duplicate column error.

    1. If the error persists despite the above steps, you can try adding the column in a different way, e.g., by simply extending the DataFrame with the new column:
    ensemble_median = crossvalidation_df.select([
        pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).median()
    ]).alias('Ensemble') # This gives you the median values in a separate DataFrame.
    
    crossvalidation_df = crossvalidation_df.hstack(ensemble_median) # Stack horizontally with original DataFrame.
    

    Combining these steps with resetting your environment (restarting kernel and rerunning the code) should help identify where the issue is occurring. If the problem still occurs, there might be a bug or unexpected behavior within the Polars library or the environment you're using, which may require further investigation or reporting to the library's maintainers.

  14. User

    check my code again from statsforecast import StatsForecast from statsforecast.models import AutoARIMA, AutoETS, AutoCES, DynamicOptimizedTheta from statsforecast.utils import ConformalIntervals import numpy as np import polars as pl

    Polars option to display all rows

    pl.Config.set_tbl_rows(None)

    Initialize the models

    models = [ AutoARIMA(season_length=52), AutoETS(season_length=52), AutoCES(season_length=52), DynamicOptimizedTheta(season_length=52) ]

    Initialize the StatsForecast model

    sf = StatsForecast(models=models, freq='1w', n_jobs=-1)

    Perform cross-validation with a step size of 1 to mimic an expanding window

    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True)

    Create an ensemble out-of-sample predictions by taking the median of the predictions from all models

    crossvalidation_df = crossvalidation_df.with_columns( pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias("Ensemble") )

    def wmape(y_true, y_pred): return np.abs(y_true - y_pred).sum() / np.abs(y_true).sum()

    Calculate the WMAPE for the ensemble model

    wmape_value = wmape(crossvalidation_df['y'], crossvalidation_df['Ensemble']) print('Average WMAPE for Ensemble: ', round(wmape_value, 4))

    Calculate the errors for the ensemble model

    errors = crossvalidation_df['y'] - crossvalidation_df['Ensemble']

    For an individual forecast

    individual_accuracy = 1 - (abs(crossvalidation_df['y'] - crossvalidation_df['Ensemble']) / crossvalidation_df['y']) individual_bias = (crossvalidation_df['Ensemble'] / crossvalidation_df['y']) - 1

    Add these calculations as new columns to your DataFrame

    crossvalidation_df = crossvalidation_df.with_columns([ individual_accuracy.alias("individual_accuracy"), individual_bias.alias("individual_bias") ])

    Print the individual accuracy and bias for each week

    for row in crossvalidation_df.to_dicts(): id = row['unique_id'] date = row['ds'] accuracy = row['individual_accuracy'] bias = row['individual_bias'] print(f"{id}, {date}, Individual Accuracy: {accuracy:.4f}, Individual Bias: {bias:.4f}")

    For groups of forecasts

    group_accuracy = 1 - (errors.abs().sum() / crossvalidation_df['y'].sum()) group_bias = (crossvalidation_df['Ensemble'].sum() / crossvalidation_df['y'].sum()) - 1

    Print the average group accuracy and group bias over all folds for the ensemble model

    print('Average Group Accuracy: ', round(group_accuracy, 4)) print('Average Group Bias: ', round(group_bias, 4))

    Fit the models on the entire dataset

    sf.fit(y_cl4)

    Instantiate the ConformalIntervals class

    prediction_intervals = ConformalIntervals()

    Generate 24 months forecasts

    forecasts_df = sf.forecast(h=4*24, prediction_intervals=prediction_intervals, level=[95], id_col='unique_id', sort_df=True)

    Create an ensemble forecast by taking the mean of the forecasts from all models

    forecasts_df = forecasts_df.with_columns( pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias("Ensemble") )

    forecasts_df = forecasts_df.with_columns([ pl.col("Ensemble").round().alias("Ensemble"), pl.col("Ensemble-lo-95").round().alias("Ensemble-lo-95"), pl.col("Ensemble-hi-95").round().alias("Ensemble-hi-95") ])

    Create an empty list

    forecasts_list = []

    Append each row to the list

    for row in forecasts_df.to_dicts(): forecasts_list.append(row)

    Print the list

    for forecast in forecasts_list: print(forecast)

  15. Assistant

    From the code you have provided, I've noticed a potential issue in the final block where you are rounding the 'Ensemble' forecast and its prediction intervals and then subsequently using .alias("Ensemble") again. This could potentially cause a conflict with the dataframe column names since 'Ensemble' column already exists from the previous operation:

    # Create an ensemble forecast by taking the mean of the forecasts from all models
    forecasts_df = forecasts_df.with_columns(
        pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias("Ensemble")
    )
    
    forecasts_df = forecasts_df.with_columns([
        pl.col("Ensemble").round().alias("Ensemble"),  # This line might be causing the issue
        pl.col("Ensemble-lo-95").round().alias("Ensemble-lo-95"),
        pl.col("Ensemble-hi-95").round().alias("Ensemble-hi-95")
    ])
    

    The second time you call .alias("Ensemble"), you are trying to create a column with the same name as an existing column, which is likely to cause the error you are getting (assuming the rest of the code is executed correctly).

    A potential fix would be to round the 'Ensemble' column without assigning the rounded values to the same column name. Instead, you could add a new column with a different name or overwrite the existing 'Ensemble' column after rounding:

    # Calculate the rounded ensemble forecast values
    rounded_ensemble = forecasts_df.select(
        pl.col("Ensemble").round().alias("RoundedEnsemble")  # Use a different alias if you want to keep the original column
    )
    
    # Join the rounded ensemble forecast back to the forecasts_df
    forecasts_df = forecasts_df.hstack(rounded_ensemble)
    
    # Now you can also round the prediction intervals and add them to forecasts_df
    forecasts_df = forecasts_df.with_columns([
        pl.col("Ensemble-lo-95").round().alias("Ensemble-lo-95"),
        pl.col("Ensemble-hi-95").round().alias("Ensemble-hi-95")
    ])
    

    Make sure you're not duplicating the 'Ensemble' column names elsewhere in your code. Each alias used in .with_columns or .alias must refer to a unique column name that does not already exist in the DataFrame unless you intend to replace the column completely.

    After making the above change, please re-run your entire code ensuring that the kernel is restarted, and verify if the error has been resolved.

  16. User

    so what should we do? I thought they are 2 different dataframe, one is crossvalidation_df and the other is forecasts_df

  17. Assistant

    You're correct, crossvalidation_df and forecasts_df are two different dataframes, and adding an 'Ensemble' column to each should not cause a conflict as long as they are not part of the same dataframe. Given the context, it appears your code should work for creating an 'Ensemble' column on both dataframes independently. The error you encountered related to a duplicate 'Ensemble' column might have originated from running some part of the code multiple times without restarting the kernel or clearing the dataframe.

    Here's what you should do to ensure that the code runs correctly:

    1. Restart the kernel to clear the environment (you've already done this, but it's good to do this again to start fresh).

    2. Refresh the data by reloading it, ensure you have fresh data in y_cl4.

    3. Run the entire code in sequence without running any part twice. This is important because running the block that adds the 'Ensemble' column multiple times will lead to the duplicate column error.

    4. Make sure to run the relevant parts only once. The parts where you're using .alias() to name columns should only be run once for each dataframe unless you're intentionally overwriting a column.

    Assuming there are no other errors in your data or code logic, and you run each piece of code once, you should not encounter the duplicate column error.

    The creation of the ensemble model and subsequent analysis should work as intended if each step is executed properly:

    • Creating the ensemble forecast by taking the median/mean of predictions from all models in both crossvalidation_df and forecasts_df.
    • Calculating accuracy, bias, and other statistics for crossvalidation_df.
    • Creating prediction intervals (ConformalIntervals), fitting the model (sf.fit), and forecasting (sf.forecast) for forecasts_df.

    Please proceed with running your complete code in a fresh, restarted environment without executing the same dataframe manipulations on 'Ensemble' more than once.

  18. User

    is this better nwo from statsforecast import StatsForecast from statsforecast.models import AutoARIMA, AutoETS, AutoCES, DynamicOptimizedTheta from statsforecast.utils import ConformalIntervals import numpy as np import polars as pl

    Polars option to display all rows

    pl.Config.set_tbl_rows(None)

    Initialize the models

    models = [ AutoARIMA(season_length=52), AutoETS(season_length=52), AutoCES(season_length=52), DynamicOptimizedTheta(season_length=52) ]

    Initialize the StatsForecast model

    sf = StatsForecast(models=models, freq='1w', n_jobs=-1)

    Perform cross-validation with a step size of 1 to mimic an expanding window

    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True)

    Create an ensemble out-of-sample predictions by taking the median of the predictions from all models

    crossvalidation_df = crossvalidation_df.with_columns( pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias("Ensemble") )

    def wmape(y_true, y_pred): return np.abs(y_true - y_pred).sum() / np.abs(y_true).sum()

    Calculate the WMAPE for the ensemble model

    wmape_value = wmape(crossvalidation_df['y'], crossvalidation_df['Ensemble']) print('Average WMAPE for Ensemble: ', round(wmape_value, 4))

    Calculate the errors for the ensemble model

    errors = crossvalidation_df['y'] - crossvalidation_df['Ensemble']

    For an individual forecast

    individual_accuracy = 1 - (abs(crossvalidation_df['y'] - crossvalidation_df['Ensemble']) / crossvalidation_df['y']) individual_bias = (crossvalidation_df['Ensemble'] / crossvalidation_df['y']) - 1

    Add these calculations as new columns to your DataFrame

    crossvalidation_df = crossvalidation_df.with_columns([ individual_accuracy.alias("individual_accuracy"), individual_bias.alias("individual_bias") ])

    Print the individual accuracy and bias for each week

    for row in crossvalidation_df.to_dicts(): id = row['unique_id'] date = row['ds'] accuracy = row['individual_accuracy'] bias = row['individual_bias'] print(f"{id}, {date}, Individual Accuracy: {accuracy:.4f}, Individual Bias: {bias:.4f}")

    For groups of forecasts

    group_accuracy = 1 - (errors.abs().sum() / crossvalidation_df['y'].sum()) group_bias = (crossvalidation_df['Ensemble'].sum() / crossvalidation_df['y'].sum()) - 1

    Print the average group accuracy and group bias over all folds for the ensemble model

    print('Average Group Accuracy: ', round(group_accuracy, 4)) print('Average Group Bias: ', round(group_bias, 4))

    Fit the models on the entire dataset

    sf.fit(y_cl4)

    Instantiate the ConformalIntervals class

    prediction_intervals = ConformalIntervals()

    Generate 24 months forecasts

    forecasts_df = sf.forecast(h=4*24, prediction_intervals=prediction_intervals, level=[95], id_col='unique_id', sort_df=True)

    Create an ensemble forecast by taking the mean of the forecasts from all models

    forecasts_df = forecasts_df.with_columns( pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias("EnsembleForecast") )

    forecasts_df = forecasts_df.with_columns([ pl.col("EnsembleForecast").round().alias("EnsembleForecast"), pl.col("Ensemble-lo-95").round().alias("Ensemble-lo-95"), pl.col("Ensemble-hi-95").round().alias("Ensemble-hi-95") ])

    Create an empty list

    forecasts_list = []

    Append each row to the list

    for row in forecasts_df.to_dicts(): forecasts_list.append(row)

    Print the list

    for forecast in forecasts_list: print(forecast)

  19. Assistant

    Your adjusted code looks much better, and changing the alias for the ensemble forecast to "EnsembleForecast" should prevent potential issues with column name conflicts. However, there's another area in your code that still needs attention:

    When you are trying to round the 'EnsembleForecast' and immediately reassigning it with the same name, you might run into problems because this overwrites the previous column. Instead, consider separating the rounding and the column renaming to avoid confusion. Here's how you could do it:

    # Create an ensemble forecast by taking the mean of the forecasts from all models
    forecasts_df = forecasts_df.with_columns(
        pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias("EnsembleForecast")
    )
    
    # Round the ensemble forecast and the prediction intervals, rename with unique aliases to avoid overwrite
    forecasts_df = forecasts_df.with_columns([
        pl.col("EnsembleForecast").round().alias("RoundedEnsembleForecast"),
        pl.col("EnsembleForecast-lo-95").round().alias("RoundedEnsembleForecast-lo-95"),
        pl.col("EnsembleForecast-hi-95").round().alias("RoundedEnsembleForecast-hi-95")
    ])
    
    # Now, you won't need to create a separate list unless you have a specific use case for it
    # Instead, you can simply print the DataFrame or convert it to a list if necessary
    print(forecasts_df.to_dicts())
    

    This approach ensures that you are not overwriting an existing column while still providing you with the rounded values that you desire, and each column has a distinct name that reflects its contents.

    Also, please ensure that "EnsembleForecast-lo-95" and "EnsembleForecast-hi-95" columns exist in forecasts_df. If they don't exist, you would need to generate these confidence intervals separately and add them to the DataFrame before rounding and renaming them.

    Your wmape function and the calculation of errors, individual accuracy, and bias look appropriate. Make sure the y_cl4 DataFrame is correctly defined and loaded before running the StatsForecast functions.

    Finally, confirm that you are only running the code once after a kernel restart, to ensure that there are no residues from previous computations that could cause unexpected behavior.

  20. User

    how about now? from statsforecast import StatsForecast from statsforecast.models import AutoARIMA, AutoETS, AutoCES, DynamicOptimizedTheta from statsforecast.utils import ConformalIntervals import numpy as np import polars as pl

    Polars option to display all rows

    pl.Config.set_tbl_rows(None)

    Initialize the models

    models = [ AutoARIMA(season_length=52), AutoETS(season_length=52), AutoCES(season_length=52), DynamicOptimizedTheta(season_length=52) ]

    Initialize the StatsForecast model

    sf = StatsForecast(models=models, freq='1w', n_jobs=-1)

    Perform cross-validation with a step size of 1 to mimic an expanding window

    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True)

    Create an ensemble out-of-sample predictions by taking the median of the predictions from all models

    crossvalidation_df = crossvalidation_df.with_columns( pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias("Ensemble") )

    def wmape(y_true, y_pred): return np.abs(y_true - y_pred).sum() / np.abs(y_true).sum()

    Calculate the WMAPE for the ensemble model

    wmape_value = wmape(crossvalidation_df['y'], crossvalidation_df['Ensemble']) print('Average WMAPE for Ensemble: ', round(wmape_value, 4))

    Calculate the errors for the ensemble model

    errors = crossvalidation_df['y'] - crossvalidation_df['Ensemble']

    For an individual forecast

    individual_accuracy = 1 - (abs(crossvalidation_df['y'] - crossvalidation_df['Ensemble']) / crossvalidation_df['y']) individual_bias = (crossvalidation_df['Ensemble'] / crossvalidation_df['y']) - 1

    Add these calculations as new columns to your DataFrame

    crossvalidation_df = crossvalidation_df.with_columns([ individual_accuracy.alias("individual_accuracy"), individual_bias.alias("individual_bias") ])

    Print the individual accuracy and bias for each week

    for row in crossvalidation_df.to_dicts(): id = row['unique_id'] date = row['ds'] accuracy = row['individual_accuracy'] bias = row['individual_bias'] print(f"{id}, {date}, Individual Accuracy: {accuracy:.4f}, Individual Bias: {bias:.4f}")

    For groups of forecasts

    group_accuracy = 1 - (errors.abs().sum() / crossvalidation_df['y'].sum()) group_bias = (crossvalidation_df['Ensemble'].sum() / crossvalidation_df['y'].sum()) - 1

    Print the average group accuracy and group bias over all folds for the ensemble model

    print('Average Group Accuracy: ', round(group_accuracy, 4)) print('Average Group Bias: ', round(group_bias, 4))

    Fit the models on the entire dataset

    sf.fit(y_cl4)

    Instantiate the ConformalIntervals class

    prediction_intervals = ConformalIntervals()

    Generate 24 months forecasts

    forecasts_df = sf.forecast(h=4*24, prediction_intervals=prediction_intervals, level=[95], id_col='unique_id', sort_df=True)

    Create an ensemble forecast by taking the mean of the forecasts from all models

    forecasts_df = forecasts_df.with_columns( pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias("EnsembleForecast") )

    forecasts_df = forecasts_df.with_columns([ pl.col("EnsembleForecast").round().alias("CombinedForecast"), pl.col("Ensemble-lo-95").round().alias("CombinedForecast-lo-95"), pl.col("Ensemble-hi-95").round().alias("CombinedForecast-hi-95") ])

    print(forecasts_df)

  21. Assistant

    The latest iteration of your code looks good, and using unique aliases for the rounded columns in forecasts_df is the correct approach to avoid duplicate column names. Your code now successfully creates an ensemble forecast by averaging predictions from several models and then rounds these predictions without overwriting any columns.

    Here are some final checks and tips:

    • Confirm that the columns "Ensemble-lo-95" and "Ensemble-hi-95" exist. If they do not, you need to ensure that the prediction_intervals calculate these and that they are included in your forecast DataFrame. If these columns are not computed as part of the forecast, you'll need to either calculate them yourself or check if there's another way to generate prediction intervals using your current setup.

    • After calling sf.fit(y_cl4), make sure y_cl4 is formatted correctly and contains the necessary data in the expected form for StatsForecast.

    • When you instantiate the ConformalIntervals class, ensure that you've set it up correctly according to the library's requirements. With some statistical packages, additional setup, calibration, or fitting procedures may be required to use prediction intervals correctly.

    • When you print the forecasts_df, if the DataFrame is very large, the output may be truncated or not displayed completely. You might want to explore saving the output to a CSV or viewing it in smaller chunks if you need to inspect the entire DataFrame.

    • For readability and maintainability, consider adding comments to describe the steps clearly, so others (or you in the future) can easily understand the flow of your forecasting process.

    Lastly, remember to run the code in sequence after restarting the kernel to prevent any state from previous runs from affecting your current session. If you still encounter errors, refer to error messages for debugging clues, and ensure your development environment packages are all up to date.

  22. User

    30 return np.abs(y_true - y_pred).sum() / np.abs(y_true).sum()

    File ~/anaconda3/lib/python3.10/site-packages/polars/dataframe/frame.py:8315, in DataFrame.with_columns(self, *exprs, **named_exprs) 8169 def with_columns( 8170 self, 8171 *exprs: IntoExpr | Iterable[IntoExpr], 8172 **named_exprs: IntoExpr, 8173 ) -> DataFrame: 8174 """ 8175 Add columns to this DataFrame. 8176 (...) 8313 └─────┴──────┴─────────────┘ 8314 """ -> 8315 return self.lazy().with_columns(*exprs, **named_exprs).collect(_eager=True)

    File ~/anaconda3/lib/python3.10/site-packages/polars/lazyframe/frame.py:1940, in LazyFrame.collect(self, type_coercion, predicate_pushdown, projection_pushdown, simplify_expression, slice_pushdown, comm_subplan_elim, comm_subexpr_elim, no_optimization, streaming, background, _eager) 1937 if background: 1938 return InProcessQuery(ldf.collect_concurrently()) -> 1940 return wrap_df(ldf.collect())

    ComputeError: The name: 'Ensemble' passed to LazyFrame.with_columns is duplicate

    Error originated just after this operation: DF ["unique_id", "ds", "cutoff", "y"]; PROJECT */8 COLUMNS; SELECTION: "None" --------------------------------------------------------------------------- ComputeError Traceback (most recent call last) Cell In[11], line 25 22 crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True) 24 # Create an ensemble out-of-sample predictions by taking the median of the predictions from all models ---> 25 crossvalidation_df = crossvalidation_df.with_columns( 26 pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias("Ensemble") 27 ) 29 def wmape(y_true, y_pred): 30 return np.abs(y_true - y_pred).sum() / np.abs(y_true).sum() still the same erorr and at the cross validation step

  23. Assistant

    The error message is clear: the "Ensemble" name is duplicated when you attempt to add it to the crossvalidation_df DataFrame. This means that at the point where you make this call:

    crossvalidation_df = crossvalidation_df.with_columns(
        pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias("Ensemble")
    )
    

    ...there is already a column named "Ensemble" in crossvalidation_df. Since you stated that a new session was started and the kernel was restarted, let's look into other possibilities that might be causing the issue.

    To troubleshoot this:

    1. Inspect the DataFrame columns right before the step that's causing the error to confirm whether the "Ensemble" column truly does not yet exist:
    print(crossvalidation_df.columns)
    
    1. Verify if cross_validation method creates an 'Ensemble' column. It's possible that the cross_validation function from StatsForecast library might be creating an "Ensemble" column by default in the crossvalidation_df DataFrame. Go through the documentation to ensure that's not the case.

    2. Ensure crossvalidation_df is the DataFrame you expect. Directly after the cross_validation call, inspect a snippet of your DataFrame to verify it contains what you expect:

    print(crossvalidation_df.head())
    
    1. Conditional column addition. To be extra sure, only add the "Ensemble" column if it doesn't already exist:
    if "Ensemble" not in crossvalidation_df.columns:
        crossvalidation_df = crossvalidation_df.with_columns(
            pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias("Ensemble")
        )
    

    These steps will help you to verify the state of crossvalidation_df just before the error occurs. Understanding the precise state of the DataFrame will help us diagnose why the 'Ensemble' column appears to be duplicated.

  24. User

    print(crossvalidation_df.columns)

    ['unique_id', 'ds', 'cutoff', 'y', 'AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']print(crossvalidation_df) unique_id ┆ ds ┆ cutoff ┆ y ┆ AutoARIMA ┆ AutoETS ┆ CES ┆ DynamicOp │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ timizedTh │ │ str ┆ datetime[μ ┆ datetime[μ ┆ f32 ┆ f32 ┆ f32 ┆ f32 ┆ eta │ │ ┆ s] ┆ s] ┆ ┆ ┆ ┆ ┆ --- │ │ ┆ ┆ ┆ ┆ ┆ ┆ ┆ f32 │ ╞═══════════╪════════════╪════════════╪═════════╪════════════╪════════════╪════════════╪═══════════╡ │ 1108284 ┆ 2023-04-17 ┆ 2023-04-10 ┆ 25394.0 ┆ 27515.5644 ┆ 27464.0410 ┆ 28000.5351 ┆ 27720.164 │ │ ┆ 00:00:00 ┆ 00:00:00 ┆ ┆ 53 ┆ 16 ┆ 56 ┆ 062 │ │ 1108284 ┆ 2023-04-24 ┆ 2023-04-10 ┆ 26566.0 ┆ 27515.5644 ┆ 27464.0410 ┆ 10834.9970 ┆ 27720.164 │ │ ┆ 00:00:00 ┆ 00:00:00 ┆ ┆ 53 ┆ 16 ┆ 7 ┆ 062 │ │ 1108284 ┆ 2023-05-01 ┆ 2023-04-10 ┆ 17182.0 ┆ 27515.5644 ┆ 27464.0410 ┆ 27656.6699 ┆ 27720.164 │

  25. Assistant

    Thank you for providing the column names and a sample of the crossvalidation_df dataframe. As you specified, there is indeed no 'Ensemble' column present before the operation that aims to add it.

    Given that the 'Ensemble' column does not exist prior to trying to add it, the duplication error is puzzling because it should not occur under these circumstances. A few other things we can consider:

    • Environmental issues: Ensure you're working in a clean environment. It's possible the environment has installations or dependencies causing conflicts. Try creating a new environment or ensure your existing one is up-to-date.

    • Library versions: Check the versions of the Polars and StatsForecast libraries. It's possible there may be a bug in the version you're using. Update them to the latest stable version, if they're not already.

    • Use DataFrame API directly: You can try to use the direct DataFrame constructor from a lazy frame to see if this allows you to bypass the issue:

    crossvalidation_df = pl.DataFrame(
        crossvalidation_df.lazy()
        .with_columns(
            pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias("Ensemble")
        )
        .collect(_eager=True)
    )
    
    • Use DataFrame.extend() method: As an alternative to with_columns, you can also try using the extend method, which concatenates Dataframes or Series:
    ensemble_column = pl.DataFrame({
        "Ensemble": crossvalidation_df.select(
            [
                pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean()
            ]
        )
    })
    crossvalidation_df = crossvalidation_df.extend(ensemble_column)
    

    Despite the approach chosen, I recommend cross-verifying each operation step by step:

    • Execute print(crossvalidation_df.columns) just before the suspicious operation to double-check the state of the dataframe.
    • Execute the operation to add the 'Ensemble' column.
    • Immediately after, again execute print(crossvalidation_df.columns) to see if the column was indeed added.

    If the error persists after this, it might be worth reaching out to the maintainers of the StatsForecast or Polars libraries with your issue and the details of your environment and library versions, as this could indicate a deeper issue within the libraries themselves.

  26. User

    look at the example for ensemble again to see if my code is done correcyly import os import time from functools import partial from multiprocessing import cpu_count os.environ['NIXTLA_NUMBA_RELEASE_GIL'] = '1' os.environ['NIXTLA_NUMBA_CACHE'] = '1'

    import fire import numpy as np import pandas as pd from statsforecast import StatsForecast from statsforecast.models import ( AutoTheta, AutoETS, AutoCES,AutoARIMA, Theta, OptimizedTheta, DynamicTheta, DynamicOptimizedTheta ) from statsforecast.utils import AirPassengers as ap

    from src.data import get_data

    def main(dataset: str = 'M3', group: str = 'Other') -> None: train, horizon, freq, seasonality = get_data('data/', dataset, group) models = [ AutoETS(season_length=seasonality), AutoCES(season_length=seasonality), AutoARIMA(season_length=seasonality), DynamicOptimizedTheta(season_length=seasonality), ]

    start = time.time()
    fcst = StatsForecast(df=train, models=models, freq=freq, n_jobs=cpu_count())
    forecasts = fcst.forecast(h=horizon)
    end = time.time()
    print(end - start)
    
    forecasts = forecasts.reset_index()
    forecasts['StatisticalEnsemble'] = forecasts.set_index(['unique_id', 'ds']).median(axis=1).values
    forecasts.to_csv(f'data/StatisticalEnsemble-forecasts-{dataset}-{group}.csv', index=False)
    
    time_df = pd.DataFrame({'time': [end - start], 'model': ['StatisticalEnsemble']})
    time_df.to_csv(f'data/StatisticalEnsemble-time-{dataset}-{group}.csv', index=False)
    

    if name == 'main': AutoTheta(season_length=12).forecast(ap.astype(np.float32), h=12) fire.Fire(main)

  27. Assistant

    Your original code indeed aligns with the example provided for creating an ensemble using the StatsForecast library, with some differences mainly due to your use of Polars rather than Pandas. The example code provides useful context in how the ensemble is calculated and provides a benchmark for correctness.

    The example uses the following steps to create an ensemble:

    1. Initialize the models and create a StatsForecast instance.
    2. Forecast the future values with each model.
    3. Calculate the median of the forecasts for each time period as the ensemble forecast.
    4. Save the resulting ensemble forecast to a CSV file.

    Comparing your code against this:

    • You do the same model initialization and forecasting but then work with the results using Polars instead of Pandas.
    • Interestingly, in the example, they compute the median of the forecasted values, while in your code, you're taking the mean:
    # Example code
    forecasts['StatisticalEnsemble'] = forecasts.set_index(['unique_id', 'ds']).median(axis=1).values
    # Your code
    crossvalidation_df = crossvalidation_df.with_columns(
        pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias("Ensemble")
    )
    

    Another differential point is that your usage aims to also provide cross-validation metrics, while the example stops after generating and recording the forecasts.

    Regarding your question on correctness, based on the structure given in the example, the part of your code generating the ensemble forecast seems correct. Other than the median vs. mean difference, you're following similar steps for creating ensemble predictions using the respective APIs (Polars vs Pandas).

    As for the error you're running into, it seems to be specific to your particular setup or library version. Assuming there's no user error (like unintentional duplicate code execution), this discrepancy suggests looking at environmental factors (like library versions) or reaching out for support from the library maintainer if the error persists. In addition, make sure to apply the proper pandas conversion if you're mixing Polars with any code that presumes Pandas DataFrames, as these are distinct data structures and are not directly interchangeable.

  28. User

    is this how the conditinal done # Initialize the StatsForecast model sf = StatsForecast(models=models, freq='1w', n_jobs=-1)

    Perform cross-validation with a step size of 1 to mimic an expanding window

    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True)

    Create an ensemble out-of-sample predictions by taking the median of the predictions from all models

    if “Ensemble” not in crossvalidation_df.columns: crossvalidation_df = crossvalidation_df.with_columns( pl.col([‘AutoARIMA’, ‘AutoETS’, ‘CES’, ‘DynamicOptimizedTheta’]).mean().alias(“Ensemble”) )

    def wmape(y_true, y_pred): return np.abs(y_true - y_pred).sum() / np.abs(y_true).sum()

  29. Assistant

    Yes, that is the correct way to conditionally add the 'Ensemble' column as long as the column names use straight quotes rather than curly quotes. Here is the correct snippet with minor adjustments:

    # Initialize the StatsForecast model
    sf = StatsForecast(models=models, freq='1w', n_jobs=-1)
    
    # Perform cross-validation with a step size of 1 to mimic an expanding window
    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True)
    
    # Create an ensemble out-of-sample predictions by taking the mean of the predictions from all models
    if "Ensemble" not in crossvalidation_df.columns:
        crossvalidation_df = crossvalidation_df.with_columns(
            pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias("Ensemble")
        )
    
    def wmape(y_true, y_pred):
        return np.abs(y_true - y_pred).sum() / np.abs(y_true).sum()
    

    Please note I changed the quotes to straight single quotes (' ') around the string literals. Curly quotes (“ ”) are typically used in typeset text but are not valid string delimiters in Python code, and this can cause syntax errors.

    With this snippet, the 'Ensemble' column should only be added if it is not already present in crossvalidation_df, which helps ensure that you do not get errors due to trying to add a duplicate column.

    Remember to ensure that y_cl4 is correctly set up as a DataFrame with the necessary columns before running the sf.cross_validation method.

  30. User

    I've tried all your suggestions, using dataframe api, extend(), conditional, none worked, I jsut rename Ensemble to Ensemble2 and see how it goes from statsforecast import StatsForecast from statsforecast.models import AutoARIMA, AutoETS, AutoCES, DynamicOptimizedTheta from statsforecast.utils import ConformalIntervals import numpy as np import polars as pl

    Polars option to display all rows

    pl.Config.set_tbl_rows(None)

    Initialize the models

    models = [ AutoARIMA(season_length=52), AutoETS(season_length=52), AutoCES(season_length=52), DynamicOptimizedTheta(season_length=52) ]

    Initialize the StatsForecast model

    sf = StatsForecast(models=models, freq='1w', n_jobs=-1)

    Perform cross-validation with a step size of 1 to mimic an expanding window

    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True)

    Create an ensemble out-of-sample predictions by taking the median of the predictions from all models

    crossvalidation_df = crossvalidation_df.with_columns( pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias("Ensemble2") )

    def wmape(y_true, y_pred): return np.abs(y_true - y_pred).sum() / np.abs(y_true).sum()

    Calculate the WMAPE for the ensemble model

    wmape_value = wmape(crossvalidation_df['y'], crossvalidation_df['Ensemble2']) print('Average WMAPE for Ensemble: ', round(wmape_value, 4))

    Calculate the errors for the ensemble model

    errors = crossvalidation_df['y'] - crossvalidation_df['Ensemble2']

    For an individual forecast

    individual_accuracy = 1 - (abs(crossvalidation_df['y'] - crossvalidation_df['Ensemble2']) / crossvalidation_df['y']) individual_bias = (crossvalidation_df['Ensemble2'] / crossvalidation_df['y']) - 1

    Add these calculations as new columns to your DataFrame

    crossvalidation_df = crossvalidation_df.with_columns([ individual_accuracy.alias("individual_accuracy"), individual_bias.alias("individual_bias") ])

    Print the individual accuracy and bias for each week

    for row in crossvalidation_df.to_dicts(): id = row['unique_id'] date = row['ds'] accuracy = row['individual_accuracy'] bias = row['individual_bias'] print(f"{id}, {date}, Individual Accuracy: {accuracy:.4f}, Individual Bias: {bias:.4f}")

    For groups of forecasts

    group_accuracy = 1 - (errors.abs().sum() / crossvalidation_df['y'].sum()) group_bias = (crossvalidation_df['Ensemble2'].sum() / crossvalidation_df['y'].sum()) - 1

    Print the average group accuracy and group bias over all folds for the ensemble model

    print('Average Group Accuracy: ', round(group_accuracy, 4)) print('Average Group Bias: ', round(group_bias, 4))

    Fit the models on the entire dataset

    sf.fit(y_cl4)

    Instantiate the ConformalIntervals class

    prediction_intervals = ConformalIntervals()

    Generate 24 months forecasts

    forecasts_df = sf.forecast(h=4*24, prediction_intervals=prediction_intervals, level=[95], id_col='unique_id', sort_df=True)

    Create an ensemble forecast by taking the mean of the forecasts from all models

    forecasts_df = forecasts_df.with_columns( pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias("EnsembleForecast") )

    forecasts_df = forecasts_df.with_columns([ pl.col("EnsembleForecast").round().alias("CombinedForecast"), pl.col("Ensemble-lo-95").round().alias("CombinedForecast-lo-95"), pl.col("Ensemble-hi-95").round().alias("CombinedForecast-hi-95") ])

    print(forecasts_df)

  31. Assistant

    Renaming the column to "Ensemble2" is a practical workaround if you've exhausted other options for addressing the error. The code you've provided should create an ensemble column without an issue, as long as no existing column is already named "Ensemble2".

    The rest of your code appears to be consistent and follows the process of creating ensemble predictions and calculating error metrics. I'll provide the full process you are following:

    1. You are initializing a series of models for time series forecasting, ensuring that they all take into account the seasonal pattern (52 weeks).
    2. You create a StatsForecast instance with these models and use it to perform cross-validation on your dataset (y_cl4).
    3. You compute the ensemble forecast as the mean of the predictions from all models and name this new column "Ensemble2" to avoid the duplication issue.
    4. You define a WMAPE function and use it to calculate the WMAPE for the ensemble forecast.
    5. You calculate errors, individual accuracy, and bias, and add these as new columns to your crossvalidation_df.
    6. You calculate group-level accuracy and bias for all forecasts and print them out.
    7. You fit the models on the entire dataset.
    8. You instantiate the ConformalIntervals class and use it to create forecasts with prediction intervals, then process this resulting forecast DataFrame separately.

    A few additional points to ensure smooth execution:

    • Keep a close watch on the naming of columns and ensure there is no overlap.
    • Verify that your models are correctly specified and that y_cl4 is appropriately formatted to work with the StatsForecast API.
    • Double-check that no cell in your Jupyter notebook (or whichever development environment you're using) accidentally redefines or processes crossvalidation_df after you've already added the "Ensemble2" column.
    • Make sure the versions of the libraries you are using are up to date and that the API usage is correct; occasionally, library updates can introduce breaking changes that require code modification.

    If renaming the column to "Ensemble2" resolves the previous error and the rest of the script runs without issues, it means there was likely a hidden state that caused the duplication problem. You should now be able to gather accurate ensemble forecasts and related statistics from your models.

  32. User

    I separated the mean() ensemble from the cross validation becuase I think the loop in cross validation is repeatedly creating a new column Ensemble so that's why we had that error, now I got another different error --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) Cell In[9], line 29 24 # Create an ensemble out-of-sample predictions by taking the median of the predictions from all models 25 crossvalidation_df = crossvalidation_df.with_columns( 26 pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']) 27 ) ---> 29 crossvalidation_df = crossvalidation_df.mean().alias("Ensemble") 31 def wmape(y_true, y_pred): 32 return np.abs(y_true - y_pred).sum() / np.abs(y_true).sum()

    AttributeError: 'DataFrame' object has no attribute 'alias' from statsforecast import StatsForecast from statsforecast.models import AutoARIMA, AutoETS, AutoCES, DynamicOptimizedTheta from statsforecast.utils import ConformalIntervals import numpy as np import polars as pl

    Polars option to display all rows

    pl.Config.set_tbl_rows(None)

    Initialize the models

    models = [ AutoARIMA(season_length=52), AutoETS(season_length=52), AutoCES(season_length=52), DynamicOptimizedTheta(season_length=52) ]

    Initialize the StatsForecast model

    sf = StatsForecast(models=models, freq='1w', n_jobs=-1)

    Perform cross-validation with a step size of 1 to mimic an expanding window

    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True)

    Create an ensemble out-of-sample predictions by taking the median of the predictions from all models

    crossvalidation_df = crossvalidation_df.with_columns( pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']) )

    crossvalidation_df = crossvalidation_df.mean().alias("Ensemble")

    def wmape(y_true, y_pred): return np.abs(y_true - y_pred).sum() / np.abs(y_true).sum()

    Calculate the WMAPE for the ensemble model

    wmape_value = wmape(crossvalidation_df['y'], crossvalidation_df['Ensemble']) print('Average WMAPE for Ensemble: ', round(wmape_value, 4))

    Calculate the errors for the ensemble model

    errors = crossvalidation_df['y'] - crossvalidation_df['Ensemble']

    For an individual forecast

    individual_accuracy = 1 - (abs(crossvalidation_df['y'] - crossvalidation_df['Ensemble']) / crossvalidation_df['y']) individual_bias = (crossvalidation_df['Ensemble'] / crossvalidation_df['y']) - 1

    Add these calculations as new columns to your DataFrame

    crossvalidation_df = crossvalidation_df.with_columns([ individual_accuracy.alias("individual_accuracy"), individual_bias.alias("individual_bias") ])

    Print the individual accuracy and bias for each week

    for row in crossvalidation_df.to_dicts(): id = row['unique_id'] date = row['ds'] accuracy = row['individual_accuracy'] bias = row['individual_bias'] print(f"{id}, {date}, Individual Accuracy: {accuracy:.4f}, Individual Bias: {bias:.4f}")

    For groups of forecasts

    group_accuracy = 1 - (errors.abs().sum() / crossvalidation_df['y'].sum()) group_bias = (crossvalidation_df['Ensemble'].sum() / crossvalidation_df['y'].sum()) - 1

    Print the average group accuracy and group bias over all folds for the ensemble model

    print('Average Group Accuracy: ', round(group_accuracy, 4)) print('Average Group Bias: ', round(group_bias, 4))

    Fit the models on the entire dataset

    sf.fit(y_cl4)

    Instantiate the ConformalIntervals class

    prediction_intervals = ConformalIntervals()

    Generate 24 months forecasts

    forecasts_df = sf.forecast(h=4*24, prediction_intervals=prediction_intervals, level=[95], id_col='unique_id', sort_df=True)

    Create an ensemble forecast by taking the mean of the forecasts from all models

    forecasts_df = forecasts_df.with_columns( pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias("EnsembleForecast") )

    forecasts_df = forecasts_df.with_columns([ pl.col("EnsembleForecast").round().alias("CombinedForecast"), pl.col("Ensemble-lo-95").round().alias("CombinedForecast-lo-95"), pl.col("Ensemble-hi-95").round().alias("CombinedForecast-hi-95") ])

    print(forecasts_df)

  33. Assistant

    The AttributeError: 'DataFrame' object has no attribute 'alias' is happening because you're calling the mean() method on crossvalidation_df, which returns a DataFrame object, and then you're trying to chain the .alias() method, which does not exist for DataFrame objects in the Polars library.

    To calculate the mean of certain columns and create a new column based on this mean, you should include the .mean() inside pl.col() like you did previously, and then use the .alias() to give it a name, while still having it inside the with_columns() method. Here's how you should modify your code:

    # Correctly calculate the ensemble mean and add as a new column
    crossvalidation_df = crossvalidation_df.with_columns(
        pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias("Ensemble")
    )
    

    Remember, with_columns() is used to add new columns to a DataFrame in Polars, and any calculations or transformations to be assigned to a new column should be completed within its scope.

    The corrected version of the code section should look like this:

    # Initialize the StatsForecast model
    sf = StatsForecast(models=models, freq='1w', n_jobs=-1)
    
    # Perform cross-validation with a step size of 1 to mimic an expanding window
    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True)
    
    # Calculate the mean of the models' forecasts and name it 'Ensemble'
    crossvalidation_df = crossvalidation_df.with_columns(
        pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias("Ensemble")
    )
    
    # Rest of your code...
    

    You don't have to call mean() separately as part of a different pipeline. Keep it within the with_columns() call to ensure that the transformation applied creates a new column in the existing DataFrame. This will avoid the DataFrame object not having an alias method, which is specific to the column expressions.

  34. User

    but do you remember that we kept getting error File ~/anaconda3/lib/python3.10/site-packages/polars/lazyframe/frame.py:1940, in LazyFrame.collect(self, type_coercion, predicate_pushdown, projection_pushdown, simplify_expression, slice_pushdown, comm_subplan_elim, comm_subexpr_elim, no_optimization, streaming, background, _eager) 1937 if background: 1938 return InProcessQuery(ldf.collect_concurrently()) -> 1940 return wrap_df(ldf.collect())

    ComputeError: The name: ‘Ensemble’ passed to LazyFrame.with_columns is duplicate

    Error originated just after this operation: DF [“unique_id”, “ds”, “cutoff”, “y”]; PROJECT */8 COLUMNS; SELECTION: “None” --------------------------------------------------------------------------- ComputeError Traceback (most recent call last) Cell In[11], line 25 22 crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True) 24 # Create an ensemble out-of-sample predictions by taking the median of the predictions from all models —> 25 crossvalidation_df = crossvalidation_df.with_columns( 26 pl.col([‘AutoARIMA’, ‘AutoETS’, ‘CES’, ‘DynamicOptimizedTheta’]).mean().alias(“Ensemble”) 27 ) 29 def wmape(y_true, y_pred): 30 return np.abs(y_true - y_pred).sum() / np.abs(y_true).sum() because of the loop in cross validation function keep creating the new "Ensemble" column, so that's why I ahd to separate them, can't we just do the 2 processes separately?

  35. Assistant

    Indeed, if the cross-validation function internally uses a loop and possibly returns a new DataFrame on each iteration, directly modifying such a DataFrame within the loop could indeed cause the duplication error you've been experiencing.

    Considering this behavior, it's reasonable to calculate the "Ensemble" column after the cross-validation function has completed its iterations and returned the final DataFrame. This approach would entail performing the calculation of the "Ensemble" column in a separate step, making sure it occurs only once without duplication. Here is how you can structure your code:

    # Initialize the StatsForecast model
    sf = StatsForecast(models=models, freq='1w', n_jobs=-1)
    
    # Perform cross-validation with a step size of 1 to mimic an expanding window
    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True)
    
    # Now that the cross-validation is complete and the DataFrame is finalized,
    # calculate the mean of the models' forecasts and add it as a new column 'Ensemble'
    crossvalidation_df = crossvalidation_df.with_columns(
        (pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean() * len(models)).alias("Ensemble")
        # Multiplication by len(models) assumes you want to incorporate the scale of the values back
    )
    
    # Rest of your code...
    

    Ensure the above operation to add the "Ensemble" column executes only once after the DataFrame has been fully constructed by the cross-validation function, avoiding the duplication issue.

    Additionally, when using mean() in Polars on multiple columns, you want to ensure that the calculation is correct based on whether mean() should be calculated across rows or columns and whether you’ll need the subsequent multiplication to account for the number of models.

    If after the cross-validation you end up with a DataFrame where each row comprises the forecasts from different models for the same period, then taking the mean() across columns (as described above) is appropriate. However, it's essential to consider the structure of the DataFrame and adjust the calculation accordingly.

    Remember, it's critical to ensure that no other part of your code is inadvertently adding an "Ensemble" column to the crossvalidation_df DataFrame before this step.

  36. User

    error right away Cell In[10], line 26 (pl.col([‘AutoARIMA’, ‘AutoETS’, ‘CES’, ‘DynamicOptimizedTheta’]).mean() * len(models)).alias(“Ensemble”) ^ SyntaxError: invalid character '‘' (U+2018)

  37. Assistant

    The error is due to the use of curly quotes (‘ ’ or “ ”) instead of straight quotes (' ' or " ") in your code. In Python, it's important to use straight quotes for string literals.

    You should replace the curly quotes with straight quotes as follows:

    crossvalidation_df = crossvalidation_df.with_columns(
        (pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean() * len(models)).alias("Ensemble")
    )
    

    This change should resolve the SyntaxError. Please ensure your code editor or development environment is configured to use straight quotes, as curly quotes can cause syntax issues in Python.

  38. User

    it's still the same error, and you provided a solution that's changed nothing

  39. Assistant

    I apologize for that oversight. The solution provided was intended to correct the specific syntax error resulting from character encoding (curly quotes instead of straight quotes), not to resolve the duplication error in the DataFrame columns.

    Let's address the duplication error with a fresh approach. If your cross-validation loop is causing the duplication error by repeatedly creating a new 'Ensemble' column, it would be best to calculate the 'Ensemble' column entirely outside and after the loop. We can do this by calculating the mean explicitly using the Polars select method, which allows for complex column-wise operations, and then joining it back to the original DataFrame:

    Here's how you can achieve this:

    # Initialize the StatsForecast model
    sf = StatsForecast(models=models, freq='1w', n_jobs=-1)
    
    # Perform cross-validation with a step size of 1 to mimic an expanding window
    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True)
    
    # Calculate the mean of the models' forecasts and create an 'Ensemble' column
    ensemble_mean = crossvalidation_df.select(
        pl.mean([
            pl.col('AutoARIMA'), 
            pl.col('AutoETS'),
            pl.col('CES'),
            pl.col('DynamicOptimizedTheta')
        ]).alias('Ensemble')
    )
    
    # Join the 'Ensemble' mean to the original cross-validation DataFrame
    crossvalidation_df = crossvalidation_df.hstack(ensemble_mean)
    

    Make sure to run this code in a fresh environment or after explicitly clearing any variables to avoid any remnants of the 'Ensemble' column from previous runs.

    By using select to calculate the mean and then using hstack to join it back, you're avoiding direct manipulation of crossvalidation_df within a loop, which should prevent the duplication error.

  40. Assistant

    It seems like your latest input is empty. Is there anything specific you would like to ask or clarify? Please feel free to post your question or share the details, and I'll be glad to help!

  41. User

    Cell In[12], line 27 pl.col(‘AutoARIMA’), ^ SyntaxError: invalid character '‘' (U+2018)

  42. Assistant

    The error message indicates that there are curly quotes used in your code instead of straight quotes. In Python, string literals should be enclosed in straight quotes, either single quotes ' ' or double quotes " ". Curly quotes are typically used in typeset text but are not valid in Python code.

    Here's the corrected line with straight single quotes:

    pl.col('AutoARIMA'),
    

    Ensure that all string literals in your code use straight quotes. If you are copying and pasting code from a source that uses stylized text, like certain websites or word processors, you may need to manually correct the quotes or type out the code directly in your text editor or IDE.

  43. User

    the solution you provded did not workLet’s address the duplication error with a fresh approach. If your cross-validation loop is causing the duplication error by repeatedly creating a new ‘Ensemble’ column, it would be best to calculate the ‘Ensemble’ column entirely outside and after the loop. We can do this by calculating the mean explicitly using the Polars select method, which allows for complex column-wise operations, and then joining it back to the original DataFrame:

  44. Assistant

    I'm sorry the previous solution didn't work for you. Let's try a different approach to calculate the ensemble column after the cross-validation, ensuring we are not duplicating column names.

    1. Perform cross-validation without creating any 'Ensemble' column.
    2. After cross-validation is complete, select the relevant forecast columns and calculate their mean separately.
    3. Join the resulting mean Series back to the original cross-validation DataFrame.

    Here is the updated code:

    # Perform cross-validation with a step size of 1 to mimic an expanding window
    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True)
    
    # Calculate the mean of forecasts from each model
    # We take the mean across the relevant columns and create a separate Series for it
    ensemble_mean = crossvalidation_df.select([
        pl.mean([
            pl.col('AutoARIMA'), 
            pl.col('AutoETS'),
            pl.col('CES'), 
            pl.col('DynamicOptimizedTheta')
        ]).alias('Ensemble')
    ])
    
    # Now, the 'Ensemble' Series can be joined back to the original DataFrame
    # We use the 'hstack' operation to horizontally stack the DataFrames, ensuring no duplicate column names
    crossvalidation_df = crossvalidation_df.hstack(ensemble_mean)
    

    In this code:

    • select is used to pick the columns and compute their mean. The output is a new DataFrame with one column ('Ensemble').
    • hstack is utilized to add the 'Ensemble' column from the new DataFrame to crossvalidation_df.

    This approach should prevent any potential duplication errors related to the 'Ensemble' column name. The .mean() here should operate across the selected columns, producing a single mean value for each row in crossvalidation_df. Then hstack is used to append this as a new column to the existing DataFrame.

    Make sure to run this code in a fresh environment or after a kernel restart to ensure that the DataFrame is not affected by any previous state.

  45. User

    this is what you proposed earlier like 2 messages ago and it did not work, error right away

  46. Assistant

    My apologies for the confusion. Let's try a different approach using the apply method in Polars to avoid directly manipulating the DataFrame inside a loop, which might cause the duplication error:

    # Perform cross-validation with a step size of 1 to mimic an expanding window
    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True)
    
    # Calculate the mean using the apply method
    ensemble_mean = crossvalidation_df.apply(lambda x: np.mean([x['AutoARIMA'], x['AutoETS'], x['CES'], x['DynamicOptimizedTheta']]), axis=1)
    
    # Convert the result to a DataFrame and give the column a name
    ensemble_mean_df = pl.DataFrame({'Ensemble': ensemble_mean})
    
    # Concatenate the ensemble_mean_df DataFrame with the original crossvalidation_df
    # This will add the 'Ensemble' column to your existing DataFrame
    crossvalidation_df = pl.concat([crossvalidation_df, ensemble_mean_df], axis=1)
    

    Here’s what's happening in this code:

    1. The sf.cross_validation function runs and yields crossvalidation_df.
    2. We use apply to compute the mean of the forecasts from the specified columns for each row. This generates a list or Series of mean values.
    3. We convert this list into a new DataFrame with a single named column ('Ensemble').
    4. Finally, we concatenate this new DataFrame to crossvalidation_df horizontally (along axis=1), effectively adding the 'Ensemble' column.

    Please ensure that pl.concat is used correctly as per the Polars library's syntax and behavior (which may differ from Pandas).

    If this still doesn't work, let's consider:

    • Ensure you are using the latest version of Polars, as API changes could cause different behavior.
    • Verify that there's no pre-existing issue with the DataFrame returned by cross_validation.
    • If you continue to encounter issues, it may be helpful to reach out to the maintainers of the StatsForecast library or the Polars library to report the issue and seek direct assistance.
  47. User

    why can't we do this way crossvalidation_df = crossvalidation_df.with_columns( pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']) )

    crossvalidation_df = crossvalidation_df.mean().alias("Ensemble")

  48. Assistant

    The attempt to utilize crossvalidation_df.mean().alias("Ensemble") is not valid, because the mean() function in Polars, when called on a DataFrame, returns a new DataFrame with the mean of each column, not a single value or series that can be passed to alias() to create a new column name. The alias() method can only be applied to a column object or expression returned from methods in Polars that allow chaining such as select() or with_columns().

    If you want to calculate the mean of several columns and store this as a new column 'Ensemble' in the DataFrame, you can do the following:

    # This will take the mean of the four specified columns and create a new column "Ensemble"
    crossvalidation_df = crossvalidation_df.with_columns(
        pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias("Ensemble")
    )
    

    This line calculates the mean across each row for the specified columns, which is presumably the intended operation if you are trying to create an 'Ensemble' forecast from individual model forecasts.

    To use mean() and then add alias(), you must first select the columns, apply mean() to the selection to create a single column result, and then apply alias() to this result.

    Therefore, the proper use of these functions would be like the example provided above, where with_columns is used to handle the aliasing of the mean value calculated from the selected columns.

  49. User

    but that is inside a loop causing duplicated values of Ensemble

  50. Assistant

    I understand now. If the duplication of the 'Ensemble' column occurs because the cross-validation is looped internally by the StatsForecast library, creating the 'Ensemble' column after the loop is the correct approach to prevent this issue.

    Let's separate the mean calculation from the cross-validation step. Make sure to calculate the mean after the cross-validation step is completed and create the 'Ensemble' column only once.

    Here's how you can do it:

    # Perform cross-validation with a step size of 1 to mimic an expanding window
    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True)
    
    # After cross-validation is complete, take the mean of the predictions from all models separately
    # Note that the mean operation should be across columns (or 'axis=1' in pandas terms), creating a Series
    ensemble_mean_series = crossvalidation_df.select([
        pl.col('AutoARIMA'), 
        pl.col('AutoETS'),
        pl.col('CES'), 
        pl.col('DynamicOptimizedTheta')
    ]).mean(axis=1)
    
    # Now create a new DataFrame with 'Ensemble' as the column name
    ensemble_df = pl.DataFrame({'Ensemble': ensemble_mean_series})
    
    # Horizontally stack the DataFrame (like pandas concat with axis=1)
    crossvalidation_df = crossvalidation_df.hstack(ensemble_df)
    

    In Polars, mean(axis=1) does not exist as it does in pandas. If you want to calculate the row-wise mean in Polars, you would have to use a different approach. Polars processes data using 'lazy' (deferred execution) by default, so we use select() to build up the query and then with_column to execute it and add it to the DataFrame.

    Let's attempt creating a UDF (User Defined Function) to calculate the row-wise mean:

    def row_mean(*args):
        return np.mean(args)
    
    # Calculate the row-wise mean using the UDF 'row_mean' with the columns of interest
    ensemble_column = crossvalidation_df.with_columns([
        row_mean(
            pl.col('AutoARIMA'), 
            pl.col('AutoETS'),
            pl.col('CES'), 
            pl.col('DynamicOptimizedTheta')
        ).alias('Ensemble')
    ])
    
    crossvalidation_df = crossvalidation_df.hstack(ensemble_column)
    

    If none of the above works, then there might be something else at play, and it would be helpful to look in more detail at the cross-validation function and the structure of crossvalidation_df.

  51. User

    both didn't work

  52. Assistant

    I apologize for the confusion earlier. Since the previous suggestions didn't work for you, let's try a different method with the following steps:

    1. Perform the cross-validation to get the forecasts for each model.
    2. After you have the resulting crossvalidation_df, we'll calculate the mean of the specified columns without using .mean() on the DataFrame. Instead, we will manually calculate the mean using the apply method to avoid duplication.

    Here's an updated section of your code:

    # Perform cross-validation with a step size of 1 to mimic an expanding window
    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True)
    
    # Calculate the mean of the models' forecasts manually for each row
    mean_expression = (pl.col('AutoARIMA') + pl.col('AutoETS') + pl.col('CES') + pl.col('DynamicOptimizedTheta')) / 4
    
    # Now create an expression to add to the DataFrame using with_columns
    crossvalidation_df = crossvalidation_df.with_columns(
        mean_expression.alias('Ensemble')
    )
    

    In this code, the mean_expression creates an expression that calculates the mean across the specified columns. Then, we use this expression when adding the new 'Ensemble' column to the DataFrame.

    This approach ensures the 'Ensemble' column is created after the loop and is only added once to the crossvalidation_df. It should not cause any duplication issues as we are manually constructing the average column and adding it to the DataFrame in one step.

  53. User

    wait a minute, the mean_expresssion is before the internal loop in crossvalidation_df,

  54. Assistant

    You're correct. My previous response didn't take into account the internal loop of the cross_validation function. Given your concern about the loop in the cross_validation causing issues with repetitive column creation, you should indeed calculate the 'Ensemble' column entirely outside of the cross_validation step.

    Let's try an approach where we first complete the cross-validation step and then calculate the 'Ensemble' outside of any potential loop:

    # Perform the cross-validation process.
    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True)
    
    # Prepare the mean expression logic to be applied on the resulting DataFrame. 
    # We create a new Polars DataFrame with the calculated 'Ensemble' means.
    ensemble_means_df = crossvalidation_df.select(
        ((pl.col('AutoARIMA') + pl.col('AutoETS') + pl.col('CES') + pl.col('DynamicOptimizedTheta')) / 4).alias('Ensemble')
    )
    
    # Now 'ensemble_means_df' contains only a single column 'Ensemble' with the mean of the forecasts.
    # We want to add this back to the original 'crossvalidation_df', aligning on the index.
    crossvalidation_df = crossvalidation_df.hstack(ensemble_means_df)
    

    This approach ensures that after the cross_validation function is fully executed, you calculate the mean of the interested columns (forecasts from the models) and form a new column 'Ensemble'. Then you stack this horizontally with the original dataframe. Since you're using hstack, we ensure that the indices are aligned without needing extra join or merge logic.

    Run this updated snippet after the cross-validation process in your code to properly integrate the 'Ensemble' forecast without encountering an issue due to an internal loop.

  55. User

    error again, okay hold on print(crossvalidation_df) unique_id ┆ ds ┆ cutoff ┆ y ┆ AutoARIMA ┆ AutoETS ┆ CES ┆ DynamicOp │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ timizedTh │ │ str ┆ datetime[μ ┆ datetime[μ ┆ f32 ┆ f32 ┆ f32 ┆ f32 ┆ eta │ │ ┆ s] ┆ s] ┆ ┆ ┆ ┆ ┆ --- │ │ ┆ ┆ ┆ ┆ ┆ ┆ ┆ f32 │ ╞═══════════╪════════════╪════════════╪═════════╪════════════╪════════════╪════════════╪═══════════╡ │ 1108284 ┆ 2023-04-17 ┆ 2023-04-10 ┆ 25394.0 ┆ 27515.5644 ┆ 27464.0410 ┆ 28000.5351 ┆ 27720.164 │ │ ┆ 00:00:00 ┆ 00:00:00 ┆ ┆ 53 ┆ 16 ┆ 56 ┆ 062 │ │ 1108284 ┆ 2023-04-24 ┆ 2023-04-10 ┆ 26566.0 ┆ 27515.5644 ┆ 27464.0410 ┆ 10834.9970 ┆ 27720.164 │ │ ┆ 00:00:00 ┆ 00:00:00 ┆ ┆ 53 ┆ 16 ┆ 7 ┆ 062 │ │ 1108284 ┆ 2023-05-01 ┆ 2023-04-10 ┆ 17182.0 ┆ 27515.5644 ┆ 27464.0410 ┆ 27656.6699 ┆ 27720.164 │ │ ┆ 00:00:00 ┆ 00:00:00 ┆ ┆ 53 ┆ 16 ┆ 22 ┆ 062 │ it's not possible to perform the mean after the internaal loop is done? crossvalidation_df = crossvalidation_df.with_columns( pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']) )

    crossvalidation_df = crossvalidation_df.mean().alias("Ensemble")

  56. Assistant

    Based on the error and the printed output of crossvalidation_df, it's clear that calling mean() on the DataFrame directly should not be paired with alias(), as mean() computes the mean of each numeric column and returns a DataFrame containing those mean values. alias() is used to rename columns, not DataFrames.

    Since we have names of the columns that need to be averaged for the 'Ensemble' column, we can use the select() method in combination with arithmetic operations to compute the ensemble mean properly.

    Here's how you can calculate the 'Ensemble' column after you've confirmed that cross-validation is complete, based on your provided DataFrame output:

    # Calculate the mean of the selected model forecast columns
    ensemble_mean = crossvalidation_df.select(
        ((pl.col('AutoARIMA') + pl.col('AutoETS') + pl.col('CES') + pl.col('DynamicOptimizedTheta')) / 4).alias('Ensemble')
    )
    
    # The previous line creates a new DataFrame with one column 'Ensemble' that contains the mean values,
    # so we want to add this to `crossvalidation_df`
    crossvalidation_df = crossvalidation_df.hstack(ensemble_mean)
    

    Note here how we construct the mean by summing up the forecasts from each model and dividing by the number of models (using / 4), and then associate the result with a new column 'Ensemble' by using alias(). Finally, we horizontally stack this column onto the crossvalidation_df.

    Make sure you run this operation only once after the cross_validation is done. Let's execute this code and see if it properly creates the 'Ensemble' column without any duplication errors. If the error persists, please share the details of the error message with me so we can troubleshoot it further.

  57. User

    jsut the same error Cell In[20], line 26 ((pl.col(‘AutoARIMA’) + pl.col(‘AutoETS’) + pl.col(‘CES’) + pl.col(‘DynamicOptimizedTheta’)) / 4).alias(‘Ensemble’) ^ SyntaxError: invalid character '‘' (U+2018)

  58. Assistant

    The error you're encountering is due to the incorrect quote characters used in the code. You should replace the curly quotes (‘ ’) with straight quotes either single (' ') or double (" ") for string literals in Python code.

    Here's the corrected snippet with straight single quotes:

    # Calculate the mean of the selected model forecast columns
    ensemble_mean = crossvalidation_df.select(
        ((pl.col('AutoARIMA') + pl.col('AutoETS') + pl.col('CES') + pl.col('DynamicOptimizedTheta')) / 4).alias('Ensemble')
    )
    
    # The previous line creates a new DataFrame with one column 'Ensemble' that contains the mean values
    # so we want to add this to `crossvalidation_df`
    crossvalidation_df = crossvalidation_df.hstack(ensemble_mean)
    

    Make sure to check your code editor or integrated development environment (IDE) settings to ensure that it uses straight quotes for string delimiters. Please run the corrected code to compute the 'Ensemble' column and add it to your DataFrame.

  59. User

    okay this is not feasible, I've tried your solutions when breaking up the alias "emsemble' into another line of code but it's been runing for 56 minutes and nothing, this is the code again, we gonna have to keep the mean and alias on the same line of code, is there any way to deal with the internal loop from cross validation? # Initialize the models models = [ AutoARIMA(season_length=52), AutoETS(season_length=52), AutoCES(season_length=52), DynamicOptimizedTheta(season_length=52) ]

    Initialize the StatsForecast model

    sf = StatsForecast(models=models, freq='1w', n_jobs=-1)

    Perform cross-validation with a step size of 1 to mimic an expanding window

    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True)

    Create an ensemble out-of-sample predictions by taking the mean of the predictions from all models

    crossvalidation_df = crossvalidation_df.with_columns( pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias('Ensemble') )

  60. Assistant

    Sorry to hear that the processing time has increased significantly in your last attempt. If the cross_validation method is inadvertently adding the 'Ensemble' column on each loop iteration internally, it would cause the duplication error you're experiencing.

    Given the constraints and what seems to be happening inside the cross_validation method, we'll need to work around this. The workaround is to ensure that adding the 'Ensemble' column is done without depending on the state inside the loop.

    Since we have confirmed that the cross_validation doesn't include the 'Ensemble' column initially and that calling mean() and alias() on the same line is necessary, you may try the following steps:

    1. Execute cross_validation and store the results in crossvalidation_df.
    2. Calculate the mean of the individual model forecasts and use alias() to assign the result to the 'Ensemble' column, all within with_columns().
    3. Immediately calculate any additional metrics and statistics right afterward to minimize the chance of unexpected behavior caused by the internal loop.

    Here's how you can do this:

    # Initialize the StatsForecast model
    sf = StatsForecast(models=models, freq='1w', n_jobs=-1)
    
    # Perform cross-validation with a step size of 1 to mimic an expanding window
    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True)
    
    # Create an ensemble out-of-sample predictions by taking the mean of the predictions from all models
    # This mean calculation and alias assignment is done outside any loop, just once
    crossvalidation_df = crossvalidation_df.with_columns(
        pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias('Ensemble')
    )
    
    # Calculate the WMAPE for the ensemble model immediately
    # ... (Your code for WMAPE, errors, individual accuracy, bias, and group statistics)
    

    This approach is essentially what you have provided, with an understanding of ensuring this is done once and that with_columns() is executed immediately after cross_validation() to prevent the introduced 'Ensemble' column from being caught in any subsequent loop iterations within cross_validation().

    Without seeing the internals of the cross_validation function, I can't provide a specific solution to alter its behavior. If you continue to have issues with 'Ensemble' being duplicated despite the above, I would recommend checking the documentation for StatsForecast or reaching out to the authors or maintainers of the library for more detailed help. There may be nuances to how cross_validation is intended to be used, or edge cases like this that the documentation or authors can help clarify.

  61. User

    read this website and see if you could learn more about cross validation from statsforecast https://nixtlaverse.nixtla.io/statsforecast/docs/tutorials/crossvalidation.html

  62. Assistant

    I'm sorry for any confusion, but as of my last update, I do not have browsing capabilities to directly access or read from external websites, including the one you've provided for the StatsForecast library documentation.

    However, cross-validation in the context of time series forecasting typically involves a rolling or expanding window approach due to the temporal nature of the data. This involves using a portion of the data for training and the subsequent portion for testing, then incrementing the training set and repeating the process.

    For StatsForecast, given your previous descriptions and assuming the library follows standard practices, the cross_validation function would likely create separate training and testing sets in each iteration of the loop.

    If you are encountering issues with cross-validation, such as long execution times or repeated creation of an 'Ensemble' column, I recommend the following general steps:

    1. Read the documentation for cross_validation carefully: Understanding the parameters and return value of the cross_validation function is critical. Look for parameters that control the behavior of the function, especially those that might affect performance or manage the results returned from the function.

    2. Check examples: Often, documentation provides examples of how to use functions correctly. Compare these examples with your implementation.

    3. Inspect return values: After calling cross_validation, examine the structure of the returned DataFrame closely. Make sure the DataFrame structure is what you expect before proceeding with further calculations.

    4. Seek support: If there are still unresolved issues or behavior you don't understand, consider reaching out for support. This could be through an official support channel for the library, a community forum, or even filing an issue if you suspect there's a bug.

    In the meantime, I recommend that you directly read the documentation using the link you have, which could provide insights specific to the library's implementation of cross-validation. If you have specific questions about the process or code examples from the documentation, feel free to share that information, and I can provide further assistance.

  63. User

    yes you can only ChatGPT Plus plugins provide browsing internet access to GPT 4.

  64. Assistant

    Thank you for the information. Since I cannot directly access the content of the link you've provided, let's go ahead with how typically cross-validation is done for time-series forecasting, and we can see if it aligns with what you have in mind from the documentation.

    When using the cross_validation method provided by the StatsForecast library for time-series data, a common approach is:

    • Partitioning the time-series data into training and testing sets based on time. Due to the sequential nature of time-series data, this often means using earlier data for training and later data for evaluation.
    • Sliding or expanding the window of training data iteratively to validate the model’s performance over different periods. This could be done in a way that enlarges the training dataset with each step (expanding window) or slides the training and validation windows together (rolling window).
    • Calculating forecast accuracy with metrics appropriate for time series, taking into account both the forecasts and the actual observed values during the testing periods.

    If the cross_validation process in StatsForecast creates an individual DataFrame for each iteration and you want to add an 'Ensemble' column at the end, you would do this after the entire cross-validation process is completed to avoid any issues with loop logic that may cause repeated column creation. Make sure that any DataFrame manipulations that depend on the results of cross_validation (like adding new columns or calculating statistics) are done after cross_validation has returned its final DataFrame.

    To debug and optimize the code further, I would also suggest the following:

    1. Test the cross-validation call independently: Run the cross_validation method on its own and make sure it executes as expected without additional manipulations.

    2. Check the output structure: After the call to cross_validation, examine the structure of crossvalidation_df to completely understand what the output DataFrame looks like.

    3. Modify the DataFrame after cross-validation: Once you're certain that cross_validation works correctly and you understand the output structure, proceed with careful DataFrame manipulations to add your 'Ensemble' column.

    The structure and usage of specific methods might vary between libraries, so for the most accurate information and best practices, it is indeed best to refer to the library's official documentation.

    If there are particular sections of the documentation that you wish to discuss or you have some code snippets from the documentation that you're unsure about, feel free to share, and I can provide assistance based on that.

  65. User

    this is the whole website Tutorials Cross validation In this example, we’ll implement time series cross-validation to evaluate model’s performance.

    Prerequesites

    This tutorial assumes basic familiarity with StatsForecast. For a minimal example visit the Quick Start

    ​ Introduction Time series cross-validation is a method for evaluating how a model would have performed in the past. It works by defining a sliding window across the historical data and predicting the period following it.

    Statsforecast has an implementation of time series cross-validation that is fast and easy to use. This implementation makes cross-validation a distributed operation, which makes it less time-consuming. In this notebook, we’ll use it on a subset of the M4 Competition hourly dataset.

    Outline:

    Install libraries Load and explore data Train model Perform time series cross-validation Evaluate results Tip

    You can use Colab to run this Notebook interactively

    Open In Colab ​ Install libraries We assume that you have StatsForecast already installed. If not, check this guide for instructions on how to install StatsForecast

    Install the necessary packages with pip install statsforecast

    pip install statsforecast from statsforecast import StatsForecast # required to instantiate StastForecast object and use cross-validation method ​ Load and explore the data As stated in the introduction, we’ll use the M4 Competition hourly dataset. We’ll first import the data from an URL using pandas.

    import pandas as pd

    Y_df = pd.read_parquet('https://datasets-nixtla.s3.amazonaws.com/m4-hourly.parquet') # load the data Y_df.head() unique_id ds y 0 H1 1 605.0 1 H1 2 586.0 2 H1 3 586.0 3 H1 4 559.0 4 H1 5 511.0 The input to StatsForecast is a data frame in long format with three columns: unique_id, ds and y:

    The unique_id (string, int, or category) represents an identifier for the series. The ds (datestamp or int) column should be either an integer indexing time or a datestamp in format YYYY-MM-DD or YYYY-MM-DD HH:MM:SS. The y (numeric) represents the measurement we wish to forecast. The data in this example already has this format, so no changes are needed.

    To keep the time required to execute this notebook to a minimum, we’ll only use one time series from the data, namely the one with unique_id == 'H1'. However, you can use as many as you want, with no additional changes to the code needed.

    df = Y_df[Y_df['unique_id'] == 'H1'] # select time series We can plot the time series we’ll work with using StatsForecast.plot method.

    StatsForecast.plot(df)

    ​ Train model For this example, we’ll use StastForecast AutoETS. We first need to import it from statsforecast.models and then we need to instantiate a new StatsForecast object.

    The StatsForecast object has the following parameters:

    models: a list of models. Select the models you want from models and import them. freq: a string indicating the frequency of the data. See panda’s available frequencies. n_jobs: n_jobs: int, number of jobs used in the parallel processing, use -1 for all cores. Any settings are passed into the constructor. Then you call its fit method and pass in the historical data frame df.

    from statsforecast.models import AutoETS

    models = [AutoETS(season_length = 24)]

    sf = StatsForecast( df = df, models = models, freq = 'H', n_jobs = -1 ) ​ Perform time series cross-validation Once the StatsForecastobject has been instantiated, we can use the cross_validation method, which takes the following arguments:

    df: training data frame with StatsForecast format h (int): represents the h steps into the future that will be forecasted step_size (int): step size between each window, meaning how often do you want to run the forecasting process. n_windows (int): number of windows used for cross-validation, meaning the number of forecasting processes in the past you want to evaluate. For this particular example, we’ll use 3 windows of 24 hours.

    crossvalidation_df = sf.cross_validation( df = df, h = 24, step_size = 24, n_windows = 3 ) The crossvaldation_df object is a new data frame that includes the following columns:

    unique_id: index. If you dont like working with index just run crossvalidation_df.resetindex() ds: datestamp or temporal index cutoff: the last datestamp or temporal index for the n_windows. y: true value "model": columns with the model’s name and fitted value. crossvalidation_df.head() ds cutoff y AutoETS unique_id H1 677 676 691.0 677.761047 H1 678 676 618.0 607.817871 H1 679 676 563.0 569.437744 H1 680 676 529.0 537.340027 H1 681 676 504.0 515.571106 We’ll now plot the forecast for each cutoff period. To make the plots clearer, we’ll rename the actual values in each period.

    crossvalidation_df.rename(columns = {'y' : 'actual'}, inplace = True) # rename actual values

    cutoff = crossvalidation_df['cutoff'].unique()

    for k in range(len(cutoff)): cv = crossvalidation_df[crossvalidation_df['cutoff'] == cutoff[k]] StatsForecast.plot(df, cv.loc[:, cv.columns != 'cutoff'])

    Notice that in each cutoff period, we generated a forecast for the next 24 hours using only the data y before said period.

    ​ Evaluate results We can now compute the accuracy of the forecast using an appropiate accuracy metric. Here we’ll use the Root Mean Squared Error (RMSE). To do this, we first need to install datasetsforecast, a Python library developed by Nixtla that includes a function to compute the RMSE.

    pip install datasetsforecast from datasetsforecast.losses import rmse The function to compute the RMSE takes two arguments:

    The actual values. The forecasts, in this case, AutoETS. rmse = rmse(crossvalidation_df['actual'], crossvalidation_df['AutoETS']) print("RMSE using cross-validation: ", rmse) RMSE using cross-validation: 33.90342 This measure should better reflect the predictive abilities of our model, since it used different time periods to test its accuracy.

    Tip

    Cross validation is especially useful when comparing multiple models. Here’s an example with multiple models and time series.

    ​

  66. Assistant

    Thank you for sharing the content from the website. Let's summarize the key points relevant to using the cross_validation function from the StatsForecast library and properly create the 'Ensemble' column:

    1. Time Series Cross-Validation: This process helps in evaluating model performance at different cutoff points in the past. Each window ends at the cutoff date, and the forecast is generated for the h steps or horizon.

    2. Performance Evaluation: After performing the cross-validation and obtaining the forecast results, typically a DataFrame is returned that includes the actual (y) and predicted values for each model across different cutoff points.

    3. Accuracy Metric: The Root Mean Squared Error (RMSE) or similar metrics are used to assess performance by comparing actual values with each model's forecasts.

    Now, regarding how to create your 'Ensemble' column without error, here is a clearer approach based on the tutorial content:

    • After Cros-Validation: Create the ensemble column once the cross-validation process is complete. This avoids creating 'Ensemble' within any loops or internal mechanisms of the function itself.
    • Mean Calculation: Using the returned crossvalidation_df, calculate the mean of the model forecasts directly.
    • Prevent Duplication: Ensure you don't overwrite or create the column twice, which could be the cause of the duplication error.

    Here's the code adapted from the process described in the content you shared, calculating the mean of the forecasts and then naming it 'Ensemble':

    from statsforecast import StatsForecast
    from statsforecast.models import AutoARIMA, AutoETS, AutoCES, DynamicOptimizedTheta
    import polars as pl
    import numpy as np
    
    # Assuming y_cl4 DataFrame and models are defined as per the tutorial
    sf = StatsForecast(models=models, freq='1w', n_jobs=-1)
    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18)
    
    # Given the printed crossvalidation_df columns, calculate mean of forecasts from all models
    forecast_columns = ['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']
    ensemble_mean = crossvalidation_df.select(
        pl.mean(forecast_columns).alias('mean_forecast')
    )
    
    # Adding the 'Ensemble' column to the original DataFrame
    crossvalidation_df = crossvalidation_df.hstack(ensemble_mean)
    
    # RMSE or other accuracy metric computation here
    # ...
    

    This code snippet uses the functionalities you'd expect based on the tutorial instructions. It selects the forecast columns, computes their mean forecast, and then stacks this back onto the results from the cross-validation. You can then use this ensemble for accuracy calculation and evaluation.

    As a reminder, select() builds the expression for the mean, and it's computed when executing hstack(). This should sidestep issues with mutable DataFrame states during looping. Make sure you adjust the forecast_columns list to match the column names in your actual crossvalidation_df.

    If you still experience issues, checking back with the documentation to ensure there haven't been any changes to the usage of cross_validation would be advisable. If the documentation is not clear or the behavior of the function still seems inconsistent with expectations, seeking direct support through the library's community or support channels may help resolve these anomalies.

  67. User

    this is the evaluation part from the example I that gave you earlier to check if my code was done correctly on ensemble from itertools import product

    import fire import numpy as np import pandas as pd from datasetsforecast.losses import mape, smape

    from src.data import get_data

    def evaluate(lib: str, model: str, dataset: str, group: str): y_test, horizon, freq, seasonality = get_data('data/', dataset, group, False) y_test = y_test['y'].values.reshape(-1, horizon)

    forecast = pd.read_csv(f'data/{lib}-forecasts-{dataset}-{group}.csv')
    y_hat = forecast[model].values.reshape(-1, horizon)
    
    evals = {}
    for metric in (mape, smape):
        metric_name = metric.__name__
        loss = metric(y_test, y_hat)
        evals[metric_name] = loss 
    
    evals = pd.DataFrame(evals, index=[f'{dataset}_{group}']).rename_axis('dataset').reset_index()
    times = pd.read_csv(f'data/{lib}-time-{dataset}-{group}.csv')
    times['model'] = model
    evals = pd.concat([evals, times], axis=1)
    
    return evals
    

    def main(test: bool = False): if test: groups = ['Other'] else: groups = ['Monthly', 'Yearly', 'Other', 'Quarterly'] lib = ['StatisticalEnsemble'] models = ['StatisticalEnsemble', 'AutoARIMA', 'CES', 'AutoETS', 'DynamicOptimizedTheta'] datasets = ['M3'] evaluation = [evaluate(lib, model, dataset, group) for lib, model, group in product(lib, models, groups) for dataset in datasets] evaluation = [eval_ for eval_ in evaluation if eval_ is not None] evaluation = pd.concat(evaluation) evaluation = evaluation[['dataset', 'model', 'mape', 'smape', 'time']] evaluation['time'] /= 60 #minutes evaluation = evaluation.set_index(['dataset', 'model']).stack().reset_index() evaluation.columns = ['dataset', 'model', 'metric', 'val'] evaluation = evaluation.set_index(['dataset', 'metric', 'model']).unstack().round(3) evaluation = evaluation.droplevel(0, 1).reset_index() evaluation.to_csv('data/evaluation.csv') smape = evaluation.query('metric=="smape"').T time = evaluation.query('metric=="time"').T if test: expected_results = { 'AutoARIMA': 4.87, 'CES': 4.85, 'AutoETS': 4.35, 'DynamicOptimizedTheta': 4.54, 'StatisticalEnsemble': 4.173 } expected_results = pd.Series(expected_results) pd.testing.assert_series_equal( smape.loc[expected_results.index].iloc[:, 0].astype(float), expected_results, check_names=False, rtol=1e-2, check_exact=False, ) assert time.loc[lib[0]].item() < 2.

    if name == 'main': fire.Fire(main)

  68. Assistant

    The evaluation code you provided takes a structured and systematic approach to computing performance metrics and execution time for different forecasting models applied to a specified dataset. It is designed to be used with the datasetsforecast package's mape and smape functions for evaluating forecasting accuracy.

    Here is an outline of the evaluation process based on the given code:

    1. Data Retrieval: The get_data function is called to obtain test data (y_test), which includes the actual time series values.

    2. Reading Forecasts: Forecasted values are read from a CSV file where each row corresponds to predictions made by a specified model.

    3. Reshaping Data: Both actual and predicted values are reshaped to align with the forecast horizon, facilitating the direct comparison between the two.

    4. Computing Errors: The Mean Absolute Percentage Error (MAPE) and Symmetric Mean Absolute Percentage Error (sMAPE) are computed for the forecast against the actual data.

    5. Aggregating Results: The results are collated into a DataFrame which includes model names, metrics, and computation times.

    6. Normalization of Time: Computation times are converted into minutes for standardization.

    7. Reformatting for Output: The final DataFrame is indexed and stacked such that each model's performance metrics and computation time are easily comparable.

    8. Saving the Results: The DataFrame is exported to a CSV file for further analysis or reporting.

    9. Testing: If in test mode, the script checks that the sMAPE values are within the expected ranges and that the computation time does not exceed a certain threshold. This could be part of automated testing or validation procedures.

    Your ensemble approach should fit into this evaluation workflow in the following way:

    • After completing the cross-validation and ensemble calculation, you would save the forecasts to a CSV file as done for the other models in the pipeline.
    • When calling the evaluate function, you will include the 'Ensemble' or 'Ensemble2' as part of the models list.
    • The reshaping and error computations would proceed similarly for the ensemble forecasts.
    • Ensure the reshaped ensemble predictions and actual values have consistent shapes and correspond to the same time points for accurate error calculation.

    Regarding your original issue with the internal loop in the cross_validation method causing the duplication of the 'Ensemble' column, this evaluation code does not appear to address that. However, once you have a DataFrame resulting from cross_validation, you could apply similar logic to compute and append your ensemble predictions before proceeding with these evaluation steps.

    To integrate the ensemble model within this pipeline, you have to ensure that:

    • The cross-validation DataFrame is prepared correctly with the ensemble column added.
    • The ensemble forecasts are stored in the expected CSV format.
    • The evaluation script is aware of the new 'Ensemble' model to perform the accuracy measurements.

    If you still encounter problems during the cross-validation step, it would be advisable to inspect the structure of crossvalidation_df closely immediately after its computation to make the necessary adjustments before appending the ensemble results.

    If you have issues with ensemble execution duration or other unexpected behavior despite these adjustments, the next step would be to seek direct assistance from the library's developers or a knowledgeable user community.

  69. User

    so this line of code, since you said there's some limitations on the polars dataframe right? would it be easier to use a list instead? or what's the alternative

  70. Assistant

    Yes, you could use a list comprehension along with Polars' ability to operate on columns to calculate the ensemble mean without directly invoking DataFrame-wide operations that might conflict with Polars' parallel processing or lazy evaluation model.

    The following alternative example uses a list to store the mean forecast values, then adds them as a new column to the crossvalidation_df DataFrame:

    # Assuming the other parts of your code have run, and we have `crossvalidation_df` ready.
    
    # Calculate ensemble mean using list comprehension and add the result as a new column
    ensemble_means = [
        np.mean([
            row['AutoARIMA'], 
            row['AutoETS'], 
            row['CES'], 
            row['DynamicOptimizedTheta']
        ]) for row in crossvalidation_df.rows()
    ]
    
    # Create a new Series and then convert to DataFrame
    ensemble_df = pl.DataFrame({'Ensemble': ensemble_means})
    
    # Use 'hstack' to concatenate the new 'Ensemble' column to the crossvalidation_df
    crossvalidation_df = crossvalidation_df.hstack(ensemble_df)
    

    In this code snippet:

    1. We calculate the mean of the specified columns for each row using a list comprehension, which gives us a Python list of ensemble means.
    2. We create a new DataFrame (ensemble_df) from this list, naming the column 'Ensemble'.
    3. We horizontally stack (hstack) this new DataFrame onto the crossvalidation_df, which effectively adds the ensemble means as a new column.

    This approach bypasses the issue of Polars' DataFrame.mean() method, which operates across columns and returns a DataFrame instead of a Series, and it avoids any ambiguity with Polars' lazy evaluation mechanism. By calculating the ensemble mean outside of Polars' DataFrame methods, we have more control over the execution and can more easily add this calculation back into the DataFrame.

  71. User

    this line of code, everything has to be on the same line, we can't break into separate lines since I told you it was taking 56 minutes and nothing working crossvalidation_df = crossvalidation_df.with_columns( pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias('Ensemble') )

  72. Assistant

    If running the operation within one line is critical due to the way Polars handles lazy evaluation or other concerns, and if the error with the internal loop within cross_validation persists, we can try constructing the ensemble mean manually for each row, but using an expression that Polars can evaluate lazily.

    Let's consider using the over clause available in Polars, which allows for window functions. This can be helpful in calculating column-wise operations without resorting to a DataFrame-wide mean that may be causing issues. Since the actual syntax and mechanics can be complex and dependent on the specific version of Polars you are using, below is a simplified conceptual approach:

    # These are the model forecast columns from which we'll create the 'Ensemble' column.
    forecast_columns = ['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']
    
    # Create an expression to calculate the row-wise mean. This creates an expression for each forecast column.
    ensemble_expr = pl.concat_list(forecast_columns).arr.mean().alias('Ensemble')
    
    # Add this expression to the crossvalidation_df
    crossvalidation_df = crossvalidation_df.with_columns(ensemble_expr)
    

    In this approach:

    1. pl.concat_list(forecast_columns) is used to combine the forecast columns into a Series of lists.
    2. That resulting Series is used .arr.mean() to calculate the mean of each list (which in this case, would be row-wise).
    3. The result of that calculation, which is now a Series, is given the column name 'Ensemble' using .alias().
    4. This new 'Ensemble' Series is added to the existing crossvalidation_df with .with_columns().

    This assumes that .arr.mean() would give us the average across each list in the Series. However, please note that the actual usage could be different based on the version of Polars and its API at the time.

    The above snippet maintains a single line for the addition of the 'Ensemble' column. It ensures the execution happens lazily until we actually need to use crossvalidation_df, at which point Polars will execute all pending operations efficiently.

    Should none of these approaches work, it would be worth checking if there is something specific about the cross_validation output that is affecting this process or if there's something about the Polars framework you're using that might require a different approach altogether. It may also be helpful to check your version of Polars or consider alternative methods of calculating the ensemble, such as using a map function or perhaps an entirely different framework that more easily supports row-wise operations.

  73. User

    where do I put this code in my code

  74. Assistant

    The code snippet to calculate the ensemble mean should be placed right after you have obtained the DataFrame from the cross-validation (crossvalidation_df). This ensures that you're not creating or trying to append the 'Ensemble' data during an internal loop used by the cross_validation method.

    Here's how your code flow should look like:

    # Initialize the StatsForecast model
    sf = StatsForecast(models=models, freq='1w', n_jobs=-1)
    
    # Perform cross-validation with a step size of 1 to mimic an expanding window
    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True)
    
    # The model forecast columns from which you'll create the 'Ensemble' column
    forecast_columns = ['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']
    
    # Construct the expression that calculates the row-wise mean
    ensemble_expr = pl.concat_list(forecast_columns).arr.mean().alias('Ensemble')
    
    # Add the ensemble expression to crossvalidation_df
    crossvalidation_df = crossvalidation_df.with_columns(ensemble_expr)
    
    # Now crossvalidation_df has the 'Ensemble' column with mean values.
    # You can proceed with further analysis or compute error metrics.
    

    Once you've inserted this snippet after obtaining crossvalidation_df, you can then continue with the rest of your forecasting analysis, such as calculating errors and other statistics.

    Please make sure that you use the updated snippet in an environment where you have already installed the required version of the Polars library that supports these operations. If the methods used in the snippet do not match the API available in your installed version of Polars, you may need to update the library or modify the code accordingly.

  75. User

    so is there another way in polars to get this line of code to work? crossvalidation_df = crossvalidation_df.with_columns( pl.col([‘AutoARIMA’, ‘AutoETS’, ‘CES’, ‘DynamicOptimizedTheta’]).mean().alias(‘Ensemble’) )I mean I think the only issue with the internal loop and .alias('Ensemble'), if we can change .alias to something else that would only run at the end or once

  76. Assistant

    In Polars, the .alias() method is used to rename columns resulting from expressions within with_columns(). However, we can't change .alias() to something else to make it execute once, because .alias() itself isn't the issue. The problem seems to arise from appending a column to a DataFrame within a loop.

    An effective way to avoid the internal loop problem is to first apply the mean calculation across the specified columns for each row and then assign the result to a new column after the loop is complete.

    We can create a 'list' of the columns we want to average and then use map to apply a function across this list for each row. After we've computed the mean, we can add it as a new column. However, keep in mind that this is a custom solution and might not be as performant as built-in vectorized operations.

    Here's an alternative way to compute the ensemble mean for each row, and to assign it to the 'Ensemble' column after the cross-validation loop:

    # Perform cross-validation with a step size of 1 to mimic an expanding window
    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True)
    
    # Assuming the following model columns
    model_columns = ['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']
    
    # Compute the mean of the model forecasts for each row
    ensemble_mean = crossvalidation_df.select(
        [(pl.col(model).cast(float) for model in model_columns)]
    ).with_columns(
        pl.sum(model_columns).alias('sum_forecasts') / len(model_columns)
    ).select(['unique_id', 'ds', 'sum_forecasts'])
    
    # This provides us with a DataFrame containing the 'unique_id', 'ds' and 'Ensemble' (sum_mean_forecasts) columns.
    
    # Then we can join this DataFrame back using unique_id and ds to match the rows.
    crossvalidation_df = crossvalidation_df.join(ensemble_mean, on=['unique_id', 'ds'], how='left')
    
    # Ensure the correct column name is used
    crossvalidation_df = crossvalidation_df.rename({'sum_forecasts': 'Ensemble'})
    

    Several aspects are crucial to this approach:

    1. We sum up the forecast columns after casting them to floats to prevent any type-related issues.
    2. We directly calculate the mean by summing and then dividing by the number of model columns.
    3. We join the mean back to our cross-validation DataFrame.

    This method allows us to calculate and join the mean in a controlled way and add the 'Ensemble' column only once, outside of any loop mechanism of the cross_validation method. This should prevent the duplication error from occurring. If Polars throws errors regarding naming or joining, you might need to adapt the column names to fit your DataFrame structure.

  77. User

    but I told you that I have tried your ways but it was taking too long and I don't know what was going on, the cross validation was done in 7 minutes but the kernel kept running to 56 minutes and nothing changes so I had to interrupt, I don't know if because there are 113 different series so there were too much value for the append or not, so we can't do this way. have to keep it this way and change something to make it work? maybe therr's another way for alias.ensemble to keep rewriting just the column name? crossvalidation_df = crossvalidation_df.with_columns( pl.col([‘AutoARIMA’, ‘AutoETS’, ‘CES’, ‘DynamicOptimizedTheta’]).mean().alias(‘Ensemble’) )

  78. Assistant

    If you're running into performance issues, it's possible that breaking down the column-wise operation and aliasing into smaller steps might help. When dealing with large DataFrames or costly computations, you typically want to ensure that you're making good use of the library's lazy evaluation.

    Let's rework the approach without using aliasing operation that might be causing the duplication error in the first place. We'll use an alternative way of appending the mean of the specified columns manually and then renaming the resultant column.

    Here's how you can perform the operation without using .alias('Ensemble'):

    1. Compute the mean outside of the with_columns operation.
    2. Create the new column manually.
    3. Append the new column to crossvalidation_df.
    # Your existing code to get `crossvalidation_df` from `cross_validation`.
    
    # Compute the mean of the forecasts manually
    mean_forecasts = (
        crossvalidation_df['AutoARIMA'] +
        crossvalidation_df['AutoETS'] +
        crossvalidation_df['CES'] +
        crossvalidation_df['DynamicOptimizedTheta']
    ) / 4
    
    # Manually create the new column (Performance-intensive step)
    crossvalidation_df = crossvalidation_df.with_column(mean_forecasts.alias('Ensemble'))
    
    # Continue with the rest of your analysis...
    

    The key here is calculating the forecast mean outside any looping structure—essentially precomputing it before the heavy with_column operation.

    Please test this adjusted snippet after the cross-validation step. If this still results in a long-running kernel, it might be worthwhile to analyze each step for potential performance bottlenecks or reach out to the Polars community for optimized patterns of use. Every operation could carry an overhead due to the sheer size of your dataset. Since the cross-validation doesn't take long, the issue may lie in how the ensemble mean computation scales with the size of your data.

    Also, if you're sure that the bottleneck is not from .alias('Ensemble'), you could try performing your operations using eager evaluation in Polars, rather than lazy evaluation, to see if performance improves:

    # Instead of using with_columns, you directly assign the calculation to a new column with eager evaluation.
    crossvalidation_df['Ensemble'] = mean_forecasts
    

    Remember, eager evaluation will execute each operation as it is encountered, whereas lazy evaluation will build a computation graph behind the scenes and only execute when necessary, often at the end of the data manipulation pipeline when results are brought into memory. With eager evaluation, performance can sometimes be better predicted and debugged since each step's execution and memory footprint are easier to track.

  79. User

    you giving the same solution that doesn't work again. please stop. I'm running your solution right now, the cross validation already but it's mean calculation and ensemble column not working, it's taking too long and there's no output

  80. Assistant

    I apologize for the confusion and for any frustration the repeated solutions may have caused. Let's take a different approach.

    If appending the calculated column directly within the with_columns method is causing issues due to an internal loop, then you might want to try a different method that doesn't rely on it. Here's an alternative method using select and mean within a separate list comprehension. This can be more efficient because it leverages Polars' execution model better.

    # Perform the cross-validation process.
    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True)
    
    # Select the columns to calculate the mean
    mean_forecasts = crossvalidation_df.select([
        pl.col('AutoARIMA'), 
        pl.col('AutoETS'), 
        pl.col('CES'), 
        pl.col('DynamicOptimizedTheta')
    ]).mean()
    
    # Construct the new column 'Ensemble'
    # By extracting only the series we are interested in, we can work with them as Python lists.
    ensemble_means = [
        mean_forecasts[0].to_numpy()
        for row in zip(
            crossvalidation_df['AutoARIMA'].to_numpy(), 
            crossvalidation_df['AutoETS'].to_numpy(),
            crossvalidation_df['CES'].to_numpy(), 
            crossvalidation_df['DynamicOptimizedTheta'].to_numpy()
        )
    ]
    
    # Convert the list to a Series before adding it to the DataFrame.
    ensemble_series = pl.Series('Ensemble', ensemble_means)
    
    # Add the series as a column to the DataFrame.
    crossvalidation_df = crossvalidation_df.with_column(ensemble_series)
    

    In this solution, we calculate the mean forecasts outside of the cross-validation loop by using a list comprehension to iterate over the rows of the selected columns. Then we create a Polars Series from the list of ensemble means and append it to the crossvalidation_df.

    This approach should help prevent performance bottlenecks associated with Polars DataFrame manipulations within loops. Please adjust any specific naming or structure issues to match your exact setup.

    If this approach is still problematic or if it's taking an excessively long time, it would be best to profile the code to identify exactly where the bottleneck is occurring. Profiling the code will give you insights into which operation or function call is taking the most time, which can then guide optimization efforts more effectively. You can use Python's built-in cProfile module, the line_profiler package, or any other profiling tool that works with your development setup.

    Moreover, if performance issues persist, consider reaching out directly to the Polars developer community or support channels, as they might provide further insights or suggestions for optimizations given the specifics of Polars' execution and evaluation model.

  81. User

    again, the similar solution that breaking up that code. the mean and alias need to be on the same line of code crossvalidation_df = crossvalidation_df.with_columns( pl.col(['AutoARIMA', 'AutoETS', 'CES', 'DynamicOptimizedTheta']).mean().alias('Ensemble') )

  82. Assistant

    If you need the mean computation to return a single value per row, you could create a custom expression that's computed across columns, and then use that expression with with_columns() combined with the alias() method.

    However, as Polars evaluates expressions lazily, performance issues when operating on large datasets might still occur. If computation time is prohibitive, you might want to either optimize the existing process or consider utilizing a different method of running this computation that takes better advantage of Polars' functionality.

    One approach could be to decompose your expression into a series of smaller, more manageable expressions that can be run efficiently by Polars. Here's a refactored version of the previous approach using smaller steps:

    # Perform cross-validation with a step size of 1 to mimic an expanding window
    crossvalidation_df = sf.cross_validation(df=y_cl4, h=16, step_size=1, n_windows=18, sort_df=True)
    
    # Calculate the Ensemble mean across the selected forecast columns
    # We create an expression for each forecast column, ensuring that this is done in separate steps.
    ensemble_expr = (pl.col('AutoARIMA') + pl.col('AutoETS') + pl.col('CES') + pl.col('DynamicOptimizedTheta')) / 4
    
    # Add the expression as a single column to the DataFrame.
    # The computation of the mean expression across rows will only be evaluated here,
    # when it's actually needed for attaching to the DataFrame.
    crossvalidation_df = crossvalidation_df.with_columns(ensemble_expr.alias('Ensemble'))
    

    This approach maintains the use of with_columns() and alias() on the same line, but if there's an enhancive loop from the StatsForecast library causing issues, we need to verify that the above workflow does not trigger it. If the 'Ensemble' column is still duplicated, it may be necessary to look into whether the cross-validation function is perhaps being called multiple times or if there is any other part of the code that may be causing the duplication.

    Should the problem persist, consider the following strategies:

    • Profiling: Use Python's profiling tools to see where the computational bottleneck is.
    • Batching: Break down the cross-validation DataFrame into smaller batches, calculate the 'Ensemble', then concatenate the results.
    • Optimization: Look to optimize the mean calculation. Depending on the length and index alignment, this may include optimizing calculations by converting to numpy arrays first.
    • Parallelization: If not already utilized, parallelize operations where possible.
    • Memory Management: Inspect whether any memory-intensive operations can be optimized.

    Finally, if you're still having difficulty with code execution, I'd advise reviewing Polars' user guides or asking for help on community forums. The issue may need a more in-depth review that accounts for the specific characteristics of your data and the resources of the system you're using.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.