intervals = [1,2,3,5] lookback = 60 optimizer =…
- User
intervals = [1,2,3,5] look_back = 60
optimizer = keras.optimizers.Adam(learning_rate=0.0003,clipvalue=0.5)
for csv_file in csv_files:
file_path = os.path.join(csv_directory, csv_file) unique_part = file_path.split('_')[-2] df = pd.read_csv(r'C:\Users\arisa\Desktop\ddd\Binance_1INCHBUSD_d.csv') include_substrings = ["y_"] exact_columns_to_keep = ["Open", "High", "Low", "Close","volume_base", "volume_crypto", "tradecount",] filtered_columns = [col for col in df.columns if any(col.startswith(s) for s in include_substrings)] columns_to_keep = list(set(exact_columns_to_keep + filtered_columns)) df = df[columns_to_keep] df.head() features = df.drop([ 'y_High_1d', 'y_Low_1d', 'y_Priority_1d', 'y_High_2d', 'y_Low_2d', 'y_Priority_2d', 'y_High_3d', 'y_Low_3d', 'y_Priority_3d', 'y_High_5d', 'y_Low_5d', 'y_Priority_5d'], axis=1) # Scale the features and targets feature_scaler = MinMaxScaler(feature_range=(0, 1)) scaled_features = feature_scaler.fit_transform(features) joblib.dump(feature_scaler,f'x_scalers/x_scaler_{unique_part}.sav') for i in intervals: y_cols = [[f'y_High_{i}d', 'y_Low_{i}d', 'y_Priority_{i}d']] targets = df[y_cols] target_scaler = MinMaxScaler(feature_range=(0, 1)) scaled_targets = target_scaler.fit_transform(targets) joblib.dump(feature_scaler,f'y_scalers/y{i}_scaler_{unique_part}.sav') x_train = [] y_train = [] for i in range(look_back, len(scaled_features)): x_train.append(scaled_features[i-look_back:i]) y_train.append(scaled_targets[i]) # Assuming the target is the next time step x_train, y_train = np.array(x_train), np.array(y_train) input_shape = (x_train.shape[1], x_train.shape[2]) model = Sequential() model.add(LSTM(units = 100, activation = 'tanh', return_sequences=True)) model.add(Dropout(0.4)) model.add(LSTM(units = 240, activation = 'tanh')) model.add(Dropout(0.5)) model.add(Dense(units = 3)) model.compile(optimizer=optimizer, loss = 'mean_squared_error', metrics=[tf.keras.metrics.MeanAbsoluteError()]) model.fit(x_train, y_train,epochs = 1000) mae = model.evaluate(x_train, y_train) model.save(f'models/lstm_model_{unique_part}_y{i}_mae_{mae}.h5')error: { "name": "KeyError", "message": ""None of [Index([('y_High_1d', 'y_Low_{i}d', 'y_Priority_{i}d')], dtype='object')] are in the [columns]"", "stack": "--------------------------------------------------------------------------- KeyError Traceback (most recent call last) Cell In[3], line 42 40 for i in intervals: 41 y_cols = [[f'y_High_{i}d', 'y_Low_{i}d', 'y_Priority_{i}d']] ---> 42 targets = df[y_cols] 43 target_scaler = MinMaxScaler(feature_range=(0, 1)) 44 scaled_targets = target_scaler.fit_transform(targets)
File c:\Users\arisa\.conda\envs\tf\lib\site-packages\pandas\core\frame.py:4108, in DataFrame.getitem(self, key) 4106 if is_iterator(key): 4107 key = list(key) -> 4108 indexer = self.columns._get_indexer_strict(key, "columns")[1] 4110 # take() does not accept boolean indexers 4111 if getattr(indexer, "dtype", None) == bool:
File c:\Users\arisa\.conda\envs\tf\lib\site-packages\pandas\core\indexes\base.py:6200, in Index._get_indexer_strict(self, key, axis_name) 6197 else: 6198 keyarr, indexer, new_indexer = self._reindex_non_unique(keyarr) -> 6200 self._raise_if_missing(keyarr, indexer, axis_name) 6202 keyarr = self.take(indexer) 6203 if isinstance(key, Index): 6204 # GH 42790 - Preserve name from an Index
File c:\Users\arisa\.conda\envs\tf\lib\site-packages\pandas\core\indexes\base.py:6249, in Index._raise_if_missing(self, key, indexer, axis_name) 6247 if nmissing: 6248 if nmissing == len(indexer): -> 6249 raise KeyError(f"None of [{key}] are in the [{axis_name}]") 6251 not_found = list(ensure_index(key)[missing_mask.nonzero()[0]].unique()) 6252 raise KeyError(f"{not_found} not in index")
KeyError: "None of [Index([('y_High_1d', 'y_Low_{i}d', 'y_Priority_{i}d')], dtype='object')] are in the [columns]"" }
- Assistant
The issue in your code lies within the line where you're attempting to access the columns of the DataFrame
dfusing the variabley_cols. Here's the problematic line:targets = df[y_cols]The error message indicates that the way you've structured
y_colsis not correct for selecting columns in a pandas DataFrame.y_colsis structured as a list of lists rather than a list of strings (which are the column names). Here’s how you can address the error:- Adjust how you create
y_colsto be a flat list of column names. - Correct the indexing to access the DataFrame with this list.
Here's the corrected portion of your code:
for i in intervals: # Corrected to create a flat list of column names y_cols = [f'y_High_{i}d', f'y_Low_{i}d', f'y_Priority_{i}d'] # Now you can properly index df with y_cols targets = df[y_cols] # Continuing with your existing code... target_scaler = MinMaxScaler(feature_range=(0, 1)) scaled_targets = target_scaler.fit_transform(targets) joblib.dump(target_scaler,f'y_scalers/y{i}_scaler_{unique_part}.sav') # Changed to target_scaler to correctly save the scaler used for targetsAdditionally, note that there was a minor mistake related to the scalers you save for later use. I corrected it to save
target_scalerinstead offeature_scalerwhile dealing with the target columns. This is important for ensuring that you're using and saving the correct scaler objects meant for feature columns and target columns separately. - Adjust how you create
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.