im getting following error: 22:19:18.216 [info] Restart requested…
- User
im getting following error: 22:19:18.216 [info] Restart requested ~\Desktop\Stock-Price-Prediction-Using-LSTM-main\lastm_all.ipynb 22:19:18.218 [warn] Cancel all remaining cells due to dead kernel 22:19:19.477 [info] Process Execution: ~.conda\envs\tf\python.exe -c "import ipykernel; print(ipykernel.version); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.file)" 22:19:19.483 [info] Process Execution:
.conda\envs\tf\python.exe -m ipykernel_launcher --f=\AppData\Roaming\jupyter\runtime\kernel-v2-95208yFYTXnN1W1U.json > cwd: ~\Desktop\Stock-Price-Prediction-Using-LSTM-main 22:19:20.524 [info] Restarted 8277bb6d-9f1c-4aed-a911-2fd5d1ef7886with following message :
The Kernel crashed while executing code in the current cell or a previous cell. Please review the code in the cell(s) to identify a possible cause of the failure. Click here for more info. View Jupyter log for further details.
my code:
%%
import pandas as pd import datetime as dt from datetime import date import numpy as np import tensorflow as tf from sklearn.preprocessing import MinMaxScaler import keras import os import joblib from tensorflow.keras.layers import Dense, Dropout, LSTM from tensorflow.keras.models import Sequential import gc import time
%%
gpus = tf.config.experimental.list_physical_devices(‘GPU’)
if gpus:
try:
# Currently, memory growth needs to be the same across GPUs
for gpu in gpus:
tf.config.experimental.set_memory_growth(gpu, True)
logical_gpus = tf.config.experimental.list_logical_devices(‘GPU’)
print(f’{len(gpus)} Physical GPUs, {len(logical_gpus)} Logical GPUs’)
except RuntimeError as e:
# Memory growth must be set before GPUs have been initialized
print(e)
%%
csv_directory = r"C:\Users\arisa\Desktop\day_spot_summary_3" csv_files = [file for file in os.listdir(csv_directory) if file.endswith(‘.csv’)]
%%
intervals = [1,2,3,5] look_back = 60
optimizer = keras.optimizers.Adam(learning_rate=0.0003,clipvalue=0.5)
for csv_file in csv_files:
file_path = os.path.join(csv_directory, csv_file) unique_part = file_path.split('')[-2]
df = pd.read_csv(file_path)
include_substrings = ["y"] exact_columns_to_keep = [“Open”, “High”, “Low”, “Close”,“volume_base”, “volume_crypto”, “tradecount”,] filtered_columns = [col for col in df.columns if any(col.startswith(s) for s in include_substrings)]
columns_to_keep = list(set(exact_columns_to_keep + filtered_columns)) df = df[columns_to_keep]
features = df.drop([ ‘y_High_1d’, ‘y_Low_1d’, ‘y_Priority_1d’, ‘y_High_2d’, ‘y_Low_2d’, ‘y_Priority_2d’, ‘y_High_3d’, ‘y_Low_3d’, ‘y_Priority_3d’, ‘y_High_5d’, ‘y_Low_5d’, ‘y_Priority_5d’], axis=1)
Scale the features and targets
feature_scaler = MinMaxScaler(feature_range=(0, 1)) scaled_features = feature_scaler.fit_transform(features)
joblib.dump(feature_scaler,f’x_scalers/x{unique_part}scaler.sav’)
for p in intervals:
Corrected to create a flat list of column names
y_cols = [f’y_High{p}d’, f’y_Low{p}d’, f’y_Priority{p}d’]
Now you can properly index df with y_cols
targets = df[y_cols]
Continuing with your existing code…
target_scaler = MinMaxScaler(feature_range=(0, 1)) scaled_targets = target_scaler.fit_transform(targets) joblib.dump(target_scaler,f’y_scalers/y{p}{unique_part}scaler.sav’)
x_train = [] y_train = []
for i in range(look_back, len(scaled_features)): x_train.append(scaled_features[i-look_back:i]) y_train.append(scaled_targets[i]) # Assuming the target is the next time step
x_train, y_train = np.array(x_train), np.array(y_train)
input_shape = (x_train.shape[1], x_train.shape[2])
model = Sequential() model.add(LSTM(units = 100, activation = ‘tanh’, return_sequences=True)) model.add(Dropout(0.4)) model.add(LSTM(units = 240, activation = ‘tanh’)) model.add(Dropout(0.5)) model.add(Dense(units = 3)) model.compile(optimizer=optimizer, loss = ‘mean_squared_error’, metrics=[tf.keras.metrics.MeanAbsoluteError()])
model.fit(x_train, y_train,epochs = 1000)
mae = model.evaluate(x_train, y_train)
if os.path.exists(f’models/lstm_model{unique_part}y{p}mae{mae}.h5’): milis = round(time.time() * 1000) model.save(f’models/lstm_model{unique_part}_y{p}mae{mae}dup{milis}.h5’) else: model.save(f’models/lstm_model{unique_part}_y{p}mae{mae}.h5’)
tf.keras.backend.clear_session()
- Assistant
The error message you're seeing, "The Kernel crashed while executing code in the current cell or a previous cell," can stem from a variety of issues. Given the nature of the problem - running deep learning models using TensorFlow/Keras with LSTM - the issue might relate to resource allocation (such as memory issues with your GPU or CPU), or there might be a bug within the code causing the kernel to crash.
From the code provided, here are a few general steps and suggestions to help diagnose and potentially resolve the problem:
1. Check for Typos and Syntax
Ensure there are no typos especially in TensorFlow/Keras function calls and your Python code is syntactically correct. This might seem basic but can often be the issue. Unicode error marks like ‘ and ’ should be replaced with the standard quotation marks ' or " in Python syntax.
2. Resource Allocation (Memory Issues)
You've commented out the section of your code that handles GPU configuration. Memory issues, especially when dealing with deep learning models like LSTM, are common. The LSTMs can be quite memory-intensive, and you should make sure your environment has enough resources to handle the computations.
Uncommenting and correctly setting the GPU memory growth might solve your problem if it's related to running out of memory:
# Ensure the device configuration is correctly uncommented and correctly set gpus = tf.config.list_physical_devices('GPU') if gpus: try: for gpu in gpus: tf.config.experimental.set_memory_growth(gpu, True) except RuntimeError as e: print(e)Note that
tf.config.experimental.list_physical_devicesis the correct way to call it (nottf.config.experimental.list_physical_devices('GPU')).3. Reduce Model Complexity
Try reducing the complexity of the LSTM model. Perhaps start with fewer neurons in each layer, or fewer layers, to see if a simpler model would not crash the kernel.
4. Reduce the Batch Size
If you're running out of memory, reducing the batch size can help. Since I could not find a
batch_sizeparameter in yourmodel.fit(), this might be a good place to start. TensorFlow/Keras defaults thebatch_sizeto 32, which you might want to lower in case of memory issues.5. Run a Smaller Subset of Data
To test whether the issue is due to the size of the data, try running your model on a smaller subset of your dataset. If it runs successfully, the issue might relate to resource constraints when the model tries to handle the full dataset.
6. Update Libraries
Make sure your TensorFlow, Keras, and other relevant libraries are up-to-date. Sometimes, kernel crashes can be caused by bugs in the libraries that have been fixed in later versions.
7. Examine the Jupyter Log
The error message suggests viewing the Jupyter log for further details. The log can provide more specific information about why the kernel crashed. It might point you to a line of code where the issue occurred or provide more insight into memory or resource-related issues.
8. Environment Conflicts
Ensure there are no conflicting installations or environment issues, especially if you've switched between TensorFlow 1.x and 2.x or have multiple environments. Sometimes, a clean installation in a new environment can solve mysterious issues.
Addressing kernel crashes can involve some trial and error because the error message is quite generic. Hopefully, these steps guide you toward a solution or at least help narrow down the cause.