Make a stock market prediction model using lstm for current status weather to buy or…
- User
Make a stock market prediction model using lstm for current status weather to buy or sell the stock right now using NIFTY dataset from yfinance :
Data Preprocessing Do data preprocess, including data clean, replacing values with percent difference. Export new data which have been preprocessed. Create Labels Create trade decisions in regard to percent change of high and low price. There are 3 trade decisions: sell, buy, hold. These are labels for machine learning later. Explore the data Visualize the count of Buy, Hold, and Sell. Check for any correlation between the future price and the current quarter's features. Get and visualize feature importances using ExtraTreesClassifier. Select top 10 features (respectively correlation and importance) and save into Excel files. Training and prediction of Machine Learning Model Scale the Data. Split the Data. Plot confusion matrix. Fit and Train. Print out Evaluation Metrics. Tune model parameters. Show tuned results and choose optimal result. Model with the best parameters. Show results and confusion matrix from optimum parameters. Save the optimal model we trained into .m file. Classify new data using well-trained models Extract trained classifier and new data for prediction. Preprocess data. Do prediction and analysis result of prediction.
- Assistant
Before diving into the various steps involved in creating the model, let's install the necessary libraries:
!pip install yfinance !pip install pandas matplotlib seaborn scikit-learnNow, let's begin implementing the steps:
Step 1: Data Preprocessing
Start by importing the necessary libraries and fetching the NIFTY stock dataset from yfinance:
import yfinance as yf import pandas as pd import numpy as np # Fetch the NIFTY dataset nifty_df = yf.download('^NSEI', start='2020-01-01', end='2021-12-31')Now, apply the necessary preprocessing steps, such as cleaning the data and replacing values with percent differences.
# Clean the data nifty_df = nifty_df.dropna() # Calculate various percent differences nifty_df['PctOpenClose'] = (nifty_df['Close'] - nifty_df['Open']) / nifty_df['Open'] * 100 nifty_df['PctHighLow'] = (nifty_df['High'] - nifty_df['Low']) / nifty_df['Low'] * 100 nifty_df['PctChange'] = nifty_df['Close'].pct_change() * 100 # Remove the first row with NaN in 'PctChange' nifty_df = nifty_df.iloc[1:] # Export cleaned dataset nifty_df.to_csv('nifty_cleaned.csv')Step 2: Create Labels
Creat trade decisions based on the percent change of high and low prices.
def create_labels(df, col_name, threshold=0.5): labels = [] for index, row in df.iterrows(): future_price = row[col_name] if future_price >= threshold: labels.append(1) # Buy elif future_price <= -threshold: labels.append(-1) # Sell else: labels.append(0) # Hold return labels nifty_df['Labels'] = create_labels(nifty_df, 'PctChange')Step 3: Explore the Data
Visualize the count of Buy, Hold, and Sell.
import matplotlib.pyplot as plt import seaborn as sns sns.countplot(nifty_df['Labels']) plt.title('Count of Buy, Hold, and Sell') plt.xlabel('Trade Decisions') plt.ylabel('Count') plt.show()Step 4: Get and Visualize Feature Importance
from sklearn.ensemble import ExtraTreesClassifier X = nifty_df.drop(['Labels'], axis=1) y = nifty_df['Labels'] # Fit the ExtraTreesClassifier et_clf = ExtraTreesClassifier() et_clf.fit(X, y) # Visualize feature importances feature_importances = pd.Series(et_clf.feature_importances_, index=X.columns) feature_importances.nlargest(10).plot(kind='barh') plt.show()Step 5: Train and Predict with Machine Learning Model
Normalize or scale the data.
from sklearn.preprocessing import MinMaxScaler scaler = MinMaxScaler() scaled_X = scaler.fit_transform(X)Split the dataset into training and testing sets.
from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(scaled_X, y, test_size=0.2, random_state=42)Fit and train the model using an LSTM (Long Short-Term Memory) neural network.
from keras.models import Sequential from keras.layers import LSTM, Dense model = Sequential() model.add(LSTM(100, activation='relu', return_sequences=True, input_shape=(8, 1))) model.add(LSTM(100, activation='relu')) model.add(Dense(3)) model.compile(optimizer='adam', loss='mse') X_train = X_train.reshape((X_train.shape[0], X_train.shape[1], 1)) model.fit(X_train, y_train, epochs=50, verbose=0)Evaluate the model and show the confusion matrix.
from sklearn.metrics import confusion_matrix, precision_score, recall_score, f1_score, accuracy_score X_test = X_test.reshape((X_test.shape[0], X_test.shape[1], 1)) y_pred = model.predict(X_test) print("Confusion Matrix:") print(confusion_matrix(y_test, y_pred)) print("\nEvaluation Metrics:") print("Accuracy: ", accuracy_score(y_test, y_pred)) print("Precision: ", precision_score(y_test, y_pred, average='macro')) print("Recall: ", recall_score(y_test, y_pred, average='macro')) print("F1-Score: ", f1_score(y_test, y_pred, average='macro'))Save the trained model for future use.
model.save('nifty_lstm_model.h5')Step 6: Classify New Data Using the Well-Trained Model
# Import the necessary libraries from keras.models import load_model # Load the trained model trained_model = load_model('nifty_lstm_model.h5') # Load the new dataset for prediction new_data = pd.read_csv('new_data.csv') # Preprocess the new dataset preprocessed_new_data = preprocess_data(new_data) # Assumes the same preprocessing is applied as before # Predict predicted_labels = trained_model.predict(preprocessed_new_data)Analyze the result of the prediction to make buy, hold or sell decisions.