Skip to content
The Chat Archive
gpt-4-0314· OpenAI WildChat-4.8M (AI2)

Make a stock market prediction model using lstm for current status weather to buy or…

  1. User

    Make a stock market prediction model using lstm for current status weather to buy or sell the stock right now using NIFTY dataset from yfinance :

    Data Preprocessing Do data preprocess, including data clean, replacing values with percent difference. Export new data which have been preprocessed. Create Labels Create trade decisions in regard to percent change of high and low price. There are 3 trade decisions: sell, buy, hold. These are labels for machine learning later. Explore the data Visualize the count of Buy, Hold, and Sell. Check for any correlation between the future price and the current quarter's features. Get and visualize feature importances using ExtraTreesClassifier. Select top 10 features (respectively correlation and importance) and save into Excel files. Training and prediction of Machine Learning Model Scale the Data. Split the Data. Plot confusion matrix. Fit and Train. Print out Evaluation Metrics. Tune model parameters. Show tuned results and choose optimal result. Model with the best parameters. Show results and confusion matrix from optimum parameters. Save the optimal model we trained into .m file. Classify new data using well-trained models Extract trained classifier and new data for prediction. Preprocess data. Do prediction and analysis result of prediction.

  2. Assistant

    Before diving into the various steps involved in creating the model, let's install the necessary libraries:

    !pip install yfinance
    !pip install pandas matplotlib seaborn scikit-learn
    

    Now, let's begin implementing the steps:

    Step 1: Data Preprocessing

    Start by importing the necessary libraries and fetching the NIFTY stock dataset from yfinance:

    import yfinance as yf
    import pandas as pd
    import numpy as np
    
    # Fetch the NIFTY dataset
    nifty_df = yf.download('^NSEI', start='2020-01-01', end='2021-12-31')
    

    Now, apply the necessary preprocessing steps, such as cleaning the data and replacing values with percent differences.

    # Clean the data
    nifty_df = nifty_df.dropna()
    
    # Calculate various percent differences
    nifty_df['PctOpenClose'] = (nifty_df['Close'] - nifty_df['Open']) / nifty_df['Open'] * 100
    nifty_df['PctHighLow'] = (nifty_df['High'] - nifty_df['Low']) / nifty_df['Low'] * 100
    nifty_df['PctChange'] = nifty_df['Close'].pct_change() * 100
    
    # Remove the first row with NaN in 'PctChange'
    nifty_df = nifty_df.iloc[1:]
    
    # Export cleaned dataset
    nifty_df.to_csv('nifty_cleaned.csv')
    

    Step 2: Create Labels

    Creat trade decisions based on the percent change of high and low prices.

    def create_labels(df, col_name, threshold=0.5):
        labels = []
        for index, row in df.iterrows():
            future_price = row[col_name]
            if future_price >= threshold:
                labels.append(1)    # Buy
            elif future_price <= -threshold:
                labels.append(-1)   # Sell
            else:
                labels.append(0)    # Hold
    
        return labels
    
    nifty_df['Labels'] = create_labels(nifty_df, 'PctChange')
    

    Step 3: Explore the Data

    Visualize the count of Buy, Hold, and Sell.

    import matplotlib.pyplot as plt
    import seaborn as sns
    
    sns.countplot(nifty_df['Labels'])
    
    plt.title('Count of Buy, Hold, and Sell')
    plt.xlabel('Trade Decisions')
    plt.ylabel('Count')
    plt.show()
    

    Step 4: Get and Visualize Feature Importance

    from sklearn.ensemble import ExtraTreesClassifier
    
    X = nifty_df.drop(['Labels'], axis=1)
    y = nifty_df['Labels']
    
    # Fit the ExtraTreesClassifier
    et_clf = ExtraTreesClassifier()
    et_clf.fit(X, y)
    
    # Visualize feature importances
    feature_importances = pd.Series(et_clf.feature_importances_, index=X.columns)
    feature_importances.nlargest(10).plot(kind='barh')
    plt.show()
    

    Step 5: Train and Predict with Machine Learning Model

    Normalize or scale the data.

    from sklearn.preprocessing import MinMaxScaler
    
    scaler = MinMaxScaler()
    scaled_X = scaler.fit_transform(X)
    

    Split the dataset into training and testing sets.

    from sklearn.model_selection import train_test_split
    
    X_train, X_test, y_train, y_test = train_test_split(scaled_X, y, test_size=0.2, random_state=42)
    

    Fit and train the model using an LSTM (Long Short-Term Memory) neural network.

    from keras.models import Sequential
    from keras.layers import LSTM, Dense
    
    model = Sequential()
    model.add(LSTM(100, activation='relu', return_sequences=True, input_shape=(8, 1)))
    model.add(LSTM(100, activation='relu'))
    model.add(Dense(3))
    
    model.compile(optimizer='adam', loss='mse')
    
    X_train = X_train.reshape((X_train.shape[0], X_train.shape[1], 1))
    model.fit(X_train, y_train, epochs=50, verbose=0)
    

    Evaluate the model and show the confusion matrix.

    from sklearn.metrics import confusion_matrix, precision_score, recall_score, f1_score, accuracy_score
    
    X_test = X_test.reshape((X_test.shape[0], X_test.shape[1], 1))
    y_pred = model.predict(X_test)
    
    print("Confusion Matrix:")
    print(confusion_matrix(y_test, y_pred))
    
    print("\nEvaluation Metrics:")
    print("Accuracy: ", accuracy_score(y_test, y_pred))
    print("Precision: ", precision_score(y_test, y_pred, average='macro'))
    print("Recall: ", recall_score(y_test, y_pred, average='macro'))
    print("F1-Score: ", f1_score(y_test, y_pred, average='macro'))
    

    Save the trained model for future use.

    model.save('nifty_lstm_model.h5')
    

    Step 6: Classify New Data Using the Well-Trained Model

    # Import the necessary libraries
    from keras.models import load_model
    
    # Load the trained model
    trained_model = load_model('nifty_lstm_model.h5')
    
    # Load the new dataset for prediction
    new_data = pd.read_csv('new_data.csv')
    
    # Preprocess the new dataset
    preprocessed_new_data = preprocess_data(new_data)  # Assumes the same preprocessing is applied as before
    
    # Predict
    predicted_labels = trained_model.predict(preprocessed_new_data)
    

    Analyze the result of the prediction to make buy, hold or sell decisions.