Skip to content
The Chat Archive
gpt-4-0125-preview· OpenAI WildChat-4.8M (AI2)

Code the following task in python. 2.3 Write a computer program using a high-level…

  1. User

    Code the following task in python. 2.3 Write a computer program using a high-level programming language of your choice (Python with NumPy or C/C++ or Java) to train and test the neural network using the Backpropagation algorithm from scratch WITHOUT using any machine learning library.

  2. Assistant

    Certainly! Given your preference, I'll provide a Python example that implements a simple neural network using the backpropagation algorithm from scratch. This will be a very basic implementation to keep things clear and understandable without delving into too much complexity.

    Goal: To implement a neural network with one hidden layer to solve a simple classification problem.

    Key Concepts:

    • Initialization: We'll initialize the weights and biases with small random values.
    • Forward Pass: Calculate the output of the neural network.
    • Backpropagation: Compute the gradients and update the weights and biases.

    Simple Neural Network in Python

    import numpy as np
    
    # Sigmoid activation function and its derivative
    def sigmoid(x):
        return 1 / (1 + np.exp(-x))
    
    def sigmoid_derivative(x):
        return x * (1 - x)
    
    # Input datasets
    inputs = np.array([[0,0],[0,1],[1,0],[1,1]])
    expected_output = np.array([[0],[1],[1],[0]])
    
    # Hyperparameters
    epochs = 10000
    learning_rate = 0.1
    inputLayerNeurons, hiddenLayerNeurons, outputLayerNeurons = 2, 2, 1
    
    # Random weights and bias initialization
    hidden_weights = np.random.uniform(size=(inputLayerNeurons, hiddenLayerNeurons))
    hidden_bias = np.random.uniform(size=(1, hiddenLayerNeurons))
    output_weights = np.random.uniform(size=(hiddenLayerNeurons, outputLayerNeurons))
    output_bias = np.random.uniform(size=(1, outputLayerNeurons))
    
    # Training algorithm
    for _ in range(epochs):
        # Forward Propagation
        hidden_layer_activation = np.dot(inputs, hidden_weights)
        hidden_layer_activation += hidden_bias
        hidden_layer_output = sigmoid(hidden_layer_activation)
    
        output_layer_activation = np.dot(hidden_layer_output, output_weights)
        output_layer_activation += output_bias
        predicted_output = sigmoid(output_layer_activation)
    
        # Backpropagation
        error = expected_output - predicted_output
        d_predicted_output = error * sigmoid_derivative(predicted_output)
        
        error_hidden_layer = d_predicted_output.dot(output_weights.T)
        d_hidden_layer = error_hidden_layer * sigmoid_derivative(hidden_layer_output)
    
        # Updating Weights and Biases
        output_weights += hidden_layer_output.T.dot(d_predicted_output) * learning_rate
        output_bias += np.sum(d_predicted_output,axis=0,keepdims=True) * learning_rate
        hidden_weights += inputs.T.dot(d_hidden_layer) * learning_rate
        hidden_bias += np.sum(d_hidden_layer,axis=0,keepdims=True) * learning_rate
    
    # Displaying results after training
    print("Final hidden weights: ", hidden_weights)
    print("Final hidden bias: ", hidden_bias)
    print("Final output weights: ", output_weights)
    print("Final output bias: ", output_bias)
    print("\nOutput from neural network after 10,000 epochs: ", predicted_output)
    

    Notes

    • This neural network setup is simplistic and designed for educational purposes. Real-world applications would require more sophisticated approaches, including deeper networks, different activation functions, regularization techniques, etc.
    • The neural network takes 2 inputs and predicts a single output using a hidden layer with 2 neurons.
    • For simplicity, we've chosen a XOR problem ([0,0] and [1,1] mapping to 0 while [0,1] and [1,0] map to 1), which cannot be solved using linear models.
    • Tune epochs and learning_rate as required. Too few epochs might result in underfitting, while too many could lead to overfitting. Adjust learning_rate to control how drastically you update the weights at each step.
    • This is a basic version of backpropagation. In practice, you might implement variations like stochastic gradient descent (SGD) or add enhancements like momentum.