Write Python code to implement a neural network from scratch without libraries.
- User
Write Python code to implement a neural network from scratch without libraries.
- Assistant
Clarifying neural network request
The user wants Python code to create a neural network from scratch without using any libraries. That likely means no machine learning libraries like TensorFlow or PyTorch, but I wonder if I can use numpy. Requests like this often imply using pure Python, which could exclude numpy altogether. Since they haven't specified details like the type of tasks or dataset, I think they might be looking for simple educational code for a fully connected network, focusing on standard library usage and basic operations.Defining neural network constraints
The user mentioned implementing a neural network from scratch without any libraries, which likely means avoiding even common libraries like numpy, but I think utilities from the standard library, like random and math, should be fine. I need to create a self-contained script featuring utility functions for operations like the dot product, activation functions, and cross-entropy loss. I have to keep explanations minimal and focus mainly on providing the code, possibly with brief comments, while ensuring it handles something simple like the XOR dataset for demonstration.Creating the neural network class
I think it would be useful to show how the training works, so I’ll create a class called
NeuralNetwork. In the__init__method, I'll randomly create weights and biases based on the provided layer sizes. Theforwardmethod will handle classification, using the sigmoid activation for hidden layers and softmax for the output layer. For thebackwardmethod, I'll compute gradients using cross-entropy and softmax, followed by updating weights with stochastic gradient descent. I’ll represent weight matrices as lists of lists, run it on single samples, and then code the backpropagation using these calculations.Implementing the backpropagation stepI’m now focusing on the backpropagation for each layer, starting from the last layer back to the first. For each layer ( L ), I’ll calculate the weight gradients and the biases using the delta values. I need to compute the deltas for the hidden layers as well and store the intermediate values during the forward pass. I might combine the forward and backward processes into a single
train_stepfunction. I’ll also need to set up the neural network class with arbitrary layer sizes and a learning rate, randomly initializing weights and biases while keeping the structure clear and systematic.Implementing forward and training methodsI’m implementing the forward method in my neural network class. This function will calculate the activations and intermediate values for each layer based on input ( x ). I’ll also create a
trainmethod that handles multiple epochs, processes data, and updates weights while calculating loss. For each training example, I’ll use a private method_train_singleto perform forward propagation, computing losses and gradients in the backward pass, ensuring I handle the shapes and indices correctly throughout. I want to keep it clear and functional.Defining layer indexing and training logicI’m setting up a 1-based layer indexing system for the neural network. Layer 0 is the input, layer 1 is the first hidden layer, up to the output layer, which is layer L. I need to ensure that weight indices align with the corresponding layers for forward and backward propagation. For the training method, I must account for cases where there may not be any hidden layers, handling special conditions gracefully. I’m also planning a small script for an XOR training example to demonstrate the neural network's functionality.Writing a simple neural network in Python
I'm going to add comments at the top of the code to explain that it's a simple fully-connected neural network built from scratch without using external libraries. I need to ensure that built-in functions from
mathandrandomare fine, as they’re not considered external libraries. I also want to implement essential methods like softmax, sigmoid, and cross-entropy, ensuring that they handle potential floating-point issues. The training method should track total loss and optionally print progress to keep it user-friendly. Lastly, I'll prepare a clear introduction to the code for users. Here’s a minimal fully-connected neural network (feedforward MLP) implemented from scratch using only the Python standard library (no NumPy, no ML libraries). It trains on the XOR problem as an example.#!/usr/bin/env python3 """ Simple neural network (MLP) from scratch using only Python standard library. - Fully-connected layers - Sigmoid activations for hidden layers - Softmax + cross-entropy for output - Trained with stochastic gradient descent (one sample at a time) """ import random import math class NeuralNetwork: def __init__(self, layer_sizes, learning_rate=0.1): """ layer_sizes: list of ints, e.g. [input_dim, hidden1, hidden2, ..., output_dim] learning_rate: SGD step size """ self.layer_sizes = layer_sizes self.num_layers = len(layer_sizes) - 1 # number of weight layers self.learning_rate = learning_rate # Initialize weights and biases self.weights = [] # list of matrices [out_size x in_size] self.biases = [] # list of vectors [out_size] for i in range(self.num_layers): in_size = layer_sizes[i] out_size = layer_sizes[i + 1] # Small random initialization W = [[random.uniform(-0.5, 0.5) * 0.1 for _ in range(in_size)] for _ in range(out_size)] b = [0.0 for _ in range(out_size)] self.weights.append(W) self.biases.append(b) # ---------- Activation functions ---------- @staticmethod def _sigmoid(x): # Numerically stable-ish sigmoid if x >= 0: z = math.exp(-x) return 1.0 / (1.0 + z) else: z = math.exp(x) return z / (1.0 + z) @staticmethod def _sigmoid_derivative(x): s = NeuralNetwork._sigmoid(x) return s * (1.0 - s) @staticmethod def _softmax(z): max_z = max(z) exps = [math.exp(v - max_z) for v in z] sum_exps = sum(exps) return [e / sum_exps for e in exps] @staticmethod def _cross_entropy(pred, target): """ pred: list of probabilities (softmax output) target: one-hot vector or probability distribution """ eps = 1e-15 loss = 0.0 for p, t in zip(pred, target): p_clamped = max(min(p, 1.0 - eps), eps) loss -= t * math.log(p_clamped) return loss # ---------- Forward & backward passes ---------- def forward(self, x): """ Forward pass through the network. x: input vector (list of floats) Returns: activations: [a^0, a^1, ..., a^L] zs: [z^1, z^2, ..., z^L] """ a = x[:] # copy activations = [a] zs = [] for layer in range(self.num_layers): W = self.weights[layer] b = self.biases[layer] # Compute z = W * a + b z = [] for i in range(len(W)): # each output neuron s = 0.0 for j in range(len(W[0])): # each input neuron s += W[i][j] * a[j] z.append(s + b[i]) zs.append(z) # Apply activation if layer == self.num_layers - 1: # Output layer: softmax a = self._softmax(z) else: # Hidden layers: sigmoid a = [self._sigmoid(v) for v in z] activations.append(a) return activations, zs def predict(self, x): """ Predict output probabilities for a single input. """ activations, _ = self.forward(x) return activations[-1] def predict_class(self, x): """ Return index of the highest-probability output neuron. """ probs = self.predict(x) return max(range(len(probs)), key=lambda i: probs[i]) def _train_single(self, x, y): """ Train on a single (x, y) pair using SGD. x: input vector y: target one-hot vector Returns: loss (float) """ # Forward pass activations, zs = self.forward(x) output = activations[-1] # Compute loss loss = self._cross_entropy(output, y) # Backward pass (backpropagation) # Gradient of loss w.r.t. z^L for softmax + cross-entropy: # delta^L = (a^L - y) delta = [output[i] - y[i] for i in range(len(y))] nabla_b = [None] * self.num_layers nabla_w = [None] * self.num_layers # Gradients for last layer nabla_b[-1] = delta a_prev = activations[-2] # a^(L-1) W_grad_last = [] for i in range(len(delta)): # output neurons row = [] for j in range(len(a_prev)): # inputs to this layer row.append(delta[i] * a_prev[j]) W_grad_last.append(row) nabla_w[-1] = W_grad_last # Backpropagate through hidden layers for l in range(self.num_layers - 2, -1, -1): z = zs[l] # z^(l+1) sp = [self._sigmoid_derivative(v) for v in z] # delta_{l} = (W_{l+1}^T * delta_{l+1}) ⊙ sigmoid'(z^{l+1}) W_next = self.weights[l + 1] delta_next = delta delta_new = [] # For each neuron j in layer l+1 for j in range(len(W_next[0])): # number of neurons in layer l+1 s = 0.0 # Sum over neurons i in layer l+2 for i in range(len(W_next)): s += W_next[i][j] * delta_next[i] delta_new.append(s * sp[j]) delta = delta_new nabla_b[l] = delta a_prev = activations[l] # a^l W_grad = [] for i in range(len(delta)): # neurons in layer l+1 row = [] for j in range(len(a_prev)): # neurons in layer l row.append(delta[i] * a_prev[j]) W_grad.append(row) nabla_w[l] = W_grad # Gradient descent update eta = self.learning_rate for l in range(self.num_layers): # Update weights for i in range(len(self.weights[l])): for j in range(len(self.weights[l][0])): self.weights[l][i][j] -= eta * nabla_w[l][i][j] # Update biases for i in range(len(self.biases[l])): self.biases[l][i] -= eta * nabla_b[l][i] return loss def train(self, X, Y, epochs=1000, verbose=True): """ Train the network. X: list of input vectors Y: list of target one-hot vectors """ n = len(X) for epoch in range(epochs): total_loss = 0.0 # Simple SGD over all samples for x, y in zip(X, Y): total_loss += self._train_single(x, y) avg_loss = total_loss / n if verbose and (epoch % max(1, epochs // 10) == 0 or epoch == epochs - 1): print(f"Epoch {epoch+1}/{epochs} - avg loss: {avg_loss:.4f}") # -------------------- Example usage (XOR problem) -------------------- if __name__ == "__main__": # XOR dataset X = [ [0.0, 0.0], [0.0, 1.0], [1.0, 0.0], [1.0, 1.0], ] # Targets as one-hot vectors: class 0 for output 0, class 1 for output 1 Y = [ [1.0, 0.0], # 0 XOR 0 = 0 [0.0, 1.0], # 0 XOR 1 = 1 [0.0, 1.0], # 1 XOR 0 = 1 [1.0, 0.0], # 1 XOR 1 = 0 ] # 2 inputs -> 2 hidden neurons -> 2 outputs nn = NeuralNetwork(layer_sizes=[2, 2, 2], learning_rate=0.5) nn.train(X, Y, epochs=5000, verbose=True) # Test predictions print("\nPredictions after training:") for x, y in zip(X, Y): probs = nn.predict(x) predicted_class = nn.predict_class(x) target_class = y.index(1.0) print(f"Input: {x} | " f"Predicted probs: {[round(p, 3) for p in probs]} | " f"Predicted class: {predicted_class}, Target: {target_class}")You can adapt
layer_sizes, activation functions, and training loop logic to build deeper or different networks.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.