o1-preview-2024-09-12· OpenAI
WildChat-4.8M (AI2)
Реализуй алгоритм стохастического градиентного спуска на python, где нужно предсказать…
- User
Реализуй алгоритм стохастического градиентного спуска на python, где нужно предсказать значение последнего столбца в таблице с данными. В качестве меры ошибки используй сигмоидную функцию потерь
- Assistant
import numpy as np def sigmoid(z): """Compute the sigmoid function.""" return 1 / (1 + np.exp(-z)) def sigmoid_loss(y_true, y_pred): """Compute the sigmoid (logistic) loss.""" return - (y_true * np.log(y_pred + 1e-15) + (1 - y_true) * np.log(1 - y_pred + 1e-15)) def stochastic_gradient_descent(X, y, learning_rate=0.01, epochs=100): """ Implement stochastic gradient descent for logistic regression. Parameters: - X: Feature matrix. - y: Target vector. - learning_rate: Learning rate for weight updates. - epochs: Number of passes over the dataset. Returns: - weights: Learned weights for the logistic regression model. """ m, n = X.shape # Number of samples and features weights = np.zeros(n) # Initialize weights for epoch in range(epochs): # Shuffle the data at the beginning of each epoch indices = np.arange(m) np.random.shuffle(indices) X = X[indices] y = y[indices] for i in range(m): xi = X[i] yi = y[i] z = np.dot(xi, weights) prediction = sigmoid(z) error = prediction - yi gradient = error * xi # Update weights weights -= learning_rate * gradient return weights def predict(X, weights): """Make predictions using the learned weights.""" z = np.dot(X, weights) predictions = sigmoid(z) return predictions # Example usage: if __name__ == "__main__": # Suppose 'data' is your dataset loaded as a NumPy array # For demonstration, we'll create a synthetic dataset np.random.seed(0) # For reproducibility num_samples = 200 num_features = 2 # Generate synthetic features X = np.random.randn(num_samples, num_features) # Generate synthetic labels (last column), using a linear combination of features plus noise true_weights = np.array([2.0, -3.5]) logits = np.dot(X, true_weights) + np.random.randn(num_samples) * 0.5 probabilities = sigmoid(logits) y = (probabilities > 0.5).astype(int) # Binary labels # Add a bias term (intercept) to feature matrix X_with_bias = np.c_[np.ones(num_samples), X] # Train the logistic regression model using SGD weights = stochastic_gradient_descent(X_with_bias, y, learning_rate=0.1, epochs=10) # Make predictions on the training set predictions = predict(X_with_bias, weights) predicted_labels = (predictions >= 0.5).astype(int) # Calculate accuracy accuracy = np.mean(predicted_labels == y) print("Trained weights:", weights) print(f"Accuracy on training set: {accuracy * 100:.2f}%")Explanation:
- Sigmoid Function: Computes the probability that each input belongs to the positive class.
- Sigmoid Loss Function: Also known as logistic loss, measures the discrepancy between the predicted probabilities and the actual binary labels.
- Stochastic Gradient Descent (SGD): Updates the model weights incrementally for each training example. It helps in handling large datasets efficiently.
- Shuffling Data: Before each epoch, the dataset is shuffled to ensure the model doesn't learn patterns based on the order of the data.
- Weight Update Rule: For each example, we compute the gradient of the loss with respect to the weights and update the weights in the opposite direction of the gradient.
- Predict Function: Uses the learned weights to compute the predicted probabilities for each example.
Notes:
- Make sure your data is properly preprocessed before training (e.g., feature scaling if necessary).
- The learning rate and the number of epochs are hyperparameters that might need tuning based on your specific dataset.
- This implementation assumes that the data is suitable for binary classification.
How to Use with Your Data:
-
Prepare Your Dataset:
- Load your data into a NumPy array or pandas DataFrame.
- Separate the features (all columns except the last) and the target variable (the last column).
- Ensure that the target variable contains binary labels (0 or 1). If not, you might need to preprocess it.
-
Add Bias Term:
- Concatenate a column of ones to your feature matrix to account for the intercept term in the model.
X = data[:, :-1] # Features y = data[:, -1] # Target variable (last column) X_with_bias = np.c_[np.ones(len(X)), X] -
Train the Model:
weights = stochastic_gradient_descent(X_with_bias, y, learning_rate=0.01, epochs=100) -
Make Predictions:
predictions = predict(X_with_bias, weights) predicted_labels = (predictions >= 0.5).astype(int) -
Evaluate the Model (Optional):
- Calculate metrics like accuracy, precision, recall, etc., to assess the performance of your model.
accuracy = np.mean(predicted_labels == y) print(f"Accuracy: {accuracy * 100:.2f}%") -
Adjust Hyperparameters (If Necessary):
- If the performance is not satisfactory, consider adjusting the learning rate, number of epochs, or performing data preprocessing.
Remember:
- The sigmoid loss function is appropriate for binary classification problems.
- Ensure there are no missing values or categorical variables that need to be encoded in your dataset.
- For multiclass classification, you would need to modify the loss function and possibly consider using softmax regression.
Sample Output:
Trained weights: [ 0.12345678 1.23456789 -2.34567890] Accuracy on training set: 85.00%Note: The sample output values are illustrative. Your actual output will depend on your dataset and hyperparameters.