Skip to content
TheChatArchiveThe Chat Archive
o1-preview-2024-09-12· OpenAIAi2 WildChat-4.8M (AI2)

Реализуй алгоритм стохастического градиентного спуска на python, где нужно предсказать…

  1. User

    Реализуй алгоритм стохастического градиентного спуска на python, где нужно предсказать значение последнего столбца в таблице с данными. В качестве меры ошибки используй сигмоидную функцию потерь

  2. Assistant
    import numpy as np
    
    def sigmoid(z):
        """Compute the sigmoid function."""
        return 1 / (1 + np.exp(-z))
    
    def sigmoid_loss(y_true, y_pred):
        """Compute the sigmoid (logistic) loss."""
        return - (y_true * np.log(y_pred + 1e-15) + (1 - y_true) * np.log(1 - y_pred + 1e-15))
    
    def stochastic_gradient_descent(X, y, learning_rate=0.01, epochs=100):
        """
        Implement stochastic gradient descent for logistic regression.
    
        Parameters:
        - X: Feature matrix.
        - y: Target vector.
        - learning_rate: Learning rate for weight updates.
        - epochs: Number of passes over the dataset.
    
        Returns:
        - weights: Learned weights for the logistic regression model.
        """
        m, n = X.shape  # Number of samples and features
        weights = np.zeros(n)  # Initialize weights
        for epoch in range(epochs):
            # Shuffle the data at the beginning of each epoch
            indices = np.arange(m)
            np.random.shuffle(indices)
            X = X[indices]
            y = y[indices]
            for i in range(m):
                xi = X[i]
                yi = y[i]
                z = np.dot(xi, weights)
                prediction = sigmoid(z)
                error = prediction - yi
                gradient = error * xi
                # Update weights
                weights -= learning_rate * gradient
        return weights
    
    def predict(X, weights):
        """Make predictions using the learned weights."""
        z = np.dot(X, weights)
        predictions = sigmoid(z)
        return predictions
    
    # Example usage:
    if __name__ == "__main__":
        # Suppose 'data' is your dataset loaded as a NumPy array
        # For demonstration, we'll create a synthetic dataset
        np.random.seed(0)  # For reproducibility
        num_samples = 200
        num_features = 2
    
        # Generate synthetic features
        X = np.random.randn(num_samples, num_features)
    
        # Generate synthetic labels (last column), using a linear combination of features plus noise
        true_weights = np.array([2.0, -3.5])
        logits = np.dot(X, true_weights) + np.random.randn(num_samples) * 0.5
        probabilities = sigmoid(logits)
        y = (probabilities > 0.5).astype(int)  # Binary labels
    
        # Add a bias term (intercept) to feature matrix
        X_with_bias = np.c_[np.ones(num_samples), X]
    
        # Train the logistic regression model using SGD
        weights = stochastic_gradient_descent(X_with_bias, y, learning_rate=0.1, epochs=10)
    
        # Make predictions on the training set
        predictions = predict(X_with_bias, weights)
        predicted_labels = (predictions >= 0.5).astype(int)
    
        # Calculate accuracy
        accuracy = np.mean(predicted_labels == y)
        print("Trained weights:", weights)
        print(f"Accuracy on training set: {accuracy * 100:.2f}%")
    

    Explanation:

    • Sigmoid Function: Computes the probability that each input belongs to the positive class.
    • Sigmoid Loss Function: Also known as logistic loss, measures the discrepancy between the predicted probabilities and the actual binary labels.
    • Stochastic Gradient Descent (SGD): Updates the model weights incrementally for each training example. It helps in handling large datasets efficiently.
      • Shuffling Data: Before each epoch, the dataset is shuffled to ensure the model doesn't learn patterns based on the order of the data.
      • Weight Update Rule: For each example, we compute the gradient of the loss with respect to the weights and update the weights in the opposite direction of the gradient.
    • Predict Function: Uses the learned weights to compute the predicted probabilities for each example.

    Notes:

    • Make sure your data is properly preprocessed before training (e.g., feature scaling if necessary).
    • The learning rate and the number of epochs are hyperparameters that might need tuning based on your specific dataset.
    • This implementation assumes that the data is suitable for binary classification.

    How to Use with Your Data:

    1. Prepare Your Dataset:

      • Load your data into a NumPy array or pandas DataFrame.
      • Separate the features (all columns except the last) and the target variable (the last column).
      • Ensure that the target variable contains binary labels (0 or 1). If not, you might need to preprocess it.
    2. Add Bias Term:

      • Concatenate a column of ones to your feature matrix to account for the intercept term in the model.
      X = data[:, :-1]  # Features
      y = data[:, -1]   # Target variable (last column)
      X_with_bias = np.c_[np.ones(len(X)), X]
      
    3. Train the Model:

      weights = stochastic_gradient_descent(X_with_bias, y, learning_rate=0.01, epochs=100)
      
    4. Make Predictions:

      predictions = predict(X_with_bias, weights)
      predicted_labels = (predictions >= 0.5).astype(int)
      
    5. Evaluate the Model (Optional):

      • Calculate metrics like accuracy, precision, recall, etc., to assess the performance of your model.
      accuracy = np.mean(predicted_labels == y)
      print(f"Accuracy: {accuracy * 100:.2f}%")
      
    6. Adjust Hyperparameters (If Necessary):

      • If the performance is not satisfactory, consider adjusting the learning rate, number of epochs, or performing data preprocessing.

    Remember:

    • The sigmoid loss function is appropriate for binary classification problems.
    • Ensure there are no missing values or categorical variables that need to be encoded in your dataset.
    • For multiclass classification, you would need to modify the loss function and possibly consider using softmax regression.

    Sample Output:

    Trained weights: [ 0.12345678  1.23456789 -2.34567890]
    Accuracy on training set: 85.00%
    

    Note: The sample output values are illustrative. Your actual output will depend on your dataset and hyperparameters.