write a code with logistic regression
- User
write a code with logistic regression
- Assistant
Certainly! Below is a complete example of implementing Logistic Regression using Python's
scikit-learnlibrary. This example uses the popular Iris dataset, which is a classic dataset in machine learning. Although the Iris dataset has three classes, for simplicity, we'll modify it to a binary classification problem.Table of Contents
- Prerequisites
- Loading and Preparing the Data
- Splitting the Data
- Training the Logistic Regression Model
- Making Predictions
- Evaluating the Model
- Complete Code
Prerequisites
Before you begin, ensure you have the necessary libraries installed. You can install them using
pipif they aren't already installed.pip install numpy pandas scikit-learn matplotlib seabornLoading and Preparing the Data
We'll use the Iris dataset, which includes three classes of iris plants. For binary classification, we'll select only two classes.
import numpy as np import pandas as pd from sklearn.datasets import load_iris # Load the Iris dataset iris = load_iris() X = iris.data # Features: sepal length, sepal width, petal length, petal width y = iris.target # Target: Species (0, 1, 2) # For binary classification, select only classes 0 and 1 binary_class_indices = y < 2 X = X[binary_class_indices] y = y[binary_class_indices] print("Features shape:", X.shape) print("Target shape:", y.shape)Output:
Features shape: (100, 4) Target shape: (100,)Splitting the Data
Split the dataset into training and testing sets to evaluate the model's performance on unseen data.
from sklearn.model_selection import train_test_split # Split the data: 80% training, 20% testing X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=42, stratify=y ) print("Training samples:", X_train.shape[0]) print("Testing samples:", X_test.shape[0])Output:
Training samples: 80 Testing samples: 20Training the Logistic Regression Model
Create and train the Logistic Regression model using the training data.
from sklearn.linear_model import LogisticRegression # Initialize the Logistic Regression model model = LogisticRegression(solver='liblinear') # 'liblinear' is good for small datasets # Train the model model.fit(X_train, y_train)Making Predictions
Use the trained model to make predictions on the test set.
# Predict probabilities y_prob = model.predict_proba(X_test)[:, 1] # Predict classes y_pred = model.predict(X_test) print("Predicted probabilities:", y_prob) print("Predicted classes:", y_pred)Sample Output:
Predicted probabilities: [0.02991865 0.97309035 0.04488833 0.98345678 0.02123456 ...] Predicted classes: [0 1 0 1 0 ...]Evaluating the Model
Evaluate how well the model performs using various metrics.
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report # Calculate accuracy accuracy = accuracy_score(y_test, y_pred) print(f"Accuracy: {accuracy:.2f}") # Confusion Matrix cm = confusion_matrix(y_test, y_pred) print("Confusion Matrix:") print(cm) # Classification Report cr = classification_report(y_test, y_pred, target_names=iris.target_names[:2]) print("Classification Report:") print(cr)Sample Output:
Accuracy: 1.00 Confusion Matrix: [[10 0] [ 0 10]] Classification Report: precision recall f1-score support setosa 1.00 1.00 1.00 10 versicolor 1.00 1.00 1.00 10 accuracy 1.00 20 macro avg 1.00 1.00 1.00 20 weighted avg 1.00 1.00 1.00 20Visualizing the Decision Boundary (Optional)
For datasets with two features, it's common to visualize the decision boundary. Since the Iris dataset has four features, we'll reduce it to two using Principal Component Analysis (PCA) for visualization purposes.
import matplotlib.pyplot as plt from sklearn.decomposition import PCA # Reduce to 2 dimensions for visualization pca = PCA(n_components=2) X_pca = pca.fit_transform(X) # Split the PCA-transformed data X_train_pca, X_test_pca, _, _ = train_test_split( X_pca, y, test_size=0.2, random_state=42, stratify=y ) # Retrain the model on PCA-transformed data model_pca = LogisticRegression(solver='liblinear') model_pca.fit(X_train_pca, y_train) # Plotting plt.figure(figsize=(8,6)) colors = ['blue', 'red'] markers = ['o', 's'] for idx, class_label in enumerate(np.unique(y)): plt.scatter( X_pca[y == class_label, 0], X_pca[y == class_label, 1], c=colors[idx], marker=markers[idx], label=iris.target_names[class_label] ) # Create a mesh to plot decision boundary x_min, x_max = X_pca[:, 0].min() - 1, X_pca[:, 0].max() + 1 y_min, y_max = X_pca[:, 1].min() - 1, X_pca[:, 1].max() + 1 xx, yy = np.meshgrid( np.arange(x_min, x_max, 0.02), np.arange(y_min, y_max, 0.02) ) Z = model_pca.predict(np.c_[xx.ravel(), yy.ravel()]) Z = Z.reshape(xx.shape) # Plot contour plt.contourf(xx, yy, Z, alpha=0.2, colors=colors) plt.xlabel('Principal Component 1') plt.ylabel('Principal Component 2') plt.legend() plt.title('Logistic Regression Decision Boundary') plt.show()Visualization:
[image: Logistic Regression Decision Boundary] open (Replace with actual image when running the code)
Complete Code
Here is the complete code combining all the steps above. You can run this script in your Python environment.
import numpy as np import pandas as pd from sklearn.datasets import load_iris from sklearn.model_selection import train_test_split from sklearn.linear_model import LogisticRegression from sklearn.metrics import accuracy_score, confusion_matrix, classification_report import matplotlib.pyplot as plt from sklearn.decomposition import PCA # 1. Load the Iris dataset iris = load_iris() X = iris.data y = iris.target # 2. Convert to binary classification: setosa vs versicolor binary_class_indices = y < 2 X = X[binary_class_indices] y = y[binary_class_indices] print("Features shape:", X.shape) print("Target shape:", y.shape) # 3. Split the data X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=42, stratify=y ) print("Training samples:", X_train.shape[0]) print("Testing samples:", X_test.shape[0]) # 4. Initialize and train the Logistic Regression model model = LogisticRegression(solver='liblinear') model.fit(X_train, y_train) # 5. Make predictions y_prob = model.predict_proba(X_test)[:, 1] y_pred = model.predict(X_test) print("Predicted probabilities:", y_prob) print("Predicted classes:", y_pred) # 6. Evaluate the model accuracy = accuracy_score(y_test, y_pred) print(f"Accuracy: {accuracy:.2f}") cm = confusion_matrix(y_test, y_pred) print("Confusion Matrix:") print(cm) cr = classification_report(y_test, y_pred, target_names=iris.target_names[:2]) print("Classification Report:") print(cr) # 7. Optional: Visualize the decision boundary using PCA # Reduce to 2 dimensions pca = PCA(n_components=2) X_pca = pca.fit_transform(X) # Split the PCA-transformed data X_train_pca, X_test_pca, _, _ = train_test_split( X_pca, y, test_size=0.2, random_state=42, stratify=y ) # Retrain the model on PCA data model_pca = LogisticRegression(solver='liblinear') model_pca.fit(X_train_pca, y_train) # Plotting plt.figure(figsize=(8,6)) colors = ['blue', 'red'] markers = ['o', 's'] for idx, class_label in enumerate(np.unique(y)): plt.scatter( X_pca[y == class_label, 0], X_pca[y == class_label, 1], c=colors[idx], marker=markers[idx], label=iris.target_names[class_label] ) # Create a mesh to plot decision boundary x_min, x_max = X_pca[:, 0].min() - 1, X_pca[:, 0].max() + 1 y_min, y_max = X_pca[:, 1].min() - 1, X_pca[:, 1].max() + 1 xx, yy = np.meshgrid( np.arange(x_min, x_max, 0.02), np.arange(y_min, y_max, 0.02) ) Z = model_pca.predict(np.c_[xx.ravel(), yy.ravel()]) Z = Z.reshape(xx.shape) # Plot contour plt.contourf(xx, yy, Z, alpha=0.2, colors=colors) plt.xlabel('Principal Component 1') plt.ylabel('Principal Component 2') plt.legend() plt.title('Logistic Regression Decision Boundary') plt.show()Explanation of the Code
-
Loading the Data:
- We load the Iris dataset and select only two classes (
setosaandversicolor) to create a binary classification problem.
- We load the Iris dataset and select only two classes (
-
Data Splitting:
- The dataset is split into training and testing sets. Stratification ensures that both sets have an equal proportion of each class.
-
Model Training:
- A
LogisticRegressionmodel is initialized and trained using the training data.
- A
-
Making Predictions:
- The model predicts both the probability of the positive class and the class labels for the test set.
-
Model Evaluation:
- We evaluate the model's performance using accuracy, confusion matrix, and a detailed classification report which includes precision, recall, and F1-score.
-
Visualization (Optional):
- Using PCA, we reduce the feature space to two dimensions to visualize the decision boundary of the logistic regression model.
Customization
-
Different Datasets: You can replace the Iris dataset with any other dataset suitable for binary classification. Ensure that the target variable is binary.
-
Hyperparameter Tuning: Explore different solvers (
'newton-cg','lbfgs','sag', etc.) and regularization strengths (C) to optimize model performance. -
Feature Engineering: Consider standardizing the features or performing feature selection to improve model accuracy.
Conclusion
This example demonstrates how to implement Logistic Regression for a binary classification task using Python's
scikit-learnlibrary. Logistic Regression is a fundamental classification algorithm that's intuitive and effective for many real-world problems. Feel free to modify the code to suit your specific needs or to experiment with different datasets and model parameters.