Classification of Iris Species Flowers using decision-tree algorithm in jupyter…
- User
Classification of Iris Species Flowers using decision-tree algorithm in jupyter notebook, I already have a dataset
- Assistant
Certainly! Below is a comprehensive guide to classifying Iris species using the Decision Tree algorithm in a Jupyter Notebook. This guide assumes you have a dataset similar to the classic Iris dataset, which typically includes features like sepal length, sepal width, petal length, petal width, and the species label.
Table of Contents
- Setup and Import Libraries
- Load and Explore the Dataset
- Data Preprocessing
- Train-Test Split
- Training the Decision Tree Classifier
- Model Evaluation
- Visualizing the Decision Tree
- Conclusion
Setup and Import Libraries
First, ensure you have the necessary libraries installed. You can install any missing libraries using
pip. For example:pip install pandas scikit-learn matplotlib seaborn graphvizNow, import the required libraries.
# Import necessary libraries import pandas as pd import numpy as np import matplotlib.pyplot as plt import seaborn as sns # For machine learning from sklearn.model_selection import train_test_split from sklearn.tree import DecisionTreeClassifier, plot_tree from sklearn.metrics import classification_report, confusion_matrix, accuracy_score # For visualization (optional) import graphviz from sklearn import tree %matplotlib inlineLoad and Explore the Dataset
Assuming you have your dataset in a CSV file, let's load it using pandas. If your dataset is in a different format (Excel, JSON, etc.), pandas can handle that as well.
# Load the dataset # Replace 'iris.csv' with your dataset's filename or path df = pd.read_csv('iris.csv') # Display the first few rows df.head()Example Output:
SepalLength SepalWidth PetalLength PetalWidth Species 5.1 3.5 1.4 0.2 setosa 4.9 3.0 1.4 0.2 setosa 4.7 3.2 1.3 0.2 setosa ... ... ... ... ... Exploratory Data Analysis (EDA)
It's essential to understand your data before building models.
# Check the shape of the dataset print(f"Dataset contains {df.shape[0]} rows and {df.shape[1]} columns.") # Check for missing values print("\nMissing values:\n", df.isnull().sum()) # Statistical summary print("\nStatistical Summary:\n", df.describe()) # Visualize the distribution of features sns.pairplot(df, hue='Species') plt.show() # Correlation heatmap plt.figure(figsize=(8,6)) sns.heatmap(df.corr(), annot=True, cmap='coolwarm') plt.title('Feature Correlation') plt.show()Data Preprocessing
Ensure that the data is clean and ready for modeling.
# Check data types print(df.dtypes) # If the species column is categorical, encode it # Most Decision Tree implementations can handle string labels, but encoding is good practice from sklearn.preprocessing import LabelEncoder le = LabelEncoder() df['Species'] = le.fit_transform(df['Species']) # Verify encoding print(df.head())Example Output:
SepalLength SepalWidth PetalLength PetalWidth Species 5.1 3.5 1.4 0.2 0 4.9 3.0 1.4 0.2 0 4.7 3.2 1.3 0.2 0 ... ... ... ... ... Train-Test Split
Split the dataset into training and testing sets to evaluate the model's performance on unseen data.
# Define features and target variable X = df.drop('Species', axis=1) y = df['Species'] # Split into training and testing sets (e.g., 80% train, 20% test) X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=42, stratify=y) print(f"Training set has {X_train.shape[0]} samples.") print(f"Testing set has {X_test.shape[0]} samples.")Training the Decision Tree Classifier
Initialize and train the Decision Tree model. You can experiment with different hyperparameters to improve performance.
# Initialize the Decision Tree Classifier clf = DecisionTreeClassifier( criterion='gini', # 'gini' for Gini impurity, 'entropy' for Information Gain splitter='best', # 'best' or 'random' max_depth=None, # The maximum depth of the tree min_samples_split=2, # The minimum number of samples required to split an internal node min_samples_leaf=1, # The minimum number of samples required to be at a leaf node random_state=42 ) # Train the model clf.fit(X_train, y_train) print("Decision Tree Classifier trained successfully.")Model Evaluation
Evaluate the trained model using the testing set.
# Predict on the test set y_pred = clf.predict(X_test) # Calculate accuracy accuracy = accuracy_score(y_test, y_pred) print(f"Accuracy on test set: {accuracy:.2f}") # Classification report print("\nClassification Report:\n", classification_report(y_test, y_pred, target_names=le.classes_)) # Confusion matrix cm = confusion_matrix(y_test, y_pred) plt.figure(figsize=(6,4)) sns.heatmap(cm, annot=True, fmt='d', cmap='Blues', xticklabels=le.classes_, yticklabels=le.classes_) plt.ylabel('Actual') plt.xlabel('Predicted') plt.title('Confusion Matrix') plt.show()Example Output:
Accuracy on test set: 1.00 Classification Report: precision recall f1-score support setosa 1.00 1.00 1.00 10 versicolor 1.00 1.00 1.00 10 virginica 1.00 1.00 1.00 10 accuracy 1.00 30 macro avg 1.00 1.00 1.00 30 weighted avg 1.00 1.00 1.00 30Visualizing the Decision Tree
Visualizing the tree helps in understanding the decision-making process of the model.
Method 1: Using Matplotlib's
plot_treeplt.figure(figsize=(20,10)) plot_tree(clf, feature_names=X.columns, class_names=le.classes_, filled=True, rounded=True, fontsize=12) plt.title("Decision Tree") plt.show()Method 2: Using
graphvizfor a More Detailed VisualizationFirst, ensure you have Graphviz installed on your system. You can download it from Graphviz Download.
Then, use the following code:
# Export the tree in DOT format dot_data = tree.export_graphviz( clf, out_file=None, feature_names=X.columns, class_names=le.classes_, filled=True, rounded=True, special_characters=True ) # Create a Graphviz object graph = graphviz.Source(dot_data) # Display the graph graph.render("iris_decision_tree") # This will save the tree as a PNG file graphNote: If you're running this in a Jupyter Notebook, the
graphvariable will display the tree inline. Thegraph.renderline saves the tree to a file namediris_decision_tree.png(you can change the filename as needed).Hyperparameter Tuning (Optional)
To improve your model, consider tuning hyperparameters using techniques like Grid Search or Random Search with cross-validation.
from sklearn.model_selection import GridSearchCV # Define the parameter grid param_grid = { 'criterion': ['gini', 'entropy'], 'max_depth': [None, 2, 4, 6, 8, 10], 'min_samples_split': [2, 5, 10], 'min_samples_leaf': [1, 2, 4] } # Initialize GridSearchCV grid_search = GridSearchCV( estimator=clf, param_grid=param_grid, cv=5, n_jobs=-1, scoring='accuracy', verbose=1 ) # Fit GridSearchCV grid_search.fit(X_train, y_train) # Best parameters print(f"Best Parameters: {grid_search.best_params_}") # Best estimator best_clf = grid_search.best_estimator_ # Evaluate the best estimator y_pred_best = best_clf.predict(X_test) accuracy_best = accuracy_score(y_test, y_pred_best) print(f"Accuracy with best parameters: {accuracy_best:.2f}")Conclusion
You've successfully built a Decision Tree classifier to classify Iris species. Here's a summary of what we've covered:
- Data Loading and Exploration: Understanding the dataset's structure and relationships.
- Data Preprocessing: Cleaning and encoding data as needed.
- Model Training: Building and fitting the Decision Tree model.
- Model Evaluation: Assessing the model's performance using accuracy, classification report, and confusion matrix.
- Visualization: Visualizing the Decision Tree to interpret its decisions.
- Hyperparameter Tuning: Optimizing the model for better performance.
Feel free to experiment further by:
- Trying different algorithms (e.g., Random Forest, SVM).
- Performing feature engineering or selection.
- Implementing cross-validation for more robust evaluations.
Full Example Code
For convenience, here's the complete code encapsulated together. You can run it step-by-step in your Jupyter Notebook.
# 1. Import Libraries import pandas as pd import numpy as np import matplotlib.pyplot as plt import seaborn as sns from sklearn.model_selection import train_test_split from sklearn.tree import DecisionTreeClassifier, plot_tree from sklearn.metrics import classification_report, confusion_matrix, accuracy_score from sklearn.preprocessing import LabelEncoder import graphviz from sklearn import tree %matplotlib inline # 2. Load Dataset df = pd.read_csv('iris.csv') # Replace with your filename print(f"Dataset shape: {df.shape}") print("\nFirst 5 rows:") display(df.head()) # 3. Exploratory Data Analysis print("\nMissing values:\n", df.isnull().sum()) print("\nStatistical Summary:\n", df.describe()) sns.pairplot(df, hue='Species') plt.show() plt.figure(figsize=(8,6)) sns.heatmap(df.corr(), annot=True, cmap='coolwarm') plt.title('Feature Correlation') plt.show() # 4. Data Preprocessing print("\nData Types:\n", df.dtypes) le = LabelEncoder() df['Species'] = le.fit_transform(df['Species']) print("\nEncoded Species:\n", df.head()) # 5. Train-Test Split X = df.drop('Species', axis=1) y = df['Species'] X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=42, stratify=y) print(f"Training samples: {X_train.shape[0]}") print(f"Testing samples: {X_test.shape[0]}") # 6. Training the Model clf = DecisionTreeClassifier(random_state=42) clf.fit(X_train, y_train) print("Model trained.") # 7. Model Evaluation y_pred = clf.predict(X_test) accuracy = accuracy_score(y_test, y_pred) print(f"Accuracy: {accuracy:.2f}") print("\nClassification Report:\n", classification_report(y_test, y_pred, target_names=le.classes_)) cm = confusion_matrix(y_test, y_pred) plt.figure(figsize=(6,4)) sns.heatmap(cm, annot=True, fmt='d', cmap='Blues', xticklabels=le.classes_, yticklabels=le.classes_) plt.ylabel('Actual') plt.xlabel('Predicted') plt.title('Confusion Matrix') plt.show() # 8. Visualizing the Tree plt.figure(figsize=(20,10)) plot_tree(clf, feature_names=X.columns, class_names=le.classes_, filled=True, rounded=True, fontsize=12) plt.title("Decision Tree") plt.show() # Optional: Using graphviz dot_data = tree.export_graphviz( clf, out_file=None, feature_names=X.columns, class_names=le.classes_, filled=True, rounded=True, special_characters=True ) graph = graphviz.Source(dot_data) graph.render("iris_decision_tree") graph # 9. Hyperparameter Tuning (Optional) from sklearn.model_selection import GridSearchCV param_grid = { 'criterion': ['gini', 'entropy'], 'max_depth': [None, 2, 4, 6, 8, 10], 'min_samples_split': [2, 5, 10], 'min_samples_leaf': [1, 2, 4] } grid_search = GridSearchCV( estimator=clf, param_grid=param_grid, cv=5, n_jobs=-1, scoring='accuracy', verbose=1 ) grid_search.fit(X_train, y_train) print(f"Best Parameters: {grid_search.best_params_}") best_clf = grid_search.best_estimator_ y_pred_best = best_clf.predict(X_test) accuracy_best = accuracy_score(y_test, y_pred_best) print(f"Accuracy with best parameters: {accuracy_best:.2f}")
Notes:
-
Dataset Path: Ensure that
'iris.csv'is replaced with the correct path to your dataset. -
Graphviz Installation: For the
graphvizvisualization to work, you need to have Graphviz installed on your machine. After installation, ensure that the Graphviz binaries are added to your system's PATH. -
Label Encoding: The
LabelEncoderconverts categorical labels into numerical values. This step is crucial if your classifier doesn't handle string labels implicitly. -
Model Performance: The Decision Tree often achieves perfect accuracy on the Iris dataset because it's simple and the dataset is small and well-separated. However, Decision Trees can overfit, so always validate performance on unseen data.
-
Hyperparameter Tuning: Tuning hyperparameters like
max_depth,min_samples_split, andmin_samples_leafcan help prevent overfitting and improve generalization.
Feel free to customize and expand upon this guide based on your specific dataset and requirements!