Build the Full project for - Face recognition using CNN
- User
Build the Full project for - Face recognition using CNN
- Assistant
Building a face recognition system using Convolutional Neural Networks (CNN) is a comprehensive project that involves several stages, including data collection, preprocessing, model building, training, evaluation, and deployment. Below is a step-by-step guide to help you create a complete face recognition project using Python and TensorFlow/Keras. This guide includes explanations, code snippets, and best practices to ensure a successful implementation.
Table of Contents
- Project Overview
- Prerequisites
- Data Collection
- Data Preprocessing
- Building the CNN Model
- Training the Model
- Evaluating the Model
- Deploying the Model
- Full Code Implementation
- Conclusion
Project Overview
Face recognition is a biometric method that uses facial features to identify or verify a person’s identity. CNNs have proven highly effective for image-related tasks, including face recognition, due to their ability to automatically and adaptively learn spatial hierarchies of features.
In this project, you'll:
- Collect and preprocess face images.
- Build a CNN model tailored for face recognition.
- Train and evaluate the model.
- Deploy the model for real-world face recognition tasks.
Prerequisites
Before you begin, ensure you have the following installed:
- Python 3.6+
- pip (Python package installer)
Python Libraries
Install the necessary Python libraries using pip:
pip install numpy pandas matplotlib seaborn tensorflow keras opencv-python scikit-learn- numpy & pandas: Data manipulation
- matplotlib & seaborn: Data visualization
- tensorflow & keras: Building and training the CNN
- opencv-python: Image processing
- scikit-learn: Utilities for model evaluation
Data Collection
Choosing a Dataset
For face recognition, you need a dataset containing images of multiple individuals. Some popular datasets include:
- Labeled Faces in the Wild (LFW): Contains over 13,000 images of faces collected from the web.
- VGGFace2: A large-scale dataset with over 3.3 million images of 9,131 subjects.
- Custom Dataset: Collect your own images using a webcam or other cameras.
For this project, we'll use the LFW dataset as it is standard and freely available.
Downloading the LFW Dataset
You can download the LFW dataset directly using
keras:from tensorflow.keras.datasets import lfw lfw_dataset = lfw.load_lfw_people(min_faces_per_person=70, resize=0.4)Alternatively, download it from the official website and extract the images.
Data Preprocessing
Proper preprocessing is crucial for achieving good performance.
Steps Involved:
- Face Alignment and Detection: Ensure faces are properly aligned within the images.
- Resizing: Standardize image sizes.
- Normalization: Scale pixel values.
- Data Augmentation: Enhance the dataset with transformations.
- Splitting the Dataset: Create training, validation, and testing sets.
Implementation
We'll proceed using the LFW dataset loaded via Keras, which is already processed to some extent.
import numpy as np import matplotlib.pyplot as plt from sklearn.model_selection import train_test_split # Load dataset from tensorflow.keras.datasets import lfw lfw_dataset = lfw.load_lfw_people(min_faces_per_person=70, resize=0.4) X = lfw_dataset.images # Shape: (n_samples, height, width) y = lfw_dataset.target # Integer labels class_names = lfw_dataset.target_names n_classes = len(class_names) print(f"Number of samples: {X.shape[0]}") print(f"Number of classes: {n_classes}")Resizing and Normalization
# Reshape data to add channel dimension X = X[..., np.newaxis] # Shape: (n_samples, height, width, 1) # Normalize pixel values to [0, 1] X = X.astype('float32') / 255.0 # One-hot encode labels from tensorflow.keras.utils import to_categorical y = to_categorical(y, num_classes=n_classes)Data Augmentation
Using Keras’
ImageDataGeneratorto perform real-time data augmentation during training.from tensorflow.keras.preprocessing.image import ImageDataGenerator datagen = ImageDataGenerator( rotation_range=20, width_shift_range=0.2, height_shift_range=0.2, horizontal_flip=True, zoom_range=0.2, shear_range=0.2, fill_mode='nearest', validation_split=0.2 # Reserve 20% for validation )
Building the CNN Model
Designing a CNN architecture suitable for face recognition.
Architecture Overview
A typical CNN for face recognition might include:
- Convolutional Layers: To extract features
- Pooling Layers: To reduce spatial dimensions
- Dropout Layers: To prevent overfitting
- Fully Connected Layers: For classification
Alternatively, leveraging pre-trained models (Transfer Learning) like VGGFace, ResNet, or Inception can expedite development and improve performance.
Here, we'll build a custom CNN and also discuss using a pre-trained model.
Custom CNN Model
from tensorflow.keras.models import Sequential from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout, BatchNormalization def create_custom_cnn(input_shape, num_classes): model = Sequential() # 1st Convolutional Block model.add(Conv2D(32, (3, 3), activation='relu', input_shape=input_shape)) model.add(BatchNormalization()) model.add(MaxPooling2D((2, 2))) model.add(Dropout(0.25)) # 2nd Convolutional Block model.add(Conv2D(64, (3, 3), activation='relu')) model.add(BatchNormalization()) model.add(MaxPooling2D((2, 2))) model.add(Dropout(0.25)) # 3rd Convolutional Block model.add(Conv2D(128, (3, 3), activation='relu')) model.add(BatchNormalization()) model.add(MaxPooling2D((2, 2))) model.add(Dropout(0.25)) # Flatten and Fully Connected Layers model.add(Flatten()) model.add(Dense(512, activation='relu')) model.add(BatchNormalization()) model.add(Dropout(0.5)) model.add(Dense(num_classes, activation='softmax')) return model input_shape = X.shape[1:] # (height, width, channels) model = create_custom_cnn(input_shape, n_classes) model.summary()Using Transfer Learning (Optional)
Leveraging a pre-trained model like VGG16 or ResNet50 can improve performance, especially with limited data.
from tensorflow.keras.applications import VGG16 from tensorflow.keras.layers import GlobalAveragePooling2D from tensorflow.keras.models import Model def create_transfer_learning_model(input_shape, num_classes): base_model = VGG16(weights='imagenet', include_top=False, input_shape=input_shape) base_model.trainable = False # Freeze the base model x = base_model.output x = GlobalAveragePooling2D()(x) x = Dense(512, activation='relu')(x) x = Dropout(0.5)(x) predictions = Dense(num_classes, activation='softmax')(x) model = Model(inputs=base_model.input, outputs=predictions) return model model = create_transfer_learning_model(input_shape, n_classes) model.summary()Note: Transfer learning is particularly useful when you have a smaller dataset or want to leverage features learned from a larger dataset.
Training the Model
Compiling the Model
Choose an optimizer, loss function, and evaluation metrics.
model.compile( optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'] )Setting Up Callbacks
Use callbacks like EarlyStopping and ModelCheckpoint to enhance training.
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint early_stop = EarlyStopping(monitor='val_loss', patience=10, restore_best_weights=True) checkpoint = ModelCheckpoint('face_recognition_model.h5', monitor='val_accuracy', save_best_only=True)Training with Data Augmentation
batch_size = 32 epochs = 50 train_generator = datagen.flow(X, y, batch_size=batch_size, subset='training') validation_generator = datagen.flow(X, y, batch_size=batch_size, subset='validation') history = model.fit( train_generator, steps_per_epoch=train_generator.samples // batch_size, validation_data=validation_generator, validation_steps=validation_generator.samples // batch_size, epochs=epochs, callbacks=[early_stop, checkpoint] )Note: If using a pre-trained model, you might consider unfreezing some of the top layers after initial training and retraining with a lower learning rate.
Evaluating the Model
After training, evaluate the model's performance on the validation and test sets.
Plotting Training History
import matplotlib.pyplot as plt # Accuracy Plot plt.figure(figsize=(12, 4)) plt.subplot(1, 2, 1) plt.plot(history.history['accuracy'], label='Train Accuracy') plt.plot(history.history['val_accuracy'], label='Validation Accuracy') plt.title('Accuracy over epochs') plt.xlabel('Epoch') plt.ylabel('Accuracy') plt.legend() # Loss Plot plt.subplot(1, 2, 2) plt.plot(history.history['loss'], label='Train Loss') plt.plot(history.history['val_loss'], label='Validation Loss') plt.title('Loss over epochs') plt.xlabel('Epoch') plt.ylabel('Loss') plt.legend() plt.show()Confusion Matrix
Visualize how well the model is performing across different classes.
from sklearn.metrics import confusion_matrix, classification_report import seaborn as sns # Predict on validation data val_steps = validation_generator.samples // batch_size y_pred = model.predict(validation_generator, steps=val_steps) y_pred_classes = np.argmax(y_pred, axis=1) y_true = np.argmax(y_pred_classes, axis=1) # Adjust if necessary # Confusion Matrix cm = confusion_matrix(validation_generator.classes[:len(y_pred_classes)], y_pred_classes) plt.figure(figsize=(20, 20)) sns.heatmap(cm, annot=True, fmt='d', xticklabels=class_names, yticklabels=class_names) plt.ylabel('Actual') plt.xlabel('Predicted') plt.title('Confusion Matrix') plt.show() # Classification Report print(classification_report(validation_generator.classes[:len(y_pred_classes)], y_pred_classes, target_names=class_names))Testing on Unseen Data
If you have a separate test set, evaluate the model's generalization.
# Assuming you have X_test and y_test test_loss, test_accuracy = model.evaluate(X_test, y_test, verbose=2) print(f"Test Accuracy: {test_accuracy*100:.2f}%")
Deploying the Model
Once trained and evaluated, the model can be deployed to recognize faces in real-time or batch processes.
Saving the Model
model.save('face_recognition_model.h5')Loading the Model
from tensorflow.keras.models import load_model model = load_model('face_recognition_model.h5')Real-Time Face Recognition using OpenCV
Here's a simple example of using OpenCV to capture video from a webcam and perform face recognition.
import cv2 import numpy as np from tensorflow.keras.models import load_model # Load the trained model model = load_model('face_recognition_model.h5') # Initialize webcam cap = cv2.VideoCapture(0) # Load Haar Cascade for face detection face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml') while True: ret, frame = cap.read() if not ret: break # Convert to grayscale for face detection gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) # Detect faces faces = face_cascade.detectMultiScale(gray, 1.3, 5) for (x, y, w, h) in faces: # Extract face ROI face = gray[y:y+h, x:x+w] face = cv2.resize(face, (lfw_dataset.images.shape[1], lfw_dataset.images.shape[2])) face = face.astype('float32') / 255.0 face = np.expand_dims(face, axis=-1) face = np.expand_dims(face, axis=0) # Predict predictions = model.predict(face) class_idx = np.argmax(predictions, axis=1) class_name = class_names[class_idx[0]] # Display bounding box and name cv2.rectangle(frame, (x, y), (x+w, y+h), (255, 0, 0), 2) cv2.putText(frame, class_name, (x, y-10), cv2.FONT_HERSHEY_SIMPLEX, 0.9, (36,255,12), 2) # Display the resulting frame cv2.imshow('Face Recognition', frame) # Break loop on 'q' key press if cv2.waitKey(1) & 0xFF == ord('q'): break # Release the capture and close windows cap.release() cv2.destroyAllWindows()Note:
- Face Detection: Before recognition, faces must be detected. Here, Haar Cascades are used for simplicity, but more accurate detectors like MTCNN or Dlib's HOG/SVM can be utilized.
- Performance: Real-time recognition may require optimization, such as resizing frames or leveraging GPU acceleration.
- Security and Privacy: Ensure ethical use and compliance with privacy laws when deploying face recognition applications.
Full Code Implementation
Putting it all together, here’s a complete Python script encapsulating the entire process.
import numpy as np import matplotlib.pyplot as plt import seaborn as sns import cv2 from sklearn.metrics import confusion_matrix, classification_report from sklearn.model_selection import train_test_split from tensorflow.keras.datasets import lfw from tensorflow.keras.models import Sequential, load_model, Model from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout, BatchNormalization, GlobalAveragePooling2D from tensorflow.keras.utils import to_categorical from tensorflow.keras.preprocessing.image import ImageDataGenerator from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint from tensorflow.keras.applications import VGG16 # 1. Load Dataset lfw_dataset = lfw.load_lfw_people(min_faces_per_person=70, resize=0.4) X = lfw_dataset.images y = lfw_dataset.target class_names = lfw_dataset.target_names n_classes = len(class_names) print(f"Number of samples: {X.shape[0]}") print(f"Number of classes: {n_classes}") # 2. Preprocess Data X = X[..., np.newaxis] # Add channel dimension X = X.astype('float32') / 255.0 # Normalize y = to_categorical(y, num_classes=n_classes) # One-hot encode # 3. Data Augmentation datagen = ImageDataGenerator( rotation_range=20, width_shift_range=0.2, height_shift_range=0.2, horizontal_flip=True, zoom_range=0.2, shear_range=0.2, fill_mode='nearest', validation_split=0.2 ) # 4. Build Model (Custom CNN) def create_custom_cnn(input_shape, num_classes): model = Sequential() # 1st Convolutional Block model.add(Conv2D(32, (3, 3), activation='relu', input_shape=input_shape)) model.add(BatchNormalization()) model.add(MaxPooling2D((2, 2))) model.add(Dropout(0.25)) # 2nd Convolutional Block model.add(Conv2D(64, (3, 3), activation='relu')) model.add(BatchNormalization()) model.add(MaxPooling2D((2, 2))) model.add(Dropout(0.25)) # 3rd Convolutional Block model.add(Conv2D(128, (3, 3), activation='relu')) model.add(BatchNormalization()) model.add(MaxPooling2D((2, 2))) model.add(Dropout(0.25)) # Fully Connected Layers model.add(Flatten()) model.add(Dense(512, activation='relu')) model.add(BatchNormalization()) model.add(Dropout(0.5)) model.add(Dense(num_classes, activation='softmax')) return model input_shape = X.shape[1:] model = create_custom_cnn(input_shape, n_classes) model.summary() # 5. Compile Model model.compile( optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'] ) # 6. Set Up Callbacks early_stop = EarlyStopping(monitor='val_loss', patience=10, restore_best_weights=True) checkpoint = ModelCheckpoint('face_recognition_model.h5', monitor='val_accuracy', save_best_only=True) # 7. Train Model batch_size = 32 epochs = 50 train_generator = datagen.flow(X, y, batch_size=batch_size, subset='training') validation_generator = datagen.flow(X, y, batch_size=batch_size, subset='validation') history = model.fit( train_generator, steps_per_epoch=train_generator.samples // batch_size, validation_data=validation_generator, validation_steps=validation_generator.samples // batch_size, epochs=epochs, callbacks=[early_stop, checkpoint] ) # 8. Evaluate Model # Plot training history plt.figure(figsize=(12, 4)) # Accuracy plt.subplot(1, 2, 1) plt.plot(history.history['accuracy'], label='Train Acc') plt.plot(history.history['val_accuracy'], label='Val Acc') plt.title('Accuracy') plt.xlabel('Epoch') plt.ylabel('Accuracy') plt.legend() # Loss plt.subplot(1, 2, 2) plt.plot(history.history['loss'], label='Train Loss') plt.plot(history.history['val_loss'], label='Val Loss') plt.title('Loss') plt.xlabel('Epoch') plt.ylabel('Loss') plt.legend() plt.show() # Confusion Matrix val_steps = validation_generator.samples // batch_size y_pred = model.predict(validation_generator, steps=val_steps) y_pred_classes = np.argmax(y_pred, axis=1) y_true = validation_generator.classes[:len(y_pred_classes)] cm = confusion_matrix(y_true, y_pred_classes) plt.figure(figsize=(20, 20)) sns.heatmap(cm, annot=True, fmt='d', xticklabels=class_names, yticklabels=class_names) plt.ylabel('Actual') plt.xlabel('Predicted') plt.title('Confusion Matrix') plt.show() print(classification_report(y_true, y_pred_classes, target_names=class_names)) # 9. Save Model model.save('face_recognition_model_final.h5') # 10. (Optional) Real-Time Recognition # Uncomment the following code to perform real-time face recognition """ # Load the saved model model = load_model('face_recognition_model_final.h5') # Initialize webcam cap = cv2.VideoCapture(0) # Load Haar Cascade for face detection face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml') while True: ret, frame = cap.read() if not ret: break # Convert to grayscale gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) # Detect faces faces = face_cascade.detectMultiScale(gray, 1.3, 5) for (x, y, w, h) in faces: # Extract ROI face = gray[y:y+h, x:x+w] face = cv2.resize(face, (lfw_dataset.images.shape[1], lfw_dataset.images.shape[2])) face = face.astype('float32') / 255.0 face = np.expand_dims(face, axis=-1) face = np.expand_dims(face, axis=0) # Predict predictions = model.predict(face) class_idx = np.argmax(predictions, axis=1) class_name = class_names[class_idx[0]] # Annotate cv2.rectangle(frame, (x, y), (x+w, y+h), (255,0,0), 2) cv2.putText(frame, class_name, (x, y-10), cv2.FONT_HERSHEY_SIMPLEX, 0.9, (36,255,12), 2) # Display cv2.imshow('Face Recognition', frame) # Exit on 'q' if cv2.waitKey(1) & 0xFF == ord('q'): break cap.release() cv2.destroyAllWindows() """Explanation:
-
Loading and Preprocessing: The script loads the LFW dataset, normalizes it, and prepares labels.
-
Data Augmentation:
ImageDataGeneratoris set up with augmentation parameters and splits the data into training and validation sets. -
Model Building: A custom CNN is defined with multiple convolutional blocks, batch normalization, dropout, and dense layers.
-
Training: The model is compiled and trained using the augmented data. Early stopping and model checkpointing are used to prevent overfitting and save the best model.
-
Evaluation: Training history is plotted, and a confusion matrix along with a classification report is generated to assess performance.
-
Deployment: Optionally, the trained model can be used for real-time face recognition with OpenCV.
Conclusion
This project provides a comprehensive approach to building a face recognition system using CNNs. By following the steps outlined above, you can develop a robust model capable of accurately recognizing faces. Depending on your requirements and dataset size, you may choose between building a custom CNN or leveraging transfer learning with pre-trained models for enhanced performance.
Next Steps:
- Improve Accuracy: Experiment with deeper architectures, fine-tuning, or advanced data augmentation techniques.
- Handle Variations: Incorporate techniques to handle different lighting conditions, angles, and occlusions.
- Optimize Performance: Utilize GPU acceleration, model pruning, or quantization for faster inference.
- Expand Functionality: Implement features like facial landmark detection, emotion recognition, or age estimation.
Ethical Considerations:
Ensure that your face recognition system respects privacy laws and ethical guidelines. Obtain necessary consents and use the technology responsibly to prevent misuse.
Feel free to ask if you need further details or assistance with specific parts of the project!