Skip to content
TheChatArchiveThe Chat Archive
o1-mini-2024-09-12· OpenAIAi2 WildChat-4.8M (AI2)

Build the Full project for - Face recognition using CNN

  1. User

    Build the Full project for - Face recognition using CNN

  2. Assistant

    Building a face recognition system using Convolutional Neural Networks (CNN) is a comprehensive project that involves several stages, including data collection, preprocessing, model building, training, evaluation, and deployment. Below is a step-by-step guide to help you create a complete face recognition project using Python and TensorFlow/Keras. This guide includes explanations, code snippets, and best practices to ensure a successful implementation.


    Table of Contents

    1. Project Overview
    2. Prerequisites
    3. Data Collection
    4. Data Preprocessing
    5. Building the CNN Model
    6. Training the Model
    7. Evaluating the Model
    8. Deploying the Model
    9. Full Code Implementation
    10. Conclusion

    Project Overview

    Face recognition is a biometric method that uses facial features to identify or verify a person’s identity. CNNs have proven highly effective for image-related tasks, including face recognition, due to their ability to automatically and adaptively learn spatial hierarchies of features.

    In this project, you'll:

    • Collect and preprocess face images.
    • Build a CNN model tailored for face recognition.
    • Train and evaluate the model.
    • Deploy the model for real-world face recognition tasks.

    Prerequisites

    Before you begin, ensure you have the following installed:

    • Python 3.6+
    • pip (Python package installer)

    Python Libraries

    Install the necessary Python libraries using pip:

    pip install numpy pandas matplotlib seaborn tensorflow keras opencv-python scikit-learn
    
    • numpy & pandas: Data manipulation
    • matplotlib & seaborn: Data visualization
    • tensorflow & keras: Building and training the CNN
    • opencv-python: Image processing
    • scikit-learn: Utilities for model evaluation

    Data Collection

    Choosing a Dataset

    For face recognition, you need a dataset containing images of multiple individuals. Some popular datasets include:

    • Labeled Faces in the Wild (LFW): Contains over 13,000 images of faces collected from the web.
    • VGGFace2: A large-scale dataset with over 3.3 million images of 9,131 subjects.
    • Custom Dataset: Collect your own images using a webcam or other cameras.

    For this project, we'll use the LFW dataset as it is standard and freely available.

    Downloading the LFW Dataset

    You can download the LFW dataset directly using keras:

    from tensorflow.keras.datasets import lfw
    lfw_dataset = lfw.load_lfw_people(min_faces_per_person=70, resize=0.4)
    

    Alternatively, download it from the official website and extract the images.


    Data Preprocessing

    Proper preprocessing is crucial for achieving good performance.

    Steps Involved:

    1. Face Alignment and Detection: Ensure faces are properly aligned within the images.
    2. Resizing: Standardize image sizes.
    3. Normalization: Scale pixel values.
    4. Data Augmentation: Enhance the dataset with transformations.
    5. Splitting the Dataset: Create training, validation, and testing sets.

    Implementation

    We'll proceed using the LFW dataset loaded via Keras, which is already processed to some extent.

    import numpy as np
    import matplotlib.pyplot as plt
    from sklearn.model_selection import train_test_split
    
    # Load dataset
    from tensorflow.keras.datasets import lfw
    lfw_dataset = lfw.load_lfw_people(min_faces_per_person=70, resize=0.4)
    
    X = lfw_dataset.images  # Shape: (n_samples, height, width)
    y = lfw_dataset.target   # Integer labels
    class_names = lfw_dataset.target_names
    n_classes = len(class_names)
    
    print(f"Number of samples: {X.shape[0]}")
    print(f"Number of classes: {n_classes}")
    

    Resizing and Normalization

    # Reshape data to add channel dimension
    X = X[..., np.newaxis]  # Shape: (n_samples, height, width, 1)
    
    # Normalize pixel values to [0, 1]
    X = X.astype('float32') / 255.0
    
    # One-hot encode labels
    from tensorflow.keras.utils import to_categorical
    y = to_categorical(y, num_classes=n_classes)
    

    Data Augmentation

    Using Keras’ ImageDataGenerator to perform real-time data augmentation during training.

    from tensorflow.keras.preprocessing.image import ImageDataGenerator
    
    datagen = ImageDataGenerator(
        rotation_range=20,
        width_shift_range=0.2,
        height_shift_range=0.2,
        horizontal_flip=True,
        zoom_range=0.2,
        shear_range=0.2,
        fill_mode='nearest',
        validation_split=0.2  # Reserve 20% for validation
    )
    

    Building the CNN Model

    Designing a CNN architecture suitable for face recognition.

    Architecture Overview

    A typical CNN for face recognition might include:

    • Convolutional Layers: To extract features
    • Pooling Layers: To reduce spatial dimensions
    • Dropout Layers: To prevent overfitting
    • Fully Connected Layers: For classification

    Alternatively, leveraging pre-trained models (Transfer Learning) like VGGFace, ResNet, or Inception can expedite development and improve performance.

    Here, we'll build a custom CNN and also discuss using a pre-trained model.

    Custom CNN Model

    from tensorflow.keras.models import Sequential
    from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout, BatchNormalization
    
    def create_custom_cnn(input_shape, num_classes):
        model = Sequential()
    
        # 1st Convolutional Block
        model.add(Conv2D(32, (3, 3), activation='relu', input_shape=input_shape))
        model.add(BatchNormalization())
        model.add(MaxPooling2D((2, 2)))
        model.add(Dropout(0.25))
    
        # 2nd Convolutional Block
        model.add(Conv2D(64, (3, 3), activation='relu'))
        model.add(BatchNormalization())
        model.add(MaxPooling2D((2, 2)))
        model.add(Dropout(0.25))
    
        # 3rd Convolutional Block
        model.add(Conv2D(128, (3, 3), activation='relu'))
        model.add(BatchNormalization())
        model.add(MaxPooling2D((2, 2)))
        model.add(Dropout(0.25))
    
        # Flatten and Fully Connected Layers
        model.add(Flatten())
        model.add(Dense(512, activation='relu'))
        model.add(BatchNormalization())
        model.add(Dropout(0.5))
        model.add(Dense(num_classes, activation='softmax'))
    
        return model
    
    input_shape = X.shape[1:]  # (height, width, channels)
    model = create_custom_cnn(input_shape, n_classes)
    model.summary()
    

    Using Transfer Learning (Optional)

    Leveraging a pre-trained model like VGG16 or ResNet50 can improve performance, especially with limited data.

    from tensorflow.keras.applications import VGG16
    from tensorflow.keras.layers import GlobalAveragePooling2D
    from tensorflow.keras.models import Model
    
    def create_transfer_learning_model(input_shape, num_classes):
        base_model = VGG16(weights='imagenet', include_top=False, input_shape=input_shape)
        base_model.trainable = False  # Freeze the base model
    
        x = base_model.output
        x = GlobalAveragePooling2D()(x)
        x = Dense(512, activation='relu')(x)
        x = Dropout(0.5)(x)
        predictions = Dense(num_classes, activation='softmax')(x)
    
        model = Model(inputs=base_model.input, outputs=predictions)
        return model
    
    model = create_transfer_learning_model(input_shape, n_classes)
    model.summary()
    

    Note: Transfer learning is particularly useful when you have a smaller dataset or want to leverage features learned from a larger dataset.


    Training the Model

    Compiling the Model

    Choose an optimizer, loss function, and evaluation metrics.

    model.compile(
        optimizer='adam',
        loss='categorical_crossentropy',
        metrics=['accuracy']
    )
    

    Setting Up Callbacks

    Use callbacks like EarlyStopping and ModelCheckpoint to enhance training.

    from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint
    
    early_stop = EarlyStopping(monitor='val_loss', patience=10, restore_best_weights=True)
    checkpoint = ModelCheckpoint('face_recognition_model.h5', monitor='val_accuracy', save_best_only=True)
    

    Training with Data Augmentation

    batch_size = 32
    epochs = 50
    
    train_generator = datagen.flow(X, y, batch_size=batch_size, subset='training')
    validation_generator = datagen.flow(X, y, batch_size=batch_size, subset='validation')
    
    history = model.fit(
        train_generator,
        steps_per_epoch=train_generator.samples // batch_size,
        validation_data=validation_generator,
        validation_steps=validation_generator.samples // batch_size,
        epochs=epochs,
        callbacks=[early_stop, checkpoint]
    )
    

    Note: If using a pre-trained model, you might consider unfreezing some of the top layers after initial training and retraining with a lower learning rate.


    Evaluating the Model

    After training, evaluate the model's performance on the validation and test sets.

    Plotting Training History

    import matplotlib.pyplot as plt
    
    # Accuracy Plot
    plt.figure(figsize=(12, 4))
    plt.subplot(1, 2, 1)
    plt.plot(history.history['accuracy'], label='Train Accuracy')
    plt.plot(history.history['val_accuracy'], label='Validation Accuracy')
    plt.title('Accuracy over epochs')
    plt.xlabel('Epoch')
    plt.ylabel('Accuracy')
    plt.legend()
    
    # Loss Plot
    plt.subplot(1, 2, 2)
    plt.plot(history.history['loss'], label='Train Loss')
    plt.plot(history.history['val_loss'], label='Validation Loss')
    plt.title('Loss over epochs')
    plt.xlabel('Epoch')
    plt.ylabel('Loss')
    plt.legend()
    
    plt.show()
    

    Confusion Matrix

    Visualize how well the model is performing across different classes.

    from sklearn.metrics import confusion_matrix, classification_report
    import seaborn as sns
    
    # Predict on validation data
    val_steps = validation_generator.samples // batch_size
    y_pred = model.predict(validation_generator, steps=val_steps)
    y_pred_classes = np.argmax(y_pred, axis=1)
    y_true = np.argmax(y_pred_classes, axis=1)  # Adjust if necessary
    
    # Confusion Matrix
    cm = confusion_matrix(validation_generator.classes[:len(y_pred_classes)], y_pred_classes)
    plt.figure(figsize=(20, 20))
    sns.heatmap(cm, annot=True, fmt='d', xticklabels=class_names, yticklabels=class_names)
    plt.ylabel('Actual')
    plt.xlabel('Predicted')
    plt.title('Confusion Matrix')
    plt.show()
    
    # Classification Report
    print(classification_report(validation_generator.classes[:len(y_pred_classes)], y_pred_classes, target_names=class_names))
    

    Testing on Unseen Data

    If you have a separate test set, evaluate the model's generalization.

    # Assuming you have X_test and y_test
    test_loss, test_accuracy = model.evaluate(X_test, y_test, verbose=2)
    print(f"Test Accuracy: {test_accuracy*100:.2f}%")
    

    Deploying the Model

    Once trained and evaluated, the model can be deployed to recognize faces in real-time or batch processes.

    Saving the Model

    model.save('face_recognition_model.h5')
    

    Loading the Model

    from tensorflow.keras.models import load_model
    model = load_model('face_recognition_model.h5')
    

    Real-Time Face Recognition using OpenCV

    Here's a simple example of using OpenCV to capture video from a webcam and perform face recognition.

    import cv2
    import numpy as np
    from tensorflow.keras.models import load_model
    
    # Load the trained model
    model = load_model('face_recognition_model.h5')
    
    # Initialize webcam
    cap = cv2.VideoCapture(0)
    
    # Load Haar Cascade for face detection
    face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')
    
    while True:
        ret, frame = cap.read()
        if not ret:
            break
    
        # Convert to grayscale for face detection
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    
        # Detect faces
        faces = face_cascade.detectMultiScale(gray, 1.3, 5)
    
        for (x, y, w, h) in faces:
            # Extract face ROI
            face = gray[y:y+h, x:x+w]
            face = cv2.resize(face, (lfw_dataset.images.shape[1], lfw_dataset.images.shape[2]))
            face = face.astype('float32') / 255.0
            face = np.expand_dims(face, axis=-1)
            face = np.expand_dims(face, axis=0)
    
            # Predict
            predictions = model.predict(face)
            class_idx = np.argmax(predictions, axis=1)
            class_name = class_names[class_idx[0]]
    
            # Display bounding box and name
            cv2.rectangle(frame, (x, y), (x+w, y+h), (255, 0, 0), 2)
            cv2.putText(frame, class_name, (x, y-10), cv2.FONT_HERSHEY_SIMPLEX, 
                        0.9, (36,255,12), 2)
    
        # Display the resulting frame
        cv2.imshow('Face Recognition', frame)
    
        # Break loop on 'q' key press
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
    
    # Release the capture and close windows
    cap.release()
    cv2.destroyAllWindows()
    

    Note:

    • Face Detection: Before recognition, faces must be detected. Here, Haar Cascades are used for simplicity, but more accurate detectors like MTCNN or Dlib's HOG/SVM can be utilized.
    • Performance: Real-time recognition may require optimization, such as resizing frames or leveraging GPU acceleration.
    • Security and Privacy: Ensure ethical use and compliance with privacy laws when deploying face recognition applications.

    Full Code Implementation

    Putting it all together, here’s a complete Python script encapsulating the entire process.

    import numpy as np
    import matplotlib.pyplot as plt
    import seaborn as sns
    import cv2
    from sklearn.metrics import confusion_matrix, classification_report
    from sklearn.model_selection import train_test_split
    from tensorflow.keras.datasets import lfw
    from tensorflow.keras.models import Sequential, load_model, Model
    from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout, BatchNormalization, GlobalAveragePooling2D
    from tensorflow.keras.utils import to_categorical
    from tensorflow.keras.preprocessing.image import ImageDataGenerator
    from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint
    from tensorflow.keras.applications import VGG16
    
    # 1. Load Dataset
    lfw_dataset = lfw.load_lfw_people(min_faces_per_person=70, resize=0.4)
    X = lfw_dataset.images
    y = lfw_dataset.target
    class_names = lfw_dataset.target_names
    n_classes = len(class_names)
    
    print(f"Number of samples: {X.shape[0]}")
    print(f"Number of classes: {n_classes}")
    
    # 2. Preprocess Data
    X = X[..., np.newaxis]  # Add channel dimension
    X = X.astype('float32') / 255.0  # Normalize
    
    y = to_categorical(y, num_classes=n_classes)  # One-hot encode
    
    # 3. Data Augmentation
    datagen = ImageDataGenerator(
        rotation_range=20,
        width_shift_range=0.2,
        height_shift_range=0.2,
        horizontal_flip=True,
        zoom_range=0.2,
        shear_range=0.2,
        fill_mode='nearest',
        validation_split=0.2
    )
    
    # 4. Build Model (Custom CNN)
    def create_custom_cnn(input_shape, num_classes):
        model = Sequential()
    
        # 1st Convolutional Block
        model.add(Conv2D(32, (3, 3), activation='relu', input_shape=input_shape))
        model.add(BatchNormalization())
        model.add(MaxPooling2D((2, 2)))
        model.add(Dropout(0.25))
    
        # 2nd Convolutional Block
        model.add(Conv2D(64, (3, 3), activation='relu'))
        model.add(BatchNormalization())
        model.add(MaxPooling2D((2, 2)))
        model.add(Dropout(0.25))
    
        # 3rd Convolutional Block
        model.add(Conv2D(128, (3, 3), activation='relu'))
        model.add(BatchNormalization())
        model.add(MaxPooling2D((2, 2)))
        model.add(Dropout(0.25))
    
        # Fully Connected Layers
        model.add(Flatten())
        model.add(Dense(512, activation='relu'))
        model.add(BatchNormalization())
        model.add(Dropout(0.5))
        model.add(Dense(num_classes, activation='softmax'))
    
        return model
    
    input_shape = X.shape[1:]
    model = create_custom_cnn(input_shape, n_classes)
    model.summary()
    
    # 5. Compile Model
    model.compile(
        optimizer='adam',
        loss='categorical_crossentropy',
        metrics=['accuracy']
    )
    
    # 6. Set Up Callbacks
    early_stop = EarlyStopping(monitor='val_loss', patience=10, restore_best_weights=True)
    checkpoint = ModelCheckpoint('face_recognition_model.h5', monitor='val_accuracy', save_best_only=True)
    
    # 7. Train Model
    batch_size = 32
    epochs = 50
    
    train_generator = datagen.flow(X, y, batch_size=batch_size, subset='training')
    validation_generator = datagen.flow(X, y, batch_size=batch_size, subset='validation')
    
    history = model.fit(
        train_generator,
        steps_per_epoch=train_generator.samples // batch_size,
        validation_data=validation_generator,
        validation_steps=validation_generator.samples // batch_size,
        epochs=epochs,
        callbacks=[early_stop, checkpoint]
    )
    
    # 8. Evaluate Model
    # Plot training history
    plt.figure(figsize=(12, 4))
    
    # Accuracy
    plt.subplot(1, 2, 1)
    plt.plot(history.history['accuracy'], label='Train Acc')
    plt.plot(history.history['val_accuracy'], label='Val Acc')
    plt.title('Accuracy')
    plt.xlabel('Epoch')
    plt.ylabel('Accuracy')
    plt.legend()
    
    # Loss
    plt.subplot(1, 2, 2)
    plt.plot(history.history['loss'], label='Train Loss')
    plt.plot(history.history['val_loss'], label='Val Loss')
    plt.title('Loss')
    plt.xlabel('Epoch')
    plt.ylabel('Loss')
    plt.legend()
    
    plt.show()
    
    # Confusion Matrix
    val_steps = validation_generator.samples // batch_size
    y_pred = model.predict(validation_generator, steps=val_steps)
    y_pred_classes = np.argmax(y_pred, axis=1)
    y_true = validation_generator.classes[:len(y_pred_classes)]
    
    cm = confusion_matrix(y_true, y_pred_classes)
    plt.figure(figsize=(20, 20))
    sns.heatmap(cm, annot=True, fmt='d', xticklabels=class_names, yticklabels=class_names)
    plt.ylabel('Actual')
    plt.xlabel('Predicted')
    plt.title('Confusion Matrix')
    plt.show()
    
    print(classification_report(y_true, y_pred_classes, target_names=class_names))
    
    # 9. Save Model
    model.save('face_recognition_model_final.h5')
    
    # 10. (Optional) Real-Time Recognition
    # Uncomment the following code to perform real-time face recognition
    
    """
    # Load the saved model
    model = load_model('face_recognition_model_final.h5')
    
    # Initialize webcam
    cap = cv2.VideoCapture(0)
    
    # Load Haar Cascade for face detection
    face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')
    
    while True:
        ret, frame = cap.read()
        if not ret:
            break
    
        # Convert to grayscale
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    
        # Detect faces
        faces = face_cascade.detectMultiScale(gray, 1.3, 5)
    
        for (x, y, w, h) in faces:
            # Extract ROI
            face = gray[y:y+h, x:x+w]
            face = cv2.resize(face, (lfw_dataset.images.shape[1], lfw_dataset.images.shape[2]))
            face = face.astype('float32') / 255.0
            face = np.expand_dims(face, axis=-1)
            face = np.expand_dims(face, axis=0)
    
            # Predict
            predictions = model.predict(face)
            class_idx = np.argmax(predictions, axis=1)
            class_name = class_names[class_idx[0]]
    
            # Annotate
            cv2.rectangle(frame, (x, y), (x+w, y+h), (255,0,0), 2)
            cv2.putText(frame, class_name, (x, y-10), cv2.FONT_HERSHEY_SIMPLEX, 
                        0.9, (36,255,12), 2)
    
        # Display
        cv2.imshow('Face Recognition', frame)
    
        # Exit on 'q'
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
    
    cap.release()
    cv2.destroyAllWindows()
    """
    
    

    Explanation:

    1. Loading and Preprocessing: The script loads the LFW dataset, normalizes it, and prepares labels.

    2. Data Augmentation: ImageDataGenerator is set up with augmentation parameters and splits the data into training and validation sets.

    3. Model Building: A custom CNN is defined with multiple convolutional blocks, batch normalization, dropout, and dense layers.

    4. Training: The model is compiled and trained using the augmented data. Early stopping and model checkpointing are used to prevent overfitting and save the best model.

    5. Evaluation: Training history is plotted, and a confusion matrix along with a classification report is generated to assess performance.

    6. Deployment: Optionally, the trained model can be used for real-time face recognition with OpenCV.


    Conclusion

    This project provides a comprehensive approach to building a face recognition system using CNNs. By following the steps outlined above, you can develop a robust model capable of accurately recognizing faces. Depending on your requirements and dataset size, you may choose between building a custom CNN or leveraging transfer learning with pre-trained models for enhanced performance.

    Next Steps:

    • Improve Accuracy: Experiment with deeper architectures, fine-tuning, or advanced data augmentation techniques.
    • Handle Variations: Incorporate techniques to handle different lighting conditions, angles, and occlusions.
    • Optimize Performance: Utilize GPU acceleration, model pruning, or quantization for faster inference.
    • Expand Functionality: Implement features like facial landmark detection, emotion recognition, or age estimation.

    Ethical Considerations:

    Ensure that your face recognition system respects privacy laws and ethical guidelines. Obtain necessary consents and use the technology responsibly to prevent misuse.


    Feel free to ask if you need further details or assistance with specific parts of the project!