Introduction We hope you had a blast at yesterday’s Rickey Kej concert! Here’s a quick…
- User
Introduction We hope you had a blast at yesterday’s Rickey Kej concert! Here’s a quick quiz for you: What genre were his songs? Stumped? Still stuck in your year-long Taylor Swift phase? Well, you're not alone. One of the 4 coordies only listens to a very niche, very questionable genre. Tonight, you’ll be the hero who upgrades their Spotify Wrapped and boosts their friend-making game :) Problem Statement You’re given a dataset with audio files from various genres. Sounds easy, right? But wait, there’s a twist. The dataset you’ll be evaluated on is full of unique challenges—audio files with all kinds of delightful distractions. Think Tom & Jerry brawls, random lecture recordings, and that unmistakable chaos from the local cheap Chinese food joint (yes, we collected those ourselves, and yes, it was worth it). Your job? Train a machine learning model to predict the genre of these noisy, chaotic audio files. How you process, clean, and augment the data is completely up to you. Good luck—you're gonna need it! Dataset: ● Dataset Time Restrictions: ● Solid 3 hours Evaluation: ● The dataset we’ll use to test your model will remain undisclosed until evaluation. See the Deliverables section for submission guidelines. ● Your predictions will be evaluated against a hidden ground truth set. Takneek PS - SnT Code Scoring: In-game total points = 100. Note that these in-game points are different from the Takneek points of the PS, made only to stop you guys from coming in every 5 minutes for evaluation. ● 100% correct predictions will fetch you full points. ● 90%-99% correct predictions will fetch you 90% of the points. ● 80%-89% correct predictions will fetch you 80% of the points. ● And so on.. ● In case of ties, number of correct predictions will be used as a tiebreaker among pools with same score. Deliverables: A Google form will be shared in which you’ll submit all these files in a zip format. ● Model weights - Your trained model weights ● prediction.py - Submit a prediction.py which when we place it in the directory containing all the audios, using your trained model weights generates a prediction.csv with all your predictions in it. See the Predictions CSV template. ● requirements.txt - Any library you’ve used and is necessary for us to run prediction.py. Few important points: ● Follow the template CSV format, predictions should be in all small letters. ● We will have one directory with all the audio files in it which genres you want to predict. ● All your deliverables will be placed in that directory along with all the audio files. ● See the following example screenshot ● Here audio1, audio2 are audio files whose genre you have to predict. ● model1.pth, predictions.py are files submitted by you. So take care of all the relative path conventions you use in your prediction.py file
- Assistant
Approach to the Audio Genre Classification Challenge
Congratulations on taking on this exciting and challenging task! Here's a comprehensive step-by-step guide to help you navigate through the competition effectively within the given 3-hour timeframe.
1. Understand the Task
- Objective: Develop a machine learning model to classify the genre of noisy and chaotic audio files.
- Challenges:
- Noisy data with various types of background sounds.
- Unknown genres (presumably multiple classes).
- Deliverables:
- Trained model weights.
prediction.pyscript that generatesprediction.csv.requirements.txtwith all necessary dependencies.
2. Environment Setup
Ensure you have the necessary libraries installed. Create a virtual environment to manage dependencies.
# Create virtual environment python -m venv audio_env # Activate virtual environment # For Windows: audio_env\Scripts\activate # For Unix or MacOS: source audio_env/bin/activate # Install necessary libraries pip install numpy pandas librosa scikit-learn torch torchvision torchaudio pandasNote: Adjust the libraries based on your preferred framework (PyTorch or TensorFlow). Below, we'll proceed with PyTorch.
3. Data Preprocessing
a. Loading and Cleaning Audio Data
Use
librosato load audio files. Considering the noisy nature, applying noise reduction techniques can be beneficial.import os import librosa import numpy as np def load_audio(file_path, sr=22050): y, sr = librosa.load(file_path, sr=sr) return y, srb. Feature Extraction
Convert audio signals into spectrograms or Mel-frequency cepstral coefficients (MFCCs), which are effective for classification tasks.
def extract_features(y, sr, n_mfcc=40): # Compute MFCCs mfcc = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=n_mfcc) # Normalize MFCCs mfcc = (mfcc - np.mean(mfcc)) / np.std(mfcc) return mfccc. Handling Variable Lengths
Ensure all input features have the same dimensions. You can pad or truncate the sequences.
def pad_features(mfcc, max_len=174): # Example max_len, adjust based on data if mfcc.shape[1] < max_len: pad_width = max_len - mfcc.shape[1] mfcc = np.pad(mfcc, pad_width=((0,0),(0,pad_width)), mode='constant') else: mfcc = mfcc[:, :max_len] return mfcc
4. Model Selection
Given the time constraints, leveraging a pre-trained model with transfer learning can expedite the process. However, for audio, models like Convolutional Neural Networks (CNNs) are commonly used.
a. Define the CNN Model
Here's a simple CNN architecture using PyTorch:
import torch import torch.nn as nn import torch.nn.functional as F class AudioClassifier(nn.Module): def __init__(self, num_classes): super(AudioClassifier, self).__init__() self.conv1 = nn.Conv2d(1, 16, kernel_size=3, padding=1) self.bn1 = nn.BatchNorm2d(16) self.pool = nn.MaxPool2d(2,2) self.conv2 = nn.Conv2d(16, 32, kernel_size=3, padding=1) self.bn2 = nn.BatchNorm2d(32) self.fc1 = nn.Linear(32 * 44 * 43, 128) # Adjust based on input size self.fc2 = nn.Linear(128, num_classes) def forward(self, x): x = self.pool(F.relu(self.bn1(self.conv1(x)))) # (batch, 16, H/2, W/2) x = self.pool(F.relu(self.bn2(self.conv2(x)))) # (batch, 32, H/4, W/4) x = x.view(x.size(0), -1) # Flatten x = F.relu(self.fc1(x)) x = self.fc2(x) return xEnsure to adjust the
fc1layer input dimensions based on the feature size after convolution and pooling.
5. Training the Model
a. Data Preparation
Assuming you have labeled training data, split it into training and validation sets.
from sklearn.model_selection import train_test_split from torch.utils.data import Dataset, DataLoader class AudioDataset(Dataset): def __init__(self, file_paths, labels, transform=None): self.file_paths = file_paths self.labels = labels self.transform = transform def __len__(self): return len(self.file_paths) def __getitem__(self, idx): y, sr = load_audio(self.file_paths[idx]) mfcc = extract_features(y, sr) mfcc = pad_features(mfcc) mfcc = mfcc[np.newaxis, ...] # Add channel dimension mfcc = torch.tensor(mfcc, dtype=torch.float) label = self.labels[idx] return mfcc, labelb. Training Loop
import torch.optim as optim # Assume file_paths and labels are defined X_train, X_val, y_train, y_val = train_test_split(file_paths, labels, test_size=0.2, random_state=42) train_dataset = AudioDataset(X_train, y_train) val_dataset = AudioDataset(X_val, y_val) train_loader = DataLoader(train_dataset, batch_size=32, shuffle=True) val_loader = DataLoader(val_dataset, batch_size=32, shuffle=False) device = torch.device("cuda" if torch.cuda.is_available() else "cpu") model = AudioClassifier(num_classes).to(device) criterion = nn.CrossEntropyLoss() optimizer = optim.Adam(model.parameters(), lr=0.001) # Training Loop epochs = 10 for epoch in range(epochs): model.train() running_loss = 0.0 for inputs, labels in train_loader: inputs, labels = inputs.to(device), labels.to(device) optimizer.zero_grad() outputs = model(inputs) loss = criterion(outputs, labels) loss.backward() optimizer.step() running_loss += loss.item() # Validation model.eval() correct = 0 total = 0 with torch.no_grad(): for inputs, labels in val_loader: inputs, labels = inputs.to(device), labels.to(device) outputs = model(inputs) _, preds = torch.max(outputs, 1) correct += (preds == labels).sum().item() total += labels.size(0) val_acc = correct / total print(f"Epoch {epoch+1}, Loss: {running_loss/len(train_loader)}, Val Acc: {val_acc}")Adjust
epochs,batch_size, and other hyperparameters as needed.c. Saving the Model Weights
After training, save the model weights.
torch.save(model.state_dict(), 'model.pth')
6. Prediction Script (
prediction.py)Create a script that loads the trained model, processes the input audio files, and generates predictions.
import os import sys import torch import librosa import numpy as np import pandas as pd from model import AudioClassifier # Ensure model.py has the AudioClassifier class def load_audio(file_path, sr=22050): y, sr = librosa.load(file_path, sr=sr) return y, sr def extract_features(y, sr, n_mfcc=40): mfcc = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=n_mfcc) mfcc = (mfcc - np.mean(mfcc)) / np.std(mfcc) return mfcc def pad_features(mfcc, max_len=174): if mfcc.shape[1] < max_len: pad_width = max_len - mfcc.shape[1] mfcc = np.pad(mfcc, pad_width=((0,0),(0,pad_width)), mode='constant') else: mfcc = mfcc[:, :max_len] return mfcc def main(): # Load model device = torch.device("cuda" if torch.cuda.is_available() else "cpu") model = AudioClassifier(num_classes).to(device) model.load_state_dict(torch.load('model.pth', map_location=device)) model.eval() # Get list of audio files audio_files = [f for f in os.listdir('.') if f.endswith(('.wav', '.mp3', '.flac'))] predictions = [] filenames = [] for file in audio_files: y, sr = load_audio(file) mfcc = extract_features(y, sr) mfcc = pad_features(mfcc) mfcc = mfcc[np.newaxis, np.newaxis, ...] # (1, 1, n_mfcc, max_len) mfcc = torch.tensor(mfcc, dtype=torch.float).to(device) with torch.no_grad(): output = model(mfcc) _, pred = torch.max(output, 1) predictions.append(label_mapping[pred.item()]) filenames.append(file) # Create DataFrame df = pd.DataFrame({ 'filename': filenames, 'genre': predictions }) # Save to CSV df.to_csv('prediction.csv', index=False) if __name__ == "__main__": main()Notes:
model.py: Ensure you have a separatemodel.pycontaining theAudioClassifierclass.label_mapping: Define a dictionary to map class indices to genre names.- File Filtering: Adjust the audio file extensions as per your dataset.
7.
requirements.txtList all the necessary libraries. You can generate this automatically if you've used a virtual environment.
pip freeze > requirements.txtEnsure it includes all essential packages like:
numpy pandas librosa torch torchaudio scikit-learn
8. Packaging Deliverables
- Model Weights:
model.pth - Prediction Script:
prediction.py - Requirements:
requirements.txt
Zip these files and ensure they can operate relative to the audio files directory.
zip submission.zip model.pth prediction.py requirements.txt
9. Additional Tips
-
Data Augmentation: To improve model robustness against noise, consider applying techniques like adding Gaussian noise, time-shifting, or altering pitch.
-
Error Handling: Ensure
prediction.pycan handle potential errors, such as unsupported file formats or corrupted files. -
Testing: Before submission, test
prediction.pyin an isolated environment to ensure it generatesprediction.csvcorrectly. -
Documentation: Comment your code for clarity, especially in
prediction.py, to make your submission understandable.
Sample
prediction.csvFormatfilename,genre audio1.wav,rock audio2.mp3,jazz audio3.flac,hiphop ...Ensure all genre names are in lowercase as per the guidelines.
Conclusion
By following this structured approach, you can efficiently develop a robust audio genre classification model within the tight time constraints. Focus on creating a solid foundation with preprocessing and a reliable model architecture, and iterate based on validation performance. Good luck!