Your goal will be to predict the price of used cars based on various attributes. Dataset…
- User
Your goal will be to predict the price of used cars based on various attributes.
Dataset
The dataset for this competition (both train and test) was generated from a deep learning model trained on the "Used Car Price Prediction Dataset". Feature distributions are close to, but not exactly the same, as the original. Feel free to use the original dataset as part of this competition, both to explore differences as well as to see whether incorporating the original in training improves model performance.
Files:
- train.csv - the training dataset;
priceis the continuous target - test.csv - the test dataset; your objective is to predict the value of
pricefor each row
train.csv
Columns:
- id (integer, e.g. 0)
- brand (string, e.g. "MINI")
- model (string, e.g. "Cooper S Base")
- model_year (integer, e.g. 2007)
- milage (integer, e.g. 213000)
- fuel_type (string, e.g. "Gasoline")
- engine (string, e.g. "172.0HP 1.6L 4 Cylinder Engine Gasoline Fuel")
- transmission (string, e.g. "A/T")
- ext_col (string, e.g. "Yellow")
- int_col (string, e.g. "Gray")
- accident (string, e.g. "None reported")
- clean_title (string, e.g. "Yes")
- price (integer, e.g. 4200)
test.csv
Columns:
- id (integer, e.g. 188533)
- brand (string, e.g. "Land")
- model (string, e.g. "Rover LR2 Base")
- model_year (integer, e.g. 2015)
- milage (integer, e.g. 98000)
- fuel_type (string, e.g. "Gasoline")
- engine (string, e.g. "240.0HP 2.0L 4 Cylinder Engine Gasoline Fuel")
- transmission (string, e.g. "6-Speed A/T")
- ext_col (string, e.g. "White")
- int_col (string, e.g. "Beige")
- accident (string, e.g. "None reported")
- clean_title (string, e.g. "Yes")
Submissions
Submissions are scored on the root mean squared error. RMSE is defined as:
RMSE = (1/N * Σ (yᵢ - ŷᵢ)²)½where ŷᵢ is the predicted value and yᵢ is the original value for each instance i.
Your submission file should look like this:
id,price 188533,43878.016 188534,43878.016 188535,43878.016 etc.For each
idin the test set, you must predict thepriceof the car. The file should contain a header and have the format above. - train.csv - the training dataset;
- Assistant
Certainly! Predicting the price of used cars is a classic regression problem in machine learning. Below, I'll guide you through the steps to build a predictive model using the provided dataset. We'll cover data loading, exploratory data analysis (EDA), preprocessing, model building, evaluation, and preparing the submission file.
Table of Contents
- Understanding the Data
- Data Loading
- Exploratory Data Analysis (EDA)
- Data Preprocessing
- Feature Engineering
- Model Building
- Model Evaluation
- Making Predictions on Test Data
- Preparing the Submission File
Understanding the Data
Before diving into code, it's crucial to understand the dataset:
- Train Dataset (
train.csv): Contains features and the target variableprice. - Test Dataset (
test.csv): Contains features, and your goal is to predictpricefor these entries.
Features Explained
- id: Unique identifier for each car.
- brand: Brand of the car (e.g., "MINI").
- model: Model of the car.
- model_year: Year the model was released.
- milage: Total miles driven.
- fuel_type: Type of fuel the car uses.
- engine: Description of the engine.
- transmission: Type of transmission (e.g., "A/T" for automatic).
- ext_col: Exterior color.
- int_col: Interior color.
- accident: Accident history.
- clean_title: Indicates if the car has a clean title.
- price: Target variable (only in train dataset).
Data Loading
First, let's load the data using Python's pandas library.
import pandas as pd # Load the training data train = pd.read_csv('train.csv') # Load the test data test = pd.read_csv('test.csv')
Exploratory Data Analysis (EDA)
Let's explore the data to understand distributions, missing values, and relationships.
1. Quick Data Overview
# View first few rows print(train.head()) # Get info about data types and missing values print(train.info()) # Statistical summary print(train.describe())2. Checking for Missing Values
# Missing values in training data print(train.isnull().sum()) # Missing values in test data print(test.isnull().sum())3. Visualizing Distributions
It's helpful to visualize the target variable and features.
import matplotlib.pyplot as plt import seaborn as sns # Histogram of the target variable 'price' sns.histplot(train['price'], kde=True) plt.title('Distribution of Car Prices') plt.show()4. Correlation Matrix
Since we are dealing with numerical and categorical data, we need to handle them differently.
# Correlation matrix (only for numerical features) numeric_features = ['model_year', 'milage', 'price'] corr_matrix = train[numeric_features].corr() sns.heatmap(corr_matrix, annot=True) plt.show()
Data Preprocessing
1. Handling Missing Values
Assuming there are missing values, decide how to handle them.
# For example, fill missing 'milage' with the median train['milage'].fillna(train['milage'].median(), inplace=True) test['milage'].fillna(test['milage'].median(), inplace=True)2. Encoding Categorical Variables
We need to convert categorical variables into numerical format.
a. Label Encoding
For ordinal categorical variables.
b. One-Hot Encoding
For nominal categorical variables.
from sklearn.preprocessing import OneHotEncoder # Combine train and test data for consistent encoding combined_data = pd.concat([train.drop('price', axis=1), test], ignore_index=True) # List of categorical columns cat_cols = ['brand', 'model', 'fuel_type', 'transmission', 'ext_col', 'int_col', 'accident', 'clean_title'] # One-Hot Encoding combined_data = pd.get_dummies(combined_data, columns=cat_cols, drop_first=True) # Split back into train and test sets train_processed = combined_data.iloc[:train.shape[0], :] test_processed = combined_data.iloc[train.shape[0]:, :] # Add the target variable back to the train set train_processed['price'] = train['price']3. Feature Scaling
For certain algorithms, scaling features can improve performance.
from sklearn.preprocessing import StandardScaler # Scale 'milage' and 'model_year' scaler = StandardScaler() train_processed[['milage', 'model_year']] = scaler.fit_transform(train_processed[['milage', 'model_year']]) test_processed[['milage', 'model_year']] = scaler.transform(test_processed[['milage', 'model_year']])
Feature Engineering
Consider extracting additional information from existing features.
1. Extracting Engine Power
From the
enginecolumn, extract horsepower.import re def extract_horsepower(engine_str): match = re.search(r'(\d+(\.\d+)?)HP', engine_str) if match: return float(match.group(1)) else: return None train_processed['horsepower'] = train['engine'].apply(extract_horsepower) test_processed['horsepower'] = test['engine'].apply(extract_horsepower) # Handle missing horsepower values train_processed['horsepower'].fillna(train_processed['horsepower'].median(), inplace=True) test_processed['horsepower'].fillna(test_processed['horsepower'].median(), inplace=True)2. Dropping Unnecessary Columns
If certain columns are not useful, drop them.
# For example, drop 'engine' and 'id' train_processed.drop(['engine', 'id'], axis=1, inplace=True) test_ids = test_processed['id'] # Save ids for submission test_processed.drop(['engine', 'id'], axis=1, inplace=True)
Model Building
Now, let's build a regression model.
1. Splitting the Data
We need to split the training data into features and target.
X = train_processed.drop('price', axis=1) y = train_processed['price']Optionally, split into training and validation sets for evaluation.
from sklearn.model_selection import train_test_split X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)2. Choosing a Model
Common regression algorithms include:
- Linear Regression
- Decision Trees
- Random Forest
- Gradient Boosting (XGBoost, LightGBM)
- Support Vector Regressor (SVR)
For this problem, ensemble methods like Random Forest or Gradient Boosting often perform well.
Using Random Forest Regressor
from sklearn.ensemble import RandomForestRegressor model = RandomForestRegressor(n_estimators=100, random_state=42)3. Training the Model
model.fit(X_train, y_train)
Model Evaluation
1. Predicting on Validation Set
from sklearn.metrics import mean_squared_error import numpy as np y_pred = model.predict(X_val) # Calculate RMSE rmse = np.sqrt(mean_squared_error(y_val, y_pred)) print(f'Validation RMSE: {rmse}')2. Cross-Validation (Optional)
To get a better estimate of the model's performance.
from sklearn.model_selection import cross_val_score cv_scores = cross_val_score(model, X, y, cv=5, scoring='neg_root_mean_squared_error') print(f'Cross-Validation RMSE: {np.abs(cv_scores.mean())}')
Hyperparameter Tuning (Optional)
Improve model performance by tuning hyperparameters.
from sklearn.model_selection import GridSearchCV param_grid = { 'n_estimators': [100, 200], 'max_depth': [None, 10, 20], 'min_samples_split': [2, 5] } grid_search = GridSearchCV(model, param_grid, cv=3, scoring='neg_root_mean_squared_error') grid_search.fit(X_train, y_train) print(f'Best parameters: {grid_search.best_params_}') print(f'Best RMSE: {np.abs(grid_search.best_score_)}') # Use the best estimator model = grid_search.best_estimator_
Making Predictions on Test Data
# Fit the model on the entire training data model.fit(X, y) # Predict on test data test_predictions = model.predict(test_processed)
Preparing the Submission File
Following the required format:
- Include headers:
id,price - Ensure the prices are in the correct order corresponding to the test
ids.
# Create a DataFrame for submission submission = pd.DataFrame({ 'id': test_ids, 'price': test_predictions }) # Ensure the DataFrame is sorted by 'id' if necessary submission = submission.sort_values(by='id') # Save to CSV without index submission.to_csv('submission.csv', index=False) print("Submission file created successfully!")
Next Steps
-
Feature Importance: Analyze which features are most influential in predicting the price.
importances = model.feature_importances_ feature_names = X.columns feature_importance_df = pd.DataFrame({'feature': feature_names, 'importance': importances}) feature_importance_df = feature_importance_df.sort_values(by='importance', ascending=False) print(feature_importance_df) -
Model Stacking/Ensembling: Combine predictions from multiple models to improve performance.
-
Handling Outliers: Detect and handle outliers in the target variable.
-
Use External Data: Since the dataset is generated from an original dataset, you can explore using the original for additional insights.
Final Tips
-
Data Quality: Always ensure that data preprocessing is correctly handled, as garbage in leads to garbage out.
-
Version Control: Keep track of different versions of your model to compare performances.
-
Experimentation: Try different algorithms and feature engineering techniques to improve RMSE.
-
Documentation: Comment your code and document your processes for future reference.
Good luck with your model building!