Skip to content
TheChatArchiveThe Chat Archive
o1-preview-2024-09-12· OpenAIAi2 WildChat-4.8M (AI2)

In order to realize your own business, it must be officially registered. Banks are…

  1. User

    In order to realize your own business, it must be officially registered. Banks are interested in people coming to them for this service. Information about which customers are ready to start their own business helps banks to optimize marketing campaigns, improve personalized offers and save resources. Your job is to find the people who are most similar to those who have already started a business after talking to the bank.

    Data:

    In this task, you are to develop a model that determines the probability that customers will register a sole proprietorship or LLC in the coming days after a personalized communication with a bank.

    The file train.csv contains data on customers who registered a business between May and August 2023 within 14 days of the communication, and data on customers who did not register a business during this time period. The test.csv file contains information about the customers for whom you need to provide forecasts for your model. You will have various customer attributes at your disposal, such as: mb - information about different sessions in the mobile application, application - applications for banking products, bki - characteristics of applications to credit bureaus, partner - information about subscriptions to partner services, graph - data on clients' contact books, feature - other anonymized characteristics, target - value of the target variable indicating the presence (1) or absence (0) of IE or LLC registration. As a solution, upload a csv file containing the feature scores from the test sample and the registration probability predictions from your solution. Metric: recall@k = completeness of counts among the k objects with the highest positive class probabilities. recall@k = number of positive class objects correctly predicted by the model among the top-k / total number of positive class objects In practice, it is only possible to produce a link to some sample of best-fit candidates. Therefore, the solution is evaluated for completeness on the top-15% of the test data, i.e. first the results are sorted by decreasing probability predicted by the model, and then the proportion of successful registrations falling into this top is evaluated.
 The number of points is calculated after the round is completed using the formula: 
Points = 400 * percentileofscore(recall@k's, recall@k's), where percentileofscore is the function that calculates percentile rank, recall@k's are the recall@k metric values of all participants, recall@k's is the recall@k metric value of a given participant.

Files:
 sample_submission.csv train.zip test.zip

    Use the best ML solutions to solve this problem. Keep in mind that the code will run in the Google Colab environment

  2. Assistant

    To solve this problem, we'll develop a machine learning model that predicts the probability of customers registering a sole proprietorship (IE) or LLC (OOO) after a communication with the bank. We'll use Python and popular machine learning libraries to build and evaluate our model. Here's an outline of the steps we'll follow:

    1. Data Loading and Exploration: Load the training and test datasets and explore their structure and content.
    2. Data Preprocessing: Handle missing values, encode categorical variables, and perform feature engineering if necessary.
    3. Model Selection and Training: Choose an appropriate classification model and train it on the training data.
    4. Evaluation: Use cross-validation and a custom metric function to optimize for the recall@15% metric.
    5. Prediction: Make predictions on the test data.
    6. Submission: Prepare the submission file as per the required format.

    Let's proceed with the implementation.

    # Step 1: Data Loading and Exploration
    
    # Import necessary libraries
    import pandas as pd
    import numpy as np
    
    # For data visualization
    import matplotlib.pyplot as plt
    import seaborn as sns
    
    # For modeling
    from sklearn.model_selection import StratifiedKFold
    from sklearn.preprocessing import LabelEncoder
    from sklearn.metrics import recall_score
    from lightgbm import LGBMClassifier
    
    # Suppress warnings
    import warnings
    warnings.simplefilter(action='ignore', category=FutureWarning)
    
    # Load the training data
    train = pd.read_csv('train.csv')
    
    # Load the test data
    test = pd.read_csv('test.csv')
    
    # Print the shapes of the datasets
    print(f'Training data shape: {train.shape}')
    print(f'Test data shape: {test.shape}')
    
    # Display the first few rows of the training data
    train.head()
    

    Data Exploration

    Let's explore the data to understand the features and check for missing values.

    # Check for missing values in training data
    print(train.isnull().sum())
    
    # Check for missing values in test data
    print(test.isnull().sum())
    
    # Examine the target variable distribution
    train['target'].value_counts(normalize=True)
    

    Observations:

    • The target variable is imbalanced. We'll need to handle this during modeling.
    • There may be missing values in the dataset that need to be addressed.
    • Features may be categorical or numerical and need appropriate encoding.

    # Step 2: Data Preprocessing
    
    # Combine train and test data for consistent preprocessing
    test['target'] = -1  # Temporarily add target column to test data
    combined = pd.concat([train, test], axis=0, ignore_index=True)
    
    # List of features to be used (exclude 'target')
    features = [col for col in combined.columns if col != 'target']
    
    # Handle missing values
    # We can fill missing numerical values with the median and categorical with the mode
    for col in features:
        if combined[col].dtype == 'object':
            combined[col] = combined[col].fillna(combined[col].mode()[0])
        else:
            combined[col] = combined[col].fillna(combined[col].median())
    
    # Encode categorical variables using Label Encoding
    le = LabelEncoder()
    for col in features:
        if combined[col].dtype == 'object':
            combined[col] = le.fit_transform(combined[col])
    
    # Split the data back into train and test sets
    train_processed = combined[combined['target'] != -1].reset_index(drop=True)
    test_processed = combined[combined['target'] == -1].reset_index(drop=True)
    
    # Drop the temporary target column from test data
    test_processed = test_processed.drop('target', axis=1)
    
    # Separate features and target variable from training data
    X = train_processed[features]
    y = train_processed['target']
    
    # Features for test data
    X_test = test_processed[features]
    

    # Step 3: Model Selection and Training
    
    # Since the metric is recall@15%, we'll define a custom evaluation metric
    
    def recall_at_k(y_true, y_prob, k):
        # y_true: true binary labels
        # y_prob: predicted probabilities
        # k: cutoff percentage (e.g., 0.15 for top 15%)
        threshold = np.quantile(y_prob, 1 - k)
        y_pred = np.where(y_prob >= threshold, 1, 0)
        recall = recall_score(y_true, y_pred)
        return recall
    
    # Use Stratified K-Fold cross-validation due to imbalance
    skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
    
    # Placeholder for out-of-fold predictions
    oof_preds = np.zeros(X.shape[0])
    
    # Placeholder for test predictions
    test_preds = np.zeros(X_test.shape[0])
    
    # LightGBM parameters (you can tune these parameters)
    lgb_params = {
        'learning_rate': 0.01,
        'n_estimators': 1000,
        'objective': 'binary',
        'metric': 'binary_logloss',
        'random_state': 42,
        'n_jobs': -1
    }
    
    # Training with cross-validation
    for fold, (train_idx, val_idx) in enumerate(skf.split(X, y)):
        print(f'Fold {fold + 1}')
        X_train, X_val = X.loc[train_idx], X.loc[val_idx]
        y_train, y_val = y.loc[train_idx], y.loc[val_idx]
        
        model = LGBMClassifier(**lgb_params)
        model.fit(
            X_train, y_train,
            eval_set=[(X_val, y_val)],
            early_stopping_rounds=100,
            verbose=100
        )
        
        # Predict on validation set
        val_preds = model.predict_proba(X_val)[:, 1]
        oof_preds[val_idx] = val_preds
        
        # Predict on test set
        test_fold_preds = model.predict_proba(X_test)[:, 1]
        test_preds += test_fold_preds / skf.n_splits
        
        # Evaluate recall@15% on validation set
        recall = recall_at_k(y_val, val_preds, 0.15)
        print(f'Recall@15% for fold {fold + 1}: {recall:.4f}\n')
    
    # Overall OOF Recall@15%
    overall_recall = recall_at_k(y, oof_preds, 0.15)
    print(f'Overall Recall@15%: {overall_recall:.4f}')
    

    # Step 4: Prediction
    
    # The test_preds array contains the average predictions over the folds
    # Now, we prepare the submission file
    
    # Assuming the test data has an 'id' column (replace with actual ID column name if different)
    submission = pd.DataFrame({
        'id': test['id'],  # Replace 'id' with the actual ID column name in test.csv
        'target': test_preds
    })
    
    # Ensure the submission file has the correct format
    submission.head()
    

    Note: If the test data does not have an 'id' column, you need to adjust the code accordingly. The submission file should contain:

    • An identifier for each record (if provided).
    • The predicted probabilities.

    # Step 5: Submission
    
    # Save the submission file
    submission.to_csv('submission.csv', index=False)
    
    # Verify that the submission file is saved correctly
    # You can download this file from Colab for submission
    

    Important: Ensure that the submission file meets the required format specified in the problem description.


    Optimizing for Recall@15%

    Since the evaluation metric is recall@15%, we can further improve our model by:

    • Adjusting the threshold to select the top 15% of customers with the highest predicted probabilities.
    • Experimenting with different models like XGBoost, CatBoost, or ensemble methods.
    • Performing hyperparameter tuning using tools like GridSearchCV or Optuna.
    • Using feature engineering techniques to create new features that may improve model performance.
    • Addressing class imbalance using techniques like oversampling the minority class or using class weights.

    Example of Threshold Adjustment

    # Determine the threshold corresponding to the top 15%
    threshold = np.quantile(test_preds, 0.85)
    
    # Assign binary predictions based on the threshold
    test['prediction'] = np.where(test_preds >= threshold, 1, 0)
    
    # Update the submission file accordingly (if required by the competition)
    submission = test[['id', 'prediction']]  # Replace 'id' and 'prediction' with the correct column names
    submission.to_csv('submission.csv', index=False)
    

    Remember to replace the placeholder column names with the actual column names in your dataset.


    Final Notes

    • Always ensure that your submission file strictly follows the format specified in the problem statement.
    • Since the code will run in a Google Colab environment, ensure that the necessary libraries are installed (e.g., !pip install lightgbm).
    • Keep in mind the computational limitations of the environment when choosing model complexity and dataset size.
    • Due to the custom metric, consider implementing a custom training loop or using callbacks that optimize for recall@15%.

    Installing Required Libraries in Colab

    # Install LightGBM if not already installed
    !pip install lightgbm
    

    References


    By following these steps and refining the model based on validation results, you can develop a solution that predicts the probability of customers registering a business, optimized for the recall@15% metric.