Skip to content
The Chat Archive
gpt-4.1-2025-04-14· OpenAI LMArena Human Preference 140K

Write code for document classification in most effective way

  1. User

    Write code for document classification in most effective way

  2. Assistant

    Certainly! Document classification is a common NLP task. The most effective way as of 2024 is to use transformer-based models (like BERT, RoBERTa, or DistilBERT) with transfer learning. Below is a complete, practical example using the Hugging Face transformers library and PyTorch. This code can be adapted for your dataset.


    1. Install Required Libraries

    pip install torch transformers datasets scikit-learn
    

    2. Prepare Your Data

    Assume you have a CSV file: data.csv with columns: text and label.


    3. Code: Document Classification with Transformers

    import pandas as pd
    from datasets import Dataset, ClassLabel
    from transformers import AutoTokenizer, AutoModelForSequenceClassification, TrainingArguments, Trainer
    from sklearn.model_selection import train_test_split
    import torch
    
    # 1. Load your data
    df = pd.read_csv('data.csv')
    train_df, test_df = train_test_split(df, test_size=0.2, stratify=df['label'], random_state=42)
    
    # 2. Convert to HuggingFace Dataset
    train_dataset = Dataset.from_pandas(train_df)
    test_dataset = Dataset.from_pandas(test_df)
    
    # 3. Encode labels
    labels = sorted(df['label'].unique())
    label2id = {label: i for i, label in enumerate(labels)}
    id2label = {i: label for label, i in label2id.items()}
    
    def encode_labels(example):
        example['label'] = label2id[example['label']]
        return example
    
    train_dataset = train_dataset.map(encode_labels)
    test_dataset = test_dataset.map(encode_labels)
    
    # 4. Tokenization
    model_name = "distilbert-base-uncased"  # Or "bert-base-uncased", etc.
    tokenizer = AutoTokenizer.from_pretrained(model_name)
    
    def tokenize_function(example):
        return tokenizer(
            example["text"],
            padding="max_length",
            truncation=True,
            max_length=256,
        )
    
    train_dataset = train_dataset.map(tokenize_function, batched=True)
    test_dataset = test_dataset.map(tokenize_function, batched=True)
    
    # 5. Set format for PyTorch
    train_dataset.set_format(type='torch', columns=['input_ids', 'attention_mask', 'label'])
    test_dataset.set_format(type='torch', columns=['input_ids', 'attention_mask', 'label'])
    
    # 6. Load model
    model = AutoModelForSequenceClassification.from_pretrained(
        model_name,
        num_labels=len(labels),
        id2label=id2label,
        label2id=label2id
    )
    
    # 7. Training arguments
    training_args = TrainingArguments(
        output_dir="./results",
        evaluation_strategy="epoch",
        save_strategy="epoch",
        learning_rate=2e-5,
        per_device_train_batch_size=16,
        per_device_eval_batch_size=16,
        num_train_epochs=3,
        weight_decay=0.01,
        load_best_model_at_end=True,
        metric_for_best_model="accuracy",
    )
    
    # 8. Metrics
    import numpy as np
    from sklearn.metrics import accuracy_score, f1_score
    
    def compute_metrics(eval_pred):
        logits, labels = eval_pred
        preds = np.argmax(logits, axis=1)
        return {
            "accuracy": accuracy_score(labels, preds),
            "f1": f1_score(labels, preds, average="weighted"),
        }
    
    # 9. Trainer
    trainer = Trainer(
        model=model,
        args=training_args,
        train_dataset=train_dataset,
        eval_dataset=test_dataset,
        compute_metrics=compute_metrics,
    )
    
    # 10. Train
    trainer.train()
    
    # 11. Evaluate
    results = trainer.evaluate()
    print(results)
    

    Tips for Effectiveness

    • Use a pretrained transformer (BERT, RoBERTa, etc.) for best results.
    • Fine-tune on your own data.
    • Use GPU for training (set up CUDA if available).
    • For large datasets, consider using the datasets library for efficient data handling.

    This approach is state-of-the-art for most document classification tasks as of 2024.
    If you have a specific dataset or need a different language/model, let me know!

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.