How to do linear probing in Transformers?
- User
How to do linear probing in Transformers?
- Assistant
Below is a practical, end-to-end recipe for running a linear probe on a pretrained Transformer (e.g. BERT, RoBERTa, GPT-2) with PyTorch + 🤗 Transformers + scikit-learn.
The same principles apply to vision models (ViT, CLIP, etc.); just swap the tokenizer / feature extractor.
1. What is “linear probing”?
- Freeze the backbone network (all weights are kept fixed).
- Extract the hidden representation
hfrom some layer ℓ (or several). - Train a single linear layer (or logistic-regression classifier / ridge regressor) on top of those representations to predict a target label y.
- Evaluate. Good accuracy ⇒ the information you care about is linearly decodable at that layer.
Why do this? • Inspect where linguistic / semantic information emerges in NLP models.
• Compare models / pre-training objectives.
• Check for memorization vs. abstraction.
• Debug fine-tuning failures (“do I really need a full fine-tune?”).
2. Set-up
pip install transformers datasets scikit-learn torch tqdm
3. Minimal code skeleton
import torch, torch.nn as nn from transformers import AutoTokenizer, AutoModel from datasets import load_dataset from sklearn.linear_model import LogisticRegression from sklearn.metrics import accuracy_score from tqdm.auto import tqdm MODEL_NAME = "bert-base-uncased" LAYER_TO_PROBE = 8 # 0-11 for BERT-base BATCH_SIZE = 32 MAX_LEN = 128 # or shorter/longer if GPU allows # 1. Load model & tokenizer (frozen) tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME) model = AutoModel.from_pretrained(MODEL_NAME, output_hidden_states=True) model.eval() # disable dropout for p in model.parameters(): p.requires_grad = False # freeze # 2. Load / prepare your probing dataset # Example: "CoLA" grammatical acceptability classification (binary) dataset = load_dataset("glue", "cola") train, test = dataset["train"], dataset["validation"] def encode(batch): return tokenizer(batch["sentence"], padding="max_length", truncation=True, max_length=MAX_LEN) train = train.map(encode, batched=True) test = test .map(encode, batched=True) cols = ["input_ids", "attention_mask", "label"] train.set_format(type="torch", columns=cols) test .set_format(type="torch", columns=cols) # 3. Extract representations def get_representations(split): reps, labels = [], [] loader = torch.utils.data.DataLoader(split, batch_size=BATCH_SIZE, shuffle=False) with torch.no_grad(): for batch in tqdm(loader): out = model(input_ids=batch["input_ids"], attention_mask=batch["attention_mask"]) # out.hidden_states: tuple(layer0, layer1, …, layer12), each: [B, T, D] h = out.hidden_states[LAYER_TO_PROBE] # choose layer ℓ cls = h[:, 0, :] # use [CLS] token reps.append(cls.cpu()) labels.append(batch["label"].cpu()) return torch.vstack(reps).numpy(), torch.cat(labels).numpy() X_train, y_train = get_representations(train) X_test , y_test = get_representations(test) print("Rep shape:", X_train.shape) # (N, hidden_size) # 4. Fit linear probe (LogReg with L2 acts like a single-layer classifier) probe = LogisticRegression(max_iter=1000, solver="lbfgs", C=1.0) probe.fit(X_train, y_train) # 5. Evaluate pred = probe.predict(X_test) acc = accuracy_score(y_test, pred) print(f"Layer {LAYER_TO_PROBE} accuracy: {acc:.3f}")
4. Tips, variants & best practices
-
Which token(s) to use
• For sentence tasks with BERT-like models,[CLS]is standard.
• Mean-pool or attention-weighted pool often works too.
• For GPT-style (no CLS) you can use the last token or mean-pool. -
Multiple layers at once
– Concatenate several layers (torch.cat([h3, h7, h11], dim=-1)).
– Or train a separate probe per layer and plot accuracy vs. depth. -
Controlling capacity
– Use ridge regression (sklearn.linear_model.RidgeClassifier) or a low C in LogisticRegression to keep the probe linear (avoid overfitting and leaking non-linear power).
– Report random orientation baseline: shuffle labels, or use random features of the same dimensionality. -
Cross-validation
– Do k-fold or dev-set tuning for hyper-parameters (C, penalty, class weight).
– Use early stopping for large datasets (LogisticRegressionhasn_jobs=-1,max_iter). -
Statistical significance
– Bootstrapping confidence intervals.
– Run probes with multiple random seeds and report mean ± sd. -
Memory-friendly extraction
– If GPU RAM is tight: send model to GPU but keep representations on CPU (reps.append(cls.cpu())).
– Or write to disk as.npy/ HF arrow file and fit the probe offline. -
Vision & audio models
– Same idea; just grab the patch embeddings, average pool, etc. -
Libraries that automate probes
–bertviz,probing(John Hewitt),dataclasses-jsonframework.
– Adapter-Transformers has a “prediction head” option that can be frozen to behave like a probe.
5. Interpreting results
• Accuracy climbs → that information is encoded earlier than you thought.
• Accuracy dips in middle layers → suggests mid-layers are re-packing information.
• Higher performance ≠ causal importance. Combine with ablation or causal tracing techniques for stronger claims.
🎉 That’s it! You now have a linear-probe pipeline to peek inside Transformer representations. Tweak the dataset, layer choice, pooling, and probe regularization to suit your research questions.
- User
Why use logistic regression or train the probe with gradient descent? Why not use an SVM?
- Assistant
Short answer ― you absolutely can use a linear SVM; it will give almost the same geometric test (“is the information linearly separable?”). In practice, most probing papers gravitate to logistic-regression / ridge because it is:
- Easier to train on millions of examples or 1000-dimensional hidden states.
- Naturally multi-class and gives probabilities.
- Smoothly differentiable, so you can optimise it with any off-the-shelf GPU SGD/Adam routine.
- Already built into every deep-learning library, which means one less dependency / solver to wrangle.
Below is a more detailed comparison.
- Computational scalability (𝑁 = #examples, 𝐷 = hidden size)
Model Common solver Complexity (rough) GPU-friendly? Logistic/Ridge LBFGS / (mini-batch) SGD O(N·D) per epoch Yes (pure matmul) Linear SVM (dual) LibLinear / SMO QP O(N² – N³) worst‐case No (CPU QP) Linear SVM (primal SGD) Pegasos / Hinge + SGD O(N·D) per epoch Yes, but needs custom loop If you have 500 k examples * 768-d vectors, a quadratic-time dual SVM may choke, while logistic regression trains in seconds with a single CUDA kernel.
- Smoothness of the loss
• Logistic loss is smooth, so optimisers (LBFGS, Adam) get fast, steady convergence.
• Hinge loss is piece-wise linear (non-differentiable at the margin). SGD works, but needs tricks (learning-rate decay, careful regularisation) to avoid oscillation.
- Probabilities & evaluation
Probing papers often want: • class probabilities → entropy curves, AUC, calibration plots
• multi-class tasks (POS tagging: 45 tags; sentiment: 5 levels)Softmax logistic regression supports this out-of-the-box.
SVM needs one-vs-rest or Platt-scaling for probabilities, adding extra code and potential instability.
- Regularisation knobs
Both models reduce to “linear weights + an ℓ2 penalty parameter C/λ”.
Logistic regression’s single C hyper-parameter is identical in spirit to SVM’s, so switching rarely changes the story.
Because results scarcely differ, authors pick the simpler tool.
- Interpretability
With a probabilistic model you can interpret the weight vector as log-odds contributions, plot odds ratios layer-by-layer, etc.
Hinge-loss weights have no probability semantics.
- Historical inertia
Early NLP probing papers (Adi 2017, Hewitt & Manning 2019, Tenney 2019) used ridge / logistic. Subsequent work kept the convention for comparability.
When might an SVM be preferable?
• Very small datasets (few hundred points) where the max-margin property often generalises a bit better.
• You explicitly care about support‐vector sparsity (e.g. counting #SVs as a capacity measure).
• You’re probing binary tasks only and want deterministic hard margins.If that’s your case, by all means run a linear SVM:
from sklearn.svm import LinearSVC probe = LinearSVC(C=1.0) # hinge loss, no probs probe.fit(X_train, y_train) acc = probe.score(X_test, y_test)Or, for probabilities:
from sklearn.svm import LinearSVC from sklearn.calibration import CalibratedClassifierCV svc = LinearSVC(C=1.0) probe = CalibratedClassifierCV(svc) # Platt scaling probe.fit(X_train, y_train)Bottom line
• Logistic / ridge and linear SVM are functionally equivalent probes; both test linear separability.
• Logistic regression is smoother, GPU-friendly, multi-class, probabilistic, and already tied into deep-learning stacks, so it wins the convenience race.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.