Evaluating feature steering: A case study in mitigating social biases

In May 2024 Anthropic's interpretability team reported that dictionary learning could extract millions of interpretable "features" from Claude 3 Sonnet, internal directions that correspond to concepts such as a famous person, a type of code or the Golden Gate Bridge, and that turning a feature up or down changed the model's output in matching ways. Those demonstrations were qualitative. This post from the Societal Impacts team asks whether feature steering works as a quantitative tool for one practical goal, reducing social bias: does it reliably move scores on standard evaluations, does it damage the model's other abilities, and can a feature's effect be predicted from the contexts in which it activates (post, introduction)? The authors describe their results as "mixed" and preliminary.

Terms used in this note

• Feature. A direction in the model's residual stream (the internal vector passed between layers) learned by a sparse autoencoder (SAE), a network trained to rewrite the model's activations as a sparse combination of many such directions. The features here come from a 34M-parameter SAE (post, "Future work"). Each feature was labeled automatically by describing the text on which it activates.
• Feature steering. Adding a constant multiple of a feature's direction to the residual stream while the model runs, so the model behaves as if that feature were more (or less) active. The post refers to the earlier paper for the exact implementation.
• Steering factor. The size of that constant, positive to amplify and negative to suppress. The post varies it between -20 and 20.
• Sweet spot. The range of steering factors within which capability benchmarks are not significantly affected.
• Off-target effect. A change in an evaluation that the feature was not expected to influence, given the contexts in which it activates.
• BBQ. The Bias Benchmark for QA, multiple-choice questions measuring reliance on social stereotypes across nine categories: age, disability status, gender identity, nationality, physical appearance, race and ethnicity, religion, socioeconomic status, and sexual orientation.

What the study does

Features. The authors chose 29 features related to social biases and political ideologies from the set learned from Claude 3 Sonnet (post, "Methods"). Appendix 2 (Table A2) lists them with their automated descriptions, grouped into categories such as age, gender and sexuality, discrimination, neutrality and objectivity, political neutrality, balanced views, political ideologies, specific issues (one feature for "Pro-life, anti-abortion" and one for "immigrants or foreigners taking away jobs from native workers"), agreement and empathy, safety, and ethnicity and culture.

Evaluations. Two capability benchmarks, MMLU (multiple-choice questions across academic subjects) and PubMedQA (biomedical questions), measure general ability. Two bias evaluations cover 11 types of bias: BBQ's nine categories and a subset of Anthropic's model-written evaluations dataset with subjective multiple-choice questions about abortion and immigration (post, "Methods"). Every evaluation was run with every feature.

Scoring. For each multiple-choice question the model was sampled 10 times and answer probabilities were estimated from the samples. The authors note this adds noise, visible as fluctuations in the plots near a steering factor of 0, and chose it because larger samples were computationally prohibitive (post, "Methods").

What it finds

A shared sweet spot. MMLU accuracy is highest for steering factors between -5 and 5 and falls more steeply beyond that range (Figure 1, across 29 features and nine steering factors); PubMedQA shows a similar pattern (Appendix 3.1, Figure A1). The authors call it surprising that the same range holds for all features. Outside it, generations degrade: in Appendix 1 (Table A1), steering a "Gender bias awareness" feature to 5 produces a completion about gender in nursing, and at 12 an off-topic reply about men in elementary teaching. All later results are reported within the -5 to 5 range.

On-target effects, sometimes in an unexpected direction. Positively steering the "Multiple perspectives and balance" feature to 5 lowers the overall BBQ bias score by about 3% (Figure A2), with decreases of 17% in age bias and 7% in disability status bias between factors 0 and 5 (post, "Measuring social biases with BBQ"). Steering the "Gender bias awareness" feature, which activates on discussions of gender bias, raises the gender identity bias score by 10% from -5 to 5 (Figure 2, left). The authors suggest the steered model may dwell on gender in a way the metric scores as biased, and caution that automated labels describe where a feature activates, not which direction it pushes behavior.

Off-target effects. The same "Gender bias awareness" feature also raises the age bias score by 13% (Figure 2, right), although age is not part of its label and it does not necessarily activate in age-related contexts. Effects for all features are in Appendices 3.2 and 3.3.

Political questions. On the abortion questions, steering the "Pro-life and anti-abortion stance" feature increased anti-abortion selections by 50%, and the "Left-wing political ideologies" feature decreased them by 47%; the "Political neutrality and independence" feature raised them from 32% to 50%, which the post notes indicates neutrality on the issue (Figure 3). On the immigration questions, the left-wing feature reduced anti-immigration selections from 25% to near 0%, a "discrimination awareness" feature reduced them by 25%, and the political neutrality feature increased them by 24% (Figure 4). The pro-life feature increased anti-immigration selections by 21.60% from -5 to 5, more than the immigration-specific feature's 3.90% change (post, "Measuring political biases"). The authors read this as a sign of underlying correlations between the concepts the features represent.

Two features reduce bias broadly. Positively steering the "Neutrality and Impartiality" and "Multiple Perspectives" features lowered BBQ bias scores in all nine categories within the sweet spot, most for age, disability status and physical appearance (Figure 5). The "Neutrality and Impartiality" feature may slightly lower BBQ accuracy, while "Multiple Perspectives" keeps accuracy steady across the range (post, "The 'Neutrality' and 'Multiple Perspectives' features"; Appendix 3.2, Figure A4).

Prompting compared with steering

Appendix 4 compares steering with written prompts, which makes this the part of the study most directly about wording. For each topic, five prompts were written that place the model on a scale from strongly supportive to strongly opposed (Tables A7 and A8). Each is a short second-person persona statement with an instruction on how to respond; for example, the strongest immigration prompt begins "You strongly oppose immigration." and the balanced one asks the model to "weigh both the positive and negative aspects of immigration".

The measured selection rates (Appendix 4):

Three patterns appear in these numbers. Prompts move the targeted measure over a wide range, from 12.8% to 83.0% anti-abortion selections. The strongest wording is not always the most extreme in effect: the "generally anti-abortion" prompt produced more anti-abortion selections than the "strongly anti-abortion" one, and "generally pro-immigration" fewer anti-immigration selections than "strongly pro-immigration". And prompts on one topic change answers on the other, which the appendix calls cross-domain effects. The appendix concludes that both methods influence selections, that neither is always predictable, and that their comparative effectiveness "is not clear-cut and likely depends on specific use cases"; the main post states that prompting showed sensitivity and specificity "comparable to feature steering" in these limited experiments (post, "Future work", item 4).

Limitations and open questions

The authors list these (post, "Limitations", "Lessons learned" and "Future work"):

• Static multiple-choice evaluations capture narrow behavior; human-judged comparisons (Elo scores) would be broader.
• Small coverage. 29 features "out of millions" and a handful of evaluations.
• Noisy scoring from 10 samples per question; log probabilities would be more precise but needed engineering effort.
• Activation context is not behavior. Features were chosen by where they activate, and there is "no inherent reason" this should predict their effect when steered.
• Not yet reliable. Steering can change outputs in unrelated domains and degrade quality at extreme values.
• Untested factors. Larger SAEs, steering only the assistant's response rather than the whole prompt, comparisons with activation steering, alternative steering methods (multiplicative, projective, conditional), and the circuits that connect features.

The post's practical conclusion is that "careful evaluations would be necessary before deploying feature steered models in practice" (post, "Conclusion").

Research questions

1. Does steering only the response tokens reduce off-target effects? The authors note that steering the whole prompt may confound results (Future work, item 2). Rerunning the gender and political experiments with response-only steering would test it.
2. Do larger SAEs give more targeted features? The authors expect this intuitively (Future work, item 1). Repeating the evaluation with features for the same concepts from SAEs of several sizes would show whether off-target effects shrink.
3. Why did "Gender bias awareness" increase measured bias? The authors suggest the model overemphasized gender. Reading the steered answers to BBQ items and classifying them as stereotyped, counter-stereotyped or off-topic would show what the score is picking up.
4. Why is the strongest prompt not the most extreme? In Appendix 4, "strongly anti-abortion" produced fewer anti-abortion selections than "generally anti-abortion". Inspecting refusals and hedged answers under each prompt would show whether stronger persona statements trigger caution.
5. Do the neutrality features help where prompts also help? The two features lowered bias in all nine BBQ categories. Comparing them on the same items with a written instruction to be neutral or balanced, as in The Capacity for Moral Self-Correction in Large Language Models, would show whether steering adds anything a prompt does not.
6. Are cross-domain effects shared by prompts and features? Both the pro-life feature and the abortion prompts changed anti-immigration answers. Measuring whether the same questions shift under both methods would indicate whether the effect comes from correlated concepts in the model rather than from either method.

How it connects

• Source of the features. The features come from the May 2024 dictionary-learning work on Claude 3 Sonnet, published as Scaling Monosemanticity (Templeton et al., 2024), which the post links and which Anthropic announced as Mapping the Mind of a Large Language Model.
• Bias measured with BBQ. The post links the BBQ evaluation to The Capacity for Moral Self-Correction in Large Language Models, which reduced BBQ bias with one-sentence instructions, and Collective Constitutional AI: Aligning a Language Model with Public Input reports a BBQ comparison between two constitutions. The three studies change different things (an instruction, a constitution, an internal activation) and measure the same benchmark.
• Model-written evaluations. The abortion and immigration questions come from the dataset released with Discovering Language Model Behaviors with Model-Written Evaluations, which the post links.
• Prompt-based mitigation. Evaluating and Mitigating Discrimination in Language Model Decisions tests written interventions against discrimination in decisions; this post's Appendix 4 compares written persona prompts with steering.

References

• Durmus, E., Tamkin, A., Clark, J., Wei, J., Marcus, J., Batson, J., Handa, K., Lovitt, L., Tong, M., McCain, M., Rausch, O., Huang, S., Bowman, S., Ritchie, S., Henighan, T., and Ganguli, D. (2024, October 25). Evaluating feature steering: A case study in mitigating social biases. Anthropic. https://www.anthropic.com/research/evaluating-feature-steering
• Durmus, E., et al. (2024). Appendix to "Evaluating Feature Steering: A Case Study in Mitigating Social Biases" [PDF]. https://assets.anthropic.com/m/6a464113e31f55d5/original/Appendix-to-Evaluating-Feature-Steering-A-Case-Study-in-Mitigating-Social-Biases.pdf
• Templeton, A., Conerly, T., Marcus, J., et al. (2024). Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet. Transformer Circuits Thread. https://transformer-circuits.pub/2024/scaling-monosemanticity/index.html
• Parrish, A., Chen, A., Nangia, N., et al. (2022). BBQ: A hand-built bias benchmark for question answering. In Findings of the Association for Computational Linguistics: ACL 2022. arXiv:2110.08193.