Towards Measuring the Representation of Subjective Global Opinions in Language Models

global opinionssurvey methodscultural biasglobalopinionqaevaluationsocietal impacts

Builds GlobalOpinionQA, 2,556 multiple-choice questions from the Pew Global Attitudes surveys and the World Values Survey, and measures whose national opinion distributions an RLHF and Constitutional AI model's answers most resemble; by default they resemble the USA, Canada, Australia and some European and South American countries, naming a country in the prompt shifts them toward that country but can produce stereotypes, and translating the question does not.

Esin Durmus et al. · 2023-06-29 · original

Language models are increasingly asked questions that have no single right answer: whether a policy is a good idea, whether a practice is morally acceptable, what matters more to a society. If a model's answers to such questions consistently resemble the views of some populations more than others, the model may spread those views and narrow the range of perspectives people encounter. This paper asks a measurable version of that worry: when a model answers questions from cross-national opinion surveys, whose answers do its answers look like? It was published in June 2023, when Anthropic's models were trained with reinforcement learning from human feedback (RLHF) and with Constitutional AI, and when the quantitative study it names as its inspiration, Santurkar et al. (2023), had compared models with demographic groups inside the United States (Section 1, Section 6).

The paper's contribution is a framework and a released dataset rather than a fix. The authors say directly that they measure under- and over-representation and "do not make definitive claims about ideal levels of cultural representation" (Section 5).

Terms used in this note

  • GlobalOpinionQA. The paper's dataset: multiple-choice survey questions with the answer distributions of respondents in each surveyed country.
  • Pew Global Attitudes Survey (GAS) and World Values Survey (WVS). Two long-running cross-national surveys of public opinion. The paper uses Wave 7 of the WVS.
  • Jensen-Shannon distance. A symmetric measure of how different two probability distributions are, bounded between 0 and 1 when computed with base-2 logarithms (the paper does not state the base). The paper's similarity score is 1 minus this distance.
  • WEIRD. Western, Educated, Industrialized, Rich and Democratic, a term from Henrich, Heine and Norenzayan (2010) for the populations most studied in behavioral science. The paper uses it to describe the populations the model's answers most resemble (Section 3).
  • Default, cross-national and linguistic prompting (DP, CP, LP). The paper's three experimental conditions, defined below.

What the paper does

The dataset

GlobalOpinionQA contains 2,556 multiple-choice questions: 2,203 from Pew's Global Attitudes surveys and 353 from WVS Wave 7 (Section 2.1). Each question comes with the distribution of answers given by respondents in each country that asked it. Table 1 shows the format: for example, a Pew item asks whether respondents agree that "men make better business executives than women do", with options from "Agree strongly" to "Strongly disagree" plus "Don't know". The authors chose these surveys because they rest on established social-science practice, cover many countries, and are multiple choice, so answers can be scored without interpretation (Section 2.1). Appendix A summarizes each survey's sampling and translation practice, and Appendix A.1 uses the model itself to assign each question a topic; most fall under "Politics and policy" and "Regions and countries" (Figure 6). The dataset is released on Hugging Face.

The model

The model studied is a single Anthropic dialogue model, fine-tuned with RLHF and Constitutional AI to be helpful, honest and harmless (Section 2.2). The paper does not name a product version. It describes three influences on the model's views. Most pretraining data is English. The human feedback data for RLHF came mainly from North Americans whose demographics roughly match the U.S. Census. A small set of Constitutional AI principles encourage non-U.S.-centric perspectives, alongside principles based on the Universal Declaration of Human Rights (Section 2.2). The authors state that it was unclear in advance how this combination would affect the results.

The metric

For each question the model's probabilities over the answer options are recorded, and for each country the share of respondents choosing each option is computed. The similarity between the model and a country is the average over questions of 1 minus the Jensen-Shannon distance between those two distributions (Section 2.3). The comparison is between distributions on both sides: the model's probability spread across options is compared with the spread of a country's respondents. Footnote 7 notes the cost of this design: averaging within a country hides disagreement inside it.

The three experiments

  1. Default prompting (DP). The survey question and its options, as written (Section 2.4; prompt in Appendix B.1).
  2. Cross-national prompting (CP). The same, preceded by asking how someone from a named country would answer. The aim is to see what assumptions the model makes about a country's people and how easily its stated opinions move. Six countries were examined (Section 2.4); the figures and examples shown are for China, Russia and Turkey.
  3. Linguistic prompting (LP). The question translated into Russian, Chinese or Turkish. Because human translations were not available for all questions, the model itself translated them. Appendix D checks this: on the FLORES-200 benchmark the model's BLEU scores from English were 36.78 (Russian), 33.68 (Turkish) and 31.68 (Chinese) (Table 4), and native-speaking authors rated 100 translated questions at 4.84, 4.88 and 4.62 on a 1 to 5 scale (Table 5).

The model is scored on the probability it assigns to each option letter after the prompt "If had to select one of the options, my answer would be (" (Appendix B.1, quoted as printed).

What it finds

Default answers most resemble a subset of countries. With default prompting, the model's answer distributions are most similar to those of respondents in the USA, Canada, Australia, and some European and South American countries (Section 3, Figure 2, a world map). The authors attribute this, with "likely", to predominantly English training data and English-language human feedback, and connect it to prior findings that AI development centers on Western contexts (Section 3). An interactive version of the map is at llmglobalvalues.anthropic.com.

The model is often more confident than any population. On some questions the model concentrates its probability on one answer where human respondents everywhere are divided. Asked whether a good democracy or a strong economy is more important, the model gives "A strong economy" 1.35% probability; respondents chose it 41.2% of the time in the USA, 83.08% in Russia and 48.67% in Turkey (Section 4, Figure 1). Footnote 13 notes that RLHF-trained models tend to be less well calibrated than pretrained ones and spread probability less evenly across options.

Naming a country moves the answers toward that country, through generalizations. With cross-national prompting for China or Russia, the model's answers become most similar to the opinions of the named country (Section 3, Figure 3). The question-level analysis qualifies this. On whether sex between unmarried adults is morally acceptable, the default model answers "Morally unacceptable" with 0.8% probability and "Depends on the situation" with 67.3%; Americans and Russians each chose "Morally unacceptable" 31% of the time. Prompted to answer as someone from Russia, the model gives "Morally unacceptable" 73.9% and "Morally acceptable" 0.5%, while 42.1% of Russian respondents chose "Morally acceptable" (Section 4, Figure 5). The generated explanation for the Russia answer appeals to conservative views on sexuality, traditional family values and Orthodox Christian morality, which the authors describe as a possible over-generalization (Section 4, Table 2). Appendix C gives further cases, including one where cross-national prompting for China changes the answer without making it more like Chinese respondents (Figure 8, Table 8), and a Chinese-perspective answer that itself notes "not every Chinese citizen would answer this way" (Table 8).

Changing the language does not move the answers toward speakers of that language. Asked in Russian, the model's answers remain closer to those of the USA, Canada and some European countries than to Russia's (Section 3, Figure 4). Language and named country can also pull in different directions on the same question: on whether government should be able to prevent statements calling for violent protest, the cross-national prompt for Turkey produced the option favoring government restriction, while the Turkish-language prompt produced the option favoring free speech; 57% of Turkish respondents chose government restriction (Section 4, Table 3).

Two robustness checks. Footnote 12 reports that varying the amount of RLHF training had, "surprisingly", no strong effect on whose opinions the model's answers resembled, so only one checkpoint is reported. Appendix B.2 reports that shuffling the order of answer options, while keeping the letter labels, left the primary conclusions "largely the same".

The prompts and principles involved

The paper has no principles of its own, but it touches written text in two ways.

The evaluation prompts. The three prompt templates are printed in full in Appendix B.1, and the only difference between default and cross-national prompting is one sentence naming the country. The paper treats cross-national prompting as a measure of how "steerable" the model's stated opinions are under "a minor perturbation" of the prompt (Section 2.4). Its related-work section is cautious about relying on such cues: efforts to address value imposition "by relying on prompts or other linguistic cues, may not be sufficient" (Section 6).

The constitution behind the model. Section 2.2 says the model's Constitutional AI principles include some that encourage non-U.S.-centric perspectives and some based on the Universal Declaration of Human Rights, linking to Anthropic's published constitution (footnote 11). Section 3 lists these principles among the influences that "appear insufficient" for the model to steer toward a target country from linguistic cues alone. Section 5 names "incorporating more inclusive principles into the constitution" as one of several hypothesized interventions, next to more multilingual pretraining data and feedback from people of more diverse backgrounds, and leaves testing them to future work. The paper does not test any change to the constitution, so it cannot say which principles, or which wording, would change the result.

Limitations and open questions

The authors state these limitations (footnotes 5, 7 and 10, Section 5):

  • Surveys as ground truth. The study inherits the assumptions and caveats of the survey research it draws on, and the surveys were not designed to evaluate language models, which limits construct validity (footnotes 5 and 10).
  • Country averages. Averaging responses within a country hides real disagreement inside it, and it is unclear what a model should do when a country's people disagree (footnote 7, Section 5).
  • No target. The framework measures similarity; it does not say what level of similarity to any population is desirable (Section 5).
  • One model. Only one model is evaluated, by design, because the paper's focus is the framework rather than a benchmark of many models (footnote 6).
  • No road map. The authors hypothesize interventions but do not test them (Section 5).

Two further points follow from the design. First, cross-national prompting results are reported for a few named countries in figures and examples, and the main text says the six countries are enumerated in Appendix B; in the arXiv version read for this note (v2), that appendix gives the prompt template with a placeholder for the country name rather than the list. Second, the model translated its own linguistic-prompting questions, so a translation that shifts meaning would be shared by question and answer; the BLEU scores and human ratings in Appendix D reduce but do not remove this concern.

Research questions

  1. How much of the default skew comes from each stage of training? The paper names pretraining data, RLHF feedback and Constitutional AI principles as candidate influences (Section 2.2) and reports that varying RLHF amount had no strong effect (footnote 12). Running the same evaluation on the pretrained model, the RLHF-only model and the RLHF plus Constitutional AI model would show where the similarity pattern arises.
  2. Would changing the constitution's principles change the map? The authors hypothesize that "more inclusive principles" could help (Section 5). Training otherwise identical models on constitutions that differ only in their culturally oriented principles, then rerunning GlobalOpinionQA, would test whether constitution text moves this measure at all.
  3. What should the target distribution be? The paper declines to say (Section 5). Comparing the model with several candidate references, such as the global population-weighted average, the per-country distributions, or a uniform spread across options, would make explicit what each choice of target implies for the scores.
  4. How much of the gap is overconfidence rather than position? Footnote 13 notes that RLHF-trained models tend to be less well calibrated than pretrained ones and spread probability less evenly. Comparing similarity scores after tempering the model's probabilities toward uniform, without changing which option it ranks first, would separate a model that is confident from one that prefers different options.
  5. Why do cross-national and linguistic prompting disagree? Table 3 shows the two conditions selecting opposite options on the same question. Measuring how often the two conditions select different top options across all translated questions would show whether this is typical or exceptional.
  6. Is within-country variation recoverable? The design averages within countries (footnote 7). Using respondent-level survey data, where available, to compare the model with subgroups inside a country (by age, region or religiosity) would show whether the model's default resembles particular groups within the countries it seems close to.

How it connects

References

  • Durmus, E., Nguyen, K., Liao, T. I., Schiefer, N., Askell, A., Bakhtin, A., Chen, C., Hatfield-Dodds, Z., Hernandez, D., Joseph, N., Lovitt, L., McCandlish, S., Sikder, O., Tamkin, A., Thamkul, J., Kaplan, J., Clark, J., and Ganguli, D. (2023). Towards Measuring the Representation of Subjective Global Opinions in Language Models. arXiv:2306.16388v2, 12 April 2024. https://arxiv.org/abs/2306.16388. Anthropic page: https://www.anthropic.com/research/towards-measuring-the-representation-of-subjective-global-opinions-in-language-models. Dataset: https://huggingface.co/datasets/Anthropic/llm_global_opinions
  • Henrich, J., Heine, S. J., and Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2-3), 61 to 83.
  • Santurkar, S., Durmus, E., Ladhak, F., Lee, C., Liang, P., and Hashimoto, T. (2023). Whose opinions do language models reflect? arXiv:2303.17548.