Language Models as Knowledge Bases?

Petroni and colleagues' EMNLP 2019 paper introduces the LAMA probe, which asks a pretrained language model to fill the masked object of a fact written as a cloze sentence and scores the top-ranked token (precision at one); BERT-large reached 32.3 on 41 Wikidata relations against 33.8 for a relation-extraction knowledge base built with an oracle entity linker, and later work found evidence that part of this comes from guessable entity names (LAMA-UHN) and that the hand-written templates understate what can be recovered.

Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H. Miller, Sebastian Riedel (Facebook AI Research; University College London) · 2019-09-03 · original

Can the facts a language model picks up in pretraining be read back out of it, without fine-tuning, the way a query reads a fact out of a knowledge base? This paper answers with a probe, LAMA (LAnguage Model Analysis), that turns each fact into a fill-in-the-blank sentence and checks whether the model ranks the missing word first. On its largest factual source, 41 Wikidata relations, BERT-large ranked the correct object first for 32.3% of facts (mean over relations), close to the 33.8% of a knowledge base built by a supervised relation extractor that was handed the sentences expressing each fact and an oracle entity linker (Table 2). The paper's own reading of what that score means is cautious, and two follow-ups qualify it from opposite sides: one removes facts that can be guessed from an entity's name, the other shows that better wording recovers more.

Version of record. The paper appeared in the Proceedings of EMNLP-IJCNLP 2019 (Hong Kong, 3 to 7 November 2019), pages 2463 to 2473, DOI 10.18653/v1/D19-1250. It was first posted to arXiv (1909.01066) on 3 September 2019, the date given above; arXiv v2 followed on 4 September, marked "accepted at EMNLP 2019". This note follows the proceedings PDF. The author order is the PDF's; the ACL Anthology's metadata lists Riedel third. Code and data were released as the LAMA repository on GitHub (named in the abstract).

Terms used in this note

  • Fact. A (subject, relation, object) triple such as (Dante, born-in, Florence), or, for one source, a question-answer pair.
  • Cloze statement. The fact written as a sentence with the object masked, for example "Dante was born in [Mask]". The paper's definition of knowing a fact: a model "knows" it if it "can successfully predict masked objects in cloze sentences" expressing it (Section 1). This is an operational definition, and the note uses "knows" only in that sense.
  • Template. The hand-written sentence pattern for a relation, such as "[S] was born in [O]" (Section 4.1.1).
  • P@k (precision at k). For one fact, 1 if the correct object is among the model's top k tokens and 0 otherwise, averaged within each relation and then across relations (Section 4.4). With one correct answer per query this is what information retrieval calls a hit or success rate at k, not precision in the usual sense; Poerner et al. (2020, Section 4.3) rename it Hits@k for that reason.

What the probe does

What is stored. Nothing is added. The knowledge under test is whatever sits in the weights of six pretrained, case-sensitive models, used "as is" (Section 6): fairseq-fconv and Transformer-XL (left-to-right, trained on WikiText-103), ELMo and ELMo 5.5B (bidirectional LSTMs), and BERT-base and BERT-large (masked language models trained on English Wikipedia and BookCorpus) (Table 1).

How it is retrieved. One forward pass per fact. The model reads the cloze statement and produces a distribution over the masked position: BERT predicts the masked token directly; the left-to-right models use the output just before the blank; ELMo averages its forward and backward predictions (Section 4.2). Candidates are restricted to a common vocabulary of about 21K case-sensitive tokens shared by all six models, so that a model with a larger vocabulary is not penalized (Section 4.5).

What enters the context. Only the cloze sentence: the subject's name, the template's words and the mask. There is no retrieved passage and no example. For the knowledge-base baseline the context is different in kind: a relation extractor is given the Wikipedia sentences known to express the test facts and builds a graph of triples, which is then queried by subject and relation (Section 4.3).

Design choices that bound the result (Section 4.5). Only single-token objects are tested, because multi-token decoding would add tunable parameters that "obscure the knowledge we are trying to measure". Only object slots are queried. One template per relation is written by hand, and the authors note that other templates did better or worse for some relations; they argue that the score is therefore "a lower bound for what language models know", by analogy with a knowledge base queried by its one relation identifier. When a subject has several valid objects (N-to-M relations), the other valid objects are removed from the candidates before ranking, following the filtered setting of Bordes et al. (2013) for knowledge-base completion.

The data

Four knowledge sources make up the probe (Section 4.1; counts from Table 2):

Source Relations Facts Template Text known to express the fact
Google-RE 3 (place of birth, date of birth, place of death) 5,527 hand-written per relation yes, manually aligned Wikipedia snippets
T-REx 41 Wikidata relations, at most 1,000 facts each 34,039 hand-written per relation yes, automatically aligned (97.8% accuracy reported by Elsahar et al., 2018)
ConceptNet 16 commonsense relations 11,458 the Open Mind Common Sense sentence containing subject and object no
SQuAD questions 305 each question rewritten by hand as a cloze yes, by construction of SQuAD

The SQuAD subset holds "context-insensitive" development-set questions with single-token answers, for example "Who developed the theory of relativity?" rewritten as "The theory of relativity was developed by ___" (Section 4.1.4).

What it finds

All numbers are mean P@1 in percent from Table 2 unless stated. No variance is reported; each number is one deterministic run of a fixed model on a fixed set.

  • Google-RE. BERT-base 9.8 and BERT-large 10.5, against 7.6 for the relation extractor with oracle entity linking (REo), 1.2 with naive string-match linking (REn), and 4.4 for a frequency baseline that always ranks the relation's most common objects. Date of birth was near zero for every system (BERT-large 1.4).
  • T-REx, all 41 relations. BERT-large 32.3, BERT-base 31.1, REo 33.8, REn 6.1, frequency baseline 22.03. The other language models scored 4.7 to 18.3. The paper describes BERT as "close to" the oracle-linked knowledge base here; it is 1.5 points below it (our arithmetic). By relation type BERT-large scored 74.5 on the 2 one-to-one relations, 34.2 on the 23 many-to-one relations and 24.3 on the 16 many-to-many relations.
  • Beyond the top rank. On T-REx the correct object was in BERT's top 10 for "around 60%" of facts and its top 100 for 80% (stated in Section 5; Figure 2 plots the curves for all six models).
  • ConceptNet. BERT-large 19.2, BERT-base 15.6, frequency baseline 4.8.
  • SQuAD as cloze questions. BERT-large 17.4 against 37.5 for DrQA, a retrieve-then-read system (TF-IDF retrieval over Wikipedia and a supervised reader, constrained to single-token answers). At P@10 the gap was 57.1 against 63.5 (Section 5).
  • What predicts a hit. For BERT-large on T-REx, P@1 correlated with the log probability of the top prediction (Pearson 0.42), with the cosine similarity of subject and object word vectors (0.31) and with how often the object is mentioned in the training corpus (0.2), but not with how often the subject is (−0.05) (Figure 3, values printed in the cells).
  • Sensitivity to wording. For up to 100 facts per T-REx relation, the object was masked in 10 different aligned Wikipedia sentences; BERT and ELMo 5.5B varied least in the rank they gave the correct object (Figure 4). The caption adds that these models are also "the most likely to have seen the query sentence during training", since the sentences come from Wikipedia.
  • More data for the extractor. Adding WikiText-103 to the relation extractor's input gave "no significant change" (Section 6; no numbers given).

The authors' conclusion is that BERT-large recalls factual knowledge "at a level remarkably competitive" with the supervised alternatives, and that language models "might become a viable alternative to traditional knowledge bases extracted from text in the future" (Section 6). Their caveat sits in the results: for Google-RE, BERT-large doing better "does not mean it does so for the right reasons", since it "could have learned associations of objects with subjects from co-occurrence patterns" (Section 5).

The prompts used

The probe's instruments are its templates, and the paper's results depend on their wording in a way the paper itself flags. One template per relation was written by the authors; Table 3 shows filled examples such as "English bulldog is a subclass of ", "The native language of Mammootty is " and "Joe Cocker is represented by music label ___". The ConceptNet templates are Open Mind Common Sense sentences with the object masked, and the SQuAD clozes were rewritten by hand. No template was tested against an alternative in the paper's main results; Figure 4's ten sentences per fact are natural Wikipedia sentences, not alternative templates.

Later work that qualifies the result

Two follow-ups test the probe directly. This note read both in full.

Guessable names: LAMA-UHN (Poerner, Waltinger and Schütze, Findings of EMNLP 2020). The authors observed that BERT predicts native languages plausible for a person's name whatever the facts are, and built LAMA-UHN ("UnHelpful Names"), a subset of Google-RE and T-REx with two filters (Section 4.4): drop facts whose answer is a substring of the subject's name (12% of facts, up to 81% for the manufacturer relation), and, for seven person relations, drop facts where BERT, asked "[X] is a common name in the following language: [MASK]." (or city, or country) for the first or last name, puts the answer in its top three. On Google-RE and T-REx combined (Table 4, Hits@1), BERT-base fell from 29.2 to 20.2 and BERT-large from 30.6 to 23.0; on the deleted facts BERT-base scored 52.7. Their reading is that BERT's score on LAMA is "partly due to the exploitation of helpful entity names". Their own entity-vector model, E-BERT-replace, which sees no name, lost under one point between LAMA and LAMA-UHN (29.1 to 28.2).

Better wording: LPAQA (Jiang, Xu, Araki and Neubig, TACL 2020). Treating each manual template as one possibly poor query, the authors mined alternative prompts from Wikipedia sentences containing known subject-object pairs and generated paraphrases by back-translation, then combined prompts with weights fitted on a separate training set of Wikidata pairs with no overlap with LAMA's T-REx (Sections 3 to 5). On the 41 T-REx relations, micro-averaged accuracy (equal to LAMA's P@1) rose from 31.1 with the manual prompts to 39.6 with the best weighted ensemble for BERT-base, and from 32.3 to 43.9 for BERT-large (Table 2). The best single prompt per relation gave 34.1 and 39.4. A fact counted as known by any of the generated prompts reached 52.6 and 56.1 (the "Oracle" column). On T-REx-UHN, BERT-base went from 21.3 to 29.4 (Table 9). The paper also reports that predicting each relation's majority object scores 22.0 micro-averaged but 2.2 when accuracy is averaged over distinct objects (macro), on which BERT-base with manual prompts scored 22.8 (Table 3). The authors call their result a "tighter lower bound". Their prompts and weights are chosen with supervision from labelled pairs, so the higher number measures the model together with a fitted query, a distinction the reader should keep when comparing it with LAMA's zero-shot score.

Limitations and open questions

  • Single-token objects and one template per relation (Section 4.5). The authors name multi-token answers as "an open challenge" and the variance across templates as future work (Section 6). Jiang et al. measured the second.
  • Floor. On T-REx the frequency baseline scores 22.03, so BERT-large's 32.3 is about 10 points above always guessing a relation's common objects (our arithmetic). Jiang et al.'s macro-averaged accuracy is one answer to this.
  • Training-data overlap. The aligned sentences for Google-RE and T-REx come from Wikipedia, which BERT was trained on; the paper says so (Section 5, Figure 4 caption). It does not separate recall of a memorized sentence from knowledge of the fact.
  • Ranking, not generation. The probe scores how a model ranks tokens in a fixed vocabulary, not what it would generate or say when asked a question.
  • What "knows" means. The definition is the paper's operational one; Poerner et al. show that a correct top-1 prediction can come from the form of a name rather than from the fact.

Research questions

  1. How much of a cloze score survives when every fact is paraphrased? Scoring the same facts with many independently written templates per relation, and reporting the spread per fact, would separate knowledge of the fact from fit to one wording.
  2. Does a zero-shot cloze score predict whether a model states the fact when asked? Comparing P@1 on LAMA facts with the correctness of free-form answers to the same facts as questions would test whether ranking and generation agree.
  3. How does the score depend on how often the fact appears in training text? Counting each fact's occurrences in a model's known pretraining corpus and plotting P@1 against that count would extend Figure 3's object-mention correlation to the fact itself.
  4. Where does a parametric store fail relative to a retrieved one? Running the same facts through a model with and without retrieved passages, split by LAMA's relation types, would show which kinds of fact (one-to-one, many-to-many, dates) need retrieval.

How it connects

  • Generative Retrieval: The Model as the Index treats a model's parameters as an index that returns document identifiers; LAMA tests the parameters as a store of facts. This note's connection: both ask what can be read out of weights without an external index.
  • Retrieval-Augmented Generation covers systems that add a retrieved, non-parametric store to the model. The SQuAD comparison here (BERT against DrQA's retrieve-then-read pipeline) is an early measurement of the gap retrieval closes. This note connects them; the unit page does not cite LAMA.
  • Evaluating Retrieval-Augmented Systems describes KILT (Petroni et al., 2021), a later benchmark from an overlapping group that scores answers together with their provenance. This note connects them.
  • Knowledge Graphs covers the structured stores LAMA compares against; LAMA's facts are Wikidata and ConceptNet triples, and its baseline is a knowledge graph extracted from text. Knowledge Graph Embeddings covers the knowledge-base completion work whose filtered ranking LAMA borrows (Bordes et al., 2013, cited in Section 4.4).
  • The components of agent memory defines agent memory to exclude weight changes. LAMA studies the excluded case: knowledge held in weights from pretraining. This note connects them.
  • GraphRAG, LoCoMo, LongMemEval and Mem0, the other notes of this unit, all study stores outside the weights. The planned synthesis memory-for-language-model-agents will draw on all of them.

References

  • Petroni, F., Rocktäschel, T., Lewis, P., Bakhtin, A., Wu, Y., Miller, A. H., and Riedel, S. (2019). Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2463–2473. https://doi.org/10.18653/v1/D19-1250 (arXiv:1909.01066) record · 2 pages
  • Poerner, N., Waltinger, U., and Schütze, H. (2020). E-BERT: Efficient-yet-effective entity embeddings for BERT. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 803–818. https://doi.org/10.18653/v1/2020.findings-emnlp.71 (arXiv:1911.03681) record
  • Jiang, Z., Xu, F. F., Araki, J., and Neubig, G. (2020). How can we know what language models know? Transactions of the Association for Computational Linguistics, 8, 423–438. https://doi.org/10.1162/tacl_a_00324 record
  • Bordes, A., Usunier, N., García-Durán, A., Weston, J., and Yakhnenko, O. (2013). Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems 26, pages 2787–2795. (Cited by the paper for the filtered ranking; not read for this note.) record · 3 pages
  • Elsahar, H., Vougiouklis, P., Remaci, A., Gravier, C., Hare, J., Laforest, F., and Simperl, E. (2018). T-REx: A large scale alignment of natural language with knowledge base triples. In Proceedings of LREC 2018. (Source of the T-REx facts; its alignment accuracy is given here as the paper reports it; not read for this note.) record
  • Petroni, F., Piktus, A., Fan, A., et al. (2021). KILT: A benchmark for knowledge intensive language tasks. In Proceedings of NAACL-HLT 2021. (Connected in "How it connects"; entry as the linked page gives it; not read for this note.) record · 2 pages