Search in Professional Domains

In the mid-1980s, David Blair and M. E. Maron evaluated a full-text retrieval system used by lawyers to find documents for a corporate lawsuit (Blair and Maron, 1985). The collection held about 40,000 documents. Experienced lawyers and paralegals searched it until they believed they had found at least three-quarters of the documents relevant to each request. When Blair and Maron measured what had actually been found, recall averaged about 20 percent. The searchers had found a fifth of what mattered while believing they had found most of it, and nothing in the system's output told them otherwise.

That gap between perceived and actual recall is the defining risk of professional search. On the web, a searcher who finds one good page is usually satisfied, and the many good pages not shown cost nothing. A lawyer who misses a controlling precedent, a reviewer who misses a clinical trial, or a patent examiner who misses prior art can make a costly and sometimes irreversible error. This article examines four professional domains where the cost of a missed document shapes how search is done: law, medicine, patents, and scholarship.

What sets professional search apart

Several features recur across these domains and distinguish them from web search.

Recall matters as much as precision. Many professional tasks require finding all the relevant documents, or as close to all as possible, not just a few good ones. Evaluation therefore emphasizes recall, and searchers are willing to read far more results than a web user would. Not every professional task is like this: a lawyer looking up a known decision needs one authoritative document, not all of them.

Authority is formal. In law, a decision from a higher court binds lower courts in its jurisdiction. In medicine, for a question about the effect of a treatment, a randomized controlled trial outweighs a case report. Authority on the web is estimated from links. In these domains it is defined by institutions and by recognized hierarchies of evidence, and ranking must respect it.

Vocabularies are controlled. Each domain maintains a curated vocabulary, built and applied by trained indexers, because the vocabulary mismatch problem described in What information retrieval is is too costly to leave to free-text matching.

Searchers are experts, and they need to see why. Professional searchers are trained, often search for hours, and must be able to justify and reproduce their searches. Many still prefer Boolean queries, because a Boolean query states exactly what was searched and returns the same result when it is run again on the same collection.

General-purpose text processing can be wrong. Domain language breaks the assumptions of generic tokenizers, stop-word lists, and stemmers, in ways described in each section below.

Legal search

Legal research draws on case law (the published decisions of courts), statutes and regulations, and secondary sources such as treatises. In litigation, it also includes electronic discovery: identifying the relevant documents among the emails and files of the parties to a lawsuit.

Editorial classification. The legal publishers built some of the most elaborate controlled vocabularies in any field. West's Key Number System organizes American case law into a hierarchy of legal topics and subtopics, each identified by a key number. Editors read each published decision, write headnotes summarizing each point of law it decides, and assign each headnote to a key number. A lawyer can then find every decision on a narrow point of law, regardless of the words the judges used. This is human indexing at large scale, and it addresses the vocabulary mismatch problem directly: the key number stands for the legal concept, whatever words a judge used to express it.

Citation and authority. Legal documents cite one another constantly, and citations carry meaning beyond the link itself: a later decision may follow, distinguish, criticize, or overrule an earlier one. Citators, such as Shepard's (first published in the nineteenth century) and Westlaw's KeyCite, record how each decision has been treated by later courts, so that a lawyer can check whether a case is still good law. Citation counts alone are a poor measure of legal authority, because they must be weighed by jurisdiction: a decision cited heavily in another state's courts may matter less to a local question than a moderately cited decision from the lawyer's own court system.

Language. Legal text defeats generic analysis. Words that general stop-word lists treat as noise can carry legal force: in a statute, may ordinarily grants permission and shall ordinarily imposes an obligation, and a system that discards modal verbs cannot tell them apart. Terms of art, Latin phrases, and citation formats (such as 410 U.S. 113, which identifies a Supreme Court decision by volume and page) need tokenizers that keep them intact. Older statutes use older vocabulary, so a query about cars may need to match motor vehicle or conveyance.

Boolean persistence. Legal databases offered ranked natural-language search from the early 1990s, yet Boolean "terms and connectors" searching, with proximity operators such as "within five words", remained the default for many professional users for well over a decade (Manning, Raghavan, and Schütze, 2008, §1.4). The reasons were control and accountability. A lawyer can state precisely what a Boolean search covered, and Blair and Maron's study showed why that statement may still be wrong.

Electronic discovery. Litigation can require reviewing millions of documents to decide which must be produced to the other side. Manual review at that scale is slow, expensive, and, as Blair and Maron's study suggests, less reliable than it appears. The TREC Legal track (2006–2011) evaluated retrieval and classification for discovery. Grossman and Cormack (2011), analyzing its results, found that the two technology-assisted review efforts they examined, in which a system uses a reviewer's judgments to rank or classify the remaining documents, matched or exceeded exhaustive manual review in both recall and precision, with far less human effort. Courts in the United States began to accept such review in 2012. Electronic discovery is an applied case of relevance feedback and text classification run to a recall target, with a documented method for deciding when enough has been found.

Medical and biomedical search

Medical search serves several groups with different needs. Researchers search the biomedical literature, clinicians search for evidence to guide a decision about a patient, authors of systematic reviews search for every study on a question, and patients search for information about their own health.

MeSH. The U.S. National Library of Medicine indexes the journal literature in MEDLINE with Medical Subject Headings (MeSH), a controlled vocabulary of about 30,000 descriptors arranged in hierarchies. Myocardial Infarction, for example, sits beneath Myocardial Ischemia, which sits beneath Heart Diseases. For decades, trained indexers read each article and assigned its headings. Since April 2022 indexing has been automated, with human curators reviewing a subset of the indexed citations. A search on a heading can be exploded to include every heading beneath it in the hierarchy, so a search for Heart Diseases also retrieves articles indexed only under narrower conditions. This is the hierarchy of a controlled vocabulary used directly as query expansion, and it is one of the oldest successful forms of concept-based retrieval.

PubMed and learned ranking. PubMed, the public interface to MEDLINE and related databases, covers more than 36 million citations. For most of its history it sorted results by date. In 2017 the National Library of Medicine introduced Best Match, which retrieves candidates with a BM25-style function and re-ranks them with a learning-to-rank model trained on aggregated user behavior, using features such as publication type and recency (Fiorini et al., 2018). The design is the same staged ranking used in web search, adapted to a literature database.

Systematic reviews. A systematic review aims to find every study addressing a clinical question, then assess and combine them. Its searches are written by information specialists as long Boolean queries combining MeSH headings and free-text terms, and they deliberately trade precision for recall: a search may return thousands of citations, of which a few dozen are eventually included. The query must be reported in full so that others can reproduce it. These requirements explain why reviewers continue to use Boolean databases even when ranked systems retrieve more relevant documents on the first page, and why the effort has shifted toward ranking and classifying the retrieved citations to speed up screening, as the CLEF eHealth technology-assisted review tasks evaluated (Kanoulas et al., 2017).

A search strategy, block by block

Systematic review searches are built from concept blocks. The question is split into its parts, typically the population, the intervention, and the type of study wanted. Each block collects every way that part might be expressed, joined by OR: a controlled-vocabulary heading, its narrower headings, and free-text words with truncation. The blocks are then joined by AND, so that a citation must mention each part in some form. OR within a block raises recall; AND across blocks raises precision.

Take the question "Do exercise programs reduce pain in knee osteoarthritis, judged from randomized controlled trials?" A small strategy for it, in a generic Boolean notation (* truncates a word, [MeSH] searches the indexers' heading, and "exploded" includes the headings beneath it):

The headings are real MeSH descriptors. Exploding Exercise Therapy brings in narrower descriptors such as Resistance Training, Endurance Training, and Muscle Stretching Exercises (checked against the National Library of Medicine's MeSH data, November 2026). Real strategies are much longer, with dozens of terms per block, but the structure is the same.

To see what each choice does, the strategies were run over a toy collection of 30 invented citations, each with a title, headings, and a publication type, judged by the review's criterion. Ten are relevant: randomized trials of exercise in knee osteoarthritis. The other twenty are near misses of the kinds a real search turns up: the right population and intervention in a cohort study, review, or protocol; the right population and design with a drug, brace, or injection; and the right intervention and design in hip osteoarthritis, back pain, or rheumatoid arthritis. Two of the relevant records are hard on purpose. One is too recent to have been indexed, so it has no headings or publication type. The other calls the condition gonarthrosis and never says knee osteoarthritis. The results (computed by tools/experiments/foundations/review_strategy.py; recall and precision rounded to two places):

Several lessons that hold for real searches show up even at this size.

• AND costs recall unless every block is complete. Strategy B keeps only the trials whose titles say exercise. Trials of quadriceps strengthening, walking, tai chi, or cycling are exercise trials, and a block that misses their vocabulary silently removes them. A block is only as good as its list of synonyms.
• Explosion is query expansion along the hierarchy. Exploding Exercise Therapy in I3 matches the trials indexed only under Resistance Training or Endurance Training, which the unexploded heading in I2 misses. In this toy collection those two trials also say strengthening and training in their titles, so I3's free-text words would have found them anyway; in a real database many records carry a narrower heading without the corresponding word, and explosion is what finds them.
• Controlled vocabulary and free text cover each other's gaps. The publication-type filter D1 is precise, but it cannot find a citation that has not been indexed yet. Adding the free-text word random* in D2 recovers it, at the cost of one protocol that mentions a randomized trial it has not yet run.
• A missing synonym defeats every other refinement. Strategies A through F all miss the gonarthrosis trial, because the population block never names it. Only adding the word, in G, reaches full recall, and nothing in the output of F signals that anything was missing. That is Blair and Maron's finding in miniature: the searcher cannot see what the search failed to find.

The toy precision is far higher than in practice. A real review's search of a database of tens of millions of citations commonly returns thousands of records for a few dozen included studies, so the number of records screened for each included study, the reciprocal of precision, runs into the dozens or hundreds. That workload is what technology-assisted screening aims to reduce.

Clinical decision support. TREC's Clinical Decision Support track (2014–2016) gave systems a patient case description and asked them to retrieve articles that would help diagnose, test, or treat the patient. Case descriptions are long and full of findings that may or may not be relevant, so the difficulty lies in deciding which parts of a long query matter.

Consumers and safety. Patients describe conditions in everyday language, and the literature uses clinical language: heart attack and myocardial infarction, high blood pressure and hypertension. Researchers have built consumer health vocabularies that map one to the other (Zeng and Tse, 2006). Medical search also adds a criterion that most domains lack: a highly ranked document that is wrong can cause harm. Evaluation for health search has therefore begun to measure whether systems promote correct and credible information over misinformation, not only whether results are on topic.

Patent search

Patent search has several distinct tasks. A prior art search, performed by examiners and applicants, looks for anything published before an application that would make its invention not new or obvious. An invalidity search looks for prior art that would defeat an existing patent. A freedom-to-operate search checks whether a product would infringe patents in force. Each is recall-oriented, and in invalidity searches a single missed document can decide the outcome of a dispute worth a great deal of money (Lupu and Hanbury, 2013).

Patent search differs from other domains in several ways.

The query is a document. A prior art search often starts from an entire patent application, thousands of words long. Turning it into an effective query means selecting the terms that characterize the invention from a long, repetitive text, an unusual problem for retrieval systems designed for short queries.

The language is deliberately abstract. Patent claims are written to cover as much as the prior art allows, so drafters often prefer general terms to specific ones. A hook may become a fastening means, and a car a vehicle having a plurality of wheels. The vocabulary mismatch is largely intentional, and it works against the searcher.

Classification carries much of the load. Patent offices assign every document to codes in the International Patent Classification and, at the European and U.S. offices, the more detailed Cooperative Patent Classification. Professional searchers combine classification codes with text terms, because classification captures the technical subject regardless of wording.

Evaluation reflects recall. The CLEF-IP and NTCIR patent evaluations built test collections for these tasks, and Magdy and Jones (2010) proposed a measure, PRES, that rewards finding all relevant documents within the number of results a professional is willing to review, rather than rewarding precision at the top of the list.

Scholarly search

Researchers search the scholarly literature to find work on a topic, to follow a line of work forward and backward through citations, and to keep up with new publications.

Citation indexing. Eugene Garfield (1955) proposed a citation index for science: for each paper, a list of the later papers that cite it. He pointed to Shepard's legal citator as a working model, so the idea passed from legal search to scholarly search. Following citations forward finds newer work on a subject without knowing its vocabulary in advance, which partly solves the vocabulary mismatch problem through the citation graph. Garfield's work on citation analysis was one of the inspirations for link-based ranking on the web, and the influence now runs in both directions: academic search engines rank papers using citation counts and link-analysis methods derived from web search.

Ranking and its biases. Google Scholar, launched in 2004, does not publish its ranking formula, but Beel and Gipp (2009) found empirically that citation counts carried substantial weight in it. Citation-weighted ranking surfaces influential work but favors older papers, which have had time to accumulate citations, and reinforces papers that are already well cited. Researchers looking for recent or less-known work must compensate, for instance by sorting by date.

Suitability for systematic searching. Gusenbauer and Haddaway (2020) tested 28 academic search systems against the requirements of systematic reviews, such as support for complex Boolean queries, reproducible results, and transparent coverage. Many widely used systems, Google Scholar among them, did not meet those requirements well enough to serve as the main search tool for a systematic review, although they remain useful supplements. The finding illustrates the theme of this article: a system tuned to give good results on the first page may still be unsuited to a task that must find everything and be reproducible.

Representations of papers. Scholarly search now also uses learned representations of papers. Cohan and colleagues (2020) trained document embeddings using the citation graph itself, treating papers that cite one another as similar, which combines Garfield's insight with dense retrieval.

Common lessons

The four domains suggest several general lessons for retrieval system design.

1. Know the cost of a miss. Whether a task needs high recall determines most of the design: the evaluation measure, the query language, how many results the interface shows, and whether the system should explain when to stop searching. Blair and Maron's study shows that searchers cannot judge recall by looking at their own results.
2. Controlled vocabularies are domain knowledge. MeSH, the Key Number System, and patent classifications encode decades of expert judgment about which concepts are the same and how they relate. Learned representations can approximate some of that knowledge from text, but the curated vocabularies remain among the most effective tools these domains have.
3. Generic preprocessing must be checked against domain language. Stop-word lists, stemmers, and tokenizers designed for general text can remove the distinctions a domain depends on. The may and shall example is the clearest case, but the same problem arises with chemical names, gene symbols, and citation formats.
4. Transparency and reproducibility can outweigh effectiveness. Professional searchers must be able to state what they searched for and repeat it. That requirement keeps Boolean retrieval in use and shapes how ranked and learned methods are introduced: usually as aids for ranking or screening the results of a documented search, rather than as replacements for it.

The next article examines settings with the opposite characteristics: collections searched by non-specialists, where convenience, access control, and commercial goals dominate.

References

• Beel, J., and Gipp, B. (2009). Google Scholar's ranking algorithm: An introductory overview. In Proceedings of the 12th International Conference on Scientometrics and Informetrics (ISSI 2009), 230–241.
• Blair, D. C., and Maron, M. E. (1985). An evaluation of retrieval effectiveness for a full-text document-retrieval system. Communications of the ACM, 28(3), 289–299.
• Cohan, A., Feldman, S., Beltagy, I., Downey, D., and Weld, D. S. (2020). SPECTER: Document-level representation learning using citation-informed transformers. In Proceedings of ACL 2020, 2270–2282.
• Fiorini, N., Canese, K., Starchenko, G., Kireev, E., Kim, W., Miller, V., Osipov, M., Kholodov, M., Ismagilov, R., Mohan, S., Ostell, J., and Lu, Z. (2018). Best Match: New relevance search for PubMed. PLOS Biology, 16(8), e2005343.
• Garfield, E. (1955). Citation indexes for science. Science, 122(3159), 108–111.
• Grossman, M. R., and Cormack, G. V. (2011). Technology-assisted review in e-discovery can be more effective and more efficient than exhaustive manual review. Richmond Journal of Law and Technology, 17(3), article 11.
• Gusenbauer, M., and Haddaway, N. R. (2020). Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Research Synthesis Methods, 11(2), 181–217.
• Kanoulas, E., Li, D., Azzopardi, L., and Spijker, R. (2017). CLEF 2017 technologically assisted reviews in empirical medicine overview. In CLEF 2017 Working Notes.
• Lupu, M., and Hanbury, A. (2013). Patent retrieval. Foundations and Trends in Information Retrieval, 7(1), 1–97.
• Magdy, W., and Jones, G. J. F. (2010). PRES: A score metric for evaluating recall-oriented information retrieval applications. In Proceedings of SIGIR 2010, 611–618.
• Manning, C. D., Raghavan, P., and Schütze, H. (2008). Introduction to Information Retrieval, section 1.4. Cambridge University Press.
• Zeng, Q. T., and Tse, T. (2006). Exploring and developing consumer health vocabularies. Journal of the American Medical Informatics Association, 13(1), 24–29.