Enterprise, Personal, and Product Search

An employee searches the company intranet for travel policy and gets forty results: three drafts of the policy, a slide deck that mentions it, a former employee's notes, and the current version in sixth place. The same person searches the web for a restaurant and finds it immediately. People often conclude that the company's search engine is poor. More often the problem is the collection. Web search relies on resources the intranet does not have: billions of links that record which pages others consider useful, and the behavior of enormous numbers of users. Without them, a search engine has little beyond the text to rank with.

This article examines three settings that share that predicament in different forms. Enterprise search works within an organization's documents, where access is restricted and links are sparse. Personal search works within one person's files and messages, where the searcher usually knows the item exists. Product search works within a store's catalog, where documents are structured records and success is measured by purchases. In each, the methods of the course apply, but the evidence available for ranking differs from the web, and so does the definition of success.

Enterprise search

What is missing

Fagin and colleagues (2003) studied search on IBM's intranet and found that it differed from the web in ways that undermined web ranking methods. Intranet documents are written to record or announce information, not to attract readers, so they are rarely linked and their links say little about quality. Many queries have only one correct answer, such as a particular form or policy, so a ranking that returns many reasonable documents but not that one has failed. And there is little or no spam, which removes one of the problems web engines spend most effort on, but also means that signals designed to detect spam provide little useful information here.

Hawking (2004) catalogued further difficulties:

• Heterogeneous content. An organization's information lives in document repositories, file shares, wikis, email, ticketing systems, databases, and chat. Each needs a connector to extract its content and metadata, and the formats differ widely.
• Duplicates and versions. The same document exists in many drafts and copies, and the current version must outrank the obsolete ones.
• Little behavioral data. An organization of a few thousand people issues too few queries for click data to be reliable for most of them.
• Jargon. Internal acronyms, project code names, and product names appear in no general dictionary, so spelling correction and synonym expansion must be built from the organization's own data.

Access control

The defining technical requirement of enterprise search is that results must respect permissions. An employee must never see a document they are not authorized to open, and should not learn from a search result that such a document exists. A snippet or even a title can disclose confidential information. There are two basic approaches (Bailey, Hawking, and Matson, 2006):

• Early binding records each document's access control list in the index, and the query is restricted to documents the searcher may read. Retrieval is efficient, but the index must be updated whenever permissions change, and permissions in many organizations change constantly.
• Late binding retrieves results without regard to permissions and checks each one against the source system before displaying it. Permissions are always current, but checks are slow, and if most top results are filtered out, the system must retrieve many more to fill a page.

The cost of late binding is easy to estimate. Suppose a searcher may read a fraction $a$ of the documents that match a query, and that readable documents are scattered through the ranking independently of score. To fill a page of 10 results, the system must check candidates one at a time until 10 have passed. The number checked follows a negative binomial distribution with mean $10/a$ (values from tools/experiments/foundations/product_search.py, rounded to whole candidates):

An employee who may read most of the collection costs one or two extra checks per page. A contractor with access to one document in twenty costs about two hundred, each a call to a source system, and occasionally more than three hundred. The users with the narrowest permissions get the slowest searches, and the ranking must retrieve deep enough to supply them.

Real systems mix the two, indexing coarse permissions such as group membership and checking fine-grained ones at query time. Access control also affects ranking in a less obvious way. Statistics such as document frequency computed over the whole collection can leak information about documents a user cannot see, so some security-sensitive systems compute them only over what each user may read.

What enterprise ranking relies on

Without links and clicks, enterprise ranking depends on evidence the web lacks or uses less:

• Metadata: author, department, document type, creation and modification dates, and the source system. A query for a policy should prefer the official policy repository over a personal folder.
• Freshness: for policies, procedures, and documentation, the newest version is usually the one wanted.
• Organizational structure: documents from the searcher's own team or project, or from colleagues they work with, are more likely to be relevant.
• Curated results: administrators can pin the authoritative answer for frequent queries. Web engines use manual curation sparingly, because their query volume is too large for it, but for an organization's handful of most frequent queries it is often the most effective intervention.

Enterprise search is also where expert finding matters most, since knowing which colleague understands a system is often more valuable than any document about it. Evaluation is difficult for the same reasons ranking is. There are few queries per topic, relevance depends on organizational knowledge, and test collections are hard to build from confidential data. Organizations often rely on indirect measures such as queries that return no results, queries reformulated immediately, and satisfaction surveys.

Personal search

Re-finding, not finding

Personal search covers one person's documents, email, photos, messages, and browsing history, on a device or in a cloud account. Its most distinctive feature is the task. People mostly search their own collections to re-find something they have seen before. Even on the web, re-finding is common: Teevan, Adar, Jones, and Potts (2007) found, in a year of Yahoo logs from 114 users, that as many as 40 percent of queries were attempts to find information the same user had found before, judged by whether the user clicked the same results again.

Re-finding changes what matters. The searcher knows the item exists, so recall matters a great deal: if the system fails to return it, the user knows the system failed. The searcher also remembers things about the item other than its content: roughly when they saw it, who sent it, what kind of file it was, what they were working on at the time. Dumais and colleagues (2003), studying Microsoft's Stuff I've Seen system, found that people used dates and people's names heavily as retrieval cues, and often preferred results sorted by date rather than by computed relevance.

Orienteering instead of searching

People often do not search their own collections at all. Teevan and colleagues (2004) observed that people looking for known information frequently orient step by step, opening a folder they remember, then a subfolder, then scanning a list, rather than jumping directly to the target with a query. The path uses what they remember about where they put things, which is often more reliable than remembering the words in a document. A good personal search tool supports both strategies. It answers queries, and it lets people filter by date, sender, and file type, which are the cues orienteering relies on.

Constraints

Personal collections bring their own technical constraints. Indexing must run on the user's device or under strict privacy guarantees. File formats vary widely, and many items, such as photos and voice memos, have little or no text unless it is extracted. There is little collective behavior to learn from about any particular item, since each collection has one user, but that user's own history is rich evidence: items opened recently or often are likely targets. Evaluation is especially hard, because no two people's collections are alike and the collections are private, so most evaluations rely on user studies and logged behavior rather than shared test collections (Elsweiler and Ruthven, 2007).

Product search

Documents are records

In product search the collection is a catalog. Each item is a structured record with a title, a description, a brand, a category, a price, attributes such as size and color, images, and customer ratings and reviews. Queries are short and often name attributes rather than describe a topic: samsung phone blue is a request for products where the brand is Samsung, the type is phone, and the color is blue. Product search therefore combines the two traditions distinguished in What information retrieval is. It uses exact filtering on structured attributes, as a database would, and ranked matching over text.

Query understanding is central. The engine must recognize that blue is a color attribute and not a word to be matched in descriptions, that iphone 15 case asks for a case and not a phone, and that running shoes size 10 specifies a size to filter on. Beyond attributes, the engine must also recognize what kind of shopping task a query represents. Sondhi and colleagues (2018) analyzed Walmart query logs and proposed five task types: shallow exploration, targeted purchase, major-item shopping, minor-item shopping, and hard-choice shopping.

Faceted navigation complements the query. After an initial search, the interface shows counts for the values of each attribute (brands, price ranges, sizes, ratings), and the user narrows the results by selecting values. Hearst (2006) argued that facets built from meaningful categories support exploration better than automatically generated clusters, because users understand them immediately and can combine them freely. Faceted search requires that the index store attribute values in a form that supports fast counting, an indexing decision of the kind discussed in the architecture article.

A worked example: facets on a small catalog

A store sells the twelve items below. Each record has a title, a brand, a price, an average rating, and the sizes in stock.

Text matching. The query running shoes, with a plural s removed so that shoes matches shoe, and both words required, matches nine items: everything except the socks P6 (no shoe), the trail shoe P9 (no running), and the basketball shoe P12. Matching on running alone would have returned the socks, which is the attribute problem of query understanding in miniature: the query names a product type, and the type is shoe.

Counting facet values. Over the nine matches, the facet counts are:

The size counts add to 17, not 9, because size is multi-valued: P1 is in stock in three sizes and is counted under each. Brand and price are single-valued, so their counts add to 9.

Selecting values. Facet interfaces conventionally combine selections with OR within a facet and AND across facets. Ticking Brooks and Saucony means "Brooks or Saucony", and adding size 10 means "and in stock in size 10". The question is what the brand counts should show once Brooks is ticked. If they are counted over the current results, every other brand shows 0, and the interface no longer tells the shopper that adding Saucony would bring in two more shoes. The usual remedy is to compute each facet's counts with every selection applied except that facet's own. Ticking Brooks, then Saucony, then size 10:

In the last row the brand counts ignore the brand selection but apply the size filter, so they answer "how many size-10 running shoes does each brand have", and Asics drops to 1 because only P4 comes in size 10. The size counts ignore the size selection but apply the brand filter. Each count is the number of results the shopper would get if that value were the only one ticked in its facet, with the other facets' selections kept: Asics (1) means one size-10 running shoe from Asics, P4. That is what makes the counts useful for deciding where to click next. Computing them costs one pass over the matches for each facet, which is why engines that support faceting keep each document's attribute values in a structure that can be read quickly for counting.

Ranking toward a purchase

Relevance in product search is entangled with commercial outcomes. Among many products that match running shoes, the store prefers to rank first those that the customer is likely to buy. That makes behavioral data the main source of training signal. Sorokina and Cantu-Paz (2016) described how Amazon's product search trains ranking models on behavioral labels, with purchases as the strongest evidence of relevance, using features such as text match, sales history, ratings, and price, and training separate models for different product categories because what predicts a purchase differs between books and electronics.

Ranking toward purchases introduces specific hazards:

• Selection bias. A product can be purchased only if it was shown. A model trained on purchases learns to favor whatever was already ranked highly, and new products struggle to be seen at all.
• Intent varies. A broad query such as gift ideas is exploratory, and users click many items and buy few. A specific query such as a model number is navigational, and users buy quickly. Treating clicks and purchases the same way across both kinds of query trains the model on very different behavior.
• Conflicting objectives. The ranking that maximizes immediate revenue may differ from the one that best serves the customer, for instance by favoring higher-margin products over better matches. Tsagkias and colleagues (2020) discuss these tensions among the open problems in e-commerce search.

Product search also tries hard not to return nothing. A query with no exact matches typically falls back to relaxing some query terms or attributes, since an empty page loses the sale. The decision of which constraints to relax is itself a ranking problem: a customer searching for red leather jacket size m may accept a different shade of red, but probably not a different size.

A six-item catalog of outerwear makes the choice concrete. The query asks for four constraints: type jacket, material leather, color red, size M.

No item satisfies all four. Each single relaxation returns something different:

Every relaxation returns a result, so "return something" does not decide among them. The order of preference encodes knowledge about the domain: colors have near neighbors that customers often accept, size is a hard requirement for clothing, and material and type sit in between. One source for such an order is behavior: which substitutes customers actually buy after a relaxed query, another place where product search depends on behavioral data.

Comparing the three settings

The table makes the general lesson explicit. A ranking method depends on evidence, and the evidence available differs from one setting to another. Methods that depend on links or crowd behavior perform poorly where links and crowds are absent. Methods that depend only on text, such as BM25, transfer everywhere, which is one reason they remain the default in open-source search engines and the first stage of many production systems. The settings also differ in what counts as success, and that difference determines the measures used to evaluate them, a theme developed in evaluation.

References

• Bailey, P., Hawking, D., and Matson, B. (2006). Secure search in enterprise webs: Tradeoffs in efficient implementation for document level security. In Proceedings of CIKM 2006, 493–502.
• Dumais, S., Cutrell, E., Cadiz, J. J., Jancke, G., Sarin, R., and Robbins, D. C. (2003). Stuff I've Seen: A system for personal information retrieval and re-use. In Proceedings of SIGIR 2003, 72–79.
• Elsweiler, D., and Ruthven, I. (2007). Towards task-based personal information management evaluations. In Proceedings of SIGIR 2007, 23–30.
• Fagin, R., Kumar, R., McCurley, K. S., Novak, J., Sivakumar, D., Tomlin, J. A., and Williamson, D. P. (2003). Searching the workplace web. In Proceedings of WWW 2003, 366–375.
• Hawking, D. (2004). Challenges in enterprise search. In Proceedings of the 15th Australasian Database Conference (ADC 2004), 15–24.
• Hearst, M. A. (2006). Clustering versus faceted categories for information exploration. Communications of the ACM, 49(4), 59–61.
• Sondhi, P., Sharma, M., Kolari, P., and Zhai, C. (2018). A taxonomy of queries for e-commerce search. In Proceedings of SIGIR 2018, 1245–1248.
• Sorokina, D., and Cantu-Paz, E. (2016). Amazon search: The joy of ranking products. In Proceedings of SIGIR 2016, 459–460.
• Teevan, J., Adar, E., Jones, R., and Potts, M. A. S. (2007). Information re-retrieval: Repeat queries in Yahoo's logs. In Proceedings of SIGIR 2007, 151–158.
• Teevan, J., Alvarado, C., Ackerman, M. S., and Karger, D. R. (2004). The perfect search engine is not enough: A study of orienteering behavior in directed search. In Proceedings of CHI 2004, 415–422.
• Tsagkias, M., King, T. H., Kallumadi, S., Murdock, V., and de Rijke, M. (2020). Challenges and research opportunities in eCommerce search and recommendations. SIGIR Forum, 54(1), article 2, 1–23.