Battling the Matthew Effect: Can Search Results be Fair?

Search engines are mediators between producers and consumers of information, and the order in which an engine presents its results affects both stakeholder groups in significant ways (e.g., economic livelihood). Top-ranked items provide their producers with much more exposure to the consumer than do lower-ranked items. If a search engine repeatedly returns comparably relevant items in the same order, the rich (more exposed) items will keep getting richer while lower-ranked items will get less and less relative exposure. The Text REtrieval Conference (TREC, trec.nist.gov) project at NIST contained a Fair Ranking track from 2019--2022 to study the problem of producing fair rankings. The track defined fairness as providing appropriate exposure to items over a set of searches, where `appropriate exposure' encompasses both relevance and rank within the search results. Organizers of the track faced multiple challenges in operationalizing the evaluation task that system implementers will also face. Chief among these challenges are defining the attributes the engine should be fair with respect to (there isn't a universally fair ranking) and having enough information regarding those attributes to implement some desirable policy. Bio: Ellen Voorhees is a scientist emeritus in the Information Technology Laboratory at the U.S. National Institute of Standards and Technology (NIST). Her primary responsibility at NIST was managing the Text REtrieval Conference (TREC), an international work...
Source: Videocast - All Events - Category: General Medicine Tags: Upcoming Events Source Type: video