|Expanding Access to Science and Technology (UNU, 1994, 462 pages)|
|Session 3: New technologies and media for information retrieval and transfer|
|Information retrieval: Theory, experiment, and operational systems|
One of the most important concepts for an understanding of our present ideas of information retrieval is that of relevance. Relevance is an extremely difficult idea to pin down and has been the subject of much work over a large number of years . It originates from well before the ASK hypothesis, and one way to think of it might be in terms of correctness. An item might be regarded as relevant to a request if it is in some sense a "correct" response to that request. There are indeed IR theorists who take a modern version of that view: an item is relevant to a request if the request can be inferred from the item, rather in the fashion of theorem proving, but with an appropriate logic .
However, probably the dominant view of relevance (and one that is rather more in sympathy with the ASK hypothesis) is a much more subjective one. An extreme version of the subjective approach would be to say simply that an item is relevant to a user's information need or ASK if the user says it is, or in other words if the user would like the system to retrieve this item. More commonly, in our experiments we rely where we can on end-users making relevance judgements in relation to their perceived needs, according to some descriptive scale that we devise; we also do some laboratory experiments with expert judges judging relevance to stated requests.
Even without agreement as to what exactly relevance is, the idea of relevance is of central importance to theory and experiment in IR, and it is becoming important to IR practice as well. From the experimental point of view, where the concept originated, we need it in order to evaluate different approaches and methods in IR. From there it has fed into theory; many recent theories in IR depend upon it. Finally, some methods of IR ask the user to make relevance judgements on-line and use that information internally as one kind of clue to help formulate a new search.
The major assumption is that users can make relevance judgements or recognize relevant items, even if not necessarily with absolute confidence. It is hard to imagine an information-seeking activity where the user was in principle unable to assess whether an item is appropriate or not, though of course users may suspend judgement or change their minds in particular cases, perhaps until or after they have more information about the item or have read some other item.