The multidimensionality and dynamic nature of relevance

Get Complete Project Material File(s) Now! »

CHAPTER 3: DEVELOPING A THEORETiCAL FRAMEWO RK

In this chapter, attributes and manifestations of relevance as defined by Saracevic (1996) and mentioned in Section 2.3.1 have been plotted against each other in a matrix in order to show that relevance functions in different dimensions for the various manifestations of relevance . The analyses reveal the necessity for a revised model of relevance types. It is argued that the manifestation of motivational or affective relevance should not be viewed as a discrete category or as part of a linear scale of relevances. Instead, motivational relevance may essentially be included in the attribute of intention, and affective relevance acts as a different dimension altogether, influencing all the other subjective relevance types. The modified model includes a socio­ cognitive type of relevance that is highly context dependent and associated with either organisational strategies or within scientific community interaction
This revised model of attributes and manifestations of relevance has then been modelled on an existing cognitive model of information transfer (Figure 3.2), (Ingwersen, 1996). The new derived model allows the mapping of relevance assessments and stresses the distinction between the use of information in work task performance and in search task activities . It is shown that relevance signifies two processes: feedback from systems (e.g . ranked output) to system, users or context, as well as the reverse process of relevance feedback from the system, users or context to the information objects. This notion of feedback from systems has already been described by Ingwersen (1984) in relation to frequency ranked terms or keyword lists applied for online query modification. It is also suggested that relevance types may be important in different ways for the search task and for the work task. Most of the work in this chapter has been reported in Cosijn and Ingwersen (2000) and Cosijn (2003) .
The structure of this chapter is as follows: The first two sections present an introduction to the nature of relevance by describing different aspects of the attributes and manifestations of relevance . The third section describes a matrix where the attributes of relevance have been plotted against the manifestations of relevance (Table 3.3). This section also includes a discussion on the emerging patterns in the matrix, by examining each of the attributes of relevance in turn. Section 3.4 contains the explanation of the modified relevance model (Table 3.4) as derived from the previous discussions. The consequences of relevance variety, and the manifestations of relevance (relevance types) are discussed in detail in Sections 3.5 and 3.6 respectively. Section 3.7 discusses the proposed consolidated model of relevance types and the modelling of relevance types on Ingwersen’s Cognitive Model of Information Transfer (Figure 3.2) . By re-organizing this model, the different types of relevance have been shown to operate in different dimensions (and over time) of the information retrieval process. In Section 3.8 the issues of work task and search task execution as related to the proposed model have been discussed. In Section 3.9 a discussion of previous empirical studies on relevance has been presented . The summarizing and consolidation of these empirical studies on the relevance judgments of actual users, has shown that the model described in Section 3.4 is a viable model to utilize in empirical testing of relevance judgments by users. Some conclusions and future research related to these issues are listed in Section 3.10.

Attributes of relevance

In his article of 1996, Saracevic sources from intuition, philosophy and communication, and ascribes the following attributes to relevance, starting from the assumption that relevance is rooted in human cognition, as described in Table 3.1 .
This is succinctly summarized by the following statement: liAs a cognitive notion, relevance involves an interactive, dynamic establishment of a relation by inference, with intentions toward a context » (Saracevic, 1996: 206).

Manifestations of relevance

Looking at the attributes of relevance as listed above, it is clear that relevance always indicates a relation. Different manifestations of relevance indicate different relations. It would therefore seem that the trend moves toward viewing relevance in IR not as a single definition of relevance, but as a system of relevances (note the plural). Consequently no single relevance in the system can be viewed in isolation. Relevance exists as an interacting system of relevances on different levels.
As with studies on the nature of relevance, manifestation studies are also widely divergent. In his article, Saracevic (1996) summarizes these studies and distinguishes the following manifestations of relevance, as represented in Table 3.2.¹
In all instances the term text (or information object) is seen to mean not only retrieved texts, but also texts in the system file or even texts which are in existence somewhere, but not necessarily in the system file (Saracevic, 1996).
Although Saracevic does not explicitly mention it, it is interesting to note that the relevances are moving (in the order listed above) from a systems approach to a user- and socially-orientated approach. Thus the whole spectrum is included.
The view that relevance is no longer simply a binary assessment between objective and subjective relevance or consisting of a binary scale, is also supported by other researchers (Greisdorf & Spink, 1999). Borlund and Ingwersen (1998) introduce the concept of relative relevance that describes the degree of agreement between various types of relevance applied in the evaluation of information retrieval systems.
The multidimensional nature of relevance was of importance in the next step of the modelling process, where the attributes of relevance have been plotted against the manifestations of relevance in a matrix format. Each of the manifestations defined by Saracevic was compared to each of the attributes defined to establish the connections and need for each of the attributes to be present in each manifestation .

Attributes and manifestations of relevance: What are the connections?

In this section, the manifestations of relevance have been plotted against the attributes of relevance (both as defined by Saracevic), and then the content of each of the cells in the matrix has been discussed in some detail, according to the attributes of relevance . The last column in the table is shaded, as the inclusion of these types of relevance is in question. These aspects have been discussed in Section 3.3.6 of this chapter.

Relation

Relevance always implies a relation . In Saracevic’s scheme, this relation is between some entity and the information object, which is simply defined as « texts » (1996: 214). We should like to argue that information objects should be defined much broader to include anything conveying information, including, for example, images . The implications of this broader definition will be discussed in more detail under 3.3.5, where the time dimension has a certain impact on the interaction process . As indicated in Table 3.3, it is clear that the relevance attribute relation moves from being purely objective (between the query and the system) to a highly subjective and individualized relation that involves the user’s intents, goals and motivations. The detail of each of these relations is as follows:
The relation defining system or algorithmic relevance may be measured in terms of the comparative effectiveness of inferring relevance . This relation is very much system-orientated because the success of the relation is entirely dependent on a given procedure or algorithm, and the intent behind it. Both the query and the objects contain identicallsimilar features , such as terms, image colour or author name.
Topical or subject relevance is characterized by a relation between the topic of the request and the topic of the information objects . It may be measured in terms of the aboutness of the information objects. This relation is system­ orientated to a large extent because the success of the relation depends on the system’s indexing and searching ability to retrieve relevant objects. However, success also depends on the formulation of the request by the user, transformed into a query by the system. The assumption is thus that both requests/queries and objects may be identified as being about the same or similar topic(s) .
Cognitive relevance or pertinence is described by a relation between the state of knowledge or the cognitive information need of the user and the information objects. Cognitive correspondence, informativeness, novelty, information quality, and the like are the criteria by which cognitive relevance is inferred. This relation encompasses both system and user, because the success of the relation depends on the system’s indexing and searching ability to retrieve relevant information objects. However, success also depends on the formulation of the request (query) by the user. In this case, the user’s ability to formulate a request is dependent on his IR and conceptual knowledge background and his understanding or perception of his information need (Ingwersen, 1992). The cognitive relevance seems to be moving towards a user-orientated relevance and away from a system-orientated relevance.
This type of relevance was also described extensively by Barry (1994) when a study was undertaken to define the criteria mentioned by users’ evaluation of the information within documents as it related to the users’ information-need situations. The results showed that the criteria employed by the users included tangible (form or feature) characteristics of documents, as well as subjective qualities together with affective and situational factors.
Situational relevance or utility is the relation between the perceived situation , task or problem at hand and the information objects as perceived. Usefulness in decision-making, appropriateness of information in problem solving, and reducing uncertainty are criteria by which situational relevance is inferred. This relation encompasses both system and user, because the success of the relation depends on the system’s indexing and searching ability to retrieve relevant texts. However, it also depends largely on the user’s ability to use the information objects for a certain purpose within a given situation or context. The utility relevance seems to be moving towards the interaction between the environment or domain (the situation) and the individual actor. An example could be that the actor or user takes part in a peer reviewing process in which he assesses the usefulness and impact of applicants’ works concerning a faculty position . To assess the appropriateness of the contributions, the actor is influenced by the community culture and domain.

CHAPTER 1: INTRODUCTION 
1.1. The concept of relevance within the context of Information Science
1.2. The research problem
1.3. Aims, goals and research objectives – research questions
1.4. Methodology
1.5. Outline of thesis
CHAPTER 2: LITERATURE REVIEW 
2.1. Introduction: demarcation of literature covered
2.2. Relevance defined
2.3. The multidimensionality and dynamic nature of relevance
2.4. Research with implications for relevance
2.5. Summary of main conclusions based on literature review
CHAPTER 3: DEVELOPING A THEORETICAL FRAMEWORK 
3.1. Attributes of relevance
3.2. Manifestations of relevance
3.3. Attributes and manifestations of relevance: What are the connections?
3.4. The modified relevance model
3.5. Some consequences of relevance variety .
3.6. Relevance types
3.7. The contexts of relevance judgements in the information seeking process
3.8. Work task and search task as depicted in the model
3.9. User criteria for relevance judgments
3.10. Summary and conclusions
CHAPTER 4: RESEARCH DESIGN AND METHODOLOGY 
4.1. Defining the research question
4.2. Construction of the questionnaire
4.3. The questionnaire
4.4. Sample design and sampling methods
4.5. Data collection methods
4.6. Data capturing and data editing
4.7. Data analysis
4.8. Limitations of the methodology
4.9. Summary
CHAPTER 5: PRESENTATION AND DISCUSSION OF RESULTS 
5.1. Sample profiles
5.2. Research questions: Data analysis and results
5.3. Summary of findings
CHAPTER 6: CONCLUSIONS REGARDING MODELLING AND EMPIRICAL DATA 
6.1. Conclusions from the literature review
6.2. Conclusions regarding the model developed
6.3. Conclusions for future research
CHAPTER 7: IMPLICATIONS OF THE MODEL FOR IR RESEARCH 
7.1. Algorithmic relevance
7.2. Topicality
7.3. Cognitive relevance I pertinence
7.4. Situational relevance
7.5. Socio-cognitive relevance
7.6. Affective relevance
7.7. Conclusions
APPENDIX
GET THE COMPLETE PROJECT