OpenAIR @ RGU >
Design and Technology >
Computing >
Journal articles (Computing) >

Please use this identifier to cite or link to this item:
This item has been viewed 4 times in the last year. View Statistics

Files in This Item:

File Description SizeFormat
Song ACM TOIS 2001.pdf302.11 kBAdobe PDFView/Open
Title: Application of aboutness to functional benchmarking in information retrieval.
Authors: Wong, Kam-Fai
Song, Dawei
Bruza, Peter D.
Cheng, Chin-Hung
Keywords: Functional benchmarking
Logic-based information retrieval
Inductive evaluation
Issue Date: Oct-2001
Publisher: ACM
Citation: WONG, K. F., SONG, D., BRUZA, P. D. and CHENG, C. H., 2001. Application of aboutness to functional benchmarking in information retrieval. ACM Transactions on Information Systems, 19 (4), pp. 337-370.
Abstract: Experimental approaches are widely employed to benchmark the performance of an information retrieval (IR) system. Measurements in terms of recall and precision are computed as performance indicators. Although they are good at assessing the retrieval effectiveness of an IR system, they fail to explore deeper aspects such as its underlying functionality and explain why the system shows such performance. Recently, inductive (i.e., theoretical) evaluation of IR systems has been proposed to circumvent the controversies of the experimental methods. Several studies have adopted the inductive approach, but they mostly focus on theoretical modeling of IR properties by using some meta-logic. In this paper, we propose to use inductive evaluation for functional benchmarking of IR models as a complement of the traditional experimental based performance benchmarking. We define a functional benchmark suite in two stages: (a) the evaluation criteria based on the notion of “aboutness”; and (b) the formal evaluation methodology using the criteria. The proposed benchmark has been successfully applied to evaluate various well-known classical and logicbased IR models. The functional benchmarking results allow us to compare and analyze the functionality of the different IR models.
ISSN: 1046-8188
Appears in Collections:Journal articles (Computing)

All items in OpenAIR are protected by copyright, with all rights reserved.


   Disclaimer | Freedom of Information | Privacy Statement |Copyright ©2012 Robert Gordon University, Garthdee House, Garthdee Road, Aberdeen, AB10 7QB, Scotland, UK: a Scottish charity, registration No. SC013781