An Evaluation of Information Retrieval
Accuracy with Simulated OCR Output
W. B. Croft and S. Harding
Computer Science Department
University of Massachusetts, Amherst
K. Taghva and J. Borsack
Information Science Research Institute
University of Nevada, Las Vegas
Optical Character Recognition (OCR) is a critical part of many text-based applications. Although some commercial systems use the output from OCR devices to index documents without editing, there is very little quantitative data on the impact of OCR errors on the accuracy of a text retrieval system. Because of the difficulty of constructing test collections to obtain this data, we have carried out evaluations using simulated OCR output on a variety of databases. The results show that high quality OCR devices have little effect on the accuracy of retrieval, but low quality devices used with databases of short documents can result in significant degradation.
Text-based information systems have become increasingly important in business, government, and academia. In many applications, the source of the text is not documents from word processors, but instead documents in their original paper form. Although imaging systems provide a simple means of storing these documents and retrieving them through manually assigned keywords, full-text access will in general be much more effective. In order to get from paper documents to full-text retrieval, OCR will be a crucial part of the process.
For printed documents, OCR techniques can recognize words with a high level
of accuracy. To produce output that is suitable for display, a significant amount
of human editing is needed. For automatic indexing and retrieval, however, the
OCR word accuracy may be sufficient. Some text retrieval systems have taken this
approach, combining OCR for indexing and imaging for display.
From an information retrieval point of view, the main issue is the impact of OCR