Query-Based Sampling using Snippets
Tigelaar, A. S. & Hiemstra, D.
In Proceedings of LSDS-IR 2010, Geneva, Switzerland (pp. 9-14).
Abstract
Query-based sampling is a commonly used approach to model the content of servers. Conventionally, queries are sent to a server and the documents in the search results returned are downloaded in full as representation of the server’s content. We present an approach that uses the document snippets in the search results as samples instead of downloading the entire documents. We show this yields equal or better modeling performance for the same bandwidth consumption depending on collection characteristics, like document length distribution and homogeneity. Query-based sampling using snippets is a useful approach for real-world systems, since it requires no extra operations beyond exchanging queries and search results.
Presented at the Large-Scale Distributed Systems for Information Retrieval Workshop
on July 23rd in Geneva, Switzerland.
