Sung-Jin Kim, Journal of Internet Computing and Services, Vol. 8, No. 4, pp. 149-158, Aug. 2007
Full Text:
Keywords: web database, web page change, web evolution

Abstract

This paper presents the estimation methods computing the probabilities of how many times web pages are downloaded and modified, respectively, in the future crawls. The methods can make web database administrators avoid unnecessarily requesting undownloadable and unmodified web pages in a page group. We postulated that the change behavior of web pages is strongly related to the past change behavior. We gather the change histories of approximately three million web pages at two-day intervals for 100 days, and estimated the future change behavior of those pages. Our estimation, which was evaluated by actual change behavior of the pages, worked well.

Statistics
Cite this article