ATTENTION: The software behind KU ScholarWorks is being upgraded to a new version. Starting July 15th, users will not be able to log in to the system, add items, nor make any changes until the new version is in place at the end of July. Searching for articles and opening files will continue to work while the system is being updated.
If you have any questions, please contact Marianne Reed at mreed@ku.edu .
Efficient Update of Indexes for Dynamically Changing Web Documents
dc.contributor.author | Lim, Lipyeow | |
dc.contributor.author | Wang, Min | |
dc.contributor.author | Padmanabhan, Sriram | |
dc.contributor.author | Vitter, Jeffrey Scott | |
dc.contributor.author | Agarwal, Ramesh | |
dc.date.accessioned | 2011-03-21T20:14:26Z | |
dc.date.available | 2011-03-21T20:14:26Z | |
dc.date.issued | 2007 | |
dc.identifier.citation | L. Lim, M. Wang, S. Padmanabhan, J. S. Vitter, and R. Agarwal. “Efficient Update of Indexes for Dynamically Changing Web Documents,” World Wide Web, 10(1), March 2007, 37–69. An extended abstract appears in “Dynamic Maintenance of Web Indexes Using Landmarks,” Proceedings of the 12th International World Wide Web Conference (WWW ’03), Budapest, May 2003, 102–111. http://dx.doi.org/10.1007/s11280-006-0009-2 | |
dc.identifier.uri | http://hdl.handle.net/1808/7225 | |
dc.description | The original publication is available at www.springerlink.com | |
dc.description.abstract | Recent work on incremental crawling has enabled the indexed document collection of a search engine to be more synchronized with the changing World Wide Web. However, this synchronized collection is not immediately searchable, because the keyword index is rebuilt from scratch less frequently than the collection can be refreshed. An inverted index is usually used to index documents crawled from the web. Complete index rebuild at high frequency is expensive. Previous work on incremental inverted index updates have been restricted to adding and removing documents. Updating the inverted index for previously indexed documents that have changed has not been addressed. In this paper, we propose an efficient method to update the inverted index for previously indexed documents whose contents have changed. Our method uses the idea of landmarks together with the diff algorithm to significantly reduce the number of postings in the inverted index that need to be updated. Our experiments verify that our landmark-diff method results in significant savings in the number of update operations on the inverted index. | |
dc.language.iso | en_US | |
dc.publisher | Springer Verlag | |
dc.subject | Web search | |
dc.subject | Indexing methods | |
dc.subject | Document management | |
dc.title | Efficient Update of Indexes for Dynamically Changing Web Documents | |
dc.type | Article | |
kusw.kuauthor | Vitter, Jeffrey Scott | |
kusw.oastatus | fullparticipation | |
dc.identifier.doi | 10.1007/s11280-006-0009-2 | |
kusw.oaversion | Scholarly/refereed, author accepted manuscript | |
kusw.oapolicy | This item meets KU Open Access policy criteria. | |
dc.rights.accessrights | openAccess |