Something doesn't quite add up. TopFieldCollector fieldCollector = TopFieldCollector.create(sort, max,true, > false, false, true); > > We use pagination, so only returning 1000 documents or so at a time. > > You say you are using pagination, yet the API you are using to create your collector isn't how you would utilize Lucene's built-in "pagination" feature (unless misunderstand the API). If the max is the snippet above is 1000, then you're simply returning top 1000 docs every time you execute your search. Otherwise... well, could you actually post a bit more of your code that runs the search here, in particular?
Assuming that the max is much larger than 1000, however, you could call fieldCollector.topDocs(int, int) after accumulating hits using this collector, but this won't work multiple times per query execution, according to the javadoc. So you either have to re-execute the full search, and then get the next chunk of ScoreDocs, or use the proper API for this, one that accepts as a parameter the end of the previous page of results, i.e. IndexSearcher.searchAfter(ScoreDoc, ...)
