Ah, of course. Thanks Mikhail. I realized it was a silly question that only made sense to me since my query was MatchAll docs.
-----Original Message----- From: Mikhail Khludnev <m...@apache.org> Sent: Monday, November 7, 2022 2:44 AM To: java-user@lucene.apache.org Subject: [EXT] Re: Efficient sort on SortedDocValues External Email: Use caution with links and attachments. Hello, Andrei. Docs are scored in-order (see Weight.scoreAll(), scoreRange()), just because underneath postings API is in-order. There are a few shortcuts/optimizations, but they only omit some iterations/segments like checking competitive scores and so one. On Sun, Nov 6, 2022 at 1:35 AM Solodin, Andrei (TR Technology) <andrei.solo...@thomsonreuters.com.invalid> wrote: > One more thing. While the test case passes now, it still iterates in > index order. Which means that it still collects ~6.4K docs out of 10k matches. > This is an improvement, but I am still wondering why it's not possible > to iterate in the field older. Seems like that would provide > substantial improvement. > > From: Solodin, Andrei (TR Technology) > Sent: Saturday, November 5, 2022 5:18 PM > To: java-user@lucene.apache.org > Subject: RE: Efficient sort on SortedDocValues > > I just realized that the problem is that the field needs to be indexed > as well. Now it works. But I noticed that this only works in Lucene 9. > Does not work in Lucene 8 (specifically 8.11.2). This must be new > functionality in Lucene 9? > > Thanks > > > From: Solodin, Andrei (TR Technology) > Sent: Saturday, November 5, 2022 1:07 PM > To: java-user@lucene.apache.org<mailto:java-user@lucene.apache.org> > Subject: Efficient sort on SortedDocValues > > Hello Lucene community, while looking into how to efficiently sort on > a field value, I came across a couple of things that I don't quite > understand. My assumption was that if I execute a search and sort on a > SortedDocValues field, lucene would only iterate over the docs in the > order of the field values or at least collect only competitive docs > (docs that made it into the topN queue). Neither of those things seems > to be happening. Instead, the iteration is happening in index order > and all matched docs are collected. Looking at the code, I see that > the optimizations are only possible if the index is sorted in the > field order to begin with, which is not possible for our use case. We > may have dozens of such fields in our index, thus there isn't any one > field that can be used to sort the index. So I guess my question if > what I am trying to achieve is possible? I tried to look though Solr > codebase, but so far couldn't come up with anything. Code example is here > https://urldefense.com/v3/__https://pastebin.com/i05E2wZy__;!!GFN0sa3rsbfR8OLyAw!c95-xVshQzkFU6u9GzqxAvTcVnO4j12lbVHEgMELOpR9AM2IQAtbX5NMIKynd2k2IkdjRAhIWUx2uOnWEm-cjg$ > . I am using 9.4.1. Thanks in advance. > > Andrei > > -- Sincerely yours Mikhail Khludnev
