Re: Multiple merge-runs from same set of segments

2021-05-27 Thread Patrick Zhai
Sorry for the delayed response, as for caching termDict data across threads, I do not aware of any existing lucene mechanism could do that (and it might be tricky since it is across threads), but maybe worth trying to see whether we can get some extra speed based on that! Patrick Ravikumar Govind

Re: Lucene/Solr and BERT

2021-05-27 Thread Julie Tibshirani
Your summary sounds right to me. There are some ideas (being discussed on the issue), but I don't think we have a detailed understanding yet of the performance difference. It would be great to get more eyes on the benchmark if you're interested in double-checking the results. Mike mentioned that h

Re: Hierarchical facet select a subtree but one child

2021-05-27 Thread nbuso
Hi, yes my aim was to introduce the functionality on the facet package. I created a ticket and added a simple patch; the use case seems to apply only to hierarchical facets, maybe we can add a validation to avoid to use the method in other cases. I'm happy to make modifications to the patch i

Re: Index backwards compatibility

2021-05-27 Thread Michael Wechner
good point! I have changed it accordingly https://cwiki.apache.org/confluence/display/LUCENE/LuceneFAQ#LuceneFAQ-WhenIupradeLucene,forexamplefrom8.8.2to9.0.0,doIhavetoreindex? Hope it is clear now :-) Am 27.05.21 um 16:39 schrieb Michael Sokolov: LGTM, but perhaps also should state that if pos

Re: Index backwards compatibility

2021-05-27 Thread Michael Sokolov
... should *reindex* ( not update ) On Thu, May 27, 2021 at 10:39 AM Michael Sokolov wrote: > > LGTM, but perhaps also should state that if possible you *should* > update because the 8.x index may not be able to be read by the > eventual 10 release. > > On Thu, May 27, 2021 at 7:52 AM Michael We

Re: Index backwards compatibility

2021-05-27 Thread Michael Sokolov
LGTM, but perhaps also should state that if possible you *should* update because the 8.x index may not be able to be read by the eventual 10 release. On Thu, May 27, 2021 at 7:52 AM Michael Wechner wrote: > > I have added a QnA > > https://cwiki.apache.org/confluence/display/LUCENE/LuceneFAQ#Luce

Re: Index backwards compatibility

2021-05-27 Thread Michael Wechner
I have added a QnA https://cwiki.apache.org/confluence/display/LUCENE/LuceneFAQ#LuceneFAQ-WhenIupradeLucene,forexamplefrom8.8.2to9.0.0,doIhavetoreindex? Hope that makes sense, otherwise let me know and I can correct/update :-) Am 26.05.21 um 23:56 schrieb Michael Wechner: using lucene-backwa

Re: Lucene/Solr and BERT

2021-05-27 Thread Michael Wechner
Thank you very much for having done these benchmarks! IIUC one could state - Indexing:   Lucene is slower than hnswlib/C++, very roughly 10x performance difference - Searching (Queries per second):   Lucene is slower than hnswlib/C++, very roughly 8x performance difference right, bu