Inconsistent ordering of results

2023-01-11 Thread Peter Lancaster
We are using solr 7.7.3 and have a collection with 20 shards each with 4 replicas. We use the default BM25 similarity algorithm for scoring. For paging through search results we would like the sort order to be deterministic to present consistent results and avoid skipping or duplicating results

Re: Inconsistent ordering of results

2023-01-11 Thread Mikhail Khludnev
Hello, Peter. Why don't you use Exact*StatsCache? I always thought that they could solve this problem. Also, I've found https://issues.apache.org/jira/browse/SOLR-13257 about introducing replica.base in 9.0. I'm not sure if it's a solution. On Wed, Jan 11, 2023 at 12:21 PM Peter Lancaster < peter.

Quoted phrase doesn't match when stemming and synonyms combined.

2023-01-11 Thread Mateusz Matela
Hi, My query is 'test polskie'. I use MorfologikFilter for Polish stemming, it turns 'polskie' into 'polski' + 'polskie'. I also use SynonymGraphFilter which turns 'polski' into 'pol'. Here's what I see in quey analysis (token position in parenthesis): Tokenizer: test(1) polskie(2) MF: test(1)

Re: Quoted phrase doesn't match when stemming and synonyms combined.

2023-01-11 Thread Dave
Hmm. As an experiment what happens when you use a range of three or four with the quotes using the tilda in the query? Also generally o find it best to use the same filters for both indexing and query, just a personal preference, I know it’s not always possible however. > On Jan 11, 2023, at 5

Re: Quoted phrase doesn't match when stemming and synonyms combined.

2023-01-11 Thread Mateusz Matela
W dniu 11.01.2023 o 12:04, Dave pisze: Hmm. As an experiment what happens when you use a range of three or four with the quotes using the tilda in the query? You mean query like "test polskie"~1 ? Yes, it does match. Unfortunately it's not a workaround I can use because the query is provided

Re: Quoted phrase doesn't match when stemming and synonyms combined.

2023-01-11 Thread Dave
Yes then that is a problem, and I agree it should be intuitive that the quotes work without the modifier. I’m not familiar with the underlying code enough to know for sure what’s going on in this instance, but reinfecting the content with the filter I wonder would fix it? You can experiment wit

RE: Inconsistent ordering of results

2023-01-11 Thread Peter Lancaster
Hi Mikhail, Thanks for the quick reply. Just to say we've now tried the ExactStatsCache/ ExactSharedStatsCache options but neither seems to help with the different docCounts/scores that are seen for different replicas. The link you posted looks more promising as it may solve the issue and impr

Re: Inconsistent ordering of results

2023-01-11 Thread Mikhail Khludnev
Searched a little bit more https://issues.apache.org/jira/browse/SOLR-13790?focusedCommentId=16942908&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16942908 https://stackoverflow.com/questions/55582874/exactstatscache-not-working-for-distributed-idf On Wed, Jan 1

Re: Inconsistent ordering of results

2023-01-11 Thread Markus Jelsma
Hello Peter, We had the same problem many years ago, replica's of the same shard having different stats. It was solved by introducing ExactStatsCache, but it was a little bit more slower, bit not too much. When Solr introduced new replica types we switched all shards from NRT, to TLOG. TLOG replic

RE: Inconsistent ordering of results

2023-01-11 Thread Peter Lancaster
Hi Markus, Thank-you for your response. I had forgotten to include that we are using NRT rather than TLOG replicas. It sounds like switching to TLOGS is exactly the right thing to do to fix this. Thanks again for your help. Peter. -Original Message- From: Markus Jelsma Sent: 11 Januar

Re: Quoted phrase doesn't match when stemming and synonyms combined.

2023-01-11 Thread Mateusz Matela
After reindexing with SGF the document matches, as expected. Still, it looks like SGF was designed to work well when used only in query, and it's just a bug revealed by an edge case. Shall I submit an issue to https://github.com/apache/lucene ? W dniu 11.01.2023 o 13:09, Dave pisze: Yes then

Re: Quoted phrase doesn't match when stemming and synonyms combined.

2023-01-11 Thread Dave
On one hand that’s great news, on the other ot probably deserves a ticket but you need to have a very specific scenario where your index filters don’t match your query filters. Also maybe spend some time putting together a reindexing plan. Solr can use multiple cores so you can index content

Error CREATEing SolrCore caused by solr.XSLTResponseWriter

2023-01-11 Thread cla
Hi all, I'm running a fresh Solr 9.1.0 on a fresh Debian 11 PC. Creating my first core (the first on this stack, not the first in my life...) finally fails with Error CREATEing SolrCore 'core_xxx': Unable to create core [core_xxx] Caused by: solr.XSLTResponseWriter Thanks for your advice whe

Re: Error CREATEing SolrCore caused by solr.XSLTResponseWriter

2023-01-11 Thread Markus Jelsma
Hello Walter, That is odd. Check the logs to see why the XSLT response writer is causing trouble. If you don't need it, you can just disable the response writer in the solrconfig.xml. Regards, Markus Op wo 11 jan. 2023 om 16:13 schreef : > Hi all, > > I'm running a fresh Solr 9.1.0 on a fresh D

What does this error mean?

2023-01-11 Thread Matthew Castrigno
Hello, I am getting the error described here: https://github.com/sepinf-inc/IPED/issues/1007 Has a fix for this been incorporated into 9.0? Any other information on what causes this error? Thank you, [cid:4e3a7e07-fdca-4746-be49-4a956b41553b] Matthew Castrigno IHT Developer II St. Luke’s Heal

Does CVE-2022-46364 affect Solr 7.3.1

2023-01-11 Thread Wesley Philip
Hello, Mend security scan has flagged cxf-core-3.4.3.jar with CVE-2022-46364. I believe this jar is pulled in as a dependency of Solr 7.3.1. I'm wondering if Solr is truly vulnerable to this issue. Thanks, Wesley ::DISCLAIMER:: The contents of this e-mail an

Re: Inconsistent ordering of results

2023-01-11 Thread Walter Underwood
Are these documents ties, with the exact same scores? Those can be ordered differently on different replicas. Using global IDF won’t fix that, plus that was 10x slower when we tried it. We fixed this by adding a sort by score, then id. The id is the same on all replicas, so that gives consisten

Solrj xml response parse fails when nested

2023-01-11 Thread Ron Haines
Seems like when I provide a 'children:[subquery]' in my &fl, and the xml response now includes a nested element, the XMLResponseParser.java throws a 'parsing error', Caused by: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[1,892] Message: must be value or array Is there a known is

RE: Inconsistent ordering of results

2023-01-11 Thread Peter Lancaster
Hi Walter, Thanks for the reply. No our documents are typically quite long (maybe 500 terms on average but varying a lot) and documents are getting slightly different scores from different replicas for the same query. The explain output gives slightly different document counts hence the differ

Re: Solrj xml response parse fails when nested

2023-01-11 Thread Mikhail Khludnev
Hi, Ron. Right. Never thought of that. It might be an issue. Feel free to raise one. On Wed, Jan 11, 2023 at 8:36 PM Ron Haines wrote: > Seems like when I provide a 'children:[subquery]' in my &fl, and the xml > response now includes a nested element, the > XMLResponseParser.java throws a 'pars

Re: Does CVE-2022-46364 affect Solr 7.3.1

2023-01-11 Thread Shawn Heisey
On 1/11/23 09:44, Wesley Philip wrote: Hello, Mend security scan has flagged cxf-core-3.4.3.jar with CVE-2022-46364. I believe this jar is pulled in as a dependency of Solr 7.3.1. I'm wondering if Solr is truly vulnerable to this issue. I don't see any file with "cxf" in its name (checked

FileBasedSpellChecker always returns "correctlySpelled": false

2023-01-11 Thread gnandre
Hi, I am using Solr 8.5.2 and when I use FileBasedSpellChecker, I always get "correctlySpelled": false even if the spellcheck.q value exists in the file. Apart from that it is working as expected. When I give slightly misspelled term to spellcheck.q param, it returns the valid suggestion present

Solr 9.1 Admin page not opening

2023-01-11 Thread Anuj Bhargava
Solr 9.1 has been installed. And java is running [root@76 ~]# /etc/init.d/solr status Found 1 Solr nodes: Solr process 2858 running on port 8983 [root@76 ~]# java -version openjdk version "11.0.17" 2022-10-18 LTS OpenJDK Runtime Environment (Red_Hat-11.0.17.0.8-2.el7_9) (build 11.0.17+8-LTS) Open

Re: Solr 9.1 Admin page not opening

2023-01-11 Thread Anuj Bhargava
Sorry using the following http://xxx.xxx.xxx.76:8983/solr/admin http://xxx.xxx.xxx.76:8983/solr/ and not as mentioned earlier - http://xxx.xxx.xxx.76:8293/solr/admin http://xxx.xxx.xxx.76:8293/solr/ On Thu, 12 Jan 2023 at 13:16, Anuj Bhargava wrote: > Solr 9.1 has been installed. And java is r