Solr relevancy score different on replicated nodes

[AshB]

Version Solr 7.4.0 zookeeper 3.4.11 Achitecture Two boxes Machine-1,Machine-2
holding single instances of solr

We are having a collection which was single shard and single replica i.e s=1
and rf=1

Few days back we tried to add replica to it.But the score for same query is
coming different from different replicas.

http://Machine-1:8983/solr/MyTestCollection/select?q=%22data%22+OR+(data)=10=score=edismax=search_field+content=json

"response":{"numFound":5836,"start":0,"maxScore":*4.418847*,"docs":[

whereas on another machine(replica)

http://Machine-2:8983/solr/MyTestCollection/select?q=%22data%22+OR+(data)=10=score=edismax=search_field+content=json

"response":{"numFound":5836,"start":0,"maxScore":*4.4952264*,"docs":[

The maxScore is different.

Relevancy gets affected due to sharding but replication was not expected as
same documents get copied to other node. score explaination gives issue with
docCount and docFreq uneven.

idf, computed as log(1 + (docCount - docFreq + 0.5) / (docFreq + 0.5)) from:
1.050635000 docCount :*10020.0* docFreq :*3504.000*

idf, computed as log(1 + (docCount - docFreq + 0.5) / (docFreq + 0.5)) from:
1.068795100

docCount :*10291.0* docFreq :*3534.000*

Is this expected?What could be wrong here?Please suggest



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Zookeeper timeout issue -

[AshB]

Hi Shawn,

Answers to your questions.

1.Yes we are aware of fault tolerance in our architecture,but its our dev
env,so we are working with solrCloud mode with limited machines.

2. Solr is running as separate app,its not on weblogic. We are using
Weblogic for rest services which further connect to zookeeper<-->Solr.

3.We used jconsole to monitor solr,zookeeper and weblogic process.In the
weblogic process looks like threads are getting stuck. One such thread
related to zookeeper is as below..

Name: zkConnectionManagerCallback-9207-thread-1
State: WAITING on
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@396cda76
Total blocked: 0  Total waited: 1

Stack trace:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)

Have attached file containing snapshots of process.

Also attached the solr  GCeasy-report-gc.pdf
  gc
log report  TimoutIssue.docx
  of solr
during the load activity.







--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Zookeeper timeout issue -

[AshB]

Hi Dominique,

Yes,we are load testing with 50 users.We tried changing the timeout but its
not reflecting.

Regards
Ashish



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: SOLR zookeeper connection timeout during startup is hardcoded to 10000ms

[AshB]

Can this timeout value be changed .

Regards
Ashish



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Zookeeper timeout issue -

[AshB]

Hi Jan,

Setup Details

Mach-1 -->20Gb RAM.
Apps running :OracleDb,WeblogicServer(services deployed to call
solr),*OneSolr Node*,*One Zookeeper node*
Mach-2 -->20Gb RAM
Apps running :*One Solr Node*,*Two zookeeper nodes*.

Solr collection details : ~8k docs,~140MB size on disc,One shard on machine
1 and two replicas on mach-1 and mach-2

We did a jmeter load testing with 50 users 30 iterations i.e 1500
requests.In each call solr is called three times due to requirements.

What we noticed is when load on mach-1 goes high upto ~12 and memory
utilization goes high and then some requests time out.

Is this expected from zookeeper when load is too high?

-Ashish





--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Zookeeper timeout issue -

[AshB]

Hi,

We are facing issue with solr/zookeeper where zookeeper timeouts after
1ms. Error below.

*SolrException: java.util.concurrent.TimeoutException: Could not connect to
ZooKeeper :9181,:9182,:9183 within 1 ms.
at org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:184)
at org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:121)
at org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:111)
at
org.apache.solr.common.cloud.ZkStateReader.(ZkStateReader.java:295)*

We are not getting any error in zookeeper logs.Except below logs
2018-12-19 04:35:22,305 [myid:2] - INFO 
[SessionTracker:ZooKeeperServer@354] - Expiring session 0x200830234de3127,
timeout of 1ms exceeded
2018-12-19 05:35:38,304 [myid:2] - INFO 
[SessionTracker:ZooKeeperServer@354] - Expiring session 0x200b4f912730086,
timeout of 1ms exceeded
2018-12-19 05:56:58,302 [myid:2] - INFO 
[SessionTracker:ZooKeeperServer@354] - Expiring session 0x100b4f9125e00bf,
timeout of 1ms exceeded


During the issue threads go high and we could notice below in weblogic
server.

Name: Connection evictor
State: TIMED_WAITING
Total blocked: 0  Total waited: 1

Stack trace: 
java.lang.Thread.sleep(Native Method)
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
java.lang.Thread.run(Thread.java:748)

What could be going wrong here?

Regards
Ashish





--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Phrase query as feature in LTR not working

[AshB]

Phrase query is not working when applied in LTR.

Feature supplied is
 {
"name" : "isPook",
"class" : "org.apache.solr.ltr.feature.SolrFeature",
"params" : {
  "fq": ["{!type=edismax qf=text v=$qq}=\"${query}\""]
}
  }

Tested this feature outside and it returns only one result ,i.e phrase but
with LTR it is matching on terms

http://localhost:8983/solr/techproducts/query?q=game%20of%20thrones=id,name,[features%20*efi.query=thrones%20of%20game*],name,cat=true

"response":{"numFound":6,"start":0,"docs":[
  {
"id":"05535734023",
"cat":["book"],
"name":"A Thrones of Game",
   
"[features]":"documentRecency=0.02011838,isBook=1.0,*isPook=1.0*,originalScore=8.337603"},
  {
"id":"05535734021",
"cat":["book"],
"name":"A Game of meeting Thrones",
   
"[features]":"documentRecency=0.02011838,isBook=1.0,*isPook=1.0*,originalScore=8.179235"},

How to set the feature so that it score only the first document containing
the phrase




--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Indexed=false for a field,but still able to search on field.

[AshB]

Hi,

Thanks ,got this issue is happening because of docValues=true.

Please elaborate on "full table scan search"

Regards
Ashish



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Indexed-false-for-a-field-but-still-able-to-search-on-field-tp4352338p4352599.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Indexed=false for a field,but still able to search on field.

[AshB]

Hi,

Yes docValues is true for fieldType







--
View this message in context: 
http://lucene.472066.n3.nabble.com/Indexed-false-for-a-field-but-still-able-to-search-on-field-tp4352338p4352442.html
Sent from the Solr - User mailing list archive at Nabble.com.

Indexed=false for a field,but still able to search on field.

[AshB]

Hi,

I created a field as,expecting I won't be able to search on it 

.

But i am able to search on it.Sample query below

fileName:"ipgb20080916_1078.xml"

What is wrong here.I am not doing any copy of this field

Solrversion:6.5.1



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Indexed-false-for-a-field-but-still-able-to-search-on-field-tp4352338.html
Sent from the Solr - User mailing list archive at Nabble.com.