dedupe doen't work on solr cloud with router field

[Luke]

I have one collection, 3 shards, 2 replicas, I defined route field: title,
and ID is the unique key.

I index two document with same ID and different title, I configured dedupe
chain and I can see signature is generated, but the old document was
removed by solr, please help, thanks

Re: Apache Solr in High Availability Primary and Secondary node.

[Kaushal Shriyan]

Hi,

Checking in again if someone can pitch in for my earlier post to this
mailing list? Thanks in Advance.

Best Regards,

On Tue, Jan 12, 2021 at 8:30 AM Kaushal Shriyan 
wrote:

>
>
> On Tue, Jan 12, 2021 at 12:10 AM Dmitri Maziuk 
> wrote:
>
>> On 1/11/2021 12:30 PM, Walter Underwood wrote:
>> > Use a load balancer. We’re in AWS, so we use an AWS ALB.
>> >
>> > If you don’t have a failure-tolerant load balancer implementation, the
>> site has bigger problems than search.
>>
>> That is the point, you have amazon doing that for you, some of us do it
>> ourselves, and it wasn't clear (to me anyway) if OP was asking about that.
>>
>> Dima
>>
>
> Hi,
>
> Thanks for all the suggestions. I am hosting my Solr search service in
> GCP. I have a follow-up question regarding Solr Nodes. Do I need to have a
> Single Master and Multiple Slaves? I am using GCP Internal Load Balancer (
> https://cloud.google.com/load-balancing/docs/l7-internal).
>
> Internal LB -> Master Node1 and Master Node2. Master Node1 will have Slave
> 1 and Master Node2 will have Slave2 as per the below diagram as an example.
> Please suggest further and correct me if the approach is incorrect. I am
> not sure how do I replicate indices when I use Google Compute Platform
> Internal LB.
>
> [image: image.png]
>
>
> Thanks in Advance.
>
> Best Regards,
>
> Kaushal
>

Re: Re:Interpreting Solr indexing times

[Alessandro Benedetti]

I agree, documents may be gigantic or very small,  with heavy text analysis
or simple strings ...
so it's not possible to give an evaluation here.
But you could make use of the nightly benchmark to give you an idea of
Lucene indexing speed (the engine inside Apache Solr) :

http://home.apache.org/~mikemccand/lucenebench/indexing.html

Not sure we have something similar for Apache Solr officially.
https://cwiki.apache.org/confluence/display/SOLR/SolrPerformanceData -> this
should be a bit outdated

Cheers



-
---
Alessandro Benedetti
Search Consultant, R Software Engineer, Director
Sease Ltd. - www.sease.io
--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: leader election stuck after hosts restarts

[Alessandro Benedetti]

I faced these problems a while ago, but at the time I created a blog post
which I hope could help:
https://sease.io/2018/05/solrcloud-leader-election-failing.html



-
---
Alessandro Benedetti
Search Consultant, R Software Engineer, Director
Sease Ltd. - www.sease.io
--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: QueryResponse ordering

[Alessandro Benedetti]

Hi Srinivas,
Filter queries don't impact scoring but only matching.
So, what is the ordering you are expecting?
A bq (boost query) parameter will add a clause to the query, impacting the
score in an additive way.
The query you posted is a bit confused, what was your intent there?
To boost search results having "abc" as the PARTY.PARTY.ID ?
https://lucene.apache.org/solr/guide/6_6/the-dismax-query-parser.html#TheDisMaxQueryParser-Thebq_BoostQuery_Parameter



-
---
Alessandro Benedetti
Search Consultant, R Software Engineer, Director
Sease Ltd. - www.sease.io
--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

RE: Query over migrating a solr database from 7.7.1 to 8.7.0

[Dyer, Jim]

I think if you have _root_ in schema.xml you should look elsewhere.  My memory 
is merely adding this one line to schema.xml took care of our problem.

From: Flowerday, Matthew J 
Sent: Tuesday, January 12, 2021 3:23 AM
To: solr-user@lucene.apache.org
Subject: RE: Query over migrating a solr database from 7.7.1 to 8.7.0

Hi Jim

Thanks for getting back to me.

I checked the schema.xml that we are using and it has the line you mentioned:



And this is the only reference (apart from within a comment) for _root_ In the 
schema.xml. Does your schema.xml have further references to _root_ that I could 
need? I also checked out solrconfig.xml file for any references to _root_ and 
there are none.

Many Thanks

Matthew

Matthew Flowerday | Consultant | ULEAF
Unisys | 01908 774830| 
matthew.flower...@unisys.com
Address Enigma | Wavendon Business Park | Wavendon | Milton Keynes | MK17 8LX

[unisys_logo]

THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is for use only by the intended recipient. If you received this in 
error, please contact the sender and delete the e-mail and its attachments from 
all devices.
[Grey_LI]  [Grey_TW] 
  [Grey_YT] 
 [Grey_FB] 
 [Grey_Vimeo]  
[Grey_UB] 

From: Dyer, Jim 
mailto:james.d...@ingramcontent.com>>
Sent: 11 January 2021 22:58
To: solr-user@lucene.apache.org
Subject: RE: Query over migrating a solr database from 7.7.1 to 8.7.0

EXTERNAL EMAIL - Be cautious of all links and attachments.
When we upgraded from 7.x to 8.x, I ran into an issue similar to yours:  when 
updating an existing document in the index, the document would be duplicated 
instead of replaced as expected.  The solution was to add a "_root_" field to 
schema.xml like this:



It appeared that when a feature was added for nested documents, this field 
somehow became mandatory in order for updates to work properly, at least in 
some cases.

From: Flowerday, Matthew J 
mailto:matthew.flower...@gb.unisys.com>>
Sent: Saturday, January 9, 2021 4:44 AM
To: solr-user@lucene.apache.org
Subject: RE: Query over migrating a solr database from 7.7.1 to 8.7.0

Hi There

As a test I stopped Solr and ran the IndexUpgrader tool on the database to see 
if this might fix the issue. It completed OK but unfortunately the issue still 
occurs - a new version of the record on solr is created rather than updating 
the original record.

It looks to me as if the record created under 7.7.1 is somehow not being 
'marked as deleted' in the way that records created under 8.7.0 are. Is there a 
way for these records to be marked as deleted when they are updated.

Many Thanks

Matthew


Matthew Flowerday | Consultant | ULEAF
Unisys | 01908 774830| 
matthew.flower...@unisys.com
Address Enigma | Wavendon Business Park | Wavendon | Milton Keynes | MK17 8LX

[unisys_logo]

THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is for use only by the intended recipient. If you received this in 
error, please contact the sender and delete the e-mail and its attachments from 
all devices.
[Grey_LI]  [Grey_TW] 
  [Grey_YT] 
 [Grey_FB] 
 [Grey_Vimeo]  
[Grey_UB] 

From: Flowerday, Matthew J 
mailto:matthew.flower...@gb.unisys.com>>
Sent: 07 January 2021 12:25
To: solr-user@lucene.apache.org
Subject: Query over migrating a solr database from 7.7.1 to 8.7.0

Hi There

I have recently upgraded a solr database from 7.7.1 to 8.7.0 and not wiped the 
database and re-indexed (as this would take too long to run on site).

On my local windows machine I have a single solr server 7.7.1 installation

I upgraded in the following manner


  *   Installed windows solr 8.7.0 on my machine in a different folder
  *   Copied the core related folder (holding conf, data, lib, core.properties) 
from 7.7.1 to the new 8.7.0 folder
  *   Brought up the solr
  *   Checked that queries work through the Solr Admin Tool and our application

This all worked fine until I tried to update a record which had been created 
under 7.7.1. Instead of marking the old record as deleted it effectively 
created a new copy of the record with the change in and left the old image as 
still visible. When I updated the record again it then correctly updated the 
new 8.7.0 version without leaving the old image behind. If I created a new 
record and then updated it the solr record would be updated correctly. 

Re: different score from different replica of same shard

[Walter Underwood]

Yes, check performance before turning on the stats cache in prod.

When we tested the LRUStatsCache in 6.6.2, searches were 11X slower.

It should be possible to do distributed IDF with little extra overhead.
Infoseek was doing that in 1995 and the patent on the technique has
expired.

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)

> On Jan 13, 2021, at 6:31 AM, Markus Jelsma  wrote:
> 
> Hallo Bernd,
> 
> I see the different replica types in the 7.1 [1] manual but not in the 6.6.
> ExactStatsCache should work in 6.6, just add it to solrconfig.xml, not the
> request handler [1]. It will slow down searches due to added overhead.
> 
> Regards,
> Markus
> 
> [1]
> https://lucene.apache.org/solr/guide/7_1/shards-and-indexing-data-in-solrcloud.html#types-of-replicas
> [2] https://lucene.apache.org/solr/guide/6_6/distributed-requests.html
> 
> Op wo 13 jan. 2021 om 15:11 schreef Bernd Fehling <
> bernd.fehl...@uni-bielefeld.de>:
> 
>> Hello Markus,
>> 
>> thanks a lot.
>> Is TLOG also for SOLR 6.6.6 or only 8.x and up?
>> 
>> I will first try ExactStatsCache.
>> Should be added as invariant to request handler, right?
>> 
>> Comparing the replica index directories they have different size and
>> the index version and generation is different. Also Max Doc.
>> But Num Docs is the same.
>> 
>> Regards,
>> Bernd
>> 
>> 
>> Am 13.01.21 um 14:54 schrieb Markus Jelsma:
>>> Hello Bernd,
>>> 
>>> This is normal for NRT replicas, because the way segments are merged and
>>> deletes are removed is not synchronized between replicas. In that case
>>> counts for TF and IDF and norms become slightly different.
>>> 
>>> You can either use ExactStatsCache that fetches counts for terms before
>>> scoring, so that all replica's use the same counts. Or change the replica
>>> types to TLOG. With TLOG segments are fetched from the leader and thus
>>> identical.
>>> 
>>> Regards,
>>> Markus
>>> 
>>> Op wo 13 jan. 2021 om 14:45 schreef Bernd Fehling <
>>> bernd.fehl...@uni-bielefeld.de>:
>>> 
 Hello list,
 
 a question for better understanding scoring of a shard in a cloud.
 
 I see different scores from different replicas of the same shard.
 Is this normal and if yes, why?
 
 My understanding until now was that replicas are always the same within
>> a
 shard
 and the same query to each replica within a shard gives always the same
 score.
 
 Can someone help me to understand this?
 
 Regards
 Bernd
 
>>> 
>> 

Re: Cursor Performance Issue

[Mike Drob]

You should be using docvalues on your id, but note that switching this
would require a reindex.

On Wed, Jan 13, 2021 at 6:04 AM Ajay Sharma 
wrote:

> Hi All,
>
> I have used cursors to search and export documents in solr according to
>
> https://lucene.apache.org/solr/guide/6_6/pagination-of-results.html#fetching-a-large-number-of-sorted-results-cursors
>
> Solr version: 6.5.0
> No of Documents: 10 crore
>
> Before implementing cursor, I was using the start and rows parameter to
> fetch records
> Service response time used to be 2 sec
>
> *Before implementing Cursor Solr URL:*
> http://localhost:8080/solr/search/select?q=bird
> toy=mapping=3=25=100
>
> Request handler Looks like this: fl contains approx 20 fields
> 
> 
> edismax
> on
> 0.01
> 
> 
> id,refid,title,smalldesc:""
> 
>
> none
> json
> 25
> 15000
> smalldesc
> title_text
> titlews^3
> sdescnisq
> 1
> 
> 2-1 470%
> 
> 
>
> Sharing Response with EchoParams=all > Qtime is 6
> responseHeader: {
> status: 0,
> QTime: 6,
> params: {
> ps: "3",
> echoParams: "all",
> indent: "on",
> fl: "id,refid,title,smalldesc:"",
> tie: "0.01",
> defType: "edismax",
> qf: "customphonetic",
> wt: "json",
>qs: "1",
>qt: "mapping",
>rows: "25",
>q: "bird toy",
>timeAllowed: "15000"
> }
> },
> response: {
> numFound: 17,
> start: 0,
> maxScore: 26.616478,
> docs: [
>   {
> id: "22347708097",
> refid: "152585558",
> title: "Round BIRD COLOURFUL SWINGING CIRCULAR SITTING TOY",
> smalldesc: "",
> score: 26.616478
>  }
> ]
> }
>
> I am facing a performance issue now after implementing the cursor. Service
> response time is increased 3 to 4 times .i.e. 8 sec in some cases
>
> *After implementing Cursor query is-*
> localhost:8080/solr/search/select?q=bird
> toy=cursor=3=1000=100=score desc,id asc=*
>
> Just added =score desc,id asc=* to the before query and
> rows to be fetched is 1000 now and fl contains just a single field
>
> Request handler remains same as before just changed the name and made fl
> change and added df in defaults
>
> 
>
>   edismax
>   on
>   0.01
>
>
>   refid
>
>
>   none
>   json
>   1000
>   smalldesc
>   title_text
>   titlews^3
>   sdescnisq
>   1
>   2-1 470%
>   product_titles
>
> 
>
> Response with Cursor and echoParams=all-> *Qtime is now 17* i.e approx 3
> time of previous qtime
> responseHeader: {
> status: 0,
> QTime: 17,
> params: {
> df: "product_titles",
> ps: "3",
> echoParams: "all",
> indent: "on",
> fl: "refid",
> tie: "0.01",
> defType: "edismax",
> qf: "customphonetic",
> qs: "1",
> qt: "cursor",
> sort: "score desc,id asc",
> rows: "1000",
> q: "bird toy",
> cursorMark: "*",
> }
> },
> response: {
> numFound: 17,
> start: 0,
> docs: [
> {
> refid: "152585558"
> },
> {
> refid: "157276077"
> }
> ]
> }
>
>
> When i curl http://localhost:8080/solr/search/select?q=bird
> toy=mapping=3=25=100, i can get results in 3 seconds.
> When i curl localhost:8080/solr/search/select?q=bird
> toy=cursor=3=1000=100=score desc,id asc=* it
> consumed 8 seconds to return result even if the result count=0
>
> BTW, the id schema definition is used in sort
>  omitNorms="true" multiValued="false"/>
>
> Is it due to the sort I have applied or I have implemented it in the wrong
> way?
> Please help or provide the direction to solve this issue
>
>
> Thanks in advance
>
> --
> Thanks & Regards,
> Ajay Sharma
> Product Search
> Indiamart Intermesh Ltd.
>
> --
>
>

Re: different score from different replica of same shard

[Vincent Brehin]

Hallo Bernd und Markus,
A very instructive article, by the creator of TLOG mode (introduced in 7.0,
btw):
https://medium.com/@caomanhdat317/indexing-flow-of-solrcloud-sharding-distributed-systems-1-bba411bf8994
It helped me when architecting our replication policy.
Not an easy matter, it's a tradeoff between consistency and performance.
Solr's choice is to be eventually consistent, this "lag" is the price to
pay for scalability and fault tolerance.
TLOG are not as realtime as NRT (obviously) so it could be a problem too,
the replicas lag behind the leader (for score stats and results)... But
TLOG make your replicas eventually identical.
>From my experience, TLOG comes with a price when you have many replicas and
*a lot* of index updates: the leader is doing indexing and is constantly
replying to update requests from replicas -> high memory, thread
contention, bad situation...
Pull replica are safer (solr process has no extra load, only the
filesystem) but they can't be leader, so no fault tolerance here. And still
a lag between leader and replica.
You could maybe stick a search session to a specific replica, in order to
have consistent results for a given client, if it's important for your use
case.
Hope this helps. Regards,
Vincent

Le mer. 13 janv. 2021 à 15:31, Markus Jelsma  a
écrit :

> Hallo Bernd,
>
> I see the different replica types in the 7.1 [1] manual but not in the 6.6.
> ExactStatsCache should work in 6.6, just add it to solrconfig.xml, not the
> request handler [1]. It will slow down searches due to added overhead.
>
> Regards,
> Markus
>
> [1]
>
> https://lucene.apache.org/solr/guide/7_1/shards-and-indexing-data-in-solrcloud.html#types-of-replicas
> [2] https://lucene.apache.org/solr/guide/6_6/distributed-requests.html
>
> Op wo 13 jan. 2021 om 15:11 schreef Bernd Fehling <
> bernd.fehl...@uni-bielefeld.de>:
>
> > Hello Markus,
> >
> > thanks a lot.
> > Is TLOG also for SOLR 6.6.6 or only 8.x and up?
> >
> > I will first try ExactStatsCache.
> > Should be added as invariant to request handler, right?
> >
> > Comparing the replica index directories they have different size and
> > the index version and generation is different. Also Max Doc.
> > But Num Docs is the same.
> >
> > Regards,
> > Bernd
> >
> >
> > Am 13.01.21 um 14:54 schrieb Markus Jelsma:
> > > Hello Bernd,
> > >
> > > This is normal for NRT replicas, because the way segments are merged
> and
> > > deletes are removed is not synchronized between replicas. In that case
> > > counts for TF and IDF and norms become slightly different.
> > >
> > > You can either use ExactStatsCache that fetches counts for terms before
> > > scoring, so that all replica's use the same counts. Or change the
> replica
> > > types to TLOG. With TLOG segments are fetched from the leader and thus
> > > identical.
> > >
> > > Regards,
> > > Markus
> > >
> > > Op wo 13 jan. 2021 om 14:45 schreef Bernd Fehling <
> > > bernd.fehl...@uni-bielefeld.de>:
> > >
> > >> Hello list,
> > >>
> > >> a question for better understanding scoring of a shard in a cloud.
> > >>
> > >> I see different scores from different replicas of the same shard.
> > >> Is this normal and if yes, why?
> > >>
> > >> My understanding until now was that replicas are always the same
> within
> > a
> > >> shard
> > >> and the same query to each replica within a shard gives always the
> same
> > >> score.
> > >>
> > >> Can someone help me to understand this?
> > >>
> > >> Regards
> > >> Bernd
> > >>
> > >
> >
>

Re: different score from different replica of same shard

[Markus Jelsma]

Hallo Bernd,

I see the different replica types in the 7.1 [1] manual but not in the 6.6.
ExactStatsCache should work in 6.6, just add it to solrconfig.xml, not the
request handler [1]. It will slow down searches due to added overhead.

Regards,
Markus

[1]
https://lucene.apache.org/solr/guide/7_1/shards-and-indexing-data-in-solrcloud.html#types-of-replicas
[2] https://lucene.apache.org/solr/guide/6_6/distributed-requests.html

Op wo 13 jan. 2021 om 15:11 schreef Bernd Fehling <
bernd.fehl...@uni-bielefeld.de>:

> Hello Markus,
>
> thanks a lot.
> Is TLOG also for SOLR 6.6.6 or only 8.x and up?
>
> I will first try ExactStatsCache.
> Should be added as invariant to request handler, right?
>
> Comparing the replica index directories they have different size and
> the index version and generation is different. Also Max Doc.
> But Num Docs is the same.
>
> Regards,
> Bernd
>
>
> Am 13.01.21 um 14:54 schrieb Markus Jelsma:
> > Hello Bernd,
> >
> > This is normal for NRT replicas, because the way segments are merged and
> > deletes are removed is not synchronized between replicas. In that case
> > counts for TF and IDF and norms become slightly different.
> >
> > You can either use ExactStatsCache that fetches counts for terms before
> > scoring, so that all replica's use the same counts. Or change the replica
> > types to TLOG. With TLOG segments are fetched from the leader and thus
> > identical.
> >
> > Regards,
> > Markus
> >
> > Op wo 13 jan. 2021 om 14:45 schreef Bernd Fehling <
> > bernd.fehl...@uni-bielefeld.de>:
> >
> >> Hello list,
> >>
> >> a question for better understanding scoring of a shard in a cloud.
> >>
> >> I see different scores from different replicas of the same shard.
> >> Is this normal and if yes, why?
> >>
> >> My understanding until now was that replicas are always the same within
> a
> >> shard
> >> and the same query to each replica within a shard gives always the same
> >> score.
> >>
> >> Can someone help me to understand this?
> >>
> >> Regards
> >> Bernd
> >>
> >
>

SockerTimeoutException in long running streaming queries

[ufuk yılmaz]

When I performa a long running streaming expression, sometimes I get:

{
"error": {
"metadata": [
"error-class",
"org.apache.solr.common.SolrException",
"root-error-class",
"java.net.SocketTimeoutException"
],
"msg": "Error trying to proxy request for url:

Is this related to SOLR-13457, or is this a completely different thing?

Regards

Sent from Mail for Windows 10

Re: different score from different replica of same shard

[Bernd Fehling]

Hello Markus,

thanks a lot.
Is TLOG also for SOLR 6.6.6 or only 8.x and up?

I will first try ExactStatsCache.
Should be added as invariant to request handler, right?

Comparing the replica index directories they have different size and
the index version and generation is different. Also Max Doc.
But Num Docs is the same.

Regards,
Bernd


Am 13.01.21 um 14:54 schrieb Markus Jelsma:

Hello Bernd,

This is normal for NRT replicas, because the way segments are merged and
deletes are removed is not synchronized between replicas. In that case
counts for TF and IDF and norms become slightly different.

You can either use ExactStatsCache that fetches counts for terms before
scoring, so that all replica's use the same counts. Or change the replica
types to TLOG. With TLOG segments are fetched from the leader and thus
identical.

Regards,
Markus

Op wo 13 jan. 2021 om 14:45 schreef Bernd Fehling <
bernd.fehl...@uni-bielefeld.de>:


Hello list,

a question for better understanding scoring of a shard in a cloud.

I see different scores from different replicas of the same shard.
Is this normal and if yes, why?

My understanding until now was that replicas are always the same within a
shard
and the same query to each replica within a shard gives always the same
score.

Can someone help me to understand this?

Regards
Bernd

RE: [Solr8.7] UI request reply empty after 8s

[ufuk yılmaz]

Hi,

A while ago I asked the same thing here. Looking at the source javascript code 
of the frontend app, I saw  a 10k millisecond timeout config in httpInterceptor 
inside app.js. I changed it to something much larger and results of long 
queries began to show.

Hope it helps

Sent from Mail for Windows 10

From: Bruno Mannina
Sent: 13 January 2021 16:39
To: solr-user@lucene.apache.org
Subject: [Solr8.7] UI request reply empty after 8s

Hi All,

 

I'm facing a problem with my Solr8.7. 

When I do a query on my collection from the Solr UI, if the request takes
more than 8s nothing happens.

I mean, Solr UI answer a blank page. No error in log, no response, nothing
at all.

And if I do the same request just behind, the answer appears quickly as
expected.

 

Is there a UI timeout somewhere?

 

Thanks,

Bruno

 



-- 
L'absence de virus dans ce courrier électronique a été vérifiée par le logiciel 
antivirus Avast.
https://www.avast.com/antivirus

Re: different score from different replica of same shard

[Markus Jelsma]

Hello Bernd,

This is normal for NRT replicas, because the way segments are merged and
deletes are removed is not synchronized between replicas. In that case
counts for TF and IDF and norms become slightly different.

You can either use ExactStatsCache that fetches counts for terms before
scoring, so that all replica's use the same counts. Or change the replica
types to TLOG. With TLOG segments are fetched from the leader and thus
identical.

Regards,
Markus

Op wo 13 jan. 2021 om 14:45 schreef Bernd Fehling <
bernd.fehl...@uni-bielefeld.de>:

> Hello list,
>
> a question for better understanding scoring of a shard in a cloud.
>
> I see different scores from different replicas of the same shard.
> Is this normal and if yes, why?
>
> My understanding until now was that replicas are always the same within a
> shard
> and the same query to each replica within a shard gives always the same
> score.
>
> Can someone help me to understand this?
>
> Regards
> Bernd
>

QueryResponse ordering

[Srinivas Kashyap]

Hello,

I have a scenario where I'm using filter query to fetch the results.

Example: Filter query(fq) - PARTY_ID:(abc OR def OR ghi)

Now I'm getting query response through solrJ in different order. Is there a way 
I can get the results in same order as specified in filter query?

Tried dismax boost bq parameter, but it is not returning any value. Please 
refere below url

http://localhost:8983/solr/party/select?bq=PARTY.PARTY_ID:"abc"^2+PARTY.PARTY_ID:"def"^1=dismax=PARTY.PARTY_ID:"abc"
 OR PARTY.PARTY_ID:"def"=*:*

Thanks,
Srinivas

DISCLAIMER:
E-mails and attachments from Bamboo Rose, LLC are confidential.
If you are not the intended recipient, please notify the sender immediately by 
replying to the e-mail, and then delete it without making copies or using it in 
any way.
No representation is made that this email or any attachments are free of 
viruses. Virus scanning is recommended and is the responsibility of the 
recipient.

Disclaimer

The information contained in this communication from the sender is 
confidential. It is intended solely for use by the recipient and others 
authorized to receive it. If you are not the recipient, you are hereby notified 
that any disclosure, copying, distribution or taking action in relation of the 
contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been 
automatically archived by Mimecast Ltd, an innovator in Software as a Service 
(SaaS) for business. Providing a safer and more useful place for your human 
generated data. Specializing in; Security, archiving and compliance. To find 
out more visit the Mimecast website.

different score from different replica of same shard

[Bernd Fehling]

Hello list,

a question for better understanding scoring of a shard in a cloud.

I see different scores from different replicas of the same shard.
Is this normal and if yes, why?

My understanding until now was that replicas are always the same within a shard
and the same query to each replica within a shard gives always the same score.

Can someone help me to understand this?

Regards
Bernd

[Solr8.7] UI request reply empty after 8s

[Bruno Mannina]

Hi All,



I'm facing a problem with my Solr8.7.

When I do a query on my collection from the Solr UI, if the request takes
more than 8s nothing happens.

I mean, Solr UI answer a blank page. No error in log, no response, nothing
at all.

And if I do the same request just behind, the answer appears quickly as
expected.



Is there a UI timeout somewhere?



Thanks,

Bruno





--
L'absence de virus dans ce courrier électronique a été vérifiée par le logiciel 
antivirus Avast.
https://www.avast.com/antivirus

What should I do when I see a collection "recovering" in SolrCloud?

[ufuk yılmaz]

Should I stop indexing new documents, or stop indexing and wait for collections 
to recover?

Recently our disk got 100% full and Solr started to throw various errors. So I 
deleted some unnecessary documents and committed with expungeDeletes=true. It 
freed some space but many collections went into recovery mode.

Is there a guideline on how to react to such situations?

Thanks!

Sent from Mail for Windows 10

Cursor Performance Issue

[Ajay Sharma]

Hi All,

I have used cursors to search and export documents in solr according to
https://lucene.apache.org/solr/guide/6_6/pagination-of-results.html#fetching-a-large-number-of-sorted-results-cursors

Solr version: 6.5.0
No of Documents: 10 crore

Before implementing cursor, I was using the start and rows parameter to
fetch records
Service response time used to be 2 sec

*Before implementing Cursor Solr URL:*
http://localhost:8080/solr/search/select?q=bird
toy=mapping=3=25=100

Request handler Looks like this: fl contains approx 20 fields


edismax
on
0.01


id,refid,title,smalldesc:""

   
none
json
25
15000
smalldesc
title_text
titlews^3
sdescnisq
1

2-1 470%



Sharing Response with EchoParams=all > Qtime is 6
responseHeader: {
status: 0,
QTime: 6,
params: {
ps: "3",
echoParams: "all",
indent: "on",
fl: "id,refid,title,smalldesc:"",
tie: "0.01",
defType: "edismax",
qf: "customphonetic",
wt: "json",
   qs: "1",
   qt: "mapping",
   rows: "25",
   q: "bird toy",
   timeAllowed: "15000"
}
},
response: {
numFound: 17,
start: 0,
maxScore: 26.616478,
docs: [
  {
id: "22347708097",
refid: "152585558",
title: "Round BIRD COLOURFUL SWINGING CIRCULAR SITTING TOY",
smalldesc: "",
score: 26.616478
 }
]
}

I am facing a performance issue now after implementing the cursor. Service
response time is increased 3 to 4 times .i.e. 8 sec in some cases

*After implementing Cursor query is-*
localhost:8080/solr/search/select?q=bird
toy=cursor=3=1000=100=score desc,id asc=*

Just added =score desc,id asc=* to the before query and
rows to be fetched is 1000 now and fl contains just a single field

Request handler remains same as before just changed the name and made fl
change and added df in defaults


   
  edismax
  on
  0.01
   
   
  refid
   
   
  none
  json
  1000
  smalldesc
  title_text
  titlews^3
  sdescnisq
  1
  2-1 470%
  product_titles
   


Response with Cursor and echoParams=all-> *Qtime is now 17* i.e approx 3
time of previous qtime
responseHeader: {
status: 0,
QTime: 17,
params: {
df: "product_titles",
ps: "3",
echoParams: "all",
indent: "on",
fl: "refid",
tie: "0.01",
defType: "edismax",
qf: "customphonetic",
qs: "1",
qt: "cursor",
sort: "score desc,id asc",
rows: "1000",
q: "bird toy",
cursorMark: "*",
}
},
response: {
numFound: 17,
start: 0,
docs: [
{
refid: "152585558"
},
{
refid: "157276077"
}
]
}


When i curl http://localhost:8080/solr/search/select?q=bird
toy=mapping=3=25=100, i can get results in 3 seconds.
When i curl localhost:8080/solr/search/select?q=bird
toy=cursor=3=1000=100=score desc,id asc=* it
consumed 8 seconds to return result even if the result count=0

BTW, the id schema definition is used in sort


Is it due to the sort I have applied or I have implemented it in the wrong
way?
Please help or provide the direction to solve this issue


Thanks in advance

-- 
Thanks & Regards,
Ajay Sharma
Product Search
Indiamart Intermesh Ltd.

-- 

Atomic Update Failures with Nested Schema and Lazy Field Loading

[Ronen Nussbaum]

Hi,



I’ve encountered another issue that might be related to nested schema.

Not always, but many times atomic updates fails for some shards with the
message “TransactionLog doesn't know how to serialize class
org.apache.lucene.document.LazyDocument$LazyField”.

I checked both options:

   1. Set false.
   2. Set true but removed
   child documents.

In both cases atomic update worked without any errors.

This might suggest that there is an issue with this combination.



Thanks in advance,
Ronen.