Question regarding replica leader

[Vishal Vaibhav]

Hi Solr folks,

I am using solr cloud 8.4.1 . I am using*
`/solr/admin/collections?action=CLUSTERSTATUS`*. Hitting this endpoint I
get a list of replicas in which one is active but neither of them is
leader. Something like this

"core_node72": {"core": "rules_shard1_replica_n71","base_url": "node3,"
node_name": "node3 base url","state": "active","type": "NRT","
force_set_state": "false"},"core_node74": {"core":
"rules_shard1_replica_n73","base_url": "node1","node_name": "node1_base_url"
,"state": "down","type": "NRT","force_set_state": "false","router": {"
name": "compositeId"},"maxShardsPerNode": "1","autoAddReplicas": "false","
nrtReplicas": "1","tlogReplicas": "0","znodeVersion": 276,"configName":
"rules"}},"live_nodes": ["node1","node2","node3","node4"] And when i see
overseer status solr/admin/collections?action=OVERSEERSTATUS I get response
like this which shows node 3 as leaderresponseHeader": {"status": 0,"QTime
": 66},"leader": "node 3","overseer_queue_size": 0,"overseer_work_queue_size
": 0,"overseer_collection_queue_size": 2,"overseer_operations": [
"addreplica",

Does it mean the cluster is having a leader node but there is no leader
replica as of now? And why the leader election is not happening?

Re: [CAUTION] SSL + Solr 8.5.1 in cloud mode + Java 8

[Kevin Risden]

Hmmm so I looked closer - it looks like a side effect of the default
passthrough of the keystore being passed to the client keystore.

https://github.com/apache/lucene-solr/blob/master/solr/bin/solr#L229

Can you remove or commout the entire SOLR_SSL_CLIENT_KEY_STORE section from
bin/solr or bin/solr.cmd depending on which version you are using? The key
being to make sure to not set "-Djavax.net.ssl.keyStore".

This assumes that you aren't using Solr client auth (which based on your
config you aren't) and you aren't trying to use Solr to connect to anything
that is secured via clientAuth (most likely you aren't).

If you can try this and report back that would be awesome. I think this
will fix the issue and it would be possible to make client auth opt in
instead of default fall back.
Kevin Risden



On Tue, Jul 14, 2020 at 1:46 AM Natarajan, Rajeswari <
rajeswari.natara...@sap.com> wrote:

> Thank you so much for the response.  Below are the configs I have in
> solr.in.sh and I followed
> https://lucene.apache.org/solr/guide/8_5/enabling-ssl.html documentation
>
> # Enables HTTPS. It is implicitly true if you set SOLR_SSL_KEY_STORE. Use
> this config
> # to enable https module with custom jetty configuration.
> SOLR_SSL_ENABLED=true
> # Uncomment to set SSL-related system properties
> # Be sure to update the paths to the correct keystore for your environment
> SOLR_SSL_KEY_STORE=etc/solr-ssl.keystore.p12
> SOLR_SSL_KEY_STORE_PASSWORD=secret
> SOLR_SSL_TRUST_STORE=etc/solr-ssl.keystore.p12
> SOLR_SSL_TRUST_STORE_PASSWORD=secret
> # Require clients to authenticate
> SOLR_SSL_NEED_CLIENT_AUTH=false
> # Enable clients to authenticate (but not require)
> SOLR_SSL_WANT_CLIENT_AUTH=false
> # SSL Certificates contain host/ip "peer name" information that is
> validated by default. Setting
> # this to false can be useful to disable these checks when re-using a
> certificate on many hosts
> SOLR_SSL_CHECK_PEER_NAME=true
>
> In local , with the below certificate it works
> ---
>
> keytool -list -keystore solr-ssl.keystore.p12
> Enter keystore password:
> Keystore type: PKCS12
> Keystore provider: SUN
>
> Your keystore contains 1 entry
>
> solr-18, Jun 26, 2020, PrivateKeyEntry,
> Certificate fingerprint (SHA1):
> AB:F2:C8:84:E8:E7:A2:BF:2D:0D:2F:D3:95:4A:98:5B:2A:88:81:50
> C02W48C6HTD6:solr-8.5.1 i843100$ keytool -list -v -keystore
> solr-ssl.keystore.p12
> Enter keystore password:
> Keystore type: PKCS12
> Keystore provider: SUN
>
> Your keystore contains 1 entry
>
> Alias name: solr-18
> Creation date: Jun 26, 2020
> Entry type: PrivateKeyEntry
> Certificate chain length: 1
> Certificate[1]:
> Owner: CN=localhost, OU=Organizational Unit, O=Organization, L=Location,
> ST=State, C=Country
> Issuer: CN=localhost, OU=Organizational Unit, O=Organization, L=Location,
> ST=State, C=Country
> Serial number: 45a822c8
> Valid from: Fri Jun 26 00:13:03 PDT 2020 until: Sun Nov 10 23:13:03 PST
> 2047
> Certificate fingerprints:
>  MD5:  0B:80:54:89:44:65:93:07:1F:81:88:8D:EC:BD:38:41
>  SHA1: AB:F2:C8:84:E8:E7:A2:BF:2D:0D:2F:D3:95:4A:98:5B:2A:88:81:50
>  SHA256:
> 9D:65:A6:55:D7:22:B2:72:C2:20:55:66:F8:0C:9C:48:B1:F6:48:40:A4:FB:CB:26:77:DE:C4:97:34:69:25:42
> Signature algorithm name: SHA256withRSA
> Subject Public Key Algorithm: 2048-bit RSA key
> Version: 3
>
> Extensions:
>
> #1: ObjectId: 2.5.29.17 Criticality=false
> SubjectAlternativeName [
>   DNSName: localhost
>   IPAddress: 172.20.10.4
>   IPAddress: 127.0.0.1
> ]
>
> #2: ObjectId: 2.5.29.14 Criticality=false
> SubjectKeyIdentifier [
> KeyIdentifier [
> : 1B 6F BB 65 A4 3C 6A F4   C9 05 08 89 88 0E 9E 76  .o.e. 0010: A1 B7 28 BE..(.
> ]
>
> /
> In a cluster env , where the deployment  , keystore everything is
> automated  (used by  multiple teams) keystore generated is as below. As you
> can see the  keystore has 2 certificates , in which case I get the
> exception  below.
>
> java.lang.UnsupportedOperationException: X509ExtendedKeyManager only
> > supported on Server
> >   at
> >
> org.apache.solr.client.solrj.impl.Http2SolrClient.createHttpClient(Http2SolrClient.java:223)
> >
>
> In both cases , the config is same except the keystore certificates . In
> the JIRA (https://issues.apache.org/jira/browse/SOLR-14105) , I see the
> fix says it supports multiple DNS and multiple certificates. So I thought
> it should be ok. Please let me know .
>
> keytool -list -keystore  /etc/nginx/certs/sidecar.p12
> Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
> Enter keystore password:
> Keystore type: PKCS12
> Keystore provider: SUN
>
> Your keystore contains 1 entry
>
> 1, Jul 7, 2020, PrivateKeyEntry,
> Certificate fingerprint (SHA1):
> E2:3B:4B:4A:0E:05:CF:DA:59:09:55:8D:4E:6D:8A:1D:4E:DD:D4:62
> bash-5.0#
> -
>
> bash-5.0#  keytool -list -v -keystore 

UpdateProcessorChains -cdcr processor along with ignore commit processor

[Natarajan, Rajeswari]

Hi ,

Would like to have these two processors (cdcr and ignorecommit)  in 
solrconfig.xml .

But cdcr fails with below error , with either cdcr-processor-chain or 
ignore-commit-from-client chain

version conflict for 60d35f0850afac66 expected=1671629672447737856 
actual=-1, retry=0 commError=false errorCode=409



 

  

cdcr-processor-chain

  







  

  









  

200

  



  





Also tried as below. Getting error comaplining about custom processor



  

custom

  
>









 200






  



Is there a way these two processors can be applied together.

Thanks,
Rajeswari

Re: In-place update vs Atomic updates

[Shawn Heisey]

On 7/14/2020 12:21 PM, raj.yadav wrote:

As per the above statement in atomic-update, it reindex the entire document
and deletes the old one.
But I was going through solr documentation regarding the  ( solr document
update policy

) and found these two contradicting statements:

1. /The first is atomic updates. This approach allows changing only one or
more fields of a document without having to reindex the entire document./


Here is how I would rewrite that paragraph to make it correct.  The 
asterisks represent bold text:


1. The first is atomic updates.  This approach allows the indexing 
request to contain *only* the desired changes, instead of the entire 
document.



2./In regular atomic updates, the entire document is reindexed internally
during the application of the update. /


This is correct as written.

Thanks,
Shawn

Re: In-place update vs Atomic updates

[raj.yadav]

Shawn Heisey-2 wrote
> Atomic updates are nearly identical to simple indexing, except that the 
> existing document is read from the index to populate a new document 
> along with whatever updates were requested, then the new document is 
> indexed and the old one is deleted.

As per the above statement in atomic-update, it reindex the entire document
and deletes the old one.
But I was going through solr documentation regarding the  ( solr document
update policy
 
) and found these two contradicting statements:

1. /The first is atomic updates. This approach allows changing only one or
more fields of a document without having to reindex the entire document./

2./In regular atomic updates, the entire document is reindexed internally
during the application of the update. /

Is there something I'm missing here?

Regards,
Raj



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Concurrent query execution and Solr

[André Widhani]

Hi,

Does anybody know if work is in progress to make Lucene's concurrent query
execution accessible through Solr? I am talking about this:
http://blog.mikemccandless.com/2019/10/concurrent-query-execution-in-apache.html

I find this compelling in particular since the changes in LUCENE-7976 /
Solr 7.5 where, even after an optimize, you end up with a number of almost
equally sized segments. And for those who would go to Solr Cloud for
parallel query execution only because they have other means of redundancy
in place, this is a nice way to avoid additional complexity with ZooKeeper.

Thanks,
André

Re: Understanding Negative Filter Queries

[Erick Erickson]

There’s another possibility if the person I _should_ shoot who
wrote the query can’t change it; add cost=101 and turn it
into a post-filter. It’s not clear to me how much difference
that’d make, but it might be worth a shot, see: 

https://yonik.com/advanced-filter-caching-in-solr-2/

Best,
Erick

> On Jul 14, 2020, at 8:33 AM, Chris Dempsey  wrote:
> 
>> 
>> Well, they’ll be exactly the same if (and only if) every document has a
>> tag. Otherwise, the
>> first one will exclude a doc that has no tag and the second one will
>> include it.
> 
> 
> That's a good point/catch.
> 
> How slow is “very slow”?
>> 
> 
> Well, in the case I was looking at it was about 10x slower but with the
> following caveats that there were 15 or so of these negative fq all some
> version of `fq={!cache=false}(tag:* -tag:)` (*don't shoot me I
> didn't write it lol*) over 15 million documents. Which to me means that
> each fq was doing each step that you described below:
> 
> The second form only has to index into the terms dictionary for the tag
>> field
>> value “email”, then zip down the posting list for all the docs that have
>> it. The
>> first form has to first identify all the docs that have a tag, accumulate
>> that list,
>> _then_ find the “email” value and zip down the postings list.
>> 
> 
> Thanks yet again Erick. That solidified in my mind how this works. Much
> appreciated!
> 
> 
> 
> 
> 
> On Tue, Jul 14, 2020 at 7:22 AM Erick Erickson 
> wrote:
> 
>> Yeah, there are optimizations there. BTW, these two queries are subtly
>> different.
>> 
>> Well, they’ll be exactly the same if (and only if) every document has a
>> tag. Otherwise, the
>> first one will exclude a doc that has no tag and the second one will
>> include it.
>> 
>> How slow is “very slow”?
>> 
>> The second form only has to index into the terms dictionary for the tag
>> field
>> value “email”, then zip down the posting list for all the docs that have
>> it. The
>> first form has to first identify all the docs that have a tag, accumulate
>> that list,
>> _then_ find the “email” value and zip down the postings list.
>> 
>> You could get around this if you require the first form functionality by,
>> say,
>> including a boolean field “has_tags”, then the first one would be
>> 
>> fq=has_tags:true -tags:email
>> 
>> Best,
>> Erick
>> 
>>> On Jul 14, 2020, at 8:05 AM, Emir Arnautović <
>> emir.arnauto...@sematext.com> wrote:
>>> 
>>> Hi Chris,
>>> tag:* is a wildcard query while *:* is match all query. I believe that
>> adjusting pure negative is turned on by default so you can safely just use
>> -tag:email and it’ll be translated to *:* -tag:email.
>>> 
>>> HTH,
>>> Emir
>>> --
>>> Monitoring - Log Management - Alerting - Anomaly Detection
>>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>>> 
>>> 
>>> 
 On 14 Jul 2020, at 14:00, Chris Dempsey  wrote:
 
 I'm trying to understand the difference between something like
 fq={!cache=false}(tag:* -tag:email) which is very slow compared to
 fq={!cache=false}(*:* -tag:email) on Solr 7.7.1.
 
 I believe in the case of `tag:*` Solr spends some effort to gather all
>> of
 the documents that have a value for `tag` and then removes those with
 `-tag:email` while in the `*:*` Solr simply uses the document set as-is
 and  then remove those with `-tag:email` (*and I believe Erick mentioned
 there were special optimizations for `*:*`*)?
>>> 
>> 
>> 

Re: Solr heap Old generation grows and it is not recovered by G1GC

[Erik Hatcher]

What kind of statistics?Are these stats that you could perhaps get from 
faceting or the stats component instead of gathering docs and accumulating 
stats yourself?



> On Jul 14, 2020, at 8:51 AM, Odysci  wrote:
> 
> Hi Erick,
> 
> I agree. The 300K docs in one search is an anomaly.
> But we do use 'fq' to return a large number of docs for the purposes of
> generating statistics for the whole index. We do use CursorMark extensively.
> Thanks!
> 
> Reinaldo
> 
> On Tue, Jul 14, 2020 at 8:55 AM Erick Erickson 
> wrote:
> 
>> I’d add that you’re abusing Solr horribly by returning 300K documents in a
>> single go.
>> 
>> Solr is built to return the top N docs where N is usually quite small, <
>> 100. If you allow
>> an unlimited number of docs to be returned, you’re simply kicking the can
>> down
>> the road, somebody will ask for 1,000,000 docs sometime and you’ll be back
>> where
>> you started.
>> 
>> I _strongly_ recommend you do one of two things for such large result sets:
>> 
>> 1> Use Streaming. Perhaps Streaming Expressions will do what you want
>>without you having to process all those docs on the client if you’re
>>doing some kind of analytics.
>> 
>> 2> if you really, truly need all 300K docs, try getting them in chunks
>> using CursorMark.
>> 
>> Best,
>> Erick
>> 
>>> On Jul 13, 2020, at 10:03 PM, Odysci  wrote:
>>> 
>>> Shawn,
>>> 
>>> thanks for the extra info.
>>> The OOM errors were indeed because of heap space. In my case most of the
>> GC
>>> calls were not full GC. Only when heap was really near the top, a full GC
>>> was done.
>>> I'll try out your suggestion of increasing the G1 heap region size. I've
>>> been using 4m, and from what you said, a 2m allocation would be
>> considered
>>> humongous. My test cases have a few allocations that are definitely
>> bigger
>>> than 2m (estimating based on the number of docs returned), but most of
>> them
>>> are not.
>>> 
>>> When i was using maxRamMB, the size used was "compatible" with the the
>> size
>>> values, assuming the avg 2K bytes docs that our index has.
>>> As far as I could tell in my runs, removing maxRamMB did change the GC
>>> behavior for the better. That is, now, heap goes up and down as expected,
>>> and before (with maxRamMB) it seemed to increase continuously.
>>> Thanks
>>> 
>>> Reinaldo
>>> 
>>> On Sun, Jul 12, 2020 at 1:02 AM Shawn Heisey 
>> wrote:
>>> 
 On 6/25/2020 2:08 PM, Odysci wrote:
> I have a solrcloud setup with 12GB heap and I've been trying to
>> optimize
 it
> to avoid OOM errors. My index has about 30million docs and about 80GB
> total, 2 shards, 2 replicas.
 
 Have you seen the full OutOfMemoryError exception text?  OOME can be
 caused by problems that are not actually memory-related.  Unless the
 error specifically mentions "heap space" we might be chasing the wrong
 thing here.
 
> When the queries return a smallish number of docs (say, below 1000),
>> the
> heap behavior seems "normal". Monitoring the gc log I see that young
> generation grows then when GC kicks in, it goes considerably down. And
 the
> old generation grows just a bit.
> 
> However, at some point i have a query that returns over 300K docs (for
>> a
> total size of approximately 1GB). At this very point the OLD generation
> size grows (almost by 2GB), and it remains high for all remaining time.
> Even as new queries are executed, the OLD generation size does not go
 down,
> despite multiple GC calls done afterwards.
 
 Assuming the OOME exceptions were indeed caused by running out of heap,
 then the following paragraphs will apply:
 
 G1 has this concept called "humongous allocations".  In order to reach
 this designation, a memory allocation must get to half of the G1 heap
 region size.  You have set this to 4 megabytes, so any allocation of 2
 megabytes or larger is humongous.  Humongous allocations bypass the new
 generation entirely and go directly into the old generation.  The max
 value that can be set for the G1 region size is 32MB.  If you increase
 the region size and the behavior changes, then humongous allocations
 could be something to investigate.
 
 In the versions of Java that I have used, humongous allocations can only
 be reclaimed as garbage by a full GC.  I do not know if Oracle has
 changed this so the smaller collections will do it or not.
 
 Were any of those multiple GCs a Full GC?  If they were, then there is
 probably little or no garbage to collect.  You've gotten a reply from
 "Zisis T." with some possible causes for this.  I do not have anything
 to add.
 
 I did not know about any problems with maxRamMB ... but if I were
 attempting to limit cache sizes, I would do so by the size values, not a
 specific RAM size.  The size values you have chosen (8192 and 16384)
 will most likely result in a total 

Re: Solr heap Old generation grows and it is not recovered by G1GC

[Odysci]

Hi Erick,

I agree. The 300K docs in one search is an anomaly.
But we do use 'fq' to return a large number of docs for the purposes of
generating statistics for the whole index. We do use CursorMark extensively.
Thanks!

Reinaldo

On Tue, Jul 14, 2020 at 8:55 AM Erick Erickson 
wrote:

> I’d add that you’re abusing Solr horribly by returning 300K documents in a
> single go.
>
> Solr is built to return the top N docs where N is usually quite small, <
> 100. If you allow
> an unlimited number of docs to be returned, you’re simply kicking the can
> down
> the road, somebody will ask for 1,000,000 docs sometime and you’ll be back
> where
> you started.
>
> I _strongly_ recommend you do one of two things for such large result sets:
>
> 1> Use Streaming. Perhaps Streaming Expressions will do what you want
> without you having to process all those docs on the client if you’re
> doing some kind of analytics.
>
> 2> if you really, truly need all 300K docs, try getting them in chunks
>  using CursorMark.
>
> Best,
> Erick
>
> > On Jul 13, 2020, at 10:03 PM, Odysci  wrote:
> >
> > Shawn,
> >
> > thanks for the extra info.
> > The OOM errors were indeed because of heap space. In my case most of the
> GC
> > calls were not full GC. Only when heap was really near the top, a full GC
> > was done.
> > I'll try out your suggestion of increasing the G1 heap region size. I've
> > been using 4m, and from what you said, a 2m allocation would be
> considered
> > humongous. My test cases have a few allocations that are definitely
> bigger
> > than 2m (estimating based on the number of docs returned), but most of
> them
> > are not.
> >
> > When i was using maxRamMB, the size used was "compatible" with the the
> size
> > values, assuming the avg 2K bytes docs that our index has.
> > As far as I could tell in my runs, removing maxRamMB did change the GC
> > behavior for the better. That is, now, heap goes up and down as expected,
> > and before (with maxRamMB) it seemed to increase continuously.
> > Thanks
> >
> > Reinaldo
> >
> > On Sun, Jul 12, 2020 at 1:02 AM Shawn Heisey 
> wrote:
> >
> >> On 6/25/2020 2:08 PM, Odysci wrote:
> >>> I have a solrcloud setup with 12GB heap and I've been trying to
> optimize
> >> it
> >>> to avoid OOM errors. My index has about 30million docs and about 80GB
> >>> total, 2 shards, 2 replicas.
> >>
> >> Have you seen the full OutOfMemoryError exception text?  OOME can be
> >> caused by problems that are not actually memory-related.  Unless the
> >> error specifically mentions "heap space" we might be chasing the wrong
> >> thing here.
> >>
> >>> When the queries return a smallish number of docs (say, below 1000),
> the
> >>> heap behavior seems "normal". Monitoring the gc log I see that young
> >>> generation grows then when GC kicks in, it goes considerably down. And
> >> the
> >>> old generation grows just a bit.
> >>>
> >>> However, at some point i have a query that returns over 300K docs (for
> a
> >>> total size of approximately 1GB). At this very point the OLD generation
> >>> size grows (almost by 2GB), and it remains high for all remaining time.
> >>> Even as new queries are executed, the OLD generation size does not go
> >> down,
> >>> despite multiple GC calls done afterwards.
> >>
> >> Assuming the OOME exceptions were indeed caused by running out of heap,
> >> then the following paragraphs will apply:
> >>
> >> G1 has this concept called "humongous allocations".  In order to reach
> >> this designation, a memory allocation must get to half of the G1 heap
> >> region size.  You have set this to 4 megabytes, so any allocation of 2
> >> megabytes or larger is humongous.  Humongous allocations bypass the new
> >> generation entirely and go directly into the old generation.  The max
> >> value that can be set for the G1 region size is 32MB.  If you increase
> >> the region size and the behavior changes, then humongous allocations
> >> could be something to investigate.
> >>
> >> In the versions of Java that I have used, humongous allocations can only
> >> be reclaimed as garbage by a full GC.  I do not know if Oracle has
> >> changed this so the smaller collections will do it or not.
> >>
> >> Were any of those multiple GCs a Full GC?  If they were, then there is
> >> probably little or no garbage to collect.  You've gotten a reply from
> >> "Zisis T." with some possible causes for this.  I do not have anything
> >> to add.
> >>
> >> I did not know about any problems with maxRamMB ... but if I were
> >> attempting to limit cache sizes, I would do so by the size values, not a
> >> specific RAM size.  The size values you have chosen (8192 and 16384)
> >> will most likely result in a total cache size well beyond the limits
> >> you've indicated with maxRamMB.  So if there are any bugs in the code
> >> with the maxRamMB parameter, you might end up using a LOT of memory that
> >> you didn't expect to be using.
> >>
> >> Thanks,
> >> Shawn
> >>
>
>

Re: Understanding Negative Filter Queries

[Chris Dempsey]

>
> Well, they’ll be exactly the same if (and only if) every document has a
> tag. Otherwise, the
> first one will exclude a doc that has no tag and the second one will
> include it.


That's a good point/catch.

How slow is “very slow”?
>

Well, in the case I was looking at it was about 10x slower but with the
following caveats that there were 15 or so of these negative fq all some
version of `fq={!cache=false}(tag:* -tag:)` (*don't shoot me I
didn't write it lol*) over 15 million documents. Which to me means that
each fq was doing each step that you described below:

The second form only has to index into the terms dictionary for the tag
> field
> value “email”, then zip down the posting list for all the docs that have
> it. The
> first form has to first identify all the docs that have a tag, accumulate
> that list,
> _then_ find the “email” value and zip down the postings list.
>

Thanks yet again Erick. That solidified in my mind how this works. Much
appreciated!





On Tue, Jul 14, 2020 at 7:22 AM Erick Erickson 
wrote:

> Yeah, there are optimizations there. BTW, these two queries are subtly
> different.
>
> Well, they’ll be exactly the same if (and only if) every document has a
> tag. Otherwise, the
> first one will exclude a doc that has no tag and the second one will
> include it.
>
> How slow is “very slow”?
>
> The second form only has to index into the terms dictionary for the tag
> field
> value “email”, then zip down the posting list for all the docs that have
> it. The
> first form has to first identify all the docs that have a tag, accumulate
> that list,
> _then_ find the “email” value and zip down the postings list.
>
> You could get around this if you require the first form functionality by,
> say,
> including a boolean field “has_tags”, then the first one would be
>
> fq=has_tags:true -tags:email
>
> Best,
> Erick
>
> > On Jul 14, 2020, at 8:05 AM, Emir Arnautović <
> emir.arnauto...@sematext.com> wrote:
> >
> > Hi Chris,
> > tag:* is a wildcard query while *:* is match all query. I believe that
> adjusting pure negative is turned on by default so you can safely just use
> -tag:email and it’ll be translated to *:* -tag:email.
> >
> > HTH,
> > Emir
> > --
> > Monitoring - Log Management - Alerting - Anomaly Detection
> > Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> >
> >
> >
> >> On 14 Jul 2020, at 14:00, Chris Dempsey  wrote:
> >>
> >> I'm trying to understand the difference between something like
> >> fq={!cache=false}(tag:* -tag:email) which is very slow compared to
> >> fq={!cache=false}(*:* -tag:email) on Solr 7.7.1.
> >>
> >> I believe in the case of `tag:*` Solr spends some effort to gather all
> of
> >> the documents that have a value for `tag` and then removes those with
> >> `-tag:email` while in the `*:*` Solr simply uses the document set as-is
> >> and  then remove those with `-tag:email` (*and I believe Erick mentioned
> >> there were special optimizations for `*:*`*)?
> >
>
>

Re: Understanding Negative Filter Queries

[Erick Erickson]

Yeah, there are optimizations there. BTW, these two queries are subtly 
different.

Well, they’ll be exactly the same if (and only if) every document has a tag. 
Otherwise, the
first one will exclude a doc that has no tag and the second one will include it.

How slow is “very slow”?

The second form only has to index into the terms dictionary for the tag field
value “email”, then zip down the posting list for all the docs that have it. The
first form has to first identify all the docs that have a tag, accumulate that 
list,
_then_ find the “email” value and zip down the postings list. 

You could get around this if you require the first form functionality by, say, 
including a boolean field “has_tags”, then the first one would be 

fq=has_tags:true -tags:email

Best,
Erick

> On Jul 14, 2020, at 8:05 AM, Emir Arnautović  
> wrote:
> 
> Hi Chris,
> tag:* is a wildcard query while *:* is match all query. I believe that 
> adjusting pure negative is turned on by default so you can safely just use 
> -tag:email and it’ll be translated to *:* -tag:email.
> 
> HTH,
> Emir
> --
> Monitoring - Log Management - Alerting - Anomaly Detection
> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> 
> 
> 
>> On 14 Jul 2020, at 14:00, Chris Dempsey  wrote:
>> 
>> I'm trying to understand the difference between something like
>> fq={!cache=false}(tag:* -tag:email) which is very slow compared to
>> fq={!cache=false}(*:* -tag:email) on Solr 7.7.1.
>> 
>> I believe in the case of `tag:*` Solr spends some effort to gather all of
>> the documents that have a value for `tag` and then removes those with
>> `-tag:email` while in the `*:*` Solr simply uses the document set as-is
>> and  then remove those with `-tag:email` (*and I believe Erick mentioned
>> there were special optimizations for `*:*`*)?
> 

Re: Understanding Negative Filter Queries

[Emir Arnautović]

Hi Chris,
tag:* is a wildcard query while *:* is match all query. I believe that 
adjusting pure negative is turned on by default so you can safely just use 
-tag:email and it’ll be translated to *:* -tag:email.

HTH,
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/



> On 14 Jul 2020, at 14:00, Chris Dempsey  wrote:
> 
> I'm trying to understand the difference between something like
> fq={!cache=false}(tag:* -tag:email) which is very slow compared to
> fq={!cache=false}(*:* -tag:email) on Solr 7.7.1.
> 
> I believe in the case of `tag:*` Solr spends some effort to gather all of
> the documents that have a value for `tag` and then removes those with
> `-tag:email` while in the `*:*` Solr simply uses the document set as-is
> and  then remove those with `-tag:email` (*and I believe Erick mentioned
> there were special optimizations for `*:*`*)?

Understanding Negative Filter Queries

[Chris Dempsey]

I'm trying to understand the difference between something like
fq={!cache=false}(tag:* -tag:email) which is very slow compared to
fq={!cache=false}(*:* -tag:email) on Solr 7.7.1.

I believe in the case of `tag:*` Solr spends some effort to gather all of
the documents that have a value for `tag` and then removes those with
`-tag:email` while in the `*:*` Solr simply uses the document set as-is
and  then remove those with `-tag:email` (*and I believe Erick mentioned
there were special optimizations for `*:*`*)?

Re: Solr heap Old generation grows and it is not recovered by G1GC

[Erick Erickson]

I’d add that you’re abusing Solr horribly by returning 300K documents in a 
single go.

Solr is built to return the top N docs where N is usually quite small, < 100. 
If you allow
an unlimited number of docs to be returned, you’re simply kicking the can down
the road, somebody will ask for 1,000,000 docs sometime and you’ll be back where
you started.

I _strongly_ recommend you do one of two things for such large result sets:

1> Use Streaming. Perhaps Streaming Expressions will do what you want
without you having to process all those docs on the client if you’re 
doing some kind of analytics.

2> if you really, truly need all 300K docs, try getting them in chunks
 using CursorMark.

Best,
Erick

> On Jul 13, 2020, at 10:03 PM, Odysci  wrote:
> 
> Shawn,
> 
> thanks for the extra info.
> The OOM errors were indeed because of heap space. In my case most of the GC
> calls were not full GC. Only when heap was really near the top, a full GC
> was done.
> I'll try out your suggestion of increasing the G1 heap region size. I've
> been using 4m, and from what you said, a 2m allocation would be considered
> humongous. My test cases have a few allocations that are definitely bigger
> than 2m (estimating based on the number of docs returned), but most of them
> are not.
> 
> When i was using maxRamMB, the size used was "compatible" with the the size
> values, assuming the avg 2K bytes docs that our index has.
> As far as I could tell in my runs, removing maxRamMB did change the GC
> behavior for the better. That is, now, heap goes up and down as expected,
> and before (with maxRamMB) it seemed to increase continuously.
> Thanks
> 
> Reinaldo
> 
> On Sun, Jul 12, 2020 at 1:02 AM Shawn Heisey  wrote:
> 
>> On 6/25/2020 2:08 PM, Odysci wrote:
>>> I have a solrcloud setup with 12GB heap and I've been trying to optimize
>> it
>>> to avoid OOM errors. My index has about 30million docs and about 80GB
>>> total, 2 shards, 2 replicas.
>> 
>> Have you seen the full OutOfMemoryError exception text?  OOME can be
>> caused by problems that are not actually memory-related.  Unless the
>> error specifically mentions "heap space" we might be chasing the wrong
>> thing here.
>> 
>>> When the queries return a smallish number of docs (say, below 1000), the
>>> heap behavior seems "normal". Monitoring the gc log I see that young
>>> generation grows then when GC kicks in, it goes considerably down. And
>> the
>>> old generation grows just a bit.
>>> 
>>> However, at some point i have a query that returns over 300K docs (for a
>>> total size of approximately 1GB). At this very point the OLD generation
>>> size grows (almost by 2GB), and it remains high for all remaining time.
>>> Even as new queries are executed, the OLD generation size does not go
>> down,
>>> despite multiple GC calls done afterwards.
>> 
>> Assuming the OOME exceptions were indeed caused by running out of heap,
>> then the following paragraphs will apply:
>> 
>> G1 has this concept called "humongous allocations".  In order to reach
>> this designation, a memory allocation must get to half of the G1 heap
>> region size.  You have set this to 4 megabytes, so any allocation of 2
>> megabytes or larger is humongous.  Humongous allocations bypass the new
>> generation entirely and go directly into the old generation.  The max
>> value that can be set for the G1 region size is 32MB.  If you increase
>> the region size and the behavior changes, then humongous allocations
>> could be something to investigate.
>> 
>> In the versions of Java that I have used, humongous allocations can only
>> be reclaimed as garbage by a full GC.  I do not know if Oracle has
>> changed this so the smaller collections will do it or not.
>> 
>> Were any of those multiple GCs a Full GC?  If they were, then there is
>> probably little or no garbage to collect.  You've gotten a reply from
>> "Zisis T." with some possible causes for this.  I do not have anything
>> to add.
>> 
>> I did not know about any problems with maxRamMB ... but if I were
>> attempting to limit cache sizes, I would do so by the size values, not a
>> specific RAM size.  The size values you have chosen (8192 and 16384)
>> will most likely result in a total cache size well beyond the limits
>> you've indicated with maxRamMB.  So if there are any bugs in the code
>> with the maxRamMB parameter, you might end up using a LOT of memory that
>> you didn't expect to be using.
>> 
>> Thanks,
>> Shawn
>> 

Re: SOLR Exact phrase search issue

[Erick Erickson]

This is usually a result of either indexing or querying not quite doing what 
you expect. The screenshots don’t help diagnose as they’re just the results, 
but don’t really help understand why.

So here’s what I do to try to figure out why:

1> add =query to the query You can check the “debugQuery” checkbox on the 
admin UI. In particular look at the “parsed query” in the results. Is it what 
you expect?

2> use the Admin/Analysis page to see how the fields you’re searching against 
are tokenized. Sometimes your analysis chain produces unexpected results. <1> 
will show you this for querying, but not indexing.

3> try turning on highlighting. You have not shown, for instance, that "Què 
t’hi jugues” all appears in a single field. It’s conceivable that you’re not 
searching that field at all and are matching
"t’hi jugues” or "Què t’hi” in a different field than you expect.

4> Another thing that fools people is that the analysis chain may break up 
“t’hi” into “t” and “hi” which then may match unexpected places.

5> Are any of these stopwords? The admin/analysis page will show you.

6> Finally, try attaching =true=id:id_of_doc_you_expect. 
That will show you now the document you expect was scored, whether or not it’s 
included in numFound. It’s intended exactly for answering the question “why 
didn’t my searche return a doc I _know_ it should?"


Seems like a lot of places to look, but <1> is certainly the first place I’d 
look. This is almost certainly a mismatch between what you think is happening 
and what you’ve actually told Solr to do ;).

Best,
Erick

> On Jul 14, 2020, at 7:05 AM, Villalba Sans, Raúl  
> wrote:
> 
> Hello,
> 
> We have an app that uses SOLR as search engine. We have detected incorrect 
> behavior for which we find no explanation. If we perform a search with the 
> phrase "Què t’hi jugues" we do not receive any results, although we know that 
> there is a result that contains this phrase. However, if we search for "Què 
> t’hi" or for "t’hi jugues" we do find results, including "Què t’hi jugues ". 
> We attach screenshots of the search tool and the xml of the results. We would 
> greatly appreciate it if you could lend a hand in trying to find a solution 
> or identify the cause of the problem.
>  
> Search 1 – “Què t’hi jugues”
> 
>  
> Search 2 – “Què t’hi”
>  
> 
> Search 3 – “t’hi jugues”
> 
> 
> Best regards,
>  
> 
>  
> Raül Villalba Sans
> Delivery Centers – Centros de Producción
>  
> Parque de Gardeny, Edificio 28
> 25071 Lleida, España
> T +34 973 193 580
>  
>