RE: Autoscaling working Examples

2021-02-18 Thread Shane Brooks
Hi Florian, We have a Solr Cloud installation with 2 nodes, each in a separate Availability Zone. We run one shard for each collection, with replicationFactor set to 2 and autoAddReplica set to True. These are the only settings we make other than using our config set. Typical settings for our

Re: Atomic Update (nested), Unified Highlighter and Lazy Field Loading => Invalid Index

2021-02-18 Thread David Smiley
age- > From: David Smiley > Sent: יום ה 18 פברואר 2021 01:03 > To: solr-user > Subject: Re: Atomic Update (nested), Unified Highlighter and Lazy Field > Loading => Invalid Index > > I think the issue is this existing bug, but needs to refer to > toSolrInputDocument i

Re: Solr 8.0 query length limit

2021-02-18 Thread Anuj Bhargava
Thanks Alex and Shawn. Regards, Anuj On Thu, 18 Feb 2021 at 18:57, Shawn Heisey wrote: > On 2/18/2021 3:38 AM, Anuj Bhargava wrote: > > Solr 8.0 query length limit > > > > We are having an issue where queries are too big, we get no result. And > if > > we remove a few keywords we get the

Re: Cannot find Solr 7.4.1 release

2021-02-18 Thread Shawn Heisey
On 2/18/2021 1:05 AM, Olivier Tavard wrote: I wanted to download Solr 7.4.1, but I cannot find the 7.4.1 release into http://archive.apache.org/dist/lucene/solr/ : there are Solr 7.4 and after directly 7.5. Of course I can build from source code, but this is frustrating because I can see that in

Re: Solr 8.0 query length limit

2021-02-18 Thread Shawn Heisey
On 2/18/2021 3:38 AM, Anuj Bhargava wrote: Solr 8.0 query length limit We are having an issue where queries are too big, we get no result. And if we remove a few keywords we get the result. The best option is to convert the request to POST, as Thomas suggested. With that, the query

Re: Solr 8.0 query length limit

2021-02-18 Thread Alexandre Rafalovitch
Also, investigate if you have repeating conditions and push those into defaults in custom request handler endpoints (in solrconfig.xml). Also, Solr supports parameter substitutions, if you have repeated subconditions. Regards, Alex On Thu., Feb. 18, 2021, 7:08 a.m. Thomas Corthals, wrote:

Re: How to get case-sensitive Terms?

2021-02-18 Thread Alexandre Rafalovitch
Terms query does not do analysis chain, but expect tokenized values. Because it matches what is returned by faceting. So I would check whether that field is string or text and difference in processing. Enabling debug will also show difference in final expanded form. Regards, Alex P. S. It is

Re: Solr 8.0 query length limit

2021-02-18 Thread Thomas Corthals
You can send big queries as a POST request instead of a GET request. Op do 18 feb. 2021 om 11:38 schreef Anuj Bhargava : > Solr 8.0 query length limit > > We are having an issue where queries are too big, we get no result. And if > we remove a few keywords we get the result. > > Error we get -

RE: Atomic Update (nested), Unified Highlighter and Lazy Field Loading => Invalid Index

2021-02-18 Thread Nussbaum, Ronen
enableLazyFieldLoading=false. I hope it wouldn't have a significant performance impact. -Original Message- From: David Smiley Sent: יום ה 18 פברואר 2021 01:03 To: solr-user Subject: Re: Atomic Update (nested), Unified Highlighter and Lazy Field Loading => Invalid Index I think the is

Re: How to get case-sensitive Terms?

2021-02-17 Thread elivis
Alexandre Rafalovitch wrote > What about copyField with the target being index only (docValue only?) and > no lowercase on the target field type? > > Solr is not a database, you are optimising for search. So duplicate, > multi-process, denormalise, create custom field types, etc. > > Regards, >

Re: Meaning of "Index" flag under properties and schema

2021-02-17 Thread Alexandre Rafalovitch
I wonder if looking more directly at the indexes would allow you to get closer to the problem source. Have you tried comparing/exploring the indexes with Luke? It is in the Lucene distribution (not Solr), and there is a small explanation here:

Re: Atomic Update (nested), Unified Highlighter and Lazy Field Loading => Invalid Index

2021-02-17 Thread David Smiley
;Solr is great"}} >> >> To reproduce: >> Do a search with surround and unified highlighter: >> >> hl.fl=text_en=unified=on=%7B!surround%7Dtext_en%3A4W("solr"%2C"great") >> >> Now, try to update the parent e.g. {id:"abc_1", categories

Re: Meaning of "Index" flag under properties and schema

2021-02-17 Thread Vivaldi
I was getting “illegal argument exception length must be >= 1” when I used significantTerms streaming expression, from this collection and field. I asked about that as a separate question on this list. I will get the whole exception stack trace the next time I am at the customer site. Why any

RE: Is 8.8.x going be stabilized and finalized?

2021-02-17 Thread Subhajit Das
apa...@elyograg.org> Sent: 17 February 2021 09:33 AM To: solr-user@lucene.apache.org<mailto:solr-user@lucene.apache.org> Subject: Re: Is 8.8.x going be stabilized and finalized? On 2/16/2021 7:57 PM, Subhajit Das wrote: > I am planning to use 8.8 line-up for production use. > > But rece

Re: Is 8.8.x going be stabilized and finalized?

2021-02-17 Thread Timothy Potter
To add to what Shawn said, RC's are made available to anyone interested in testing them and that helps us find bugs before release. RC2 for 8.8.1 is available for testing now, see dev mailing list for location. Please download it and verify it is stable for your use cases and environment. Tim

Re: Change field to DocValues

2021-02-17 Thread Mahmoud Almokadem
17, 2021 at 4:51 PM xiefengchang wrote: > Hi: > I think you are just trying to avoid complete re-index right? > why don't you take a look at this: > https://lucene.apache.org/solr/guide/8_0/updating-parts-of-documents.html > > > > > > > > > > > &

Re: Atomic Update (nested), Unified Highlighter and Lazy Field Loading => Invalid Index

2021-02-17 Thread David Smiley
h with surround and unified highlighter: > > hl.fl=text_en=unified=on=%7B!surround%7Dtext_en%3A4W("solr"%2C"great") > > Now, try to update the parent e.g. {id:"abc_1", categories_i:{add:1}} > > Important: it happens only when "id" contains unde

Re: [SOLVED] UPDATE collection's Rule-based Replica Placement

2021-02-17 Thread mosheB
Thanks Ilan and Aroop for replying. So not exactly move but rather *update* the existing set of rules so future replica placement will enforced by them. I managed to do so using the MODIFYCOLLECTION action:

RE: Atomic Update (nested), Unified Highlighter and Lazy Field Loading => Invalid Index

2021-02-17 Thread Nussbaum, Ronen
y to update the parent e.g. {id:"abc_1", categories_i:{add:1}} Important: it happens only when "id" contains underscore characters! If you'll use "abc-1" it would work. Thanks in advance, Ronen. -Original Message- From: David Smiley Sent: יום א 14 פברואר 2

Re: Is 8.8.x going be stabilized and finalized?

2021-02-16 Thread Shawn Heisey
On 2/16/2021 7:57 PM, Subhajit Das wrote: I am planning to use 8.8 line-up for production use. But recently, a lot of people are complaining on 8.7 and 8.8. Also, there is a clearly known issue on 8.8 as well. Following trends of earlier versions (5.x, 6.x and 7.x), will 8.8 will also be

Re: tlog keeps growing

2021-02-16 Thread mmb1234
Erik, Looks like we're also running into this issue. https://www.mail-archive.com/solr-user@lucene.apache.org/msg153798.html Is there any think we can do to remedy this besides a node restart, which causes leader re-election on the good shards which causes them to also become un-operational

Re: Elevation in dataDir in Solr Cloud

2021-02-16 Thread Chris Hostetter
almost certainly need to be : > an index change for it to re-load, since I don't see any indication that : > it's watching for changes in zk) : > : > FWIW: the way most people seem to be using QEC these days is to have an : > empty elevate.xml file, and then have their applica

Re: Down Replica is elected as Leader (solr v8.7.0)

2021-02-16 Thread mmb1234
> Are yours growing always, on all nodes, forever? Or is it one or two who ends up in a bad state? Randomly on some of the shards and some of the followers in the collection. Then whichever tlog was open on follower when it was the leader, that one doesn't stops growing. And that shard had

Re: Down Replica is elected as Leader (solr v8.7.0)

2021-02-16 Thread matthew sporleder
I've run into this (or similar) issues in the past (solr6? I don't remember exactly) where tlogs get stuck either growing indefinitely and/or refusing to commit on restart. What I ended up doing was writing a monitor to check for the number of tlogs and alert if they got over some limit (100 or

Re: Down Replica is elected as Leader (solr v8.7.0)

2021-02-16 Thread mmb1234
Looks like the problem is related to tlog rotation on the follower shard. We did the following for a specific shard. 0. start solr cloud 1. solr-0 (leader), solr-1, solr-2 2. rebalance to make solr-1 as preferred leader 3. solr-0, solr-1 (leader), solr-2 The tlog file on solr-0 kept on growing

Re: Down Replica is elected as Leader (solr v8.7.0)

2021-02-16 Thread mmb1234
Looks like the problem is related to tlog rotation on the follower shard. We did the following for a specific shard. 0. start solr cloud 1. solr-0 (leader), solr-1, solr-2 2. rebalance to make solr-1 as preferred leader 3. solr-0, solr-1 (leader), solr-2 The tlog file on solr-0 kept on growing

Re: SolrJ: SolrInputDocument.addField()

2021-02-16 Thread Shawn Heisey
On 2/15/2021 10:17 AM, Steven White wrote: Yes, I have managed schema enabled like so: true cp-schema.xml The reason why I enabled it is so that I can dynamically customize the schema based on what's in the DB. So that I can add fields to the schema dynamically. A

Re: Meaning of "Index" flag under properties and schema

2021-02-16 Thread Shawn Heisey
On 2/16/2021 9:16 AM, ufuk yılmaz wrote: I didn’t realise that, sorry. The table is like: Flags Indexed Tokenized Stored UnInvertible Properties YesYesYes Yes Schema YesYesYes Yes Index Yes

Re: Significant terms expression giving error "length needs to be >= 1"

2021-02-16 Thread Joel Bernstein
Can you include the stack trace from the logs? Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Feb 15, 2021 at 3:53 PM ufuk yılmaz wrote: > We have a SolrCloud cluster, version 8.4 > > At the customer’s site there’s a collection with very few documents, > around 12. We usually have

RE: Meaning of "Index" flag under properties and schema

2021-02-16 Thread ufuk yılmaz
NO Problematic collection has a Index row under Schema row. No other collection has it. I was asking about what the “Index” meant -ufuk Sent from Mail for Windows 10 From: Charlie Hull Sent: 16 February 2021 18:48 To: solr-user@lucene.apache.org Subject: Re: Meaning of "Index" flag under

Re: Meaning of "Index" flag under properties and schema

2021-02-16 Thread Charlie Hull
This list strips attachments so you'll have to figure out another way to show the difference, Cheers Charlie On 16/02/2021 15:16, ufuk yılmaz wrote: There’s a collection at our customer’s site giving weird exceptions when a particular field is involved (asked another question detailing

Re: SolrJ: SolrInputDocument.addField()

2021-02-16 Thread Jimi Hullegård
Hi Steven, Just a thought, from someone who never have used schema-less mode: Have you considered using a regular schema file, with a bunch of dynamicField definitions? Then you can for example define a dynamic boolean field like this: Then, when you index the data, you can append "_b" to

Re: SolrJ: SolrInputDocument.addField()

2021-02-15 Thread Steven White
Hi Shawn, Yes, I have managed schema enabled like so: true cp-schema.xml The reason why I enabled it is so that I can dynamically customize the schema based on what's in the DB. So that I can add fields to the schema dynamically. I didn't know about the field "guessing" part.

Re: SolrJ: SolrInputDocument.addField()

2021-02-15 Thread Shawn Heisey
On 2/15/2021 6:52 AM, Steven White wrote: It looks to me that SolrInputDocument.addField() is either missnamed or isn't well implemented. When it is called on a field that doesn't exist in the schema, it will create that field and give it a type based on the data. Not only that, it will set

Re: SolrJ: SolrInputDocument.addField()

2021-02-15 Thread Steven White
Thanks Shawn. It looks to me that SolrInputDocument.addField() is either missnamed or isn't well implemented. When it is called on a field that doesn't exist in the schema, it will create that field and give it a type based on the data. Not only that, it will set default values. For example,

Re: Elevation in dataDir in Solr Cloud

2021-02-15 Thread Mónica Marrero
in the data dir in cloud > mode.) > > I don't know if it will work, but one thing you might want to experiment > with is putting your elevate.xml back the configset in zk, and updating it > on the fly in zk -- then see if it gets reloaded by each core the next > time the index cha

Re: SolrJ: SolrInputDocument.addField()

2021-02-14 Thread Shawn Heisey
On 2/14/2021 9:00 AM, Steven White wrote: It looks like I'm misusing SolrJ API SolrInputDocument.addField() thus I need clarification. Here is an example of what I have in my code: SolrInputDocument doc = new SolrInputDocument(); doc.addField("MyFieldOne", "some data");

Re: Down Replica is elected as Leader (solr v8.7.0)

2021-02-14 Thread mmb1234
We found that for the shard that does not get a leader, the tlog replay did not complete (we don't see "log replay finished", "creating leader registration node", "I am the new leader" etc log messages) for hours. Also not sure why the TLOG are 10's of GBs (anywhere from 30 to 40GB).

Re: Atomic Update (nested), Unified Highlighter and Lazy Field Loading => Invalid Index

2021-02-14 Thread David Smiley
Hello Ronen, Can you please file a JIRA issue? Some quick searches did not turn anything up. It would be super helpful to me if you could list a series of steps with Solr out-of-the-box in 8.8 including what data to index and query. Solr already includes the "tech products" sample data; maybe

Re: Asymmetric Key Size not sufficient

2021-02-14 Thread Mahir Kabir
Hi, Thanks for letting me know. Best, Mahir On Sun, Feb 14, 2021, 9:08 AM Mike Drob wrote: > Future vulnerability reports should be sent to secur...@apache.org so > that they can be resolved privately. > > Thank you > > On Fri, Feb 12, 2021 at 10:17 AM Ishan Chattopadhyaya < >

Re: Asymmetric Key Size not sufficient

2021-02-14 Thread Mike Drob
Future vulnerability reports should be sent to secur...@apache.org so that they can be resolved privately. Thank you On Fri, Feb 12, 2021 at 10:17 AM Ishan Chattopadhyaya < ichattopadhy...@gmail.com> wrote: > Recent versions of Solr use 2048. > >

Re: CVE-2019-17558 on SOLR 6.1

2021-02-13 Thread TK Solr
(Resending to the list. Sorry, Rick.) FYI, my client was using 8.3.1, which should have mitigated the attack. But the server was suffering a sudden death of the Solr process, and the log showed it was being attacked using CVE-2019-17558. We blocked the external access of Solr API. Then this

Re: Down Replica is elected as Leader (solr v8.7.0)

2021-02-13 Thread mmb1234
By tracing the output in the log files we see the following sequence. Overseer role list has POD-1, POD-2, POD-3 in that order POD-3 has 2 shard leaders. POD-3 restarts. A) Logs for the shard whose leader moves successfully from POD-3 to POD-1 On POD-1: o.a.s.c.ShardLeaderElectionContext

Re: CVE-2019-17558 on SOLR 6.1

2021-02-13 Thread Rick Tham
Thanks Shawn. On Fri, Feb 12, 2021 at 7:43 PM Shawn Heisey wrote: > On 2/12/2021 11:17 AM, Rick Tham wrote: > > I am trying to figure out if the following is an additioanal valid > > mitigation step for CVE-2019-17558 on SOLR 6.1. None of our > solrconfig.xml > > contains the lib references to

Re: CVE-2019-17558 on SOLR 6.1

2021-02-12 Thread Shawn Heisey
On 2/12/2021 11:17 AM, Rick Tham wrote: I am trying to figure out if the following is an additioanal valid mitigation step for CVE-2019-17558 on SOLR 6.1. None of our solrconfig.xml contains the lib references to the velocity jar files as follows: l It doesn't appear that you can add these

Re: Extremely Small Segments

2021-02-12 Thread Shawn Heisey
On 2/12/2021 4:30 AM, yasoobhaider wrote: Note: Nothing out of the ordinary in logs. Only /update request logs. Can you share your logs? The best option would be to include everything in the logs directory. Hopefully you have not altered the default logging config, which sets the detail to

Re: Why Solr questions on stackoverflow get very few views and answers, if at all?

2021-02-12 Thread Walter Underwood
Many questions have responses as comments, but no actual answers. One frequent contributor doesn’t understand how StackOverflow works, so he’s posting answers as comments. He’s also doing conversations instead of crafting a useful, complete answer. I just answered a few. Mostly with “don’t use

Re: Elevation in dataDir in Solr Cloud

2021-02-12 Thread Chris Hostetter
nges (NOTE that there will almost certainly need to be an index change for it to re-load, since I don't see any indication that it's watching for changes in zk) FWIW: the way most people seem to be using QEC these days is to have an empty elevate.xml file, and then have their application use some othe

Re: [ANNOUNCE] Apache Solr 8.8.0 released

2021-02-12 Thread Ishan Chattopadhyaya
Hi all, This release contains a critical bug, that should be fixed in 8.8.1 shortly. Please avoid upgrading to this release for the moment. https://twitter.com/ichattopadhyaya/status/1360163382171586562 Apologies for the inconvenience. Thanks, Ishan On Mon, Feb 1, 2021 at 6:01 PM Noble Paul

Re: SOLR upgrade

2021-02-12 Thread David Hastings
i generally will only upgrade every other release. since i started with 1.4, went to 3->5->7.X, and never EVER a .0 or an even .X release, On Fri, Feb 12, 2021 at 12:01 PM Ishan Chattopadhyaya < ichattopadhy...@gmail.com> wrote: > Just avoid 8.8.0 for the moment, until 8.8.1 is released. 8.7.x

Re: SOLR upgrade

2021-02-12 Thread Ishan Chattopadhyaya
Just avoid 8.8.0 for the moment, until 8.8.1 is released. 8.7.x should be fine. On Fri, Feb 12, 2021 at 10:28 PM Alessandro Benedetti wrote: > Hi, > following up on Charlie's detailed response I would recommend carefully > assess the code you are using to interact with Apache Solr (on top of

Re: SOLR upgrade

2021-02-12 Thread Alessandro Benedetti
Hi, following up on Charlie's detailed response I would recommend carefully assess the code you are using to interact with Apache Solr (on top of the Solr changes themselves). Assuming you are using some sort of client, it's extremely important to fully understand both the syntax and semantic of

Re: Extremely Small Segments

2021-02-12 Thread Alessandro Benedetti
Hi Yasoob, Can you check in the log when hard commits really happen? I ended up sometimes with auto soft/hard commit config in the wrong place of the solrconfig.xml and for that reason getting un-expected behaviour. Your assumptions are correct, the ramBuffer flushes as soon as one of the

Re: Asymmetric Key Size not sufficient

2021-02-12 Thread Ishan Chattopadhyaya
Recent versions of Solr use 2048. https://github.com/apache/lucene-solr/blob/branch_8_6/solr/core/src/java/org/apache/solr/util/CryptoKeys.java#L332 Thanks for your report. On Fri, Feb 12, 2021 at 3:44 PM Mahir Kabir wrote: > Hello, > > I am a Ph.D. student at Virginia Tech, USA. While working

Re: Why Solr questions on stackoverflow get very few views and answers, if at all?

2021-02-12 Thread samuel...@grupoinditex.mail.onmicrosoft.com


Re: Why Solr questions on stackoverflow get very few views and answers, if at all?

2021-02-12 Thread Alexandre Rafalovitch
I answered quite a bunch a whole ago, as part of book writing process. I think a lot of them were missing core information like version of Solr. So they were not very timeless. The list allows a conversation and multiple perspectives, which is better than a one shot answer. Regards, Alex On

Re: Why Solr questions on stackoverflow get very few views and answers, if at all?

2021-02-12 Thread Charlie Hull
I've answered a few in my time, but my experience is that if you do so you then get emailed a whole load more questions some of which aren't even relevant to Solr! Also, quite a few of them are 'here is 3 pages of code please debug it for me no I won't tell the actual error I got'. This is

Re: Excessive logging 8.8.0

2021-02-11 Thread Ishan Chattopadhyaya
This should be fixed now in https://issues.apache.org/jira/browse/SOLR-15136. Thanks Markus. On Sat, Feb 6, 2021 at 7:33 PM Ishan Chattopadhyaya < ichattopadhy...@gmail.com> wrote: > I think we should release a 8.8.1 with that fixed. > > On Fri, 5 Feb, 2021, 4:09 pm Markus Jelsma, > wrote: > >>

Re: Collection Creation across DC

2021-02-11 Thread Dominique Bejean
Hi, Sorry, it is in French, but here is my suggestion in order to replace the deprecated CDCR and achieve HA https://www.eolya.fr/2020/11/16/solrcloud-disaster-recovery-alternative-a-cdcr/ In short, each shard has one PULL replica on remote datacenter and these PULL replicas are excluded from

Re: Down Replica is elected as Leader (solr v8.7.0)

2021-02-11 Thread Rahul Goswami
I haven’t delved into the exact reason for this, but what generally helps to avoid this situation in a cluster is i) During shutdown (in case you need to restart the cluster), let the overseer node be the last one to shut down. ii) While restarting, let the Overseer node be the first one to start

Re: Using multiple language stop words in Solr Core

2021-02-11 Thread Markus Jelsma
Hell Abhay, Do not enable stopwords unless you absolutely know what you are doing. In general, it is a bad practice that somehow still lingers on. But to answer the question, you must have one field and fieldType for each language, so language specific filters go there. Also, using edismax and

Re: UPDATE collection's Rule-based Replica Placement

2021-02-11 Thread Aroop Ganguly
Moshe An indirect way to do this could be to take backup of this collection and then restore with the desired placement rules. Backup: Example: curl “https://solr.foo.com/solr/admin/collections?

Re: support Rich Document

2021-02-10 Thread Jörn Franke
You can store them on the filesystem and a link to them in Solr. Your search application could fetch them from the filesystem and serve them to the users. Alternatively serve them as WebDAV, SharePoint or whatever your organization sets as standard. It does not make sense to store them in

Re: UPDATE collection's Rule-based Replica Placement

2021-02-10 Thread Ilan Ginzburg
Do you look for something that would move existing collection replicas to comply with a new set of rules? I'm afraid that doesn't exist, but you can use the Collection API to move replicas "manually". Ilan On Tue, Feb 9, 2021 at 1:10 PM mosheB wrote: > > Hi community, > Using Solr 8.3, is there

RE: Ghost Documents or Shards out of Sync

2021-02-09 Thread Chris Hostetter
: Let me add some background. A user triggers an operation which under the : hood needs to update a single field. Atomic update fails with a message : that one of the mandatory fields is missing (which is strange by : itself). When I query Solr for the exact document (fq with the document :

Re: SOLR upgrade

2021-02-09 Thread Charlie Hull
Hi Lulu, I'm afraid you're going to have to recognise that Solr 5.2.1 is very out-of-date and the changes between this version and the current 8.x releases are significant. A direct jump is I think the only sensible option. Although you could take the current configuration and attempt to

Re: Unable to connect to an 8.8.0 Solr Cloud database via API

2021-02-08 Thread Timothy Potter
<https://vimeo.com/unisys>[image: Grey_UB] <http://blogs.unisys.com/> > > > > *From:* Timothy Potter > *Sent:* 08 February 2021 15:44 > *To:* solr-user@lucene.apache.org > *Subject:* Re: Unable to connect to an 8.8.0 Solr Cloud database via API > > > > *EX

Re: Unable to connect to an 8.8.0 Solr Cloud database via API

2021-02-08 Thread Timothy Potter
Thanks for the additional details Matthew. I created this JIRA to track this problem: https://issues.apache.org/jira/browse/SOLR-15145. Please add any additional information to that ticket if needed. Are you able to upgrade your SolrJ client JAR to 8.8.0? If not, I understand but that would be a

Re: Unable to connect to an 8.8.0 Solr Cloud database via API

2021-02-08 Thread Timothy Potter
What version of SolrJ is embedded in your uleaf.ear file? There have been changes in how we deal with URLs stored in ZK in 8.8 --> SOLR-12182 On Fri, Feb 5, 2021 at 2:34 AM Flowerday, Matthew J < matthew.flower...@gb.unisys.com> wrote: > Hi There > > > > I have been checking out the latest

Re: Authentication for all but selects

2021-02-07 Thread Robert Douglas
Thanks Craig! I got the following to work. { "authentication":{ "blockUnknown":false, "class":"solr.BasicAuthPlugin", "credentials":{"solr":"..."}}, "authorization":{ "class":"solr.RuleBasedAuthorizationPlugin", "user-role":{"solr":"admin"}, "permissions":[ {

RE: Ghost Documents or Shards out of Sync

2021-02-07 Thread Nussbaum, Ronen
: יום ב 01 פברואר 2021 22:58 To: solr-user@lucene.apache.org Subject: Re: Ghost Documents or Shards out of Sync To expand on what Jason suggested, if the issue is the non-deterministic ordering due to staggered commits per replica, you may have more consistency with TLOG replicas rather than the NRT

Re: Solr Slack Workspace

2021-02-06 Thread Ishan Chattopadhyaya
in and start using. > > > > > > > > On Tue, Jan 26, 2021 at 5:38 AM Ishan Chattopadhyaya < > > > > ichattopadhy...@gmail.com> wrote: > > > > > > > > > Thanks ufuk. I'll take a look. > > > > > > > > >

Re: Excessive logging 8.8.0

2021-02-06 Thread Ishan Chattopadhyaya
I think we should release a 8.8.1 with that fixed. On Fri, 5 Feb, 2021, 4:09 pm Markus Jelsma, wrote: > Thanks! > > Op do 4 feb. 2021 om 20:04 schreef Chris Hostetter < > hossman_luc...@fucit.org > >: > > > > > FWIW: that log message was added to branch_8x by 3c02c9197376 as part of > >

Re: Json Faceting Performance Issues on solr v8.7.0

2021-02-05 Thread Michael Gibney
Ah! that's significant. The latency is likely due to building the OrdinalMap (which maps segment ords to global ords) ... "dvhash" (assuming the relevant fields are not multivalued) will very likely work; "dvhash" doesn't map to global ords, so doesn't need to build the OrdinalMap (which gets

Re: Solr Slack Workspace

2021-02-05 Thread Anshum Gupta
26, 2021 at 5:38 AM Ishan Chattopadhyaya < > > > ichattopadhy...@gmail.com> wrote: > > > > > > > Thanks ufuk. I'll take a look. > > > > > > > > On Tue, 26 Jan, 2021, 4:05 pm ufuk yılmaz, > > > > > > > wrote: &g

Re: Json Faceting Performance Issues on solr v8.7.0

2021-02-05 Thread mmb1234
> Does this happen on a warm searcher (are subsequent requests with no intervening updates _ever_ fast?)? Subsequent response times very fast if searcher remains open. As a control test, I faceted on the same field that I used in the q param. 1. Start solr 2. Execute q=resultId:x=0 =>

Re: Json Faceting Performance Issues on solr v8.7.0

2021-02-05 Thread Michael Gibney
Apologies, I missed deducing from the request url that you're already talking strictly about single-shard requests (so everything I was suggesting about shards.preference etc. is not applicable). "dvhash" is still worth a try though, esp. with `numFound` being 943 (out of 185 million!). Does this

Re: Solr Slack Workspace

2021-02-05 Thread Justin Sweeney
shan Chattopadhyaya < > > > ichattopadhy...@gmail.com> wrote: > > > > > > > Thanks ufuk. I'll take a look. > > > > > > > > On Tue, 26 Jan, 2021, 4:05 pm ufuk yılmaz, > > > > > > > wrote: > > > > > > > >

Re: Json Faceting Performance Issues on solr v8.7.0

2021-02-05 Thread mmb1234
Ok. I'll try that. Meanwhile query on resultId is subsecond response. But the immediate next query for faceting takes 40+secs. The core has 185million docs and 63GB index size. curl 'http://localhost:8983/solr/TestCollection_shard1_replica_t3/query?q=resultId:x=0' {

Re: Json Faceting Performance Issues on solr v8.7.0

2021-02-05 Thread Michael Gibney
`resultId` sounds like it might be a relatively high-cardinality field (lots of unique values)? What's your number of shards, and replicas per shard? SOLR-15008 (note: not a bug) describes a situation that may be fundamentally similar to yours (though to be sure it's impossible to say for sure

RE: Authentication for all but selects

2021-02-05 Thread Oakley, Craig (NIH/NLM/NCBI) [C]
What works for us is having something like this at the bottom of security.json: { "name":"open_select", "path":"/select/*", "role":null, "index":9}, { "name":"catch-all-nocollection", "collection":null, "path":"/*",

Re: Extract a list of the most recent field values?

2021-02-05 Thread Alexandre Rafalovitch
Rewriting: *) https://lucene.apache.org/solr/guide/8_8/json-request-api.html#json-parameter-merging , there is a way to represent most (all?) of the structure with json.x parameter. *) Also, you can create custom Request Handlers in solrconfig.xml with a lot of those parameters either as defaults

Re: Clarification on term facet method dvhash

2021-02-05 Thread Michael Gibney
t; > One thing I can add is I tried dvhash with a string multi-valued field, it > worked and didn’t throw any error but I don’t know if it got silently > ignored or just worked. > > Sent from Mail for Windows 10 > > From: Michael Gibney > Sent: 05 February 2021 20:52 > To

RE: Clarification on term facet method dvhash

2021-02-05 Thread ufuk yılmaz
To: solr-user@lucene.apache.org Subject: Re: Clarification on term facet method dvhash Correction!: wrt "dvhash" and numeric types, it looks like I had it exactly backwards! single-valued numeric types _do_ use (even default to) "dvhash" ... sorry about that! I stand by the

Re: Clarification on term facet method dvhash

2021-02-05 Thread Michael Gibney
Correction!: wrt "dvhash" and numeric types, it looks like I had it exactly backwards! single-valued numeric types _do_ use (even default to) "dvhash" ... sorry about that! I stand by the rest of the previous message though, which applies at a minimum to string-like fields. On Fri, Feb 5, 2021 at

Re: Clarification on term facet method dvhash

2021-02-05 Thread Michael Gibney
> Performance and resource is still affected by 30M unique values of T right? Yes. The main performance issue would be the per-request allocation of a 30M-element `long[]` for "dv" or "uif" methods (which are by far the most common methods in practice). With low enough request volume and large

Re: Extract a list of the most recent field values?

2021-02-05 Thread Alexandre Rafalovitch
This feels like basic faceting on category, but you are trying to make a latest record, rather than count as a sorting/grouping principle. How about using JSON Facets? https://lucene.apache.org/solr/guide/8_8/json-facet-api.html I would do the first level as range facet and do your dates at

Re: 404 Errors on update/extract

2021-02-05 Thread Alexandre Rafalovitch
Hi Leon, Feel free to create JIRA issue https://issues.apache.org/jira/secure/Dashboard.jspa and then do Github pull request to fix the example name. The documentation is in asciidoc format at: https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide/src with names matching those on

Re: 404 Errors on update/extract

2021-02-05 Thread nq
Hi Alex, Thanks a lot for your help! I have tested the same using the 'techproducts' example as proposed, and it worked fine. You are right, the documentation seems to be outdated in this aspect. I have just reviewed the solrconfig.xml of the 'schemaless' example and found all the Solr

Re: 404 Errors on update/extract

2021-02-05 Thread Alexandre Rafalovitch
I think the extract handler is not defined in schemaless. This may be a change from before and the documentation is out of sync. Can you try 'techproducts' example instead of schemaless: bin/solr stop (if you are still running it) bin/solr start -e techproducts Then the import command. The Tika

Re: Extract a list of the most recent field values?

2021-02-05 Thread Emir Arnautović
Hi Jimi, It seems to me that you could get the results using collapsing query parse: https://lucene.apache.org/solr/guide/6_6/collapse-and-expand-results.html HTH, Emir -- Monitoring - Log Management - Alerting -

Re: Recovering forever after upgrade to 8.8.0: Timeout waiting for collection state

2021-02-05 Thread Henrik B A
On Fri, Feb 5, 2021 at 10:58 AM Henrik Brautaset Aronsen wrote: > After upgrading our Solr Cloud collections from 8.7.0 to 8.8.0 I struggle > to get a consistent state. We have 8 servers hosting 3 collections, with > shards/replicas spread over alle the servers. > > All replicas on solr3577 is

Re: Excessive logging 8.8.0

2021-02-05 Thread Markus Jelsma
Thanks! Op do 4 feb. 2021 om 20:04 schreef Chris Hostetter : > > FWIW: that log message was added to branch_8x by 3c02c9197376 as part of > SOLR-15052 ... it's based on master commit 8505d4d416fd -- but that does > not add that same logging message ... so it definitely smells like a > mistake to

Re: Urgent- General Question about document Indexing frequency in solr

2021-02-04 Thread Scott Stults
Manisha, The most general recommendation around commits is to not explicitly commit after every update. There are settings that will let Solr automatically commit after some threshold is met, and by delegating commits to that mechanism you can generally ingest faster. See this blog post that

Re: Excessive logging 8.8.0

2021-02-04 Thread Chris Hostetter
FWIW: that log message was added to branch_8x by 3c02c9197376 as part of SOLR-15052 ... it's based on master commit 8505d4d416fd -- but that does not add that same logging message ... so it definitely smells like a mistake to me that 8x would add this INFO level log message that master

Re: SSL using CloudSolrClient

2021-02-03 Thread ChienHuaWang
Thanks for the information. Could you advise whether CloudSolrClient is compatible with non-TLS? even client is not configure, it can still connect to Solr (TLS enabled)? -- Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: SolrCloud keeps crashing

2021-02-03 Thread TK Solr
Oops, I should have referenced this document rather: https://www.tenable.com/cve/CVE-2019-17558 On 2/3/21 2:42 PM, TK Solr wrote: Victor & Satish, Is your Solr accessible from the Internet by anyone? If so, your site is being attacked by a bot

Re: SolrCloud keeps crashing

2021-02-03 Thread TK Solr
Victor & Satish, Is your Solr accessible from the Internet by anyone? If so, your site is being attacked by a bot using this security hole: https://www.tenable.com/blog/cve-2019-17558-apache-solr-vulnerable-to-remote-code-execution-zero-day-vulnerability If that is the case, try blocking the

Re: How to get case-sensitive Terms?

2021-02-03 Thread Alexandre Rafalovitch
What about copyField with the target being index only (docValue only?) and no lowercase on the target field type? Solr is not a database, you are optimising for search. So duplicate, multi-process, denormalise, create custom field types, etc. Regards, Alex On Wed., Feb. 3, 2021, 4:43 p.m.

Re: How to get case-sensitive Terms?

2021-02-03 Thread elivis
Alexandre Rafalovitch wrote > It is documented in the reference guide: > https://lucene.apache.org/solr/guide/8_8/analysis-screen.html > > Hope it helps, >Alex. > > On Tue, 2 Feb 2021 at 00:57, elivis > elivis@ > wrote: >> >> Alexandre Rafalovitch wrote >> > Admin UI also allows you to

<    1   2   3   4   5   6   7   8   9   10   >