Re: SolrJ: SolrInputDocument.addField()

2021-02-15 Thread Shawn Heisey
On 2/15/2021 6:52 AM, Steven White wrote: It looks to me that SolrInputDocument.addField() is either missnamed or isn't well implemented. When it is called on a field that doesn't exist in the schema, it will create that field and give it a type based on the data. Not only that, it will set

Re: SolrJ: SolrInputDocument.addField()

2021-02-15 Thread Steven White
Thanks Shawn. It looks to me that SolrInputDocument.addField() is either missnamed or isn't well implemented. When it is called on a field that doesn't exist in the schema, it will create that field and give it a type based on the data. Not only that, it will set default values. For example,

Re: Elevation in dataDir in Solr Cloud

2021-02-15 Thread Mónica Marrero
Of course, here is the full stack trace (collection 'techproducts' with just one core to make it easier): org.apache.solr.common.SolrException: Unable to reload core [techproducts2_shard1_replica_n1] at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1472) at

Re: SolrJ: SolrInputDocument.addField()

2021-02-14 Thread Shawn Heisey
On 2/14/2021 9:00 AM, Steven White wrote: It looks like I'm misusing SolrJ API SolrInputDocument.addField() thus I need clarification. Here is an example of what I have in my code: SolrInputDocument doc = new SolrInputDocument(); doc.addField("MyFieldOne", "some data");

Re: Down Replica is elected as Leader (solr v8.7.0)

2021-02-14 Thread mmb1234
We found that for the shard that does not get a leader, the tlog replay did not complete (we don't see "log replay finished", "creating leader registration node", "I am the new leader" etc log messages) for hours. Also not sure why the TLOG are 10's of GBs (anywhere from 30 to 40GB).

Re: Atomic Update (nested), Unified Highlighter and Lazy Field Loading => Invalid Index

2021-02-14 Thread David Smiley
Hello Ronen, Can you please file a JIRA issue? Some quick searches did not turn anything up. It would be super helpful to me if you could list a series of steps with Solr out-of-the-box in 8.8 including what data to index and query. Solr already includes the "tech products" sample data; maybe

Re: Asymmetric Key Size not sufficient

2021-02-14 Thread Mahir Kabir
Hi, Thanks for letting me know. Best, Mahir On Sun, Feb 14, 2021, 9:08 AM Mike Drob wrote: > Future vulnerability reports should be sent to secur...@apache.org so > that they can be resolved privately. > > Thank you > > On Fri, Feb 12, 2021 at 10:17 AM Ishan Chattopadhyaya < >

Atomic Update (nested), Unified Highlighter and Lazy Field Loading => Invalid Index

2021-02-14 Thread Ronen Nussbaum
Hi All, I discovered a strange behaviour with this combination. Not only the atomic update fails, the child documents are not properly indexed, and you can't use highlights on their text fields. Currently there is no workaround other than reindex. Checked on 8.3.0, 8.6.1 and 8.8.0. 1. Configure

SolrJ: SolrInputDocument.addField()

2021-02-14 Thread Steven White
Hi everyone, It looks like I'm misusing SolrJ API SolrInputDocument.addField() thus I need clarification. Here is an example of what I have in my code: SolrInputDocument doc = new SolrInputDocument(); doc.addField("MyFieldOne", "some data"); doc.addField("MyFieldTwo", 100); The

Re: Asymmetric Key Size not sufficient

2021-02-14 Thread Mike Drob
Future vulnerability reports should be sent to secur...@apache.org so that they can be resolved privately. Thank you On Fri, Feb 12, 2021 at 10:17 AM Ishan Chattopadhyaya < ichattopadhy...@gmail.com> wrote: > Recent versions of Solr use 2048. > >

Re: CVE-2019-17558 on SOLR 6.1

2021-02-13 Thread TK Solr
(Resending to the list. Sorry, Rick.) FYI, my client was using 8.3.1, which should have mitigated the attack. But the server was suffering a sudden death of the Solr process, and the log showed it was being attacked using CVE-2019-17558. We blocked the external access of Solr API. Then this

Re: Down Replica is elected as Leader (solr v8.7.0)

2021-02-13 Thread mmb1234
By tracing the output in the log files we see the following sequence. Overseer role list has POD-1, POD-2, POD-3 in that order POD-3 has 2 shard leaders. POD-3 restarts. A) Logs for the shard whose leader moves successfully from POD-3 to POD-1 On POD-1: o.a.s.c.ShardLeaderElectionContext

Re: CVE-2019-17558 on SOLR 6.1

2021-02-13 Thread Rick Tham
Thanks Shawn. On Fri, Feb 12, 2021 at 7:43 PM Shawn Heisey wrote: > On 2/12/2021 11:17 AM, Rick Tham wrote: > > I am trying to figure out if the following is an additioanal valid > > mitigation step for CVE-2019-17558 on SOLR 6.1. None of our > solrconfig.xml > > contains the lib references to

Re: CVE-2019-17558 on SOLR 6.1

2021-02-12 Thread Shawn Heisey
On 2/12/2021 11:17 AM, Rick Tham wrote: I am trying to figure out if the following is an additioanal valid mitigation step for CVE-2019-17558 on SOLR 6.1. None of our solrconfig.xml contains the lib references to the velocity jar files as follows: l It doesn't appear that you can add these

Re: Extremely Small Segments

2021-02-12 Thread Shawn Heisey
On 2/12/2021 4:30 AM, yasoobhaider wrote: Note: Nothing out of the ordinary in logs. Only /update request logs. Can you share your logs? The best option would be to include everything in the logs directory. Hopefully you have not altered the default logging config, which sets the detail to

Re: Why Solr questions on stackoverflow get very few views and answers, if at all?

2021-02-12 Thread Walter Underwood
Many questions have responses as comments, but no actual answers. One frequent contributor doesn’t understand how StackOverflow works, so he’s posting answers as comments. He’s also doing conversations instead of crafting a useful, complete answer. I just answered a few. Mostly with “don’t use

Re: Elevation in dataDir in Solr Cloud

2021-02-12 Thread Chris Hostetter
: I need to have the elevate.xml file updated frequently and I was wondering : if it is possible to put this file in the dataDir folder when using Solr : Cloud. I know that this is possible in the standalone mode, and I haven't : seen in the documentation [1] that it can not be done in Cloud. :

CVE-2019-17558 on SOLR 6.1

2021-02-12 Thread Rick Tham
We are using Solr 6.1 and at the moment we can not upgrade due to application dependencies. We have mitigation steps in place to only trust specific machines within our DMZ. I am trying to figure out if the following is an additioanal valid mitigation step for CVE-2019-17558 on SOLR 6.1. None of

Re: [ANNOUNCE] Apache Solr 8.8.0 released

2021-02-12 Thread Ishan Chattopadhyaya
Hi all, This release contains a critical bug, that should be fixed in 8.8.1 shortly. Please avoid upgrading to this release for the moment. https://twitter.com/ichattopadhyaya/status/1360163382171586562 Apologies for the inconvenience. Thanks, Ishan On Mon, Feb 1, 2021 at 6:01 PM Noble Paul

Re: SOLR upgrade

2021-02-12 Thread David Hastings
i generally will only upgrade every other release. since i started with 1.4, went to 3->5->7.X, and never EVER a .0 or an even .X release, On Fri, Feb 12, 2021 at 12:01 PM Ishan Chattopadhyaya < ichattopadhy...@gmail.com> wrote: > Just avoid 8.8.0 for the moment, until 8.8.1 is released. 8.7.x

Re: SOLR upgrade

2021-02-12 Thread Ishan Chattopadhyaya
Just avoid 8.8.0 for the moment, until 8.8.1 is released. 8.7.x should be fine. On Fri, Feb 12, 2021 at 10:28 PM Alessandro Benedetti wrote: > Hi, > following up on Charlie's detailed response I would recommend carefully > assess the code you are using to interact with Apache Solr (on top of

Re: SOLR upgrade

2021-02-12 Thread Alessandro Benedetti
Hi, following up on Charlie's detailed response I would recommend carefully assess the code you are using to interact with Apache Solr (on top of the Solr changes themselves). Assuming you are using some sort of client, it's extremely important to fully understand both the syntax and semantic of

Re: Extremely Small Segments

2021-02-12 Thread Alessandro Benedetti
Hi Yasoob, Can you check in the log when hard commits really happen? I ended up sometimes with auto soft/hard commit config in the wrong place of the solrconfig.xml and for that reason getting un-expected behaviour. Your assumptions are correct, the ramBuffer flushes as soon as one of the

Re: Asymmetric Key Size not sufficient

2021-02-12 Thread Ishan Chattopadhyaya
Recent versions of Solr use 2048. https://github.com/apache/lucene-solr/blob/branch_8_6/solr/core/src/java/org/apache/solr/util/CryptoKeys.java#L332 Thanks for your report. On Fri, Feb 12, 2021 at 3:44 PM Mahir Kabir wrote: > Hello, > > I am a Ph.D. student at Virginia Tech, USA. While working

Elevation in dataDir in Solr Cloud

2021-02-12 Thread Mónica Marrero
Hi, I need to have the elevate.xml file updated frequently and I was wondering if it is possible to put this file in the dataDir folder when using Solr Cloud. I know that this is possible in the standalone mode, and I haven't seen in the documentation [1] that it can not be done in Cloud. I am

Re: Why Solr questions on stackoverflow get very few views and answers, if at all?

2021-02-12 Thread samuel...@grupoinditex.mail.onmicrosoft.com


Re: Why Solr questions on stackoverflow get very few views and answers, if at all?

2021-02-12 Thread Alexandre Rafalovitch
I answered quite a bunch a whole ago, as part of book writing process. I think a lot of them were missing core information like version of Solr. So they were not very timeless. The list allows a conversation and multiple perspectives, which is better than a one shot answer. Regards, Alex On

Extremely Small Segments

2021-02-12 Thread samuel...@grupoinditex.mail.onmicrosoft.com


Extremely Small Segments

2021-02-12 Thread yasoobhaider
Hi I am migrating from master slave to Solr Cloud but I'm running into problems with indexing. Cluster details: 8 machines of 64GB memory, each hosting 1 replica. 4 shards, 2 replica of each. Heap size is 16GB. Collection details: Total number of docs: ~250k (but only 50k are indexed right

Extremely small segments

2021-02-12 Thread samuel...@grupoinditex.mail.onmicrosoft.com


Extremely small segments

2021-02-12 Thread Yasoob Haider
Hi I am migrating from master slave to Solr Cloud but I'm running into problems with indexing. Cluster details: 8 machines of 64GB memory, each hosting 1 replica. 4 shards, 2 replica of each. Heap size is 16GB. Collection details: Total number of docs: ~250k (but only 50k are indexed right

Re: Why Solr questions on stackoverflow get very few views and answers, if at all?

2021-02-12 Thread Charlie Hull
I've answered a few in my time, but my experience is that if you do so you then get emailed a whole load more questions some of which aren't even relevant to Solr! Also, quite a few of them are 'here is 3 pages of code please debug it for me no I won't tell the actual error I got'. This is

Why Solr questions on stackoverflow get very few views and answers, if at all?

2021-02-12 Thread ufuk yılmaz
Is it because the main place for q is this mailing list, or somewhere else that I don’t know? Or Solr isn’t ‘hot’ as some other topics? Sent from Mail for Windows 10

Asymmetric Key Size not sufficient

2021-02-12 Thread Mahir Kabir
Hello, I am a Ph.D. student at Virginia Tech, USA. While working on a security project-related work, we came across the following vulnerability in the source code - In file https://github.com/apache/lucene-solr/blob/branch_6_6/solr/core/src/java/org/apache/solr/util/CryptoKeys.java

Re: Excessive logging 8.8.0

2021-02-11 Thread Ishan Chattopadhyaya
This should be fixed now in https://issues.apache.org/jira/browse/SOLR-15136. Thanks Markus. On Sat, Feb 6, 2021 at 7:33 PM Ishan Chattopadhyaya < ichattopadhy...@gmail.com> wrote: > I think we should release a 8.8.1 with that fixed. > > On Fri, 5 Feb, 2021, 4:09 pm Markus Jelsma, > wrote: > >>

Re: Collection Creation across DC

2021-02-11 Thread Dominique Bejean
Hi, Sorry, it is in French, but here is my suggestion in order to replace the deprecated CDCR and achieve HA https://www.eolya.fr/2020/11/16/solrcloud-disaster-recovery-alternative-a-cdcr/ In short, each shard has one PULL replica on remote datacenter and these PULL replicas are excluded from

Re: Down Replica is elected as Leader (solr v8.7.0)

2021-02-11 Thread Rahul Goswami
I haven’t delved into the exact reason for this, but what generally helps to avoid this situation in a cluster is i) During shutdown (in case you need to restart the cluster), let the overseer node be the last one to shut down. ii) While restarting, let the Overseer node be the first one to start

Sv: [SPAM] Process copyField only when field is absent in update

2021-02-11 Thread Hullegård , Jimi
I had a similar need in an old solr project. I was able to handle it with this configuration in solrconfig.xml: lastModified realLastModified realLastModified The logic is basically: Copy the default value (lastModified here) into the target field (realLastModified). If the target

Solr wiki page update

2021-02-11 Thread Vincent Brehin
Hi community members, I work for Adelean https://www.adelean.com/ , we are offering services around everything Search related, and especially Solr consulting and support. We are based in Paris and operate mainly in France. Is it possible to list our company on the support page (Support - SOLR -

Process copyField only when field is absent in update

2021-02-11 Thread ufuk yılmaz
When I have a copyfield directive like,

Re: Using multiple language stop words in Solr Core

2021-02-11 Thread Markus Jelsma
Hell Abhay, Do not enable stopwords unless you absolutely know what you are doing. In general, it is a bad practice that somehow still lingers on. But to answer the question, you must have one field and fieldType for each language, so language specific filters go there. Also, using edismax and

Using multiple language stop words in Solr Core

2021-02-11 Thread Abhay Kumar
Hello Team, Solr provides some data type out of box in managed schema for different languages such as english, french, japanies etc. We are using common data type "text_general" for fields declaration and using stopwards.txt for stopword filtering.

Re: UPDATE collection's Rule-based Replica Placement

2021-02-11 Thread Aroop Ganguly
Moshe An indirect way to do this could be to take backup of this collection and then restore with the desired placement rules. Backup: Example: curl “https://solr.foo.com/solr/admin/collections?

Re: support Rich Document

2021-02-10 Thread Jörn Franke
You can store them on the filesystem and a link to them in Solr. Your search application could fetch them from the filesystem and serve them to the users. Alternatively serve them as WebDAV, SharePoint or whatever your organization sets as standard. It does not make sense to store them in

Down Replica is elected as Leader (solr v8.7.0)

2021-02-10 Thread mmb1234
Hello, On reboot of one of the solr nodes in the cluster, we often see a collection's shards with 1. LEADER replica in DOWN state, and/or 2. shard with no LEADER Output from /solr/admin/collections?action=CLUSTERSTATUS is below. Even after 5 to 10 minutes, the collection often does not recover.

support Rich Document

2021-02-10 Thread Luke
HI, I know Solr can index rich documents, but I have one requirement. I have all kind of documents, such as word, pdf, excel, ppt, jpg etcs when Solr indexes them with Tika or OCR, it will extract text and save to solr, but the format will be lost, so when the user opens the document, it is not

Re: UPDATE collection's Rule-based Replica Placement

2021-02-10 Thread Ilan Ginzburg
Do you look for something that would move existing collection replicas to comply with a new set of rules? I'm afraid that doesn't exist, but you can use the Collection API to move replicas "manually". Ilan On Tue, Feb 9, 2021 at 1:10 PM mosheB wrote: > > Hi community, > Using Solr 8.3, is there

Collection Creation across DC

2021-02-10 Thread Revas
Hello, Can we create a collection across data Center ( shard replica is in a different data center) for HA ? Thanks Revas

Index rich document and view

2021-02-10 Thread Luke Oak
Hi, I have all kind of rich documents, such as excel, ppt, PDF, word, jpg ..., I knew Tika or ocr can convert them to text and index it. But when I open the document, the format is changed, how can I keep original document format, is it possible in solr? If not, can I use external field type

Without custom updateRequestProcessorChain: DistributedUpdateProcessorFactory was explicitly disabled from this updateRequestProcessorChain

2021-02-10 Thread diego_70
Hello, We are using SOLR cloud 8.5. Several times per hour we can see these kind of errors in logs: /RunUpdateProcessor has received an AddUpdateCommand containing a document that appears to still contain Atomic document update operations, most likely because DistributedUpdateProcessorFactory

RE: Ghost Documents or Shards out of Sync

2021-02-09 Thread Chris Hostetter
: Let me add some background. A user triggers an operation which under the : hood needs to update a single field. Atomic update fails with a message : that one of the mandatory fields is missing (which is strange by : itself). When I query Solr for the exact document (fq with the document :

UPDATE collection's Rule-based Replica Placement

2021-02-09 Thread mosheB
Hi community, Using Solr 8.3, is there any way to change the replica placment of "running" collection say "from this point forward" or should I recreate the collection and migrate all my data from the existing collection to the new one? Tried to use the COLLECTIONPROP action which doesn't do the

Re: SOLR upgrade

2021-02-09 Thread Charlie Hull
Hi Lulu, I'm afraid you're going to have to recognise that Solr 5.2.1 is very out-of-date and the changes between this version and the current 8.x releases are significant. A direct jump is I think the only sensible option. Although you could take the current configuration and attempt to

SOLR upgrade

2021-02-08 Thread Paul, Lulu
Hi SOLR team, Please may I ask for advice regarding upgrading the SOLR version (our project currently running on solr-5.2.1) to the latest version? What are the steps, breaking changes and potential issues ? Could this be done as an incremental version upgrade or a direct jump to the newest

Re: Unable to connect to an 8.8.0 Solr Cloud database via API

2021-02-08 Thread Timothy Potter
Hi Matthew, Ok, that's great to hear. Thanks for reporting back! Cheers, Tim On Mon, Feb 8, 2021 at 12:26 PM Flowerday, Matthew J < matthew.flower...@gb.unisys.com> wrote: > Hi Tim > > > > Upgrading to solrJ 8.8.0 fixed the issue. > > > > Many Thanks for your help! > > > > Matthew > > > >

Re: Unable to connect to an 8.8.0 Solr Cloud database via API

2021-02-08 Thread Timothy Potter
Thanks for the additional details Matthew. I created this JIRA to track this problem: https://issues.apache.org/jira/browse/SOLR-15145. Please add any additional information to that ticket if needed. Are you able to upgrade your SolrJ client JAR to 8.8.0? If not, I understand but that would be a

Re: Unable to connect to an 8.8.0 Solr Cloud database via API

2021-02-08 Thread Timothy Potter
What version of SolrJ is embedded in your uleaf.ear file? There have been changes in how we deal with URLs stored in ZK in 8.8 --> SOLR-12182 On Fri, Feb 5, 2021 at 2:34 AM Flowerday, Matthew J < matthew.flower...@gb.unisys.com> wrote: > Hi There > > > > I have been checking out the latest

Re: Authentication for all but selects

2021-02-07 Thread Robert Douglas
Thanks Craig! I got the following to work. { "authentication":{ "blockUnknown":false, "class":"solr.BasicAuthPlugin", "credentials":{"solr":"..."}}, "authorization":{ "class":"solr.RuleBasedAuthorizationPlugin", "user-role":{"solr":"admin"}, "permissions":[ {

RE: Ghost Documents or Shards out of Sync

2021-02-07 Thread Nussbaum, Ronen
Hi, Thank you your replies - much appreciated! I'm afraid it's neither one... Let me add some background. A user triggers an operation which under the hood needs to update a single field. Atomic update fails with a message that one of the mandatory fields is missing (which is strange by

Re: Solr Slack Workspace

2021-02-06 Thread Ishan Chattopadhyaya
No, it is not ASF slack. A separate slack org, just for Solr. On Sat, 6 Feb, 2021, 6:35 am Anshum Gupta, wrote: > Hey Ishan, > > Thanks for doing this. Is this the ASF Slack space or something else? > > > On Tue, Feb 2, 2021 at 2:04 AM Ishan Chattopadhyaya < > ichattopadhy...@gmail.com> wrote:

Re: Excessive logging 8.8.0

2021-02-06 Thread Ishan Chattopadhyaya
I think we should release a 8.8.1 with that fixed. On Fri, 5 Feb, 2021, 4:09 pm Markus Jelsma, wrote: > Thanks! > > Op do 4 feb. 2021 om 20:04 schreef Chris Hostetter < > hossman_luc...@fucit.org > >: > > > > > FWIW: that log message was added to branch_8x by 3c02c9197376 as part of > >

Re: Json Faceting Performance Issues on solr v8.7.0

2021-02-05 Thread Michael Gibney
Ah! that's significant. The latency is likely due to building the OrdinalMap (which maps segment ords to global ords) ... "dvhash" (assuming the relevant fields are not multivalued) will very likely work; "dvhash" doesn't map to global ords, so doesn't need to build the OrdinalMap (which gets

Re: Solr Slack Workspace

2021-02-05 Thread Anshum Gupta
Hey Ishan, Thanks for doing this. Is this the ASF Slack space or something else? On Tue, Feb 2, 2021 at 2:04 AM Ishan Chattopadhyaya < ichattopadhy...@gmail.com> wrote: > Hi all, > I've created an invite link for the Slack workspace: > https://s.apache.org/solr-slack. > Please test it out.

Re: Json Faceting Performance Issues on solr v8.7.0

2021-02-05 Thread mmb1234
> Does this happen on a warm searcher (are subsequent requests with no intervening updates _ever_ fast?)? Subsequent response times very fast if searcher remains open. As a control test, I faceted on the same field that I used in the q param. 1. Start solr 2. Execute q=resultId:x=0 =>

Re: Json Faceting Performance Issues on solr v8.7.0

2021-02-05 Thread Michael Gibney
Apologies, I missed deducing from the request url that you're already talking strictly about single-shard requests (so everything I was suggesting about shards.preference etc. is not applicable). "dvhash" is still worth a try though, esp. with `numFound` being 943 (out of 185 million!). Does this

Re: Solr Slack Workspace

2021-02-05 Thread Justin Sweeney
Worked for me and a few others, thanks for doing that! On Tue, Feb 2, 2021 at 5:04 AM Ishan Chattopadhyaya < ichattopadhy...@gmail.com> wrote: > Hi all, > I've created an invite link for the Slack workspace: > https://s.apache.org/solr-slack. > Please test it out. I'll send a broader

Re: Json Faceting Performance Issues on solr v8.7.0

2021-02-05 Thread mmb1234
Ok. I'll try that. Meanwhile query on resultId is subsecond response. But the immediate next query for faceting takes 40+secs. The core has 185million docs and 63GB index size. curl 'http://localhost:8983/solr/TestCollection_shard1_replica_t3/query?q=resultId:x=0' {

timeouts when update sent to non-Leader

2021-02-05 Thread Oakley, Craig (NIH/NLM/NCBI) [C]
We have a problem on a 3.5gig collection running Solr7.4 (we will soon upgrade to Solr8.5.2) Users were often encountering timeout errors of the type shown below My colleague found a blog post at

Re: Json Faceting Performance Issues on solr v8.7.0

2021-02-05 Thread Michael Gibney
`resultId` sounds like it might be a relatively high-cardinality field (lots of unique values)? What's your number of shards, and replicas per shard? SOLR-15008 (note: not a bug) describes a situation that may be fundamentally similar to yours (though to be sure it's impossible to say for sure

Json Faceting Performance Issues on solr v8.7.0

2021-02-05 Thread mmb1234
Hello, I am seeing very slow response from json faceting against a single core (though core is shard leader in a collection). Fields processId and resultId are non-multivalued, indexed and docvalues string (not text). Soft Commit = 5sec (opensearcher=true) and Hard Commit = 10sec because new

RE: Authentication for all but selects

2021-02-05 Thread Oakley, Craig (NIH/NLM/NCBI) [C]
What works for us is having something like this at the bottom of security.json: { "name":"open_select", "path":"/select/*", "role":null, "index":9}, { "name":"catch-all-nocollection", "collection":null, "path":"/*",

Re: Extract a list of the most recent field values?

2021-02-05 Thread Alexandre Rafalovitch
Rewriting: *) https://lucene.apache.org/solr/guide/8_8/json-request-api.html#json-parameter-merging , there is a way to represent most (all?) of the structure with json.x parameter. *) Also, you can create custom Request Handlers in solrconfig.xml with a lot of those parameters either as defaults

Re: Clarification on term facet method dvhash

2021-02-05 Thread Michael Gibney
Happy to help! If I'm correctly reading the block of code linked to above, "dvhash" is silently ignored for multi-valued fields. So probably not much performance difference there ;-) On Fri, Feb 5, 2021 at 2:12 PM ufuk yılmaz wrote: > This is a huge help Mr. Gibney thank you! > > One thing I

Sv: Extract a list of the most recent field values?

2021-02-05 Thread Hullegård , Jimi
Ah, I never thought about grouping on date ranges, and nesting the faceting like that. Interesting! I managed to do a quick test query that seems to give me what I want: { "query": "*:*", "filter": "+category:* +modified:[NOW/DAY-60DAYS TO *]", "limit": 0, "facet": {

RE: Clarification on term facet method dvhash

2021-02-05 Thread ufuk yılmaz
This is a huge help Mr. Gibney thank you! One thing I can add is I tried dvhash with a string multi-valued field, it worked and didn’t throw any error but I don’t know if it got silently ignored or just worked. Sent from Mail for Windows 10 From: Michael Gibney Sent: 05 February 2021 20:52

Authentication for all but selects

2021-02-05 Thread Robert Douglas
Hello all, We are working on some migrations and we want to be incorporating authentication more uniformly across all our installations of Solr, but we are getting stuck on allowing Select statements through without authentication while having authentication on with RBAP for everything else.

Re: Clarification on term facet method dvhash

2021-02-05 Thread Michael Gibney
Correction!: wrt "dvhash" and numeric types, it looks like I had it exactly backwards! single-valued numeric types _do_ use (even default to) "dvhash" ... sorry about that! I stand by the rest of the previous message though, which applies at a minimum to string-like fields. On Fri, Feb 5, 2021 at

Re: Clarification on term facet method dvhash

2021-02-05 Thread Michael Gibney
> Performance and resource is still affected by 30M unique values of T right? Yes. The main performance issue would be the per-request allocation of a 30M-element `long[]` for "dv" or "uif" methods (which are by far the most common methods in practice). With low enough request volume and large

Clarification on term facet method dvhash

2021-02-05 Thread ufuk yılmaz
Hello, I’m using Solr 8.4. Very excited about performance improvements in 8.8: http://joelsolr.blogspot.com/2021/01/optimizations-coming-to-solr.html As I understand the main determinator of performance and RAM usage of a terms facet is cardinality of the field in whole collection, but not the

Re: Extract a list of the most recent field values?

2021-02-05 Thread Alexandre Rafalovitch
This feels like basic faceting on category, but you are trying to make a latest record, rather than count as a sorting/grouping principle. How about using JSON Facets? https://lucene.apache.org/solr/guide/8_8/json-facet-api.html I would do the first level as range facet and do your dates at

Re: 404 Errors on update/extract

2021-02-05 Thread Alexandre Rafalovitch
Hi Leon, Feel free to create JIRA issue https://issues.apache.org/jira/secure/Dashboard.jspa and then do Github pull request to fix the example name. The documentation is in asciidoc format at: https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide/src with names matching those on

Re: 404 Errors on update/extract

2021-02-05 Thread nq
Hi Alex, Thanks a lot for your help! I have tested the same using the 'techproducts' example as proposed, and it worked fine. You are right, the documentation seems to be outdated in this aspect. I have just reviewed the solrconfig.xml of the 'schemaless' example and found all the Solr

Sv: Extract a list of the most recent field values?

2021-02-05 Thread Hullegård , Jimi
Hi Emir, But that page says: "The field that is being collapsed on. The field must be a single valued String, Int or Float" And the field in question is a multi value field. And when I try using fq={!collapse field=myField} I get: "org.apache.solr.search.SyntaxError: Collapsing not supported

Re: 404 Errors on update/extract

2021-02-05 Thread Alexandre Rafalovitch
I think the extract handler is not defined in schemaless. This may be a change from before and the documentation is out of sync. Can you try 'techproducts' example instead of schemaless: bin/solr stop (if you are still running it) bin/solr start -e techproducts Then the import command. The Tika

Re: Extract a list of the most recent field values?

2021-02-05 Thread Emir Arnautović
Hi Jimi, It seems to me that you could get the results using collapsing query parse: https://lucene.apache.org/solr/guide/6_6/collapse-and-expand-results.html HTH, Emir -- Monitoring - Log Management - Alerting -

Extract a list of the most recent field values?

2021-02-05 Thread Hullegård , Jimi
Hi, Say we have a bunch of documents in Solr, and each document has a multi value field "category". Now I would like to get the N most recently used categories, ordered so that the most recently used category comes first and then in falling order. My simplistic solution to this would be: 1.

404 Errors on update/extract

2021-02-05 Thread nq
Hi, I am new to Solr and tried to follow the guide to upload PDF data using Tika, on Solr 8.7.0 (running on Debian 10): https://lucene.apache.org/solr/guide/8_7/uploading-data-with-solr-cell-using-apache-tika.html but I get an HTTP 404 error when trying to import the file. In the solr

Empty shard1 - -:{"shard1":[]} cannot add new replicas

2021-02-05 Thread Dirk Wintergruen
Dear all, I cannot add or remove any replicas of one collection. Diagnostics in the log file shows empty shards "gmpg-fulltext3":{"shard1":[]}, see below. What can I do? ng.error.diagnostics.3897285248187441 { "sortedNodes":[{

Re: Recovering forever after upgrade to 8.8.0: Timeout waiting for collection state

2021-02-05 Thread Henrik B A
On Fri, Feb 5, 2021 at 10:58 AM Henrik Brautaset Aronsen wrote: > After upgrading our Solr Cloud collections from 8.7.0 to 8.8.0 I struggle > to get a consistent state. We have 8 servers hosting 3 collections, with > shards/replicas spread over alle the servers. > > All replicas on solr3577 is

Recovering forever after upgrade to 8.8.0: Timeout waiting for collection state

2021-02-05 Thread Henrik B A
After upgrading our Solr Cloud collections from 8.7.0 to 8.8.0 I struggle to get a consistent state. We have 8 servers hosting 3 collections, with shards/replicas spread over all the servers. All replicas on solr3577 is in "Recovering" state, and is repeating every five minutes:

Re: Excessive logging 8.8.0

2021-02-05 Thread Markus Jelsma
Thanks! Op do 4 feb. 2021 om 20:04 schreef Chris Hostetter : > > FWIW: that log message was added to branch_8x by 3c02c9197376 as part of > SOLR-15052 ... it's based on master commit 8505d4d416fd -- but that does > not add that same logging message ... so it definitely smells like a > mistake to

Recovering forever after upgrade to 8.8.0: Timeout waiting for collection state

2021-02-05 Thread Henrik Brautaset Aronsen
Hi! After upgrading our Solr Cloud collections from 8.7.0 to 8.8.0 I struggle to get a consistent state. We have 8 servers hosting 3 collections, with shards/replicas spread over alle the servers. All replicas on solr3577 is in "Recovering" state, and is repeating every five minutes:

Unable to connect to an 8.8.0 Solr Cloud database via API

2021-02-05 Thread Flowerday, Matthew J
Hi There I have been checking out the latest (8.8.0) SolrCloud database (using Zookeeper 3.6.2) against our application which talks to Solr via the Solr API (I am not too sure of the details as I am not a java developer unfortunately!). The software has Solr 8.7.0/ZooKeeper 3.6.2 libraries

Re: Urgent- General Question about document Indexing frequency in solr

2021-02-04 Thread Scott Stults
Manisha, The most general recommendation around commits is to not explicitly commit after every update. There are settings that will let Solr automatically commit after some threshold is met, and by delegating commits to that mechanism you can generally ingest faster. See this blog post that

Re: Excessive logging 8.8.0

2021-02-04 Thread Chris Hostetter
FWIW: that log message was added to branch_8x by 3c02c9197376 as part of SOLR-15052 ... it's based on master commit 8505d4d416fd -- but that does not add that same logging message ... so it definitely smells like a mistake to me that 8x would add this INFO level log message that master

Excessive logging 8.8.0

2021-02-04 Thread Markus Jelsma
Hello all, We upgraded some nodes to 8.8.0 and notice there is excessive logging on INFO when some traffic/indexing is going on: 2021-02-04 11:42:48.535 INFO (qtp261748192-268) [c:data s:shard2 r:core_node4 x:data_shard2_replica_t2] o.a.s.c.c.ZkStateReader already watching , added to s

Re: SSL using CloudSolrClient

2021-02-03 Thread ChienHuaWang
Thanks for the information. Could you advise whether CloudSolrClient is compatible with non-TLS? even client is not configure, it can still connect to Solr (TLS enabled)? -- Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: SolrCloud keeps crashing

2021-02-03 Thread TK Solr
Oops, I should have referenced this document rather: https://www.tenable.com/cve/CVE-2019-17558 On 2/3/21 2:42 PM, TK Solr wrote: Victor & Satish, Is your Solr accessible from the Internet by anyone? If so, your site is being attacked by a bot

Urgent- General Question about document Indexing frequency in solr

2021-02-03 Thread Manisha Rahatadkar
Hi All Looking for some help on document indexing frequency. I am using apache solr 7.7 and SolrNet library to commit documents to Solr. Summary for this function is: // Summary: // Commits posted documents, blocking until index changes are flushed to disk and // blocking until a new

Re: SolrCloud keeps crashing

2021-02-03 Thread TK Solr
Victor & Satish, Is your Solr accessible from the Internet by anyone? If so, your site is being attacked by a bot using this security hole: https://www.tenable.com/blog/cve-2019-17558-apache-solr-vulnerable-to-remote-code-execution-zero-day-vulnerability If that is the case, try blocking the

<    1   2   3   4   5   6   7   8   9   10   >