Keep Solr Indexing live

Hello All, I am using DIH to import data from SQL to Solr using Url "/dataimport?command=full-import&clean=true". My problem is, When SQL query return zero record then Solr also return zero records. But as per my project requirement, Solr indexing should be clean only when SQL query return reco

Re: recurring Solr warning messages

Hi,Can someone respond on this please?Or, can you direct me to the right contact who may know about these issues.Regards,RiteshFrom: "Ritesh"Sent: Tue, 19 Dec 2017 18:06:13To: Subject: Re: recurring Solr warning messagesHello,Can you help on

DocValues for multivalued strings and boolean fields

Hi, One of our Solr users is trying to set docValues="true" for multivalued string fields and boolean-type fields. I am not sure what the performance impact of that would be. Can docValues negatively affect performance in any way? We are using Solr 6.5.1 and also experimenting with 7.1.0 Thanks

Re: Solr 7.1 Solrcloud dynamic/automatic replicas

If you specify the node parameter for ADDREPLICA I don't think so, but as you know you have to understand the topology via CLUSTERSTATUS or some such. If you don't specify the "node" parameter, I think if you take a look at the "Rule-based Replica Placement" here: https://lucene.apache.org/solr/gu

Re: Solr 7.1 Solrcloud dynamic/automatic replicas

Thanks again Erick. It looks like I've got this working. One final question I think: Is there a way to prevent ADDREPLICA from adding another core if a core for the collection already exists on the node? I've noticed that if I call ADDREPLICA twice for the same IP:PORT_solr, I get multiple cores.

Re: Trouble with mm and SynonymQuery and KeywordRepeatFilter

Hi Markus, My suggestion: rewrite your synonyms to include the triggering word in the expanded synonyms list. That way you won’t need KeywordRepeat/RemoveDuplicates filters, and mm=100% will work as you expect. I don’t think this situation is a bug, since mm applies to the built query, not to

RE: Trouble with mm and SynonymQuery and KeywordRepeatFilter

Hello, Yes of course, index time synonyms lessens the query time complexity and will solve the mm problem. It also screws IDF and the flexibility of adding synonyms on demand. The first we do not want, the second is impossible for us (very large main search index). We are looking for a solutio

Re: Filtering Solr pivot facet values

Thanks for your reply, Shawn. I think multi-select faceting does the opposite of what I want. I want the facet to include the filters. Example: The following 8 documents are the only ones in my Solr core: [ {"id": "1", "hierarchy": ["1", "16", "169"], "interests": ["soccer", "futbol"]}, {"i

Re: Trouble with mm and SynonymQuery and KeywordRepeatFilter

On 12/19/2017 4:38 AM, Markus Jelsma wrote: > I have an interesting issue with mm and SynonymQuery and KeywordRepeatFilter. > We do query time synonym expansion and use KeywordRepeat for not only finding > stemmed tokens. Our synonyms are already preprocessed and contain only > stemmed tokens. S

Re: Filtering Solr pivot facet values

On 12/20/2017 1:31 PM, Arun Rangarajan wrote: > Sorry to bother you again on this. Is there no way in Solr to filter pivot > facets? > [Or did I attract the wrath of the group by posting the question first on > StackOverflow? :-)] StackOverflow and this list are pretty much unaware of each other u

Re: Build suggester in different directory (not /tmp).

Erick, oh, yes, I think I was misunderstanding buildOnCommit. I presumed it would run following the completion of my DIH. The behavior you described would be very problematic! Thank you for taking the time to point that out! Best, Matt On Wed, Dec 20, 2017 at 3:47 PM, Erick Erickson wrote: >

Re: Build suggester in different directory (not /tmp).

Matthew: I think you'll be awfully unhappy with buildOnCommit. Say you're bulk-indexing and committing every 15 seconds buildOnStartup is problematical as well since it'd rebuild everytime you bounced Solr even if the index hadn't changed. Personally I'd alter my indexing process to fire a b

Re: Build suggester in different directory (not /tmp).

Thanks Erick, I'll head your warning. Ultimately, the index will be rather static so I do not fear much from buildingOnComit. But I think building on startup would likely be set to false regardless. Shawn, Thank you as well. That is very informative regarding java.io.tmpdir. I am starting this a

RE: Trouble with mm and SynonymQuery and KeywordRepeatFilter

Hello - any ideas to share on this topic? Many thanks, Markus -Original message- > From:Markus Jelsma > Sent: Tuesday 19th December 2017 12:38 > To: Solr-user > Subject: Trouble with mm and SynonymQuery and KeywordRepeatFilter > > Hello, > > I have an interesting issue with mm and

Re: Filtering Solr pivot facet values

Hello Solr Gurus, Sorry to bother you again on this. Is there no way in Solr to filter pivot facets? [Or did I attract the wrath of the group by posting the question first on StackOverflow? :-)] Thanks once again. On Mon, Dec 18, 2017 at 10:59 AM, Arun Rangarajan wrote: > Solr version: 6.6.0 >

Re: No space left on device - When I execute suggester component.

It's kind of scary how often serendipity plays it's part. See the thread titled: "Build suggester in different directory (not /tmp)." Which basically says that the suggester is being built in /tmp which may be limited. And yes, that's where it gets built by default, although the thread I mention

Re: Build suggester in different directory (not /tmp).

On 12/20/2017 10:05 AM, Matthew Roth wrote: > I am building a few suggester's and I am receiving the error that I have no > space left on device. > At first this threw me. df showed I had over 100 G free. the /data dir the > suggester is being constructed from is only 4G. On a subsequent run I >

Re: No space left on device - When I execute suggester component.

Hi Shawn/Erick/Matt, I agree with you. When I execute the command df -h I am getting the complete list of nfs mount info and Size and available space. I just shared one liner out of it.. One more thing I observed whenever I run suggest.build. *http://rn.com:8989/solr/LW_Data/

Re: Build suggester in different directory (not /tmp).

bq: this means I will need to set buildOnCommit and buildOnStartup to false. Be _very_ careful with these settings. Building your suggester can read the stored field(s) from _every_ document in your index to build which can take a very long time (perhaps hours). You'd pay that penalty every time y

Re: Are the entries in managed-schema order dependent?

Actually, I think Solr does rearrange everything to its liking (alphabetical?) when it rewrites managed-schema. So, if the definitions are added via API, the order will be deterministic. That's what I believe though, I can't remember testing it exhaustively with physically rearranged types. Regar

Re: Are the entries in managed-schema order dependent?

Thanks! On 12/20/2017 11:37 AM, Erick Erickson wrote: The schema is not order dependent, I freely mix-n-match the fieldType, copyField and field definitions for instance. On Wed, Dec 20, 2017 at 8:29 AM, Michael Joyner wrote: Hey all, I'm wanting to update our managed-schemas to include t

Re: Build suggester in different directory (not /tmp).

I have an incomplete solution. I was trying to build three suggester's at once. If I added the ?suggest.dictionary= parameter and built one at a time it worked out fine. However, this means I will need to set buildOnCommit and buildOnStartup to false. This is less than ideal. Building in a differen

Re: No space left on device - When I execute suggester component.

Oh, this seems relevant to my recent post to the list. My problem is that the suggester's are first being built in /tmp and moved to /var. tmp has a total of 2g's free whereas /var has near 100G. Perhaps you are running into the same problem I am in this regard? How does your /tmp dir look when bu

Build suggester in different directory (not /tmp).

Hi List, I am building a few suggester's and I am receiving the error that I have no space left on device. No space left on device java.io.IOException: No space left on device at sun.nio.ch.FileDispatcherImpl.write0(Native Method) at ... At first this threw me. df showed I had over 100 G fr

Re: Are the entries in managed-schema order dependent?

The schema is not order dependent, I freely mix-n-match the fieldType, copyField and field definitions for instance. On Wed, Dec 20, 2017 at 8:29 AM, Michael Joyner wrote: > Hey all, > > I'm wanting to update our managed-schemas to include the latest options > available in the 6.6.2 branch. (po

Are the entries in managed-schema order dependent?

Hey all, I'm wanting to update our managed-schemas to include the latest options available in the 6.6.2 branch. (point types for one) I would like to be able to sort them and diff them (production vs dist supplied) to create a simple patch that can be reviewed, edited if necessary, and then

Re: Solr 7.1 Solrcloud dynamic/automatic replicas

The internal method is ZkController.generateNodeName(), although it's fairly simple, there are bunches of samples in ZkControllerTest But yeah, it requires that you know your hostname and port, and the context is "solr". On Tue, Dec 19, 2017 at 8:04 PM, Greg Roodt wrote: > Ok, thanks. I'

RE: DocTransformer: Float cannot be cast to org.apache.lucene.document.StoredField

Ah of course, it worked before i enabled docValues for that field. Got it working again! Thanks! -Original message- > From:Emir Arnautović > Sent: Wednesday 20th December 2017 16:02 > To: solr-user@lucene.apache.org > Subject: Re: DocTransformer: Float cannot be cast to > org.apache.lu

Re: DocTransformer: Float cannot be cast to org.apache.lucene.document.StoredField

I did not check the code, but that is what error is suggesting. Can you check if field definition is the same locally and on other Solr. Since Solr can use doc values as stored, I would guess that it is not always StoredField that is returned. Regards, Emir -- Monitoring - Log Management - Aler

RE: DocTransformer: Float cannot be cast to org.apache.lucene.document.StoredField

Are you telling my that SolrDocument.get(key) can return both StoredField or the actual class of the value? The code ran fine locally. There i got a StoredField and had to use numericValue() to get my float. Thanks, Markus -Original message- > From:Emir Arnautović > Sent: Wednesda

Authentication Plugin

Hi all, We've got a solrcloud cluster set up on 6.3.0 with the BasicAuthentication plugin enabled. All of the hosts are time synchronized using ntp and are on the same network switch. We're periodically experiencing issues where follower replicas are put into down states by the leader in the case

Re: DocTransformer: Float cannot be cast to org.apache.lucene.document.StoredField

Hi Markus, You are trying to cast to stored field without checking if that is actually StoredField. What you can do is check first if StoredField or Float or… and cast to appropriate value. HTH, Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Su

DocTransformer: Float cannot be cast to org.apache.lucene.document.StoredField

Hello, Recently i had to make yet another DocTransformer. It ran fine on my local machine, this is what i get in production, on freshly reindexed data. 2017-12-20 12:12:58.987 ERROR (qtp329611835-17) [c:documents s:shard2 r:core_node1 x:documents_shard2_replica2] o.a.s.s.HttpSolrCall null:java

Re: Keep indexed records

Hi Shashi, IMO it would be best if you put that logic on your controller where you start import. If you are doing that through admin console, the only solution I am aware of is to write your custom component. HTH, Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elastics

Keep indexed records

Hello All, I want to keep indexed records live in solr, during data import when sql query not returning any record. We also need to clean= true, because when SQL query return records then solr should be reindexed. Only avoid reindexing, when SQL query not return any rows. Please suggests. Regard

Re: No space left on device - When I execute suggester component.

On 12/20/2017 12:21 AM, Fiz Newyorker wrote: I tried df -h , during suggest.build command. Size. Used Avail Use% Mounted on 63G 17G 44G 28% /ngs/app That cannot be the entire output of that command. Here's what I get when I do it: root@smeagol:~# df -h Filesystem Size