Re: Solr Cloud with 5 servers cluster failed due to Leader out of memory

On 8/4/2016 8:14 PM, Tim Chen wrote: > Couple of thoughts: 1, If Leader goes down, it should just go down, > like dead down, so other servers can do the election and choose the > new leader. This at least avoids bringing down the whole cluster. Am I > right? Supplementing what Erick told you: Wh

Re: Solr Cloud with 5 servers cluster failed due to Leader out of memory

The fact that all the shards have the same leader is somewhat of a red herring. Until you get hundreds of shards (perhaps across a _lot_ of collections), the additional load on the leaders is hard to measure. If you really see this as a problem, consider the BALANCESHARDUNIQUE and REBALANCELEADERS

Re: Problems using fieldType text_general in copyField

Just as a note, TYPO3 uses a lot of include files though I do not remember which specific mechanism they rely on. Regards, Alex On 5 Aug 2016 10:51 AM, "John Bickerstaff" wrote: > Many thanks for your time! Yes, it does make sense. > > I'll give your recommendation a shot tomorrow and upda

Solr Cloud with 5 servers cluster failed due to Leader out of memory

Hi Guys, Me again. :) We have 5 Solr servers: 01 -04 running Solr version 4.10 and ZooKeeper service 05 running ZooKeeper only. JVM Max Memory set to 10G. We have around 20 collections, and for each collection, there are 4 shards, for each shard, there are 4 replica sitting across on 4 Solr se

Re: Problems using fieldType text_general in copyField

Many thanks for your time! Yes, it does make sense. I'll give your recommendation a shot tomorrow and update the thread. On Aug 4, 2016 6:22 PM, "Chris Hostetter" wrote: TL;DR: use entity includes *WITH OUT TOP LEVEL WRAPPER ELEMENTS* like in this example... https://github.com/apache/lucene-

Re: Problems using fieldType text_general in copyField

TL;DR: use entity includes *WITH OUT TOP LEVEL WRAPPER ELEMENTS* like in this example... https://github.com/apache/lucene-solr/blob/master/solr/core/src/test-files/solr/collection1/conf/schema-snippet-types.incl https://github.com/apache/lucene-solr/blob/master/solr/core/src/test-files/solr/coll

Re: Problems using fieldType text_general in copyField

Thanks and sorry for the misunderstanding. The file I pasted last time is the file I was trying to include into the main schema.xml. It was when that file was getting processed that I got the error ['content' is not a glob and doesn't match any explicit field or dynamicField. ] I should note th

Re: Problems using fieldType text_general in copyField

: The schema is a copy of the techproducts sample. : : Entire include here - and I take your point about the possibility of : malformation - thanks. : : I assumed (perhaps wrongly) that I could duplicate the : arrangement from the schema.xml file. I really can't make heads or tails of what y

Re: Problems using fieldType text_general in copyField

I get the same error with the Entity Includes - with or without the tag... I'm probably just going to make a section in schema.xml rather than worry about this. Includes are "nice to have" but not critical. On Thu, Aug 4, 2016 at 4:25 PM, John Bickerstaff wrote: > Found the Entity Includes -

Re: Out of sync deletions causing differing IDF

Thx for these both, we'll give them both a try, see what difference they make. Upayavira On Thu, 4 Aug 2016, at 12:27 PM, Erick Erickson wrote: > Upayavira: > > bq: I would have expected that, because the data is being indexed > concurrently across replicas, that the pattern of delete/merge woul

Re: Problems using fieldType text_general in copyField

Found the Entity Includes - thanks. On Thu, Aug 4, 2016 at 4:22 PM, John Bickerstaff wrote: > Thanks! > > The schema is a copy of the techproducts sample. > > Entire include here - and I take your point about the possibility of > malformation - thanks. > > I assumed (perhaps wrongly) that I coul

Re: Problems using fieldType text_general in copyField

Thanks! The schema is a copy of the techproducts sample. Entire include here - and I take your point about the possibility of malformation - thanks. I assumed (perhaps wrongly) that I could duplicate the arrangement from the schema.xml file. I'm unfamiliar with xml entity includes, but I'll

Re: Can a MergeStrategy filter returned docs?

Collapse will have dups unless you use the _route_ parameter to co-locate documents with the same group, onto the same shard. In you're scenario, co-locating docs sounds like it won't work because you may have different grouping criteria. The doc counts would be inflated unless you sent all the d

Re: Can a MergeStrategy filter returned docs?

Perhaps my assumptions about merge are wrong. When I run a search with the collapsing filter (q=*:*&fq={!collapse field=VENDOR_NAME}...) I get "dupes" if the same VENDOR_NAME is on shard1 and shard2. Here's the response: "response": { "numFound": 24158, "start": 0, "docs": [ {

Re: Problems using fieldType text_general in copyField

you mentioned that the problem only happens when you use xinclude, but you havne't shown us hte details of your xinclude -- what exactly does your schema.xml look like (with the xinclude call) and what exactly does the file being included look like (entire contents) (I suspect the problem you

Re: Problems using fieldType text_general in copyField

I would call this a bug... I'm going out on a limb and say that if you define a field in the included XML file, you will get this error. As long as the field is defined first in schema.xml, you can "copyFIeld" it or whatever in the include file, but apparently fields MUST be created in the schema

Re: Can a MergeStrategy filter returned docs?

Can you describe more about what you're trying to do in the merge? Why does it seem it's too late to drop documents in the merge? If you can provide a very simple example with some sample records and a sample output, that would be helpful. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Aug

Can a MergeStrategy filter returned docs?

I've been struggling just to get my search plugin working for sharded collections, but I haven't ascertained if my end goal is even achievable. I have a plugin that groups documents that are considered duplicates (based on multiple fields - like the CollapsingQParserPlugin). When responses come bac

Problems using fieldType text_general in copyField

Summary: Using xinclude to include an xml file into schema.xml The following line generates an error: about a field being "not a glob and not matching an explicit field" even though I declare the field in the line just above. This seems to happen only for for fieldType text_general? ===

Re: Difference in boolean query parsing. Solr-5.4.0 VS Solr.6.1.0

It’s fairly likely these differences are as a result of SOLR-2649[1] (released with 5.5) and SOLR-8812[2] (released with 6.1). If you haven’t seen it, I recommend you read Hoss'ss blog “Why Not AND, OR, And NOT?” . If you can, add

unique( )- How to override default of 100

** PROTECTED 関係者外秘 Hi, I was looking at Solr’s countdistinct feature with unique and hll functions. I am interested in getting accurate cardinality in cloud setup. As per the link, unique() function provides exact counts if the number of values per node does not exceed 100 by default. How do I o

Re: Out of sync deletions causing differing IDF

Upayavira: bq: I would have expected that, because the data is being indexed concurrently across replicas, that the pattern of delete/merge would be similar across replicas. Except for the pesky timing issue. The timers start for autocommit when a request is received. So the time the autocommit t

Re: Replication with managed resources?

Raised as https://issues.apache.org/jira/browse/SOLR-9382 -- View this message in context: http://lucene.472066.n3.nabble.com/Replication-with-managed-resources-tp4289880p4290386.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: QParsePlugin not working on sharded collection

So my implementation with a DocTransformer is causing an exception (with a sharded collection): ERROR - 2016-08-04 09:41:44.247; [ShardTest1 shard1_0 core_node3 ShardTest1_shard1_0_replica1] org.apache.solr.common.SolrException; null:org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrExcep

Re: Why Nested document 'child' entity query (iterative count)repeatedly executing?

It seems like debug reporting issue. It deserves a minor Jira. On Thu, Jul 14, 2016 at 2:38 PM, Rajendran, Prabaharan wrote: > Hi All, > > I am trying to index nested document. While start full data-import (from > UI) following options selected Verbose, Commit, Debug & Debug-Mode. > > Raw Debug-

Re: Sort Facet Values by "Interestingness"?

Ok let's explore how to use scoreNodes() with the facet() expression. scoreNodes is a graph expression, so it expects certain fields to be on each Tuple. The fields are: 1) node: The node field is the node id gathered by the gatherNodes() function. 2) collection: This is the collection that the n

Re: QParsePlugin not working on sharded collection

Thanks Erick, you answered my question by pointing out the aggregator. I didn't realize a merge strategy was _required_ to return stats info when there are multiple shards. I'm having trouble with my actual plugin so I've scaled back to the simplest possible example. I'm adding to it little by litt

RE: Out of sync deletions causing differing IDF

Hello - your similarity should rely on numDoc instead, it solves the problem. I believe it is already fixed in trunk, but i am not sure. Markus -Original message- > From:Upayavira > Sent: Thursday 4th August 2016 13:59 > To: solr-user@lucene.apache.org > Subject: Out of sync deletions c

Out of sync deletions causing differing IDF

We have a system that has a reasonable number of changes going on on a daily basis (maybe 60m docs, and around 1m updates per day). Using Solr Cloud, the data is split into 10 shards and those shards are replicated. What we are finding is that the number of deletions is causing differing maxDocs a

ComplexPhraseQuery and range query

Hi, We are running solr 6.1.0 We are using ComplexPhraseQuery As default parser. It seems that range query are not working with this parser. Here is the query: nbaff:({0 TO * }) Here is the field definition in the schema: positionIncrementGap="0" precisionStep="0"/> Running query in debug m

Re: problems with bulk indexing with concurrent DIH

After updating to version 5.5.3 it looks good now. I think LUCENE-6161 has fixed my problem. Nevertheless, after updating my development system and recompyling my plugins I will have a look at DIH about the "update" and also your advise about the uniqueKey. Best regards Bernd Am 02.08.2016 um 16: