Setting routerField/shardKey on specific collection?

2013-12-04 Thread Daniel Bryant
Hi, I'm using Solr 4.6 and trying to specify a router.field (shard key) on a specific collection so that all documents with the same value in the specified field end up in the same collection. However, I can't find an example of how to do this via the solr.xml? I see in this ticket

RE: Setting routerField/shardKey on specific collection?

2013-12-04 Thread Tim Potter
, Timothy Potter Sr. Software Engineer, LucidWorks www.lucidworks.com From: Daniel Bryant daniel.bry...@tai-dev.co.uk Sent: Wednesday, December 04, 2013 9:40 AM To: solr-user@lucene.apache.org Subject: Setting routerField/shardKey on specific collection

Re: Setting routerField/shardKey on specific collection?

2013-12-04 Thread Daniel Bryant
Subject: Setting routerField/shardKey on specific collection? Hi, I'm using Solr 4.6 and trying to specify a router.field (shard key) on a specific collection so that all documents with the same value in the specified field end up in the same collection. However, I can't find an example of how to do

RE: shardkey

2013-06-28 Thread Joshi, Shital
] Sent: Thursday, June 27, 2013 5:35 PM To: solr-user@lucene.apache.org Subject: Re: shardkey You might be seeing https://issues.apache.org/jira/browse/SOLR-4923 ? The commit true part of the request that add documents? If so, it might be SOLR-4923 and you should try the commit in a request after

Re: shardkey

2013-06-28 Thread Mark Miller
:8983/solr/core3/update?commit=true; Is that correct? It doesn't matter whether HOST is leader or replica. -Original Message- From: Mark Miller [mailto:markrmil...@gmail.com] Sent: Thursday, June 27, 2013 5:35 PM To: solr-user@lucene.apache.org Subject: Re: shardkey You

RE: shardkey

2013-06-28 Thread Joshi, Shital
Thanks! -Original Message- From: Mark Miller [mailto:markrmil...@gmail.com] Sent: Friday, June 28, 2013 5:06 PM To: solr-user@lucene.apache.org Subject: Re: shardkey Yeah, that is what I would try until 4.4 comes out - and it should not matter replica or leader. - Mark On Jun 28

RE: shardkey

2013-06-27 Thread Joshi, Shital
Hi, We finally decided on using custom sharding (implicit document routing) for our project. We will have ~3 mil documents per shardkey. We're maintaining shardkey - shardid mapping in a database table. While adding documents we always specify _shard_ parameter in update URL but while

Re: shardkey

2013-06-27 Thread Mark Miller
wrote: Hi, We finally decided on using custom sharding (implicit document routing) for our project. We will have ~3 mil documents per shardkey. We're maintaining shardkey - shardid mapping in a database table. While adding documents we always specify _shard_ parameter in update URL

RE: shardkey

2013-06-25 Thread Joshi, Shital
a String, without timestamp) and per business date we're expecting about 3 million documents. -Original Message- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: Friday, June 21, 2013 8:50 PM To: solr-user@lucene.apache.org Subject: Re: shardkey On Fri, Jun

RE: shardkey

2013-06-21 Thread Joshi, Shital
Thanks for answering my questions on shardkey. We experimented with implicit router and it works like you said. We're experimenting with composite id router. This document http://searchhub.org/2013/06/13/solr-cloud-document-routing/ says: A shard key can be pre-pended to the unique document id

Re: shardkey

2013-06-21 Thread Yonik Seeley
On Fri, Jun 21, 2013 at 6:08 PM, Joshi, Shital shital.jo...@gs.com wrote: But now Solr stores composite id in the document id Correct, it's the document id itself that contains everything needed for tje compositeId router to determine the hash. It would only use it to calculate hash key but

RE: shardkey

2013-06-17 Thread Joshi, Shital
Thanks for the links. It was very useful. Is there a way to use implicit router WITH numShards parameter? We have 5 shards and business day (Monday-Friday) is our shardkey. We want to be able to say Monday - shard1, Tuesday - shard2. -Original Message- From: Joel Bernstein

Re: shardkey

2013-06-17 Thread Shalin Shekhar Mangar
No, there is no way to do that right now. I think you'd be better off using custom sharding because you can't really control that two shardKeys must go to two different shards. We can only guarantee that docs with the same shardKey will goto the same shard. On Mon, Jun 17, 2013 at 9:47 PM, Joshi

Re: shardkey

2013-06-13 Thread Joel Bernstein
/display/solr/Shards+and+Indexing+Data+in+SolrCloud -Original Message- From: Rishi Easwaran [mailto:rishi.easwa...@aol.com] Sent: Wednesday, June 12, 2013 1:40 PM To: solr-user@lucene.apache.org Subject: Re: shardkey From my understanding. In SOLR cloud the CompositeIdDocRouter uses

shardkey

2013-06-12 Thread Joshi, Shital
Hi, We are using Solr 4.3.0 SolrCloud (5 shards, 10 replicas). I have couple questions on shard key. 1. Looking at the admin GUI, how do I know which field is being used for shard key. 2. What is the default shard key used? 3. How do I override the default shard key?

Re: shardkey

2013-06-12 Thread bbarani
I suppose you can implement custom hashing by using _shard_ field. I am not sure on this, but I have come across this approach sometime back.. At query time, you can specify shard.keys parameter... -- View this message in context: http://lucene.472066.n3.nabble.com/shardkey-tp4069940p4069990

Re: shardkey

2013-06-12 Thread Rishi Easwaran
Subject: shardkey Hi, We are using Solr 4.3.0 SolrCloud (5 shards, 10 replicas). I have couple questions on shard key. 1. Looking at the admin GUI, how do I know which field is being used for shard key. 2. What is the default shard key used? 3. How do I override

RE: shardkey

2013-06-12 Thread James Thomas
Subject: Re: shardkey From my understanding. In SOLR cloud the CompositeIdDocRouter uses HashbasedDocRouter. CompositeId router is default if your numShards1 on collection creation. CompositeId router generates an hash using the uniqueKey defined in your schema.xml to route your documents