Thanks Mark. We use commit=true as part of the request to add documents. Something like this:
echo "$data"| curl --proxy "" --silent "http://HOST:9983/solr/collection1/update/csv?commit=true&separator=|&fieldnames=$fieldnames&_shard_=shard1" --data-binary @- -H 'Content-type:text/plain; charset=utf-8' You're suggesting that after this update, we should always execute, curl --proxy "" --silent "http://HOST:8983/solr/core3/update?commit=true"; Is that correct? It doesn't matter whether HOST is leader or replica. -----Original Message----- From: Mark Miller [mailto:markrmil...@gmail.com] Sent: Thursday, June 27, 2013 5:35 PM To: solr-user@lucene.apache.org Subject: Re: shardkey You might be seeing https://issues.apache.org/jira/browse/SOLR-4923 ? The commit true part of the request that add documents? If so, it might be SOLR-4923 and you should try the commit in a request after adding the docs. - Mark On Jun 27, 2013, at 4:42 PM, "Joshi, Shital" <shital.jo...@gs.com> wrote: > Hi, > > We finally decided on using custom sharding (implicit document routing) for > our project. We will have ~3 mil documents per shardkey. We're maintaining > shardkey -> shardid mapping in a database table. While adding documents we > always specify _shard_ parameter in update URL but while querying, we don't > specify shards parameter. We want to search across shards. > > While experimenting we found that right after hard committing (commit=true in > update URL), at times the query didn't return documents across shards (40% of > the time) But many times (60% of the time) it returned documents across > shards. When queried after few hours, the query always returned documents > across shards. Is that expected behavior? Is there a parameter to enforce > querying across all shards? This is very important point for us to move > further with SolrCloud. > > We're experimenting with adding a new shard and start directing all new > documents to this new shard. Hopefully that should work. > > Many Thanks! > > -----Original Message----- > From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley > Sent: Friday, June 21, 2013 8:50 PM > To: solr-user@lucene.apache.org > Subject: Re: shardkey > > On Fri, Jun 21, 2013 at 6:08 PM, Joshi, Shital <shital.jo...@gs.com> wrote: >> But now Solr stores composite id in the document id > > Correct, it's the document id itself that contains everything needed > for tje compositeId router to determine the hash. > >> It would only use it to calculate hash key but while storing > > compositeId routing is when it makes sense to make the routing part of > the unique id so that an id is all the information needed to find the > document in the cluster. For example customer_id!document_name. From > your example of 20130611!test_14 it looks like you're doing time based > sharding, and one would normally not use the compositeId router for > that. > > -Yonik > http://lucidworks.com
