[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536676#comment-13536676 ] Commit Tag Bot commented on SOLR-4221: -- [trunk commit] Yonik Seeley http://svn.apache.org/viewvc?view=revision&revision=1424263 SOLR-4221: pick correct router for collection props > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536681#comment-13536681 ] Commit Tag Bot commented on SOLR-4221: -- [branch_4x commit] Yonik Seeley http://svn.apache.org/viewvc?view=revision&revision=1424265 SOLR-4221: pick correct router for collection props > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13679494#comment-13679494 ] Yonik Seeley commented on SOLR-4221: Off the top of my head, some things left to do in this general area of "custom sharding": - ability to create a new shard via collections API (and other lifecycle operations... delete, rename?) - probably want to have the option of either leaving it empty, or creating new replicas - anything that may make time based sharding easier - this probably includes shard aliasing, like we have collection aliasing now - when creating a collection, the ability to specify a router (and associated config for the router) and have that persisted, etc - using request parameters or document fields to direct what shard an update is for - I think a lot of this is done/working (implicit router), but lacks tests Further down the road, we also need to consider what to do about deletes, real-time get, or other operations where we would normally derive the shard from the ID... if it's not provided in another form, then we could decide to broadcast to all shards. > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13679508#comment-13679508 ] Noble Paul commented on SOLR-4221: -- bq. ability to create a new shard via collections API What do we do if the cluster already have docs which would belong to this shard? Split them too? bq.when creating a collection, the ability to specify a router (and associated config for the router) and have that persisted, etc I guess this can be a separate issue and can be fixed right away bq.using request parameters or document fields to direct what shard an update is for Is there a DocRouter which shards on the basis of document fields? (Other than the composite Id router) . I feel we need another one which does not change id of the document > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681012#comment-13681012 ] Noble Paul commented on SOLR-4221: -- Assigning 'implicit' router in the absence of 'numShards' param is confusing for the user. I would like it this way. there is a param caller 'router' whose default is "compositeId" . if both 'numShards' and 'router' params are absent the create action should fail stating 'numShards is a required param for default 'compositeId' router. Proposed modifications to the collection CREATE action * creating a shard should be possible even if no nodes can be allocated to the shard. As and when nodes come up , it can be allocated to the shard * new required parameter 'shard' (multivalued) for 'implicit' route New collection action , CREATE_SHARD * params . 'shard' (multivalued) optional . It creates new shards and try to allocate nodes for the same. If enough nodes are not available the shards will remain in 'construction' state * replicationFactor : (optional) if absent the value will be inherited from the collection New collection action , UPDATE_SHARD * replicationFactor : provide a new value to add capacity to an existing shard The name of the Router is a bit confusing. I fee it should be called "explicit" instead of "implicit". But let us leave it > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681207#comment-13681207 ] Yonik Seeley commented on SOLR-4221: bq. 'numShards is a required param for default 'compositeId' router. We want things to be as easy as possible to get going... it may make sense to have a default of 1 for numShards. bq. The name of the Router is a bit confusing. I fee it should be called "explicit" instead of "implicit". But let us leave it Yeah... I guess what I was thinking at the time was when someone sent and update to a shard w/ no further routing info. The update is assumed to belong on the receiving shard (that's the implicit part). Of course it should work with explict shard info too. bq. new required parameter 'shard' (multivalued) for 'implicit' route One should be able to create a collection w/ no shards also (although normally the next steps would be to create shards). > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681213#comment-13681213 ] Noble Paul commented on SOLR-4221: -- bq.We want things to be as easy as possible to get going... it may make sense to have a default of 1 for numShards. In case of "implicit" router , we do not wish to assign shard names . So the parameter is pretty useless. We can only create as many shards as user provides. If no 'shard' params are present we will have to create a collection with no shards bq.The update is assumed to belong on the receiving shard (that's the implicit part). yeah. makes sense. I can't imagine a name which makes sense foe both cases bq.One should be able to create a collection w/ no shards also (although normally the next steps would be to create shards). Is it possible to create collection w/ no shard?, or we do we need to add support for that first? > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681234#comment-13681234 ] Yonik Seeley commented on SOLR-4221: bq. In case of "implicit" router , we do not wish to assign shard names Right - I was more talking about the default when we were already using the compositeId router. It seems like that for the collections API, defaulting to router=compositeId and numShards=1 is reasonable. bq. Is it possible to create collection w/ no shard?, or we do we need to add support for that first? I don't think the collections API currently has support for routers other than the default compositeId (just tried it... I get a NPE if I don't provide numShards). There may be issues at a lower level too - not sure (initially, collections were created by bringing up a core - the collections API came later). > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681258#comment-13681258 ] Mark Miller commented on SOLR-4221: --- bq. I get a NPE if I don't provide numShards) That's a weird little bug I ran into a couple days ago - you should be getting a message about numShards being required (it is currently). > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681494#comment-13681494 ] Noble Paul commented on SOLR-4221: -- Yes, It gives an NPE if numShards is missing. It should throw a proper message why it is failing. That'll be fixed as a part of this > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13682291#comment-13682291 ] Mark Miller commented on SOLR-4221: --- Nice tracking that down! It was weird as hell. I could not figure out what I was missing. I think we should fix it as it's own bug - it can go into 4.4 when this may not. > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684936#comment-13684936 ] Otis Gospodnetic commented on SOLR-4221: [~noble.paul] should SOLR-4059 be closed as dupe? > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13686639#comment-13686639 ] Mark Miller commented on SOLR-4221: --- It's only a dupe by text name afaict - really it should be a part tracked as part of this issue IMO. > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689573#comment-13689573 ] Noble Paul commented on SOLR-4221: -- why do we keep the numShards in the CoreDescriptor? It says " // setting only matters on core creation" .Is it the right place to keep it > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689925#comment-13689925 ] Shalin Shekhar Mangar commented on SOLR-4221: - bq. why do we keep the numShards in the CoreDescriptor? It says " // setting only matters on core creation" .Is it the right place to keep it No it is not the right place. Besides numShards, there's shardRange and shardState too which are used only for core creation. We need to figure out where we can keep such properties that are useful only during core creation. > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689991#comment-13689991 ] Mark Miller commented on SOLR-4221: --- I don't think it's a big deal myself - if it can be improved without adding complication, thats fine - but you can still simply view a CoreDescriptor as information to be used for core creation. I don't know that it ever promised to have 'live' information or something - the SolrCore reads the info on the CoreDescriptor to create the SolrCore. It is not 100% satisfying, but unless the solution is pretty simple, I don't think it's worth worrying about near term. > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13694670#comment-13694670 ] Noble Paul commented on SOLR-4221: -- Ideally a user would want new shards to go to new nodes so that the existing nodes are not overloaded. Now it is not possible to achieve it. If I start a node before t5he command is issued it would just join some random shard and it is taken up. If I issue the CREATESHARD command before adding nodes it will/should fail saying insufficient nodes . Actually, it is relevant for SPLITSHARDas well. Now the new shards created will go and live in an already loaded node. The proposed behavior is this. If I startup a node w/o specifying the collection , it would just be a part of the live nodes and no shard and collection should be assigned to it. So when a new shard has to be created , the command can lookup and see if there are unutilized live nodes and it can be assigned there > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13694736#comment-13694736 ] Mark Miller commented on SOLR-4221: --- bq. If I startup a node w/o specifying the collection To get that behavior we should be starting with no cores defined out of the box - you start up a node and it joins nothing because it has no cores. Then collection create commands and what not can start using that empty node. I don't think we care about starting a core that doesn't belong to a collection - we just want to get to the point where you start a solrcloud node and there is no predefined cores. > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13694789#comment-13694789 ] Yonik Seeley commented on SOLR-4221: bq. The proposed behavior is this. If I startup a node w/o specifying the collection , it would just be a part of the live nodes and no shard and collection should be assigned to it. That's not so friendly though... seems like most of the time we would want to just be able to start a new node and have it utilized automatically. I think your use case (reserving some nodes for a new collection/shard/whatever you know you are going to create) may be best served by temporarily setting autoCreateReplicas=false (or whatever) via API. If autoCreateReplicas==false, then there should be a way to bring up a node and not have cores created on it. > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13694815#comment-13694815 ] Noble Paul commented on SOLR-4221: -- I miss something here. What is the work flow like? Do I add nodes first or do I create the shards/collection first? I can pass these api params only when the shard/collection is happening. Automatically assigning nodes to shards even when we have sufficient replicas is something I find strange > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13694823#comment-13694823 ] Yonik Seeley commented on SOLR-4221: bq. What is the work flow like? Do I add nodes first or do I create the shards/collection first? Both? bq. Automatically assigning nodes to shards even when we have sufficient replicas is something I find strange Yeah, that's sort of a different case... I was sort of assuming that we either didn't have sufficient replicas, or that the overseer may chose to rebalance some replicas off more heavily loaded nodes. > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13694827#comment-13694827 ] Noble Paul commented on SOLR-4221: -- If I create the shard first and I don't have enough nodes to sign it to, it should probably just fai just the way create collection does. That would be the consistent behavior Auto rebalancing should be an opt in thing , right? > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13694844#comment-13694844 ] Yonik Seeley commented on SOLR-4221: bq. If I create the shard first and I don't have enough nodes to sign it to, it should probably just fail Seems like it should be fine to create a shard w/ no current replicas (but it should also be fine to specify a certain minimum replication factor for the create to succeed). As far as consistency with how collection creation works, I think we're going to need an additional parameter now that we can store the the replicationFactor in ZK and it can act as a target. We need something like createReplicationFactor or something that specifies the minimum replication factor to achieve for the collection creation to succeed. bq. Auto rebalancing should be an opt in thing , right? It should certainly be optional/configurable. What the default should be is a decision we can tackle once it's implemented. > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695252#comment-13695252 ] Noble Paul commented on SOLR-4221: -- bq.As far as consistency with how collection creation works, I think we're going to need an additional parameter now that we can store the the replicationFactor in ZK and it can act as a target. Do we really need to have so many configuration params and complicate this? I believe we should define a standard recipe for doing something and people will just follow . Adding more options like createReplictionFactor is not elegant . If you wish to create a new collection/shard w/o enough nodes we can just use something like forceCreate=true so that the collection and shards are created empty + inactive . if forceCreate=false let us just fail if nodes are insufficient > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695723#comment-13695723 ] Jan Høydahl commented on SOLR-4221: --- bq. If you wish to create a new collection/shard w/o enough nodes we can just use something like forceCreate=true so that the collection and shards are created empty + inactive . if forceCreate=false let us just fail if nodes are insufficient +1 When adding a new empty node, it should only get auto-assigned replicas if any of the existing collections have not filled up its numShards and replicationFactor yet. Else it should be left empty and then be utilized by future create collection requests or rebalance/split requests. > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695754#comment-13695754 ] Yonik Seeley commented on SOLR-4221: bq. Do we really need to have so many configuration params and complicate this? Eh, you replaced one parameter (a minimum replication factor to achieve) with another (a boolean meaning 0 or replicationFactor). This seems akin to facet.includeZeros vs facet.minCount... the latter incorporates the former and is more powerful (hence fewer parameters in the long run), and the former would have never existed if the latter had been thought of first. Now, I don't like the name "createReplictionFactor" (maybe "minReplicas" or something like that would be better), but the functionality is a superset of "forceCreate" and is more descriptive (after all, there are many ways to force things). It also seems useful to say something like "create this collection and make sure it's usable (at least one replica for every shard) but don't worry about trying to satisfy the replciationFactor just yet). That would be minReplicas=1 > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13696165#comment-13696165 ] Noble Paul commented on SOLR-4221: -- I agree with you Yonik. forceCreate is a superset and can mean a lot of things. minReplicas make sense. But we need a clear definition of the following . Otherwise we are getting into a slippery slope * replicationFactor : I'm still not very clear about this because it has a different meanoing in other nosql systems. * minReplicas does it also mean if ever the no:of replicas fall below the 'minReplicas' , write ops would fail? Or does it only mean that it is only for creation of cluster/shard ? we should also go ahead and define what those will mean when the 'autoManageCluster' (or whatever it is going to be called) is implemented . > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13696766#comment-13696766 ] Noble Paul commented on SOLR-4221: -- If I specify replicationFactor=2 and minReplicas=1 and Ithe cluster could create only 1 core , What is the state of the shard? ACTIVE|INACTIVE or something new ? > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13697930#comment-13697930 ] Yonik Seeley commented on SOLR-4221: bq. does it also mean if ever the no:of replicas fall below the 'minReplicas' , write ops would fail? I was considering it only as a create-time parameter (not persisted), and had originally thought about a parameter on updates that could specify the required replication level for safety. But we really need to think about all the use-cases we can up front - it could make sense to have additional persisted properties around replicationFactor. One use case is expanding your search capacity dynamically by simply starting up new nodes that automatically become replicas of existing shards. This is the current behavior today. We need to think about if/how to handle this use case when we start persisting replicationFactor and if the overseer will destroy nodes to bring things back to the replication factor. One way would be to set the replicationFactor really high (to prevent this distruction), but this could lead to other negatives such as a future GUI displaying the status of the collection as degraded since the replicationFactor isn't met. Requiring the user to bump the replicationFactor up before adding new nodes isn't the friendliest thing given that this will be a very common use case. > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13698040#comment-13698040 ] Shai Erera commented on SOLR-4221: -- Another usecase is to have multiple collections, each with different number of shards and different #replicas. I would not want to see 10 replicas for each shard just because I loaded 10 servers. Rather, I think it would be useful if I could say "collection1, #shards=3, #replicas=2" (total 6 indexes) and "collection2, #shards=4, #replicas=5" (total 20 indexes) and if I bring up a 10 nodes cluster, the indexes are somehow distributed across them. > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13698051#comment-13698051 ] Yonik Seeley commented on SOLR-4221: bq. Another usecase is to have multiple collections, each with different number of shards and different #replicas. Right - we need to figure out how to accomodate all these use cases. > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13698142#comment-13698142 ] Noble Paul commented on SOLR-4221: -- bq.Another usecase is to have multiple collections, each with different number of shards and different #replicas. This is why I feel we should not automatically create cores in nodes that come up. The best thing to do is Any new node will just not participate in any collection (unless a shard has fewer nodes than replicationFactor) There should be an explicit ASSIGN_NODE command to add/remove nodes to/from a shard. It should be possible to do an ASSIGN_NODE without specifying a nodename in which case the overseer would look for free nodes in the cluster and add to the specified shard when a new shard is created by CREATESHARD command or when a SPLITSHARD command this nodes could be automatically be taken up. > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13698158#comment-13698158 ] Shai Erera commented on SOLR-4221: -- Exactly. New nodes mean just more resources, I don't think they should affect the number of resources the search cluster needs, only define how many resources are available. Conceptually, we could have a cost function which takes into account the node's RAM/CPU/Disk + indexes size (GBs, #docs), maybe even based on performance history, and assign indexes to nodes following that cost function. The way I picture it (and I don't know how to map that to Solr jargon): * A new node comes up, broadcasts "I'm alive" * Maybe communicates with other "overbooked" nodes to transfer indexes over * A ClusterBalancer (Overseer?) periodically checks if the cluster can be better balanced (e.g. splitting shards, moving indexes) * Maybe even we drop numShards entirely and make this a completely dynamic and automatic setting (or at least optional), but this is for future work > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13702066#comment-13702066 ] Noble Paul commented on SOLR-4221: -- The working code is posted here https://github.com/shalinmangar/lucene-solr . We will start moving it to SVN after 4.4 branching > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13725021#comment-13725021 ] Anshum Gupta commented on SOLR-4221: You may want to check on the current router being used, before being able to add a shard. > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13725165#comment-13725165 ] Anshum Gupta commented on SOLR-4221: Spoke to Noble offline and perhaps it's fine to deliberately not check it in OCP and just in the CollectionsHandler. This lets Shards be created even when the router is non-implicit for cases like ShardSplitting. > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13725494#comment-13725494 ] ASF subversion and git services commented on SOLR-4221: --- Commit 1508968 from [~noble.paul] in branch 'dev/trunk' [ https://svn.apache.org/r1508968 ] SOLR-4221 SOLR-4808 SOLR-5006 SOLR-5017 SOLR-4222 > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13725549#comment-13725549 ] ASF subversion and git services commented on SOLR-4221: --- Commit 1508981 from [~noble.paul] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1508981 ] SOLR-4221 SOLR-4808 SOLR-5006 SOLR-5017 SOLR-4222 > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13725625#comment-13725625 ] Noble Paul commented on SOLR-4221: -- A custom collection can be created by explicitly specifying the router as 'implicit' and by giving the names of the shards explicitly as follows http://host:port/solr/admin/collections?action=CREATE&name=monthly&router=implicit&collection.configName=x&shards=jan13,feb13,mar13 where 3 shards named jan13,feb13,mar13 will be created h2.Document routing * add a special request param called \_route_=june13’ . It means all docs added in the batch would belong to the same shard or, * add a special field \_route_ in the document to signal the shard where it belongs to * if the collection is created with ‘routeField’ value , the docRouter would look at that field to get the shard name SOLR-5017 * If it is unable to identify the shard using any of the above methods , the write operation fails. If the shard is not present , then the write fails too h2.Query routing It remain same as other routers. \_route_ parameter can specify the actual shard name h2.Creating new shards There is a a new command CREATESHARD . refer to SOLR-5006. > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13725670#comment-13725670 ] ASF subversion and git services commented on SOLR-4221: --- Commit 1509017 from sha...@apache.org in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1509017 ] SOLR-4221: Fix compile error on Java6 due to use of diamond operator > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13764968#comment-13764968 ] Hoss Man commented on SOLR-4221: Is this feature complete? (all of the subtasks are marked resolved and several commits associated with this isue are in branch 4x) Can someone who understands all of the various changes made in these issues please update the ref guide (or post a comment suggestion what additions should be made)... https://cwiki.apache.org/confluence/display/solr/Collections+API > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[
https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13769685#comment-13769685
]
Noble Paul commented on SOLR-4221:
--
Makes sense
when the collection is created let us accept the parameters with the 'router'
prefix . eg: router.name=implicit&router.field=somefield. So any value that has
a router prefix will go into the router attribute. we can use it in the future
for more attributes and even custom attributes for custom routers. eg:
router.class=my.ClassName&router.attr1=x&router.attr2=y
when it is persisted, it can be
router : {"name":"implicit","field":"somefield"}
> Custom sharding
> ---
>
> Key: SOLR-4221
> URL: https://issues.apache.org/jira/browse/SOLR-4221
> Project: Solr
> Issue Type: New Feature
>Reporter: Yonik Seeley
>Assignee: Noble Paul
> Fix For: 4.5, 5.0
>
> Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch,
> SOLR-4221.patch, SOLR-4221.patch
>
>
> Features to let users control everything about sharding/routing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13769675#comment-13769675 ] Yonik Seeley commented on SOLR-4221: Hmmm, it feels like the "routeField" parameter should be scoped under "router" somehow... I think there will be additional parameters to configure a router in the future (such as number of bits to allocate to parts of the compositeId router, etc), as well as custom routers, where their configuration could include additional parameters that would best be scoped. router.routeField? > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Fix For: 4.5, 5.0 > > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13770854#comment-13770854 ] ASF subversion and git services commented on SOLR-4221: --- Commit 1524442 from [~noble.paul] in branch 'dev/trunk' [ https://svn.apache.org/r1524442 ] SOLR-4221 changed format router is stored > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Fix For: 4.5, 5.0 > > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13770865#comment-13770865 ] ASF subversion and git services commented on SOLR-4221: --- Commit 1524446 from [~noble.paul] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1524446 ] SOLR-4221 changed format router is stored > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Fix For: 4.5, 5.0 > > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13771910#comment-13771910 ] ASF subversion and git services commented on SOLR-4221: --- Commit 1524742 from [~jpountz] in branch 'dev/branches/lucene_solr_4_5' [ https://svn.apache.org/r1524742 ] SOLR-4221 changed format router is stored > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Fix For: 4.5, 5.0 > > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776649#comment-13776649 ] Yonik Seeley commented on SOLR-4221: I think the last patch here broke back compat with clusterstate (Markus hit this, reported in SOLR-5261) I'll try to come up with a patch. It shouldn't be too hard to fix (longest part will be testing). > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Fix For: 4.5, 5.0 > > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776723#comment-13776723 ] ASF subversion and git services commented on SOLR-4221: --- Commit 1526003 from [~yo...@apache.org] in branch 'dev/trunk' [ https://svn.apache.org/r1526003 ] SOLR-4221: back compat for router spec > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Fix For: 4.5, 5.0 > > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776740#comment-13776740 ] ASF subversion and git services commented on SOLR-4221: --- Commit 1526007 from [~yo...@apache.org] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1526007 ] SOLR-4221: back compat for router spec > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Fix For: 4.5, 5.0 > > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776742#comment-13776742 ] ASF subversion and git services commented on SOLR-4221: --- Commit 1526008 from [~yo...@apache.org] in branch 'dev/branches/lucene_solr_4_5' [ https://svn.apache.org/r1526008 ] SOLR-4221: back compat for router spec > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Fix For: 4.5, 5.0 > > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13777141#comment-13777141 ] Noble Paul commented on SOLR-4221: -- [~yo...@apache.org] I don't think we can fix this in a a backcompat manner if we reuse the name router and change the Object type. There will be atleast one node in the cluster which will have the old binaries and will expect the router instanceof String. the new binaries will keep writing router as Map. The backward compatible solution would be to choose another name say "route" and if the 'router' value is missing, the old binaries would not complain > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Fix For: 4.5, 5.0 > > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13777142#comment-13777142 ] Yonik Seeley commented on SOLR-4221: bq. I don't think we can fix this in a a backcompat manner There are different degrees of back compat, and I've already committed something that at least works a lot better than what was there. I tested by creating a 4.4 cluster, bringing it down, and then replacing the jar file with 4.5. > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Fix For: 4.5, 5.0 > > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13777161#comment-13777161 ] Noble Paul commented on SOLR-4221: -- Yeah , that works But if my usecase is, Bring down each node , replace the jar and restart the node this will fail. right? If this does not work , we will have to document it clearly > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Fix For: 4.5, 5.0 > > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[
https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13777407#comment-13777407
]
Markus Jelsma commented on SOLR-4221:
-
Ah, i come from SOLR-5261 and noticed the discussion here so i'll comment here
as well. The stack trace i posted earlier is gone now and is replaced by the
trace below. This only happens if a new SolrJ attempts to talk to a slightly
older cluster.
{code}
java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Map
at
org.apache.solr.common.cloud.DocRouter.getRouteField(DocRouter.java:54)
at
org.apache.solr.common.cloud.CompositeIdRouter.sliceHash(CompositeIdRouter.java:64)
at
org.apache.solr.common.cloud.HashBasedRouter.getTargetSlice(HashBasedRouter.java:33)
at
org.apache.solr.client.solrj.request.UpdateRequest.getRoutes(UpdateRequest.java:190)
at
org.apache.solr.client.solrj.impl.CloudSolrServer.directUpdate(CloudSolrServer.java:313)
at
org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:506)
at
org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
{code}
> Custom sharding
> ---
>
> Key: SOLR-4221
> URL: https://issues.apache.org/jira/browse/SOLR-4221
> Project: Solr
> Issue Type: New Feature
>Reporter: Yonik Seeley
>Assignee: Noble Paul
> Fix For: 4.5, 5.0
>
> Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch,
> SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch,
> SOLR-4221.patch
>
>
> Features to let users control everything about sharding/routing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13777436#comment-13777436 ] Noble Paul commented on SOLR-4221: -- Thanks [~markus17] good catch > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Fix For: 4.5, 5.0 > > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13777807#comment-13777807 ] ASF subversion and git services commented on SOLR-4221: --- Commit 1526244 from [~yo...@apache.org] in branch 'dev/trunk' [ https://svn.apache.org/r1526244 ] SOLR-4221: make new solrj client/router able to read old clusterstate > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Fix For: 4.5, 5.0 > > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13777862#comment-13777862 ] ASF subversion and git services commented on SOLR-4221: --- Commit 1526255 from [~yo...@apache.org] in branch 'dev/branches/lucene_solr_4_5' [ https://svn.apache.org/r1526255 ] SOLR-4221: make new solrj client/router able to read old clusterstate > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Fix For: 4.5, 5.0 > > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13778509#comment-13778509 ] ASF subversion and git services commented on SOLR-4221: --- Commit 1526395 from [~noble.paul] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1526395 ] SOLR-4221 make new solrj client/router able to read old clusterstate > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Fix For: 4.5, 5.0 > > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[
https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13778609#comment-13778609
]
Markus Jelsma commented on SOLR-4221:
-
I tested it again with a SolrJ of just now and an older cluster 18th of
september. It keeps throwing the exception:
{code}
java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Map
at
org.apache.solr.common.cloud.DocRouter.getRouteField(DocRouter.java:54)
at
org.apache.solr.common.cloud.CompositeIdRouter.sliceHash(CompositeIdRouter.java:64)
at
org.apache.solr.common.cloud.HashBasedRouter.getTargetSlice(HashBasedRouter.java:33)
at
org.apache.solr.client.solrj.request.UpdateRequest.getRoutes(UpdateRequest.java:190)
at
org.apache.solr.client.solrj.impl.CloudSolrServer.directUpdate(CloudSolrServer.java:313)
at
org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:506)
at
org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
{code}
Due to the commit message i assumed it would work now.
> Custom sharding
> ---
>
> Key: SOLR-4221
> URL: https://issues.apache.org/jira/browse/SOLR-4221
> Project: Solr
> Issue Type: New Feature
>Reporter: Yonik Seeley
>Assignee: Noble Paul
> Fix For: 4.5, 5.0
>
> Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch,
> SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch,
> SOLR-4221.patch
>
>
> Features to let users control everything about sharding/routing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13778613#comment-13778613 ] Noble Paul commented on SOLR-4221: -- Did u test with the newest build? The line numbers aere referring to old build > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Fix For: 4.5, 5.0 > > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13778617#comment-13778617 ] Markus Jelsma commented on SOLR-4221: - Ah, perhaps the current 5.0-SNAPSHOT doesn't yet contain the fix. I'll try again a bit later. > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Fix For: 4.5, 5.0 > > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13778664#comment-13778664 ] Adrien Grand commented on SOLR-4221: Thanks for doing these tests Markus! I'll wait for your confirmation that the issue is fixed before building new artifacts. > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Fix For: 4.5, 5.0 > > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4221) Custom sharding
[ https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13778679#comment-13778679 ] Markus Jelsma commented on SOLR-4221: - Adrien, SolrJ can now index to an older cluster successfully! Thanks Yonik, Noble! This is great! > Custom sharding > --- > > Key: SOLR-4221 > URL: https://issues.apache.org/jira/browse/SOLR-4221 > Project: Solr > Issue Type: New Feature >Reporter: Yonik Seeley >Assignee: Noble Paul > Fix For: 4.5, 5.0 > > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch > > > Features to let users control everything about sharding/routing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Commented] (SOLR-4221) Custom sharding
just grand me edit access On Thu, Sep 12, 2013 at 5:11 AM, Hoss Man (JIRA) wrote: > > [ > https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13764968#comment-13764968] > > Hoss Man commented on SOLR-4221: > > > Is this feature complete? (all of the subtasks are marked resolved and > several commits associated with this isue are in branch 4x) > > > Can someone who understands all of the various changes made in these > issues please update the ref guide (or post a comment suggestion what > additions should be made)... > > https://cwiki.apache.org/confluence/display/solr/Collections+API > > > > > > Custom sharding > > --- > > > > Key: SOLR-4221 > > URL: https://issues.apache.org/jira/browse/SOLR-4221 > > Project: Solr > > Issue Type: New Feature > >Reporter: Yonik Seeley > >Assignee: Noble Paul > > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, SOLR-4221.patch > > > > > > Features to let users control everything about sharding/routing. > > -- > This message is automatically generated by JIRA. > If you think it was sent incorrectly, please contact your JIRA > administrators > For more information on JIRA, see: http://www.atlassian.com/software/jira > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > > -- - Noble Paul
Re: [jira] [Commented] (SOLR-4221) Custom sharding
: just grand me edit access Are you talking about edit access for the ref guide? there's a well documented process for that... https://cwiki.apache.org/confluence/display/solr/Internal+-+Maintaining+Documentation#Internal-MaintainingDocumentation-WhoCanEditThisDocumentation https://cwiki.apache.org/confluence/display/solr/Internal+-+CWIKI+ACLs That doesn't answer my first question however: is SOLR-4221 complete? should the issue be resolved as "fixed in 4.5" ? : On Thu, Sep 12, 2013 at 5:11 AM, Hoss Man (JIRA) wrote: : : > : > [ : > https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13764968#comment-13764968] : > : > Hoss Man commented on SOLR-4221: : > : > : > Is this feature complete? (all of the subtasks are marked resolved and : > several commits associated with this isue are in branch 4x) : > : > : > Can someone who understands all of the various changes made in these : > issues please update the ref guide (or post a comment suggestion what : > additions should be made)... : > : > https://cwiki.apache.org/confluence/display/solr/Collections+API : > : > : > : > : > > Custom sharding : > > --- : > > : > > Key: SOLR-4221 : > > URL: https://issues.apache.org/jira/browse/SOLR-4221 : > > Project: Solr : > > Issue Type: New Feature : > >Reporter: Yonik Seeley : > >Assignee: Noble Paul : > > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, : > SOLR-4221.patch, SOLR-4221.patch : > > : > > : > > Features to let users control everything about sharding/routing. : > : > -- : > This message is automatically generated by JIRA. : > If you think it was sent incorrectly, please contact your JIRA : > administrators : > For more information on JIRA, see: http://www.atlassian.com/software/jira : > : > - : > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org : > For additional commands, e-mail: dev-h...@lucene.apache.org : > : > : : : -- : - : Noble Paul : -Hoss - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Commented] (SOLR-4221) Custom sharding
Noble, I've added you to the solr-committers confluence group, so you should now be able to edit the wiki. Steve On Sep 12, 2013, at 1:08 AM, Noble Paul നോബിള് नोब्ळ् wrote: > just grand me edit access > > > On Thu, Sep 12, 2013 at 5:11 AM, Hoss Man (JIRA) wrote: > > [ > https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13764968#comment-13764968 > ] > > Hoss Man commented on SOLR-4221: > > > Is this feature complete? (all of the subtasks are marked resolved and > several commits associated with this isue are in branch 4x) > > > Can someone who understands all of the various changes made in these issues > please update the ref guide (or post a comment suggestion what additions > should be made)... > > https://cwiki.apache.org/confluence/display/solr/Collections+API > > > > > > Custom sharding > > --- > > > > Key: SOLR-4221 > > URL: https://issues.apache.org/jira/browse/SOLR-4221 > > Project: Solr > > Issue Type: New Feature > >Reporter: Yonik Seeley > >Assignee: Noble Paul > > Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, > > SOLR-4221.patch, SOLR-4221.patch > > > > > > Features to let users control everything about sharding/routing. > > -- > This message is automatically generated by JIRA. > If you think it was sent incorrectly, please contact your JIRA administrators > For more information on JIRA, see: http://www.atlassian.com/software/jira > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > > > > > -- > - > Noble Paul - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Commented] (SOLR-4221) Custom sharding
Yes, It is resolved. On 12 Sep 2013 21:22, "Chris Hostetter" wrote: > > : just grand me edit access > > Are you talking about edit access for the ref guide? > > there's a well documented process for that... > > > https://cwiki.apache.org/confluence/display/solr/Internal+-+Maintaining+Documentation#Internal-MaintainingDocumentation-WhoCanEditThisDocumentation > https://cwiki.apache.org/confluence/display/solr/Internal+-+CWIKI+ACLs > > That doesn't answer my first question however: is SOLR-4221 complete? > should the issue be resolved as "fixed in 4.5" ? > > > : On Thu, Sep 12, 2013 at 5:11 AM, Hoss Man (JIRA) > wrote: > : > : > > : > [ > : > > https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13764968#comment-13764968 > ] > : > > : > Hoss Man commented on SOLR-4221: > : > > : > > : > Is this feature complete? (all of the subtasks are marked resolved and > : > several commits associated with this isue are in branch 4x) > : > > : > > : > Can someone who understands all of the various changes made in these > : > issues please update the ref guide (or post a comment suggestion what > : > additions should be made)... > : > > : > https://cwiki.apache.org/confluence/display/solr/Collections+API > : > > : > > : > > : > > : > > Custom sharding > : > > --- > : > > > : > > Key: SOLR-4221 > : > > URL: https://issues.apache.org/jira/browse/SOLR-4221 > : > > Project: Solr > : > > Issue Type: New Feature > : > >Reporter: Yonik Seeley > : > >Assignee: Noble Paul > : > > Attachments: SOLR-4221.patch, SOLR-4221.patch, > SOLR-4221.patch, > : > SOLR-4221.patch, SOLR-4221.patch > : > > > : > > > : > > Features to let users control everything about sharding/routing. > : > > : > -- > : > This message is automatically generated by JIRA. > : > If you think it was sent incorrectly, please contact your JIRA > : > administrators > : > For more information on JIRA, see: > http://www.atlassian.com/software/jira > : > > : > - > : > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > : > For additional commands, e-mail: dev-h...@lucene.apache.org > : > > : > > : > : > : -- > : - > : Noble Paul > : > > -Hoss > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > >
