[jira] [Commented] (SOLR-4321) SolrCloud collection CREATE will put multiple master shards on a single node (and no shards on some)
[ https://issues.apache.org/jira/browse/SOLR-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561682#comment-13561682 ] Commit Tag Bot commented on SOLR-4321: -- [branch_4x commit] Mark Robert Miller http://svn.apache.org/viewvc?view=revision&revision=1438046 SOLR-4321: Collections API will sometimes use a node more than once, even when more unused nodes are available. > SolrCloud collection CREATE will put multiple master shards on a single node > (and no shards on some) > > > Key: SOLR-4321 > URL: https://issues.apache.org/jira/browse/SOLR-4321 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Affects Versions: 4.1 >Reporter: Brett Hoerner >Assignee: Mark Miller > Fix For: 4.2, 5.0 > > Attachments: SOLR-4321.patch, SOLR-4321.patch, SOLR-4321.patch > > > This is best described by a photo of my SolrCloud admin: > http://i.imgur.com/hW4xnxy.png > I have a 4 node cluster, with shard count of 2 and replication factor of 2. > After running something like, > > http://localhost:8983/solr/admin/collections?action=CREATE&name=15724&numShards=2&replicationFactor=2 > The shards seem to be completely randomly placed, which is fine, but some > nodes receive more of the share than others (some even receiving none). > For example, 15724 has given node "backfill-1d" 2 "parts", > 15724_shard1_replica2 and 15724_shard2_replica1 but has given "backfill-2d" > nothing at all. > This creates unnecessary load on some nodes, no? Is this something I am > supposed to handle myself (I looked, but don't see how). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4321) SolrCloud collection CREATE will put multiple master shards on a single node (and no shards on some)
[ https://issues.apache.org/jira/browse/SOLR-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561676#comment-13561676 ] Commit Tag Bot commented on SOLR-4321: -- [trunk commit] Mark Robert Miller http://svn.apache.org/viewvc?view=revision&revision=1438043 SOLR-4321: Collections API will sometimes use a node more than once, even when more unused nodes are available. > SolrCloud collection CREATE will put multiple master shards on a single node > (and no shards on some) > > > Key: SOLR-4321 > URL: https://issues.apache.org/jira/browse/SOLR-4321 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Affects Versions: 4.1 >Reporter: Brett Hoerner >Assignee: Mark Miller > Fix For: 4.2, 5.0 > > Attachments: SOLR-4321.patch, SOLR-4321.patch, SOLR-4321.patch > > > This is best described by a photo of my SolrCloud admin: > http://i.imgur.com/hW4xnxy.png > I have a 4 node cluster, with shard count of 2 and replication factor of 2. > After running something like, > > http://localhost:8983/solr/admin/collections?action=CREATE&name=15724&numShards=2&replicationFactor=2 > The shards seem to be completely randomly placed, which is fine, but some > nodes receive more of the share than others (some even receiving none). > For example, 15724 has given node "backfill-1d" 2 "parts", > 15724_shard1_replica2 and 15724_shard2_replica1 but has given "backfill-2d" > nothing at all. > This creates unnecessary load on some nodes, no? Is this something I am > supposed to handle myself (I looked, but don't see how). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4321) SolrCloud collection CREATE will put multiple master shards on a single node (and no shards on some)
[ https://issues.apache.org/jira/browse/SOLR-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13559784#comment-13559784 ] Mark Miller commented on SOLR-4321: --- Thanks guys - I've added a test. Will commit soon. > SolrCloud collection CREATE will put multiple master shards on a single node > (and no shards on some) > > > Key: SOLR-4321 > URL: https://issues.apache.org/jira/browse/SOLR-4321 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Affects Versions: 4.1 >Reporter: Brett Hoerner >Assignee: Mark Miller > Attachments: SOLR-4321.patch, SOLR-4321.patch, SOLR-4321.patch > > > This is best described by a photo of my SolrCloud admin: > http://i.imgur.com/hW4xnxy.png > I have a 4 node cluster, with shard count of 2 and replication factor of 2. > After running something like, > > http://localhost:8983/solr/admin/collections?action=CREATE&name=15724&numShards=2&replicationFactor=2 > The shards seem to be completely randomly placed, which is fine, but some > nodes receive more of the share than others (some even receiving none). > For example, 15724 has given node "backfill-1d" 2 "parts", > 15724_shard1_replica2 and 15724_shard2_replica1 but has given "backfill-2d" > nothing at all. > This creates unnecessary load on some nodes, no? Is this something I am > supposed to handle myself (I looked, but don't see how). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4321) SolrCloud collection CREATE will put multiple master shards on a single node (and no shards on some)
[ https://issues.apache.org/jira/browse/SOLR-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13558372#comment-13558372 ] Brett Hoerner commented on SOLR-4321: - Eric's patch works for me, 4 node cluster, 2 shards, 2 replicas and each node gets 1 piece of the pie. > SolrCloud collection CREATE will put multiple master shards on a single node > (and no shards on some) > > > Key: SOLR-4321 > URL: https://issues.apache.org/jira/browse/SOLR-4321 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Affects Versions: 4.1 >Reporter: Brett Hoerner > Attachments: SOLR-4321.patch > > > This is best described by a photo of my SolrCloud admin: > http://i.imgur.com/hW4xnxy.png > I have a 4 node cluster, with shard count of 2 and replication factor of 2. > After running something like, > > http://localhost:8983/solr/admin/collections?action=CREATE&name=15724&numShards=2&replicationFactor=2 > The shards seem to be completely randomly placed, which is fine, but some > nodes receive more of the share than others (some even receiving none). > For example, 15724 has given node "backfill-1d" 2 "parts", > 15724_shard1_replica2 and 15724_shard2_replica1 but has given "backfill-2d" > nothing at all. > This creates unnecessary load on some nodes, no? Is this something I am > supposed to handle myself (I looked, but don't see how). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4321) SolrCloud collection CREATE will put multiple master shards on a single node (and no shards on some)
[ https://issues.apache.org/jira/browse/SOLR-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13558149#comment-13558149 ] Eric Falcao commented on SOLR-4321: --- Here is a better permalink for brett's issue: http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201301.mbox/%3CCAC8vSzZ2hmdtm6iWc9Byf6ESSATuKc7Xq%3DoPHFNeEAWqCTtJ8Q%40mail.gmail.com%3E > SolrCloud collection CREATE will put multiple master shards on a single node > (and no shards on some) > > > Key: SOLR-4321 > URL: https://issues.apache.org/jira/browse/SOLR-4321 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Affects Versions: 4.1 >Reporter: Brett Hoerner > > This is best described by a photo of my SolrCloud admin: > http://i.imgur.com/hW4xnxy.png > I have a 4 node cluster, with shard count of 2 and replication factor of 2. > After running something like, > > http://localhost:8983/solr/admin/collections?action=CREATE&name=15724&numShards=2&replicationFactor=2 > The shards seem to be completely randomly placed, which is fine, but some > nodes receive more of the share than others (some even receiving none). > For example, 15724 has given node "backfill-1d" 2 "parts", > 15724_shard1_replica2 and 15724_shard2_replica1 but has given "backfill-2d" > nothing at all. > This creates unnecessary load on some nodes, no? Is this something I am > supposed to handle myself (I looked, but don't see how). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4321) SolrCloud collection CREATE will put multiple master shards on a single node (and no shards on some)
[ https://issues.apache.org/jira/browse/SOLR-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13558146#comment-13558146 ] Eric Falcao commented on SOLR-4321: --- I am able to confirm this using the SolrCloud example documentation found at http://wiki.apache.org/solr/SolrCloud#Example_C:_Two_shard_cluster_with_shard_replicas_and_zookeeper_ensemble Reproduction steps: - Follow example C - Create a new collection: curl "http://localhost:8983/solr/admin/collections?action=CREATE&name=foo&numShards=2&replicationFactor=2"; - Verify that in the cloud admin, the new collection is imbalanced (it's not using the right mix of nodes) I annotated the node numbers, hoping to provide some clarity (since the ip is the same, i hope it adds some clarity): http://imgur.com/05L7eZ4 > SolrCloud collection CREATE will put multiple master shards on a single node > (and no shards on some) > > > Key: SOLR-4321 > URL: https://issues.apache.org/jira/browse/SOLR-4321 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Affects Versions: 4.1 >Reporter: Brett Hoerner > > This is best described by a photo of my SolrCloud admin: > http://i.imgur.com/hW4xnxy.png > I have a 4 node cluster, with shard count of 2 and replication factor of 2. > After running something like, > > http://localhost:8983/solr/admin/collections?action=CREATE&name=15724&numShards=2&replicationFactor=2 > The shards seem to be completely randomly placed, which is fine, but some > nodes receive more of the share than others (some even receiving none). > For example, 15724 has given node "backfill-1d" 2 "parts", > 15724_shard1_replica2 and 15724_shard2_replica1 but has given "backfill-2d" > nothing at all. > This creates unnecessary load on some nodes, no? Is this something I am > supposed to handle myself (I looked, but don't see how). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4321) SolrCloud collection CREATE will put multiple master shards on a single node (and no shards on some)
[ https://issues.apache.org/jira/browse/SOLR-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13558131#comment-13558131 ] Brett Hoerner commented on SOLR-4321: - Here are some logs from the very first boot of this cluster (repeatable). "default" is in solr.xml to start, I try creating test1, test2, etc using the same curl command from above (which works fine in 4.0): Jan 19, 2013 11:37:05 PM org.apache.solr.core.SolrCore initIndex WARNING: [default] Solr index directory '/mnt/solr/data/default/index' doesn't exist. Creating new index... Jan 19, 2013 11:45:18 PM org.apache.solr.core.SolrCore initIndex WARNING: [test1_shard2_replica2] Solr index directory '/mnt/solr/data/test1_shard2_replica2/index' doesn't exist. Creating new index... Jan 19, 2013 11:45:19 PM org.apache.solr.common.SolrException log SEVERE: Sync Failed Jan 19, 2013 11:48:29 PM org.apache.solr.core.SolrCore initIndex WARNING: [test2_shard2_replica2] Solr index directory '/mnt/solr/data/test2_shard2_replica2/index' doesn't exist. Creating new index... Jan 19, 2013 11:48:29 PM org.apache.solr.common.SolrException log SEVERE: Sync Failed Jan 19, 2013 11:48:31 PM org.apache.solr.core.SolrCore initIndex WARNING: [test3_shard1_replica1] Solr index directory '/mnt/solr/data/test3_shard1_replica1/index' doesn't exist. Creating new index... Jan 19, 2013 11:48:32 PM org.apache.solr.common.SolrException log SEVERE: Sync Failed Jan 19, 2013 11:48:34 PM org.apache.solr.core.SolrCore initIndex WARNING: [test4_shard2_replica1] Solr index directory '/mnt/solr/data/test4_shard2_replica1/index' doesn't exist. Creating new index... Jan 19, 2013 11:48:34 PM org.apache.solr.core.SolrCore initIndex WARNING: [test4_shard1_replica2] Solr index directory '/mnt/solr/data/test4_shard1_replica2/index' doesn't exist. Creating new index... Jan 19, 2013 11:48:36 PM org.apache.solr.common.SolrException log SEVERE: Sync Failed Jan 19, 2013 11:48:37 PM org.apache.solr.core.SolrCore initIndex WARNING: [test5_shard2_replica2] Solr index directory '/mnt/solr/data/test5_shard2_replica2/index' doesn't exist. Creating new index... Jan 19, 2013 11:48:39 PM org.apache.solr.common.SolrException log SEVERE: Sync Failed Jan 19, 2013 11:48:41 PM org.apache.solr.update.PeerSync sync WARNING: no frame of reference to tell of we've missed updates Jan 19, 2013 11:50:07 PM org.apache.solr.core.SolrCore initIndex WARNING: [foo_shard1_replica1] Solr index directory '/mnt/solr/data/foo_shard1_replica1/index' doesn't exist. Creating new index... Jan 19, 2013 11:50:08 PM org.apache.solr.common.SolrException log SEVERE: Sync Failed > SolrCloud collection CREATE will put multiple master shards on a single node > (and no shards on some) > > > Key: SOLR-4321 > URL: https://issues.apache.org/jira/browse/SOLR-4321 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Affects Versions: 4.1 >Reporter: Brett Hoerner > > This is best described by a photo of my SolrCloud admin: > http://i.imgur.com/hW4xnxy.png > I have a 4 node cluster, with shard count of 2 and replication factor of 2. > After running something like, > > http://localhost:8983/solr/admin/collections?action=CREATE&name=15724&numShards=2&replicationFactor=2 > The shards seem to be completely randomly placed, which is fine, but some > nodes receive more of the share than others (some even receiving none). > For example, 15724 has given node "backfill-1d" 2 "parts", > 15724_shard1_replica2 and 15724_shard2_replica1 but has given "backfill-2d" > nothing at all. > This creates unnecessary load on some nodes, no? Is this something I am > supposed to handle myself (I looked, but don't see how). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4321) SolrCloud collection CREATE will put multiple master shards on a single node (and no shards on some)
[ https://issues.apache.org/jira/browse/SOLR-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13558130#comment-13558130 ] Brett Hoerner commented on SOLR-4321: - I should note that the first collection created (by being in the solr.xml before the SolrCloud instances are brought up) is fine. I have a single collection called "default" in the config before I start my first node, and it starts with 1/2 shards and 0/2 replica (the first node). As each node is brought up for the first time it takes (in order), shard2, replica1, then replica2. I'd like to note I'm also having another problem that is only happening on 4.1 (not 4.0), and I'm starting to think it could be related: http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201301.mbox/browser > SolrCloud collection CREATE will put multiple master shards on a single node > (and no shards on some) > > > Key: SOLR-4321 > URL: https://issues.apache.org/jira/browse/SOLR-4321 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Affects Versions: 4.1 >Reporter: Brett Hoerner > > This is best described by a photo of my SolrCloud admin: > http://i.imgur.com/hW4xnxy.png > I have a 4 node cluster, with shard count of 2 and replication factor of 2. > After running something like, > > http://localhost:8983/solr/admin/collections?action=CREATE&name=15724&numShards=2&replicationFactor=2 > The shards seem to be completely randomly placed, which is fine, but some > nodes receive more of the share than others (some even receiving none). > For example, 15724 has given node "backfill-1d" 2 "parts", > 15724_shard1_replica2 and 15724_shard2_replica1 but has given "backfill-2d" > nothing at all. > This creates unnecessary load on some nodes, no? Is this something I am > supposed to handle myself (I looked, but don't see how). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
