[jira] [Comment Edited] (SOLR-5311) Avoid registering replicas which are removed

2013-12-30 Thread Noble Paul (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-5311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13858870#comment-13858870
 ] 

Noble Paul edited comment on SOLR-5311 at 12/30/13 3:30 PM:


People managing the clusterstate explicitly is not really a requirement.

They just need to create cores and the system should automatically assign 
coreNodeName .





was (Author: noble.paul):
People managing the clusterstate explicitly is not really a requirement.

They just need to create cores and the system should automatically assign 
coreNodeName .



 Avoid registering replicas which are removed 
 -

 Key: SOLR-5311
 URL: https://issues.apache.org/jira/browse/SOLR-5311
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Reporter: Noble Paul
Assignee: Noble Paul
 Fix For: 4.6, 5.0

 Attachments: SOLR-5311.patch, SOLR-5311.patch, SOLR-5311.patch, 
 SOLR-5311.patch, SOLR-5311.patch, SOLR-5311.patch, SOLR-5311.patch


 If a replica is removed from the clusterstate and if it comes back up it 
 should not be allowed to register. 
 Each core ,when comes up, checks if it was already registered and if yes is 
 it still there. If not ,throw an error and do an unregister . If such a 
 request come to overseer it should ignore such a core



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Comment Edited] (SOLR-5311) Avoid registering replicas which are removed

2013-12-30 Thread Noble Paul (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-5311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13858935#comment-13858935
 ] 

Noble Paul edited comment on SOLR-5311 at 12/30/13 5:18 PM:


Actually it was an undocumented feature. 
https://cwiki.apache.org/confluence/display/solr/CoreAdminHandler+Parameters+and+Usage#CoreAdminHandlerParametersandUsage-{{CREATE}}




was (Author: noble.paul):
Actually it was an undocumented feature. 
https://cwiki.apache.org/confluence/display/solr/CoreAdminHandler+Parameters+and+Usage



 Avoid registering replicas which are removed 
 -

 Key: SOLR-5311
 URL: https://issues.apache.org/jira/browse/SOLR-5311
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Reporter: Noble Paul
Assignee: Noble Paul
 Fix For: 4.6, 5.0

 Attachments: SOLR-5311.patch, SOLR-5311.patch, SOLR-5311.patch, 
 SOLR-5311.patch, SOLR-5311.patch, SOLR-5311.patch, SOLR-5311.patch


 If a replica is removed from the clusterstate and if it comes back up it 
 should not be allowed to register. 
 Each core ,when comes up, checks if it was already registered and if yes is 
 it still there. If not ,throw an error and do an unregister . If such a 
 request come to overseer it should ignore such a core



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Comment Edited] (SOLR-5311) Avoid registering replicas which are removed

2013-12-30 Thread Noble Paul (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-5311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13858958#comment-13858958
 ] 

Noble Paul edited comment on SOLR-5311 at 12/30/13 6:04 PM:


I'm not sure I understand you. I'm missing something I guess 
Thisproperty is persisted to solr.xml or core.properties . And that is how it 
works. Without that it can't work . I made the change so that it is persisted 
to solr.xml/core.properties 


was (Author: noble.paul):
I'm not sure I understand you. I'm missing something I guess 
This property is a property is persisted to solr.xml or core.properties . And 
that is how it works. Without that it can't work . I made the change so that it 
is persisted to solr.xml/core.properties 

 Avoid registering replicas which are removed 
 -

 Key: SOLR-5311
 URL: https://issues.apache.org/jira/browse/SOLR-5311
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Reporter: Noble Paul
Assignee: Noble Paul
 Fix For: 4.6, 5.0

 Attachments: SOLR-5311.patch, SOLR-5311.patch, SOLR-5311.patch, 
 SOLR-5311.patch, SOLR-5311.patch, SOLR-5311.patch, SOLR-5311.patch


 If a replica is removed from the clusterstate and if it comes back up it 
 should not be allowed to register. 
 Each core ,when comes up, checks if it was already registered and if yes is 
 it still there. If not ,throw an error and do an unregister . If such a 
 request come to overseer it should ignore such a core



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Comment Edited] (SOLR-5311) Avoid registering replicas which are removed

2013-12-30 Thread Mark Miller (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-5311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13859292#comment-13859292
 ] 

Mark Miller edited comment on SOLR-5311 at 12/31/13 3:42 AM:
-

This is part of a larger direction we have been working towards, which is 
essentially making ZooKeeper the truth.

With the current SolrCloud, you cannot do this like you have. The Core Admin 
API, and pre configured cores, and the collections API all need to work in 
concert. That makes this approach a complete no go right now.

The path I have been working towards with the Collections API is a new mode 
where everything is handled by the Collections API. In this case, it will not 
be valid for users to try and mess with things at a per core level.

In this way, the cluster can truly match the truth in ZooKeeper because both 
the nodes and the Overseer can work together to keep the truth enforced. This 
includes things like being able to easily change the replication factor for a 
collection, or add a new host that automatically gets used depending on your 
settings. You should not have to address individual nodes to manage 
collections, nor should your replicationFactor stop mattering simply because 
you added another core via core admin. To me, this is the ultimate cloud 
situation. The system needs full control. We can add the ability to override or 
prefer certain things, but in general, we want to get to the point of 
optionally have the cluster mostly managed for you given some simple 
directectives. Of course, I think it should be implemented as a bunch of option 
features. You should also be easy to really lock things done unless you manage 
things manually.

All of this requires we have a mode a user can decide to use (the collections 
api, perhaps with an option for back compat) so that we are in control of 
everything. We know when a collection is created and deleted - it won't be able 
to just pop back into existence. 

Until we have this special mode, the way that we had to build this, lots of 
historical reasons, we currently have to support what we have with pre 
configured cores and core admin and the collections api. This is silly form a 
user perspective though. It can all be done much nicer with just a collections 
API that doesn't have to be directed to any single node. 

Doing what you want to do in a back compat way is not some simple fix. We have 
been working towards this for a long time now - if you could just slap in a 
band aid and make it work like this, I would have done it a long time go.


was (Author: markrmil...@gmail.com):
This is part of a larger direction we have been working towards, which is 
essentially making ZooKeeper the truth.

With the current SolrCloud, you cannot do this like you have. The Core Admin 
API, and pre configured cores, and the collections API all need to work in 
concert. That makes this approach a complete no go right now.

The path I have been working towards with the Collections API is a new mode 
where everything is handled by the Collections API. In this case, it will not 
be valid for users to try and mess with things at a per core level.

In this way, the cluster can truly match the truth in ZooKeeper because both 
the nodes and the Overseer can work together to keep the truth enforced. This 
includes things like being able to easily change the replication factor for a 
collection, or a new host that automatically gets used depending on your 
settings. You should not have to address individual nodes to manage 
collections, nor should your replicationFactor stop mattering simply because 
you added another core via core admin. To me, this is the ultimate cloud 
situation. The system needs full control. We can add the ability to override or 
prefer certain things, but in general, we want to get to the point of 
optionally have the cluster mostly managed for you given some simple 
directectives. Of course, I think it should be implemented as a bunch of option 
features. You should also be easy to really lock things done unless you manage 
things manually.

All of this requires we have a mode a user can decide to use (the collections 
api, perhaps with an option for back compat) so that we are in control of 
everything. We know when a collection is created and deleted - it won't be able 
to just pop back into existence. 

Until we have this special mode, the way that we had to build this, lots of 
historical reasons, we currently have to support what we have with pre 
configured cores and core admin and the collections api. This is silly form a 
user perspective though. It can all be done much nicer with just a collections 
API that doesn't have to be directed to any single node. 

Doing what you want to do in a back compat way is not some simple fix. We have 
been working towards this for a long time now - if you could just slap in a 
band aid and 

[jira] [Comment Edited] (SOLR-5311) Avoid registering replicas which are removed

2013-10-17 Thread Noble Paul (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-5311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13797836#comment-13797836
 ] 

Noble Paul edited comment on SOLR-5311 at 10/17/13 12:25 PM:
-

all tests pass in trunk. please verify. this is a combined patch for SOLR-5310 
too


was (Author: noble.paul):
all tests pass in trunk. please verify

 Avoid registering replicas which are removed 
 -

 Key: SOLR-5311
 URL: https://issues.apache.org/jira/browse/SOLR-5311
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Reporter: Noble Paul
Assignee: Noble Paul
 Attachments: SOLR-5311.patch, SOLR-5311.patch, SOLR-5311.patch, 
 SOLR-5311.patch


 If a replica is removed from the clusterstate and if it comes back up it 
 should not be allowed to register. Overseer should send an error back to the 
 node saying it is not a member. The cores corresponding to that replica aslo 
 should be cleaned up



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org