[
https://issues.apache.org/jira/browse/SOLR-3488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295689#comment-13295689
]
Mark Miller commented on SOLR-3488:
-----------------------------------
I've got my first patch ready - still some things to address, but it currently
does queue based collection creation.
One thing I recently realized when I put some last minute pieces together is
that I cannot share the same Overseer queue that already exists - it will cause
a deadlock as we wait for states to be registered. Processing the queue with
multiple threads still seemed scary if you were to create a lot of collections
at once - so it seems just safer to use a different queue.
I'm still somewhat unsure about handing failures though - for the moment I'm
simply adding the job back onto the queue - this gets complicated quickly
though. Especially if you add in delete collection and it can fail. If you
start putting commands back on the queue you could have weird create/delete
command retry reordering?
I also have not switched to requiring or respecting a replication factor - I
was thinking perhaps specifying nothing or -1 would give you what you have now?
An infinite rep factor? And we would enforce a lower rep factor if requested?
For now I still require that you pass a collection template and new nodes are
created on the nodes that host the collection template.
I'm not sure how replication factor would be enforced though? The Oveerseer
just periodically prunes and adds given what it sees and what the rep factor
is? Is that how failures should be handled? Don't readd to the queue, just let
the periodic job attempt to fix things later?
What if someone starts a new node with a new replicas pre configured in
solr.xml? The Overseer periodic job would simply remove them shortly thereafter
if the rep factor was not high enough?
One issue with pruning at the moment is that unloading a core will not remove
it's data dir. We probably want to fix that for collection removal.
If we go too far down this path, it seems rebalancing starts to become
important as well.
I've got some other thoughts and ideas to get down, but that is a start so I
can gather some feedback.
I have not yet integrated Tomasso's work, but will if we don't end up changing
things much from now.
> Create a Collections API for SolrCloud
> --------------------------------------
>
> Key: SOLR-3488
> URL: https://issues.apache.org/jira/browse/SOLR-3488
> Project: Solr
> Issue Type: New Feature
> Components: SolrCloud
> Reporter: Mark Miller
> Assignee: Mark Miller
> Attachments: SOLR-3488.patch, SOLR-3488.patch, SOLR-3488_2.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
