[
https://issues.apache.org/jira/browse/SOLR-5477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881093#comment-13881093
]
Anshum Gupta commented on SOLR-5477:
------------------------------------
I had a discussion with Noble (offline) and we thought that holding this stuff
for persistence in zk didn't make much sense. Here are the reasons:
* Every request would need to de-duplicate against the
submitted/completed/failed tasks and zk queues aren't fit for that. Every dedup
would translate to fetching all children and running a compare or something.
With all cores trying to use the same zk setup, not sure if it's even required.
In memory hashes would work better at handling this use case.
* We may not be interested in persistence of results over a longer duration
i.e. if the node goes down, we should be fine with losing the request
information as in general, the time taken to bring the same node back up would
be more than someone else doing the job in the meanwhile (fair assumption?).
> Async execution of OverseerCollectionProcessor tasks
> ----------------------------------------------------
>
> Key: SOLR-5477
> URL: https://issues.apache.org/jira/browse/SOLR-5477
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
> Reporter: Noble Paul
> Assignee: Anshum Gupta
> Attachments: SOLR-5477-CoreAdminStatus.patch, SOLR-5477.patch,
> SOLR-5477.patch, SOLR-5477.patch, SOLR-5477.patch
>
>
> Typical collection admin commands are long running and it is very common to
> have the requests get timed out. It is more of a problem if the cluster is
> very large.Add an option to run these commands asynchronously
> add an extra param async=true for all collection commands
> the task is written to ZK and the caller is returned a task id.
> as separate collection admin command will be added to poll the status of the
> task
> command=status&id=7657668909
> if id is not passed all running async tasks should be listed
> A separate queue is created to store in-process tasks . After the tasks are
> completed the queue entry is removed. OverSeerColectionProcessor will perform
> these tasks in multiple threads
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
