[
https://issues.apache.org/jira/browse/CONNECTORS-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13823616#comment-13823616
]
Graeme Seaton commented on CONNECTORS-781:
------------------------------------------
Hi Karl,
Will the connection allocation's need to be job-based and sticky?
If we have resource A with 10 available connections and resource B with 5 and
10 nodes, then we potentially have only 5 available nodes which are able to
process jobs for that combination (I've simplified the issue of resourcing
freeing for the moment). If all work is then allocated across the nodes based
on those that have the appropriate resource combinations in a round-robin
manner, would this simplify things? (Probably at the cost of making
job/task-allocation more problematic?)
> Fault-Tolerant Setup for ManifoldCF Agent.
> ------------------------------------------
>
> Key: CONNECTORS-781
> URL: https://issues.apache.org/jira/browse/CONNECTORS-781
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Framework agents process, Framework core, Framework
> crawler agent
> Affects Versions: ManifoldCF 1.5
> Reporter: Swami Rajamohan
> Assignee: Karl Wright
> Labels: agents, crawler, fault-tolerance
> Fix For: ManifoldCF 1.5
>
>
> It should be possible to setup ManifoldCF as a Fault-Tolerant infrastructure.
> The Agent component of ManifoldCF should support multiple instances of an
> agent crawling against a single crawl store, to be able to both distribute
> (share) the crawl load as well as to be able to pick up a request that gets
> abruptly terminated due to either partitioning of the instance/failure of the
> instance itself.
> Since there is a proposal to move to a store like Voldemort, it would be nice
> to be able to have a fault tolerant infrastructure.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
