[ https://issues.apache.org/jira/browse/OAK-2844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stefan Egli reassigned OAK-2844: -------------------------------- Assignee: Stefan Egli > Introducing a simple document-based discovery-light service (to circumvent > documentMk's eventual consistency delays) > -------------------------------------------------------------------------------------------------------------------- > > Key: OAK-2844 > URL: https://issues.apache.org/jira/browse/OAK-2844 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: mongomk > Reporter: Stefan Egli > Assignee: Stefan Egli > Labels: resilience > Fix For: 1.4 > > Attachments: InstanceStateChangeListener.java, OAK-2844.WIP-02.patch, > OAK-2844.patch, OAK-2844.v3.patch > > > When running discovery.impl on a mongoMk-backed jcr repository, there are > risks of hitting problems such as described in "SLING-3432 > pseudo-network-partitioning": this happens when a jcr-level heartbeat does > not reach peers within the configured heartbeat timeout - it then treats that > affected instance as dead, removes it from the topology, and continues with > the remainings, potentially electing a new leader, running the risk of > duplicate leaders. This happens when delays in mongoMk grow larger than the > (configured) heartbeat timeout. These problems ultimately are due to the > 'eventual consistency' nature of, not only mongoDB, but more so of mongoMk. > The only alternative so far is to increase the heartbeat timeout to match the > expected or measured delays that mongoMk can produce (under say given > load/performance scenarios). > Assuming that mongoMk will always carry a risk of certain delays and a > maximum, reasonable (for discovery.impl timeout that is) maximum cannot be > guaranteed, a better solution is to provide discovery with more 'real-time' > like information and/or privileged access to mongoDb. > Here's a summary of alternatives that have so far been floating around as a > solution to circumvent eventual consistency: > # expose existing (jmx) information about active 'clusterIds' - this has > been proposed in SLING-4603. The pros: reuse of existing functionality. The > cons: going via jmx, binding of exposed functionality as 'to be maintained > API' > # expose a plain mongo db/collection (via osgi injection) such that a higher > (sling) level discovery could directly write heartbeats there. The pros: > heartbeat latency would be minimal (assuming the collection is not sharded). > The cons: exposes a mongo db/collection potentially also to anyone else, with > the risk of opening up to unwanted possibilities > # introduce a simple 'discovery-light' API to oak which solely provides > information about which instances are active in a cluster. The implementation > of this is not exposed. The pros: no need to expose a mongoDb/collection, > allows any other jmx-functionality to remain unchanged. The cons: a new API > that must be maintained > This ticket is about the 3rd option, about a new mongo-based discovery-light > service that is introduced to oak. The functionality in short: > * it defines a 'local instance id' that is non-persisted, ie can change at > each bundle activation. > * it defines a 'view id' that uniquely identifies a particular incarnation > of a 'cluster view/state' (which is: a list of active instance ids) > * and it defines a list of active instance ids > * the above attributes are passed to interested components via a listener > that can be registered. that listener is called whenever the discovery-light > notices the cluster view has changed. > While the actual implementation could in fact be based on the existing > {{getActiveClusterNodes()}} {{getClusterId()}} of the > {{DocumentNodeStoreMBean}}, the suggestion is to not fiddle with that part, > as that has dependencies to other logic. But instead, the suggestion is to > create a dedicated, other, collection ('discovery') where heartbeats as well > as the currentView are stored. > Will attach a suggestion for an initial version of this for review. -- This message was sent by Atlassian JIRA (v6.3.4#6332)