Hi, There are various options here: you could distribute the 'queue chiefs' via some hashing amongst all instances in the cluster view, and reassign those chiefs upon a topology change. Or you could let the leader explicitly send a 'become-chief' job/command to particular instances that would then blush and proudly start as chief of a particular queue - and a 'step-down-as-chief' command in case the leader detects it needs to redistribute load.
Cheers, Stefan On 24/09/15 10:01, "Carsten Ziegeler" <cziege...@apache.org> wrote: >Hi Julian, > >I think that makes sense, it just adds one indirection on determining >who will be processing ordered queues - but that shouldn't pose a problem. > >Regards >Carsten > >Am 24.09.15 um 09:53 schrieb Julian Sedding: >> Hi all >> >> I was looking into the following deployment scenario recently: >> >> - Oak Mongo with 4 attached, clustered Sling nodes >> - 2 nodes to be used to render the HTML UI >> - 2 nodes to be used for background processing >> >> The motivation was to remove background load from the rendering nodes >> in order to improve user experience (responsiveness etc). >> >> I got the load distribution working, however, there is one aspect I >> think could be improved. Ordered Job queues get processed on the >> leader instance. >> >> I would like ordered Jobs to be processed on one of the processing >> nodes. However, this cannot currently be achieved reliably, because I >> cannot control which node is the leader. Especially in the case of a >> failover situation. >> >> Instead of mandating a leader, I thought it may be better to let the >> leader nominate an "ordered-queue processing node". The leader could >> then assign this role to a node that signals its availability for that >> role. As a bonus, with such a setup, it may even be possible to >> distribute the processing of different ordered queues to differenet >> nodes. >> >> So in my scenario, I would configure the "processing" nodes to be >> available for processing ordered queues. I don't care any more which >> node is the leader, because the leader would assign jobs from ordered >> queues to one of the processing nodes. >> >> WDYT? Is this a realistic scenario, or have I got any misconceptions >> that would prevent such a setup? >> >> Regards >> Julian >> > > >-- >Carsten Ziegeler >Adobe Research Switzerland >cziege...@apache.org
