Re: Job Distribution
Am 24.09.15 um 10:45 schrieb Stefan Egli: > Hi, > > There are various options here: you could distribute the 'queue chiefs' > via some hashing amongst all instances in the cluster view, and reassign > those chiefs upon a topology change. Or you could let the leader > explicitly send a 'become-chief' job/command to particular instances that > would then blush and proudly start as chief of a particular queue - and a > 'step-down-as-chief' command in case the leader detects it needs to > redistribute load. > Right, I guess a hash based selection is easier as it avoids any additional communication which then might require acks and so on. Regards Carsten -- Carsten Ziegeler Adobe Research Switzerland cziege...@apache.org
Re: Job Distribution
Hi, There are various options here: you could distribute the 'queue chiefs' via some hashing amongst all instances in the cluster view, and reassign those chiefs upon a topology change. Or you could let the leader explicitly send a 'become-chief' job/command to particular instances that would then blush and proudly start as chief of a particular queue - and a 'step-down-as-chief' command in case the leader detects it needs to redistribute load. Cheers, Stefan On 24/09/15 10:01, "Carsten Ziegeler" wrote: >Hi Julian, > >I think that makes sense, it just adds one indirection on determining >who will be processing ordered queues - but that shouldn't pose a problem. > >Regards >Carsten > >Am 24.09.15 um 09:53 schrieb Julian Sedding: >> Hi all >> >> I was looking into the following deployment scenario recently: >> >> - Oak Mongo with 4 attached, clustered Sling nodes >> - 2 nodes to be used to render the HTML UI >> - 2 nodes to be used for background processing >> >> The motivation was to remove background load from the rendering nodes >> in order to improve user experience (responsiveness etc). >> >> I got the load distribution working, however, there is one aspect I >> think could be improved. Ordered Job queues get processed on the >> leader instance. >> >> I would like ordered Jobs to be processed on one of the processing >> nodes. However, this cannot currently be achieved reliably, because I >> cannot control which node is the leader. Especially in the case of a >> failover situation. >> >> Instead of mandating a leader, I thought it may be better to let the >> leader nominate an "ordered-queue processing node". The leader could >> then assign this role to a node that signals its availability for that >> role. As a bonus, with such a setup, it may even be possible to >> distribute the processing of different ordered queues to differenet >> nodes. >> >> So in my scenario, I would configure the "processing" nodes to be >> available for processing ordered queues. I don't care any more which >> node is the leader, because the leader would assign jobs from ordered >> queues to one of the processing nodes. >> >> WDYT? Is this a realistic scenario, or have I got any misconceptions >> that would prevent such a setup? >> >> Regards >> Julian >> > > >-- >Carsten Ziegeler >Adobe Research Switzerland >cziege...@apache.org
Re: Job Distribution
Hi Julian, I think that makes sense, it just adds one indirection on determining who will be processing ordered queues - but that shouldn't pose a problem. Regards Carsten Am 24.09.15 um 09:53 schrieb Julian Sedding: > Hi all > > I was looking into the following deployment scenario recently: > > - Oak Mongo with 4 attached, clustered Sling nodes > - 2 nodes to be used to render the HTML UI > - 2 nodes to be used for background processing > > The motivation was to remove background load from the rendering nodes > in order to improve user experience (responsiveness etc). > > I got the load distribution working, however, there is one aspect I > think could be improved. Ordered Job queues get processed on the > leader instance. > > I would like ordered Jobs to be processed on one of the processing > nodes. However, this cannot currently be achieved reliably, because I > cannot control which node is the leader. Especially in the case of a > failover situation. > > Instead of mandating a leader, I thought it may be better to let the > leader nominate an "ordered-queue processing node". The leader could > then assign this role to a node that signals its availability for that > role. As a bonus, with such a setup, it may even be possible to > distribute the processing of different ordered queues to differenet > nodes. > > So in my scenario, I would configure the "processing" nodes to be > available for processing ordered queues. I don't care any more which > node is the leader, because the leader would assign jobs from ordered > queues to one of the processing nodes. > > WDYT? Is this a realistic scenario, or have I got any misconceptions > that would prevent such a setup? > > Regards > Julian > -- Carsten Ziegeler Adobe Research Switzerland cziege...@apache.org
Job Distribution
Hi all I was looking into the following deployment scenario recently: - Oak Mongo with 4 attached, clustered Sling nodes - 2 nodes to be used to render the HTML UI - 2 nodes to be used for background processing The motivation was to remove background load from the rendering nodes in order to improve user experience (responsiveness etc). I got the load distribution working, however, there is one aspect I think could be improved. Ordered Job queues get processed on the leader instance. I would like ordered Jobs to be processed on one of the processing nodes. However, this cannot currently be achieved reliably, because I cannot control which node is the leader. Especially in the case of a failover situation. Instead of mandating a leader, I thought it may be better to let the leader nominate an "ordered-queue processing node". The leader could then assign this role to a node that signals its availability for that role. As a bonus, with such a setup, it may even be possible to distribute the processing of different ordered queues to differenet nodes. So in my scenario, I would configure the "processing" nodes to be available for processing ordered queues. I don't care any more which node is the leader, because the leader would assign jobs from ordered queues to one of the processing nodes. WDYT? Is this a realistic scenario, or have I got any misconceptions that would prevent such a setup? Regards Julian
