While it does create a separate process I run my batch jobs with Jenkins. The jobs are just pages so you don't have weird lifecycle problems and they are easy to test because you can just go to the page. Your load balancer will distribute the load, you get history and you don't need to write any code. Just drop the war file into your server.
On Thu, Jun 29, 2017 at 1:18 AM, Ilya Obshadko <ilya.obsha...@gmail.com> wrote: > Yes, my case is about the same logical queue. I currently don’t have any > abstraction for “units of work” or whatever you may call it - application > didn’t need it when it was running on a single machine. I’ll have to > introduce something similar to isolate jobs from actual business objects. > > > On Wed, Jun 28, 2017 at 11:24 PM, Dmitry Gusev <dmitry.gu...@gmail.com> > wrote: > > > Not sure I understand where those pessimistic locks came from. > > > > In out case there's no locking at all, every machine in a cluster > processes > > jobs simultaneously, unless, of course, the jobs are not from the same > > logical queue and must be executed in order. > > > > By row-level locking I mean PostgreSQL's SELECT ... FOR UPDATE, i.e.: > > > > UPDATE units_of_work > > SET started_at = ? > > WHERE id = (SELECT id > > FROM units_of_work > > WHERE started_at IS NULL > > LIMIT 1 > > FOR UPDATE) > > RETURNING id > > > > This is a simplified version of what's actually happening, but > illustrates > > the idea: different coordinators don't lock each other. > > > > > > On Wed, Jun 28, 2017 at 11:05 PM, Ilya Obshadko <ilya.obsha...@gmail.com > > > > wrote: > > > > > I was actually looking at Spring Batch (and a couple of other > > solutions). I > > > don’t think Spring Batch could be of much help here. > > > > > > My conclusion is similar to what you are saying - implementing > > lightweight > > > job coordinator is much easier. > > > > > > Row-level locking works well when you are dealing with a simple queue > > table > > > - you do a pessimistic lock on N rows, process them and give a chance > to > > > another host in the cluster. Unfortunately only one of my background > jobs > > > is suitable for this type of refactoring. > > > > > > Other jobs process records that shouldn’t be locked for a considerable > > > amount of time. > > > > > > So currently I’m thinking of the following scenario: > > > > > > - pass deployment ID via environment to all containers (ECS can do this > > > quite easily) > > > - use a simple table with records containing job name, current cluster > > > deployment ID and state > > > - first background executor that is able to lock an appropriate job row > > > starts working, the other(s) are cancelled > > > > > > > > > > > > On Tue, Jun 27, 2017 at 10:16 PM, Dmitry Gusev <dmitry.gu...@gmail.com > > > > > wrote: > > > > > > > Hi Ilya, > > > > > > > > If you have Spring in your classpath you may look at Spring Batch. > > > > > > > > For our projects we've built something similar -- a custom jobs > > framework > > > > on top of PostgreSQL. > > > > > > > > The idea is that there a coordinator service (Tapestry service) that > > runs > > > > in a thread pool and constantly polls special DB tables for new > > records. > > > > For every new unit of work it creates instance of a worker (using > > > > `ObjectLocator.autobuild()`) that's capable of processing the job. > > > > > > > > The polling can be optimised well for performance using row-level > > locks & > > > > DB indexing. > > > > > > > > Coordinator runs in the same JVM as the rest of the app so there's no > > > > dedicated process. > > > > It integrates with tapestry's EntityManager so that you could create > a > > > job > > > > in transaction. > > > > > > > > When running in a cluster every JVM has its own coordinator -- this > it > > > how > > > > the jobs get distributed. > > > > > > > > But you're saying that row-level locking doesn't work for some of > your > > > > use-cases, can you be more concrete here? > > > > > > > > > > > > On Tue, Jun 27, 2017 at 9:35 PM, Ilya Obshadko < > > ilya.obsha...@gmail.com> > > > > wrote: > > > > > > > > > I’ve recently expanded my Tapestry application to run multiple > hosts. > > > > While > > > > > it’s quite OK for the web-faced part (sticky load balancer does > most > > of > > > > the > > > > > job), it’s not very straightforward with background jobs. > > > > > > > > > > Some of them can be quite easily distributed using database > row-level > > > > > locks, but this doesn’t work for every use case I have. > > > > > > > > > > Are there any suggestions about this? I’d prefer not to have a > > > dedicated > > > > > process running background tasks. Ideally, I want to dynamically > > > > distribute > > > > > background jobs between hosts in cluster, based on current load > > status. > > > > > > > > > > > > > > > -- > > > > > Ilya Obshadko > > > > > > > > > > > > > > > > > > > > > -- > > > > Dmitry Gusev > > > > > > > > AnjLab Team > > > > http://anjlab.com > > > > > > > > > > > > > > > > -- > > > Ilya Obshadko > > > > > > > > > > > -- > > Dmitry Gusev > > > > AnjLab Team > > http://anjlab.com > > > > > > -- > Ilya Obshadko >