In the context of my remote processing management system, where my client
server is sending many tasks to a set of remote servers through a set of
spawned tasks running in parallel, I need to be able to pause the client so
that it stops sending new tasks to the remote servers.
So far I've been using a single document stored in ML as my mechanism for
indicating that a job is in progress and capturing the job details (job ID,
start time, servers in use, etc.). This works fine because it was only updated
at the start and end of the job.
But for the pause/resume use case I need to have a flag that indicates that the
job is paused and have other processes (e.g., my task-submission code)
immediately respond to a change. For example, if I'm looping over 100 tasks to
load up a remote task queue and the job is paused, I want that loop to end
immediately.
So basically, in this loop, for every iteration, check the "is paused" status,
which requires reading the job doc to see if a @paused attribute is present
(the @paused attribute captures the time the pause was requested and serves as
the "is paused" flag). However, because the loop is a single transaction, it
will see the same version of the job doc for every iteration, even if it's
changed.
I tried using xdmp:eval() to read the job doc but that didn't seem to change
the behavior.
E.g., doing this in query console:
return (er:is-job-paused(), er:pause-job(), er:is-job-paused())
Results in (false, false)
So this isn't going to work.
So my question: what's the best way to manage this kind of dynamic flag in ML?
I could use file system files instead of docs in the database, which would
avoid the ML transaction behavior but that seems a little hackier than I'd like.
What I'd really like is some kind of "shared memory" mechanism where I can set
and reset variables at will across different modules running in parallel but I
haven't seen anything like that in my study of the ML API.
Is there such a mechanism that I've missed?
Or am I just thinking about the problem the wrong way?
Thanks,
Eliot
--
Eliot Kimber
http://contrext.com
_______________________________________________
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
