Hi, Eliot: Have you considered a server field -- where any code that changes the status also updates the server field and the iterator checks the server field?
The server fields are local to the host, so there's no concern about a separate iterator running on a different host. If multiple iterators run on the same host, each would need to distinguish its status by an id, which the iterator could generate from a random id when it starts. Hoping that helps, Erik Hennum ________________________________________ From: [email protected] <[email protected]> on behalf of Eliot Kimber <[email protected]> Sent: Thursday, December 7, 2017 7:48:44 AM To: MarkLogic Developer Discussion Subject: [MarkLogic Dev General] Best Approach to Manage "Flags" That Might Change Within a Single Transaction In the context of my remote processing management system, where my client server is sending many tasks to a set of remote servers through a set of spawned tasks running in parallel, I need to be able to pause the client so that it stops sending new tasks to the remote servers. So far I've been using a single document stored in ML as my mechanism for indicating that a job is in progress and capturing the job details (job ID, start time, servers in use, etc.). This works fine because it was only updated at the start and end of the job. But for the pause/resume use case I need to have a flag that indicates that the job is paused and have other processes (e.g., my task-submission code) immediately respond to a change. For example, if I'm looping over 100 tasks to load up a remote task queue and the job is paused, I want that loop to end immediately. So basically, in this loop, for every iteration, check the "is paused" status, which requires reading the job doc to see if a @paused attribute is present (the @paused attribute captures the time the pause was requested and serves as the "is paused" flag). However, because the loop is a single transaction, it will see the same version of the job doc for every iteration, even if it's changed. I tried using xdmp:eval() to read the job doc but that didn't seem to change the behavior. E.g., doing this in query console: return (er:is-job-paused(), er:pause-job(), er:is-job-paused()) Results in (false, false) So this isn't going to work. So my question: what's the best way to manage this kind of dynamic flag in ML? I could use file system files instead of docs in the database, which would avoid the ML transaction behavior but that seems a little hackier than I'd like. What I'd really like is some kind of "shared memory" mechanism where I can set and reset variables at will across different modules running in parallel but I haven't seen anything like that in my study of the ML API. Is there such a mechanism that I've missed? Or am I just thinking about the problem the wrong way? Thanks, Eliot -- Eliot Kimber http://contrext.com _______________________________________________ General mailing list [email protected] Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
