You should use recursive functions for this kind of thing:
declare function local:while($test, $body) {
if($test()) then ($body(), local:while($test,$body)) else ()
};
let $tasks := ...
return
local:while(
function() { exists($tasks) },
function() {
submit-task(head($tasks)),
xdmp:set($tasks,tail($tasks))
}
)
MarkLogic's tail call optimization will mean that the local:while()
function will use a constant amount of stack space.
However in your specific example you really just want to execute a
function on each member of a sequence. In that specific case you can use
fn:map:
fn:map(submit-task#1,$tasks)
John
On 27/11/17 16:56, Eliot Kimber wrote:
> I have a client-server system where the client is spawning 100s of 1000s of
> jobs on the client. The client polls the servers to see when each server’s
> task queue is ready for more jobs. This all works fine.
>
> Logically this polling is a while-true() loop that will continue until either
> all the servers are offline or all the tasks to be submitted are consumed.
>
> In a procedural language this is trivial, but in XQuery 2 I’m not finding a
> way to do it that works. In XQuery 3 I could use the new iterate operator but
> that doesn’t seem to be available in MarkLogic 9.
>
> My first attempt was to use a recursive process, relying on tail recursion
> optimization to avoid blowing the stack buffer. That worked logically but I
> still ran into out-of-memory on the server at some point (around 200K jobs
> submitted) and it seems likely that it was runaway recursion doing it.
>
> So I tried using a simple loop with xdmp:set() to iterate over the tasks and
> use an exception to break out when all the tasks are done:
>
> try {
> for $i in 1 to 1000000 (: i.e., loop forever :)
> if (empty($tasks))
> then error()
> else submit-task(head($tasks))
> xdmp:set($tasks, tail($tasks))
> } catch ($e) {
> (: We’re done. (
> }
>
> Is there a better way to do this kind of looping forever?
>
> I’m also having a very strange behavior where in my new looping code I’m
> getting what I think must be a pending commit deadlock that I didn’t get in
> my recursive version of the code. I can trace the code to the xdmp:eval()
> that would commit an update to the task and that code never returns.
>
> Each task is a document that I update to reflect the details of the task’s
> status (start and end times, current processing status, etc.). Those updates
> are all done either in separately-run modules or via xdmp:eval(), so as far
> as I can tell there shouldn’t be any issues with uncommitted updates. I
> didn’t change anything in the logic that updates the task documents, only the
> loop that iterates over the tasks.
>
> Could it be that the use of xdmp:set() to modify the $tasks variable (a
> sequence of <task> elements) would be causing some kind of commit lock?
>
> Thanks,
>
> Eliot
>
> --
> Eliot Kimber
> http://contrext.com
>
>
>
>
> _______________________________________________
> General mailing list
> [email protected]
> Manage your subscription at:
> http://developer.marklogic.com/mailman/listinfo/general
--
John Snelson, Principal Engineer http://twitter.com/jpcs
MarkLogic Corporation http://www.marklogic.com
_______________________________________________
General mailing list
[email protected]
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
