Thanks again Geert for your insightful ideas and suggestions! Tim
-----Original Message----- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Geert Josten Sent: Wednesday, July 28, 2010 6:20 AM To: General Mark Logic Developer Discussion Subject: Re: [MarkLogic Dev General] Prioritizingentries inthetaskserverqueue Hi Tim, > Using scheduled jobs is an interesting idea and has a lot of > merit. What are you thinking about using for suspending > processing until the high priority directory is empty? Are > you talking about waking the low priority job on a scheduled > basis and just having it do nothing if high priority > directory entries exist? I'd say the scheduled job would do the spawning for you, but simply skip until the high prio directory is empty, perhaps just logging an entry mentioning 'Waiting for high prio tasks to finish' or something like that. No real suspending or waking of tasks or jobs, which is not really possible. > I would like to be able to instantaneously kick off servicing > of a document whenever the CPF completes instead of waiting > for a scheduled job to perform the task. I don't want to > limit document processing to the minimum job scheduling > interval of what appears to be one minute. I think what > could work would be a series of priority tasks that can be > spawned either by a CPF event or a if necessary a scheduled > job as back-up, but to prevent multiple tasks from running > for the same priority I need some kind of tail recursion > semaphore to indicate if that particular priority task is > already running - perhaps using a locking mechanism on a > document uniquely created for each given priority and > releasing the lock when the priority task finds no more > documents in its queue. Tail-recusion is not a very ideal approach to work around your priority problems. I did it as an experiment to see if I could lower the task load, in which I was also doing bulk loads, which weren't triggered by CPF events. If you don't want a CPF task to be cued automatically the only ways to prevent that is to delay the actual insert/update that triggers the event (you could do some sleeping, but that blocks a thread I believe), or insert/update outside the domain/scope of CPF. Limit job scheduling: good point, but you can spawn multiple tasks from one job if you like, proportional to the number of available threads. You could also create a wrapper if you like, that does the tail-recursion you want, continue to spawn the next task once the first reaches the end, only terminating recursion when it runs out of tasks or perhaps until it detects higher prio tasks. What you need for this tail-recusion is a main module that takes some external vars the tell it which folders to scan, a small sleeping time interval perhaps. It should call (not spawn) a function that executes the task at hand, and finish with spawning a call to itself. There will be a small overlap between the running task and spawn to itself.. I think it is best to rely on job schedule to initiate the recursion, but you indeed need to detect already running recursions somehow. Acquiring an exclusive non-root lock might work, you'll have to try.. Kind regards, Geert drs. G.P.H. (Geert) Josten Consultant Daidalos BV Hoekeindsehof 1-4 2665 JZ Bleiswijk T +31 (0)10 850 1200 F +31 (0)10 850 1199 mailto:geert.jos...@daidalos.nl http://www.daidalos.nl/ KvK 27164984 De informatie - verzonden in of met dit e-mailbericht - is afkomstig van Daidalos BV en is uitsluitend bestemd voor de geadresseerde. Indien u dit bericht onbedoeld hebt ontvangen, verzoeken wij u het te verwijderen. Aan dit bericht kunnen geen rechten worden ontleend. _______________________________________________ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
