Michael, I'll have a go to further investigate "soon" (on holiday for a week). Thanks for all the useful suggestions!
Does it make sense to actually increase the paging file to 2x the memory size (instead of the current 1.5x), or is this just some rule of thumb and doesn't really matter? thanks, Jakob. On Fri, Aug 14, 2009 at 18:54, Michael Blakeley<[email protected]> wrote: > Jakob, > > On (2), are you sure you've reduced the right in-memory limits? The error > message says it's trying to allocate 128-MB. If it was 256-MB, and chopping > it to 128-MB didn't help... keep chopping it in half. But if that > '134217728' number isn't changing when you change the config, then you are > probably looking at the wrong configuration parameter. > > Also, MarkLogic Server is not PAE-aware, so it won't ever see 4-GB on a > 32-GB OS. The best you can do is to enabled the /3GB switch, for a 3-GB > process space. > > Finally, I don't use Windows every day, but it seems to recycle address > spaces sometimes. So I've sometimes been forced to reboot the OS to make > SVC-MEMALLOC go away. > > Life is much easier with a 64-bit OS. > > On (1), it's possible that the TaskServer isn't looking at the right > database. The spawned task should default to the database from which it was > spawned, but you could try adding a 'database' child element to the options > node to make sure you're in the right database. Or you could call > xdmp:log(xdmp:database-name(xdmp:database())) to find out what database the > module spawn is using. > > Another possibility is that the range index on DOI exists, but doesn't match > the collation of the DOI string, or doesn't match the data type of the > supplied DOI value. You could log the output of fn:default-collation() to > check the string collation, and/or use xdmp:describe() to check the data > type. > > -- Mike > > On 2009-08-13 13:40, Jakob Fix wrote: >> >> Well, things don't get any easier or clearer ... >> >> 1) I now get error messages like this one when running the scheduled >> task as described in previous messages: >> >> 2009-08-12 22:21:00.458 Notice: TaskServer: XDMP-ELEMRIDXNOTFOUND: >> cts:element-values(QName("", "DOI")) -- No element range index for >> QName("", "DOI") >> 2009-08-12 22:21:00.458 Notice: TaskServer: in >> /app/backend/update.xqy, on line 21 [1.0-ml] >> >> bizarrely enough, running the same task (either by spawning it from >> cq, or be simply calling the page in a web browser or with curl), >> everything works fine, and no such error message is reported. Also, >> this doesn't seem to be reproducible. the element range index exists, >> of course. >> >> >> 2) When invoking my module which spawns many tasks (and I don't get >> the error above), either from cq or a scheduled task I also get the >> memory allocation errors like this one (one for each spawn, I'd think, >> but I haven't counted): >> >> 2009-08-10 23:05:14.619 Notice: TaskServer: SVC-MEMALLOC: Memory >> allocation error: VirtualAlloc 134217728 bytes: Not enough storage is >> available to process this command. >> 2009-08-10 23:05:14.619 Notice: TaskServer: >> 2009-08-10 23:05:14.619 Notice: TaskServer: in >> /app/backend/query-doi.xqy [1.0-ml] >> 2009-08-10 23:05:15.291 Notice: TaskServer: SVC-MEMALLOC: Memory >> allocation error: VirtualAlloc 134217728 bytes: Not enough storage is >> available to process this command. >> >> Note that calling the xquery script as a web page in a browser or curl >> does not create these errors. >> >> I carefully read the hints and ideas suggested by Michael Blakeley in >> his response to a similar problem here: >> http://markmail.org/message/xgpdbv3qm62doxkv >> >> Yes, I'm on 32-bit Windows (Server 2003 SP2), not my choice. Windows >> sees all 4GB of RAM because of PAE. The paging file is 6142MB (1.5x of >> the available RAM, not 2x as recommended) on the C: drive. Reducing >> list size limit and tree size limit by 50% does not decrease the >> number of errors. MarkLogic service was restarted but the errors >> persist. >> >> Could it be that are still some teething problems with the scheduled >> tasks? I am wondering because of the differences between the problems >> I've encountered using a scheduled task and when I call my script >> externally. At the moment, I use Windows' scheduled task feature, but >> I would like to use the MarkLogic alternative if possible. >> >> cheers, >> Jakob. >> >> >> >> On Tue, Aug 11, 2009 at 00:58, Jakob Fix<[email protected]> wrote: >>> >>> Danny, >>> >>> thanks. I've followed your example. Adapting it slightly as to use my >>> doistats-database, but keeping your sample script, I get the same >>> results as you do, same results for the manual spawn in cq. >>> >>> 2009-08-11 00:40:00.739 Info: TaskServer: hello >>> 2009-08-11 00:42:48.269 Info: TaskServer: hello >>> >>> I've just spawned my own task in cq and it works, the queue of the >>> task server is full. I've recreated the task to run in 10 minutes' >>> time (i.e. at one o'clock in the morning), we'll see what that does. >>> >>> Maybe I should add at this point that the update.xqy spawns itself a >>> big number of tasks, like this: >>> >>> for $doi in cts:element-values(xs:QName("DOI")) >>> return >>> <div> >>> { >>> try { >>> xdmp:spawn( >>> "/app/backend/query-doi.xqy", >>> (xs:QName("doi"), $doi, >>> xs:QName("date"), $date) (: corresponds to >>> fn:current-date() :) >>> ) >>> } catch($e) { >>> <i xmlns:e="http://marklogic.com/xdmp/error";>OOPS {$doi}: >>> {$e/e:message/text()}</i> >>> } >>> } >>> </div> >>> >>> >>> thanks for your help so far, >>> Jakob. >>> >>> >>> >>> >>> On Mon, Aug 10, 2009 at 23:49, Danny >>> Sokolsky<[email protected]> wrote: >>>> >>>> Hi Jakob, >>>> >>>> The root is just the root from which to evaluate the path. This is just >>>> like the root option to xdmp:spawn. The path is then relative to the root. >>>> So if your root is /foo/ and your path is hello.xqy, then it will resolve >>>> to /foo/hello.xqy. >>>> >>>> It looks like things are configured correctly in your system, but it is >>>> easy to accidentally overlook something. Is it possible you added that >>>> module to a different database? >>>> >>>> Here is what I tried successfully: >>>> >>>> 1) create a text file that is an xquery main module. I created mine in >>>> the database names "Modules": >>>> >>>> xdmp:document-insert("/test/test.xqy", text {"xdmp:log('hello')"}) >>>> >>>> 2) configure a scheduled task. Mine has these setting: >>>> >>>> task path /test/test.xqy >>>> task root / >>>> task type daily >>>> task period 1 >>>> task start time 2:12 PM >>>> task database Modules >>>> task modules Modules >>>> task user admin >>>> task host server-name.marklogic.com >>>> >>>> At 2:12 I saw the following in my ErrorLog.txt: >>>> >>>> 2009-08-10 14:12:00.532 Info: TaskServer: hello >>>> >>>> 3) Test my module in cq as follows: >>>> >>>> xquery version "1.0-ml"; >>>> xdmp:spawn("/test/test.xqy", (), >>>> <options xmlns="xdmp:eval"> >>>> <modules>{xdmp:database("Modules")}</modules> >>>> <root>/</root> >>>> </options>) >>>> >>>> I see the following in my log: >>>> >>>> 2009-08-10 14:18:41.499 Info: TaskServer: hello >>>> >>>> Maybe if you can try something like that, then compare the process with >>>> what you are doing, it will help find the solution? >>>> >>>> -Danny >>>> >>>> >>>> -----Original Message----- >>>> From: [email protected] >>>> [mailto:[email protected]] On Behalf Of Jakob Fix >>>> Sent: Monday, August 10, 2009 1:54 PM >>>> To: General Mark Logic Developer Discussion >>>> Subject: Re: [MarkLogic Dev General] scheduled task (task root?) >>>> >>>> Hi Danny, >>>> >>>> thanks for answering. please see my comments inline. >>>> >>>> On Mon, Aug 10, 2009 at 19:03, Danny Sokolsky >>>> <[email protected]> wrote: >>>>> >>>>> Hi Jakob, >>>>> >>>>> Debugging things on the task server can be a little tricky... >>>>> >>>>> When you say manually it works, what do you mean? If you do an: >>>>> >>>>> xdmp:invoke("/app/backend/update.xqy") it works? >>>> >>>> yes, it does. >>>> >>>>> The exception you are seeing can happen if the document >>>>> /app/backend/update.xqy is not loaded as a text document, or if the >>>>> document >>>>> cannot be found (or if there is a permission issue causing it not to be >>>>> found, but since you are running as admin that should not be the >>>>> case--assuming your admin user has the admin role). So the first thing to >>>>> verify is that the doistats-database database has the >>>>> app/backend/update.xqy >>>>> document loaded as a text file. >>>>> >>>>> For example, if you load a main module as follows, then you can confirm >>>>> it is a text document as follows: >>>>> >>>>> xdmp:document-insert("/test/test.xqy", text {"'hello'"}) ; >>>>> >>>>> if ( doc("/test/test.xqy")/node() >>>>> instance of text() ) >>>>> then "text" >>>>> else "not text" >>>> >>>> doing this for my xquery file returns, as hoped for, "text". >>>> >>>>> And yes, to change a scheduled task you must delete it and then >>>>> recreate it. >>>> >>>> yes, this has become an RFE, it seems. :) >>>> >>>> so, does this mean, on paper everything works as it should? what's >>>> the next step to test? could you explain the idea behind the task >>>> root property, I haven't grasped it so far. >>>> >>>>> -Danny >>>> >>>> thanks, >>>> Jakob. >>>> >>>> >>>>> -----Original Message----- >>>>> From: [email protected] >>>>> [mailto:[email protected]] On Behalf Of Jakob Fix >>>>> Sent: Monday, August 10, 2009 3:09 AM >>>>> To: General Mark Logic Developer Discussion >>>>> Subject: [MarkLogic Dev General] scheduled task (task root?) >>>>> >>>>> I've created a scheduled task in MLS 4.1. It does not run: >>>>> >>>>> not with "/" as the task root >>>>> 2009-08-08 21:32:00.106 Notice: TaskServer: XDMP-TEXTNODE: >>>>> /app/backend/upate.xqy -- Server unable to build program from non-text >>>>> document >>>>> 2009-08-08 21:32:00.106 Notice: TaskServer: in /app/backend/upate.xqy >>>>> [1.0-ml] >>>>> >>>>> nor with "http://server:8004/"; >>>>> 2009-08-10 12:00:00.050 Notice: TaskServer: XDMP-TEXTNODE: >>>>> http://server:8004/app/backend/update.xqy -- Server unable to build >>>>> program from non-text document >>>>> 2009-08-10 12:00:00.050 Notice: TaskServer: in /app/backend/update.xqy >>>>> [1.0-ml] >>>>> >>>>> manually, it works (of course). >>>>> >>>>> I have searched the marklogic mailing list for discussions with this >>>>> kind of problem and only found one message from May 2008 which had >>>>> encountered this type of exception >>>>> (http://markmail.org/thread/nk26f2eizhqy3scy). In my case, it seems >>>>> to be a problem with the "task root" property of the scheduled task >>>>> which is explained in the help as follows: >>>>> >>>>> "task root specifies the root directory (files [sic] system) or URI >>>>> root (database) that contains the module." >>>>> >>>>> I'm in the latter case where the xquery sits in the >>>>> "doistats-database" database in the /app/backend/ directory. Or could >>>>> the problem lie somewhere else (permissions? I'm running this as admin >>>>> with all privileges)? >>>>> >>>>> here are the task details: >>>>> >>>>> task path: /app/backend/update.xqy >>>>> task root: / >>>>> task type: daily >>>>> task period: 1 >>>>> task start time 9:32 PM >>>>> task database: doistats-database >>>>> task modules: doistats-database >>>>> task user: admin >>>>> task host: server >>>>> >>>>> >>>>> BTW, it seems a scheduled task cannot be modified, but only deleted? >>>>> (I filed a case with support@). >>>>> >>>>> thanks, >>>>> Jakob. >>>>> _______________________________________________ >>>>> General mailing list >>>>> [email protected] >>>>> http://xqzone.com/mailman/listinfo/general >>>>> _______________________________________________ >>>>> General mailing list >>>>> [email protected] >>>>> http://xqzone.com/mailman/listinfo/general >>>> >>>> _______________________________________________ >>>> General mailing list >>>> [email protected] >>>> http://xqzone.com/mailman/listinfo/general >>>> _______________________________________________ >>>> General mailing list >>>> [email protected] >>>> http://xqzone.com/mailman/listinfo/general >>>> >>> >> _______________________________________________ >> General mailing list >> [email protected] >> http://xqzone.com/mailman/listinfo/general > > _______________________________________________ > General mailing list > [email protected] > http://xqzone.com/mailman/listinfo/general > _______________________________________________ General mailing list [email protected] http://xqzone.com/mailman/listinfo/general
