Yes, and spawning also mean it will run on the task server, which usually has a much larger timeout time specified..
Good luck! Geert > -----Oorspronkelijk bericht----- > Van: [email protected] [mailto:general- > [email protected]] Namens Danny Sokolsky > Verzonden: woensdag 15 februari 2012 2:52 > Aan: MarkLogic Developer Discussion > Onderwerp: Re: [MarkLogic Dev General] How to Stop Time Out > > Hi Betty, > > You might start by creating a string range index on RGNum, then instead of the > for loop you show, run a lexicon call to get the distinct values. Something like: > > cts:element-values(xs:QName("RGNum"), (), (), > cts:and-query(( > cts:directory-query('/opt/documents/'), > cts:element-query(xs:QName("era:MyDocument"), cts:and-query( () )) > )) ) > > That should make that distinct values calculation much faster. > > Now you could also xdmp:spawn each spreadsheet, passing the value from the > distict-values calculation (or the element-values call if you create the range > index). > > -Danny > > -----Original Message----- > From: [email protected] [mailto:general- > [email protected]] On Behalf Of Betty Harvey > Sent: Tuesday, February 14, 2012 6:06 PM > To: MarkLogic Developer Discussion > Subject: Re: [MarkLogic Dev General] How to Stop Time Out > > It is not possible to run the queries separately because I don't know what > the unique values are. There are a possibility of 1100+ values but only > 186 are currently in the database. Ultimately the number of documents are > going to increase tremendously and all 1100 spreadsheets will need to be > created. 2.4 million documents are the tip of the iceberg. > > I am getting the unique values: > > for $RG in > fn:distinct- > values(xdmp:directory('/opt/documents/')/descendant::era:MyDocument/desce > ndant::RGNum/text()) > > I then create the spreadsheet and call a function that gathers information > for each document that matches the $RG. > > I have a variable that maintains the contents of each spreadsheet. > > The only command in the return is: > > xdmp:save($RGNum, document {$report} ) > > This writes out every spreadsheet. I am running the report from the > browser or CQ. > > Thanks! > > Betty > > > Hi Betty, > > > > You say 'the script stops at 36'. That sounds to me as if you are creating > > all 186 spreadsheets in one request. Is it possible to generate them one > > by one? That might also allow parallel processing them. > > > > You also say that a larger case deals with a spreadsheet of 664k+ rows and > > takes an hour. You will have to tell more about the strategy you use in > > order to help with optimizations. I have one thought though: could taking > > multiple steps be a solution? > > > > Colleen asks whether you are running this in the background or not. App > > and Task server both have their own timeout setting. Background processing > > makes employing CPF interesting too. > > > > Kind regards, > > Geert > > > >> -----Oorspronkelijk bericht----- > >> Van: [email protected] [mailto:general- > >> [email protected]] Namens Betty Harvey > >> Verzonden: woensdag 15 februari 2012 0:04 > >> Aan: MarkLogic Developer Discussion > >> Onderwerp: [MarkLogic Dev General] How to Stop Time Out > >> > >> Is there anyway to tell MarkLogic not to time out. I have developed a > >> query that is cycles through the database and creates an Excel > > spreadsheet > >> for each distinct group of values. > >> > >> There are 2.4 million documents in the database. There should be a > > total > >> of 186 spreadsheets that get created. The script stops at 36 > >> spreadsheets. I tested one of the larger values that resulted in a > > total > >> of 664,000+ rows in the Excel spreadsheet and it took approximately an > >> hour. > >> > >> I need someway of telling MarkLogic not to time out on this query. I > >> really don't want to increase the time for timeout. I am working in a > >> test environment and this option isn't an available option in > > production. > >> > >> TIA! > >> > >> Betty > >> > >> /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/ > >> Betty Harvey | Phone: 410-787-9200 FAX: 9830 > >> Electronic Commerce Connection, Inc. | > >> [email protected] | Washington,DC XML Users Grp > >> URL: http://www.eccnet.com | http://www.eccnet.com/xmlug > >> /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\\/\/ > >> Member of XML Guild (www.xmlguild.org) > >> _______________________________________________ > >> General mailing list > >> [email protected] > >> http://developer.marklogic.com/mailman/listinfo/general > > _______________________________________________ > > General mailing list > > [email protected] > > http://developer.marklogic.com/mailman/listinfo/general > > > > > /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/ > Betty Harvey | Phone: 410-787-9200 FAX: 9830 > Electronic Commerce Connection, Inc. | > [email protected] | Washington,DC XML Users Grp > URL: http://www.eccnet.com | http://www.eccnet.com/xmlug > /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\\/\/ > Member of XML Guild (www.xmlguild.org) > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
