Hi Christian, and thanks a lot for the pointer to fetch:xml - it seems to
do the trick! Now, a little recoding, and it should be working.
Best,
Lars
2015-03-27 10:48 GMT+01:00 Christian Grün <christian.gr...@gmail.com>:
> Hi Lars,
>
> Here is some background information for the reported behavior (sorry
> in advance if this is known to you anyway): The functional semantics
> of XQuery requires that repeated calls to fn:doc and fn:collection
> return the same documents. This can e.g. be shown by the following
> query:
>
> doc('x.xml') is doc('x.xml')
>
> As it's difficult to guess in advance which of the opened documents
> will possibly be requested again in the same query, they are all kept
> in main-memory until query evaluation is completed.
>
> However, things are different with functions like fetch:xml [1]. You
> may need to tweak your query a little bit, because the function will
> always give you single XML documents.
>
> Does this help?
> Christian
>
> [1] http://docs.basex.org/wiki/Fetch_Module#fetch:xml
>
>
> On Fri, Mar 27, 2015 at 10:41 AM, Lars Johnsen <yoon...@gmail.com> wrote:
> > Hi all
> >
> > Here is code that gradually eats up memory, whether run in GUI or as
> > command. All it does is creating temporary collections out of folders,
> and
> > writing them to file.
> >
> > Is there a simple way to avoid this code to eat up memory? It runs out of
> > memory (set at 12GB for command, 18GB in GUI) after 300 folders or so,
> and
> > it has to process 20 000 of them.
> >
> > Best
> > Lars G Johnsen
> > Norwegian National Library
> >
> > Here is the actual code
> >
> > (: process list of folders :)
> >
> > for $collections in file:list($digibooks)
> > let $html := $htmlfiles || substring-before($collections, "_ocr") ||
> > ".html"
> >
> > return
> >
> > (: code is rerun so check if files exist :)
> >
> > if (not(file:exists($html))) then
> > try {
> >
> > (: create a temporary collection of the files and write
> result to
> > disk :)
> >
> > file:write(
> > $html,
> > db:digibok-to-html(
> > collection($digibooks || $collections))
> > )
> >
> > } catch * {
> > $err:code
> > }
> > else
> > ()
> >
> >
>
