Re: [basex-talk] Tail Recursion Error on startup
Thanks. Any thoughts on why it would clear up after 3-4 attempts of running the query? From: Christian Grün To: buddyonweb-software Cc: BaseX Talk Sent: Wednesday, December 21, 2016 8:31 AM Subject: Re: [basex-talk] Tail Recursion Error on startup Hi Buddy, > If we attempt to execute it again (sometimes twice, but have done so > repeatedly up to 5 times), the error goes away and all is fine. Difficult to say why this happens. Maybe your code is rewritten by Java’s JIT compiler. > Is there anything you can suggest or anything we can provide you to see how > to eliminate this. This is particularly problematic on our server when we > reboot our production application. > Anything you can suggest or anyway we can help is much appreciated. The problem itself is a generic one and not limited to BaseX or XQuery. Did you try rewrite your function calls to tail calls [1]? Another alternative is to increase the stack size of Java (via the -Xss flag). Hope this helps, Christian [1] https://en.wikipedia.org/wiki/Tail_call
[basex-talk] Tail Recursion Error on startup
When we start up BaseX (whether the server or the GUI) and run a particular a query against a particular .xqm file, we always get a tail recursion error. Error:[bxerr:BASX0005] Stack Overflow: Try tail recursion? If we attempt to execute it again (sometimes twice, but have done so repeatedly up to 5 times), the error goes away and all is fine. Is there anything you can suggest or anything we can provide you to see how to eliminate this. This is particularly problematic on our server when we reboot our production application.Anything you can suggest or anyway we can help is much appreciated. Buddy
[basex-talk] Dynamic lookup of namespaces
Is there a way to get a list of the namespaces declared/used? I am wanting to right a function that does a function lookup, but I want to right it in such a way that I don't have to know the namespaces ahead of time. Are there any functions that will return a list of the namespaces being used? Thanks in advance, Buddy
[basex-talk] Out of memory getting document from Client API
Using the client API I am trying to retrieve a 600MB file from BaseX. Version
I am using is BaseX 8.3.
I have successfully stored this document using the client API without issue.
As an aside, I have update the basexserver script to include 4GB of memory as
Java command line option. Here is the snippet from that script:# Options for
virtual machine (can be extended by global options)
BASEX_JVM="-Xmx4g $BASEX_JVM"
In my Java client side application I have memory set to 3 GB of memory (script
snippet below):exec "$JAVACMD" -Xms1g -Xmx3g .rest of args.
Here is a snippet of my Java code. Note that this works just fine when
operation on smaller documents, but I wanted to stress test the code.
String query = "doc('theCollectionName/extract.xml')";ByteArrayOutputStream
baos = new ByteArrayOutputStream();
ClientSession session = new ClientSession(host, port, user,
pwd);session.setOutputStream(baos);ClientQuery cq = session.query(query);
cq.execute();
However, I keep getting out of memory with the below stack trace. I would
think that the amount of space I am providing in both thebasexserver script
(4GB) and my client side app (3GB) would be plenty to read in a 600MB file from
BaseX. Is there something I am overlookingor something I am not doing
correctly? Any insights/feedback is appreciated.
Exception in thread "queue" java.lang.OutOfMemoryError: Java heap space
at java.lang.StringCoding$StringDecoder.decode(StringCoding.java:149)
at java.lang.StringCoding.decode(StringCoding.java:193)
at java.lang.StringCoding.decode(StringCoding.java:254)
at java.lang.String.(String.java:534)
at java.io.ByteArrayOutputStream.toString(ByteArrayOutputStream.java:221)
at org.basex.api.client.ClientSession.exec(ClientSession.java:261)
at org.basex.api.client.ClientQuery.execute(ClientQuery.java:102)
[basex-talk] Binding external variable in GUI
At the risk of a) sounding like a real newbie b) being accused of not Googling for an answer (which I did - but apparently overlooked) I am wondering how I can bind an external variable using just XQuery syntax in the BaseX GUI. I have found how to set the variable via command line options, and how to do it in Java, but I merelywanted to run a quick interactive test in the GUI to test an idea of mine (before writing a bunch of Java code).What I have done is in a tab in the GUI, I have included the module, and before running a function, want to set that external variable. Surprisingly I found myself at a loss on how to do what appears to be a simple task. So, thanks in advance for any helpand for the rest of this community for not laughing at me too loudly Buddy
[basex-talk] JavaDoc link?
I may have overlooked it (and if so, sorry), but I can't seem to find the link to the JavaDocs on any of the online documentation pages.Can you direct me to the online documentation page that has that link? Again sorry if it was right in front of me and I missed it. Thanks, Buddy
[basex-talk] DB Contention
I have a Java process that continually scans for incoming .xml files deposited into a system folder on the OS. When it finds a file (say a.xml), it will create a DB for that file (call it DB-A) and load the XML file into that database (DB-A).Note: DB names will be guaranteed to be unique when created, so there will never be 2 DB-A databases created. As this file is processed by a seondar, other XML files will be added into DB-A - but all documents will relate only to the processing of the a.xml file (for example, various statistics, etc.). Now, there is a secondary/separate java process, that scans the database instances that have been created with the XML file loaded. This secondary process does some querying on that file and adds additional XML documents to that database. That is only this process will add new documents to the database. So my question is this: 1. With process 1 creating and inserting the original .xml file, is there a chance for database contention or is this architecture pretty safe from contention? Note: both Java process are simply using the BaseX.jar file. 2. If I added a third Java process in the future, that would a) only access existing documents in read only mode b) could add new documents to the database that no other process would read or update, is this safe from contention? Thanks in advance.
Re: [basex-talk] Outputting large output
The arg basically is a map that contains elements the other functions may
need.So it can not only contain the root element, but any other sub elements.
Since this map is beingpassed around, we are free to put in that map anything
you suggest.
Write now we just have the output being written to disk, but that is just
because of where we are in the development right now.We could most certainly
store it in BaseX as well. We are sort of open to options with respect to the
storage.
Let me know if there is any other details I can provide you with or that you
may need.
On Monday, September 21, 2015 12:06 PM, Christian Grün
wrote:
> let $src := collection("someDataBase")[1] - illustrating with just 1
> document
>
> mynsp:mytransform(args.)
So args is an XQuery variable containing a document? What do you with
the results? Do you output them as result, or do you store them
somewhere?
Once again, please provide us something that's both complete and
self-contained (this may help: http://sscce.org), or be more specific
when it comes to the problem description. Take your time.
>
> The mynsp:mytransform function would then look something as follows
>
> declare function spec:mytransform(args) as element()*
> {
>
> (
> ...code that then produces other sub elements. This is primarily
> achieved by calling other functions similar to this one. For example.
>
> spec:someTransform1(...),
> spec:someTransform2(...),
> etc.
> )
>
> }
>
> Let me know if there is any other details you'd like or that I left out.
> Once again, I appreciate your efforts.
>
>
>
> On Monday, September 21, 2015 11:45 AM, Christian Grün
> wrote:
>
>
> Once again, please provide us with more specific information, such as
> e.g. self-contained code snippets; otherwise we cannot help. There are
> simply too many ways to transform documents, query data, etc.
>
>
> On Mon, Sep 21, 2015 at 5:41 PM, wrote:
>> I am using BaseX to transform some an XML documents into another XML
>> layout.
>> When the incoming document gets large, run out of memory. I've bumped up
>> memory in the
>> JVM and that worked.
>>
>> However, what if there was a really large document? Is there a way to
>> redirect the output
>> so that instead of the results stored in memory they are sent to disk
>> somewhere (whether to a file
>> on the file system or even back into BaseX - destination doesn't quite
>> matter).
>>
>> Thanks.
>>
>
>
Re: [basex-talk] Outputting large output
Sure, let me provide some more details - and sorry for the original vagueness.
The main query that "kicks" the process off looks something like this
(simplified a bit):
let $src := collection("someDataBase")[1] - illustrating with just 1
document
mynsp:mytransform(args.)
The mynsp:mytransform function would then look something as follows
declare function spec:mytransform(args) as element()*{(
...code that then produces other sub elements. This is primarilyachieved by
calling other functions similar to this one. For example.
spec:someTransform1(...),spec:someTransform2(...),etc.
)
}
Let me know if there is any other details you'd like or that I left out.Once
again, I appreciate your efforts.
On Monday, September 21, 2015 11:45 AM, Christian Grün
wrote:
Once again, please provide us with more specific information, such as
e.g. self-contained code snippets; otherwise we cannot help. There are
simply too many ways to transform documents, query data, etc.
On Mon, Sep 21, 2015 at 5:41 PM, wrote:
> I am using BaseX to transform some an XML documents into another XML layout.
> When the incoming document gets large, run out of memory. I've bumped up
> memory in the
> JVM and that worked.
>
> However, what if there was a really large document? Is there a way to
> redirect the output
> so that instead of the results stored in memory they are sent to disk
> somewhere (whether to a file
> on the file system or even back into BaseX - destination doesn't quite
> matter).
>
> Thanks.
>
Re: [basex-talk] Opening a database outside of the BaseX directory
The DBPATH worked - boy, don't I feel dumb...oops! Thanks so much again! On Monday, September 21, 2015 11:39 AM, Christian Grün wrote: > Having an issue opening up a database (using just the BaseX jar file). I > execute > XQuery (inside Java program) and I try specifying the path to the database > (e.g. /home/somedirectory/basex/data/MyDB). However whenever I use the full > path to get > the database, it can't find it. I execute the program locally (without the > full path) and everything works just fine! We need more information: How did you specify the database path? Did you assign it to DBPATH, as described in the documentation [1]? A .basex file may simplify your life; if your setting allows it, you can also move your database directory to your Java working directory. Christian [1] http://docs.basex.org/wiki/Configuration
[basex-talk] Outputting large output
I am using BaseX to transform some an XML documents into another XML layout.When the incoming document gets large, run out of memory. I've bumped up memory in the JVM and that worked. However, what if there was a really large document? Is there a way to redirect the outputso that instead of the results stored in memory they are sent to disk somewhere (whether to a fileon the file system or even back into BaseX - destination doesn't quite matter). Thanks.
[basex-talk] Opening a database outside of the BaseX directory
OK, I've been working with BaseX for a little while with some great success. But I am now putting some 'glue' code together and I have a simple task that I am struggling with (believe it or not) Having an issue opening up a database (using just the BaseX jar file). I execute XQuery (inside Java program) and I try specifying the path to the database (e.g. /home/somedirectory/basex/data/MyDB). However whenever I use the full path to get the database, it can't find it. I execute the program locally (without the full path) and everything works just fine! I've tried using db:open() and collection(). Any advice - and sorry for such a simple question. I'm sure I am overlooking something both simple and silly. Thanks again
