Grab a copy of the source distribution.
In the directory, java\src\org\apache\xalan\lib\sql
There are two central files
DTMDocument.java and SQLDocument.java
DTMDocument is a class that provides the foundation to create
a DTM indexed collection of object.
SQLDocument, populates that collection with results from the
execution of an SQL Statement / ResultSet... A lot of the magic
occurs in the function addRowToDTMFromResultSet( )
A DTM (Document Table Model) consists of integer arrays
that allow you to navigate the collection efficiently where XPath
statements are being evaluated. The first implementation of Xalan2
used the whole integer for index value but that proved to be a problem
to handle many documents. To solve that problem they segmented
the integer where so many bits represented the Document ID and
the rest represented the Nodes within that document. So any one ID
could represent an ID in a document.There are some bas functions that
help you split DocID from NodeID and recombine. The trick part is
that to save memory, the Metadata for each result set is only read once
then its mirrored on each row. To do this, the SQLDocument code
is dependent of DTMID's always being in incremental order. There
is code that makes a statement "if current DTMID > LastRowID"
In a multi segment DTM Document, there is no guarantee that
NodeID's are allocated in an increasing order as you cross through
segments. So once you get you head around DTMID's, the problem
is fairly simple but as I said the devil will be in the details
Dan Feather wrote:
Ah! That's what I was afraid of.
Okay, well to get past the problem for the time being, I will look into
the copy-of approach... could you by chance point me to or provide a
little more detail about that? I am not well versed in XSLT so anything
to kick start my progress would be greatly appreciated.
As for writing a fix... I could take a crack at fixing it. I would be
interested in seeing what is involved. It would be something I would
need to work on after hours (i.e. not at work). I will contact you off
list with a different e-mail address shortly to work out the details.
Thanks for your response!
-Dan
-----Original Message-----
From: John Gentilin [mailto:[EMAIL PROTECTED]
Sent: Thursday, February 23, 2006 2:42 PM
To: Dan Feather
Cc: [email protected]
Subject: Re: Sql Extension - Possible limitation in number of records?
Hi Dan,
You are correct in there is currently somewhat of a limitation in
non-streaming mode on the
number of records that can be handled. This is because of the segmented
DTM mechanism
internal to Xalan. Currently the SQL extensions only can handle one
segment so eventually
you run out of DTMID's. The project to overcome this limitation is on my
list but there have
been almost no inquiries so its priority has been fairly low. As a work
around, you can enable
streaming mode then create a variable that is a "copy-of" of your query.
How are your Java skills, if you have the time I would be willing to
walk you through the
problem, then review the code once you come up with a fix. Its not a
particularly hard
problem but the devil is in the details..
Regards
John Gentilin
Dan Feather wrote:
Hello,
I am using the SQL-Extension and I running into a problem that I
think is related to the number of records in a result set and I am not
using streaming mode. I need to make recursive stylesheet calls and
access previous rows, so I cannot use streaming mode.
My table has 10 columns, and 3,183 rows in this case. When I run
my stylesheet over this table I immediately get a
NullPointerException.
However, if I delete around 80 rows and run it, everything works fine.
I
can delete a different 80 rows (to eliminate it being a problem in the
data contained in one of the rows) and it works fine. It seems to
happen
right around 3100 rows. I haven't gone and deleted one row at a time
yet
to determine the exact number of rows in this case.
Anyway, my question is has anyone experienced similar behavior?
Are there any known changes or work arounds for this problem? Is this
a
known issue? If you would like more information I would be happy to
provide it, but I figured I would just see if this rang any bells with
anyone before we dove to far into the code. I appreciate your help.
-Dan
--
--------------------------------------
John Gentilin
Eye Catching Solutions Inc.
18314 Carlwyn Drive
Castro Valley CA 94546
Contact Info
[EMAIL PROTECTED]
Ca Office 1-510-881-4821
NJ Office 1-732-422-4917
