[jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
[ http://issues.apache.org/jira/browse/DERBY-688?page=all ] A B updated DERBY-688: -- Derby Info: [Release Note Needed] Adding a release note to call out Derby's dependency on a JAXP parser and Apache Xalan when executing XML operators. First attempt at the release note is as follows: DERBY-688: XML datatype and corresponding SQL/XML operators. As of the 10.2 release, Derby supports a new XML data type and a set of operators that work with the XML data type. The XML data type and operators are based on a small subset of the SQL/XML specification. PROBLEM For the XML operators to work properly, Derby requires that a JAXP parser, such as Apache Xerces, and Apache Xalan are included in the Java classpath. If either the parser or Xalan are missing from the classpath, Derby disallows any XML-related operations. SYMPTOM If a user attempts to use any of the of XML operators but does not have a JAXP parser AND Apache Xalan in his/her classpath, the result will be an error similar to the following: Failed to locate '' API or implementation classes. XML operations are not permitted unless these classes are in your classpath. CAUSE Derby does all XML parsing by making calls to JAXP methods as defined in the JDK 1.4 API. Similarly, Derby does all evaluation of XPath queries, manipulation of XPath results, and serialization of XML values by making calls to Xalan-specific methods defined in Xalan classes. Thus if either JAXP or Xalan is missing from the classpath, the Derby XML operators would not function correctly, leading to ClassNotFound and similar exceptions at various points during code execution. In order to avoid this, Derby checks to see if JAXP and Xalan are in the classpath, and if either is missing, Derby will not even attempt to perform XML operations. Instead, it will just issue the error mentioned above. SOLUTION If you intend to use any of the Derby XML operators, make sure that you have 1) a JAXP parser and 2) Apache Xalan in your classpath. If you are running in client/server mode, the JAXP and Xalan classes must be in the classpath for the Derby server, since that is where XML-related processing actually occurs. Apache Derby version 10.2 has been tested with Xalan 2.7.0. If you have a version of Xalan that is earlier than 2.7, the Derby XML operators may still work. However, it is possible that you will experience unexpected errors when using the Derby XML operators. For example, there is a bug in Xalan versions earlier than 2.5 that can lead to failures with Derby XML operators when Derby is running with a security manager. The exact error can vary depending on the situation, but generally includes a line similar to the following: org.apache.xml.utils.WrappedRuntimeException: The resource [ file:org/apache/xalan/serialize/XMLEntities.res ] could not load: java.security.AccessControlException: access denied This problem has been fixed as of Xalan 2.5, so you can avoid the problem by using a more recent version of Xalan. Note: Most Java virtual machines (JVMs) that are version 1.4 or later have a JAXP parser embedded in the JVM. If you are using one of these JVMs, you do not need to add any other JAXP classes to your classpath. Additionally, if the JVM that you are using includes an embedded version of Xalan, you should confirm that the version of Xalan satisfies the minimum requirements for Derby. For example, if your JVM is Sun JDK 1.4.2, you must override the version of Xalan in the JVM with a newer version. Use Java's Endorsed Standards Override Mechanisms described at http://java.sun.com/j2se/1.4.2/docs/guide/standards/ to override the version of Xalan. If the JVM that you are using does not have a JAXP parser or a version of Xalan, you can add external versions of those classes in your classpath and Derby will pick up those classes. WORKAROUND There is no way to use the Derby XML operators without first including a JAXP parser and Apache Xalan in your classpath. Attempts to do so will result in an error. > Enhancements to XML functionality to move toward XPath/XQuery support... > > > Key: DERBY-688 > URL: http://issues.apache.org/jira/browse/DERBY-688 > Project: Derby > Issue Type: Improvement > Components: SQL, JDBC >Reporter: A B > Assigned To: A B >Priority: Minor > Fix For: 10.2.2.0, 10.3.0.0 > > Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat, > d688_phase1_v2.patch, d688_phase1_v3.patch, d688_phase2_v1_code.patch, > d688_phase2_v1_tests.patch, d688_phase2_v2_tests.patch, > d688_phase2_v3_tests.patch, d688_phase3_v1_code.patch, > d688_phase3_v1_tests.patch, d688_phase4_v1.patch, d688_phase4_v2.patch, > d688_phase5_v1.patch, d688_phase6_v1.patch, d688_phase7_v1.patch, > derbyXMLSpec
Re: XML in Derby - WAS : [jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
Kristian Waagan wrote:
> Daniel John Debrunner wrote:
>
>> David Van Couvering wrote:
>>
>>> Wow, great work Army, and thanks for the careful review and effort by
>>> Brian and Yip! It's great to have this in. If I could only grok
>>> exactly what the feature is and how I might use it :) I am telling
>>> people "we have XML features" and I know it's something to do with
>>> XQuery and XPath, but I couldn't say what.
>>
>>
>> I was messing with the XML support last night for buddy testing and it's
>> very cool. Three new Derby features combined together to make the
>> application development easier:
>>
>> 1) XML support
>> 2) CALL procedure in trigger
>> 3) Lengthless overrides for PreparedStatement.setCharacterStream
>>
>> OK - I didn't test 3) cos I would have to set up Xalan for Mustang and I
>> didn't want to spend time on how to figure that out, but it would have
>> been useful.
>
>
> Hi Dan,
>
> This sounds really cool :)
> Any change you can share your trigger procedure?
> Then maybe someone else can take on the work to test the new lengthless
> overrides added by JDBC 4.0. I don't think they have received much
> testing yet. They are now in the 10.2 branch, but I believe they are not
> (fully) included in the latest beta (10.2.1.1). I'll add an entry to the
> buddy testing page when the time is right.
Here's the Java method:
public static void get_url_content(String id, String path)
throws SQLException, IOException
{
Connection conn =
DriverManager.getConnection("jdbc:default:connection");
PreparedStatement ps = conn.prepareStatement(
"UPDATE WDD.WEB_DOCS SET WD_CONTENT = " +
"XMLPARSE (DOCUMENT CAST (? AS CLOB) PRESERVE WHITESPACE)" +
" , WD_ACCESSTIME = CURRENT TIMESTAMP " +
"WHERE WD_ID = ?");
ps.setString(2, id);
URL url = new URL(path);
URLConnection urlConn = url.openConnection();
urlConn.connect();
int length = urlConn.getContentLength();
String enc = urlConn.getContentEncoding();
if (enc == null)
enc = "UTF-8";
InputStream in = urlConn.getInputStream();
InputStreamReader isr = new InputStreamReader(in, enc);
// HACK - Assume number of characters will be
// the same as the number of bytes.
ps.setCharacterStream(1, isr, length);
ps.execute();
in.close();
ps.close();
conn.close();
}
and the SQL
DROP TABLE WDD.WEB_DOCS;
CREATE TABLE WDD.WEB_DOCS (
WD_ID VARCHAR(128) PRIMARY KEY,
WD_URL VARCHAR(1000),
WD_CONTENT XML,
WD_ACCESSTIME TIMESTAMP
);
DROP PROCEDURE WDD.GET_URL_CONTENT;
CREATE PROCEDURE WDD.GET_URL_CONTENT(ID VARCHAR(128), URL VARCHAR(1000))
LANGUAGE JAVA PARAMETER STYLE JAVA
EXTERNAL NAME 'wdd.get_url_content';
CREATE TRIGGER WDD.WD_I AFTER INSERT
ON WDD.WEB_DOCS
REFERENCING NEW AS NEW
FOR EACH ROW MODE DB2SQL
CALL WDD.GET_URL_CONTENT(NEW.WD_ID, NEW.WD_URL);
Dan.
Re: XML in Derby - WAS : [jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
Daniel John Debrunner wrote:
David Van Couvering wrote:
Wow, great work Army, and thanks for the careful review and effort by
Brian and Yip! It's great to have this in. If I could only grok
exactly what the feature is and how I might use it :) I am telling
people "we have XML features" and I know it's something to do with
XQuery and XPath, but I couldn't say what.
I was messing with the XML support last night for buddy testing and it's
very cool. Three new Derby features combined together to make the
application development easier:
1) XML support
2) CALL procedure in trigger
3) Lengthless overrides for PreparedStatement.setCharacterStream
OK - I didn't test 3) cos I would have to set up Xalan for Mustang and I
didn't want to spend time on how to figure that out, but it would have
been useful.
Hi Dan,
This sounds really cool :)
Any change you can share your trigger procedure?
Then maybe someone else can take on the work to test the new lengthless
overrides added by JDBC 4.0. I don't think they have received much
testing yet. They are now in the 10.2 branch, but I believe they are not
(fully) included in the latest beta (10.2.1.1). I'll add an entry to the
buddy testing page when the time is right.
Regards,
--
Kristian
I setup a table with a column that contained a URL as a VARCHAR and an
XML column, a procedure in an INSERT trigger then fetched the data from
the URL and updated the XML column using XMLPARSE, streaming directly
from the remote site using setCharacterStream.
I used this to download DERBY Jira issues, each row holds an XML
document that corresponds to a single Jira issue. E.g. from
http://issues.apache.org/jira/browse/DERBY-434?decorator=none&view=rss
Then I can execute queries against the issues locally, using XPath (and
SQL).
-- Sequence of all comments made by Sunitha against bugs reported by me
-- wd_id is the DERBY-XXX identifier
-- wd_accesstime is the time the data was downloaded from the web.
-- wd_content is the XML column
select wd_id, wd_accesstime,
XMLSERIALIZE(
XMLQUERY('//item/comments/[EMAIL PROTECTED]"skambha"]' PASSING BY REF
wd_content EMPTY ON EMPTY)
AS VARCHAR(3))
from wdd.web_docs where
XMLEXISTS('//reporter[text() = "Daniel John Debrunner"]' PASSING BY REF
wd_content);
-- Jira status of all bugs entered by me
select wd_id,
XMLSERIALIZE(
XMLQUERY('//item/status/text()' PASSING BY REF wd_content EMPTY ON EMPTY)
AS VARCHAR(20)),
wd_accesstime
from wdd.web_docs where
XMLEXISTS('//reporter[text() = "Daniel John Debrunner"]' PASSING BY REF
wd_content)
order by 2,1;
I'll think I wil expand this to use it in my talk at ApacheCon US.
Dan.
XML in Derby - WAS : [jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
David Van Couvering wrote:
> Wow, great work Army, and thanks for the careful review and effort by
> Brian and Yip! It's great to have this in. If I could only grok
> exactly what the feature is and how I might use it :) I am telling
> people "we have XML features" and I know it's something to do with
> XQuery and XPath, but I couldn't say what.
I was messing with the XML support last night for buddy testing and it's
very cool. Three new Derby features combined together to make the
application development easier:
1) XML support
2) CALL procedure in trigger
3) Lengthless overrides for PreparedStatement.setCharacterStream
OK - I didn't test 3) cos I would have to set up Xalan for Mustang and I
didn't want to spend time on how to figure that out, but it would have
been useful.
I setup a table with a column that contained a URL as a VARCHAR and an
XML column, a procedure in an INSERT trigger then fetched the data from
the URL and updated the XML column using XMLPARSE, streaming directly
from the remote site using setCharacterStream.
I used this to download DERBY Jira issues, each row holds an XML
document that corresponds to a single Jira issue. E.g. from
http://issues.apache.org/jira/browse/DERBY-434?decorator=none&view=rss
Then I can execute queries against the issues locally, using XPath (and
SQL).
-- Sequence of all comments made by Sunitha against bugs reported by me
-- wd_id is the DERBY-XXX identifier
-- wd_accesstime is the time the data was downloaded from the web.
-- wd_content is the XML column
select wd_id, wd_accesstime,
XMLSERIALIZE(
XMLQUERY('//item/comments/[EMAIL PROTECTED]"skambha"]' PASSING BY REF
wd_content EMPTY ON EMPTY)
AS VARCHAR(3))
from wdd.web_docs where
XMLEXISTS('//reporter[text() = "Daniel John Debrunner"]' PASSING BY REF
wd_content);
-- Jira status of all bugs entered by me
select wd_id,
XMLSERIALIZE(
XMLQUERY('//item/status/text()' PASSING BY REF wd_content EMPTY ON EMPTY)
AS VARCHAR(20)),
wd_accesstime
from wdd.web_docs where
XMLEXISTS('//reporter[text() = "Daniel John Debrunner"]' PASSING BY REF
wd_content)
order by 2,1;
I'll think I wil expand this to use it in my talk at ApacheCon US.
Dan.
[jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
[ http://issues.apache.org/jira/browse/DERBY-688?page=all ] Bryan Pendleton updated DERBY-688: -- Derby Info: (was: [Patch Available]) Merged the phase7 patch to the 10.2 branch as revision 434583. Cleared Patch Available. Is it time to mark this issue resolved? > Enhancements to XML functionality to move toward XPath/XQuery support... > > > Key: DERBY-688 > URL: http://issues.apache.org/jira/browse/DERBY-688 > Project: Derby > Issue Type: Improvement > Components: SQL, JDBC >Reporter: A B > Assigned To: A B >Priority: Minor > Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat, > d688_phase1_v2.patch, d688_phase1_v3.patch, d688_phase2_v1_code.patch, > d688_phase2_v1_tests.patch, d688_phase2_v2_tests.patch, > d688_phase2_v3_tests.patch, d688_phase3_v1_code.patch, > d688_phase3_v1_tests.patch, d688_phase4_v1.patch, d688_phase4_v2.patch, > d688_phase5_v1.patch, d688_phase6_v1.patch, d688_phase7_v1.patch, > derbyXMLSpec.html > > > As of DERBY-334, Derby has some very basic support for XML that consists of > an XML datatype and three operators (XMLPARSE, XMLSERIALIZE, and XMLEXISTS). > I would like to enhance this existing functionality and, by doing so, help to > move Derby incrementally toward a more usable and more complete XPath/XQuery > solution (with emphasis on "incrementally"). > I have attached to this issue a document describing the particular changes > that I am looking to make. At a high level, they consist of: > 1) Making it easier to use the XML operators and datatype from within JDBC > (ex. by implicit parsing/serialization of XML values). > 2) Adding a new operator, XMLQUERY, to allow a user to retrieve the results > of an XPath expression (instead of just determining whether or not the > expression evaluates to an empty sequence, which is what XMLEXISTS does). > 3) Making changes to the existing operators to line them up with the SQL/XML > 2005 specification, and also to take steps toward my eventual hope of having > support for XQuery (as opposed to just XPath) in Derby. > If anyone has time and interest enough to look at the document and provide > feedback, that'd be great... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
[ http://issues.apache.org/jira/browse/DERBY-688?page=all ] A B updated DERBY-688: -- Derby Info: [Patch Available] > Enhancements to XML functionality to move toward XPath/XQuery support... > > > Key: DERBY-688 > URL: http://issues.apache.org/jira/browse/DERBY-688 > Project: Derby > Issue Type: Improvement > Components: JDBC, SQL >Reporter: A B > Assigned To: A B >Priority: Minor > Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat, > d688_phase1_v2.patch, d688_phase1_v3.patch, d688_phase2_v1_code.patch, > d688_phase2_v1_tests.patch, d688_phase2_v2_tests.patch, > d688_phase2_v3_tests.patch, d688_phase3_v1_code.patch, > d688_phase3_v1_tests.patch, d688_phase4_v1.patch, d688_phase4_v2.patch, > d688_phase5_v1.patch, d688_phase6_v1.patch, d688_phase7_v1.patch, > derbyXMLSpec.html > > > As of DERBY-334, Derby has some very basic support for XML that consists of > an XML datatype and three operators (XMLPARSE, XMLSERIALIZE, and XMLEXISTS). > I would like to enhance this existing functionality and, by doing so, help to > move Derby incrementally toward a more usable and more complete XPath/XQuery > solution (with emphasis on "incrementally"). > I have attached to this issue a document describing the particular changes > that I am looking to make. At a high level, they consist of: > 1) Making it easier to use the XML operators and datatype from within JDBC > (ex. by implicit parsing/serialization of XML values). > 2) Adding a new operator, XMLQUERY, to allow a user to retrieve the results > of an XPath expression (instead of just determining whether or not the > expression evaluates to an empty sequence, which is what XMLEXISTS does). > 3) Making changes to the existing operators to line them up with the SQL/XML > 2005 specification, and also to take steps toward my eventual hope of having > support for XQuery (as opposed to just XPath) in Derby. > If anyone has time and interest enough to look at the document and provide > feedback, that'd be great... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
[ http://issues.apache.org/jira/browse/DERBY-688?page=all ]
A B updated DERBY-688:
--
Attachment: d688_phase7_v1.patch
Attaching a "phase 7" patch, d1688_v1.patch, that does the following:
1 - Makes changes to catch all "Throwable" errors that might be
thrown by Xalan or JAXP, instead of just catching the exceptions
declared by the APIs. This is per the email thread here:
http://www.nabble.com/xalan-assertion-when-execution-a-xml-query...-tf2149830.html#a5953476
This allows Derby to continue working as normal if/when an
unexpected Xalan/JAXP error (such NPE or assertion failure)
occurs. In that case the statement itself will fail and the
error will be printed, but Derby will continue to work as
expected after the failure.
2 - Slight change so that, in the event of an unexpected Xalan
compilation error, the name of the operator that encountered
the error will be printed as part of Derby's message. Currently
the operator name isn't passed in and thus "{0}" shows up
in the error message, which is incorrect.
3 - Fixes a small bug in XML query execution code that was leading
to NPEs in Xalan. Namely, the current code passes a null argument
into Xalan where a non-null is expected (and required) for namespace
prefix resolution.
4 - Makes the first of two changes required to ensure Derby SQL/XML
support agrees with the specification. The two changes are
mentioned in my previous comments; this phase 7 patch addresses
the first one (insertion of a non-Document node into a Derby XML
column should not be allowed).
While that may look like a lot of changes, the changes for each of these is
quite small and I believe the changes to be easily reviewable and commitable as
a single patch. If anyone disagrees, though, please let me know and I can
break the changes up into separate patches.
Note that most of d688_phase7_v1.patch is made up of test changes for items #3
and #4.
I applied d688_phase7_v1.patch and ran xmlSuite against sane jars with ibm142
(Windows) and saw no failures. So this patch is ready for review/commit.
> Enhancements to XML functionality to move toward XPath/XQuery support...
>
>
> Key: DERBY-688
> URL: http://issues.apache.org/jira/browse/DERBY-688
> Project: Derby
> Issue Type: Improvement
> Components: JDBC, SQL
>Reporter: A B
> Assigned To: A B
>Priority: Minor
> Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat,
> d688_phase1_v2.patch, d688_phase1_v3.patch, d688_phase2_v1_code.patch,
> d688_phase2_v1_tests.patch, d688_phase2_v2_tests.patch,
> d688_phase2_v3_tests.patch, d688_phase3_v1_code.patch,
> d688_phase3_v1_tests.patch, d688_phase4_v1.patch, d688_phase4_v2.patch,
> d688_phase5_v1.patch, d688_phase6_v1.patch, d688_phase7_v1.patch,
> derbyXMLSpec.html
>
>
> As of DERBY-334, Derby has some very basic support for XML that consists of
> an XML datatype and three operators (XMLPARSE, XMLSERIALIZE, and XMLEXISTS).
> I would like to enhance this existing functionality and, by doing so, help to
> move Derby incrementally toward a more usable and more complete XPath/XQuery
> solution (with emphasis on "incrementally").
> I have attached to this issue a document describing the particular changes
> that I am looking to make. At a high level, they consist of:
> 1) Making it easier to use the XML operators and datatype from within JDBC
> (ex. by implicit parsing/serialization of XML values).
> 2) Adding a new operator, XMLQUERY, to allow a user to retrieve the results
> of an XPath expression (instead of just determining whether or not the
> expression evaluates to an empty sequence, which is what XMLEXISTS does).
> 3) Making changes to the existing operators to line them up with the SQL/XML
> 2005 specification, and also to take steps toward my eventual hope of having
> support for XQuery (as opposed to just XPath) in Derby.
> If anyone has time and interest enough to look at the document and provide
> feedback, that'd be great...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
Bryan Pendleton (JIRA) wrote: I reviewed d688_phase6_v1.patch and it looks good to me. The build was fine, and derbyall ran cleanly. xml_general.sql and xmlBinding.java ran as expected with Xalan 2.7. Committed this patch to subversion as revision 433450, and cleared the PatchAvailable flag. Thanks Bryan! Army
[jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
[ http://issues.apache.org/jira/browse/DERBY-688?page=all ] Bryan Pendleton updated DERBY-688: -- Derby Info: (was: [Patch Available]) I reviewed d688_phase6_v1.patch and it looks good to me. The build was fine, and derbyall ran cleanly. xml_general.sql and xmlBinding.java ran as expected with Xalan 2.7. Committed this patch to subversion as revision 433450, and cleared the PatchAvailable flag. > Enhancements to XML functionality to move toward XPath/XQuery support... > > > Key: DERBY-688 > URL: http://issues.apache.org/jira/browse/DERBY-688 > Project: Derby > Issue Type: Improvement > Components: JDBC, SQL >Reporter: A B > Assigned To: A B >Priority: Minor > Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat, > d688_phase1_v2.patch, d688_phase1_v3.patch, d688_phase2_v1_code.patch, > d688_phase2_v1_tests.patch, d688_phase2_v2_tests.patch, > d688_phase2_v3_tests.patch, d688_phase3_v1_code.patch, > d688_phase3_v1_tests.patch, d688_phase4_v1.patch, d688_phase4_v2.patch, > d688_phase5_v1.patch, d688_phase6_v1.patch, derbyXMLSpec.html > > > As of DERBY-334, Derby has some very basic support for XML that consists of > an XML datatype and three operators (XMLPARSE, XMLSERIALIZE, and XMLEXISTS). > I would like to enhance this existing functionality and, by doing so, help to > move Derby incrementally toward a more usable and more complete XPath/XQuery > solution (with emphasis on "incrementally"). > I have attached to this issue a document describing the particular changes > that I am looking to make. At a high level, they consist of: > 1) Making it easier to use the XML operators and datatype from within JDBC > (ex. by implicit parsing/serialization of XML values). > 2) Adding a new operator, XMLQUERY, to allow a user to retrieve the results > of an XPath expression (instead of just determining whether or not the > expression evaluates to an empty sequence, which is what XMLEXISTS does). > 3) Making changes to the existing operators to line them up with the SQL/XML > 2005 specification, and also to take steps toward my eventual hope of having > support for XQuery (as opposed to just XPath) in Derby. > If anyone has time and interest enough to look at the document and provide > feedback, that'd be great... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
A B (JIRA) wrote: So this patch, d688_phase6_v1.patch, is ready for commit. Thanks Army! I intend to review and commit this change. If anyone else is reviewing it, please let me know. thanks, bryan
[jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
[ http://issues.apache.org/jira/browse/DERBY-688?page=all ] A B updated DERBY-688: -- Derby Info: [Patch Available] > Enhancements to XML functionality to move toward XPath/XQuery support... > > > Key: DERBY-688 > URL: http://issues.apache.org/jira/browse/DERBY-688 > Project: Derby > Issue Type: Improvement > Components: JDBC, SQL >Reporter: A B > Assigned To: A B >Priority: Minor > Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat, > d688_phase1_v2.patch, d688_phase1_v3.patch, d688_phase2_v1_code.patch, > d688_phase2_v1_tests.patch, d688_phase2_v2_tests.patch, > d688_phase2_v3_tests.patch, d688_phase3_v1_code.patch, > d688_phase3_v1_tests.patch, d688_phase4_v1.patch, d688_phase4_v2.patch, > d688_phase5_v1.patch, d688_phase6_v1.patch, derbyXMLSpec.html > > > As of DERBY-334, Derby has some very basic support for XML that consists of > an XML datatype and three operators (XMLPARSE, XMLSERIALIZE, and XMLEXISTS). > I would like to enhance this existing functionality and, by doing so, help to > move Derby incrementally toward a more usable and more complete XPath/XQuery > solution (with emphasis on "incrementally"). > I have attached to this issue a document describing the particular changes > that I am looking to make. At a high level, they consist of: > 1) Making it easier to use the XML operators and datatype from within JDBC > (ex. by implicit parsing/serialization of XML values). > 2) Adding a new operator, XMLQUERY, to allow a user to retrieve the results > of an XPath expression (instead of just determining whether or not the > expression evaluates to an empty sequence, which is what XMLEXISTS does). > 3) Making changes to the existing operators to line them up with the SQL/XML > 2005 specification, and also to take steps toward my eventual hope of having > support for XQuery (as opposed to just XPath) in Derby. > If anyone has time and interest enough to look at the document and provide > feedback, that'd be great... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
[ http://issues.apache.org/jira/browse/DERBY-688?page=all ] A B updated DERBY-688: -- Attachment: d688_phase6_v1.patch Attaching a phase 6 patch, d688_phase6_v1.patch, that changes the SQLSTATEs for XML errors to match the SQL/XML specification where possible, and that moves the Derby-specific XML sql states to a compile-time range (42Z70 - 42Z7Z) instead of an execution-time range (X0Xxx). I applied the patch, did an ant clobber followed by an ant all, then ran xmlSuite with ibm142 and all tests passed (Windows). So this patch, d688_phase6_v1.patch, is ready for commit. > Enhancements to XML functionality to move toward XPath/XQuery support... > > > Key: DERBY-688 > URL: http://issues.apache.org/jira/browse/DERBY-688 > Project: Derby > Issue Type: Improvement > Components: JDBC, SQL >Reporter: A B > Assigned To: A B >Priority: Minor > Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat, > d688_phase1_v2.patch, d688_phase1_v3.patch, d688_phase2_v1_code.patch, > d688_phase2_v1_tests.patch, d688_phase2_v2_tests.patch, > d688_phase2_v3_tests.patch, d688_phase3_v1_code.patch, > d688_phase3_v1_tests.patch, d688_phase4_v1.patch, d688_phase4_v2.patch, > d688_phase5_v1.patch, d688_phase6_v1.patch, derbyXMLSpec.html > > > As of DERBY-334, Derby has some very basic support for XML that consists of > an XML datatype and three operators (XMLPARSE, XMLSERIALIZE, and XMLEXISTS). > I would like to enhance this existing functionality and, by doing so, help to > move Derby incrementally toward a more usable and more complete XPath/XQuery > solution (with emphasis on "incrementally"). > I have attached to this issue a document describing the particular changes > that I am looking to make. At a high level, they consist of: > 1) Making it easier to use the XML operators and datatype from within JDBC > (ex. by implicit parsing/serialization of XML values). > 2) Adding a new operator, XMLQUERY, to allow a user to retrieve the results > of an XPath expression (instead of just determining whether or not the > expression evaluates to an empty sequence, which is what XMLEXISTS does). > 3) Making changes to the existing operators to line them up with the SQL/XML > 2005 specification, and also to take steps toward my eventual hope of having > support for XQuery (as opposed to just XPath) in Derby. > If anyone has time and interest enough to look at the document and provide > feedback, that'd be great... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
[ http://issues.apache.org/jira/browse/DERBY-688?page=all ] Bryan Pendleton updated DERBY-688: -- Derby Info: (was: [Patch Available]) patch d688_patch_v5.txt committed to subversion as revision 430670. Clearing patch_available as all pending patches have been committed. > Enhancements to XML functionality to move toward XPath/XQuery support... > > > Key: DERBY-688 > URL: http://issues.apache.org/jira/browse/DERBY-688 > Project: Derby > Issue Type: Improvement > Components: JDBC, SQL >Reporter: A B > Assigned To: A B >Priority: Minor > Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat, > d688_phase1_v2.patch, d688_phase1_v3.patch, d688_phase2_v1_code.patch, > d688_phase2_v1_tests.patch, d688_phase2_v2_tests.patch, > d688_phase2_v3_tests.patch, d688_phase3_v1_code.patch, > d688_phase3_v1_tests.patch, d688_phase4_v1.patch, d688_phase4_v2.patch, > d688_phase5_v1.patch, derbyXMLSpec.html > > > As of DERBY-334, Derby has some very basic support for XML that consists of > an XML datatype and three operators (XMLPARSE, XMLSERIALIZE, and XMLEXISTS). > I would like to enhance this existing functionality and, by doing so, help to > move Derby incrementally toward a more usable and more complete XPath/XQuery > solution (with emphasis on "incrementally"). > I have attached to this issue a document describing the particular changes > that I am looking to make. At a high level, they consist of: > 1) Making it easier to use the XML operators and datatype from within JDBC > (ex. by implicit parsing/serialization of XML values). > 2) Adding a new operator, XMLQUERY, to allow a user to retrieve the results > of an XPath expression (instead of just determining whether or not the > expression evaluates to an empty sequence, which is what XMLEXISTS does). > 3) Making changes to the existing operators to line them up with the SQL/XML > 2005 specification, and also to take steps toward my eventual hope of having > support for XQuery (as opposed to just XPath) in Derby. > If anyone has time and interest enough to look at the document and provide > feedback, that'd be great... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
A B updated DERBY-688: -- Attachment: d688_phase4_v2.patch Thanks for the quick response Army! I am intending to review and commit the phase4 and phase5 patches tonight. thanks, bryan
[jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
[ http://issues.apache.org/jira/browse/DERBY-688?page=all ] A B updated DERBY-688: -- Attachment: d688_phase4_v2.patch Attaching a second version of the phase 4 patch that corrects the test failure reported by Bryan. The real reason I had the EXPORT commands in the test to begin with was because I assumed that the IMPORT command would check for the target file before checking the validity of the columns, and so I used a (successful) EXPORT command to create the file. Luckily, that assumption was wrong: the import command first checks the types of the columns and *then* looks for the file, and since the new test cases should fail due to the XML type, the actual file specified is never accessed, and thus it doesn't even have to exist. So I've updated the phase 4 test case to give bogus file names for the IMPORT commands since they don't matter. And I also removed the successful EXPORT command that created the file which was throwing the SecurityException. No other changes exist between d688_phase4_v1 and d688_phase4_v2. Also, my derbyall run with these XML changes ran with no new errors last night. I ran on SuSE Linux with ibm142 against insane jars. I think the phase 4 and phase 5 patches are now ready for commit (in that order), unless further review comments are made. This wraps up the *functional* changes for XML that I'd like to get into 10.2 There are still, however, some other tasks to complete: - Fix-up new error messages. All of the error messages for the DERBY-688 changes thus far have been added as "X0X...", but it recently occured to me that are those reserved for "execution time" errors. Many of the new XML messages are actually compile-time errors and thus they probably should have different SQLSTATEs. So I'll have to look at that. - Complete the documentation of the new operators (see DERBY-1655) - Fix xmlBinding test to address diff reported by Bryan and which I also see when running on Linux. - Add new tests to verify Derby behavior when XML classes are not present. - Enable tests to run as part of derbyall (for those jvms that have the required classes). - Add more descriptive comments of the diff between BY REF and BY VALUE, probably in sqlgrammar.jj - For completeness, I need to update the spec attached to this issue to reflect the behavior that's actually been implemented (which is a subset of the functionality described in the spec (ex. no JDBC support). The first two items are the only ones that are visible to the user, so those need to be completed first. > Enhancements to XML functionality to move toward XPath/XQuery support... > > > Key: DERBY-688 > URL: http://issues.apache.org/jira/browse/DERBY-688 > Project: Derby > Issue Type: Improvement > Components: JDBC, SQL >Reporter: A B > Assigned To: A B >Priority: Minor > Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat, > d688_phase1_v2.patch, d688_phase1_v3.patch, d688_phase2_v1_code.patch, > d688_phase2_v1_tests.patch, d688_phase2_v2_tests.patch, > d688_phase2_v3_tests.patch, d688_phase3_v1_code.patch, > d688_phase3_v1_tests.patch, d688_phase4_v1.patch, d688_phase4_v2.patch, > d688_phase5_v1.patch, derbyXMLSpec.html > > > As of DERBY-334, Derby has some very basic support for XML that consists of > an XML datatype and three operators (XMLPARSE, XMLSERIALIZE, and XMLEXISTS). > I would like to enhance this existing functionality and, by doing so, help to > move Derby incrementally toward a more usable and more complete XPath/XQuery > solution (with emphasis on "incrementally"). > I have attached to this issue a document describing the particular changes > that I am looking to make. At a high level, they consist of: > 1) Making it easier to use the XML operators and datatype from within JDBC > (ex. by implicit parsing/serialization of XML values). > 2) Adding a new operator, XMLQUERY, to allow a user to retrieve the results > of an XPath expression (instead of just determining whether or not the > expression evaluates to an empty sequence, which is what XMLEXISTS does). > 3) Making changes to the existing operators to line them up with the SQL/XML > 2005 specification, and also to take steps toward my eventual hope of having > support for XQuery (as opposed to just XPath) in Derby. > If anyone has time and interest enough to look at the document and provide > feedback, that'd be great... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
[ http://issues.apache.org/jira/browse/DERBY-688?page=all ] A B updated DERBY-688: -- Derby Info: [Patch Available] > Enhancements to XML functionality to move toward XPath/XQuery support... > > > Key: DERBY-688 > URL: http://issues.apache.org/jira/browse/DERBY-688 > Project: Derby > Issue Type: Improvement > Components: SQL, JDBC >Reporter: A B > Assigned To: A B >Priority: Minor > Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat, > d688_phase1_v2.patch, d688_phase1_v3.patch, d688_phase2_v1_code.patch, > d688_phase2_v1_tests.patch, d688_phase2_v2_tests.patch, > d688_phase2_v3_tests.patch, d688_phase3_v1_code.patch, > d688_phase3_v1_tests.patch, d688_phase4_v1.patch, d688_phase5_v1.patch, > derbyXMLSpec.html > > > As of DERBY-334, Derby has some very basic support for XML that consists of > an XML datatype and three operators (XMLPARSE, XMLSERIALIZE, and XMLEXISTS). > I would like to enhance this existing functionality and, by doing so, help to > move Derby incrementally toward a more usable and more complete XPath/XQuery > solution (with emphasis on "incrementally"). > I have attached to this issue a document describing the particular changes > that I am looking to make. At a high level, they consist of: > 1) Making it easier to use the XML operators and datatype from within JDBC > (ex. by implicit parsing/serialization of XML values). > 2) Adding a new operator, XMLQUERY, to allow a user to retrieve the results > of an XPath expression (instead of just determining whether or not the > expression evaluates to an empty sequence, which is what XMLEXISTS does). > 3) Making changes to the existing operators to line them up with the SQL/XML > 2005 specification, and also to take steps toward my eventual hope of having > support for XQuery (as opposed to just XPath) in Derby. > If anyone has time and interest enough to look at the document and provide > feedback, that'd be great... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
[ http://issues.apache.org/jira/browse/DERBY-688?page=all ] A B updated DERBY-688: -- Attachment: d688_phase4_v1.patch d688_phase5_v1.patch Attaching two patches, one for "Phase 4" and one for "Phase 5". Phase 4: - The phase 4 patch, d688_phase4_v1.patch, adds some additional restrictions to the ways in which XML values can be used. In particular: 1. XML types cannot be used in CREATE PROCEDURE or CREATE FUNCTION statements. 2. XML types cannot be used in import/export functions. 3. XML types cannot be declared as columns in a global temp table. I admit that I'm a bit fuzzy as to *why* these restrictions need to be in place, but these are the restrictions that apply to the other "long" datatypes in Derby (LOBs, LONG VARCHAR) so I'm enforcing them for XML, as well, to be safe. It'll be easier to remove these restrictions in the future than it will be to block them after users have potentially been relying on them. One final restriction added by this patch is as follows: if a parameter is used for the operand to an XMLPARSE operation, the parameter must have an explicit cast to a character string type. I've put this restriction in place because, based on my reading of the spec, this is required for SQL/XML[2006] conformance. Further explanation can be found in comments for the relevant code. d688_phase4_v1.patch also adds some simple test cases for each of these restrictions to the lang/xml_general.sql test, with the corresponding master updates. And finally, the phase 4 patch includes two new error messages: one for the XMLPARSE restriction, and one for missing XML classes, which is actually for phase 5 but I included it in the phase 4 patch so that the two patches can be applied sequentially (phase 4 then phase 5) without conflict. Phase 5: - The phase 5 patch, d688_phase5_v1.patch, adds code to determine whether or not the user's classpath has the required XML classes and, if not, to throw a user-friendly(-ier) error message whenver the user attempts to use any of the XML operators. I inquired as to the best way to do this in the following thread: http://thread.gmane.org/gmane.comp.apache.db.derby.devel/25307/focus=25315 Dan suggested a) looking at the Derby code that loads modules, and b) adding a new utility method to the ClassInspector class. I looked at the module-loading code and it ultimately just makes a call to Class.forName() and ignores a module if that call throws a LinkageError or a ClassNotFoundException; see the getImplementations() method in BaseMonitor.java. So based on that I added a utility method to ClassInspector that does the same thing, except that it just returns "true" if the call to Class.forName() succeeds and "false" otherwise. I made the new method static because it doesn't rely on any state specific to ClassInspector and because it would have taken a good deal of searching for me to figure out how to instantiate an instance of ClassInspector correctly from within the XML datatype class. I ran quite a few tests manually with this change to verify that things are working, but I haven't figured out a good way to add corresponding tests to derbyall. So there are no such tests yet. If anyone has any suggestions that'd be great; otherwise I'm thinking it'd probably be best to figure that out as a follow-up task to this issue. After applying both patches, I ran xmlSuite on Windows 2000 with ibm142 and there were no failures (I still have to look into a diff in xmlBinding.java that occurs on Linux, but that was already reported by Bryan Pendleton and is not related to these specific patches). I'll run derbyall tonight as a sanity check, but in the meantime I'd appreciate any reviews/feedback. As mentioned above, the patches are intended to be applied in sequential order, phase 4 followed by phase 5, but that does not mean that they both have to be committed at the same time. So long as phase 4 is committed first, there shouldn't be any issues. > Enhancements to XML functionality to move toward XPath/XQuery support... > > > Key: DERBY-688 > URL: http://issues.apache.org/jira/browse/DERBY-688 > Project: Derby > Issue Type: Improvement > Components: JDBC, SQL >Reporter: A B > Assigned To: A B >Priority: Minor > Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat, > d688_phase1_v2.patch, d688_phase1_v3.patch, d688_phase2_v1_code.patch, > d688_phase2_v1_tests.patch, d688_phase2_v2_tests.patch, > d688_phase2_v3_tests.patch, d688_phase3_v1_code.patch, > d688_phase3_v1_tests.patch, d688_phase4_v1.patch, d688_phase5_v1.patch, > derbyXMLSpec.html > > > As of DERBY-334, Derby has some very basic support for XML that consists of > an XML datatype and three operators (XMLPAR
Re: [jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
David Van Couvering wrote: and thanks for the careful review and effort by Brian and Yip! Agreed! I am telling people "we have XML features" and I know it's something to do with XQuery and XPath, but I couldn't say what. Did these patches include documentation? Or is that forthcoming? Documentation is still forthcoming, as are some additional patches to finalize the work. There're still a couple of tasks to complete--such as enabling tests to run as part of derbyall--but the bulk of it is now in the codeline. Documentation is one of the remaining tasks. Army
Re: [jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
Wow, great work Army, and thanks for the careful review and effort by Brian and Yip! It's great to have this in. If I could only grok exactly what the feature is and how I might use it :) I am telling people "we have XML features" and I know it's something to do with XQuery and XPath, but I couldn't say what. Did these patches include documentation? Or is that forthcoming? David Bryan Pendleton (JIRA) wrote: [ http://issues.apache.org/jira/browse/DERBY-688?page=all ] Bryan Pendleton updated DERBY-688: -- Derby Info: (was: [Patch Available]) Committed d688_phase3_v1_code.patch and d688_phase3_v1_tests.patch to subversion as revision 429847. Committed the two patches together per Army's recommendation to commit these patches as a unit to avoid test diffs. Clearing the patch available flag because all the pending patches have now been committed. Thanks for all the hard work on this, Army! Enhancements to XML functionality to move toward XPath/XQuery support... Key: DERBY-688 URL: http://issues.apache.org/jira/browse/DERBY-688 Project: Derby Issue Type: Improvement Components: SQL, JDBC Reporter: A B Assigned To: A B Priority: Minor Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat, d688_phase1_v2.patch, d688_phase1_v3.patch, d688_phase2_v1_code.patch, d688_phase2_v1_tests.patch, d688_phase2_v2_tests.patch, d688_phase2_v3_tests.patch, d688_phase3_v1_code.patch, d688_phase3_v1_tests.patch, derbyXMLSpec.html As of DERBY-334, Derby has some very basic support for XML that consists of an XML datatype and three operators (XMLPARSE, XMLSERIALIZE, and XMLEXISTS). I would like to enhance this existing functionality and, by doing so, help to move Derby incrementally toward a more usable and more complete XPath/XQuery solution (with emphasis on "incrementally"). I have attached to this issue a document describing the particular changes that I am looking to make. At a high level, they consist of: 1) Making it easier to use the XML operators and datatype from within JDBC (ex. by implicit parsing/serialization of XML values). 2) Adding a new operator, XMLQUERY, to allow a user to retrieve the results of an XPath expression (instead of just determining whether or not the expression evaluates to an empty sequence, which is what XMLEXISTS does). 3) Making changes to the existing operators to line them up with the SQL/XML 2005 specification, and also to take steps toward my eventual hope of having support for XQuery (as opposed to just XPath) in Derby. If anyone has time and interest enough to look at the document and provide feedback, that'd be great...
[jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
[ http://issues.apache.org/jira/browse/DERBY-688?page=all ] Bryan Pendleton updated DERBY-688: -- Derby Info: (was: [Patch Available]) Committed d688_phase3_v1_code.patch and d688_phase3_v1_tests.patch to subversion as revision 429847. Committed the two patches together per Army's recommendation to commit these patches as a unit to avoid test diffs. Clearing the patch available flag because all the pending patches have now been committed. Thanks for all the hard work on this, Army! > Enhancements to XML functionality to move toward XPath/XQuery support... > > > Key: DERBY-688 > URL: http://issues.apache.org/jira/browse/DERBY-688 > Project: Derby > Issue Type: Improvement > Components: SQL, JDBC >Reporter: A B > Assigned To: A B >Priority: Minor > Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat, > d688_phase1_v2.patch, d688_phase1_v3.patch, d688_phase2_v1_code.patch, > d688_phase2_v1_tests.patch, d688_phase2_v2_tests.patch, > d688_phase2_v3_tests.patch, d688_phase3_v1_code.patch, > d688_phase3_v1_tests.patch, derbyXMLSpec.html > > > As of DERBY-334, Derby has some very basic support for XML that consists of > an XML datatype and three operators (XMLPARSE, XMLSERIALIZE, and XMLEXISTS). > I would like to enhance this existing functionality and, by doing so, help to > move Derby incrementally toward a more usable and more complete XPath/XQuery > solution (with emphasis on "incrementally"). > I have attached to this issue a document describing the particular changes > that I am looking to make. At a high level, they consist of: > 1) Making it easier to use the XML operators and datatype from within JDBC > (ex. by implicit parsing/serialization of XML values). > 2) Adding a new operator, XMLQUERY, to allow a user to retrieve the results > of an XPath expression (instead of just determining whether or not the > expression evaluates to an empty sequence, which is what XMLEXISTS does). > 3) Making changes to the existing operators to line them up with the SQL/XML > 2005 specification, and also to take steps toward my eventual hope of having > support for XQuery (as opposed to just XPath) in Derby. > If anyone has time and interest enough to look at the document and provide > feedback, that'd be great... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
[ http://issues.apache.org/jira/browse/DERBY-688?page=all ] A B updated DERBY-688: -- Attachment: d688_phase3_v1_code.patch d688_phase3_v1_tests.patch Attaching a "phase 3" patch that implements the XMLQUERY operator. The patch is in two parts: - d688_phase3_v1_code.patch - d688_phase3_v1_tests.patch When committed, though, *both* patches should be committed together in order to avoid test diffs. The SQL parsing/compile time logic was added as part of the phase 2 patch; this patch handles the execution-time logic by making the necessary Xalan calls to evaluate an expression and to retrieve the results. The phase 3 patch also adds logic to distinguish between two "types" of XML: XML(DOCUMENT(ANY)) and XML(SEQUENCE), as defined in the SQL/XML[2006] specification. The reason we need to distinguish between the two is that the result of evaluating an XML query expression against an XML document can be an arbitrary list of items including atomic values, attributes, etc.--i.e. a sequence of items that is *not* required to form a valid DOCUMENT node. For now, though, we only allow valid DOCUMENTs to be inserted into XML columns, so we have to be able to look at the results of the XMLQUERY operator to determine whether or not it's a valid DOCUMENT, and if not we disallow insertion of that result into an XML column. We can, however, keep the result transiently and pass it into other operations that accept an XML value (namely, XMLSERIALIZE, which a user can then use to retrieve the results in serialized form). NOTE: The phase 3 changes rely on the Phase 1 AND Phase 2 changes, which have not yet been committed. The phase 3 diffs are created w.r.t the phase 1 and phase 2 diffs and thus the phase 3 patches will not apply unless the other have been applied first. I ran derbyall on Red Hat Linux with ibm142 after applying all patches and there were no new failures. I also ran xmlSuite on Windows 2000 with ibm142 and all tests passed there, as well. I'm still looking for someone to review/commit any/all of these patches. Just to clarify, the following patches are up for review/commit and must be applied/committed in the order shown: 1. d688_phase1_v3.patch 2. d688_phase2_v1_code.patch 3. d688_phase2_v3_tests.patch 4. d688_phase3_v1_code.patch 5. d688_phase3_v1_tests.patch Thanks. > Enhancements to XML functionality to move toward XPath/XQuery support... > > > Key: DERBY-688 > URL: http://issues.apache.org/jira/browse/DERBY-688 > Project: Derby > Issue Type: Improvement > Components: JDBC, SQL >Reporter: A B > Assigned To: A B >Priority: Minor > Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat, > d688_phase1_v2.patch, d688_phase1_v3.patch, d688_phase2_v1_code.patch, > d688_phase2_v1_tests.patch, d688_phase2_v2_tests.patch, > d688_phase2_v3_tests.patch, d688_phase3_v1_code.patch, > d688_phase3_v1_tests.patch, derbyXMLSpec.html > > > As of DERBY-334, Derby has some very basic support for XML that consists of > an XML datatype and three operators (XMLPARSE, XMLSERIALIZE, and XMLEXISTS). > I would like to enhance this existing functionality and, by doing so, help to > move Derby incrementally toward a more usable and more complete XPath/XQuery > solution (with emphasis on "incrementally"). > I have attached to this issue a document describing the particular changes > that I am looking to make. At a high level, they consist of: > 1) Making it easier to use the XML operators and datatype from within JDBC > (ex. by implicit parsing/serialization of XML values). > 2) Adding a new operator, XMLQUERY, to allow a user to retrieve the results > of an XPath expression (instead of just determining whether or not the > expression evaluates to an empty sequence, which is what XMLEXISTS does). > 3) Making changes to the existing operators to line them up with the SQL/XML > 2005 specification, and also to take steps toward my eventual hope of having > support for XQuery (as opposed to just XPath) in Derby. > If anyone has time and interest enough to look at the document and provide > feedback, that'd be great... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
[ http://issues.apache.org/jira/browse/DERBY-688?page=all ] A B updated DERBY-688: -- Attachment: d688_phase2_v3_tests.patch Oops, I was wrong about the "phase 2" test patch--that was user error. I have too many uncommitted patches for this issue on my machine so I mixed them up. Attaching another version, d688_phase2_v3_tests.patch, that corrects my mix up. Sorry for the confusion. > Enhancements to XML functionality to move toward XPath/XQuery support... > > > Key: DERBY-688 > URL: http://issues.apache.org/jira/browse/DERBY-688 > Project: Derby > Issue Type: Improvement > Components: JDBC, SQL >Reporter: A B > Assigned To: A B >Priority: Minor > Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat, > d688_phase1_v2.patch, d688_phase1_v3.patch, d688_phase2_v1_code.patch, > d688_phase2_v1_tests.patch, d688_phase2_v2_tests.patch, > d688_phase2_v3_tests.patch, derbyXMLSpec.html > > > As of DERBY-334, Derby has some very basic support for XML that consists of > an XML datatype and three operators (XMLPARSE, XMLSERIALIZE, and XMLEXISTS). > I would like to enhance this existing functionality and, by doing so, help to > move Derby incrementally toward a more usable and more complete XPath/XQuery > solution (with emphasis on "incrementally"). > I have attached to this issue a document describing the particular changes > that I am looking to make. At a high level, they consist of: > 1) Making it easier to use the XML operators and datatype from within JDBC > (ex. by implicit parsing/serialization of XML values). > 2) Adding a new operator, XMLQUERY, to allow a user to retrieve the results > of an XPath expression (instead of just determining whether or not the > expression evaluates to an empty sequence, which is what XMLEXISTS does). > 3) Making changes to the existing operators to line them up with the SQL/XML > 2005 specification, and also to take steps toward my eventual hope of having > support for XQuery (as opposed to just XPath) in Derby. > If anyone has time and interest enough to look at the document and provide > feedback, that'd be great... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
[ http://issues.apache.org/jira/browse/DERBY-688?page=all ] A B updated DERBY-688: -- Attachment: d688_phase2_v2_tests.patch Attaching a new version of the "phase2_tests" patch since the first version doesn't appear to apply cleanly. In particular, the DerbyNetClient/xml_general.out master file looks to have conflicts. The new patch is "d688_phase2_v2_tests.patch". > Enhancements to XML functionality to move toward XPath/XQuery support... > > > Key: DERBY-688 > URL: http://issues.apache.org/jira/browse/DERBY-688 > Project: Derby > Issue Type: Improvement > Components: SQL, JDBC >Reporter: A B > Assigned To: A B >Priority: Minor > Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat, > d688_phase1_v2.patch, d688_phase1_v3.patch, d688_phase2_v1_code.patch, > d688_phase2_v1_tests.patch, d688_phase2_v2_tests.patch, derbyXMLSpec.html > > > As of DERBY-334, Derby has some very basic support for XML that consists of > an XML datatype and three operators (XMLPARSE, XMLSERIALIZE, and XMLEXISTS). > I would like to enhance this existing functionality and, by doing so, help to > move Derby incrementally toward a more usable and more complete XPath/XQuery > solution (with emphasis on "incrementally"). > I have attached to this issue a document describing the particular changes > that I am looking to make. At a high level, they consist of: > 1) Making it easier to use the XML operators and datatype from within JDBC > (ex. by implicit parsing/serialization of XML values). > 2) Adding a new operator, XMLQUERY, to allow a user to retrieve the results > of an XPath expression (instead of just determining whether or not the > expression evaluates to an empty sequence, which is what XMLEXISTS does). > 3) Making changes to the existing operators to line them up with the SQL/XML > 2005 specification, and also to take steps toward my eventual hope of having > support for XQuery (as opposed to just XPath) in Derby. > If anyone has time and interest enough to look at the document and provide > feedback, that'd be great... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
[ http://issues.apache.org/jira/browse/DERBY-688?page=all ] A B updated DERBY-688: -- Attachment: d688_phase2_v1_code.patch d688_phase2_v1_tests.patch Attaching a "phase 2" patch that does the following: - Adds syntax and binding logic for XMLQUERY operator - Updates syntax for XMLEXISTS to match XMLQUERY and does some refactoring so that the two can share code. The most user-visible change in this area is the change from BY VALUE to BY REF syntax, which required test updates and corresponding master file updates. Also as part of these changes I've rewritten some of the parse logic to make it more easily extendible to a broader XMLQUERY syntax (as defined in SQL/XML[2006]) when/if Derby supports the broader syntax (esp. optional context items and variable bindings) in the future. - These phase 2 patches do *not* actually implement the XMLQUERY operator yet; they just allow parsing and binding of the operator. Execution-time logic will come in a subsequent patch. For the sake of review I've separated the Phase 2 patches into two different parts: d688_phase2_v1_code.patch: The code changes. d688_phase2_v1_tests.patch: The test/master changes. When committed, though, *both* patches should be committed together in order to avoid test diffs. NOTE: The phase 2 changes rely on the Phase 1 changes, which have not yet been committed. The phase 2 diffs are created w.r.t the phase 1 diffs and thus the phase 2 patches will not apply unless the phase 1 patch has been applied first. I'm still looking for someone to review/commit the phase 1patch (d688_phase1_v3.patch). I ran derbyall on Windows 2000 with ibm142 after applying the phase 1 patch AND both phase 2 patches; there were no new failures. I also ran xmlSuite and all XML-specific tests passed. If anyone has time to review the phase 1 and/or phase 2 patches, I'd be grateful. > Enhancements to XML functionality to move toward XPath/XQuery support... > > > Key: DERBY-688 > URL: http://issues.apache.org/jira/browse/DERBY-688 > Project: Derby > Issue Type: Improvement > Components: JDBC, SQL >Reporter: A B > Assigned To: A B >Priority: Minor > Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat, > d688_phase1_v2.patch, d688_phase1_v3.patch, d688_phase2_v1_code.patch, > d688_phase2_v1_tests.patch, derbyXMLSpec.html > > > As of DERBY-334, Derby has some very basic support for XML that consists of > an XML datatype and three operators (XMLPARSE, XMLSERIALIZE, and XMLEXISTS). > I would like to enhance this existing functionality and, by doing so, help to > move Derby incrementally toward a more usable and more complete XPath/XQuery > solution (with emphasis on "incrementally"). > I have attached to this issue a document describing the particular changes > that I am looking to make. At a high level, they consist of: > 1) Making it easier to use the XML operators and datatype from within JDBC > (ex. by implicit parsing/serialization of XML values). > 2) Adding a new operator, XMLQUERY, to allow a user to retrieve the results > of an XPath expression (instead of just determining whether or not the > expression evaluates to an empty sequence, which is what XMLEXISTS does). > 3) Making changes to the existing operators to line them up with the SQL/XML > 2005 specification, and also to take steps toward my eventual hope of having > support for XQuery (as opposed to just XPath) in Derby. > If anyone has time and interest enough to look at the document and provide > feedback, that'd be great... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
[ http://issues.apache.org/jira/browse/DERBY-688?page=all ] A B updated DERBY-688: -- Attachment: d688_phase1_v3.patch Attaching an updated patch synced with the latest codeline. The only difference between _v2 and _v3 is that the client master file for lang/xml_general.sql has been updated to reflect the latest checkin for DERBY-1029. > Enhancements to XML functionality to move toward XPath/XQuery support... > > > Key: DERBY-688 > URL: http://issues.apache.org/jira/browse/DERBY-688 > Project: Derby > Issue Type: Improvement > Components: JDBC, SQL >Reporter: A B > Assigned To: A B >Priority: Minor > Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat, > d688_phase1_v2.patch, d688_phase1_v3.patch, derbyXMLSpec.html > > > As of DERBY-334, Derby has some very basic support for XML that consists of > an XML datatype and three operators (XMLPARSE, XMLSERIALIZE, and XMLEXISTS). > I would like to enhance this existing functionality and, by doing so, help to > move Derby incrementally toward a more usable and more complete XPath/XQuery > solution (with emphasis on "incrementally"). > I have attached to this issue a document describing the particular changes > that I am looking to make. At a high level, they consist of: > 1) Making it easier to use the XML operators and datatype from within JDBC > (ex. by implicit parsing/serialization of XML values). > 2) Adding a new operator, XMLQUERY, to allow a user to retrieve the results > of an XPath expression (instead of just determining whether or not the > expression evaluates to an empty sequence, which is what XMLEXISTS does). > 3) Making changes to the existing operators to line them up with the SQL/XML > 2005 specification, and also to take steps toward my eventual hope of having > support for XQuery (as opposed to just XPath) in Derby. > If anyone has time and interest enough to look at the document and provide > feedback, that'd be great... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
[ http://issues.apache.org/jira/browse/DERBY-688?page=all ] A B updated DERBY-688: -- Derby Info: [Patch Available] > Enhancements to XML functionality to move toward XPath/XQuery support... > > > Key: DERBY-688 > URL: http://issues.apache.org/jira/browse/DERBY-688 > Project: Derby > Issue Type: Improvement > Components: SQL, JDBC >Reporter: A B > Assigned To: A B >Priority: Minor > Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat, > d688_phase1_v2.patch, derbyXMLSpec.html > > > As of DERBY-334, Derby has some very basic support for XML that consists of > an XML datatype and three operators (XMLPARSE, XMLSERIALIZE, and XMLEXISTS). > I would like to enhance this existing functionality and, by doing so, help to > move Derby incrementally toward a more usable and more complete XPath/XQuery > solution (with emphasis on "incrementally"). > I have attached to this issue a document describing the particular changes > that I am looking to make. At a high level, they consist of: > 1) Making it easier to use the XML operators and datatype from within JDBC > (ex. by implicit parsing/serialization of XML values). > 2) Adding a new operator, XMLQUERY, to allow a user to retrieve the results > of an XPath expression (instead of just determining whether or not the > expression evaluates to an empty sequence, which is what XMLEXISTS does). > 3) Making changes to the existing operators to line them up with the SQL/XML > 2005 specification, and also to take steps toward my eventual hope of having > support for XQuery (as opposed to just XPath) in Derby. > If anyone has time and interest enough to look at the document and provide > feedback, that'd be great... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
[ http://issues.apache.org/jira/browse/DERBY-688?page=all ] A B updated DERBY-688: -- Attachment: d688_phase1_v2.patch Attaching a second patch, d688_phase1_v2.patch, that makes the appropriate changes to ensure that the new classes (esp. SqlXmlExecutor) are included in the jar builds. For the thread indicating what changes to make, see: http://article.gmane.org/gmane.comp.apache.db.derby.devel/24135 The patch is the same as phase1_v1 except that it has the following additional diffs: Index: tools/jar/extraDBMSclasses.properties === --- tools/jar/extraDBMSclasses.properties (revision 423655) +++ tools/jar/extraDBMSclasses.properties (working copy) @@ -84,4 +84,4 @@ derby.module.store.urlf=org.apache.derby.impl.io.URLFile derby.module.store.cpf=org.apache.derby.impl.io.CPFile -derby.module.xml.typec=org.apache.derby.impl.sql.compile.XMLTypeCompiler +derby.module.xml.sqlxmle=org.apache.derby.impl.sql.execute.SqlXmlExecutor Index: tools/jar/extraDBMStypes.properties === --- tools/jar/extraDBMStypes.properties (revision 423655) +++ tools/jar/extraDBMStypes.properties (working copy) @@ -13,3 +13,4 @@ derby.module.type.m=org.apache.derby.impl.sql.compile.RefTypeCompiler derby.module.type.n=org.apache.derby.impl.sql.compile.TimeTypeCompiler derby.module.type.o=org.apache.derby.impl.sql.compile.TimestampTypeCompiler +derby.module.type.p=org.apache.derby.impl.sql.compile.XMLTypeCompiler I ran the XML test suite (xmlSuite) with classes and with jars using IBM 1.4.2 and they passed in both cases. Reviews/feedback are appreciated. > Enhancements to XML functionality to move toward XPath/XQuery support... > > > Key: DERBY-688 > URL: http://issues.apache.org/jira/browse/DERBY-688 > Project: Derby > Issue Type: Improvement > Components: SQL, JDBC >Reporter: A B > Assigned To: A B >Priority: Minor > Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat, > d688_phase1_v2.patch, derbyXMLSpec.html > > > As of DERBY-334, Derby has some very basic support for XML that consists of > an XML datatype and three operators (XMLPARSE, XMLSERIALIZE, and XMLEXISTS). > I would like to enhance this existing functionality and, by doing so, help to > move Derby incrementally toward a more usable and more complete XPath/XQuery > solution (with emphasis on "incrementally"). > I have attached to this issue a document describing the particular changes > that I am looking to make. At a high level, they consist of: > 1) Making it easier to use the XML operators and datatype from within JDBC > (ex. by implicit parsing/serialization of XML values). > 2) Adding a new operator, XMLQUERY, to allow a user to retrieve the results > of an XPath expression (instead of just determining whether or not the > expression evaluates to an empty sequence, which is what XMLEXISTS does). > 3) Making changes to the existing operators to line them up with the SQL/XML > 2005 specification, and also to take steps toward my eventual hope of having > support for XQuery (as opposed to just XPath) in Derby. > If anyone has time and interest enough to look at the document and provide > feedback, that'd be great... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
[ http://issues.apache.org/jira/browse/DERBY-688?page=all ] A B updated DERBY-688: -- Attachment: d688_phase1_v1.patch d688_phase1_v1.stat Attaching a "phase 1" patch, d688_phase1_v1.patch, for this issue that does the following: 1. Reorganizes XML-specific code as follows: - Moves all code that relies on JAXP and Xalan classes out of XML.java and into a new class, SqlXmlUtil.java. See comments at the beginning of SqlXmlUtil for an explanation of why this was done. - Creates a new class, SqlXmlExecutor, in the impl.sql.execute package that serves as the class on which all XML operator calls are generated. Ex. for XMLEXISTS, instead of generating: .XMLExists(, xmlOperand) we now generate: .XMLSerialize(, xmlOperand) Along with making the code cleaner by allowing all XML operator calls to be defined in the same class, this new class has other benefits, as well--see comments at the beginning of SqlXmlExecutor for more of an explanation. 2. Changes implementation of XPath from XSLT processing to the low-level Xalan API, which is faster, more flexible, and better for implementation of the XMLQUERY operator (the XMLQUERY operator will be coming in subsequent phases). Note that as part of this change I've removed the dependency on an explicit declaration of Xerces as the parser; Derby will now pick up the parser from the JVM (i.e. this patch resolves DERBY-567). 3. Makes a small change to the XMLEXISTS operator to bring it more in line with SQL/XML spec. More specifically, the query expression that is specified must now be a string literal; parameters and other expressions are not allowed. 4. Updates the XML test and master files (lang/xml_general.sql and lang/xmlBinding.java) to bring them in sync with the latest Derby codeline. Since the XML tests are not (yet) run as part of derbyall, the master files need to be updated to reflect some client/server changes that have gone into the codeline for 10.2 (for example, server pre-fetching behavior). Of these changes the only one that is a functional change is #3; the rest are internal changes that should have no external effect on the way the XML operators work. They are important, though, because they lay the groundwork for subsequent work as described in the HTML document for this issue. I did try to break the patch up into the different pieces outlined above, but after two days of trying to undo my work for the sake of elegant, smaller patches, I decided the patch is acceptable as it is (believe it or not, I did actually take a lot out of it already). I'll shoot for incremental development and smaller patches as I work on the subquent changes for this issue... I ran derbyall with an earlier version of this patch and there were no new failures. The latest version (which is the d688_phase1_v1.patch that I'm posting here) has some updates, though, so I need to run derbyall again. In the meantime, reviews/comments/feedback would be appreciated. > Enhancements to XML functionality to move toward XPath/XQuery support... > > > Key: DERBY-688 > URL: http://issues.apache.org/jira/browse/DERBY-688 > Project: Derby > Issue Type: Improvement > Components: JDBC, SQL >Reporter: A B > Assigned To: A B >Priority: Minor > Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat, > derbyXMLSpec.html > > > As of DERBY-334, Derby has some very basic support for XML that consists of > an XML datatype and three operators (XMLPARSE, XMLSERIALIZE, and XMLEXISTS). > I would like to enhance this existing functionality and, by doing so, help to > move Derby incrementally toward a more usable and more complete XPath/XQuery > solution (with emphasis on "incrementally"). > I have attached to this issue a document describing the particular changes > that I am looking to make. At a high level, they consist of: > 1) Making it easier to use the XML operators and datatype from within JDBC > (ex. by implicit parsing/serialization of XML values). > 2) Adding a new operator, XMLQUERY, to allow a user to retrieve the results > of an XPath expression (instead of just determining whether or not the > expression evaluates to an empty sequence, which is what XMLEXISTS does). > 3) Making changes to the existing operators to line them up with the SQL/XML > 2005 specification, and also to take steps toward my eventual hope of having > support for XQuery (as opposed to just XPath) in Derby. > If anyone has time and interest enough to look at the document and provide > feedback, that'd be great... -- This message is automatically generated by JIRA. - If you thin
[jira] Updated: (DERBY-688) Enhancements to XML functionality to move toward XPath/XQuery support...
[ http://issues.apache.org/jira/browse/DERBY-688?page=all ] A B updated DERBY-688: -- Attachment: derbyXMLSpec.html > Enhancements to XML functionality to move toward XPath/XQuery support... > > > Key: DERBY-688 > URL: http://issues.apache.org/jira/browse/DERBY-688 > Project: Derby > Type: Improvement > Components: JDBC, SQL > Reporter: A B > Assignee: A B > Priority: Minor > Attachments: derbyXMLSpec.html > > As of DERBY-334, Derby has some very basic support for XML that consists of > an XML datatype and three operators (XMLPARSE, XMLSERIALIZE, and XMLEXISTS). > I would like to enhance this existing functionality and, by doing so, help to > move Derby incrementally toward a more usable and more complete XPath/XQuery > solution (with emphasis on "incrementally"). > I have attached to this issue a document describing the particular changes > that I am looking to make. At a high level, they consist of: > 1) Making it easier to use the XML operators and datatype from within JDBC > (ex. by implicit parsing/serialization of XML values). > 2) Adding a new operator, XMLQUERY, to allow a user to retrieve the results > of an XPath expression (instead of just determining whether or not the > expression evaluates to an empty sequence, which is what XMLEXISTS does). > 3) Making changes to the existing operators to line them up with the SQL/XML > 2005 specification, and also to take steps toward my eventual hope of having > support for XQuery (as opposed to just XPath) in Derby. > If anyone has time and interest enough to look at the document and provide > feedback, that'd be great... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
