Re: [basex-talk] Destination of result-document from XSLT module
Marc,
Thanks for mentioning that! I am running into problems with whitespace being
lost on input to the transformation too. For example, an XSLT contains inside a
template:
within the past days
The output has lost the whitespace around the element:
within the past15days
Set CHOP false before the transformation helps, for example:
(# db:chop false #) { let $xslt := doc('stylesheet.xsl') return
xslt:transform-text($doc, $xslt) }
Provides the expected output:
within the past 15 days
I will have to watch for other instances of lost whitespace in documents.
In regard to my original question about setting Base Output URI, the solution
I've chosen for now is to pass a URI path to a folder as a parameter into the
stylesheet, and update each xsl:result-document href attribute to include the
parameter value.
Thanks,
Vincent
-Original Message-
From: basex-talk-boun...@mailman.uni-konstanz.de
[mailto:basex-talk-boun...@mailman.uni-konstanz.de] On Behalf Of Marc
Sent: Monday, August 03, 2015 5:49 PM
Cc: basex-talk@mailman.uni-konstanz.de
Subject: Re: [basex-talk] Destination of result-document from XSLT module
Thank Vincent,
But it the input of the xslt I want to control.
I'm using BaseX with the CHOP false because I use text documentation and space
is important at the end or the begining of an element, and when I try to use
xslt (with baseX 7) it serialize with indent true the input of my xslt, so I
lost the space.
Marc
Le 03/08/2015 23:07, Lizzi, Vincent a écrit :
> Thanks, Max and Andy. The XQuery 3.1 transform function looks promising.
>
> Marc, For controlling the serialization of XSLT output, you can specify
> serialization for the XSLT within the XSLT itself using xsl:output, run the
> XSLT using the xslt:transform-text, and then write the output to a file.
>
> file:write-text('file.txt', xslt:transform-text($doc, $xslt))
>
> I'm not sure if this will do what you need, but so far it has been working
> for me.
>
> Vincent
>
>
>
> -Original Message-
> From: basex-talk-boun...@mailman.uni-konstanz.de
> [mailto:basex-talk-boun...@mailman.uni-konstanz.de] On Behalf Of Marc
> Sent: Monday, August 03, 2015 4:53 PM
> Cc: basex-talk@mailman.uni-konstanz.de
> Subject: Re: [basex-talk] Destination of result-document from XSLT
> module
>
> Hi,
> I have an other problem is to control the serialization of the node passed to
> the transform function.
> I don't see how to control it.
> Marc
> Le 03/08/2015 22:32, Andy Bunce a écrit :
> > Hi Max,
> > This sounds like a good thing.
> > Another solution to the result-document issue might be to implement
> > the XQuery 3.1 transform function [1]
> >
> > /Andy
> > [1] http://www.w3.org/TR/xpath-functions-31/#func-transform
> >
> > On 3 August 2015 at 20:54, Max Goltzsche > > wrote:
> >
> > Hello Vincent,
> >
> > besides an URI resolver I also want to set XSLT 2.0's output
> > destination in BaseX.
> > Currently as you can see in BaseX' xslt:transform and
> > xslt:transform-text implementation in
> >
> > https://github.com/BaseXdb/basex/blob/master/basex-core/src/main/java/org/basex/query/func/xslt/XsltTransform.java
> > there is neither an URIResolver or OutputURIResolver set on the
> > Transformer nor a destination systemId set on its StreamResult. Thus
> > Saxon resolves output paths relative to your Java process' working
> > directory (in fact all relative XSL include, import, document and
> > collection paths in the XSLT passed to the transform method).
> > Unfortunately to change this behaviour the XsltTransform class must
> > be enhanced.
> > If you need a quicker* solution for your problem you may have to
> > build your own BaseX transform Java Module based on the
> > XsltTransform class setting at least the StreamResult's systemId I
> > think.
> > I will be also working on this the next evenings.
> >
> > best regards,
> > Max
> >
> >
> > On 03.08.2015 20:54, Lizzi, Vincent wrote:
> >>
> >> I’m trying to use the XSLT Module in BaseX 8.2.3 with Saxon 9.6 to
> >> run an XSLT that produces several output documents using
> >> xsl:result-document. I’m having trouble setting the location of
> >> the output documents. I want to have xsl:result-document create
> >> the output documents in a temporary folder because the documents
> >> need to be zipped together.
> >>
> >> __ __
> >>
> >> According to Saxon’s documentation, the a relative path in the
> >> href attribute of xsl:result-document will be resolved using
> >> either the path of the Destination, or the current directory. The
> >> XSLT Module does not appear have a way to provide a path for a
> >> destination document. What I’m seeing is that the result
> >> documents are created in BaseX’s home directory. The XSLT works as
> >> expected when run using Saxon
Re: [basex-talk] Performance with joins?
> Executes in 257.13 ms. But it shouldn't be necessary to explicitly specify > the text() node, right (or would even be bad practice to do so[1])? The results of both variants should be equivalent indeed (unless a BIB_ID does not have more than one text node). I wouldn't call it bad practice to explicitly address the text node. In this particular case, it helps the optimizer to detect that the value index can be applied. Cheers, Christian
Re: [basex-talk] Performance with joins?
Adding a text() step to the predicate does dramatically reduce the
execution time.
This query:
declare namespace marc="http://www.loc.gov/MARC21/slim";;
for $m in collection(
"latin_hold_20150730"
)/marc:collection/marc:record,
$r in collection(
"latin_hold_20150730"
)/root/row[BIB_ID/text() = $m/marc:controlfield[@tag = '001']]
group by $key := $r/ITEM_ID
return
{
$m/marc:datafield[@tag = '245']/marc:subfield[@code = 'a']/text()
}
Executes in 257.13 ms. But it shouldn't be necessary to explicitly specify
the text() node, right (or would even be bad practice to do so[1])?
Thanks,
Tim
[1]
http://stackoverflow.com/questions/12970162/xquery-join-query#comment-17592189
--
Tim A. Thompson
Metadata Librarian (Spanish/Portuguese Specialty)
Princeton University Library
On Tue, Aug 4, 2015 at 8:25 AM, Tim Thompson wrote:
> I was mistaken; I hadn't enabled indexes on this database. However, after
> creating attribute and text indexes, the query actually seems to take
> longer to execute (483622.95 ms on last run), although the query plan
> itself doesn't seem to have changed:
>
> Compiling:
> - pre-evaluating collection("latin_hold_20150730")
> - pre-evaluating collection("latin_hold_20150730")
> Query:
> declare namespace marc="http://www.loc.gov/MARC21/slim";; for $m in
> collection( "latin_hold_20150730" )/marc:collection/marc:record, $r in
> collection( "latin_hold_20150730" )/root/row[BIB_ID =
> $m/marc:controlfield[@tag = '001']] group by $key := $r/ITEM_ID return
> { $m/marc:datafield[@tag = '245']/marc:subfield[@code =
> 'a']/string() }
> Optimized Query:
> for $m_0 in (db:open-pre("latin_hold_20150730",0),
> ...)/marc:collection/marc:record for $r_1 in
> (db:open-pre("latin_hold_20150730",0), ...)/root/row[(BIB_ID =
> $m_0/marc:controlfield[(@tag = "001")])] let (: post-group :) $m_4 := $m_0
> group by $key_2 := $r_1/ITEM_ID return element test { (attribute n {
> ($key_2) }, $m_4/marc:datafield[(@tag = "245")]/marc:subfield[(@code =
> "a")]/string()) }
> Result:
> - Hit(s): 7587 Items
> - Updated: 0 Items
> - Printed: 505 KB
> - Read Locking: local [latin_hold_20150730]
> - Write Locking: none
> Timing:
> - Parsing: 1.04 ms
> - Compiling: 0.86 ms
> - Evaluating: 483604.85 ms
> - Printing: 16.2 ms
> - Total Time: 483622.95 ms
>
> Query plan:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
> Tim A. Thompson
> Metadata Librarian (Spanish/Portuguese Specialty)
> Princeton University Library
>
>
> On Tue, Aug 4, 2015 at 3:23 AM, Christian Grün
> wrote:
>
>> Dear Tim,
>>
>> The query plan indicates that no index is applied. Your query may be
>> evaluated faster when rewriting "BIB_ID" to "BIB_ID/text()". I will
>> see if this can automatically be done by the query compiler.
>>
>> Best,
>> Christian
>>
>>
>> On Tue, Aug 4, 2015 at 4:42 AM, Tim Thompson wrote:
>> > Hello,
>> >
>> > I have a database containing two resources/documents: they both
>> represent
>> > the same set of library catalog records (7728 "records" in each), but
>> they
>> > each contain different data that I want to join.
>> >
>> > The first resource looks like this:
>> >
>> > > > xmlns:marc="http://www.loc.gov/MARC21/slim";>
>> >
>> > 01225cam a2200373Mi 4500
>> > 5323084
>> > > > ind1="1"
>> > ind2="4"
>> > tag="245">
>> > Els teleclubs a les illes
>> Balears
>> > :
>> >
>> >
>> >
>> > 01225cam a2200373Mi 4500
>> > 5323084
>> > > > ind1="1"
>> > ind2="4"
>> > tag="245">
>> > Els teleclubs a les illes
>> Balears
>> > :
>> >
>> >
>> >
>> > 00818cam a2200241Mi 4500
>> > 6310976
>> > > > ind1="0"
>> > ind2="0"
>> > tag="245">
>> > Diccionari manual de
Re: [basex-talk] Performance with joins?
I was mistaken; I hadn't enabled indexes on this database. However, after
creating attribute and text indexes, the query actually seems to take
longer to execute (483622.95 ms on last run), although the query plan
itself doesn't seem to have changed:
Compiling:
- pre-evaluating collection("latin_hold_20150730")
- pre-evaluating collection("latin_hold_20150730")
Query:
declare namespace marc="http://www.loc.gov/MARC21/slim";; for $m in
collection( "latin_hold_20150730" )/marc:collection/marc:record, $r in
collection( "latin_hold_20150730" )/root/row[BIB_ID =
$m/marc:controlfield[@tag = '001']] group by $key := $r/ITEM_ID return
{ $m/marc:datafield[@tag = '245']/marc:subfield[@code =
'a']/string() }
Optimized Query:
for $m_0 in (db:open-pre("latin_hold_20150730",0),
...)/marc:collection/marc:record for $r_1 in
(db:open-pre("latin_hold_20150730",0), ...)/root/row[(BIB_ID =
$m_0/marc:controlfield[(@tag = "001")])] let (: post-group :) $m_4 := $m_0
group by $key_2 := $r_1/ITEM_ID return element test { (attribute n {
($key_2) }, $m_4/marc:datafield[(@tag = "245")]/marc:subfield[(@code =
"a")]/string()) }
Result:
- Hit(s): 7587 Items
- Updated: 0 Items
- Printed: 505 KB
- Read Locking: local [latin_hold_20150730]
- Write Locking: none
Timing:
- Parsing: 1.04 ms
- Compiling: 0.86 ms
- Evaluating: 483604.85 ms
- Printing: 16.2 ms
- Total Time: 483622.95 ms
Query plan:
--
Tim A. Thompson
Metadata Librarian (Spanish/Portuguese Specialty)
Princeton University Library
On Tue, Aug 4, 2015 at 3:23 AM, Christian Grün
wrote:
> Dear Tim,
>
> The query plan indicates that no index is applied. Your query may be
> evaluated faster when rewriting "BIB_ID" to "BIB_ID/text()". I will
> see if this can automatically be done by the query compiler.
>
> Best,
> Christian
>
>
> On Tue, Aug 4, 2015 at 4:42 AM, Tim Thompson wrote:
> > Hello,
> >
> > I have a database containing two resources/documents: they both represent
> > the same set of library catalog records (7728 "records" in each), but
> they
> > each contain different data that I want to join.
> >
> > The first resource looks like this:
> >
> > > xmlns:marc="http://www.loc.gov/MARC21/slim";>
> >
> > 01225cam a2200373Mi 4500
> > 5323084
> > > ind1="1"
> > ind2="4"
> > tag="245">
> > Els teleclubs a les illes Balears
> > :
> >
> >
> >
> > 01225cam a2200373Mi 4500
> > 5323084
> > > ind1="1"
> > ind2="4"
> > tag="245">
> > Els teleclubs a les illes Balears
> > :
> >
> >
> >
> > 00818cam a2200241Mi 4500
> > 6310976
> > > ind1="0"
> > ind2="0"
> > tag="245">
> > Diccionari manual de sinònims i
> > antònims de la llengua catalana /
> >
> >
> >
> >
> > The second one looks like this:
> >
> >
> >
> > cat
> > 5912416
> > 5323084
> > MXBKSMX
> >
> >
> > cat
> > 5912416
> > 5323084
> > PUVILL
> >
> >
> > cat
> > 5935043
> > 6310976
> > PUVILL
> >
> >
> >
> > I have a simple query that joins the two using the value of the
> > marc:controlfield[@tag = '001'] from resource 1 and the BIB_ID from
> resource
> > 2.
> >
> > The query:
> >
> > declare namespace marc="http://www.loc.gov/MARC21/slim";;
> >
> > for $m in collection(
> > "latin_hold_20150730"
> > )/marc:collection/marc:record,
> > $r in collection(
> > "latin_hold_20150730"
> > )/root/row[BIB_ID = $m/marc:controlfield[@tag = '001']]
> > group by $key := $r/ITEM_ID
> >
> > return
> > {
> > $m/marc:datafield[@tag = '245']/marc:subfield[@code = 'a']/string()
> > }
> >
> > When I run this using Saxon (substituting fn:doc for fn:collection),
Re: [basex-talk] Round tripping JSON
Yes, I think it's a CACHERESTXQ bug. The handler I posted earlier
triggers it. Not sure what exactly triggers it.
Yes, if only we had a time machine. Kidding aside, REST
services/microservices/UI clients/Javascript frameworks all are
pushing on REST. I was triggered by recent developments by
Facebook/Netflix and recently summarized in a talk on InfoQ called
"Demand Driven architecture"
(https://qconnewyork.com/ny2015/presentation/demand-driven-architecture).
I think that it solves issues for UI clients. I'm looking into which
forum to raise the issue or if I should just do the pragmatic thing,
use POST, and shut up. I'm a bit scared for "religuous fanatism". ;-)
On Tue, Aug 4, 2015 at 12:01 PM, Christian Grün
wrote:
>> I figured it out. The way to repro is with the function I posted
>> earlier. Start the server but with CACHERESTXQ=true. Test the
>> function: works fine. Change the handler function/file and save it.
>> Retry the POST, boom. NPE. Switching CACHERESTXQ=false no fixes this.
>
> I see; so it seems to be a CACHERESTXQ bug, right? (Andy: thanks as well).
>
>> Maybe I get crucified for saying this but I think we should
>> allow GET with body.
>
> Maybe even Roy agrees with you today, but we should probably have
> intervened and fixed that 20 years ago ;) We could obviously weaken
> the RESTXQ constraints, but it may be advisable to first check out if
> there has been a similar discussion on JAX-RS.
>
>
>
> And yes, it's very useful and can simplify
>> clients that need to specify complex queries to a REST server. I stop
>> ranting now, I probably should take this somewhere else but I'm still
>> building up courage to do so ;-)
>>
>>
>>
>> On Tue, Aug 4, 2015 at 11:05 AM, Christian Grün
>> wrote:
>>> All existing test cases seem to work, and I didn't come across this in
>>> my simple queries. Could you please provide me with a function and
>>> call that triggers the exception?
>>>
>>>
>>> On Tue, Aug 4, 2015 at 11:00 AM, Marc van Grootel
>>> wrote:
I've double checked the version and manually downloaded latest. I'm
now on BaseX 8.3 beta e13a5f0 which I assume is correct.
Not sure what's going on but in most cases (with format=map) gives me
a NPE (and {"foo": "bar"}):
Unexpected error: Improper use? Potential bug? Your feedback is welcome:
Contact: basex-talk@mailman.uni-konstanz.de
Version: BaseX 8.3 beta e13a5f0
Java: Oracle Corporation, 1.7.0_25
OS: Windows 7, amd64
Stack Trace:
java.lang.NullPointerException
at org.basex.http.restxq.RestXqFunction.parse(RestXqFunction.java:124)
at org.basex.http.restxq.RestXqModule.process(RestXqModule.java:100)
at org.basex.http.restxq.RestXqFunction.process(RestXqFunction.java:109)
at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:44)
at org.basex.http.BaseXServlet.service(BaseXServlet.java:64)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:370)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
at
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
at
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool
Re: [basex-talk] Round tripping JSON
> I figured it out. The way to repro is with the function I posted
> earlier. Start the server but with CACHERESTXQ=true. Test the
> function: works fine. Change the handler function/file and save it.
> Retry the POST, boom. NPE. Switching CACHERESTXQ=false no fixes this.
I see; so it seems to be a CACHERESTXQ bug, right? (Andy: thanks as well).
> Maybe I get crucified for saying this but I think we should
> allow GET with body.
Maybe even Roy agrees with you today, but we should probably have
intervened and fixed that 20 years ago ;) We could obviously weaken
the RESTXQ constraints, but it may be advisable to first check out if
there has been a similar discussion on JAX-RS.
And yes, it's very useful and can simplify
> clients that need to specify complex queries to a REST server. I stop
> ranting now, I probably should take this somewhere else but I'm still
> building up courage to do so ;-)
>
>
>
> On Tue, Aug 4, 2015 at 11:05 AM, Christian Grün
> wrote:
>> All existing test cases seem to work, and I didn't come across this in
>> my simple queries. Could you please provide me with a function and
>> call that triggers the exception?
>>
>>
>> On Tue, Aug 4, 2015 at 11:00 AM, Marc van Grootel
>> wrote:
>>> I've double checked the version and manually downloaded latest. I'm
>>> now on BaseX 8.3 beta e13a5f0 which I assume is correct.
>>>
>>> Not sure what's going on but in most cases (with format=map) gives me
>>> a NPE (and {"foo": "bar"}):
>>>
>>> Unexpected error: Improper use? Potential bug? Your feedback is welcome:
>>> Contact: basex-talk@mailman.uni-konstanz.de
>>> Version: BaseX 8.3 beta e13a5f0
>>> Java: Oracle Corporation, 1.7.0_25
>>> OS: Windows 7, amd64
>>> Stack Trace:
>>> java.lang.NullPointerException
>>> at org.basex.http.restxq.RestXqFunction.parse(RestXqFunction.java:124)
>>> at org.basex.http.restxq.RestXqModule.process(RestXqModule.java:100)
>>> at org.basex.http.restxq.RestXqFunction.process(RestXqFunction.java:109)
>>> at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:44)
>>> at org.basex.http.BaseXServlet.service(BaseXServlet.java:64)
>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>>> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
>>> at
>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503)
>>> at
>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
>>> at
>>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
>>> at
>>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
>>> at
>>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
>>> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429)
>>> at
>>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
>>> at
>>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
>>> at
>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
>>> at
>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
>>> at org.eclipse.jetty.server.Server.handle(Server.java:370)
>>> at
>>> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
>>> at
>>> org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982)
>>> at
>>> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043)
>>> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
>>> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
>>> at
>>> org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
>>> at
>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
>>> at
>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
>>> at
>>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
>>> at
>>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
>>> at java.lang.Thread.run(Thread.java:724)
>>>
>>>
>>>
>>> On Tue, Aug 4, 2015 at 10:48 AM, Marc van Grootel
>>> wrote:
Hi Christian,
I pulled it via the Maven repo using Gradle. It says BaseX 8.3 beta
7f8299f.
Maybe that doesn't carry the latest?
Re format name suggestions: format=item (not good, includes XML
nodes), format=function (correct per
http://www.w3.org/TR/xpath-datamodel-31/#types-representation as it
includes array and map) but confusing. format=native (maybe?)
I agree fully that the server should have the last say in how to
process the content so I would definitely be in favor of prioritizing
server parameters. Not sure if that breaks something out there though.
--Marc
On Tue, Aug 4, 2015 at 10:31
Re: [basex-talk] Round tripping JSON
Thanks, came to that conclusion too, strange things is that
CACHERESTXQ=true never bit me before. First time I have issues with
it. I have it set to true almost all the time.
--Marc
On Tue, Aug 4, 2015 at 11:24 AM, Andy Bunce wrote:
> This looks a bit like errors I have seen with CACHERESTXQ=true [1]
> and then hot swapping the XQuery files. Requesting /.init sorts it for me.
> /Andy
> [1] http://docs.basex.org/wiki/Options#CACHERESTXQ
>
> On 4 August 2015 at 10:05, Christian Grün wrote:
>>
>> All existing test cases seem to work, and I didn't come across this in
>> my simple queries. Could you please provide me with a function and
>> call that triggers the exception?
>>
>>
>> On Tue, Aug 4, 2015 at 11:00 AM, Marc van Grootel
>> wrote:
>> > I've double checked the version and manually downloaded latest. I'm
>> > now on BaseX 8.3 beta e13a5f0 which I assume is correct.
>> >
>> > Not sure what's going on but in most cases (with format=map) gives me
>> > a NPE (and {"foo": "bar"}):
>> >
>> > Unexpected error: Improper use? Potential bug? Your feedback is welcome:
>> > Contact: basex-talk@mailman.uni-konstanz.de
>> > Version: BaseX 8.3 beta e13a5f0
>> > Java: Oracle Corporation, 1.7.0_25
>> > OS: Windows 7, amd64
>> > Stack Trace:
>> > java.lang.NullPointerException
>> > at org.basex.http.restxq.RestXqFunction.parse(RestXqFunction.java:124)
>> > at org.basex.http.restxq.RestXqModule.process(RestXqModule.java:100)
>> > at org.basex.http.restxq.RestXqFunction.process(RestXqFunction.java:109)
>> > at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:44)
>> > at org.basex.http.BaseXServlet.service(BaseXServlet.java:64)
>> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>> > at
>> > org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
>> > at
>> > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503)
>> > at
>> > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
>> > at
>> > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
>> > at
>> > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
>> > at
>> > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
>> > at
>> > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429)
>> > at
>> > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
>> > at
>> > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
>> > at
>> > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
>> > at
>> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
>> > at org.eclipse.jetty.server.Server.handle(Server.java:370)
>> > at
>> > org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
>> > at
>> > org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982)
>> > at
>> > org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043)
>> > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
>> > at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
>> > at
>> > org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
>> > at
>> > org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
>> > at
>> > org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
>> > at
>> > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
>> > at
>> > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
>> > at java.lang.Thread.run(Thread.java:724)
>> >
>> >
>> >
>> > On Tue, Aug 4, 2015 at 10:48 AM, Marc van Grootel
>> > wrote:
>> >> Hi Christian,
>> >>
>> >> I pulled it via the Maven repo using Gradle. It says BaseX 8.3 beta
>> >> 7f8299f.
>> >> Maybe that doesn't carry the latest?
>> >>
>> >> Re format name suggestions: format=item (not good, includes XML
>> >> nodes), format=function (correct per
>> >> http://www.w3.org/TR/xpath-datamodel-31/#types-representation as it
>> >> includes array and map) but confusing. format=native (maybe?)
>> >>
>> >> I agree fully that the server should have the last say in how to
>> >> process the content so I would definitely be in favor of prioritizing
>> >> server parameters. Not sure if that breaks something out there though.
>> >>
>> >> --Marc
>> >>
>> >>
>> >>
>> >> On Tue, Aug 4, 2015 at 10:31 AM, Christian Grün
>> >> wrote:
>> >>> Hi Marc,
>> >>>
>> I can get this to work with %input:json format=direct/format=basic
>> but
>> when I change to format=map I still get
>>
>> [bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1):
>> No
>> text allowed before root element.
>> >>>
>> >>> Hm, it seems to work on my machine. Here is again the minimiz
Re: [basex-talk] Round tripping JSON
Hi Christian,
I figured it out. The way to repro is with the function I posted
earlier. Start the server but with CACHERESTXQ=true. Test the
function: works fine. Change the handler function/file and save it.
Retry the POST, boom. NPE. Switching CACHERESTXQ=false no fixes this.
About the conversion. I'm fine with the default being parse into XML
as long as i can fix it to parse as array/map using a handler
annotation (and in a way that the client cannot mess with this). No
need to change defaults.
Maybe the wrong forum here but that REST discussion:
Server semantics for GET, however, are restricted such that a body,
if any, has no semantic meaning to the request. The requirements
on parsing are separate from the requirements on method semantics.
What is that supposed to mean (pun intended)? If with semantics they
mean HTTP verb semantics. Eg. GET is idempotent etc. then my use case
where the body is simply an easier way to specify a query to the
server and thus fully compatible with GET semantics. I see no
difference with providing params via the URL string or path. Somehow
in the past somebody (not sure if it was Fielding himself, maybe the
people from the HTTP spec) decided body should not be part of a GET
and all that needs to be done with GET semantics can be done with
query strings. Now I'm forced to use POST which HTTP semantics are
create a new resource. To me this seems worse then using GET and most
implementers have chosen to conflate specs with "morals" (the REST
morals). I already found out that going for GET is useless as the road
is riddled with tools and libraries that forbid you from doing so. And
also ES had to allow the same queries to be POSTed instead of provided
with GET. Maybe I get crucified for saying this but I think we should
allow GET with body. And yes, it's very useful and can simplify
clients that need to specify complex queries to a REST server. I stop
ranting now, I probably should take this somewhere else but I'm still
building up courage to do so ;-)
On Tue, Aug 4, 2015 at 11:05 AM, Christian Grün
wrote:
> All existing test cases seem to work, and I didn't come across this in
> my simple queries. Could you please provide me with a function and
> call that triggers the exception?
>
>
> On Tue, Aug 4, 2015 at 11:00 AM, Marc van Grootel
> wrote:
>> I've double checked the version and manually downloaded latest. I'm
>> now on BaseX 8.3 beta e13a5f0 which I assume is correct.
>>
>> Not sure what's going on but in most cases (with format=map) gives me
>> a NPE (and {"foo": "bar"}):
>>
>> Unexpected error: Improper use? Potential bug? Your feedback is welcome:
>> Contact: basex-talk@mailman.uni-konstanz.de
>> Version: BaseX 8.3 beta e13a5f0
>> Java: Oracle Corporation, 1.7.0_25
>> OS: Windows 7, amd64
>> Stack Trace:
>> java.lang.NullPointerException
>> at org.basex.http.restxq.RestXqFunction.parse(RestXqFunction.java:124)
>> at org.basex.http.restxq.RestXqModule.process(RestXqModule.java:100)
>> at org.basex.http.restxq.RestXqFunction.process(RestXqFunction.java:109)
>> at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:44)
>> at org.basex.http.BaseXServlet.service(BaseXServlet.java:64)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
>> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503)
>> at
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
>> at
>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
>> at
>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
>> at
>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
>> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429)
>> at
>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
>> at
>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
>> at
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
>> at
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
>> at org.eclipse.jetty.server.Server.handle(Server.java:370)
>> at
>> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
>> at
>> org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982)
>> at
>> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043)
>> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
>> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
>> at
>> org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
>> at
>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
>> at
>> org.eclipse.jetty.io.nio.SelectChannelEndPoint$
Re: [basex-talk] Round tripping JSON
This looks a bit like errors I have seen with CACHERESTXQ=true [1]
and then hot swapping the XQuery files. Requesting /.init sorts it for me.
/Andy
[1] http://docs.basex.org/wiki/Options#CACHERESTXQ
On 4 August 2015 at 10:05, Christian Grün wrote:
> All existing test cases seem to work, and I didn't come across this in
> my simple queries. Could you please provide me with a function and
> call that triggers the exception?
>
>
> On Tue, Aug 4, 2015 at 11:00 AM, Marc van Grootel
> wrote:
> > I've double checked the version and manually downloaded latest. I'm
> > now on BaseX 8.3 beta e13a5f0 which I assume is correct.
> >
> > Not sure what's going on but in most cases (with format=map) gives me
> > a NPE (and {"foo": "bar"}):
> >
> > Unexpected error: Improper use? Potential bug? Your feedback is welcome:
> > Contact: basex-talk@mailman.uni-konstanz.de
> > Version: BaseX 8.3 beta e13a5f0
> > Java: Oracle Corporation, 1.7.0_25
> > OS: Windows 7, amd64
> > Stack Trace:
> > java.lang.NullPointerException
> > at org.basex.http.restxq.RestXqFunction.parse(RestXqFunction.java:124)
> > at org.basex.http.restxq.RestXqModule.process(RestXqModule.java:100)
> > at org.basex.http.restxq.RestXqFunction.process(RestXqFunction.java:109)
> > at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:44)
> > at org.basex.http.BaseXServlet.service(BaseXServlet.java:64)
> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
> > at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
> > at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503)
> > at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
> > at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
> > at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> > at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
> > at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429)
> > at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
> > at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
> > at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
> > at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
> > at org.eclipse.jetty.server.Server.handle(Server.java:370)
> > at
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
> > at
> org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982)
> > at
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043)
> > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
> > at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
> > at
> org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
> > at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
> > at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
> > at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
> > at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
> > at java.lang.Thread.run(Thread.java:724)
> >
> >
> >
> > On Tue, Aug 4, 2015 at 10:48 AM, Marc van Grootel
> > wrote:
> >> Hi Christian,
> >>
> >> I pulled it via the Maven repo using Gradle. It says BaseX 8.3 beta
> 7f8299f.
> >> Maybe that doesn't carry the latest?
> >>
> >> Re format name suggestions: format=item (not good, includes XML
> >> nodes), format=function (correct per
> >> http://www.w3.org/TR/xpath-datamodel-31/#types-representation as it
> >> includes array and map) but confusing. format=native (maybe?)
> >>
> >> I agree fully that the server should have the last say in how to
> >> process the content so I would definitely be in favor of prioritizing
> >> server parameters. Not sure if that breaks something out there though.
> >>
> >> --Marc
> >>
> >>
> >>
> >> On Tue, Aug 4, 2015 at 10:31 AM, Christian Grün
> >> wrote:
> >>> Hi Marc,
> >>>
> I can get this to work with %input:json format=direct/format=basic but
> when I change to format=map I still get
>
> [bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1): No
> text allowed before root element.
> >>>
> >>> Hm, it seems to work on my machine. Here is again the minimized
> >>> version (I also tried the function you sent to me; I removed the
> >>> %rest:consumes and %rest:produces annotations, because they only serve
> >>> as filters, and do not influence the conversion of $query):
> >>>
> >>> declare %rest:POST("{$query}") %rest:path("/json")
> >>> function local:json($query) {
> >>> $query
> >>> };
> >>>
> >>> curl -XPOST -H"Content-Type:application/json;format=map
Re: [basex-talk] Round tripping JSON
All existing test cases seem to work, and I didn't come across this in
my simple queries. Could you please provide me with a function and
call that triggers the exception?
On Tue, Aug 4, 2015 at 11:00 AM, Marc van Grootel
wrote:
> I've double checked the version and manually downloaded latest. I'm
> now on BaseX 8.3 beta e13a5f0 which I assume is correct.
>
> Not sure what's going on but in most cases (with format=map) gives me
> a NPE (and {"foo": "bar"}):
>
> Unexpected error: Improper use? Potential bug? Your feedback is welcome:
> Contact: basex-talk@mailman.uni-konstanz.de
> Version: BaseX 8.3 beta e13a5f0
> Java: Oracle Corporation, 1.7.0_25
> OS: Windows 7, amd64
> Stack Trace:
> java.lang.NullPointerException
> at org.basex.http.restxq.RestXqFunction.parse(RestXqFunction.java:124)
> at org.basex.http.restxq.RestXqModule.process(RestXqModule.java:100)
> at org.basex.http.restxq.RestXqFunction.process(RestXqFunction.java:109)
> at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:44)
> at org.basex.http.BaseXServlet.service(BaseXServlet.java:64)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
> at org.eclipse.jetty.server.Server.handle(Server.java:370)
> at
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
> at
> org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982)
> at
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043)
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
> at
> org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
> at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
> at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
> at java.lang.Thread.run(Thread.java:724)
>
>
>
> On Tue, Aug 4, 2015 at 10:48 AM, Marc van Grootel
> wrote:
>> Hi Christian,
>>
>> I pulled it via the Maven repo using Gradle. It says BaseX 8.3 beta 7f8299f.
>> Maybe that doesn't carry the latest?
>>
>> Re format name suggestions: format=item (not good, includes XML
>> nodes), format=function (correct per
>> http://www.w3.org/TR/xpath-datamodel-31/#types-representation as it
>> includes array and map) but confusing. format=native (maybe?)
>>
>> I agree fully that the server should have the last say in how to
>> process the content so I would definitely be in favor of prioritizing
>> server parameters. Not sure if that breaks something out there though.
>>
>> --Marc
>>
>>
>>
>> On Tue, Aug 4, 2015 at 10:31 AM, Christian Grün
>> wrote:
>>> Hi Marc,
>>>
I can get this to work with %input:json format=direct/format=basic but
when I change to format=map I still get
[bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1): No
text allowed before root element.
>>>
>>> Hm, it seems to work on my machine. Here is again the minimized
>>> version (I also tried the function you sent to me; I removed the
>>> %rest:consumes and %rest:produces annotations, because they only serve
>>> as filters, and do not influence the conversion of $query):
>>>
>>> declare %rest:POST("{$query}") %rest:path("/json")
>>> function local:json($query) {
>>> $query
>>> };
>>>
>>> curl -XPOST -H"Content-Type:application/json;format=map"
>>> -Tinput.json "http://localhost:8984/json";
>>>
>>> Are you sure you tried the latest snapshot?
>>>
A second point regarding format=map. Not sure if this is the correct
name as I could be posting "[1,2,3]" which is valid JSON. Does this
mean there should be a format=array or would it be better to give a
different name for this format.
>>>
>>> This is a valid point. It stems from the beginnings of th
Re: [basex-talk] Round tripping JSON
> I got it working using this function. One requirement though is that > the client must send the JSON with a text content type or no > content-type. This way the function receives a string argument and I > can do with the string what I want. So you mean that your clients will always send JSON as text/plain or without content type? In that case, you can indeed use json:parse($input). > Not 100% satisfied because the client has to lie about it's content > type in order to get this to work. I need to ask for more details: Do you want to say that you are bothered by the implicit JSON conversion once the user specifies the content-type? Would you like to have disabled this conversion? > although the HTTP specs more or less allow it many tools and libraries > (including BaseX/RESTXQ) do not support sending bodies with GET. So > now I'm doing this with POST which feels wrong too. In that case, Elasticsearch seems to have deviated from Roy Field's initial suggestions, which seem to also have been adopted by JAX-RS and, consistently, RESTXQ (see e.g. [1]). [1] https://groups.yahoo.com/neo/groups/rest-discuss/conversations/messages/9962
Re: [basex-talk] Round tripping JSON
I've double checked the version and manually downloaded latest. I'm
now on BaseX 8.3 beta e13a5f0 which I assume is correct.
Not sure what's going on but in most cases (with format=map) gives me
a NPE (and {"foo": "bar"}):
Unexpected error: Improper use? Potential bug? Your feedback is welcome:
Contact: basex-talk@mailman.uni-konstanz.de
Version: BaseX 8.3 beta e13a5f0
Java: Oracle Corporation, 1.7.0_25
OS: Windows 7, amd64
Stack Trace:
java.lang.NullPointerException
at org.basex.http.restxq.RestXqFunction.parse(RestXqFunction.java:124)
at org.basex.http.restxq.RestXqModule.process(RestXqModule.java:100)
at org.basex.http.restxq.RestXqFunction.process(RestXqFunction.java:109)
at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:44)
at org.basex.http.BaseXServlet.service(BaseXServlet.java:64)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:370)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
at
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
at
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:724)
On Tue, Aug 4, 2015 at 10:48 AM, Marc van Grootel
wrote:
> Hi Christian,
>
> I pulled it via the Maven repo using Gradle. It says BaseX 8.3 beta 7f8299f.
> Maybe that doesn't carry the latest?
>
> Re format name suggestions: format=item (not good, includes XML
> nodes), format=function (correct per
> http://www.w3.org/TR/xpath-datamodel-31/#types-representation as it
> includes array and map) but confusing. format=native (maybe?)
>
> I agree fully that the server should have the last say in how to
> process the content so I would definitely be in favor of prioritizing
> server parameters. Not sure if that breaks something out there though.
>
> --Marc
>
>
>
> On Tue, Aug 4, 2015 at 10:31 AM, Christian Grün
> wrote:
>> Hi Marc,
>>
>>> I can get this to work with %input:json format=direct/format=basic but
>>> when I change to format=map I still get
>>>
>>> [bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1): No
>>> text allowed before root element.
>>
>> Hm, it seems to work on my machine. Here is again the minimized
>> version (I also tried the function you sent to me; I removed the
>> %rest:consumes and %rest:produces annotations, because they only serve
>> as filters, and do not influence the conversion of $query):
>>
>> declare %rest:POST("{$query}") %rest:path("/json")
>> function local:json($query) {
>> $query
>> };
>>
>> curl -XPOST -H"Content-Type:application/json;format=map"
>> -Tinput.json "http://localhost:8984/json";
>>
>> Are you sure you tried the latest snapshot?
>>
>>> A second point regarding format=map. Not sure if this is the correct
>>> name as I could be posting "[1,2,3]" which is valid JSON. Does this
>>> mean there should be a format=array or would it be better to give a
>>> different name for this format.
>>
>> This is a valid point. It stems from the beginnings of that feature,
>> where the top object was always a map. Do you possibly have a
>> suggestion for a better name?
>>
>>> A last point I would like to make is that I still find it dubious that
>>> the client can specify things in a header (Content Type) that override
>>> the way it's porcessed on the server.
>>
>> I think it's basically a good idea to allow a client to specify
>> content-type parameters.
Re: [basex-talk] Round tripping JSON
Hi Christian,
I pulled it via the Maven repo using Gradle. It says BaseX 8.3 beta 7f8299f.
Maybe that doesn't carry the latest?
Re format name suggestions: format=item (not good, includes XML
nodes), format=function (correct per
http://www.w3.org/TR/xpath-datamodel-31/#types-representation as it
includes array and map) but confusing. format=native (maybe?)
I agree fully that the server should have the last say in how to
process the content so I would definitely be in favor of prioritizing
server parameters. Not sure if that breaks something out there though.
--Marc
On Tue, Aug 4, 2015 at 10:31 AM, Christian Grün
wrote:
> Hi Marc,
>
>> I can get this to work with %input:json format=direct/format=basic but
>> when I change to format=map I still get
>>
>> [bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1): No
>> text allowed before root element.
>
> Hm, it seems to work on my machine. Here is again the minimized
> version (I also tried the function you sent to me; I removed the
> %rest:consumes and %rest:produces annotations, because they only serve
> as filters, and do not influence the conversion of $query):
>
> declare %rest:POST("{$query}") %rest:path("/json")
> function local:json($query) {
> $query
> };
>
> curl -XPOST -H"Content-Type:application/json;format=map"
> -Tinput.json "http://localhost:8984/json";
>
> Are you sure you tried the latest snapshot?
>
>> A second point regarding format=map. Not sure if this is the correct
>> name as I could be posting "[1,2,3]" which is valid JSON. Does this
>> mean there should be a format=array or would it be better to give a
>> different name for this format.
>
> This is a valid point. It stems from the beginnings of that feature,
> where the top object was always a map. Do you possibly have a
> suggestion for a better name?
>
>> A last point I would like to make is that I still find it dubious that
>> the client can specify things in a header (Content Type) that override
>> the way it's porcessed on the server.
>
> I think it's basically a good idea to allow a client to specify
> content-type parameters. But it's true that the server parameter
> should probably be priorized and overwrite a client parameter, instead
> of the other way round. Do you agree?
>
> Christian
--
--Marc
Re: [basex-talk] Round tripping JSON
I got it working using this function. One requirement though is that
the client must send the JSON with a text content type or no
content-type. This way the function receives a string argument and I
can do with the string what I want.
Not 100% satisfied because the client has to lie about it's content
type in order to get this to work.
declare %rest:POST("{$query}") %rest:path("/json")
%rest:produces("application/json")
function api:json($query)
{
,
serialize(parse-json($query), map { 'method': 'json'})
};
BTW some background: I'm trying to implement a pattern for sending
query specification via REST (this allows a much better query language
instead of forcing everything through query params). Elastic Search
does this too. My first instinct was to use a body with GET but,
although the HTTP specs more or less allow it many tools and libraries
(including BaseX/RESTXQ) do not support sending bodies with GET. So
now I'm doing this with POST which feels wrong too.
--Marc
On Tue, Aug 4, 2015 at 10:16 AM, Marc van Grootel
wrote:
> A last point I would like to make is that I still find it dubious that
> the client can specify things in a header (Content Type) that override
> the way it's porcessed on the server. Isn't this a case of "The server
> knows best". As a client I just specify that this is JSON and leave it
> up to the server how to process. If I do that and put an
> %input:json(...) annotation on the RESTXQ handler it can still be
> overridden by things specified in the header by the client.
> But again this probably should be directed at RESTXQ rather than BaseX.
>
> --Marc
>
>
> On Tue, Aug 4, 2015 at 10:01 AM, Marc van Grootel
> wrote:
>> Hi Christian,
>>
>> Super-fast as always ;-)
>>
>> I'm still struggling though. Couple of points
>>
>> I have this RESTXQ function that I'm trying to get working.
>>
>>
>> declare %rest:POST("{$query}") %rest:path("/json")
>> %rest:consumes("application/json")
>> %rest:produces("application/json")
>> %input:json("format=map")
>> function api:json($query)
>> {
>> $query
>> };
>>
>> I'm posting this as the body (and have Content-Type=application/json)
>>
>> {"foo": "bar"}
>>
>> I can get this to work with %input:json format=direct/format=basic but
>> when I change to format=map I still get
>>
>> [bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1): No
>> text allowed before root element.
>>
>> What I want to achieve is not go through JSON > XML parsing but get
>> the same effect as using fn:parse-json() on the incoming JSON and
>> fn:serialize(..., map { 'method': JSON }) on the output.
>>
>> I can imagine it's the RESTXQ layer itself that preps the args that
>> insists on turning the JSON into XML. Maybe because it hasn't been
>> revamped to fit XQuery 3.1 where we can have native arrays and maps.
>>
>> A second point regarding format=map. Not sure if this is the correct
>> name as I could be posting "[1,2,3]" which is valid JSON. Does this
>> mean there should be a format=array or would it be better to give a
>> different name for this format.
>>
>> --Marc
>>
>> On Mon, Aug 3, 2015 at 6:47 PM, Christian Grün
>> wrote:
>>> …fixed, and available via the latest snapshot [1].
>>>
>>> Thanks,
>>> Christian
>>>
>>> [1] http://files.basex.org/releases/latest
>>>
>>>
>>>
>>> On Mon, Aug 3, 2015 at 5:11 PM, Christian Grün
>>> wrote:
Hi Marc,
> I'm using Postman. I already had Content-Type = application/json and
> Accept = application/json.
> I tried changing it to application/json;format=map
I noticed that you get different result when specify 'format=basic' or
'format=direct':
curl -XPUT
-H"Content-Type:application/json;format=basic"
-Tinput.json "http://localhost:8984/test";
..so I guess something is going wrong here. I'll dive into this soon.
> (btw do you think it's necessary to have the basex/restxq specific
> pseudo attributes (format= etc) in the HTTP request headers?)
You can also use %input:json(...) if you don't want to specify the
format client-side.
> Maybe something to forward to Adam Retter / RESTXQ?
So far, the content-type parameter extensions have not been adopted in
RESTXQ yet, but feel free to give him an update!
Christian
>>
>>
>>
>> --
>> --Marc
>
>
>
> --
> --Marc
--
--Marc
Re: [basex-talk] Round tripping JSON
Hi Marc,
> I can get this to work with %input:json format=direct/format=basic but
> when I change to format=map I still get
>
> [bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1): No
> text allowed before root element.
Hm, it seems to work on my machine. Here is again the minimized
version (I also tried the function you sent to me; I removed the
%rest:consumes and %rest:produces annotations, because they only serve
as filters, and do not influence the conversion of $query):
declare %rest:POST("{$query}") %rest:path("/json")
function local:json($query) {
$query
};
curl -XPOST -H"Content-Type:application/json;format=map"
-Tinput.json "http://localhost:8984/json";
Are you sure you tried the latest snapshot?
> A second point regarding format=map. Not sure if this is the correct
> name as I could be posting "[1,2,3]" which is valid JSON. Does this
> mean there should be a format=array or would it be better to give a
> different name for this format.
This is a valid point. It stems from the beginnings of that feature,
where the top object was always a map. Do you possibly have a
suggestion for a better name?
> A last point I would like to make is that I still find it dubious that
> the client can specify things in a header (Content Type) that override
> the way it's porcessed on the server.
I think it's basically a good idea to allow a client to specify
content-type parameters. But it's true that the server parameter
should probably be priorized and overwrite a client parameter, instead
of the other way round. Do you agree?
Christian
Re: [basex-talk] Round tripping JSON
A last point I would like to make is that I still find it dubious that
the client can specify things in a header (Content Type) that override
the way it's porcessed on the server. Isn't this a case of "The server
knows best". As a client I just specify that this is JSON and leave it
up to the server how to process. If I do that and put an
%input:json(...) annotation on the RESTXQ handler it can still be
overridden by things specified in the header by the client.
But again this probably should be directed at RESTXQ rather than BaseX.
--Marc
On Tue, Aug 4, 2015 at 10:01 AM, Marc van Grootel
wrote:
> Hi Christian,
>
> Super-fast as always ;-)
>
> I'm still struggling though. Couple of points
>
> I have this RESTXQ function that I'm trying to get working.
>
>
> declare %rest:POST("{$query}") %rest:path("/json")
> %rest:consumes("application/json")
> %rest:produces("application/json")
> %input:json("format=map")
> function api:json($query)
> {
> $query
> };
>
> I'm posting this as the body (and have Content-Type=application/json)
>
> {"foo": "bar"}
>
> I can get this to work with %input:json format=direct/format=basic but
> when I change to format=map I still get
>
> [bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1): No
> text allowed before root element.
>
> What I want to achieve is not go through JSON > XML parsing but get
> the same effect as using fn:parse-json() on the incoming JSON and
> fn:serialize(..., map { 'method': JSON }) on the output.
>
> I can imagine it's the RESTXQ layer itself that preps the args that
> insists on turning the JSON into XML. Maybe because it hasn't been
> revamped to fit XQuery 3.1 where we can have native arrays and maps.
>
> A second point regarding format=map. Not sure if this is the correct
> name as I could be posting "[1,2,3]" which is valid JSON. Does this
> mean there should be a format=array or would it be better to give a
> different name for this format.
>
> --Marc
>
> On Mon, Aug 3, 2015 at 6:47 PM, Christian Grün
> wrote:
>> …fixed, and available via the latest snapshot [1].
>>
>> Thanks,
>> Christian
>>
>> [1] http://files.basex.org/releases/latest
>>
>>
>>
>> On Mon, Aug 3, 2015 at 5:11 PM, Christian Grün
>> wrote:
>>> Hi Marc,
>>>
I'm using Postman. I already had Content-Type = application/json and
Accept = application/json.
I tried changing it to application/json;format=map
>>>
>>> I noticed that you get different result when specify 'format=basic' or
>>> 'format=direct':
>>>
>>> curl -XPUT
>>> -H"Content-Type:application/json;format=basic"
>>> -Tinput.json "http://localhost:8984/test";
>>>
>>> ..so I guess something is going wrong here. I'll dive into this soon.
>>>
(btw do you think it's necessary to have the basex/restxq specific
pseudo attributes (format= etc) in the HTTP request headers?)
>>>
>>> You can also use %input:json(...) if you don't want to specify the
>>> format client-side.
>>>
Maybe something to forward to Adam Retter / RESTXQ?
>>>
>>> So far, the content-type parameter extensions have not been adopted in
>>> RESTXQ yet, but feel free to give him an update!
>>>
>>> Christian
>
>
>
> --
> --Marc
--
--Marc
Re: [basex-talk] Round tripping JSON
Hi Christian,
Super-fast as always ;-)
I'm still struggling though. Couple of points
I have this RESTXQ function that I'm trying to get working.
declare %rest:POST("{$query}") %rest:path("/json")
%rest:consumes("application/json")
%rest:produces("application/json")
%input:json("format=map")
function api:json($query)
{
$query
};
I'm posting this as the body (and have Content-Type=application/json)
{"foo": "bar"}
I can get this to work with %input:json format=direct/format=basic but
when I change to format=map I still get
[bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1): No
text allowed before root element.
What I want to achieve is not go through JSON > XML parsing but get
the same effect as using fn:parse-json() on the incoming JSON and
fn:serialize(..., map { 'method': JSON }) on the output.
I can imagine it's the RESTXQ layer itself that preps the args that
insists on turning the JSON into XML. Maybe because it hasn't been
revamped to fit XQuery 3.1 where we can have native arrays and maps.
A second point regarding format=map. Not sure if this is the correct
name as I could be posting "[1,2,3]" which is valid JSON. Does this
mean there should be a format=array or would it be better to give a
different name for this format.
--Marc
On Mon, Aug 3, 2015 at 6:47 PM, Christian Grün
wrote:
> …fixed, and available via the latest snapshot [1].
>
> Thanks,
> Christian
>
> [1] http://files.basex.org/releases/latest
>
>
>
> On Mon, Aug 3, 2015 at 5:11 PM, Christian Grün
> wrote:
>> Hi Marc,
>>
>>> I'm using Postman. I already had Content-Type = application/json and
>>> Accept = application/json.
>>> I tried changing it to application/json;format=map
>>
>> I noticed that you get different result when specify 'format=basic' or
>> 'format=direct':
>>
>> curl -XPUT
>> -H"Content-Type:application/json;format=basic"
>> -Tinput.json "http://localhost:8984/test";
>>
>> ..so I guess something is going wrong here. I'll dive into this soon.
>>
>>> (btw do you think it's necessary to have the basex/restxq specific
>>> pseudo attributes (format= etc) in the HTTP request headers?)
>>
>> You can also use %input:json(...) if you don't want to specify the
>> format client-side.
>>
>>> Maybe something to forward to Adam Retter / RESTXQ?
>>
>> So far, the content-type parameter extensions have not been adopted in
>> RESTXQ yet, but feel free to give him an update!
>>
>> Christian
--
--Marc
Re: [basex-talk] Performance with joins?
Dear Tim,
The query plan indicates that no index is applied. Your query may be
evaluated faster when rewriting "BIB_ID" to "BIB_ID/text()". I will
see if this can automatically be done by the query compiler.
Best,
Christian
On Tue, Aug 4, 2015 at 4:42 AM, Tim Thompson wrote:
> Hello,
>
> I have a database containing two resources/documents: they both represent
> the same set of library catalog records (7728 "records" in each), but they
> each contain different data that I want to join.
>
> The first resource looks like this:
>
> xmlns:marc="http://www.loc.gov/MARC21/slim";>
>
> 01225cam a2200373Mi 4500
> 5323084
> ind1="1"
> ind2="4"
> tag="245">
> Els teleclubs a les illes Balears
> :
>
>
>
> 01225cam a2200373Mi 4500
> 5323084
> ind1="1"
> ind2="4"
> tag="245">
> Els teleclubs a les illes Balears
> :
>
>
>
> 00818cam a2200241Mi 4500
> 6310976
> ind1="0"
> ind2="0"
> tag="245">
> Diccionari manual de sinònims i
> antònims de la llengua catalana /
>
>
>
>
> The second one looks like this:
>
>
>
> cat
> 5912416
> 5323084
> MXBKSMX
>
>
> cat
> 5912416
> 5323084
> PUVILL
>
>
> cat
> 5935043
> 6310976
> PUVILL
>
>
>
> I have a simple query that joins the two using the value of the
> marc:controlfield[@tag = '001'] from resource 1 and the BIB_ID from resource
> 2.
>
> The query:
>
> declare namespace marc="http://www.loc.gov/MARC21/slim";;
>
> for $m in collection(
> "latin_hold_20150730"
> )/marc:collection/marc:record,
> $r in collection(
> "latin_hold_20150730"
> )/root/row[BIB_ID = $m/marc:controlfield[@tag = '001']]
> group by $key := $r/ITEM_ID
>
> return
> {
> $m/marc:datafield[@tag = '245']/marc:subfield[@code = 'a']/string()
> }
>
> When I run this using Saxon (substituting fn:doc for fn:collection), it only
> takes a second to execute. In BaseX 8.2.3 (using the GUI), however, it takes
> a very long time to execute (around 6 minutes!). The BaseX database has both
> attribute and text indexes enabled.
>
> Any idea what is causing it to take so long?
>
> Here is the BaseX Query Info:
>
> Total Time: 383756.3 ms
>
> Compiling:
> - pre-evaluating collection("latin_hold_20150730")
> - pre-evaluating collection("latin_hold_20150730")
> Query:
> declare namespace marc="http://www.loc.gov/MARC21/slim";; for $m in
> collection( "latin_hold_20150730" )/marc:collection/marc:record, $r in
> collection( "latin_hold_20150730" )/root/row[BIB_ID =
> $m/marc:controlfield[@tag = '001']] group by $key := $r/ITEM_ID return n="{ $key }">{ $m/marc:datafield[@tag = '245']/marc:subfield[@code =
> 'a']/string() }
> Optimized Query:
> for $m_0 in (db:open-pre("latin_hold_20150730",0),
> ...)/marc:collection/marc:record for $r_1 in
> (db:open-pre("latin_hold_20150730",0), ...)/root/row[(BIB_ID =
> $m_0/marc:controlfield[(@tag = "001")])] let (: post-group :) $m_4 := $m_0
> group by $key_2 := $r_1/ITEM_ID return element test { (attribute n {
> ($key_2) }, $m_4/marc:datafield[(@tag = "245")]/marc:subfield[(@code =
> "a")]/string()) }
> Result:
> - Hit(s): 7587 Items
> - Updated: 0 Items
> - Printed: 505 KB
> - Read Locking: local [latin_hold_20150730]
> - Write Locking: none
> Timing:
> - Parsing: 0.43 ms
> - Compiling: 1.96 ms
> - Evaluating: 383737.5 ms
> - Printing: 16.41 ms
> - Total Time: 383756.3 ms
> Query plan:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Thanks in advance!
>
> Tim
>
>
> --
> Tim
