Hello Christian,
Thanks for the response. I figured out how to get around my issue. Add the
method="text" attribute to the http:body.
{json:serialize($body)}
- James
On Tue, Feb 17, 2015 at 1:37 PM, Christian Grün
wrote:
> Hi James,
>
> > I am however having an issue when posting JSON with markup in it. For
> > example:
>
> I tried to reproduce your example, but it didn't work out. It seems
> you were storing data in a database, right? Will the data be stored in
> a wrong format, or is the subsequent output (serialization) that gives
> wrongly escaped results? Could you possibly give me a little complete
> example that we can try out?
>
> Thanks in advance,
> Christian
>
>
> >
> > let $out := map {
> > name: 'myView',
> > markup: "Some other stuff"
> > }
> >
> > http:send-request(, uri, $out)
> >
> > The issue is that the resulting JSON body being posted has the 'markup'
> > property xml escaped. IE:
> >
> > { "name": "myView",
> > "markup": "Some other . "
> > }
> >
> > I had a similar issue with file:write but resolved that by using
> > file:write-string.
> >
> > I have tried calling json:serialize on $out manually as well as declaring
> > the option output:method to 'json'. I even tried changing the request
> object
> > by inserting a http:body. IE:
> >
> >
> >
> > {json:serialize($out)}
> >
> >
> >
> > Lastly, I have tried declaring the cdata-section option:
> > declare option output:cdata-section-elements 'http:body';
> >
> > Still no go...
> >
> > Is this not possible? One option would be to serialize the content using
> > file:write and then call curl. This is a workaround I have used in the
> past
> > however I was hoping I was behind that now with the latest improvements
> to
> > XQuery 3.1. Maybe I need to send the http:request object as a map
> instead of
> > an xml node?
> >
> > Thanks again for your time on all of this.
> > - James
>
Hello everyone,
First, I want to say that I really like the JSON improvements in BaseX 8.0.
Great job.
I am however having an issue when posting JSON with markup in it. For
example:
let $out := map {
name: 'myView',
markup: "Some other stuff"
}
http:send-request(, uri, $out)
The issue is that the resulting JSON body being posted has the 'markup'
property xml escaped. IE:
{ "name": "myView",
"markup": "Some other . "
}
I had a similar issue with file:write but resolved that by using
file:write-string.
I have tried calling json:serialize on $out manually as well as declaring
the option output:method to 'json'. I even tried changing the request
object by inserting a http:body. IE:
{json:serialize($out)}
Lastly, I have tried declaring the cdata-section option:
declare option output:cdata-section-elements 'http:body';
Still no go...
Is this not possible? One option would be to serialize the content using
file:write and then call curl. This is a workaround I have used in the past
however I was hoping I was behind that now with the latest improvements to
XQuery 3.1. Maybe I need to send the http:request object as a map instead
of an xml node?
Thanks again for your time on all of this.
- James
A great online open source editor is ace.js. There is XQuery 3.0 compliant
language processor for it, plus its styleable and again open source. Supports
20+ languages including but not limited too:
CSSHTMLJavaC#PythonRubyPHPJavascriptXQuery 3.0
etc.
Also its extremely extensible. To add it to a page is a few lines of javascript
and one or 2 script references. One for the ace editor, one for the language
processor. Language processors can allow for a plethora of functionality
including type-ahead etc.. You could easily add BaseX, or your Apps model to
the processing for validation etc...
Just thought I would share. - James
> From: christian.gr...@gmail.com
> Date: Sun, 17 Nov 2013 13:03:55 +0100
> To: lin...@126.com
> CC: basex-talk@mailman.uni-konstanz.de
> Subject: Re: [basex-talk] Is there a better way to do develop or test with
> c/s mode server?
>
> >I found the Gui can only work with local mode, but I want do some xquery
> > test / sandbox with basex server, only found can do in client mode, Is there
> > some other way ?esp for restful web mode.
>
> That’s true. We have no existing web admin interface for BaseX, but if
> you want to perform simple queries on a server, you can create a
> simple web page which executes queries via REST [1].
>
> [1] http://docs.basex.org/wiki/REST
> ___
>
> >
> > thanks
> >
> >
> > --
> > 此致
> >
> >easy
> >
> > 莫愁前路无知己，天下谁人不识君。
> >
> >
> >
> > ___
> > BaseX-Talk mailing list
> > BaseX-Talk@mailman.uni-konstanz.de
> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
> >
> ___
> BaseX-Talk mailing list
> BaseX-Talk@mailman.uni-konstanz.de
> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Would this work
$someDoc//@*/text()
--- Original Message ---
From: "buzz chopra"
Sent: November 2, 2013 11:38 PM
To: "Dirk Kirsten"
Cc: "BaseX"
Subject: Re: [basex-talk] xquery and hash:md5
Hello Dirk,
Yes in that case, the issue exists in the nature of our data - and my poor
choice of the return value of fn:string as the content to hash.
Our data looks like:
As you state, fn:string behaves the way
http://www.w3.org/TR/xpath-functions-30/#func-string says it should. i.e.
the empty string is returned for the document above.
I'll have to try and find something that returns all attribute values in
each document.
Thanks again for your help.
Buzz
On Fri, Nov 1, 2013 at 10:37 AM, Dirk Kirsten wrote:
> Hello Buzz,
>
> The result of string() depends on the kind of element as an input. For
> an attribute value it will simply be the value of the attribute. For an
> element node it will be a concatination of the node content and all
> descendant nodes. So if string($r) for $r being an element node is
> always empty I would guess it is an empty node (something like />), which will always yield an empty result.
>
> If you want to create different hashs for empty elements with different
> attribute values you might want to do something like the following
>
> let $reps := fn:collection($coll)//reps
> for $r at $lcv in $reps
> let $l := string-join($r/@*, '')
> let $hash := xs:hexBinary(hash:md5(fn:string($l)))
> return {$hash}
>
> If this is not what you actually want to achieve it would be very
> helpful to actually get a small example of your data (especiall the reps
> nodes) to get an idea how it looks like.
>
> Cheers,
> Dirk
>
> On 01/11/13 16:19, buzz chopra wrote:
> > Dirk,
> >
> > Thank you for your quick response.
> >
> > Upon your suggestion (and prior to that) I checked that the values for $r
> > are different. Indeed they are. But, what I didn't check until your
> > suggestion is that the values of fn:string($r) are different. They in
> fact
> > are the same - thank you for your suggestion.
> >
> > Is it possible that there is some sort of type casting related issue
> going
> > on where I call fn:string($r)?
> >
> > I get different values for the hash of each attribute value when I select
> > attributes in the collection like:
> >
> > let $reps := fn:collection($coll)//reps/@att0
> > for $r at $lcv in $reps
> > let $hash := xs:hexBinary(hash:md5(fn:string($r)))
> > return {$hash}
> >
> >
> > when I select elements in the collection like:
> >
> > let $reps := fn:collection($coll)//reps
> > for $r at $lcv in $reps
> > let $hash := xs:hexBinary(hash:md5(fn:string($r)))
> > return {$hash}
> >
> > I found that the value of fn:string($r) is blank for all values of $r
> > (which are different) which is why the hash is the same.
> >
> > I appreciate your help.
> >
> > Buzz
> >
> >
> > On Fri, Nov 1, 2013 at 2:52 AM, Dirk Kirsten wrote:
> >
> >> Hello buzz,
> >>
> >> it is ok to also ask more general XQuery questions on this list.
> >>
> >> For testing I executed the following query:
> >>
> >> for $r in ("123", "234", "345")
> >> let $hash := xs:hexBinary(hash:md5(fn:string($r)))
> >> return $hash
> >>
> >> which yields different results for each value of $r. Are you absolutely
> >> sure you have different values for $r (you can check using trace())?
> >> Also, given they query I would be very suprised if indeed it shows the
> >> same result.
> >>
> >> Cheers,
> >> Dirk
> >>
> >> On 01/11/13 08:24, buzz chopra wrote:
> >>> This may be a question better suited for an xquery forum but I'll start
> >>> here since I'm using basex:
> >>>
> >>> Can someone suggest why I'd be getting the same result for the hash
> call
> >>> even though
> >>> the content of $r should be changing?
> >>>
> >>> let $reps := fn:collection($coll)/Rep
> >>> for $r at $lcv in $reps
> >>> let $hash := xs:hexBinary(hash:md5(fn:string($r)))
> >>> return {$hash}
> >>> };
> >>>
> >>> Please let me know if I should post this elsewhere (and please include
> >>> where).
> >>>
> >>> Thanks in advance for any suggestions.
> >>>
> >>> Buzz
> >>>
> >>>
> >>>
> >>> ___
> >>> BaseX-Talk mailing list
> >>> BaseX-Talk@mailman.uni-konstanz.de
> >>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
> >>>
> >>
> >> --
> >> Dirk Kirsten, BaseX GmbH, http://basex.org
> >> |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz
> >> |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer:
> >> | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle
> >> `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22
> >> ___
> >> BaseX-Talk mailing list
> >> BaseX-Talk@mailman.uni-konstanz.de
> >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
> >>
> >
>
> --
> Dirk Kirsten, BaseX GmbH, http://basex.org
> |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz
> |-- Registergericht Freiburg, HRB: 708285, Geschäftsfü
AH!
Well its my fault. Several weeks of work lost because I uninstalled 7.7 and it
removed the repo directory with all my modules! Just wanted to give the fair
warning.. Time to get a drink... I was going to put that in Drop Box last week
and never got to it!!
- James ___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Christian,
That is a grand idea. I obviously took the wrong fork in the road somewhere in
my trials/research.
Thanks once again. - James
> From: christian.gr...@gmail.com
> Date: Fri, 9 Aug 2013 18:49:52 +0200
> Subject: Re: [basex-talk] Analyze-string unsupported?
> To: james...@hotmail.com
> CC: basex-talk@mailman.uni-konstanz.de
>
> What about simply using XQuery?…
>
> analyze-string('test MM23423 junk MM1 junkagain man', 'MM')
> ___
>
> > I am trying to use analyze-string. I have this query:
> >
> > declare namespace str = 'str.analysis';
> > declare function str:analyze-string($string as xs:string, $regex as
> > xs:string,$n as xs:integer ) as xs:string {
> > xslt:transform(, (
> > http://www.w3.org/1999/XSL/Transform";
> > version="2.0">
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > ))
> > };
> > let $item as xs:string := 'test MM23423 junk MM1 junkagain man'
> > let $result := str:analyze-string($item, 'MM', 10)
> > return $result
> >
> > However I am getting an error that this is unsupported. Upon further
> > investigation I found that this is only available in Saxon EE. I'm guessing
> > this is my problem. Is there anything of similar functionality included in
> > basex?
> >
> > Any help would be appreciated. thanks.
> >
> > - james
> >
> > ___
> > BaseX-Talk mailing list
> > BaseX-Talk@mailman.uni-konstanz.de
> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
> >
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Hey,
I am trying to use analyze-string. I have this query:
declare namespace str = 'str.analysis';declare function
str:analyze-string($string as xs:string, $regex as xs:string,$n as xs:integer )
as xs:string { xslt:transform(, ( http://www.w3.org/1999/XSL/Transform"; version="2.0">
))}; let $item as xs:string := 'test
MM23423 junk MM1 junkagain man'let $result := str:analyze-string($item,
'MM', 10)return $result
However I am getting an error that this is unsupported. Upon further
investigation I found that this is only available in Saxon EE. I'm guessing
this is my problem. Is there anything of similar functionality included in
basex?
Any help would be appreciated. thanks.
- james ___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Christian,
Thanks for the help. This works, but as with the other method. It affects the
entire script which is not desired. I just need the call to fn:serialize to
create json. No where else.
Once I set this I have to adjust a lot of the methods i'm using to accept json
instead of xml since thats what 90% of our stuff works with. Can you change the
output mid script? The output is xml, i just need the json for mapping an http
request. For example:
declare option db:parser "html";declare option db:htmlopt "html=true";
for $product in
doc("http://www.telvent-gis.com/support/versions.shtml";)//div[@id='contenedor']/div[@id]
return let $request := { json:serialize(
Test Subject 22
text/html
{fn:serialize($product/*)}
document) }
return try { http:send-request($request) } catch
HC0001 { 'Not Found' }
Thanks again for spending time on this. I might need to look into some other
method. :(- James
> From: christian.gr...@gmail.com
> Date: Fri, 2 Aug 2013 18:01:13 +0200
> Subject: Re: [basex-talk] Question around JSON Serialization
> To: james...@hotmail.com
> CC: basex-talk@mailman.uni-konstanz.de
>
> I have attached a possible solution; don’t hesitate to tell us if you
> expect a different solution.
>
> Best,
> Christian
> ___
>
> declare option output:method 'json';
>
> let $htmlContent := Issue:
> return
>
> Test Subject 10
>
> text/html
> { fn:serialize($htmlContent) }
>
> document
>
> ___
>
> 2013/8/2 James Wright :
> > Christian,
> >
> > Not a problem. Here is a script that perfectly demonstrates my issue and
> > should run on your end:
> >
> > declare variable $serializationParameters :=
> > > xmlns:output="http://www.w3.org/2010/xslt-xquery-serialization";
> > xmlns="http://example.org/ext";>
> >
> > ;
> >
> > let $htmlContent := Issue:
> > return json:serialize(
> >
> > Test Subject 10
> >
> > text/html
> > {fn:serialize($htmlContent, $serializationParameters
> > )}
> >
> > document
> > )
> >
> > This returns:
> >
> > {
> > "subject": "Test Subject 10",
> > "content": {
> > "type": "text\/html",
> > "text": "Issue:"
> > },
> > "type": "document"
> > }
> >
> > However we need it to return:
> >
> > {
> > "subject": "Test Subject 10",
> > "content": {
> > "type": "text\/html",
> > "text": "Issue:<\/h2><\/div>"
> > },
> > "type": "document"
> > }
> >
> > Thanks,
> > James
> >
> >> From: christian.gr...@gmail.com
> >> Date: Fri, 2 Aug 2013 17:32:11 +0200
> >
> >> Subject: Re: [basex-talk] Question around JSON Serialization
> >> To: james...@hotmail.com
> >> CC: basex-talk@mailman.uni-konstanz.de
> >>
> >> Hi James,
> >>
> >> could you please have another look at your example and provide us with
> >> some code that runs out of the box?
> >>
> >> Thanks in advance,
> >> Christian
> >> ___
> >>
> >> 2013/8/2 James Wright :
> >> > Hey Christian,
> >> >
> >> > Thank definitely did the trick however it affects the entire query. The
> >> > query should output xml of the api call requests. I tried adding
> >> > serialization parameters to the single fn:serialize method and set the
> >> > output to 'text' however now it only returns the 'text' inside of the
> >> > elements and not the xml. The script as a whole is supposed to return
> >> > xml
> >> > with results. The only json part is for the API calls. Here is a more
> >> > full
> >> > script:
> >> >
> >> > declare option db:parser "html";
> >> > declare option db:htmlopt "html=true";
> >>
Christian,
Not a problem. Here is a script that perfectly demonstrates my issue and should
run on your end:
declare variable $serializationParameters :=
http://www.w3.org/2010/xslt-xquery-serialization";
xmlns="http://example.org/ext";>
; let $htmlContent :=
Issue: return json:serialize( Test Subject 10
text/html
{fn:serialize($htmlContent, $serializationParameters )}
document )
This returns:
{ "subject": "Test Subject 10", "content": {"type": "text\/html",
"text": "Issue:" }, "type": "document"}
However we need it to return:
{ "subject": "Test Subject 10", "content": {"type": "text\/html",
"text": "Issue:<\/h2><\/div>" }, "type": "document"}
Thanks,James
> From: christian.gr...@gmail.com
> Date: Fri, 2 Aug 2013 17:32:11 +0200
> Subject: Re: [basex-talk] Question around JSON Serialization
> To: james...@hotmail.com
> CC: basex-talk@mailman.uni-konstanz.de
>
> Hi James,
>
> could you please have another look at your example and provide us with
> some code that runs out of the box?
>
> Thanks in advance,
> Christian
> ___
>
> 2013/8/2 James Wright :
> > Hey Christian,
> >
> > Thank definitely did the trick however it affects the entire query. The
> > query should output xml of the api call requests. I tried adding
> > serialization parameters to the single fn:serialize method and set the
> > output to 'text' however now it only returns the 'text' inside of the
> > elements and not the xml. The script as a whole is supposed to return xml
> > with results. The only json part is for the API calls. Here is a more full
> > script:
> >
> > declare option db:parser "html";
> > declare option db:htmlopt "html=true";
> > declare variable $js := '{';
> > declare variable $je := '}';
> > let $serializationParameters :=
> > >
> > xmlns:output="http://www.w3.org/2010/xslt-xquery-serialization";
> > xmlns="http://example.org/ext";>
> >
> >
> >
> > for $productInfo in
> > doc("http://www.someUrl.com/support/versions.shtml";)//div[@id='contenedor']/div[@id]
> > return
> > let $request :=
> > > href='http://www.someExternalService.com/api/core/v3/contents'
> > method='post' username='admin' password='admin'
> > send-authorization='true'>
> >
> > {json:serialize(
> >
> > $productInfo/name
> >
> > text/html
> > {fn:serialize($productInfo/* ,
> > $serializationParameters )}
> >
> > document
> > )}
> >
> > return
> >
> >try {http:send-request($request)[2]}
> >catch HC0001 {'Not Found'}
> >
> >
> >
> > I guess I would expect this to operate as you described since I set the
> > output:method to 'text'. Let me know if you have any ideas? I have read that
> > serialization page several times and I guess I'm just not understanding this
> > piece.
> >
> > Thanks again.
> > - James
> >
> >> From: christian.gr...@gmail.com
> >> Date: Fri, 2 Aug 2013 09:28:48 +0200
> >> Subject: Re: [basex-talk] Question around JSON Serialization
> >> To: james...@hotmail.com
> >> CC: basex-talk@mailman.uni-konstanz.de
> >
> >>
> >> Hi James,
> >>
> >> by default, all string results are XML-encoded. You’ll get the
> >> expected result by declaring "text" as output option in the prolog
> >> (header) of your query:
> >>
> >> declare option output:method 'text';
> >> let $someItem := ...
> >>
> >> Our Wiki page on serialization may give you more hints [1].
> >>
> >> Hope this helps,
> >> Christian
> >>
> >> [1] http://docs.basex.org/wiki/Serialization
> >> __
Hey Christian,
Thank definitely did the trick however it affects the entire query. The query
should output xml of the api call requests. I tried adding serialization
parameters to the single fn:serialize method and set the output to 'text'
however now it only returns the 'text' inside of the elements and not the xml.
The script as a whole is supposed to return xml with results. The only json
part is for the API calls. Here is a more full script:
declare option db:parser "html";declare option db:htmlopt "html=true";declare
variable $js := '{';declare variable $je := '}';let $serializationParameters :=
http://www.w3.org/2010/xslt-xquery-serialization";
xmlns="http://example.org/ext";>
for $productInfo in
doc("http://www.someUrl.com/support/versions.shtml";)//div[@id='contenedor']/div[@id]
return let $request :=
{json:serialize(
$productInfo/name
text/html
{fn:serialize($productInfo/* , $serializationParameters )}
document )}
return
try {http:send-request($request)[2]} catch HC0001 {'Not Found'}
I guess I would expect this to operate as you described since I set the
output:method to 'text'. Let me know if you have any ideas? I have read that
serialization page several times and I guess I'm just not understanding this
piece.
Thanks again. - James
> From: christian.gr...@gmail.com
> Date: Fri, 2 Aug 2013 09:28:48 +0200
> Subject: Re: [basex-talk] Question around JSON Serialization
> To: james...@hotmail.com
> CC: basex-talk@mailman.uni-konstanz.de
>
> Hi James,
>
> by default, all string results are XML-encoded. You’ll get the
> expected result by declaring "text" as output option in the prolog
> (header) of your query:
>
> declare option output:method 'text';
> let $someItem := ...
>
> Our Wiki page on serialization may give you more hints [1].
>
> Hope this helps,
> Christian
>
> [1] http://docs.basex.org/wiki/Serialization
> ___
>
> 2013/8/2 James Wright :
> > Hey BaseX Team,
> >
> > Im trying to serialize some JSON in order to make a few calls to an external
> > api. In the json I need to include 'html' for example:
> >
> > {
> >"subject": "Test Subject",
> >"content": {
> > type: "text/html",
> > text: "Issue:
> > },
> > "type": "document"
> > }
> >
> > When I call the following script however I get:
> >
> > {
> > "subject": "Test Subject 10",
> > "content": {
> > "type": "text\/html",
> > "text": "<html>\n <div>\n<h2>Issue:<\/h2>\n
> > <\/div>\n<\/html>"
> > },
> > "type": "document"
> > }
> >
> > I tried messing with the serialization parameters however with no success. I
> > use the resulting string as the content for a http post request. Any help
> > would be appreciated.
> >
> > Script:
> > let $htmlContent := Issue:
> > return json:serialize(
> >
> > Test Subject 10
> >
> > text/html
> > {fn:serialize({ $someItem })}
> >
> > document
> > )
> >
> > Thanks,
> > James
> >
> > ___
> > BaseX-Talk mailing list
> > BaseX-Talk@mailman.uni-konstanz.de
> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
> >
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Hey BaseX Team,
Im trying to serialize some JSON in order to make a few calls to an external
api. In the json I need to include 'html' for example:
{ "subject": "Test Subject", "content": { type: "text/html",
text: "Issue:},"type": "document"}
When I call the following script however I get:
{ "subject": "Test Subject 10", "content": {"type": "text\/html",
"text": "\n \n
Issue:<\/h2>\n
<\/div>\n<\/html>" }, "type": "document"}
I tried messing with the serialization parameters however with no success. I
use the resulting string as the content for a http post request. Any help would
be appreciated.
Script:let $htmlContent := Issue: return
json:serialize( Test
Subject 10
text/html {fn:serialize({ $someItem
})}document
)
Thanks,James ___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Hey Again,
Sorry if I am here often. I got my system up and running and at first was
seeing 1 second query times which is unacceptable. I started doing some
research and trying different things. Here are some interesting things I found:
1) Using data() is SLOW!!! For example:
(//DataElement(data(Name) = 'Transformer'])[1] : this takes 350 ms avg to
complete (Most of it evaluation)(//DataElement(Name = 'Transformer')[1] : this
takes 4 - 10 ms.
Not sure why this is, but its good to know. I'm assuming the data() usage
circumvents the indexes or something...
2) In a dynamic context is often worthwhile to 'construct' a query with the
names embedded and execute xquery:eval rather than using variables within the
expression. For example when trying to find a node based on a key property name
which you don't know until run time:
($page:database//DataElement[(*|@*)[name() = $keyPropName)] = $entityId])[1]
Above query takes about 1200 ms to complete in our data. If you have more than
one of these being executed, it jumps to 10,000 ms or more!!
xquery:eval("($context//" || $entityListName || "[" || $keyPropertyName || " =
$entityId])[1]",
map{ '$context' := $page:database,
'$entityId' :=
$entityId })
This query takes between 2 - 8 ms to complete. Awesome!!
Just thought I would share my findings! So far I am really enjoying BaseX!!
Thanks for all the hard work!
- James ___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Hello Christian,
Thanks for the response. I should have been more descriptive. I am not having
issues just wanted to make sure I was using the best approach. I notice that I
could accomplish the task two ways:
subseqence(somequery, 0, 10)(somequery)[(1 to 10)]
My specific query was really insignificant. Basically I just wanted to ensure I
was writing queries in a way where once all predicates are achieved the
execution is terminated.
On a side note. What would take a query a long time to compile? I'm seeing 60
second compilations coupled with 52ms execution times. Its really not an issues
since this type of query would only be executed on 'schema' changes, but I was
just curious if you had a comment on this from a BaseX perspective.
- James
> From: christian.gr...@gmail.com
> Date: Tue, 21 May 2013 23:24:20 +0200
> Subject: Re: [basex-talk] Quick question regarding subsequences
> To: james...@hotmail.com
> CC: basex-talk@mailman.uni-konstanz.de
>
> Hi James,
>
> regarding your specific query, I’m afraid it’s probably too nested in
> order to simply say how it’s gonna be evaluated, so I would suggest to
> simply run the query, see what happens and check out the optimization
> steps int the query info. In general, I can confirm that the
> evaluation of a filter expression with a positional predicate will be
> stopped as soon as all results in the specified range have been
> returned. The following query demonstrates this behavior (it would run
> quite a while if the iteration were not interrupted):
>
> (1 to 10)[position() = 1 to 10]
>
> Next, you may get better results if you move expressions out of a
> predicate if they may need to be evaluated multiple times:
>
> let $key := $entityMetadata/property[@key = 'true']/@name
> return ($context//*
> [name() = $type]
> [data((*|@*)[name() = $key]) = $entityId]
> )[1]
>
> In the given case, it might be unnecessary, as you want to stop
> evaluation after the first hit anyway.
>
> I agree that each XQuery implementation may find different strategies
> to evaluate this query – and I’d like to add that it‘s often not
> possible to say what a specific implementation does, as optimization
> strategies change over time, and may also be dependent on the input
> data, available index structures, etc. For example, if multiple
> predicates are specified, we are trying to find and evaluate the
> cheapest predicate earlier than others, and this estimation also
> depends on the database statistics.
>
> In a nutshell: you will often succeed in speeding up your queries by
> rewriting them. On the other hand, it’s sometimes impossible to say in
> advance if a query will be evaluated faster than another.
>
> Hope this helps,
> Christian
> ___
>
> On Tue, May 21, 2013 at 9:42 PM, James Wright wrote:
> > Hey,
> >
> > In the BaseX implementation this query:
> >
> > ($context//*[name() = $type and data((*|@*)[name() =
> > $entityMetadata/property[@key = 'true']/@name]) = $entityId])[1]
> >
> > Will the query execute completely and then a 'subset' of 1 will be
> > extracted, or will it only execute until it find the '1st' item that matches
> > the query? Maybe this is a general XQuery question but I figured it was
> > implementation specific.
> >
> > - James
> >
> > ___
> > BaseX-Talk mailing list
> > BaseX-Talk@mailman.uni-konstanz.de
> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
> >
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Hey,
In the BaseX implementation this query:
($context//*[name() = $type and data((*|@*)[name() =
$entityMetadata/property[@key = 'true']/@name]) = $entityId])[1]
Will the query execute completely and then a 'subset' of 1 will be extracted,
or will it only execute until it find the '1st' item that matches the query?
Maybe this is a general XQuery question but I figured it was implementation
specific.
- James ___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Christian,
Thanks for the update. Speed was my thinking and it looks like what you updated
the wiki to state...
Thanks again!
- James
> From: christian.gr...@gmail.com
> Date: Mon, 20 May 2013 23:50:18 +0200
> Subject: Re: [basex-talk] BaseX generate-id()
> To: james...@hotmail.com; basex-talk@mailman.uni-konstanz.de
>
> I’m not quite sure on what you were asking, but I noticed that the
> explanation of PRE and ID values in our Wiki was somewhat deficient.
> Please don‘t hesitate to have yet another look at the revised articles
> [1,2], which are hopefully more self-explanatory. If not, you are
> always invited to read the dissertation in more detail ;)
>
> [1] http://docs.basex.org/wiki/Database_Module#db:open-pre
> [2] http://docs.basex.org/wiki/Node_Storage
> ___
>
> On Mon, May 20, 2013 at 10:46 PM, James Wright wrote:
> > How did I not notice this!! I have read through those modules so many times
> > now!! I think this is exactly what I was looking for. I expected there to be
> > some sort of persistent id because I skimmed the initial dissertation on
> > BaseX from several years ago and it talks about a persistent ID. In regards
> > to the 'pre' this ID represents the id of a node which 'hasn't' been
> > inserted yet but it does not necessarily denote the ID the node will take
> > when actually inserted... So whats the point? Is this more for internal
> > uses? Maybe I don't fully understand...
> >
> > Thanks for the quick reply!
> >
> > - James
> >
> >> From: christian.gr...@gmail.com
> >> Date: Mon, 20 May 2013 22:30:08 +0200
> >> Subject: Re: [basex-talk] BaseX generate-id()
> >> To: james...@hotmail.com
> >> CC: basex-talk@mailman.uni-konstanz.de
> >
> >>
> >> Hi James,
> >>
> >> a node id generated via generate-id() is unique and the same during
> >> the runtime of a query. If you want to ensure that you are requesting
> >> the same nodes of a database, I recommend you to check out the
> >> node-id() and node-pre() functions of our database module [1].
> >>
> >> Hope this helps,
> >> Christian
> >>
> >> [1] http://docs.basex.org/wiki/Database_Module
> >> ___
> >>
> >> On Mon, May 20, 2013 at 10:09 PM, James Wright
> >> wrote:
> >> > In the W3C specs it says this method should generate the same ID for a
> >> > node
> >> > regardless of when its called. Is this how it is implemented in BaseX or
> >> > does it simply generate a random number? My testing seems to implicate
> >> > the
> >> > ladder. If it does generate the same ID for each node each time its run,
> >> > does it do ensure uniqueness at the db or resource level?
> >> >
> >> > Thx for your time.
> >> > - James
> >> >
> >> > ___
> >> > BaseX-Talk mailing list
> >> > BaseX-Talk@mailman.uni-konstanz.de
> >> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
> >> >
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
In the W3C specs it says this method should generate the same ID for a node
regardless of when its called. Is this how it is implemented in BaseX or does
it simply generate a random number? My testing seems to implicate the ladder.
If it does generate the same ID for each node each time its run, does it do
ensure uniqueness at the db or resource level?
Thx for your time.- James ___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Is it possible to write a custom serializer parser for BaseX?
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Liam,
Thanks again. We luckily are not talking terabytes. The larger documents I have
seen are in the 1 - 5 gigs with most under 15mb. I would think each 'document'
would be a resource in the database. Its the 1+ gig DOMs that are our problem..
Also having to load the entire Blob/DOM at once is horrible for load
performance. The RestXQ/Hypermedia approach would allow lazy loading via
hypermedia driven discover-ability but still support full XPath/XQuery level
querying as previously relegated to the DOM alone.
The nice thing about our metadata approach is we actually have several levels
of metadata which is then 'combined' into user/group specific metadata to allow
for user/group level configuration without undermining core system functions
and caching mechanisms.
The RestXQ endpoints simply consume the 'System wide' metadata for building
entity representations and the client consumes their own 'personal' metadata
for client side representation This personal metadata is a mashup of the
system metadata, any group metadata they are apart of and personal metadata
alterations. For example Bob could make the 'NeedsRepair' field invisible on
Transformer Inspection records...
This 'custom' metadata is consumed by the client and the html markup is
generated based on it. Again we use clietside js via knockout.js for this. The
key benefit here is the data representation does not change and can be consumed
by all clients regardless of their metadata alterations. This single
representation can then be subsequently cached efficiently through the network
with caching headers which are configurable via the system wide entity
metadata. For example maybe Transformers have a freshness of 1 minute while
Inspection Records are fresh for 2 days...
Although there is only 'one' representation of each resource private/sensitive
data is always present and encrypted with a secret key per resource where the
secret key is only available to those with privileges.
So for example:
api/employees/2
may return
{ name: 'Bob' salary: 'encryptedstringhere', isActive: 'true' }
This resource can be 'shared' by everyone. If someone has permissions to see
'salary' they can request:
api/employees/2/keys
This would return a sequence of secret keys for the properties encrypted in the
resource for all properties the user has permissions to see.
Anyway I think I got carried away... I am just excited!
I really just wanted to say thanks again, there is a ton of documentation on
XML and its hard to wade through it all efficiently.. Your input was
invaluable. The approach you outlined is in line with the way I was thinking I
would implement query-able relationships should xml not have those facilities
inhouse. Also thanks for the SML reference. It looks promising.
Ill leave you alone and check out that query-talk group..
Once again! Thanks and have a great rest of your week!
Hopefully in the coming weeks Ill know if this will all work as we envision or
fails miserably. :)
- James
> Subject: Re: [basex-talk] Referential Queries
> From: l...@w3.org
> To: james...@hotmail.com
> CC: basex-talk@mailman.uni-konstanz.de
> Date: Tue, 14 May 2013 15:13:59 -0400
>
> [I think this thread is getting further away from BaseX, and might
> belong on query-talk instead, but on the other hand the use of XQuery as
> a back-end for Web Apps is definitely on the increase]
>
> On Tue, 2013-05-14 at 11:14 -0600, James Wright wrote:
> > Hello Again,
> > If this is the wrong forum for these type of questions let me know. By
> > the way Liam I picked up your book last night, I like the flavor as it
> > differs from my other reads such as those from Kay. Although I have
> > been using XML for years and understand the core concepts it should be
> > a great refresher.
>
> Thanks, I wrote the boring chapters :-)
>
> > The Organizational Overall Problem:
> > There aren't many people in my industry that use XQuery and xml in
> > the way it was intended (IMHO). In fact most developers in my
> > organization are rather uneducated in it and as you know there is some
> > un-rational backlash as many correlate XML to the DOM and XPath/XSLT
> > 1.0 and as a competitor to JSON which is ludicrous.
>
> You're right, it's crazy and unfortunate.
>
> XML was originally designed as an interoperable way to put SGML
> technical documentation on the Web in Netscape plugins!
>
> > The DOM has its issues of scale-ability which our products are
> > currently running into. This isn't really xml's or the DOM's problem
> > but simply poor implementation. As you know though, all that matters
> > is perception.
>
> If it helps, XQuery, Xpath 2 and lat
Hello Again,
If this is the wrong forum for these type of questions let me know. By the way
Liam I picked up your book last night, I like the flavor as it differs from my
other reads such as those from Kay. Although I have been using XML for years
and understand the core concepts it should be a great refresher. If you have
time to read this and respond I appreciate it... If not I understand. :)
The Organizational Overall Problem:
There aren't many people in my industry that use XQuery and xml in the way it
was intended (IMHO). In fact most developers in my organization are rather
uneducated in it and as you know there is some un-rational backlash as many
correlate XML to the DOM and XPath/XSLT 1.0 and as a competitor to JSON which
is ludicrous. The DOM has its issues of scale-ability which our products are
currently running into. This isn't really xml's or the DOM's problem but simply
poor implementation. As you know though, all that matters is perception.
I am having to work with a large number of un-schemad, basically hack job,
developed xml documents and workflows. A lot of our product utilizes XSLT for
reporting and transformation however only a few in the team understand the
concepts and due to MS/Managerial BS we are stuck with a .NET XSLT 1.0
processor. Also almost all of our XSLT scripts utilize a Pull pattern which I
find overly verbose and inefficient, but that is a personal opinion.
I have been researching alternatives to the DOM and .NET's standard processor
(I have used Saxon too) because I personally find XML useful and the query
semantics of XQuery 3.0 awesome.
Proposed Solutions:
We have two primary use cases:
1) as a local db to replace the context DOM for our 'documents' which in our
case relates to Utilitiy GIS Designs of circuit, subdivisions, fiber, etc... I
am thinking BaseX coupled with RestXQ could replace our DOM for local installs
and allow ourselves to decouple from the Geodatabase and provide a browser
based UI. (Currently our xml documents are stored in a blob in the GIS
database) The GIS we are using is ESRI however we are interested in also
supporting Open Street Maps. (Noticed the Geo Module)
2) as a service for hosting uploading and allowing users/delivery and support
to view, query and modify complex sets of interrelated XML configuration files.
Some of our applications have hundreds. Again all these documents follow a
similar semantics however their is no defined schema for any of them. Currently
the client has to read a 50 page manual and edit the files one at time. Often
there are nodes in several files which must match exactly or the entire
application fails Its a nightmare and there is no concept of generalization
in my organization when it comes to development. Every tool we do have to
'configure' is hand crafted and unique. Its abysmal! Because of this we
have tools that cover probably only 20% of the configuration.
I think we can accomplish both of the above tasks using a single codebase and
restXQ
I have written an XQuery expression which using our 'common' xml semantics can
ascertain entities/properties/relationships and distill this in the form of
metadata which then using RestXQ is distilled into a metadata driven api for
manipulated data centric xml documents. Similar to how you can transform a well
formed xml document into an xsd however our metadata format compliments XSD but
servers broader purposes for example to annotate triggers/mappings etc.
This metadata is then consumed for each RestXQ operation to allow for a
generalized API and all markup is applied on the Client to allow for more
efficient caching mechanisms since users can share resources between different
representations more efficiently. We currently use Knockout.js on the client
for this.
Here is an example of a generalized API endpoint for retrieving a unique entity
by type.
(: Removed error handling and some other stuff for brevity. Its not fully
functional/vindicated but is more a representation of the concept :)declare
%restxq:path("api/{$entityListName}/{$entityId}")%restxq:GET
function page:GetEntity($entityListName as xs:string, $entityId as xs:string) {
let $entityMetadata := $page:database/metadata/entity[@type =
$entityListName] let $entity :=
if($entityMetadata/property[@key = 'true']) then (: Use the properties marked
in the metadata as the key :) $page:database//*[name() =
$entityListName and data((*|@*)[name() = $entityMetadata/property[@key =
'true']/@name]) = $entityId] {
(: Construct the representation based on the metadata :) for
$propMeta in $entityMetadata/property return attribute
{$propMeta/@name} { data($entity/(*|@*)[name() = $propMeta/@name]) }
}{ (: Include a link to all related entities for
further discovery by the client.. :)
Hey Liam,
I really appreciate your time with this. That really clears everything up.
Thanks again.- James
> Subject: Re: [basex-talk] Question regarding BaseX support for id and idref
> From: l...@w3.org
> To: james...@hotmail.com
> CC: basex-talk@mailman.uni-konstanz.de
> Date: Mon, 13 May 2013 23:53:54 -0400
>
> On Mon, 2013-05-13 at 21:01 -0600, James Wright wrote:
> > Liam,
> > I think that answers my question and is what I expected. So basically
> > BaseX and maybe XQuery as a whole does not respect or understand the
> > id/idref concept formally and thus will not automatically join.
>
> It's actually not strictly true - a DTD-aware XQuery processor can
> supply the id() and idref() XPath functions. However, these do not work
> across documents, and the values must be legal XML identifiers.
>
> Having said that, most XML databases make indexes, so joins using FLWOR
> and values will work based on values.
>
> Although there are no integrity constraints as such, you can use XML
> Schema to validate incoming data in some implementations automatically,
> and in others by running a validator as part of some external process.
>
> > [...]
>
> > I need to handle the relationship explicitly.
>
> Yes.
>
> Note also, by the way, that XML databases tend to go fastest when you
> have
> . few or zero XML namespaces
> . lots of different XML element names
> . content rather than attributes (as then you can use full text).
>
> But this is only a general rule of thumb; like all database query
> languages you can sometimes write a query that takes milliseconds and
> sometimes you write one that takes days. Luckily XQuery has been around
> for a while and the surviving implementations generally do some pretty
> good optimization.
>
> Liam
>
> --
> Liam Quin - XML Activity Lead, W3C, http://www.w3.org/People/Quin/
> Pictures from old books: http://fromoldbooks.org/
> Ankh: irc.sorcery.net irc.gnome.org freenode/#xml
>
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Liam,
I think that answers my question and is what I expected. So basically BaseX and
maybe XQuery as a whole does not respect or understand the id/idref concept
formally and thus will not automatically join. This isn't a 'problem' so much
as an attempt to insure I do not reproduce what already exists.
Does this sound correct? For example if I had a node named item with 2 child
nodes simply referencing another childitem node by ID but not inline.
with item as the context node. If I wrote:
childitem
I would expect the nodes being referenced not the inline nodes with the
references on them. But this is not how it works.. Right? I need to handle the
relationship explicitly.
- James
> Subject: Re: [basex-talk] Question regarding BaseX support for id and idref
> From: l...@w3.org
> To: james...@hotmail.com
> CC: basex-talk@mailman.uni-konstanz.de
> Date: Mon, 13 May 2013 21:36:34 -0400
>
> On Mon, 2013-05-13 at 19:15 -0600, James Wright wrote:
> > [...] I know the xml standard defines id and idref however I have not
> > been able to find any documentation on these in BaseX or XML in
> > general. For example does BaseX handle idiosyncrasies of id and idref
> > or must I handle these in my queries? For example:
> > let $context :=
>
> Note that an XML ID must be an identifier, so must start with a name
> start character (a letter)...
>
> > let $context :=
> >
> >
> >
> > for $itemWithFirstChild in $context/item[child/@name = 'firstChild']return
> > $itemWithFirstChild/@id
>
> I think this got garbled somewhere - e.g. there aren't enough end tags.
> Make sure you turn off HTML formatting in your email program.
>
> > I would like this to return "10 20"Now this example uses a dynamic
> > context node however in the application these would exist as nodes in
> > the database...
> > If it does fully support this how do I enable it or get it to work?
>
> One way to get started with BaseX would be to use the "basexgui" program
> in the bin directory to create a database (there's also a command-line
> program to do it; I use the Perl and PHP APIs too on
> www.fromoldbooks.org).
>
> People usually do joins by value in XML and XQuery; ID/IDREF only work
> with older DTD technology, or with xml:id; I don't know if BaseX has
> support for them. But if it does the values must be IDs :)
>
> So you're on the right lines.
>
> > Also what about XLink?
>
> It's not used very much. As a standard it solved the wrong problems,
> unfortunately. Easy for me to say in hindsight.
>
> The XQuery Use Cases on www.w3.org/TR may be helpful; there's also a
> chapter on XQuery using BaseX for examples in a book I co-authored last
> year, "Beginning XML" 5th edition.
>
> Liam
>
> --
> Liam Quin - XML Activity Lead, W3C, http://www.w3.org/People/Quin/
> Pictures from old books: http://fromoldbooks.org/
> Ankh: irc.sorcery.net irc.gnome.org freenode/#xml
>
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Hello BaseX-Talk,
I am new to the boards and BaseX in general. I have been looking for a open
source NoSQL database with expressive query abilities and I think BaseX fits
the bill. Its been an exciting journey as I am new to the whole X platform
aside from the angle brackets... As I think many are.
I am currently exploring the capabilities of RestXQ and the xml stack. I
especially like its potential coupled with knockout.js, and a truly flexible
RESTful metadata/hypermedia driven api. I have implemented an api of this
nature in .Net however the metadata and querying capabilities are hindered and
unnecessarily complex where I feel xml and XQuery make this dynamic
orchestration quite gracefully.
Unfortunately I am stuck now that I am exploring referential relationships and
xml. I am trying to understand the support BaseX and XML in general has for
referential nodes especially when dealing with queries. For example I know the
xml standard defines id and idref however I have not been able to find any
documentation on these in BaseX or XML in general. For example does BaseX
handle idiosyncrasies of id and idref or must I handle these in my queries? For
example:
let $context :=
for $itemWithFirstChild in $context/item[child/@name = 'firstChild']return
$itemWithFirstChild/@id
I would like this to return "10 20"Now this example uses a dynamic context node
however in the application these would exist as nodes in the database...
If it does fully support this how do I enable it or get it to work? Also what
about XLink?
Any help would be greatly appreciated. Thanks again for your time.- James
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
25 matches
Mail list logo
