Re: [basex-talk] proc:system ?
Andy,
Thank you for your answer. Indeed, the length of my parameters are now much
more than the Windows limit.
I guess that creating a temporary file, then passing the name of the file
as parameters should solve the issue.
Cheers,
Jean-Marc
2017-10-13 10:42 GMT+02:00 Andy Bunce :
> Hi Jean-Marc,
>
> I have hit this kind of issue before and it is not BaseX specific. The
> problem is Windows(tm) limits [1]
> This is compounded by the Java call from proc:system passing in a copy of
> the environment of the current process [2].
> Mine has built up a large class path over time with my lib/custom
> experiements. My `proc:property("java.class.path")` is nearly 4k
>
> You could try to "clean" your classpath and switch your file locations to
> shorter paths
>
> /Andy
>
> [1] https://support.microsoft.com/en-gb/help/830473/command-prom
> pt-cmd--exe-command-line-string-limitation
> [2] https://docs.oracle.com/javase/7/docs/api/java/lang/ProcessB
> uilder.html#environment()
>
> On 12 October 2017 at 11:19, jean-marc Mercier > wrote:
>
>> Hello BaseX team,
>>
>> I am an extensive user of the proc:system BaseX command (and looking
>> forward for proc:fork !).
>> I am using this command with in-memory arguments, it means that my way
>> of using it is :
>> proc:system("my_exe", $my_arguments_for_exe)
>>
>> $my_arguments_for_exe is an xml node serialized to string by BaseX. As my
>> test are getting more complex, the size of $my_arguments_for_exe are
>> increasing. It seems that we are reaching a limit now, since the compiler
>> refuses to pass arguments :
>>
>> "Stopped at CoDeFi_Interface.xqm, 44/22:
>> [BXPR] Cannot run program
>> "C:\Jiheme\scientifique\recherche\CoDeFi\CoDeFi\CoDeFi.exe":
>> CreateProcess error=206, Nom de fichier ou extension trop long
>> "
>> Is there a way to bypass this ?
>>
>> Thank you for your answer,
>>
>> Cheers,
>>
>> Jean-Marc
>>
>
>
[basex-talk] proc:system ?
Hello BaseX team,
I am an extensive user of the proc:system BaseX command (and looking
forward for proc:fork !).
I am using this command with in-memory arguments, it means that my way of
using it is :
proc:system("my_exe", $my_arguments_for_exe)
$my_arguments_for_exe is an xml node serialized to string by BaseX. As my
test are getting more complex, the size of $my_arguments_for_exe are
increasing. It seems that we are reaching a limit now, since the compiler
refuses to pass arguments :
"Stopped at CoDeFi_Interface.xqm, 44/22:
[BXPR] Cannot run program
"C:\Jiheme\scientifique\recherche\CoDeFi\CoDeFi\CoDeFi.exe": CreateProcess
error=206, Nom de fichier ou extension trop long
"
Is there a way to bypass this ?
Thank you for your answer,
Cheers,
Jean-Marc
Re: [basex-talk] Strange compilator behavior ?
Christian, Yes absolutely. Thx for your answer. Cheers, Jean-Marc 2017-10-09 18:00 GMT+02:00 Christian Grün : > > Yes. But my question is : why did the BaseX compiler ran the query let > $idd > > := $elt[d]/@id ? > > The query is valid, because $elt could have an element named d. For > example, the following query returns the value of $elt: > > let $elt := > return $elt[d] > > Does this help? > Christian > > > > > Shouldn't it raise a compilation error ? > > > > > > > > 2017-10-09 17:49 GMT+02:00 Christian Grün : > >> > >> Dear Jean-Marc, > >> > >> > let $idd := $elt[d]/@id > >> > >> Shouldn’t it be '$d' instead of 'd' ? > >> > >> let $idd := $elt[$d]/@id > >> > >> Cheers, > >> Christian > >> > >> > >> > >> > return ($elt[$d]/@id,$idd) > >> > > >> > output : id="yolo". > >> > > >> > (Hence missing the second "yolo", because the correct query is : let > >> > $idd := > >> > $elt[$d]/@id) > >> > > >> > However I can't figure out why the BaseX compiler ran this query ? > >> > Shouldn't > >> > it refused the compilation and point out this error ? > >> > > >> > Cheers, > >> > > >> > Jean-Marc > >> > > >> > > >> > > > > > >
Re: [basex-talk] Strange compilator behavior ?
Hello Christian, Yes. But my question is : why did the BaseX compiler ran the query let $idd := $elt[d]/@id ? Shouldn't it raise a compilation error ? 2017-10-09 17:49 GMT+02:00 Christian Grün : > Dear Jean-Marc, > > > let $idd := $elt[d]/@id > > Shouldn’t it be '$d' instead of 'd' ? > > let $idd := $elt[$d]/@id > > Cheers, > Christian > > > > > return ($elt[$d]/@id,$idd) > > > > output : id="yolo". > > > > (Hence missing the second "yolo", because the correct query is : let > $idd := > > $elt[$d]/@id) > > > > However I can't figure out why the BaseX compiler ran this query ? > Shouldn't > > it refused the compilation and point out this error ? > > > > Cheers, > > > > Jean-Marc > > > > > > >
[basex-talk] Strange compilator behavior ?
Hello BaseX team ! I lost some time to detect an error in my code, and wonder if there is a minor bug into the BaseX compiler. Here is the test let $elt as element(*)*:= for $d in (1 to 1) return let $idd := $elt[d]/@id return ($elt[$d]/@id,$idd) output : id="yolo". (Hence missing the second "yolo", because the correct query is : let $idd := $elt[$d]/@id) However I can't figure out why the BaseX compiler ran this query ? Shouldn't it refused the compilation and point out this error ? Cheers, Jean-Marc
Re: [basex-talk] map behavior ?
Hello Christian !
Nice to read you again ! I am striking back to BaseX, using it as my main
database server for a integrating / pricing / unittesting engine.
Ok, fair enough, I implement a sort_map function.
2017-09-26 12:01 GMT+02:00 Christian Grün :
> Dear Jean-Marc,
>
> Welcome back!
>
> Maps in XQuery are simple key/value pairs; neither keys nor values
> will be arranged in a specific order.
>
> If you want to sort map by keys before outputting them, you can sort
> them in a second step:
>
> let $map := map:merge((
> map:entry(0.1, "Sunday"),
> map:entry(5.0, "Thursday"),
> map:entry(2.0, "Monday"),
> map:entry(3.0, "Wednesday"),
> map:entry(0.5, "Tuesday"),
> map:entry(6.0, "Friday"),
> map:entry(7.0, "Saturday")
> ))
> for $key in sort(map:keys($map))
> return "* " || $key || ": " || $map($key)
>
> Hope this helps,
> Christian
>
>
>
>
> On Tue, Sep 26, 2017 at 11:29 AM, jean-marc Mercier
> wrote:
> > Hello BaseX team,
> >
> > It has been a while since I last wrote to this mailing list ! Happy to
> > report again :)
> >
> > I have a strange behavior concerning map, it doesn't seem to sort as I
> > expected. Is there any explanation ?
> >
> > let $map as map(xs:double,xs:string) := map:merge((
> > map:entry(xs:double(0.1), "Sunday"),
> > map:entry(xs:double(5.0), "Thursday"),
> > map:entry(xs:double(2.0), "Monday"),
> > map:entry(xs:double(3.0), "Wednesday"),
> > map:entry(xs:double(0.5), "Tuesday"),
> > map:entry(xs:double(6.0), "Friday"),
> > map:entry(xs:double(7.0), "Saturday")
> > ))
> > return $map
> >
> > output :
> > map {
> > 2: "Monday",
> > 3: "Wednesday",
> > 5: "Thursday",
> > 6: "Friday",
> > 7: "Saturday",
> > 0.1: "Sunday",
> > 0.5: "Tuesday"
> > }
> >
> > Cheers
>
[basex-talk] map behavior ?
Hello BaseX team,
It has been a while since I last wrote to this mailing list ! Happy to
report again :)
I have a strange behavior concerning map, it doesn't seem to sort as I
expected. Is there any explanation ?
let $map as map(xs:double,xs:string) := map:merge((
map:entry(xs:double(0.1), "Sunday"),
map:entry(xs:double(5.0), "Thursday"),
map:entry(xs:double(2.0), "Monday"),
map:entry(xs:double(3.0), "Wednesday"),
map:entry(xs:double(0.5), "Tuesday"),
map:entry(xs:double(6.0), "Friday"),
map:entry(xs:double(7.0), "Saturday")
))
return $map
output :
map {
2: "Monday",
3: "Wednesday",
5: "Thursday",
6: "Friday",
7: "Saturday",
0.1: "Sunday",
0.5: "Tuesday"
}
Cheers
Re: [basex-talk] Basex Standalone : [bxerr:BXSQ0007] Could not initialize supplied driver: 'com.mysql.jdbc.Driver'
Christian,
Hello. Thx for your answer. I've tried your suggestion, but I can't set the
Java Build path using Eclipse/ Project / Properties (my IDE is eclipse
together with BaseX Interpreter) :
[image: Images intégrées 1]
This is a minor issue : I can survive using the BaseX server. It is only
hardening dev / debug, since I don't know how to trace using BAseX server
(fn:trace seems not to output anything using the basex server, maybe I
missed something here ?).
Cheers
2015-05-11 22:05 GMT+02:00 Christian Grün :
> Hi Jean-Marc,
>
> > I guess that it might be due to the fact that I don't know how to include
> > multiple jar while using with eclipse. I tried to go to
> > Windows/Preferences/XQUERY/Interpreters and set -classpath
> > {BaseXDirectory}\lib\*, but it does not work.
>
> You will probably have to include the libraries as follows:
>
> Project → Properties → Java Build Path →
> Libraries → Add JARs...
>
> Hope this helps,
> Christian
>
>
>
> >
> > Maybe somebody already met this small issue before ?
> >
> > Thanks for your answer,
> >
> > cheers
>
[basex-talk] Basex Standalone : [bxerr:BXSQ0007] Could not initialize supplied driver: 'com.mysql.jdbc.Driver'
Hi,
I am struggling to use Basex sql module together with Basex Standalone. I
am getting the error : [bxerr:BXSQ0007] Could not initialize supplied
driver: 'com.mysql.jdbc.Driver'
The sql connection works finely while using the Basex server.
I guess that it might be due to the fact that I don't know how to include
multiple jar while using with eclipse. I tried to go to
Windows/Preferences/XQUERY/Interpreters and set -classpath
{BaseXDirectory}\lib\*, but it does not work.
Maybe somebody already met this small issue before ?
Thanks for your answer,
cheers
Re: [basex-talk] C++ BaseX Client
Christian, Nice, thanks. Take care however, this C++ client has not been tested by anyone except me ! I will add a readme.txt for reporting issues to BaseX list. 2015-02-02 13:34 GMT+01:00 Christian Grün : > Jean-Marc, > > Thanks for the new client! I have added your link to our list of bindings > [1]. > > All the best, > Christian > > [1] http://docs.basex.org/wiki/Clients > > On Mon, Feb 2, 2015 at 12:51 PM, jean-marc Mercier > wrote: > > Christian, > > > > Hello. I just commited changes to > https://github.com/JohnLeM/BasexCPPAPI/ > > Major changes : > > - C++ client for BaseX 8.0, according to the new digest protocol > > (http://docs.basex.org/wiki/Clients) but it should also works for > ancient > > version (I tested a 7.8). > > - simplified C++ interface to BaseX > > > > I started also to standardize the client to your client architecture : > > execute(), info(), close() > > I will slowly implement other commands. > > > > Note : > > - I compiled it under VS2015, boost 1_57, Platform 64 (important for > > addressing memory larger than 1,5 Gb) and 32. Seems to work finely. > > - I guess than the makefile is outdated now. I did not install gcc on my > > comp (always a pain to Mingw / gcc a windows installation ...), maybe > Dirk > > could have a look at it ? > > > > Cheers, Jean-Marc > > > > > > > > 2015-01-31 17:55 GMT+01:00 jean-marc Mercier >: > >> > >> Hi christian, > >> > >> I should probably update this old repository : > >> https://github.com/JohnLeM/BasexCPPAPI/ > >> > >> However you already made a remark concerning this code : I should update > >> this C++ client to your standard client architecture. I'll try to > update the > >> client before commiting, let me some days left. > >> > >> For information, I am toying with this client to switch from XQUERY to > >> C++, due to performance issues related to BaseX map and array > computations. > >> I will try to report this problem, it might be a quite instructive and > >> interesting one. > >> > >> > >> 2015-01-31 16:31 GMT+01:00 Christian Grün : > >>> > >>> Hi Jean-Marc, > >>> > >>> Good to hear! Where can I find the client code? > >>> > >>> Christian > >>> > >>> > >>> On Sat, Jan 31, 2015 at 4:10 PM, jean-marc Mercier > >>> wrote: > >>> > Hi all, > >>> > > >>> > Just to report that I updated a C++ client for BaseX 8.0, according > to > >>> > the > >>> > new digest protocol (http://docs.basex.org/wiki/Clients) > >>> > > >>> > Cheers > >> > >> > > >
Re: [basex-talk] C++ BaseX Client
Christian, Hello. I just commited changes to https://github.com/JohnLeM/BasexCPPAPI/ Major changes : - C++ client for BaseX 8.0, according to the new digest protocol ( http://docs.basex.org/wiki/Clients) but it should also works for ancient version (I tested a 7.8). - simplified C++ interface to BaseX I started also to standardize the client to your client architecture : execute(), info(), close() I will slowly implement other commands. Note : - I compiled it under VS2015, boost 1_57, Platform 64 (important for addressing memory larger than 1,5 Gb) and 32. Seems to work finely. - I guess than the makefile is outdated now. I did not install gcc on my comp (always a pain to Mingw / gcc a windows installation ...), maybe Dirk could have a look at it ? Cheers, Jean-Marc 2015-01-31 17:55 GMT+01:00 jean-marc Mercier : > Hi christian, > > I should probably update this old repository : > https://github.com/JohnLeM/BasexCPPAPI/ > > However you already made a remark concerning this code : I should update > this C++ client to your standard client architecture. I'll try to update > the client before commiting, let me some days left. > > For information, I am toying with this client to switch from XQUERY to > C++, due to performance issues related to BaseX map and array computations. > I will try to report this problem, it might be a quite instructive and > interesting one. > > > 2015-01-31 16:31 GMT+01:00 Christian Grün : > >> Hi Jean-Marc, >> >> Good to hear! Where can I find the client code? >> >> Christian >> >> >> On Sat, Jan 31, 2015 at 4:10 PM, jean-marc Mercier >> wrote: >> > Hi all, >> > >> > Just to report that I updated a C++ client for BaseX 8.0, according to >> the >> > new digest protocol (http://docs.basex.org/wiki/Clients) >> > >> > Cheers >> > >
Re: [basex-talk] C++ BaseX Client
Hi christian, I should probably update this old repository : https://github.com/JohnLeM/BasexCPPAPI/ However you already made a remark concerning this code : I should update this C++ client to your standard client architecture. I'll try to update the client before commiting, let me some days left. For information, I am toying with this client to switch from XQUERY to C++, due to performance issues related to BaseX map and array computations. I will try to report this problem, it might be a quite instructive and interesting one. 2015-01-31 16:31 GMT+01:00 Christian Grün : > Hi Jean-Marc, > > Good to hear! Where can I find the client code? > > Christian > > > On Sat, Jan 31, 2015 at 4:10 PM, jean-marc Mercier > wrote: > > Hi all, > > > > Just to report that I updated a C++ client for BaseX 8.0, according to > the > > new digest protocol (http://docs.basex.org/wiki/Clients) > > > > Cheers >
[basex-talk] C++ BaseX Client
Hi all, Just to report that I updated a C++ client for BaseX 8.0, according to the new digest protocol (http://docs.basex.org/wiki/Clients) Cheers
Re: [basex-talk] try catch mechanisms and inlining
Hi christian,
In this case it would be useless : it appears that the optimized query
consists in dumping this big map (about 400 000 000 integers), consisting
in several Giga octets of data when serialized.
2015-01-17 18:27 GMT+01:00 Christian Grün :
> Hi Jean-Marc,
>
> > Note that this analysis is quite difficult to
> > do because I can't use xquery info view for my "real" queries : the gui
> is
> > freezing. It appears that it is trying to dump an enormous map with
> which I
> > am working.
>
> You can also print the query info with the command-line flag -V.
>
> Hope this helps,
> C.
>
>
> >
> >
> > declare function local:test($i){fn:for-each(1 to $i,
> function($a){2*$a})};
> > declare function local:fun-profile($fun,$args) {try{$fun($args)} catch
> > *{()}};
> > declare function local:fun-profile2($fun,$args) {$fun($args)};
> >
> > prof:time(count(local:fun-profile(local:test#1, 1000))) : 843.18 ms
> > (with BaseX gui)
> >
> >
> > 2015-01-17 17:00 GMT+01:00 Christian Grün :
> >>
> >> Hi Jean-Marc,
> >>
> >> This time, it was not that obvious to run your query (I stlil didn't
> >> find local:call). But my guess is that the try/catch query is simply
> >> slower, because catching errors simply takes time. I invite you to
> >> look into the info view; it will show you what is inlined and what is
> >> not.
> >>
> >> Best,
> >> Christian
> >>
> >>
> >> On Sat, Jan 17, 2015 at 3:40 PM, jean-marc Mercier
> >> wrote:
> >> > Hello,
> >> >
> >> > To complete the previous threads :
> >> > 1) I forgot a definition : declare function local:call($fun,$args)
> >> > {$fun($args)};
> >> > 2) The same behavior seems to occurs with the following code, not
> >> > involving
> >> > try catch expression :
> >> >
> >> > declare function local:test($i){fn:for-each(1 to $i,
> >> > function($a){2*$a})};
> >> > declare variable $test := prof:time(local:test(1000),false(),"2
> :");
> >> > prof:time(count(local:test(1000)),false(),"1 :"),count($test)
> >> >
> >> > Output :
> >> > 2 :2508.42 ms
> >> > 1 :710.51 ms
> >> >
> >> >
> >> >
> >> > 2015-01-17 13:27 GMT+01:00 jean-marc Mercier
> >> > :
> >> >>
> >> >> Hello all,
> >> >>
> >> >> I wrote some basic debugging / profiling tools for xquery code, using
> >> >> try{} catch *{} mechanisms. However, profiling tools are not
> reliable.
> >> >>
> >> >> The suspected reason is that inlining is not activated in try catch
> >> >> expression. If confirmed, is it a desired behavior ?
> >> >>
> >> >> Here is the basic test :
> >> >>
> >> >> declare function local:test($i){fn:for-each(1 to $i,
> >> >> function($a){2*$a})};
> >> >> declare function local:fun-profile($fun,$args)
> >> >> {try{local:call($fun,$args)} catch *{()}};
> >> >> declare function local:fun-profile2($fun,$args)
> >> >> {local:call($fun,$args)};
> >> >>
> >> >> With default options (ie inlining activated)
> >> >>
> >> >> prof:time(count(local:fun-profile2(local:test#1, 1000))) Output :
> >> >> 683.12 ms
> >> >> prof:time(count(local:fun-profile(local:test#1, 1000))) Output
> :
> >> >> 2465.47 ms
> >> >>
> >> >> With inlining desactivated (ie INLINELIMIT=0)
> >> >> prof:time(count(local:fun-profile2(local:test#1, 1000))) Output :
> >> >> 2480.32 ms
> >> >>
> >> >
> >
> >
>
Re: [basex-talk] try catch mechanisms and inlining
Christian,
Hi, thx for your answer, sry for forgetting local:call($fun,$args)
{$fun($args)} definition. I ran the test within BaseX gui as you suggested,
but it is showing a "correct" behavior (see below). Thus this point
is probably due to an overhead in using eclipse.
I registered anyhow some suspicious behavior while profiling, but clearly
need more time to analyze the point. I will probably come back over this
topic soon if you don't mind. Note that this analysis is quite difficult to
do because I can't use xquery info view for my "real" queries : the gui is
freezing. It appears that it is trying to dump an enormous map with which I
am working.
declare function local:test($i){fn:for-each(1 to $i, function($a){2*$a})};
declare function local:fun-profile($fun,$args) {try{$fun($args)} catch
*{()}};
declare function local:fun-profile2($fun,$args) {$fun($args)};
prof:time(count(local:fun-profile(local:test#1, 1000))) : 843.18 ms
(with BaseX gui)
2015-01-17 17:00 GMT+01:00 Christian Grün :
> Hi Jean-Marc,
>
> This time, it was not that obvious to run your query (I stlil didn't
> find local:call). But my guess is that the try/catch query is simply
> slower, because catching errors simply takes time. I invite you to
> look into the info view; it will show you what is inlined and what is
> not.
>
> Best,
> Christian
>
>
> On Sat, Jan 17, 2015 at 3:40 PM, jean-marc Mercier
> wrote:
> > Hello,
> >
> > To complete the previous threads :
> > 1) I forgot a definition : declare function local:call($fun,$args)
> > {$fun($args)};
> > 2) The same behavior seems to occurs with the following code, not
> involving
> > try catch expression :
> >
> > declare function local:test($i){fn:for-each(1 to $i,
> function($a){2*$a})};
> > declare variable $test := prof:time(local:test(1000),false(),"2 :");
> > prof:time(count(local:test(1000)),false(),"1 :"),count($test)
> >
> > Output :
> > 2 :2508.42 ms
> > 1 :710.51 ms
> >
> >
> >
> > 2015-01-17 13:27 GMT+01:00 jean-marc Mercier >:
> >>
> >> Hello all,
> >>
> >> I wrote some basic debugging / profiling tools for xquery code, using
> >> try{} catch *{} mechanisms. However, profiling tools are not reliable.
> >>
> >> The suspected reason is that inlining is not activated in try catch
> >> expression. If confirmed, is it a desired behavior ?
> >>
> >> Here is the basic test :
> >>
> >> declare function local:test($i){fn:for-each(1 to $i,
> function($a){2*$a})};
> >> declare function local:fun-profile($fun,$args)
> >> {try{local:call($fun,$args)} catch *{()}};
> >> declare function local:fun-profile2($fun,$args)
> {local:call($fun,$args)};
> >>
> >> With default options (ie inlining activated)
> >>
> >> prof:time(count(local:fun-profile2(local:test#1, 1000))) Output :
> >> 683.12 ms
> >> prof:time(count(local:fun-profile(local:test#1, 1000))) Output :
> >> 2465.47 ms
> >>
> >> With inlining desactivated (ie INLINELIMIT=0)
> >> prof:time(count(local:fun-profile2(local:test#1, 1000))) Output :
> >> 2480.32 ms
> >>
> >
>
Re: [basex-talk] try catch mechanisms and inlining
Hello,
To complete the previous threads :
1) I forgot a definition : declare function local:call($fun,$args)
{$fun($args)};
2) The same behavior seems to occurs with the following code, not involving
try catch expression :
declare function local:test($i){fn:for-each(1 to $i, function($a){2*$a})};
declare variable $test := prof:time(local:test(1000),false(),"2 :");
prof:time(count(local:test(1000)),false(),"1 :"),count($test)
Output :
2 :2508.42 ms
1 :710.51 ms
2015-01-17 13:27 GMT+01:00 jean-marc Mercier :
> Hello all,
>
> I wrote some basic debugging / profiling tools for xquery code, using
> try{} catch *{} mechanisms. However, profiling tools are not reliable.
>
> The suspected reason is that inlining is not activated in try catch
> expression. If confirmed, is it a desired behavior ?
>
> Here is the basic test :
>
> declare function local:test($i){fn:for-each(1 to $i, function($a){2*$a})};
> declare function local:fun-profile($fun,$args)
> {try{local:call($fun,$args)} catch *{()}};
> declare function local:fun-profile2($fun,$args) {local:call($fun,$args)};
>
> With default options (ie inlining activated)
>
> prof:time(count(local:fun-profile2(local:test#1, 1000))) Output
> : 683.12 ms
> prof:time(count(local:fun-profile(local:test#1, 1000))) Output
> : 2465.47 ms
>
> With inlining desactivated (ie INLINELIMIT=0)
> prof:time(count(local:fun-profile2(local:test#1, 1000))) Output
> : 2480.32 ms
>
>
[basex-talk] try catch mechanisms and inlining
Hello all,
I wrote some basic debugging / profiling tools for xquery code, using try{}
catch *{} mechanisms. However, profiling tools are not reliable.
The suspected reason is that inlining is not activated in try catch
expression. If confirmed, is it a desired behavior ?
Here is the basic test :
declare function local:test($i){fn:for-each(1 to $i, function($a){2*$a})};
declare function local:fun-profile($fun,$args) {try{local:call($fun,$args)}
catch *{()}};
declare function local:fun-profile2($fun,$args) {local:call($fun,$args)};
With default options (ie inlining activated)
prof:time(count(local:fun-profile2(local:test#1, 1000))) Output
: 683.12 ms
prof:time(count(local:fun-profile(local:test#1, 1000))) Output
: 2465.47 ms
With inlining desactivated (ie INLINELIMIT=0)
prof:time(count(local:fun-profile2(local:test#1, 1000))) Output
: 2480.32 ms
Re: [basex-talk] Optimization for procedure call ?
Christian,
Note that executing within BaseX gui leads to different results, but seems
also to behave non linearly : for instance for $i in (15 to 20) return
prof:time(local:test1($i))
returns random results : for instance
Evaluating:
- 1368.54 ms
- 1314.6 ms
- 5151.12 ms
- 1719.01 ms
- 6023.38 ms
- 1996.37 ms
Might it be garbage collector stuff ?
2015-01-14 8:18 GMT+01:00 jean-marc Mercier :
> Glory to our holy Leo for transmitting *brahmavidya* to shishya !!
>
> Turning back to this optimization problem, this might be trickier since
> execution time does not seem to behave linearly. For instance, consider
>
> declare function local:test1($i){let $data_3 as item() *:= fn:for-each((1
> to 100*$i), function($a_0) as item()? { (2 * $a_0) }) return for $n in
> (1 to 2) return fn:count($data_3)};
> declare function local:test2($i){count(fn:for-each((1 to 100*$i),
> function($a_0) as item()? { (2 * $a_0) }))};
>
> Then execution time is ( the results are stable while repeating the
> experience )
>
> prof:time(local:test1(7)) : 631.32 ms prof:time(local:test2(7)) : 451.32
> ms
> prof:time(local:test1(8)) : 2550.17 ms prof:time(local:test2(8)) :
> 507.32 ms
>
>
>
> 2015-01-13 22:43 GMT+01:00 Christian Grün :
>
>> Hi Jean-Marc,
>>
>> beforehand: The first issue you reported back to us recently [1] has
>> been fixed by our functional guru Leo! A new snapshot is available
>> [2].
>>
>> > Hi. Thanks for the FLWOR trick, behaving in constant time. However it
>> seems
>> > to generates an unexpected 5X time overhead
>>
>> Most probably this is because the results of for-each will be cached,
>> while in the previous case (if no inlining takes place), all items
>> will be iterated.
>>
>> Hope this helps,
>> Christian
>>
>> [1] https://github.com/BaseXdb/basex/issues/1052
>> [2] http://files.basex.org/releases/latest/
>>
>>
>> On Tue, Jan 13, 2015 at 9:19 AM, jean-marc Mercier
>> wrote:
>> > Christian,
>> >
>> > Hi. Thanks for the FLWOR trick, behaving in constant time. However it
>> seems
>> > to generates an unexpected 5X time overhead, and I can't use it
>> > straightforwardly as workaround. Can you reproduce it, since this
>> overhead
>> > might be due to eclipse ?
>> >
>> > declare function local:test(){fn:for-each(1 to 1000,
>> > function($a){2*$a})};
>> > declare function local:test1(){count(local:test())};
>> > declare function local:test2(){let $ data := local:test() return for $n
>> in
>> > (1 to 10) return count($data)};
>> >
>> > prof:time(local:test1()) ouput : 639.12 ms
>> > prof:time(local:test2()) ouput : 2431.67 ms
>> >
>> >
>> >
>> > 2015-01-13 1:59 GMT+01:00 Christian Grün :
>> >>
>> >> Hi Jean-Marc,
>> >>
>> >> interesting one; it seems that $data is inlined and thus evaluated
>> >> more than once. This doesn't happen if a FLWOR expression is used:
>> >>
>> >> let $data := fn:for-each ...
>> >> for $n in 1 to $i
>> >> return local:test2($data, $n)
>> >>
>> >> Should be easy to resolve.
>> >>
>> >> Thanks for your (always concise) examples,
>> >> Christian
>> >>
>> >>
>> >> On Mon, Jan 12, 2015 at 9:08 PM, jean-marc Mercier
>> >> wrote:
>> >> > Hello,
>> >> >
>> >> > I am facing some performances issues, probably due to unexpected
>> query
>> >> > reevaluation. Might it be due to the optimizer ? Here is a code to
>> test
>> >> >
>> >> > declare function local:test($i){
>> >> > let $data := fn:for-each(1 to 1000, function($a){2*$a} )
>> >> > (: let $data := xquery:eval("fn:for-each(1 to 1000,
>> >> > function($a){2*$a} )") :)
>> >> > return (1 to $i) ! local:test2($data,.)
>> >> > };
>> >> >
>> >> > declare function local:test2($data, $dummy) { count($data) };
>> >> >
>> >> > (1 to 10) ! prof:time(local:test(.))
>> >> >
>> >> > Output :
>> >> >
>> >> > 683.76 ms
>> >> > 1137.95 ms
>> >> > 1727.68 ms
>> >> > 2151.06 ms
>> >> > 2694.84 ms
>> >> > 3189.57 ms
>> >> > 3725.1 ms
>> >> > 4277.51 ms
>> >> > 4815.74 ms
>> >> > 5417.79 ms
>> >> >
>> >> > Note that using xquery:eval (toggling the comment inside local:test).
>> >> > seems
>> >> > to perform correctly. However I can't use this trick as a workaround.
>> >> >
>> >> > (1 to 10) ! prof:time(local:test(.))
>> >> > Ouput :
>> >> >
>> >> > 2472.71 ms
>> >> > 825.61 ms
>> >> > 3013.14 ms
>> >> > 986.81 ms
>> >> > 2982.97 ms
>> >> > 778.24 ms
>> >> > 1195.33 ms
>> >> > 3280.7 ms
>> >> > 976.96 ms
>> >> > 913.69 ms
>> >
>> >
>>
>
>
Re: [basex-talk] Optimization for procedure call ?
Glory to our holy Leo for transmitting *brahmavidya* to shishya !!
Turning back to this optimization problem, this might be trickier since
execution time does not seem to behave linearly. For instance, consider
declare function local:test1($i){let $data_3 as item() *:= fn:for-each((1
to 100*$i), function($a_0) as item()? { (2 * $a_0) }) return for $n in
(1 to 2) return fn:count($data_3)};
declare function local:test2($i){count(fn:for-each((1 to 100*$i),
function($a_0) as item()? { (2 * $a_0) }))};
Then execution time is ( the results are stable while repeating the
experience )
prof:time(local:test1(7)) : 631.32 ms prof:time(local:test2(7)) : 451.32 ms
prof:time(local:test1(8)) : 2550.17 ms prof:time(local:test2(8)) : 507.32
ms
2015-01-13 22:43 GMT+01:00 Christian Grün :
> Hi Jean-Marc,
>
> beforehand: The first issue you reported back to us recently [1] has
> been fixed by our functional guru Leo! A new snapshot is available
> [2].
>
> > Hi. Thanks for the FLWOR trick, behaving in constant time. However it
> seems
> > to generates an unexpected 5X time overhead
>
> Most probably this is because the results of for-each will be cached,
> while in the previous case (if no inlining takes place), all items
> will be iterated.
>
> Hope this helps,
> Christian
>
> [1] https://github.com/BaseXdb/basex/issues/1052
> [2] http://files.basex.org/releases/latest/
>
>
> On Tue, Jan 13, 2015 at 9:19 AM, jean-marc Mercier
> wrote:
> > Christian,
> >
> > Hi. Thanks for the FLWOR trick, behaving in constant time. However it
> seems
> > to generates an unexpected 5X time overhead, and I can't use it
> > straightforwardly as workaround. Can you reproduce it, since this
> overhead
> > might be due to eclipse ?
> >
> > declare function local:test(){fn:for-each(1 to 1000,
> > function($a){2*$a})};
> > declare function local:test1(){count(local:test())};
> > declare function local:test2(){let $ data := local:test() return for $n
> in
> > (1 to 10) return count($data)};
> >
> > prof:time(local:test1()) ouput : 639.12 ms
> > prof:time(local:test2()) ouput : 2431.67 ms
> >
> >
> >
> > 2015-01-13 1:59 GMT+01:00 Christian Grün :
> >>
> >> Hi Jean-Marc,
> >>
> >> interesting one; it seems that $data is inlined and thus evaluated
> >> more than once. This doesn't happen if a FLWOR expression is used:
> >>
> >> let $data := fn:for-each ...
> >> for $n in 1 to $i
> >> return local:test2($data, $n)
> >>
> >> Should be easy to resolve.
> >>
> >> Thanks for your (always concise) examples,
> >> Christian
> >>
> >>
> >> On Mon, Jan 12, 2015 at 9:08 PM, jean-marc Mercier
> >> wrote:
> >> > Hello,
> >> >
> >> > I am facing some performances issues, probably due to unexpected query
> >> > reevaluation. Might it be due to the optimizer ? Here is a code to
> test
> >> >
> >> > declare function local:test($i){
> >> > let $data := fn:for-each(1 to 1000, function($a){2*$a} )
> >> > (: let $data := xquery:eval("fn:for-each(1 to 1000,
> >> > function($a){2*$a} )") :)
> >> > return (1 to $i) ! local:test2($data,.)
> >> > };
> >> >
> >> > declare function local:test2($data, $dummy) { count($data) };
> >> >
> >> > (1 to 10) ! prof:time(local:test(.))
> >> >
> >> > Output :
> >> >
> >> > 683.76 ms
> >> > 1137.95 ms
> >> > 1727.68 ms
> >> > 2151.06 ms
> >> > 2694.84 ms
> >> > 3189.57 ms
> >> > 3725.1 ms
> >> > 4277.51 ms
> >> > 4815.74 ms
> >> > 5417.79 ms
> >> >
> >> > Note that using xquery:eval (toggling the comment inside local:test).
> >> > seems
> >> > to perform correctly. However I can't use this trick as a workaround.
> >> >
> >> > (1 to 10) ! prof:time(local:test(.))
> >> > Ouput :
> >> >
> >> > 2472.71 ms
> >> > 825.61 ms
> >> > 3013.14 ms
> >> > 986.81 ms
> >> > 2982.97 ms
> >> > 778.24 ms
> >> > 1195.33 ms
> >> > 3280.7 ms
> >> > 976.96 ms
> >> > 913.69 ms
> >
> >
>
Re: [basex-talk] Optimization for procedure call ?
Christian,
Hi. Thanks for the FLWOR trick, behaving in constant time. However it seems
to generates an unexpected 5X time overhead, and I can't use it
straightforwardly as workaround. Can you reproduce it, since this overhead
might be due to eclipse ?
declare function local:test(){fn:for-each(1 to 1000,
function($a){2*$a})};
declare function local:test1(){count(local:test())};
declare function local:test2(){let $ data := local:test() return for $n in
(1 to 10) return count($data)};
prof:time(local:test1()) ouput : 639.12 ms
prof:time(local:test2()) ouput : 2431.67 ms
2015-01-13 1:59 GMT+01:00 Christian Grün :
> Hi Jean-Marc,
>
> interesting one; it seems that $data is inlined and thus evaluated
> more than once. This doesn't happen if a FLWOR expression is used:
>
> let $data := fn:for-each ...
> for $n in 1 to $i
> return local:test2($data, $n)
>
> Should be easy to resolve.
>
> Thanks for your (always concise) examples,
> Christian
>
>
> On Mon, Jan 12, 2015 at 9:08 PM, jean-marc Mercier
> wrote:
> > Hello,
> >
> > I am facing some performances issues, probably due to unexpected query
> > reevaluation. Might it be due to the optimizer ? Here is a code to test
> >
> > declare function local:test($i){
> > let $data := fn:for-each(1 to 1000, function($a){2*$a} )
> > (: let $data := xquery:eval("fn:for-each(1 to 1000,
> > function($a){2*$a} )") :)
> > return (1 to $i) ! local:test2($data,.)
> > };
> >
> > declare function local:test2($data, $dummy) { count($data) };
> >
> > (1 to 10) ! prof:time(local:test(.))
> >
> > Output :
> >
> > 683.76 ms
> > 1137.95 ms
> > 1727.68 ms
> > 2151.06 ms
> > 2694.84 ms
> > 3189.57 ms
> > 3725.1 ms
> > 4277.51 ms
> > 4815.74 ms
> > 5417.79 ms
> >
> > Note that using xquery:eval (toggling the comment inside local:test).
> seems
> > to perform correctly. However I can't use this trick as a workaround.
> >
> > (1 to 10) ! prof:time(local:test(.))
> > Ouput :
> >
> > 2472.71 ms
> > 825.61 ms
> > 3013.14 ms
> > 986.81 ms
> > 2982.97 ms
> > 778.24 ms
> > 1195.33 ms
> > 3280.7 ms
> > 976.96 ms
> > 913.69 ms
>
[basex-talk] Optimization for procedure call ?
Hello,
I am facing some performances issues, probably due to unexpected query
reevaluation. Might it be due to the optimizer ? Here is a code to test
declare function local:test($i){
let $data := fn:for-each(1 to 1000, function($a){2*$a} )
(: let $data := xquery:eval("fn:for-each(1 to 1000,
function($a){2*$a} )") :)
return (1 to $i) ! local:test2($data,.)
};
declare function local:test2($data, $dummy) { count($data) };
(1 to 10) ! prof:time(local:test(.))
Output :
683.76 ms
1137.95 ms
1727.68 ms
2151.06 ms
2694.84 ms
3189.57 ms
3725.1 ms
4277.51 ms
4815.74 ms
5417.79 ms
Note that using xquery:eval (toggling the comment inside local:test). seems
to perform correctly. However I can't use this trick as a workaround.
(1 to 10) ! prof:time(local:test(.))
Ouput :
2472.71 ms
825.61 ms
3013.14 ms
986.81 ms
2982.97 ms
778.24 ms
1195.33 ms
3280.7 ms
976.96 ms
913.69 ms
[basex-talk] Fwd: Storing sequences of atomic type into BaseX ?
Christian,
Excellent ! thanks a lot !
prof:time(local:savebin((1 to 1000),$binfile))
,prof:time(local:loadbin($binfile))
output is now
930.71 ms
907.04 ms
2015-01-08 22:11 GMT+01:00 Christian Grün :
> Hi Jean-Marc,
>
> > 2) However unserialization seems to perform awfully, or I do not know
> how to
> > do it properly.
>
> I haven't tried the query, but my guess is that the binary data is
> streamed again and again. The stream:materialize function should help
> you [1]:
>
> let $data := stream:materialize(file:read-binary($file))
> ...
>
> Christian
>
> [1] http://docs.basex.org/wiki/Streaming_Module
>
>
>
> > declare function local:savebin($seq,$file as xs:string) {
> > file:write-binary($file,bin:join( (bin:pack-integer(count($seq),4)
> ,$seq
> > ! bin:pack-integer(.,4
> > };
> > declare function local:loadbin($file as xs:string) {
> > let $data := file:read-binary($file)
> > let $size:= bin:unpack-integer($data,0,4)
> > let $seq := for $i in (1 to ($size)) return
> > bin:unpack-integer($data,$i*4,4)
> > return count($seq)
> > };
> >
> > prof:time(local:savebin((1 to 10),"Bin.dat"))
> > ,prof:time(local:loadbin("Bin.dat"))
> >
> > output :
> >
> > 46.38 ms
> > 10775.12 ms
> > 10
> >
> > To compare, unserializing a sequence (1 to 10 000 000) stored in a file
> as a
> > big string using fn:tokenize takes about 10 sec (100 x faster). Did I
> > mistake something ?
> >
> >
> > 2015-01-08 16:44 GMT+01:00 Christian Grün :
> >>
> >> > This way of doing stores integers as string, then call a cast string /
> >> > integer to unserialize it. For large integer list (I am dealing with
> >> > lists
> >> > of size 134 Mo), it is quite time and mem consuming.
> >> >
> >> > I was wondering if there exists a more efficient way to store and
> >> > retrieve
> >> > atomic list into BaseX ?
> >>
> >> One alternative is to store the integers in a binary file:
> >>
> >> let $size := 4
> >> let $data := bin:join(
> >> for $n in 1 to 100
> >> return bin:pack-integer($n, $size)
> >> )
> >> return db:store('db', 'integers.bin', $data)
> >>
> >> This way, every integer will occupy the supplied number of bytes
> >> (here: 4, allowing you to address 2^32 integers).
> >
> >
>
Re: [basex-talk] Storing sequences of atomic type into BaseX ?
Christian,
Hi. Thx for the trick. It seems to work finely for serializing, but awfully
for unserializing :
1) Serialization using pack-integer is great. I am saving time and disk
space (around a factor 4).
2) However unserialization seems to perform awfully, or I do not know how
to do it properly.
Here is a test :
declare function local:savebin($seq,$file as xs:string) {
file:write-binary($file,bin:join( (bin:pack-integer(count($seq),4)
,$seq ! bin:pack-integer(.,4
};
declare function local:loadbin($file as xs:string) {
let $data := file:read-binary($file)
let $size:= bin:unpack-integer($data,0,4)
let $seq := for $i in (1 to ($size)) return
bin:unpack-integer($data,$i*4,4)
return count($seq)
};
prof:time(local:savebin((1 to 10),"Bin.dat"))
,prof:time(local:loadbin("Bin.dat"))
output :
46.38 ms
10775.12 ms
10
To compare, unserializing a sequence (1 to 10 000 000) stored in a file as
a big string using fn:tokenize takes about 10 sec (100 x faster). Did I
mistake something ?
2015-01-08 16:44 GMT+01:00 Christian Grün :
> > This way of doing stores integers as string, then call a cast string /
> > integer to unserialize it. For large integer list (I am dealing with
> lists
> > of size 134 Mo), it is quite time and mem consuming.
> >
> > I was wondering if there exists a more efficient way to store and
> retrieve
> > atomic list into BaseX ?
>
> One alternative is to store the integers in a binary file:
>
> let $size := 4
> let $data := bin:join(
> for $n in 1 to 100
> return bin:pack-integer($n, $size)
> )
> return db:store('db', 'integers.bin', $data)
>
> This way, every integer will occupy the supplied number of bytes
> (here: 4, allowing you to address 2^32 integers).
>
[basex-talk] Database corrupted while inserting big string
Hello,
I am trying to insert big string into BaseX, and ended corrupting the whole
database. Here a test code , can you reproduce it ?
declare updating function local:dbtest($db as xs:string) {
db:replace($db,"map",{fn:serialize(1 to 1000)})
};
local:dbtest("MyDataBase")
output :
Improper use? Potential bug? Your feedback is welcome:
Contact: basex-talk@mailman.uni-konstanz.de
Version: BaseX 8.0 beta 769b53b
Java: Oracle Corporation, 1.8.0_25
OS: Windows 7, amd64
Stack Trace:
java.lang.IndexOutOfBoundsException
at java.io.RandomAccessFile.writeBytes(Native Method)
at java.io.RandomAccessFile.write(Unknown Source)
at org.basex.io.random.DataAccess.writeBlock(DataAccess.java:448)
at org.basex.io.random.DataAccess.flush(DataAccess.java:49)
at org.basex.io.random.DataAccess.close(DataAccess.java:61)
at org.basex.data.DiskData.close(DiskData.java:151)
at org.basex.core.cmd.Close.close(Close.java:45)
at org.basex.query.QueryResources.close(QueryResources.java:110)
at org.basex.query.QueryContext.close(QueryContext.java:564)
at org.basex.query.QueryProcessor.close(QueryProcessor.java:235)
at org.basex.core.cmd.AQuery.query(AQuery.java:129)
at org.basex.core.cmd.XQuery.run(XQuery.java:22)
at org.basex.core.Command.run(Command.java:379)
at org.basex.core.Command.execute(Command.java:95)
at org.basex.api.client.LocalSession.execute(LocalSession.java:132)
at org.basex.api.client.Session.execute(Session.java:36)
at org.basex.core.CLI.execute(CLI.java:106)
at org.basex.BaseX.(BaseX.java:124)
at org.basex.BaseX.main(BaseX.java:42)
Re: [basex-talk] Performance issue due to evaluation during function composition ?
Hi Christian,
Thx a lot. Jean-Marc
2015-01-06 12:20 GMT+01:00 Christian Grün :
> Hi Jean-Marc,
>
> Dirk has already outlined well what this is about. I have added a new
> GitHub request [1].
>
> Cheers,
> Christian
>
> [1] https://github.com/BaseXdb/basex/issues/1052
>
>
>
> On Fri, Jan 2, 2015 at 11:30 PM, Dirk Kirsten wrote:
> > Hello Jean-Marc,
> >
> > yeah, I can reproduce that. However, I would not consider it a bug but
> rather a not-so-clever optimizer. The optimized query looks something like
> this:
> >
> > declare function local:dummy($db_0) { let $fun_8 :=
> local:count_elements#1 return $fun_8($db_0) };
> > (prof:time(22383), prof:time(local:dummy(db:open-pre("my-db",0
> >
> > It quite nicely shows that in the second case the function isn't inlined
> but instead it is fully evaluated (and counting all elements will take some
> time instead of just looking up the value from the index).
> > So yes, I agree the optimizer could (and probably should) do better, but
> right now it is not strictly incorrect (but unexpected). I guess Christian
> will fix this as soon as he is back from Christmas vacation.
> >
> > Cheers,
> > Dirk
> >
> > On 01/02/2015 10:02 PM, jean-marc Mercier wrote:
> >> errata : * declare variable $BigDb := db:open('MyBigDataBase');
> >>
> >> 2015-01-02 21:54 GMT+01:00 jean-marc Mercier <
> jeanmarc.merc...@gmail.com>:
> >>
> >> > Dirk,
> >> >
> >> > Hello, sorry, but a very similar behavior seems to appear also in the
> >> > latest version BaseX80-20141225.174535. Here is a new code to test :
> >> >
> >> > declare variable $BigDb := AppParam:DBInputOpen($ConfigDoc);
> >> > declare function local:dummy($db)
> {local:call(local:count_elements#1,$db)};
> >> > declare function local:count_elements($db)
> >> > {count($db/descendant-or-self::element())};
> >> > declare function local:call($fun,$args) {$fun($args)};
> >> >
> >> > prof:time(local:call(local:count_elements#1,$BigDb))
> >> > ,prof:time(local:dummy($BigDb))
> >> >
> >> > output :
> >> > 3.05 ms
> >> > 104170.17 ms
> >> > 134666491 134666491
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > 2015-01-02 20:05 GMT+01:00 jean-marc Mercier <
> jeanmarc.merc...@gmail.com>:
> >> >
> >> >> Hi Dirk,
> >> >>
> >> >> You are right, I was using an old 8.0 BaseX version (seems to be the
> one
> >> >> from 23/09/2014). The problem does not appear in the latest release.
> Thank
> >> >> you !
> >> >>
> >> >>
> >> >> 2015-01-02 19:29 GMT+01:00 Dirk Kirsten :
> >> >>
> >> >>> Hello Jean-Marc,
> >> >>>
> >> >>> what version of BaseX did you use? I tried with the latest snapshot
> and
> >> >>> used the XMark dataset (with factor 80, which produces around the
> same
> >> >>> number of elements (133653910 to be exact) you had) and got the
> following
> >> >>> output with an up-to-date index:
> >> >>>
> >> >>> - 0.01 ms
> >> >>> - 0.01 ms
> >> >>> - 0.0 ms
> >> >>>
> >> >>> If it isn't a bug in a previous version of BaseX, I would guess you
> >> >>> reached the inline limit (
> http://docs.basex.org/wiki/Options#INLINELIMIT).
> >> >>> You might want to try setting it to a higher value.
> >> >>>
> >> >>> Cheers,
> >> >>> Dirk
> >> >>>
> >> >>>
> >> >>> On 01/02/2015 10:19 AM, jean-marc Mercier wrote:
> >> >>>> Hello (and happy new year 2015 !)
> >> >>>>
> >> >>>> I am facing some performance issues opening "big" databases due to
> an
> >> >>>> (unexpected for me) argument evaluation during function
> composition. Is
> >> >>>> this a normal behavior ?
> >> >>>>
> >> >>>> Cheers
> >> >>>>
> >> >>>> Here is a code description :
> >> >>>>
> >> >>>> declare variable $db := db:open("MyBigDataBase");
> >> >>>> declare function local:elts($db){$db//element()};
> >> >>>> declare function local:compose($f1 as function(*), $f2 as
> function(*))
> >> >>> as
> >> >>>> function(*){ function($a) { $f1($f2($a)) }};
> >> >>>>
> >> >>>> declare function local:count_elt($db) {count(local:elts($db))};
> >> >>>> prof:time(local:count_elt($db)),
> >> >>>> prof:time(count(local:elts($db))),
> >> >>>> prof:time(local:compose(count#1,local:elts#1) ($db))
> >> >>>>
> >> >>>> ouput :
> >> >>>> 3.1 ms
> >> >>>> 0.02 ms
> >> >>>> 100047.53 ms
> >> >>>> 134666491 134666491 134666491
> >> >>>>
> >> >>>
> >> >>> --
> >> >>> Dirk Kirsten, BaseX GmbH, http://basex.org
> >> >>> |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz
> >> >>> |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer:
> >> >>> | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle
> >> >>> `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22
> >> >>>
> >> >>>
> >> >>
> >> >
> >>
> >
> > --
> > Dirk Kirsten, BaseX GmbH, http://basex.org
> > |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz
> > |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer:
> > | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle
> > `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22
> >
>
[basex-talk] prof:time and updating expression
Hi all, I was trying to profile updating expression, and the result is [XUST0001] prof:time(value[,cache[,label]]): no updating expression allowed. Is this a desired behavior ? Cheers, Jean-Marc
[basex-talk] Storing sequences of atomic type into BaseX ?
Hello,
I am currently trying to store precomputed maps. The basic issue is to
store a sequence $seq of integers (that are indeed element id, i.e.
retrieved using db:node-pre).
Actually, I am storing it using
db:replace($db,"map",{fn:serialize($seq)})
and retrieve the sequence using using fn:tokenize(db:open($db,"map")/map,"
") ! xs:integer(.).
This way of doing stores integers as string, then call a cast string /
integer to unserialize it. For large integer list (I am dealing with lists
of size 134 Mo), it is quite time and mem consuming.
I was wondering if there exists a more efficient way to store and retrieve
atomic list into BaseX ?
[basex-talk] Question / Suggestion : Template mechanisms and operator overloading for XQUERY 3.++ ?
Hello,
I was wondering if it is possible to improve template programming and
operator overloading with XQUERY 3.?
1) template programming : XQUERY provides a quite natural template
mechanism using type switch and instance:
declare function yell($animal) {
if ($animal instance of element()) then
return typeswitch($animal)
case element(cow) return "Moo" default return "..."
else "this is not an animal"
};
However, I wonder whether it could be possible to provide such a mechanism
directly at the interpreter level ? For instance, the following code seems
coherent and concise :
declare function yell($animal as element(cow) ) {"Moo"}
declare function yell($animal as element() ) {".."}
declare function yell($animal ) {"This is not an animal"}
As far as I understand, XQUERY functions identification seems to be achieve
through a couple (name, arity). Would it be difficult to extend it with
(name,signature or predicate) ?
2) Operator overloading (quite linked to the previous question)
I don't know how to overload operators with XQUERY. For instance I would
like to define something like
declare function array:operator+($left,$right){array:for
-each-pair($left,$right,function($a + $b) { $a + $b } )}
Is it possible ?
I know that I can survive writing
declare function array:plus($left,$right){array:for-each-pair($left,$right,
function($a + $b) { $a + $b } )}
but when one start to write algorithmic code, it is far easier and
understandable to write $a+$b than array:plus($a,$b)
Re: [basex-talk] Performance issue due to evaluation during function composition ?
errata : * declare variable $BigDb := db:open('MyBigDataBase');
2015-01-02 21:54 GMT+01:00 jean-marc Mercier :
> Dirk,
>
> Hello, sorry, but a very similar behavior seems to appear also in the
> latest version BaseX80-20141225.174535. Here is a new code to test :
>
> declare variable $BigDb := AppParam:DBInputOpen($ConfigDoc);
> declare function local:dummy($db) {local:call(local:count_elements#1,$db)};
> declare function local:count_elements($db)
> {count($db/descendant-or-self::element())};
> declare function local:call($fun,$args) {$fun($args)};
>
> prof:time(local:call(local:count_elements#1,$BigDb))
> ,prof:time(local:dummy($BigDb))
>
> output :
> 3.05 ms
> 104170.17 ms
> 134666491 134666491
>
>
>
>
>
> 2015-01-02 20:05 GMT+01:00 jean-marc Mercier :
>
>> Hi Dirk,
>>
>> You are right, I was using an old 8.0 BaseX version (seems to be the one
>> from 23/09/2014). The problem does not appear in the latest release. Thank
>> you !
>>
>>
>> 2015-01-02 19:29 GMT+01:00 Dirk Kirsten :
>>
>>> Hello Jean-Marc,
>>>
>>> what version of BaseX did you use? I tried with the latest snapshot and
>>> used the XMark dataset (with factor 80, which produces around the same
>>> number of elements (133653910 to be exact) you had) and got the following
>>> output with an up-to-date index:
>>>
>>> - 0.01 ms
>>> - 0.01 ms
>>> - 0.0 ms
>>>
>>> If it isn't a bug in a previous version of BaseX, I would guess you
>>> reached the inline limit (http://docs.basex.org/wiki/Options#INLINELIMIT).
>>> You might want to try setting it to a higher value.
>>>
>>> Cheers,
>>> Dirk
>>>
>>>
>>> On 01/02/2015 10:19 AM, jean-marc Mercier wrote:
>>> > Hello (and happy new year 2015 !)
>>> >
>>> > I am facing some performance issues opening "big" databases due to an
>>> > (unexpected for me) argument evaluation during function composition. Is
>>> > this a normal behavior ?
>>> >
>>> > Cheers
>>> >
>>> > Here is a code description :
>>> >
>>> > declare variable $db := db:open("MyBigDataBase");
>>> > declare function local:elts($db){$db//element()};
>>> > declare function local:compose($f1 as function(*), $f2 as function(*))
>>> as
>>> > function(*){ function($a) { $f1($f2($a)) }};
>>> >
>>> > declare function local:count_elt($db) {count(local:elts($db))};
>>> > prof:time(local:count_elt($db)),
>>> > prof:time(count(local:elts($db))),
>>> > prof:time(local:compose(count#1,local:elts#1) ($db))
>>> >
>>> > ouput :
>>> > 3.1 ms
>>> > 0.02 ms
>>> > 100047.53 ms
>>> > 134666491 134666491 134666491
>>> >
>>>
>>> --
>>> Dirk Kirsten, BaseX GmbH, http://basex.org
>>> |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz
>>> |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer:
>>> | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle
>>> `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22
>>>
>>>
>>
>
Re: [basex-talk] Performance issue due to evaluation during function composition ?
Dirk,
Hello, sorry, but a very similar behavior seems to appear also in the
latest version BaseX80-20141225.174535. Here is a new code to test :
declare variable $BigDb := AppParam:DBInputOpen($ConfigDoc);
declare function local:dummy($db) {local:call(local:count_elements#1,$db)};
declare function local:count_elements($db)
{count($db/descendant-or-self::element())};
declare function local:call($fun,$args) {$fun($args)};
prof:time(local:call(local:count_elements#1,$BigDb))
,prof:time(local:dummy($BigDb))
output :
3.05 ms
104170.17 ms
134666491 134666491
2015-01-02 20:05 GMT+01:00 jean-marc Mercier :
> Hi Dirk,
>
> You are right, I was using an old 8.0 BaseX version (seems to be the one
> from 23/09/2014). The problem does not appear in the latest release. Thank
> you !
>
>
> 2015-01-02 19:29 GMT+01:00 Dirk Kirsten :
>
>> Hello Jean-Marc,
>>
>> what version of BaseX did you use? I tried with the latest snapshot and
>> used the XMark dataset (with factor 80, which produces around the same
>> number of elements (133653910 to be exact) you had) and got the following
>> output with an up-to-date index:
>>
>> - 0.01 ms
>> - 0.01 ms
>> - 0.0 ms
>>
>> If it isn't a bug in a previous version of BaseX, I would guess you
>> reached the inline limit (http://docs.basex.org/wiki/Options#INLINELIMIT).
>> You might want to try setting it to a higher value.
>>
>> Cheers,
>> Dirk
>>
>>
>> On 01/02/2015 10:19 AM, jean-marc Mercier wrote:
>> > Hello (and happy new year 2015 !)
>> >
>> > I am facing some performance issues opening "big" databases due to an
>> > (unexpected for me) argument evaluation during function composition. Is
>> > this a normal behavior ?
>> >
>> > Cheers
>> >
>> > Here is a code description :
>> >
>> > declare variable $db := db:open("MyBigDataBase");
>> > declare function local:elts($db){$db//element()};
>> > declare function local:compose($f1 as function(*), $f2 as function(*))
>> as
>> > function(*){ function($a) { $f1($f2($a)) }};
>> >
>> > declare function local:count_elt($db) {count(local:elts($db))};
>> > prof:time(local:count_elt($db)),
>> > prof:time(count(local:elts($db))),
>> > prof:time(local:compose(count#1,local:elts#1) ($db))
>> >
>> > ouput :
>> > 3.1 ms
>> > 0.02 ms
>> > 100047.53 ms
>> > 134666491 134666491 134666491
>> >
>>
>> --
>> Dirk Kirsten, BaseX GmbH, http://basex.org
>> |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz
>> |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer:
>> | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle
>> `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22
>>
>>
>
Re: [basex-talk] Performance issue due to evaluation during function composition ?
Hi Dirk,
You are right, I was using an old 8.0 BaseX version (seems to be the one
from 23/09/2014). The problem does not appear in the latest release. Thank
you !
2015-01-02 19:29 GMT+01:00 Dirk Kirsten :
> Hello Jean-Marc,
>
> what version of BaseX did you use? I tried with the latest snapshot and
> used the XMark dataset (with factor 80, which produces around the same
> number of elements (133653910 to be exact) you had) and got the following
> output with an up-to-date index:
>
> - 0.01 ms
> - 0.01 ms
> - 0.0 ms
>
> If it isn't a bug in a previous version of BaseX, I would guess you
> reached the inline limit (http://docs.basex.org/wiki/Options#INLINELIMIT).
> You might want to try setting it to a higher value.
>
> Cheers,
> Dirk
>
>
> On 01/02/2015 10:19 AM, jean-marc Mercier wrote:
> > Hello (and happy new year 2015 !)
> >
> > I am facing some performance issues opening "big" databases due to an
> > (unexpected for me) argument evaluation during function composition. Is
> > this a normal behavior ?
> >
> > Cheers
> >
> > Here is a code description :
> >
> > declare variable $db := db:open("MyBigDataBase");
> > declare function local:elts($db){$db//element()};
> > declare function local:compose($f1 as function(*), $f2 as function(*)) as
> > function(*){ function($a) { $f1($f2($a)) }};
> >
> > declare function local:count_elt($db) {count(local:elts($db))};
> > prof:time(local:count_elt($db)),
> > prof:time(count(local:elts($db))),
> > prof:time(local:compose(count#1,local:elts#1) ($db))
> >
> > ouput :
> > 3.1 ms
> > 0.02 ms
> > 100047.53 ms
> > 134666491 134666491 134666491
> >
>
> --
> Dirk Kirsten, BaseX GmbH, http://basex.org
> |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz
> |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer:
> | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle
> `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22
>
>
[basex-talk] Performance issue due to evaluation during function composition ?
Hello (and happy new year 2015 !)
I am facing some performance issues opening "big" databases due to an
(unexpected for me) argument evaluation during function composition. Is
this a normal behavior ?
Cheers
Here is a code description :
declare variable $db := db:open("MyBigDataBase");
declare function local:elts($db){$db//element()};
declare function local:compose($f1 as function(*), $f2 as function(*)) as
function(*){ function($a) { $f1($f2($a)) }};
declare function local:count_elt($db) {count(local:elts($db))};
prof:time(local:count_elt($db)),
prof:time(count(local:elts($db))),
prof:time(local:compose(count#1,local:elts#1) ($db))
ouput :
3.1 ms
0.02 ms
100047.53 ms
134666491 134666491 134666491
Re: [basex-talk] Eclipse vs BaseX Gui differences ?
Hi christian, Hope your holidays went finely ! I can send you the XQUERY files, that consists mainly the graph database V0.0 open sourced files. However, you won't be able to reproduce the issue without a database sample. Actually, I can't send you my database, that has been built with my client data, thus containing confidential informations. Don't you have another idea to go ahead with this issue ? If not, I'll have to build a database from public data, like a wikipedia dump, but it will take some time. Cheers 2014-10-06 13:53 GMT+02:00 Christian Grün : > Hi Jean-Marc, > > back again. Could you possibly send us the problematic query files > (provided that these can be run without too much initialization > overhead)? > > Cheers, > Christian > > > > On Sat, Sep 27, 2014 at 11:34 AM, jean-marc Mercier > wrote: > > Dirk, > > > > Hello ! > > Sorry, the problem seems to be on BaseX side. > > > > 1- I have tried to use eclipse IDE setting the interpreter as a client. > > Doesn't work, it saturated my Java machine. > > 2- I have tried to sollicitate directly the BaseX client module through > > client:connect and client:query. Doesn't work, it saturated my Java > machine. > > 3- Then I toggled totally the eclipse IDE : I opened a local server > instance > > basexserver.bat, I opened a dos windows basexclient.bat, and tried to RUN > > the query file directly on the client console. Doesn't work, it still > > saturated my JAVA Machine. > > 4- I finally tested that the same query ran perfectly using the BaseX > Gui. > > > > My first penny guess is that this problem could be linked to this closed > > bug, that we encountered some 7 months ago : > > > https://github.com/BaseXdb/basex/commit/5a926e0ea74b3e4a09f9a735ef7f190064133b05 > > . This impression is motivated because I have the same symptoms than for > > this bug : as I terminate the query, the eclipse console IDE freeze and I > > have to kill the BAseX server process to retrieve. > > > > 5- Thus I tried also to force non verbose mode using Basex client > > interpreter (setting -v options). Doesn't work, it still saturated my > JAVA > > Machine. > > > > > > Do we have any way to check if a query run in verbose mode ? > > > > Cheeres, > > > > Jean-Marc > > > > > > 2014-09-26 16:20 GMT+02:00 jean-marc Mercier >: > >> > >> Hi Dirk, > >> > >> Ok, thanks for the trick, that should work. The only point is that it > >> should be less user friendly for developing than using the eclipse IDE, > >> since I will have to set up the BaseX repo first of running a query. > >> > >> 2014-09-26 12:11 GMT+02:00 Dirk Kirsten : > >>> > >>> Hi Jean-Marc, > >>> > >>> I guess you set it up using the standalone version? Could you maybe > test > >>> using the Client/Server set-up. You could then test with a plain > BaseXClient > >>> and the eclipse environment. As they both should access the same > >>> BaseXServer, there shouldn't be any difference in execution. > >>> > >>> But I don't know what XQDT is doing internally and I would pretty much > >>> guess the error is somewhere there (because plain BaseX seems to > execute > >>> just fine). As the development seems to have pretty much stalled, I am > not > >>> sure we can do much about that... > >>> > >>> > >>> On 09/26/2014 11:54 AM, jean-marc Mercier wrote: > >>> > Hi Dirk, > >>> > > >>> > Thanks for your answer. I am setting up eclipse indigo, following the > >>> > step-up procedure http://docs.basex.org/wiki/Integrating_Eclipse. > The > >>> > only > >>> > difference is that I set using -Xmx12G as Interpreter Arguments. > >>> > > >>> > Cheers > >>> > > >>> > > >>> > 2014-09-26 11:44 GMT+02:00 Dirk Kirsten : > >>> > > >>> > > Hello Jean-Marc, > >>> > > > >>> > > I am not sure how you are calling BaseX from within Eclipse, I > >>> > > remember > >>> > > there are different ways. Normally, I would expect it to simply > >>> > > execute > >>> > > basex, so there shouldn't be any difference. The query optimization > >>> > > should > >>> > > be done in both cases. However, it could be for example that your > >>&
Re: [basex-talk] Eclipse vs BaseX Gui differences ?
Dirk, Hello. I also tried to run the query on a Standalone BaseX version. It does not work neither, saturating my JAVA machine. Note that the verbose mode is off, here the result of INFO run on the standalone console : BaseX 8.0 beta 5910246 [Standalone] Try help to get more information. > OPEN SOPHISIMPORTDB Database 'SOPHISIMPORTDB' was opened in 244.02 ms. > help Try HELP [...] to get info on a specific command. ADD (TO [path]) [input] Add resource to database. ALTER [DATABASE|USER] [...] Alter database name or user password. CHECK [input] Open or create database. CLOSE Close current database. COPY [name] [newname] Copy database. CREATE [BACKUP|DATABASE|EVENT|INDEX|USER] [...] Create database, index, user, backup or event. CS [query] Run XQuery and set result as new context set. DELETE [path] Delete resources from database. DROP [BACKUP|DATABASE|EVENT|INDEX|USER] [...] Drop database, index, user, backup or event. EXIT Exit application. EXPORT [path] Export database to XML. FIND [keywords] Run a keyword query. FLUSH Flush database. GET [option] Show global option. GRANT [NONE|READ|WRITE|CREATE|ADMIN] (ON [database]) TO [user] Grant user permissions. HELP ([command]) Get help on BaseX commands. INFO ([DATABASE|INDEX|STORAGE]) Show information on current database. INSPECT Perform integrity checks. KILL [name] Kill user sessions. LIST ([name] ([path])) List databases or resources of database. OPEN [name] ([path]) Open database. OPTIMIZE (ALL) Optimize the database. PASSWORD ([password]) Change password. QUIT Exit application. RENAME [path] [newpath] Rename resources in database. REPLACE [path] [input] Replace resources in database. REPO [DELETE|INSTALL|LIST] Install, delete or list packages. RESTORE [name-(date)] Restore database. RETRIEVE [path] Retrieve raw data. RUN [path] Run query or command script. EXECUTE [input] Execute command script. SET [option] ([value]) Set global options. SHOW [BACKUPS|EVENTS|SESSIONS|USERS] Show server information. STORE (TO [path]) [input] Store raw data. TEST [path] Run XQUnit tests. XQUERY [query] Run XQuery. > INFO General Information: Version: 8.0 beta 5910246 Used Memory: 126 MB Global options: DBPATH: C:/Jiheme/informatique/basex8.0/data DEBUG: false EVENTPORT: 1985 GLOBALLOCK: false HOST: localhost HTTPLOCAL: false KEEPALIVE: 600 LANG: English LANGKEYS: false LOG: true LOGMSGMAXLEN: 1000 NONPROXYHOSTS: PARALLEL: 8 PASSWORD: PORT: 1984 PROXYHOST: PROXYPORT: 80 REPOPATH: C:/Jiheme/informatique/basex8.0/repo RESTXQPATH: SERVERHOST: SERVERPORT: 1984 STOPPORT: 8985 TIMEOUT: 30 USER: WEBPATH: C:/Jiheme/informatique/basex8.0/webapp Local options ADDARCHIVES: true ADDCACHE: false ADDRAW: false ATTRINDEX: true AUTOFLUSH: true BINDINGS: CACHEQUERY: false CASESENS: false CATFILE: CHECKSTRINGS: true CHOP: true COMPPLAN: true CREATEFILTER: *.xml CREATEONLY: false CSVPARSER: DEFAULTDB: false DIACRITICS: false DOTCOMPACT: false DOTPLAN: false DTD: false EXPORTER: FORCECREATE: false FTINDEX: false FTINDEXSPLITSIZE: 0 HTMLPARSER: INDEXSPLITSIZE: 0 INLINELIMIT: 100 INTPARSE: false JSONPARSER: LANGUAGE: en LSERROR: 0 MAINMEM: false MAXCATS: 100 MAXHITS: -1 MAXLEN: 96 MAXSTAT: 30 MIXUPDATES: false PARSER: xml QUERYINFO: false QUERYPATH: RUNQUERY: true RUNS: 1 SERIALIZE: true SERIALIZER: SKIPCORRUPT: false STEMMING: false STOPWORDS: STRIPNS: false TAILCALLS: 256 TEXTINDEX: true TEXTPARSER: UPDINDEX: false WRITEBACK: false XINCLUDE: true XMLPLAN: false > 2014-09-27 11:34 GMT+02:00 jean-marc Mercier : > Dirk, > > Hello ! > Sorry, the problem seems to be on BaseX side. > > 1- I have tried to use eclipse IDE setting the interpreter as a client. > Doesn't work, it saturated my Java machine. > 2- I have tried to sollicitate directly the BaseX client module through > client:connect and client:query. Doesn't work, it saturated my Java machine. > 3- Then I toggled totally the eclipse IDE : I opened a local server > instance basexserver.bat, I opened a dos windows basexclient.bat, and tried > to RUN the query file directly on the client console. Doesn't work, it > still saturated my JAVA Machine. > 4- I finally tested that the same query ran perfectly using the BaseX Gui. > > My first penny guess is that this problem could be linked to this closed > bug, that we encountered some 7 months ago : > https://github.com/BaseXdb/basex/commit/5a926e0ea74b3e4a09f9a735ef7f190064133b05 > . > This impression is motivated because I have the same symptoms than for this > bug : as I terminate the query, the eclipse console IDE freeze and I have > to kill the BAseX server process to retrieve. > > 5- Thus I tried also to force non verbose mode using Basex client > interpreter (setting -v options). Doesn't work, it still saturated my JAVA &
Re: [basex-talk] Eclipse vs BaseX Gui differences ?
Dirk, Hello ! Sorry, the problem seems to be on BaseX side. 1- I have tried to use eclipse IDE setting the interpreter as a client. Doesn't work, it saturated my Java machine. 2- I have tried to sollicitate directly the BaseX client module through client:connect and client:query. Doesn't work, it saturated my Java machine. 3- Then I toggled totally the eclipse IDE : I opened a local server instance basexserver.bat, I opened a dos windows basexclient.bat, and tried to RUN the query file directly on the client console. Doesn't work, it still saturated my JAVA Machine. 4- I finally tested that the same query ran perfectly using the BaseX Gui. My first penny guess is that this problem could be linked to this closed bug, that we encountered some 7 months ago : https://github.com/BaseXdb/basex/commit/5a926e0ea74b3e4a09f9a735ef7f190064133b05 . This impression is motivated because I have the same symptoms than for this bug : as I terminate the query, the eclipse console IDE freeze and I have to kill the BAseX server process to retrieve. 5- Thus I tried also to force non verbose mode using Basex client interpreter (setting -v options). Doesn't work, it still saturated my JAVA Machine. Do we have any way to check if a query run in verbose mode ? Cheeres, Jean-Marc 2014-09-26 16:20 GMT+02:00 jean-marc Mercier : > Hi Dirk, > > Ok, thanks for the trick, that should work. The only point is that it > should be less user friendly for developing than using the eclipse IDE, > since I will have to set up the BaseX repo first of running a query. > > 2014-09-26 12:11 GMT+02:00 Dirk Kirsten : > >> Hi Jean-Marc, >> >> I guess you set it up using the standalone version? Could you maybe test >> using the Client/Server set-up. You could then test with a plain >> BaseXClient and the eclipse environment. As they both should access the >> same BaseXServer, there shouldn't be any difference in execution. >> >> But I don't know what XQDT is doing internally and I would pretty much >> guess the error is somewhere there (because plain BaseX seems to execute >> just fine). As the development seems to have pretty much stalled, I am not >> sure we can do much about that... >> >> >> On 09/26/2014 11:54 AM, jean-marc Mercier wrote: >> > Hi Dirk, >> > >> > Thanks for your answer. I am setting up eclipse indigo, following the >> > step-up procedure http://docs.basex.org/wiki/Integrating_Eclipse. The >> only >> > difference is that I set using -Xmx12G as Interpreter Arguments. >> > >> > Cheers >> > >> > >> > 2014-09-26 11:44 GMT+02:00 Dirk Kirsten : >> > >> > > Hello Jean-Marc, >> > > >> > > I am not sure how you are calling BaseX from within Eclipse, I >> remember >> > > there are different ways. Normally, I would expect it to simply >> execute >> > > basex, so there shouldn't be any difference. The query optimization >> should >> > > be done in both cases. However, it could be for example that your >> BaseX >> > > integration in eclipse uses XQJ, which could possibly introduce some >> > > performance bottlenecks. >> > > >> > > So if you could shortly describe how you set up BaseX within Eclipse, >> that >> > > would be great. >> > > >> > > Cheers, >> > > Dirk >> > > >> > > On 09/24/2014 06:33 AM, jean-marc Mercier wrote: >> > >> Hi all, >> > >> >> > >> Happy to write again in this mailing-list, hope you are all doing >> well >> > > guys >> > >> ! >> > >> >> > >> I am encountering a small problem running my old xquery modules. I >> guess >> > >> that it is a classical issue, but I can't remember how to look into >> BaseX >> > >> archive :( >> > >> >> > >> Here is the description of the problem : I am running a query, >> calling >> > >> module_of_mine.xqm, on two different environments. >> > >> >> > >> The first one is eclipse, linked to the latest interpreter from >> > >> BaseX80-20140923.183154 (but I tested also against older >> interpreter). In >> > >> this way, this query saturated my 12 G Java machine after some >> minutes. >> > >> >> > >> Then I installed module_of_mine.xqm into BaseX repo, and ran the same >> > > query >> > >> using basex gui with 4G heap size. The query ran perfectly in some 30 >> > > sec. >> >
Re: [basex-talk] Eclipse vs BaseX Gui differences ?
Hi Dirk, Ok, thanks for the trick, that should work. The only point is that it should be less user friendly for developing than using the eclipse IDE, since I will have to set up the BaseX repo first of running a query. 2014-09-26 12:11 GMT+02:00 Dirk Kirsten : > Hi Jean-Marc, > > I guess you set it up using the standalone version? Could you maybe test > using the Client/Server set-up. You could then test with a plain > BaseXClient and the eclipse environment. As they both should access the > same BaseXServer, there shouldn't be any difference in execution. > > But I don't know what XQDT is doing internally and I would pretty much > guess the error is somewhere there (because plain BaseX seems to execute > just fine). As the development seems to have pretty much stalled, I am not > sure we can do much about that... > > > On 09/26/2014 11:54 AM, jean-marc Mercier wrote: > > Hi Dirk, > > > > Thanks for your answer. I am setting up eclipse indigo, following the > > step-up procedure http://docs.basex.org/wiki/Integrating_Eclipse. The > only > > difference is that I set using -Xmx12G as Interpreter Arguments. > > > > Cheers > > > > > > 2014-09-26 11:44 GMT+02:00 Dirk Kirsten : > > > > > Hello Jean-Marc, > > > > > > I am not sure how you are calling BaseX from within Eclipse, I remember > > > there are different ways. Normally, I would expect it to simply execute > > > basex, so there shouldn't be any difference. The query optimization > should > > > be done in both cases. However, it could be for example that your BaseX > > > integration in eclipse uses XQJ, which could possibly introduce some > > > performance bottlenecks. > > > > > > So if you could shortly describe how you set up BaseX within Eclipse, > that > > > would be great. > > > > > > Cheers, > > > Dirk > > > > > > On 09/24/2014 06:33 AM, jean-marc Mercier wrote: > > >> Hi all, > > >> > > >> Happy to write again in this mailing-list, hope you are all doing well > > > guys > > >> ! > > >> > > >> I am encountering a small problem running my old xquery modules. I > guess > > >> that it is a classical issue, but I can't remember how to look into > BaseX > > >> archive :( > > >> > > >> Here is the description of the problem : I am running a query, calling > > >> module_of_mine.xqm, on two different environments. > > >> > > >> The first one is eclipse, linked to the latest interpreter from > > >> BaseX80-20140923.183154 (but I tested also against older > interpreter). In > > >> this way, this query saturated my 12 G Java machine after some > minutes. > > >> > > >> Then I installed module_of_mine.xqm into BaseX repo, and ran the same > > > query > > >> using basex gui with 4G heap size. The query ran perfectly in some 30 > > > sec. > > >> > > >> My first penny guess is that the built-in BaseX xquery profiler is > doing > > >> some nice work there ? If so, is there a way to activate / desactivate > > > this > > >> profiler from eclipse or from some configuration files ? > > >> > > >> Cheers, > > >> > > >> Jean-Marc > > >> > > > > > > -- > > > Dirk Kirsten, BaseX GmbH, http://basex.org > > > |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz > > > |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: > > > | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle > > > `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22 > > > > > > > > > > -- > Dirk Kirsten, BaseX GmbH, http://basex.org > |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz > |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: > | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle > `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22 > >
Re: [basex-talk] Eclipse vs BaseX Gui differences ?
Hi Dirk, Thanks for your answer. I am setting up eclipse indigo, following the step-up procedure http://docs.basex.org/wiki/Integrating_Eclipse. The only difference is that I set using -Xmx12G as Interpreter Arguments. Cheers 2014-09-26 11:44 GMT+02:00 Dirk Kirsten : > Hello Jean-Marc, > > I am not sure how you are calling BaseX from within Eclipse, I remember > there are different ways. Normally, I would expect it to simply execute > basex, so there shouldn't be any difference. The query optimization should > be done in both cases. However, it could be for example that your BaseX > integration in eclipse uses XQJ, which could possibly introduce some > performance bottlenecks. > > So if you could shortly describe how you set up BaseX within Eclipse, that > would be great. > > Cheers, > Dirk > > On 09/24/2014 06:33 AM, jean-marc Mercier wrote: > > Hi all, > > > > Happy to write again in this mailing-list, hope you are all doing well > guys > > ! > > > > I am encountering a small problem running my old xquery modules. I guess > > that it is a classical issue, but I can't remember how to look into BaseX > > archive :( > > > > Here is the description of the problem : I am running a query, calling > > module_of_mine.xqm, on two different environments. > > > > The first one is eclipse, linked to the latest interpreter from > > BaseX80-20140923.183154 (but I tested also against older interpreter). In > > this way, this query saturated my 12 G Java machine after some minutes. > > > > Then I installed module_of_mine.xqm into BaseX repo, and ran the same > query > > using basex gui with 4G heap size. The query ran perfectly in some 30 > sec. > > > > My first penny guess is that the built-in BaseX xquery profiler is doing > > some nice work there ? If so, is there a way to activate / desactivate > this > > profiler from eclipse or from some configuration files ? > > > > Cheers, > > > > Jean-Marc > > > > -- > Dirk Kirsten, BaseX GmbH, http://basex.org > |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz > |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: > | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle > `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22 > >
[basex-talk] how generating wiki from Basex inspect:xqdoc ?
Hi all, I am using BaseX' inspect:xqdoc module to generate xquery docs. I was wondering how generating html pages for accessing this documentation from the web, as for instance Leo did for http://www.woerteler.de/xquery/modules/base64 ?? Thanx cheers, Jean-Marc
[basex-talk] Eclipse vs BaseX Gui differences ?
Hi all, Happy to write again in this mailing-list, hope you are all doing well guys ! I am encountering a small problem running my old xquery modules. I guess that it is a classical issue, but I can't remember how to look into BaseX archive :( Here is the description of the problem : I am running a query, calling module_of_mine.xqm, on two different environments. The first one is eclipse, linked to the latest interpreter from BaseX80-20140923.183154 (but I tested also against older interpreter). In this way, this query saturated my 12 G Java machine after some minutes. Then I installed module_of_mine.xqm into BaseX repo, and ran the same query using basex gui with 4G heap size. The query ran perfectly in some 30 sec. My first penny guess is that the built-in BaseX xquery profiler is doing some nice work there ? If so, is there a way to activate / desactivate this profiler from eclipse or from some configuration files ? Cheers, Jean-Marc
Re: [basex-talk] Introspection bug in BaseX7.8 beta 4cfa54c ?
Topic closed : the last release (31/01/14) fixed the bug. Thx all 2014-01-31 jean-marc Mercier : > Mmhh...ok. I will try to find it that way. I am actually trying to send > you the prof:time(prof:mem)) of the main algorithm, to compare the main > perf algorithm between the two betas. > My first penny guess is that it could come from Leo's work, since INLINE > and TAILCALL impacted the result. Could you be kind enough to point me out > his January check-in ? Having a look to his check-in, if we are lucky, > could help me isolate the problem. > > > 2014-01-31 Christian Grün > > If you got time, it may already be helpful if you provide us with your >> code base. You could also try to run BaseX with less memory assigned (e.g. >> 128m) and send us the stack trace, which will be triggered via the debug >> flag (-d) as soon as RAM is exhausted, or you could run BaseX with Java >> profiling arguments (e.g. -Xrunhprof:cpu=sample,depth=15) and provide us >> with the output. I can't promise anything, though. >> >> >> On Thu, Jan 30, 2014 at 9:39 PM, jean-marc Mercier < >> jeanmarc.merc...@gmail.com> wrote: >> >>> Christian, >>> >>> The point is that I don't know how to provide a self-contained query. >>> >>> The only strategy I could propose, to provide a self-contained query >>> isolating the memory management problem, is really too time-consuming : I >>> can imagine a binary-chop search, comparing the memory management of the >>> two BaseX beta versions, over a very complex algorithm, to isolate the >>> problem. This is probably a week work, I can't spend such a time. >>> >>> Do you have a better strategy to find the problem ? >>> >>> >>> >>> >>> 2014-01-30 Christian Grün >>> >>> > A first clue: I set INLINELIMIT = 0 TAILCALL = -1, and the process >>>> ended, >>>> > even if it is taking now some 3/4 minutes and nearly 5,5 Go memory >>>> > footprint. >>>> >>>> Thanks. Could you provide us with a self-contained query? >>>> Christian >>>> >>> >>> >> > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] Make external variable declarations mandatory #833
Ahok ! Thanks very much, it does work now. Topic closed.
2014-02-01 Christian Grün :
> Hi Jean-Marc (answering on behalf of Dirk... Let's see who is faster ;),
>
> your query will work if you change the xquery:eval call as follows:
>
> xquery:eval($query || $exp,$vars)
>
> Best,
> Christian
>
>
>
> On Sat, Feb 1, 2014 at 11:44 AM, jean-marc Mercier <
> jeanmarc.merc...@gmail.com> wrote:
>
>> Dirkk, thx for your answer.
>>
>> I am really sorry, but definitively, I don't know how to use xquery:eval
>> with the last release.
>>
>> Following your advise, I tried the following one, but it does not work:
>>
>> declare function local:eval_helper($exp, $args){
>> let $query := 'declare variable $xml external; $xml'
>> let $vars := map { '$xml':= $args }
>> return
>> xquery:eval('$xml'||$exp,$vars)
>> };
>> declare variable $test := 1;
>> local:eval_helper('//tata', $test)
>>
>> Could you try yourself ? I mean, can you make the above user case working
>> ? This use case worked fineley with BaseX7.8 beta 6aeaebf, before the
>> external variable correction.
>>
>>
>>
>>
>> 2014-02-01 Dirk Kirsten :
>>
>> Hi Jean-Marc,
>>>
>>> external variables now have to be explicitly declared. You can do so by:
>>>
>>> declare variable $xml external;
>>>
>>> Cheers,
>>> Dirk
>>>
>>> On 01/02/14 09:16, jean-marc Mercier wrote:
>>> > Hi,
>>> >
>>> > I am extensively using a function wrapping xquery:eval. My use case is
>>> > roughly the following :
>>> >
>>> > declare function local:eval_helper($exp,
>>> > $args){xquery:eval('$xml'||$exp,map{'$xml' := $args})};
>>> > declare variable $test := 1;
>>> > local:eval_helper3('//tata', $test)
>>> >
>>> >
>>> > Since the following correction : [1]
>>> > https://github.com/BaseXdb/basex/issues/833, the previous code raises
>>> now a
>>> > [XPST0008] Undefined variable $xml.
>>> >
>>> > I have tried several combinations to find a workaround, none are
>>> working :
>>> >
>>> > - declare function local:eval_helper($exp,
>>> > $args){xquery:eval(''||$exp,map{'' := $args})};
>>> > ([XPDY0050] Root of the context item must be a document node.).
>>> > - declare function local:eval_helper($exp,
>>> $args){xquery:evaluate($arg||$exp)};
>>> >
>>> > [XPTY0019] Steps within a path expression must yield nodes; xs:integer
>>> > found.
>>> > -- declare variable $xml external := "xml", : [XPST0008] Undefined
>>> variable
>>> > $xml.
>>> >
>>> > What is the correct workaround ?
>>> >
>>> >
>>> > Cheers
>>> >
>>> > Jean-Marc
>>> >
>>> >
>>> >
>>> > ___
>>> > BaseX-Talk mailing list
>>> > BaseX-Talk@mailman.uni-konstanz.de
>>> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
>>> >
>>>
>>> --
>>> Dirk Kirsten, BaseX GmbH, http://basex.org
>>> |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz
>>> |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer:
>>> | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle
>>> `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22
>>> ___
>>> BaseX-Talk mailing list
>>> BaseX-Talk@mailman.uni-konstanz.de
>>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
>>>
>>
>>
>> ___
>> BaseX-Talk mailing list
>> BaseX-Talk@mailman.uni-konstanz.de
>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
>>
>>
>
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] Make external variable declarations mandatory #833
Dirkk, thx for your answer.
I am really sorry, but definitively, I don't know how to use xquery:eval
with the last release.
Following your advise, I tried the following one, but it does not work:
declare function local:eval_helper($exp, $args){
let $query := 'declare variable $xml external; $xml'
let $vars := map { '$xml':= $args }
return
xquery:eval('$xml'||$exp,$vars)
};
declare variable $test := 1;
local:eval_helper('//tata', $test)
Could you try yourself ? I mean, can you make the above user case working ?
This use case worked fineley with BaseX7.8 beta 6aeaebf, before the
external variable correction.
2014-02-01 Dirk Kirsten :
> Hi Jean-Marc,
>
> external variables now have to be explicitly declared. You can do so by:
>
> declare variable $xml external;
>
> Cheers,
> Dirk
>
> On 01/02/14 09:16, jean-marc Mercier wrote:
> > Hi,
> >
> > I am extensively using a function wrapping xquery:eval. My use case is
> > roughly the following :
> >
> > declare function local:eval_helper($exp,
> > $args){xquery:eval('$xml'||$exp,map{'$xml' := $args})};
> > declare variable $test := 1;
> > local:eval_helper3('//tata', $test)
> >
> >
> > Since the following correction : [1]
> > https://github.com/BaseXdb/basex/issues/833, the previous code raises
> now a
> > [XPST0008] Undefined variable $xml.
> >
> > I have tried several combinations to find a workaround, none are working
> :
> >
> > - declare function local:eval_helper($exp,
> > $args){xquery:eval(''||$exp,map{'' := $args})};
> > ([XPDY0050] Root of the context item must be a document node.).
> > - declare function local:eval_helper($exp,
> $args){xquery:evaluate($arg||$exp)};
> >
> > [XPTY0019] Steps within a path expression must yield nodes; xs:integer
> > found.
> > -- declare variable $xml external := "xml", : [XPST0008] Undefined
> variable
> > $xml.
> >
> > What is the correct workaround ?
> >
> >
> > Cheers
> >
> > Jean-Marc
> >
> >
> >
> > ___
> > BaseX-Talk mailing list
> > BaseX-Talk@mailman.uni-konstanz.de
> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
> >
>
> --
> Dirk Kirsten, BaseX GmbH, http://basex.org
> |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz
> |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer:
> | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle
> `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22
> ___
> BaseX-Talk mailing list
> BaseX-Talk@mailman.uni-konstanz.de
> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
>
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
[basex-talk] Make external variable declarations mandatory #833
Hi,
I am extensively using a function wrapping xquery:eval. My use case is
roughly the following :
declare function local:eval_helper($exp,
$args){xquery:eval('$xml'||$exp,map{'$xml' := $args})};
declare variable $test := 1;
local:eval_helper3('//tata', $test)
Since the following correction : [1]
https://github.com/BaseXdb/basex/issues/833, the previous code raises now a
[XPST0008] Undefined variable $xml.
I have tried several combinations to find a workaround, none are working :
- declare function local:eval_helper($exp,
$args){xquery:eval(''||$exp,map{'' := $args})};
([XPDY0050] Root of the context item must be a document node.).
- declare function local:eval_helper($exp, $args){xquery:evaluate($arg||$exp)};
[XPTY0019] Steps within a path expression must yield nodes; xs:integer
found.
-- declare variable $xml external := "xml", : [XPST0008] Undefined variable
$xml.
What is the correct workaround ?
Cheers
Jean-Marc
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] Introspection bug in BaseX7.8 beta 4cfa54c ?
Mmhh...ok. I will try to find it that way. I am actually trying to send you the prof:time(prof:mem)) of the main algorithm, to compare the main perf algorithm between the two betas. My first penny guess is that it could come from Leo's work, since INLINE and TAILCALL impacted the result. Could you be kind enough to point me out his January check-in ? Having a look to his check-in, if we are lucky, could help me isolate the problem. 2014-01-31 Christian Grün > If you got time, it may already be helpful if you provide us with your > code base. You could also try to run BaseX with less memory assigned (e.g. > 128m) and send us the stack trace, which will be triggered via the debug > flag (-d) as soon as RAM is exhausted, or you could run BaseX with Java > profiling arguments (e.g. -Xrunhprof:cpu=sample,depth=15) and provide us > with the output. I can't promise anything, though. > > > On Thu, Jan 30, 2014 at 9:39 PM, jean-marc Mercier < > jeanmarc.merc...@gmail.com> wrote: > >> Christian, >> >> The point is that I don't know how to provide a self-contained query. >> >> The only strategy I could propose, to provide a self-contained query >> isolating the memory management problem, is really too time-consuming : I >> can imagine a binary-chop search, comparing the memory management of the >> two BaseX beta versions, over a very complex algorithm, to isolate the >> problem. This is probably a week work, I can't spend such a time. >> >> Do you have a better strategy to find the problem ? >> >> >> >> >> 2014-01-30 Christian Grün >> >> > A first clue: I set INLINELIMIT = 0 TAILCALL = -1, and the process >>> ended, >>> > even if it is taking now some 3/4 minutes and nearly 5,5 Go memory >>> > footprint. >>> >>> Thanks. Could you provide us with a self-contained query? >>> Christian >>> >> >> > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] Introspection bug in BaseX7.8 beta 4cfa54c ?
Christian, The point is that I don't know how to provide a self-contained query. The only strategy I could propose, to provide a self-contained query isolating the memory management problem, is really too time-consuming : I can imagine a binary-chop search, comparing the memory management of the two BaseX beta versions, over a very complex algorithm, to isolate the problem. This is probably a week work, I can't spend such a time. Do you have a better strategy to find the problem ? 2014-01-30 Christian Grün > > A first clue: I set INLINELIMIT = 0 TAILCALL = -1, and the process ended, > > even if it is taking now some 3/4 minutes and nearly 5,5 Go memory > > footprint. > > Thanks. Could you provide us with a self-contained query? > Christian > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] Introspection bug in BaseX7.8 beta 4cfa54c ?
Christian, A first clue: I set INLINELIMIT = 0 TAILCALL = -1, and the process ended, even if it is taking now some 3/4 minutes and nearly 5,5 Go memory footprint. 2014-01-30 jean-marc Mercier > Hi Christian, > > Thx, I tested it. It seems that a regression has been introduced : my main > process, running in 30sec with 4 Go in memory with BaseX7.8 6aeaebf, > never ended with this latest release. Indeed, it started disk-swapping as > it is consuming more than 6 Go now (the size heap of my JAVA machine). > > Do you have any clue, or suggestion, that could help us finding this > regression ? > > Cheers, > > Jean-Marc > > > > > 2014-01-30 Christian Grün > > Hi Jean-Marc, >> >> you got it right, we fixed the bug! I've just updated the snapshot; >> feel free to give it a try! >> >> Thanks, >> Christian >> >> >> >> On Thu, Jan 30, 2014 at 5:07 PM, jean-marc Mercier >> wrote: >> > Hello, >> > >> > As bug https://github.com/BaseXdb/basex/issues/839 has been closed, I >> tried >> > to upgrade from BaseX7.8 6aeaebf to BaseX beta_4cfa54c. >> > Unfortunately, this bug still appears in 4cfa54c, and I rolled back >> again to >> > beta 6aeaebf . >> > Did I misunderstoog something ? >> > >> > Cheers >> > >> > ___ >> > BaseX-Talk mailing list >> > BaseX-Talk@mailman.uni-konstanz.de >> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> > >> > > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] Introspection bug in BaseX7.8 beta 4cfa54c ?
Hi Christian, Thx, I tested it. It seems that a regression has been introduced : my main process, running in 30sec with 4 Go in memory with BaseX7.8 6aeaebf, never ended with this latest release. Indeed, it started disk-swapping as it is consuming more than 6 Go now (the size heap of my JAVA machine). Do you have any clue, or suggestion, that could help us finding this regression ? Cheers, Jean-Marc 2014-01-30 Christian Grün > Hi Jean-Marc, > > you got it right, we fixed the bug! I've just updated the snapshot; > feel free to give it a try! > > Thanks, > Christian > > > > On Thu, Jan 30, 2014 at 5:07 PM, jean-marc Mercier > wrote: > > Hello, > > > > As bug https://github.com/BaseXdb/basex/issues/839 has been closed, I > tried > > to upgrade from BaseX7.8 6aeaebf to BaseX beta_4cfa54c. > > Unfortunately, this bug still appears in 4cfa54c, and I rolled back > again to > > beta 6aeaebf . > > Did I misunderstoog something ? > > > > Cheers > > > > ___ > > BaseX-Talk mailing list > > BaseX-Talk@mailman.uni-konstanz.de > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
[basex-talk] Introspection bug in BaseX7.8 beta 4cfa54c ?
Hello, As bug https://github.com/BaseXdb/basex/issues/839 has been closed, I tried to upgrade from BaseX7.8 6aeaebf to BaseX beta_4cfa54c. Unfortunately, this bug still appears in 4cfa54c, and I rolled back again to beta 6aeaebf . Did I misunderstoog something ? Cheers ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] BaseX as a Windows Service
Ok, I missed the point sorry : BaseX being a Java executable, we have to wrap it before registring it as a service with sc or installutil. Wrapping Java for windows servicing is quite easy, I did it some years ago with JSW. I had a look to YAJSW : it seems to be a better choice than JSW now. Flexible enough licensing, and most importantly support for windows 64 bits. @Christian : Installing a service with YAJSW seems really straightforward (see : http://yajsw.sourceforge.net/). Is it worth writing a Wiki page ? 2014/1/22 jean-marc Mercier > Hi, > > Maybe I don't understand your needs, but there exists native tools to > install services with windows. For instance the batch command sc, or the > visual studio tool installutil. > Is there any incentive for not using them ? > > > 2014/1/22 Christian Grün > >> Hi Sam, hi Huib, >> >> if one of you succeeds installing BaseX as service, would you be >> willing to contribute a little HowTo article for our Wiki? >> >> Christian >> >> >> On Wed, Jan 22, 2014 at 2:19 PM, Huib Verweij >> wrote: >> > Hi Sam, >> > >> > I don't know about any special tricks BaseX has up it's sleeve, but I >> have >> > had succes running Solr, BigData and other Java server apps as a Windows >> > service using YAJSW. It's fairly easy to accomplish. There are other >> > applications that can run Java apps as a Windows Service for you as >> well. >> > >> > Op 22 jan. 2014, om 14:12 heeft Sam Eklund het >> > volgende geschreven: >> > >> > Is there a possibility to install BaseX server as a service under >> Windows? >> > >> > The –S switch to start BaseX in the background is nice, however a proper >> > windows service would be even better! >> > >> > Anyone that has accomplished this? >> > >> > >> > >> > Hartelijke groet, >> > >> > Huib Verweij >> > >> > >> > >> > >> > ___ >> > BaseX-Talk mailing list >> > BaseX-Talk@mailman.uni-konstanz.de >> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> > >> ___ >> BaseX-Talk mailing list >> BaseX-Talk@mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> > > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] BaseX as a Windows Service
Hi, Maybe I don't understand your needs, but there exists native tools to install services with windows. For instance the batch command sc, or the visual studio tool installutil. Is there any incentive for not using them ? 2014/1/22 Christian Grün > Hi Sam, hi Huib, > > if one of you succeeds installing BaseX as service, would you be > willing to contribute a little HowTo article for our Wiki? > > Christian > > > On Wed, Jan 22, 2014 at 2:19 PM, Huib Verweij > wrote: > > Hi Sam, > > > > I don't know about any special tricks BaseX has up it's sleeve, but I > have > > had succes running Solr, BigData and other Java server apps as a Windows > > service using YAJSW. It's fairly easy to accomplish. There are other > > applications that can run Java apps as a Windows Service for you as well. > > > > Op 22 jan. 2014, om 14:12 heeft Sam Eklund het > > volgende geschreven: > > > > Is there a possibility to install BaseX server as a service under > Windows? > > > > The –S switch to start BaseX in the background is nice, however a proper > > windows service would be even better! > > > > Anyone that has accomplished this? > > > > > > > > Hartelijke groet, > > > > Huib Verweij > > > > > > > > > > ___ > > BaseX-Talk mailing list > > BaseX-Talk@mailman.uni-konstanz.de > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > > ___ > BaseX-Talk mailing list > BaseX-Talk@mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] Possible bug in BaseX interpreter, might be the introspection module ?
Hi all,
Due to this issue, I am pulling back to the BaseX beta 6aeaebf (that is not
showing this issue), waiting for a fix.
Cheers,
Jean-Marc
2014/1/10 jean-marc Mercier
> Hi all,
>
> it seems that this bug is linked to the introspection module. I isolated a
> code that produces on my config a exception, see at end of mail.
> Note :
> - It seems to be a regression, as this code worked finely with the 3/12
> version of BaseX.
> - It took me a whole day to isolate this exception. This "overhead" is due
> to the JAVA catch / raising exception BaseX mechanism, letting the user
> blind with its XQUERY code. As bugs are becoming harder and harder to spot,
> maybe we should think about a way, or a tool, to easen trace / debug while
> the interpreter raises a JAVA exception ?
>
> Cheers
>
>
> declare function local:map_inspect_functions() { map:new(for $fun in
> inspect:functions() return map:entry(local:name-function($fun),$fun)) };
> declare function
> local:name-function($fun){fn:data(inspect:function($fun)/@name)};
> declare function local:get_void() { function(){ () }};
> declare variable $local:MapIntrospecting := local:map_inspect_functions();
> let $fun_2 := map:get($local:MapIntrospecting,"local:get_void")
> return count($fun_2())
>
> Improper use? Potential bug? Your feedback is welcome:
> Contact: basex-talk@mailman.uni-konstanz.de
> Version: BaseX 7.8 beta 132460f
> Java: Oracle Corporation, 1.7.0_45
> OS: Windows 7, amd64
> Stack Trace:
> java.lang.ClassCastException: org.basex.query.value.type.AtomType cannot
> be cast to org.basex.query.value.type.FuncType
> at org.basex.query.func.InlineFunc.item(InlineFunc.java:261)
> at org.basex.query.func.InlineFunc.value(InlineFunc.java:280)
> at org.basex.query.QueryContext.value(QueryContext.java:366)
> at org.basex.query.func.StaticFunc.invValue(StaticFunc.java:203)
> at org.basex.query.func.FuncCall.invoke(FuncCall.java:96)
> at org.basex.query.func.FuncCall.value(FuncCall.java:52)
> at org.basex.query.QueryContext.value(QueryContext.java:366)
> at org.basex.query.value.item.FuncItem.invValue(FuncItem.java:130)
> at org.basex.query.func.FuncCall.invoke(FuncCall.java:96)
> at org.basex.query.func.FuncCall.value(FuncCall.java:52)
> at org.basex.query.func.FuncCall.iter(FuncCall.java:57)
> at org.basex.query.QueryContext.iter(QueryContext.java:355)
> at org.basex.query.func.FNAggr.item(FNAggr.java:36)
> at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46)
> at org.basex.query.MainModule.iter(MainModule.java:96)
> at org.basex.query.QueryContext.iter(QueryContext.java:309)
> at org.basex.query.QueryProcessor.iter(QueryProcessor.java:81)
> at org.basex.core.cmd.AQuery.query(AQuery.java:89)
> at org.basex.core.cmd.XQuery.run(XQuery.java:22)
> at org.basex.core.Command.run(Command.java:329)
> at org.basex.core.Command.execute(Command.java:94)
> at org.basex.server.LocalSession.execute(LocalSession.java:121)
> at org.basex.server.Session.execute(Session.java:37)
> at org.basex.core.Main.execute(Main.java:146)
> at org.basex.BaseX.(BaseX.java:119)
> at org.basex.BaseX.main(BaseX.java:38)
>
>
>
>
> 2014/1/6 jean-marc Mercier
>
>> Note : casewhere, I checked that this issue does not depend upon the
>> values of INLINELIMIT and TAILCALL.
>>
>>
>> 2014/1/6 jean-marc Mercier
>>
>>> Hello,
>>>
>>> I have a quite strange behavior, that might be a bug in the
>>> introspection module. I am just describing it, and will work a bit to
>>> produce a reproductible code if you don't have a quick clue about the
>>> problem.
>>>
>>> I created a map of string to functions, called $common:MapIntrospecting,
>>> generated by the introspection module.
>>>
>>> The following code works perfectly (you can't run it, but believe me :)
>>>
>>> declare variable $map :=
>>> map:entry("Alimentation:nodes#1",Alimentation:nodes#1);
>>> declare variable $nodes := a big node...
>>>
>>> let $fun_1 := map:get($map,"Alimentation:nodes#1")
>>> let $fun_2 := map:get($common:MapIntrospecting,"Alimentation:nodes#1")
>>> return $fun_2($nodes)
>>>
>>> And now, the following code (I just cut the useless first line) produces
>>> a spurious and ferocious exception
>>>
>>> let $fun_2 := map:get($common:MapIntrospecting,"Alimentation:nodes#1")
>>> return $fun_2($nodes)
>>>
>>> Any clue ?
>>>
>>> Improper use? Potential bug? Your feedback is welcome:
>>> Contact: basex-talk@mailman.uni-konstanz.de
>>>
Re: [basex-talk] C++ API for BaseX v0.0
Dirk, Hi. I just succeeded in compiling a first version with Cygwin compiler support (i.e. 100%unix compiler), and made it work under a windows machine. Could you try to compile and test it with a unix based one ? Thanks. There will probably be errors, let discuss them. You should fin at https://github.com/JohnLeM/BaseXCPPAPI the resulting first trial. Note : - there are now two project. One is for VS2012 compiler, located in /VS2012 - the other is a first trial, for a third-eye check (you). You should find a auto-generated (debug) makefile at root. Cheers, Jean-Marc 2014/1/9 Dirk Kirsten > Hi Jean-Marc, > > sure, I am happy to test. > > Cheers, > Dirk > > On 09/01/14 11:21, jean-marc Mercier wrote: > > Hi Dirk, > > > > I am currently trying to compile this project with GCC / MINGW under > > eclipse. With eclipse, I should be able to provide a unix compatible > > makefile. Once produced, could you be kind enough to test ? > > > > To answer your thoughts : > > - PugiXML (or any C++ library with XPATH support) is a very efficient way > > to bind C++ with XML. It seems natural to provide an example using it. > I'll > > provide also a very simple example without. > > - std::wstring_convert. It is necessary for utf8 support, but you're > right, > > it seems not to be supported. I removed all references to it, but there > > will not be any utf8 support. > > > > Cheers > > > > Jean-Marc > > > > > > > > 2014/1/8 Dirk Kirsten > > > >> Hi Jean-Marc, > >> > >> I did take a (very) quick look at it. However, I failed to compile this > >> on my linux machine using gcc. Some thoughts: > >> > >> - PugiXML seems to be used just for the example, I guess (because > >> requiring another XPath processor to connect to BaseX would be weird). > >> Seems easier to me to simplify the example and just execute a simple > >> XQuery without relying on PugiXML > >> - You use TCHAR, this seems to be windows specific. > >> - std::wstring_convert isn't supported by gcc, it seems > >> - It would be really nice to have a simply Makefile > >> - Please be careful with case-sensitivity. BasexClient.h and > >> BaseXClient.h aren't the same using a case-sensitive file system. > >> > >> In general, I think it would be very beneficial for BaseX to have an C++ > >> client which compiles on every major platform. So we might want to add a > >> more polished version of your connector to the official BaseX release. > >> However, as already mentioned, there still seems to be some work to do > >> and I am not sure anyone in the BaseX core team has any particular C++ > >> skills ... so every outside contribution is (as always) more than > welcome. > >> > >> Cheers, > >> Dirk > >> > >> On 08/01/14 15:13, jean-marc Mercier wrote: > >>> Hi all, > >>> > >>> C++ users could be interested in this first "quick and dirty" version > of > >> a > >>> C++ API for BaseX, located at https://github.com/JohnLeM/BaseXCPPAPI. > >> Even > >>> if this v0.0 version is built upon Visual C++ 2012 Express, the C++ > code > >>> should be multi platform. Hence, if there is any C++ / Unix / gcc BaseX > >>> users, do not hesitate to ask for a make file. > >>> > >>> To BaseX team : you could also be interested to including a cleaner > >> version > >>> of this work to your API set for your C++ clients. > >>> > >>> This API is built upon : > >>> > >>> - boost::asio libray for the TCP connector (http://www.boost.org/) > >>> - PugiXML (https://code.google.com/p/pugixml/) for a lightweight C++ > >> parser > >>> with XPATH 1.0 support. > >>> > >>> The provided example is a simple construction of an integer list. > >>> > >>> Cheers > >>> > >>> Jean-Marc > >>> > >>> > >>> > >>> ___ > >>> BaseX-Talk mailing list > >>> BaseX-Talk@mailman.uni-konstanz.de > >>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >>> > >> > >> -- > >> Dirk Kirsten, BaseX GmbH, http://basex.org > >> |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz > >> |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: > >> | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle > >> `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22 > >> > > > > -- > Dirk Kirsten, BaseX GmbH, http://basex.org > |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz > |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: > | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle > `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22 > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] Possible bug in BaseX interpreter, might be the introspection module ?
Hi all,
it seems that this bug is linked to the introspection module. I isolated a
code that produces on my config a exception, see at end of mail.
Note :
- It seems to be a regression, as this code worked finely with the 3/12
version of BaseX.
- It took me a whole day to isolate this exception. This "overhead" is due
to the JAVA catch / raising exception BaseX mechanism, letting the user
blind with its XQUERY code. As bugs are becoming harder and harder to spot,
maybe we should think about a way, or a tool, to easen trace / debug while
the interpreter raises a JAVA exception ?
Cheers
declare function local:map_inspect_functions() { map:new(for $fun in
inspect:functions() return map:entry(local:name-function($fun),$fun)) };
declare function
local:name-function($fun){fn:data(inspect:function($fun)/@name)};
declare function local:get_void() { function(){ () }};
declare variable $local:MapIntrospecting := local:map_inspect_functions();
let $fun_2 := map:get($local:MapIntrospecting,"local:get_void")
return count($fun_2())
Improper use? Potential bug? Your feedback is welcome:
Contact: basex-talk@mailman.uni-konstanz.de
Version: BaseX 7.8 beta 132460f
Java: Oracle Corporation, 1.7.0_45
OS: Windows 7, amd64
Stack Trace:
java.lang.ClassCastException: org.basex.query.value.type.AtomType cannot be
cast to org.basex.query.value.type.FuncType
at org.basex.query.func.InlineFunc.item(InlineFunc.java:261)
at org.basex.query.func.InlineFunc.value(InlineFunc.java:280)
at org.basex.query.QueryContext.value(QueryContext.java:366)
at org.basex.query.func.StaticFunc.invValue(StaticFunc.java:203)
at org.basex.query.func.FuncCall.invoke(FuncCall.java:96)
at org.basex.query.func.FuncCall.value(FuncCall.java:52)
at org.basex.query.QueryContext.value(QueryContext.java:366)
at org.basex.query.value.item.FuncItem.invValue(FuncItem.java:130)
at org.basex.query.func.FuncCall.invoke(FuncCall.java:96)
at org.basex.query.func.FuncCall.value(FuncCall.java:52)
at org.basex.query.func.FuncCall.iter(FuncCall.java:57)
at org.basex.query.QueryContext.iter(QueryContext.java:355)
at org.basex.query.func.FNAggr.item(FNAggr.java:36)
at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46)
at org.basex.query.MainModule.iter(MainModule.java:96)
at org.basex.query.QueryContext.iter(QueryContext.java:309)
at org.basex.query.QueryProcessor.iter(QueryProcessor.java:81)
at org.basex.core.cmd.AQuery.query(AQuery.java:89)
at org.basex.core.cmd.XQuery.run(XQuery.java:22)
at org.basex.core.Command.run(Command.java:329)
at org.basex.core.Command.execute(Command.java:94)
at org.basex.server.LocalSession.execute(LocalSession.java:121)
at org.basex.server.Session.execute(Session.java:37)
at org.basex.core.Main.execute(Main.java:146)
at org.basex.BaseX.(BaseX.java:119)
at org.basex.BaseX.main(BaseX.java:38)
2014/1/6 jean-marc Mercier
> Note : casewhere, I checked that this issue does not depend upon the
> values of INLINELIMIT and TAILCALL.
>
>
> 2014/1/6 jean-marc Mercier
>
>> Hello,
>>
>> I have a quite strange behavior, that might be a bug in the introspection
>> module. I am just describing it, and will work a bit to produce a
>> reproductible code if you don't have a quick clue about the problem.
>>
>> I created a map of string to functions, called $common:MapIntrospecting,
>> generated by the introspection module.
>>
>> The following code works perfectly (you can't run it, but believe me :)
>>
>> declare variable $map :=
>> map:entry("Alimentation:nodes#1",Alimentation:nodes#1);
>> declare variable $nodes := a big node...
>>
>> let $fun_1 := map:get($map,"Alimentation:nodes#1")
>> let $fun_2 := map:get($common:MapIntrospecting,"Alimentation:nodes#1")
>> return $fun_2($nodes)
>>
>> And now, the following code (I just cut the useless first line) produces
>> a spurious and ferocious exception
>>
>> let $fun_2 := map:get($common:MapIntrospecting,"Alimentation:nodes#1")
>> return $fun_2($nodes)
>>
>> Any clue ?
>>
>> Improper use? Potential bug? Your feedback is welcome:
>> Contact: basex-talk@mailman.uni-konstanz.de
>> Version: BaseX 7.8 beta 132460f
>> Java: Oracle Corporation, 1.7.0_45
>> OS: Windows 7, amd64
>> Stack Trace:
>> java.lang.ClassCastException: org.basex.query.value.type.AtomType cannot
>> be cast to org.basex.query.value.type.FuncType
>> at org.basex.query.func.InlineFunc.item(InlineFunc.java:261)
>> at org.basex.query.func.InlineFunc.value(InlineFunc.java:280)
>> at org.basex.query.QueryContext.value(QueryContext.java:366)
>> at org.basex.query.func.StaticFuncCall.evalArgs(StaticFuncCall.java:154)
>> at org.basex.query.func.FuncCall.value(FuncCall.java:5
Re: [basex-talk] C++ API for BaseX v0.0
Hi Dirk, I am currently trying to compile this project with GCC / MINGW under eclipse. With eclipse, I should be able to provide a unix compatible makefile. Once produced, could you be kind enough to test ? To answer your thoughts : - PugiXML (or any C++ library with XPATH support) is a very efficient way to bind C++ with XML. It seems natural to provide an example using it. I'll provide also a very simple example without. - std::wstring_convert. It is necessary for utf8 support, but you're right, it seems not to be supported. I removed all references to it, but there will not be any utf8 support. Cheers Jean-Marc 2014/1/8 Dirk Kirsten > Hi Jean-Marc, > > I did take a (very) quick look at it. However, I failed to compile this > on my linux machine using gcc. Some thoughts: > > - PugiXML seems to be used just for the example, I guess (because > requiring another XPath processor to connect to BaseX would be weird). > Seems easier to me to simplify the example and just execute a simple > XQuery without relying on PugiXML > - You use TCHAR, this seems to be windows specific. > - std::wstring_convert isn't supported by gcc, it seems > - It would be really nice to have a simply Makefile > - Please be careful with case-sensitivity. BasexClient.h and > BaseXClient.h aren't the same using a case-sensitive file system. > > In general, I think it would be very beneficial for BaseX to have an C++ > client which compiles on every major platform. So we might want to add a > more polished version of your connector to the official BaseX release. > However, as already mentioned, there still seems to be some work to do > and I am not sure anyone in the BaseX core team has any particular C++ > skills ... so every outside contribution is (as always) more than welcome. > > Cheers, > Dirk > > On 08/01/14 15:13, jean-marc Mercier wrote: > > Hi all, > > > > C++ users could be interested in this first "quick and dirty" version of > a > > C++ API for BaseX, located at https://github.com/JohnLeM/BaseXCPPAPI. > Even > > if this v0.0 version is built upon Visual C++ 2012 Express, the C++ code > > should be multi platform. Hence, if there is any C++ / Unix / gcc BaseX > > users, do not hesitate to ask for a make file. > > > > To BaseX team : you could also be interested to including a cleaner > version > > of this work to your API set for your C++ clients. > > > > This API is built upon : > > > > - boost::asio libray for the TCP connector (http://www.boost.org/) > > - PugiXML (https://code.google.com/p/pugixml/) for a lightweight C++ > parser > > with XPATH 1.0 support. > > > > The provided example is a simple construction of an integer list. > > > > Cheers > > > > Jean-Marc > > > > > > > > ___ > > BaseX-Talk mailing list > > BaseX-Talk@mailman.uni-konstanz.de > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > > > -- > Dirk Kirsten, BaseX GmbH, http://basex.org > |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz > |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: > | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle > `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22 > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
[basex-talk] C++ API for BaseX v0.0
Hi all, C++ users could be interested in this first "quick and dirty" version of a C++ API for BaseX, located at https://github.com/JohnLeM/BaseXCPPAPI. Even if this v0.0 version is built upon Visual C++ 2012 Express, the C++ code should be multi platform. Hence, if there is any C++ / Unix / gcc BaseX users, do not hesitate to ask for a make file. To BaseX team : you could also be interested to including a cleaner version of this work to your API set for your C++ clients. This API is built upon : - boost::asio libray for the TCP connector (http://www.boost.org/) - PugiXML (https://code.google.com/p/pugixml/) for a lightweight C++ parser with XPATH 1.0 support. The provided example is a simple construction of an integer list. Cheers Jean-Marc ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] BaseX for noobs: C++ API ?
I finally succeeded connecting to BaseX server. This issue was stupidly due to the last trailing '\0' character in the server time stamp response, messing out the password MD5 computations. 2014/1/7 jean-marc Mercier > @Dirk thx : I saw this with WireShark, even if I could have realized it > without. > > > 2014/1/7 jean-marc Mercier > >> a...I finally succeeded ! This was so stupidly due to the last >> trailing '\0' character in the server time stamp response, that was so >> stupidly messed out the password MD5 computations. I do like computers when >> I am loosing two days on such b***t. >> >> I am actually eating my keyboard >> >> >> 2014/1/6 jean-marc Mercier >> >>> I installed WireShark. Thx Dirkk, this should definitively helps me. I >>> keep you informed. >>> >>> >>> 2014/1/6 jean-marc Mercier >>> >>>> Dirk, yes I read it. I also have the C# connector that works, and, even >>>> if it does not compile, the C Unix code. But still I can't connect :( >>>> >>>> >>>> 2014/1/6 Dirk Kirsten >>>> >>>>> One other thing, based on your last mail: Did you have a look at >>>>> >>>>> https://docs.basex.org/wiki/Server_Protocol#Authentication_.28via_cram-md5.29 >>>>> ? >>>>> So the basic thing is that you have to hash the password first and then >>>>> combine the result with the timestamp send by the server and then hash >>>>> again to prevent time-based attacks. >>>>> >>>>> Cheers, >>>>> Dirk >>>>> >>>>> On 06/01/14 21:36, jean-marc Mercier wrote: >>>>> > ah ! nice ! I did not thought about this ! >>>>> > >>>>> > >>>>> > 2014/1/6 Dirk Kirsten >>>>> > >>>>> >> Hello Jean-Marc, >>>>> >> >>>>> >> No, there is no such echoing service. Especially, because you sait >>>>> that >>>>> >> you can't connect, so the authentification does not seem to have >>>>> taken >>>>> >> place. So doing anything before authentification would be rather >>>>> weird. >>>>> >> >>>>> >> Also, you did not mention _what_ excactly does not work. However, >>>>> you >>>>> >> can use tools like Wireshark (which should also be available on >>>>> Windows, >>>>> >> I guess) to analyze all your traffic (you can also filter for >>>>> traffic to >>>>> >> specific ports, so just relevant stuff is shown to you). >>>>> >> >>>>> >> Cheers, >>>>> >> Dirk >>>>> >> >>>>> >> On 06/01/14 21:15, jean-marc Mercier wrote: >>>>> >>> Hello, >>>>> >>> >>>>> >>> I still can't connect to BaseX. I checked that I sent to the server >>>>> >> exactly >>>>> >>> the same bytes than the C# connector that worked on my >>>>> configuration. >>>>> >>> >>>>> >>> Is there any way to make the BaseX server echoing any input >>>>> request on a >>>>> >>> particular socket to check exactly what my TCP connector is >>>>> sending ? >>>>> >>> >>>>> >>> >>>>> >>> >>>>> >>> 2014/1/5 jean-marc Mercier >>>>> >>> >>>>> >>>> Dirk, >>>>> >>>> >>>>> >>>> Thx. Well, I am suffering a lot to develop this C++ BaseX API. I >>>>> did not >>>>> >>>> succeed in installing the openssl MD5 algorithm, but I finally >>>>> found a >>>>> >> MD5 >>>>> >>>> algorithm that seems to work (at least, it seems to compute the >>>>> MD5 hash >>>>> >>>> for "admin" correctly). However, after two days, I still can't >>>>> connect >>>>> >> to >>>>> >>>> the BaseX server, and can't figure out why... >>>>> >>>> >>>>> >>>> Concerning TCP connectors, I think that it is wiser to use >>>>> bo
Re: [basex-talk] BaseX for noobs: C++ API ?
Liam, Thx for your proposal. Unix C++ user can use the already existing BaseX API, that unfortunately don't work for poor Windows users. My program do send information to BaseX server, since the server replies through its log "admin (the user account I am using to log in) -- acces denied". There is probably something in the MD5 algorithm that I am using. However, I checked that the MD5 algorithm computes correctly the hash for 'admin'. If BaseX server could echo the incoming request, that would be of great help, but I don't know how to do this. If nobody knows, I will have to fork the BAseX code and to try compile a server to see what is happening on the server side...that is probably some days of hard works :( 2014/1/6 Liam R E Quin > On Mon, 2014-01-06 at 21:15 +0100, jean-marc Mercier wrote: > > Hello, > > > > I still can't connect to BaseX. I checked that I sent to the server > exactly > > the same bytes than the C# connector that worked on my configuration. > > > > Is there any way to make the BaseX server echoing any input request on a > > particular socket to check exactly what my TCP connector is sending ? > I do have an old C program I wrote years ago that saves a copy of data > sent on a socket into a file; I wrote it for Unix but maybe it'd work > for you. There are some packages for Linux that do soemthing similar > too. > > Make sure that you check the return value of every system call you use - > writing to a socket does not necessarily succeed. Make sure also that > your code connects to the right IP address and port - e.g. try > connecting to something like a Web server where you can see the logs, > and sending "GET / HTTP/1.0\r\nAccept: */*\r\n\r\n" at it. Forgetting to > convert addresses and port numbers to network order is sometimes an > issue. > > Liam > > -- > Liam Quin - XML Activity Lead, W3C, http://www.w3.org/People/Quin/ > Pictures from old books: http://fromoldbooks.org/ > Ankh: irc.sorcery.net irc.gnome.org freenode/#xml > > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] BaseX for noobs: C++ API ?
Hello, I still can't connect to BaseX. I checked that I sent to the server exactly the same bytes than the C# connector that worked on my configuration. Is there any way to make the BaseX server echoing any input request on a particular socket to check exactly what my TCP connector is sending ? 2014/1/5 jean-marc Mercier > Dirk, > > Thx. Well, I am suffering a lot to develop this C++ BaseX API. I did not > succeed in installing the openssl MD5 algorithm, but I finally found a MD5 > algorithm that seems to work (at least, it seems to compute the MD5 hash > for "admin" correctly). However, after two days, I still can't connect to > the BaseX server, and can't figure out why... > > Concerning TCP connectors, I think that it is wiser to use boost::asio > library : it is not MS, and seems to becoming the C++ network standard > library. > > Since this C++ BaseX API'fate is to be published (provided we can make it > work!), maybe somebody in BaseX team, that have some experience with C++, > could have a look to it ? I put the current code at > https://github.com/JohnLeM/BaseXCPPAPI, as a bottle in the sea :( > > > > > > > 2014/1/5 Dirk Kirsten > >> Hello Jean-Marc, >> >> I see your point. Unfortunately, I don't have any experience doing >> Windows socket programming (nor do I run any Windows OS), so I am of not >> much help here. But I would guess the C code can be quite easily >> replaced (e.g. OpenSSL is most likely just used for the md5 computation) >> and based on some articles like >> >> http://www.codeproject.com/Articles/13071/Programming-Windows-TCP-Sockets-in-C-for-the-Begin >> I would guess windows socket programming can be done quite similar to >> Unix sockets. >> >> Cheers, >> Dirk >> >> On 03/01/14 11:33, jean-marc Mercier wrote: >> > Hi Dirk, >> > >> > Thanx for your answer. Indeed, I tried to install the existing C API. >> For >> > windows user : >> > >> > Some *.h, *.c files of the BaseX C API seems to work only with a gcc >> > compiler. Some others one don't even exists under windows, at least I >> did >> > not found them : >> > 1) the BaseX C API seems to use the open SSL framework. I did download >> and >> > install it, but have some problem to compile it with Visual C++ Express. >> > 2) Some needed c/h files (see for instance "#include ) are >> > located in the MinGW distribution for windows. However, these files are >> > bound to GCC compiler, and can not be compiled with Microsoft compiler. >> > >> > Indeed, I am trying to mimic in C++ the C# API, that works on my >> > configuration. But it is not easy for me, since I never manipulated >> sockets >> > and TCP connectors, and will take me a lot of time. Thus, if anyone >> already >> > did the work, that would be so nice to share :) >> > >> > Cheers, >> > >> > Jean-Marc >> > >> > >> > >> > >> > 2014/1/3 Dirk Kirsten >> > >> >> Hello Jean-Marc, >> >> >> >> What problems did you experience when using the provided C API >> >> (https://github.com/BaseXdb/basex-api/tree/master/src/main/c) under >> >> Windows 7? It should also work under windows. >> >> Additionally, you can find the source code of the C API at the >> basex-api >> >> repository, so this should give you a starting point if you ant to >> >> rewrite anything. >> >> >> >> Cheers, >> >> Dirk >> >> >> >> On 03/01/14 11:12, jean-marc Mercier wrote: >> >>> Hi all, >> >>> >> >>> going back to the root... I am trying to connect my favorite C++ >> tools to >> >>> BaseX. >> >>> Unfortunately, I am using visual C++ 12 Express under windows 7. It >> seems >> >>> that windows users will not be able to use the existing BaseX API C >> >>> connector, and need another one. Thus I started to code a C++ BaseX >> API >> >>> connector. >> >>> >> >>> Writing a API connector is quite difficult for me, since I don't have >> the >> >>> required networks knowledge. >> >>> >> >>> Has somebody already some code to that purposes ? >> >>> >> >>> >> >>> >> >>> ___ >> >>> Ba
Re: [basex-talk] Possible bug in BaseX interpreter, might be the introspection module ?
Note : casewhere, I checked that this issue does not depend upon the values
of INLINELIMIT and TAILCALL.
2014/1/6 jean-marc Mercier
> Hello,
>
> I have a quite strange behavior, that might be a bug in the introspection
> module. I am just describing it, and will work a bit to produce a
> reproductible code if you don't have a quick clue about the problem.
>
> I created a map of string to functions, called $common:MapIntrospecting,
> generated by the introspection module.
>
> The following code works perfectly (you can't run it, but believe me :)
>
> declare variable $map :=
> map:entry("Alimentation:nodes#1",Alimentation:nodes#1);
> declare variable $nodes := a big node...
>
> let $fun_1 := map:get($map,"Alimentation:nodes#1")
> let $fun_2 := map:get($common:MapIntrospecting,"Alimentation:nodes#1")
> return $fun_2($nodes)
>
> And now, the following code (I just cut the useless first line) produces a
> spurious and ferocious exception
>
> let $fun_2 := map:get($common:MapIntrospecting,"Alimentation:nodes#1")
> return $fun_2($nodes)
>
> Any clue ?
>
> Improper use? Potential bug? Your feedback is welcome:
> Contact: basex-talk@mailman.uni-konstanz.de
> Version: BaseX 7.8 beta 132460f
> Java: Oracle Corporation, 1.7.0_45
> OS: Windows 7, amd64
> Stack Trace:
> java.lang.ClassCastException: org.basex.query.value.type.AtomType cannot
> be cast to org.basex.query.value.type.FuncType
> at org.basex.query.func.InlineFunc.item(InlineFunc.java:261)
> at org.basex.query.func.InlineFunc.value(InlineFunc.java:280)
> at org.basex.query.QueryContext.value(QueryContext.java:366)
> at org.basex.query.func.StaticFuncCall.evalArgs(StaticFuncCall.java:154)
> at org.basex.query.func.FuncCall.value(FuncCall.java:52)
> at org.basex.query.QueryContext.value(QueryContext.java:366)
> at org.basex.query.func.StaticFunc.invValue(StaticFunc.java:203)
> at org.basex.query.func.FuncCall.invoke(FuncCall.java:96)
> at org.basex.query.func.FuncCall.value(FuncCall.java:52)
> at org.basex.query.QueryContext.value(QueryContext.java:366)
> at org.basex.query.func.StaticFuncCall.evalArgs(StaticFuncCall.java:154)
> at org.basex.query.func.FuncCall.value(FuncCall.java:52)
> at org.basex.query.QueryContext.value(QueryContext.java:366)
> at org.basex.query.func.StaticFuncCall.evalArgs(StaticFuncCall.java:154)
> at org.basex.query.func.FuncCall.value(FuncCall.java:52)
> at org.basex.query.QueryContext.value(QueryContext.java:366)
> at org.basex.query.func.StaticFunc.invValue(StaticFunc.java:203)
> at org.basex.query.func.FuncCall.invoke(FuncCall.java:96)
> at org.basex.query.func.FuncCall.value(FuncCall.java:52)
> at org.basex.query.QueryContext.value(QueryContext.java:366)
> at org.basex.query.func.StaticFunc.invValue(StaticFunc.java:203)
> at org.basex.query.func.FuncCall.invoke(FuncCall.java:96)
> at org.basex.query.func.FuncCall.value(FuncCall.java:52)
> at org.basex.query.var.StaticVar.value(StaticVar.java:129)
> at org.basex.query.var.StaticVarRef.value(StaticVarRef.java:58)
> at org.basex.query.QueryContext.value(QueryContext.java:366)
> at org.basex.query.func.StaticFuncCall.evalArgs(StaticFuncCall.java:154)
> at org.basex.query.func.FuncCall.value(FuncCall.java:52)
> at org.basex.query.QueryContext.value(QueryContext.java:366)
> at org.basex.query.expr.List.value(List.java:133)
> at org.basex.query.QueryContext.value(QueryContext.java:366)
> at org.basex.query.func.StaticFunc.invValue(StaticFunc.java:203)
> at org.basex.query.func.FuncCall.invoke(FuncCall.java:96)
> at org.basex.query.func.FuncCall.value(FuncCall.java:52)
> at org.basex.query.QueryContext.value(QueryContext.java:366)
> at org.basex.query.value.item.FuncItem.invValue(FuncItem.java:130)
> at org.basex.query.func.FuncCall.invoke(FuncCall.java:96)
> at org.basex.query.func.FuncCall.value(FuncCall.java:52)
> at org.basex.query.QueryContext.value(QueryContext.java:366)
> at org.basex.query.MainModule.value(MainModule.java:79)
> at org.basex.query.QueryContext.value(QueryContext.java:323)
> at org.basex.query.QueryContext.iter(QueryContext.java:309)
> at org.basex.query.QueryProcessor.iter(QueryProcessor.java:81)
> at org.basex.core.cmd.AQuery.query(AQuery.java:89)
> at org.basex.core.cmd.XQuery.run(XQuery.java:22)
> at org.basex.core.Command.run(Command.java:329)
> at org.basex.core.Command.execute(Command.java:94)
> at org.basex.server.LocalSession.execute(LocalSession.java:121)
> at org.basex.server.Session.execute(Session.java:37)
> at org.basex.core.Main.execute(Main.java:146)
> at org.basex.BaseX.(BaseX.java:119)
> at org.basex.BaseX.main(BaseX.java:38)
>
>
>
>
>
>
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
[basex-talk] Possible bug in BaseX interpreter, might be the introspection module ?
Hello,
I have a quite strange behavior, that might be a bug in the introspection
module. I am just describing it, and will work a bit to produce a
reproductible code if you don't have a quick clue about the problem.
I created a map of string to functions, called $common:MapIntrospecting,
generated by the introspection module.
The following code works perfectly (you can't run it, but believe me :)
declare variable $map :=
map:entry("Alimentation:nodes#1",Alimentation:nodes#1);
declare variable $nodes := a big node...
let $fun_1 := map:get($map,"Alimentation:nodes#1")
let $fun_2 := map:get($common:MapIntrospecting,"Alimentation:nodes#1")
return $fun_2($nodes)
And now, the following code (I just cut the useless first line) produces a
spurious and ferocious exception
let $fun_2 := map:get($common:MapIntrospecting,"Alimentation:nodes#1")
return $fun_2($nodes)
Any clue ?
Improper use? Potential bug? Your feedback is welcome:
Contact: basex-talk@mailman.uni-konstanz.de
Version: BaseX 7.8 beta 132460f
Java: Oracle Corporation, 1.7.0_45
OS: Windows 7, amd64
Stack Trace:
java.lang.ClassCastException: org.basex.query.value.type.AtomType cannot be
cast to org.basex.query.value.type.FuncType
at org.basex.query.func.InlineFunc.item(InlineFunc.java:261)
at org.basex.query.func.InlineFunc.value(InlineFunc.java:280)
at org.basex.query.QueryContext.value(QueryContext.java:366)
at org.basex.query.func.StaticFuncCall.evalArgs(StaticFuncCall.java:154)
at org.basex.query.func.FuncCall.value(FuncCall.java:52)
at org.basex.query.QueryContext.value(QueryContext.java:366)
at org.basex.query.func.StaticFunc.invValue(StaticFunc.java:203)
at org.basex.query.func.FuncCall.invoke(FuncCall.java:96)
at org.basex.query.func.FuncCall.value(FuncCall.java:52)
at org.basex.query.QueryContext.value(QueryContext.java:366)
at org.basex.query.func.StaticFuncCall.evalArgs(StaticFuncCall.java:154)
at org.basex.query.func.FuncCall.value(FuncCall.java:52)
at org.basex.query.QueryContext.value(QueryContext.java:366)
at org.basex.query.func.StaticFuncCall.evalArgs(StaticFuncCall.java:154)
at org.basex.query.func.FuncCall.value(FuncCall.java:52)
at org.basex.query.QueryContext.value(QueryContext.java:366)
at org.basex.query.func.StaticFunc.invValue(StaticFunc.java:203)
at org.basex.query.func.FuncCall.invoke(FuncCall.java:96)
at org.basex.query.func.FuncCall.value(FuncCall.java:52)
at org.basex.query.QueryContext.value(QueryContext.java:366)
at org.basex.query.func.StaticFunc.invValue(StaticFunc.java:203)
at org.basex.query.func.FuncCall.invoke(FuncCall.java:96)
at org.basex.query.func.FuncCall.value(FuncCall.java:52)
at org.basex.query.var.StaticVar.value(StaticVar.java:129)
at org.basex.query.var.StaticVarRef.value(StaticVarRef.java:58)
at org.basex.query.QueryContext.value(QueryContext.java:366)
at org.basex.query.func.StaticFuncCall.evalArgs(StaticFuncCall.java:154)
at org.basex.query.func.FuncCall.value(FuncCall.java:52)
at org.basex.query.QueryContext.value(QueryContext.java:366)
at org.basex.query.expr.List.value(List.java:133)
at org.basex.query.QueryContext.value(QueryContext.java:366)
at org.basex.query.func.StaticFunc.invValue(StaticFunc.java:203)
at org.basex.query.func.FuncCall.invoke(FuncCall.java:96)
at org.basex.query.func.FuncCall.value(FuncCall.java:52)
at org.basex.query.QueryContext.value(QueryContext.java:366)
at org.basex.query.value.item.FuncItem.invValue(FuncItem.java:130)
at org.basex.query.func.FuncCall.invoke(FuncCall.java:96)
at org.basex.query.func.FuncCall.value(FuncCall.java:52)
at org.basex.query.QueryContext.value(QueryContext.java:366)
at org.basex.query.MainModule.value(MainModule.java:79)
at org.basex.query.QueryContext.value(QueryContext.java:323)
at org.basex.query.QueryContext.iter(QueryContext.java:309)
at org.basex.query.QueryProcessor.iter(QueryProcessor.java:81)
at org.basex.core.cmd.AQuery.query(AQuery.java:89)
at org.basex.core.cmd.XQuery.run(XQuery.java:22)
at org.basex.core.Command.run(Command.java:329)
at org.basex.core.Command.execute(Command.java:94)
at org.basex.server.LocalSession.execute(LocalSession.java:121)
at org.basex.server.Session.execute(Session.java:37)
at org.basex.core.Main.execute(Main.java:146)
at org.basex.BaseX.(BaseX.java:119)
at org.basex.BaseX.main(BaseX.java:38)
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] BaseX for noobs: C++ API ?
Dirk, Thx. Well, I am suffering a lot to develop this C++ BaseX API. I did not succeed in installing the openssl MD5 algorithm, but I finally found a MD5 algorithm that seems to work (at least, it seems to compute the MD5 hash for "admin" correctly). However, after two days, I still can't connect to the BaseX server, and can't figure out why... Concerning TCP connectors, I think that it is wiser to use boost::asio library : it is not MS, and seems to becoming the C++ network standard library. Since this C++ BaseX API'fate is to be published (provided we can make it work!), maybe somebody in BaseX team, that have some experience with C++, could have a look to it ? I put the current code at https://github.com/JohnLeM/BaseXCPPAPI, as a bottle in the sea :( 2014/1/5 Dirk Kirsten > Hello Jean-Marc, > > I see your point. Unfortunately, I don't have any experience doing > Windows socket programming (nor do I run any Windows OS), so I am of not > much help here. But I would guess the C code can be quite easily > replaced (e.g. OpenSSL is most likely just used for the md5 computation) > and based on some articles like > > http://www.codeproject.com/Articles/13071/Programming-Windows-TCP-Sockets-in-C-for-the-Begin > I would guess windows socket programming can be done quite similar to > Unix sockets. > > Cheers, > Dirk > > On 03/01/14 11:33, jean-marc Mercier wrote: > > Hi Dirk, > > > > Thanx for your answer. Indeed, I tried to install the existing C API. For > > windows user : > > > > Some *.h, *.c files of the BaseX C API seems to work only with a gcc > > compiler. Some others one don't even exists under windows, at least I did > > not found them : > > 1) the BaseX C API seems to use the open SSL framework. I did download > and > > install it, but have some problem to compile it with Visual C++ Express. > > 2) Some needed c/h files (see for instance "#include ) are > > located in the MinGW distribution for windows. However, these files are > > bound to GCC compiler, and can not be compiled with Microsoft compiler. > > > > Indeed, I am trying to mimic in C++ the C# API, that works on my > > configuration. But it is not easy for me, since I never manipulated > sockets > > and TCP connectors, and will take me a lot of time. Thus, if anyone > already > > did the work, that would be so nice to share :) > > > > Cheers, > > > > Jean-Marc > > > > > > > > > > 2014/1/3 Dirk Kirsten > > > >> Hello Jean-Marc, > >> > >> What problems did you experience when using the provided C API > >> (https://github.com/BaseXdb/basex-api/tree/master/src/main/c) under > >> Windows 7? It should also work under windows. > >> Additionally, you can find the source code of the C API at the basex-api > >> repository, so this should give you a starting point if you ant to > >> rewrite anything. > >> > >> Cheers, > >> Dirk > >> > >> On 03/01/14 11:12, jean-marc Mercier wrote: > >>> Hi all, > >>> > >>> going back to the root... I am trying to connect my favorite C++ tools > to > >>> BaseX. > >>> Unfortunately, I am using visual C++ 12 Express under windows 7. It > seems > >>> that windows users will not be able to use the existing BaseX API C > >>> connector, and need another one. Thus I started to code a C++ BaseX API > >>> connector. > >>> > >>> Writing a API connector is quite difficult for me, since I don't have > the > >>> required networks knowledge. > >>> > >>> Has somebody already some code to that purposes ? > >>> > >>> > >>> > >>> ___ > >>> BaseX-Talk mailing list > >>> BaseX-Talk@mailman.uni-konstanz.de > >>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >>> > >> > >> -- > >> Dirk Kirsten, BaseX GmbH, http://basex.org > >> |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz > >> |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: > >> | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle > >> `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22 > >> ___ > >> BaseX-Talk mailing list > >> BaseX-Talk@mailman.uni-konstanz.de > >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >> > > > > -- > Dirk Kirsten, BaseX GmbH, http://basex.org > |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz > |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: > | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle > `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22 > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] BaseX for noobs: C++ API ?
Hi Dirk, Thanx for your answer. Indeed, I tried to install the existing C API. For windows user : Some *.h, *.c files of the BaseX C API seems to work only with a gcc compiler. Some others one don't even exists under windows, at least I did not found them : 1) the BaseX C API seems to use the open SSL framework. I did download and install it, but have some problem to compile it with Visual C++ Express. 2) Some needed c/h files (see for instance "#include ) are located in the MinGW distribution for windows. However, these files are bound to GCC compiler, and can not be compiled with Microsoft compiler. Indeed, I am trying to mimic in C++ the C# API, that works on my configuration. But it is not easy for me, since I never manipulated sockets and TCP connectors, and will take me a lot of time. Thus, if anyone already did the work, that would be so nice to share :) Cheers, Jean-Marc 2014/1/3 Dirk Kirsten > Hello Jean-Marc, > > What problems did you experience when using the provided C API > (https://github.com/BaseXdb/basex-api/tree/master/src/main/c) under > Windows 7? It should also work under windows. > Additionally, you can find the source code of the C API at the basex-api > repository, so this should give you a starting point if you ant to > rewrite anything. > > Cheers, > Dirk > > On 03/01/14 11:12, jean-marc Mercier wrote: > > Hi all, > > > > going back to the root... I am trying to connect my favorite C++ tools to > > BaseX. > > Unfortunately, I am using visual C++ 12 Express under windows 7. It seems > > that windows users will not be able to use the existing BaseX API C > > connector, and need another one. Thus I started to code a C++ BaseX API > > connector. > > > > Writing a API connector is quite difficult for me, since I don't have the > > required networks knowledge. > > > > Has somebody already some code to that purposes ? > > > > > > > > ___ > > BaseX-Talk mailing list > > BaseX-Talk@mailman.uni-konstanz.de > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > > > -- > Dirk Kirsten, BaseX GmbH, http://basex.org > |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz > |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: > | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle > `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22 > ___ > BaseX-Talk mailing list > BaseX-Talk@mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
[basex-talk] BaseX for noobs: C++ API ?
Hi all, going back to the root... I am trying to connect my favorite C++ tools to BaseX. Unfortunately, I am using visual C++ 12 Express under windows 7. It seems that windows users will not be able to use the existing BaseX API C connector, and need another one. Thus I started to code a C++ BaseX API connector. Writing a API connector is quite difficult for me, since I don't have the required networks knowledge. Has somebody already some code to that purposes ? ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] BaseX for noobs : how to add user module ?
Hi, @Andy, @Florent, Thx for answering ! @Andy. I know a little bit the repository. Indeed, I am using it to add Java bindings to BaseX. However, I am looking to an alternative solution. @Florent. Thanx for pointing me this link, it seems to be exactly what I am looking for. Cheers, Jean-Marc 2013/12/30 Andy Bunce > Hi Jean-marc, > > Have you looked at using the repository? [1] There is a more complex > example at [2] > > Regards > /Andy > > [1] http://docs.basex.org/wiki/Repository#Java > [2] http://files.basex.org/modules/org/basex/modules/StringSimilarity/ > > > On Mon, Dec 30, 2013 at 11:03 AM, jean-marc Mercier < > jeanmarc.merc...@gmail.com> wrote: > >> Hello, >> >> I am trying to add some external features to BaseX, and would like to use >> them into a BaseX client XQUERY module. I can't see any simple way to do >> it, is there any ? >> >> Here is the scenario : suppose that you want to add a super-mega map >> module to BaseX, let us call it supermegamap, that is a collection of >> functions : supermegamap:new, supermegamap:contains, supermegamap:get. >> >> I would like to use my supermegamap through BaseX via a xquery module : >> let $my_supermegamp := supermegamap:new(). >> >> I see only two quite not very handy ways to do this : >> >> 1) The first one is to use JAVA bindings. Ok, it will work, but is not >> really elegant. >> 2) The second one is to fork BaseX code : adding the JAVA code to BaseX, >> modifying the file Function.JAVA. That might work, should be efficient, but >> it is quite heavy, and overall, will be most probably a dead-code the next >> time I will have to upgrade BaseX. >> >> Is there any alternative way ? >> >> Thx >> >> Jean-Marc >> >> >> >> >> >> >> >> ___ >> BaseX-Talk mailing list >> BaseX-Talk@mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> >> > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
[basex-talk] BaseX for noobs : how to add user module ?
Hello, I am trying to add some external features to BaseX, and would like to use them into a BaseX client XQUERY module. I can't see any simple way to do it, is there any ? Here is the scenario : suppose that you want to add a super-mega map module to BaseX, let us call it supermegamap, that is a collection of functions : supermegamap:new, supermegamap:contains, supermegamap:get. I would like to use my supermegamap through BaseX via a xquery module : let $my_supermegamp := supermegamap:new(). I see only two quite not very handy ways to do this : 1) The first one is to use JAVA bindings. Ok, it will work, but is not really elegant. 2) The second one is to fork BaseX code : adding the JAVA code to BaseX, modifying the file Function.JAVA. That might work, should be efficient, but it is quite heavy, and overall, will be most probably a dead-code the next time I will have to upgrade BaseX. Is there any alternative way ? Thx Jean-Marc ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] Memory usage !!
Hi John, I am a simple BaseX user, not belonging to BaseX team, but my first penny guess is that you are providing too few information to your request to be proceeded: what are these numbers expressed in, how did you produced them, what kind of query are you trying to run, etc etc... Hope this helps, Cheers, Jean-Marc 2013/12/30 John Best > Dear Team, > > I have developed and application using Eclipse RCP. I am using BaseX 7.6 > to store XML data. > > Yesterday, when I was testing the application with various kind of > searches, the response was quite slow as I went ahead with more and more > searches. > > To my surprise, I notices the in Windows Task Manager, the javaw.exe was > consuming more memory... Then I restarted the laptop and did this procedure > again > > Following is the memory usage - > > 1 - 84920 > 2 - 101584 > 3 - 108224 > 4 - 109284 > 5 - 111768 > 6 - 128468 > 7 - 160616 > 8 - 180212 > 9 - 198313 > 10 - 212592 > > > Why this is so ? and how can I reduce this memory usage by javaw.exe ? > > > > -- > Have a nice day > JBest > > > > ___ > BaseX-Talk mailing list > BaseX-Talk@mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] INLINER : bug ?
Leo, You are right, there was a type error in my code, thx a lot for pointing it out. I am recomputing the "profile" result of your code with full inlining. Cheers, Jean-Marc 2013/12/29 Leo Wörteler > Dear Jean-Marc, > > Am 29.12.2013 13:09, schrieb jean-marc Mercier: > > @Leo, there might be a bug in the BaseX INLINER (I am currently >> profiling your module https://github.com/LeoWoerteler/xq-modules/). >> > > thanks first for looking at my code! > > > With these settings, I am raising a [XPTY0004] Single item expected >> error. >> > > The behavior *with* inlining is actually the correct one. There is a type > error in your code that is not reported unless some functions are inlined. > > You declare function parameters in your `idmap` module as `$idmap as > function() as function(*)`, which means that `$idmap` must be bound to a > function item with no arguments that returns a *single* function item with > arbitrary arity. The functions you bind to those parameters however > actually return *two* function items, namely the two elements of a pair. A > correct type would thus be `$idmap as function() as function(*)+`. > > The actual bug is that this type is not always checked if the function > item is not inlined, this will be fixed soon. > > Hope that helps, > Leo > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
[basex-talk] INLINER : bug ?
Hi Everybody @Leo, there might be a bug in the BaseX INLINER (I am currently profiling your module https://github.com/LeoWoerteler/xq-modules/). To reproduce it : - use the 3 files attached. - Edit baseX config file .baseX, set the following Local options : # Local Options INLINELIMIT = 5 - launch the XQUERY file MapProfiler.xquery. With these settings, the profile function works finely. Then - Edit baseX config file .baseX, set the following Local options : # Local Options INLINELIMIT = 6 With these settings, I am raising a [XPTY0004] Single item expected error. Cheers, Jean-Marc idmap.xqm Description: Binary data set.xqm Description: Binary data MapProfiler.xquery Description: Binary data ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] [ANN] Higher-Order XQuery Modules
Leonard, Hello. I'am just reading this annoucement, trashed erroneously by gmail. Sorry for not having reacted before. It sounds very interesting. First probably stupid question : how do we "install" these modules ? 2013/12/12 Leonard Wörteler > Hello everyone, > > I want to announce the small collection of XQuery 3.0 modules [1] that > I've written over the last few weeks. I use them mostly for testing the > BaseX optimizer, but there should be no BaseX-specific XQuery code in the > repository. > > I have implemented some purely functional data structures including maps > based on AVL and Red-Black Trees that both support arbitrary values as keys > and efficient deletion. Some documentation can be found under each module's > namespace URI or under [2]. > > Constructive feedback of any kind (suggestions, bug reports, pull > requests) would be much appreciated. > > Hope you like it, > cheers, Leo > > [1] https://github.com/LeoWoerteler/xq-modules/ > [2] http://www.woerteler.de/xquery/modules/ > ___ > BaseX-Talk mailing list > BaseX-Talk@mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
[basex-talk] add-dayTimeDuration-to-date
Hello, I am trying to add dates and duration through BaseX, in conformance with http://www.w3.org/TR/xpath-functions-30/#func-add-dayTimeDuration-to-date. However, BaseX interpreter does not recognize the function add-dayTimeDuration-to-date, nor op:add-dayTimeDuration-to-date. Did I misunderstood something ? Thanx for your answer, Cheers, Jean-Marc ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] map:module. Behavior of maps of nodes
Hi, Casewhere someone is using these files, here is a more stable version, including some quite handy union, intersection and except "operators". By stable I mean that I am using them extensively since some weeks without any modifications ! Cheers Jean-Marc 2013/12/8 jean-marc Mercier > Christian, > Hi > > >thanks a lot for your profiling results and the attached XQuery modules > > Since I am currently extensively using these files, I spotted and > corrected some bugs. I will provide a more robust code as soon as I > consider them as stables. > > Notice that I am also using sets to have a meaningful *except, > intersection or union* operators for nodes or functions. Indeed, the > definition given in > http://www.w3.org/TR/xpath-functions/#union-intersection-except is not > very clear for me. > > Jean-Marc > > > 2013/12/7 Christian Grün > >> Hi Jean-Marc, >> >> thanks a lot for your profiling results and the attached XQuery modules, >> very appreciated. Your code may serve as good inspiration for other users >> getting into the world of higher order functions. We’ll consider your input >> once we make existing modules public in a central repository (sth. that has >> been planned for a while, but... well, is one of many other projects we are >> working on). >> >> Christian >> ___ >> >> On Fri, Dec 6, 2013 at 4:29 PM, jean-marc Mercier < >> jeanmarc.merc...@gmail.com> wrote: >> >>> Christian, >>> >>> Hi. >>> >>> To end this topic, you should find enclosed two files that fulfills now >>> my requirements. I found these objects very practical, their purpose are to >>> handle set or maps of nodes / functions. >>> I am publishing them here very informally, hoping other BaseX users >>> could find them as helpful as I do, or maybe yourself, or a standardization >>> committee. >>> >>> - The first one, idmap.xqm, allows to use maps, with nodes or functions >>> as keys. >>> - The second one, set.xqm, allows to define set of items (I use such an >>> object to define a stack). >>> >>> These files consists in small overload of the BaseX map module. Both >>> require an external id function of atomic type, that must be provided by >>> the user. >>> >>> Below are the profile tests for these maps and set. The profile function >>> is located into idmap:profile($i,$j) to be run by others. Basically, the >>> "idmap" seems to be five time slower than BaseX map for inserting atomic >>> types. Set seems to be quite comparable to BaseX map. This overload come >>> from perf issues in XQUERY, but I can handle this small overload for my >>> purposes. Obviously, for atomic type, it is better to use directly the map >>> module, and these objects embed a basic protection to avoid a misusing. >>> >>> Some remarks : >>> >>> 1) It means that users are linked to BaseX map module when they are >>> using these objects. As you already know, I tried unsuccessfully to write >>> them in pure XQUERY, one of the motivation was not to be linked an external >>> module. >>> 2) Using a map module to define a set is not really the best way to >>> proceed, but it seems that XQUERY 3.0 users don't have any other choice. >>> 3) I think that BaseX could provide a simplified access to such objects. >>> Indeed, the BaseX interpreter must have a default id for nodes or functions >>> that it could expose to users, or use it directly through the BaseX map >>> module. >>> >>> Hope that helps >>> >>> Cheers, >>> >>> Jean-Marc >>> >>> >>> Tag name/Text nb integer_set node_set integer_map Basex_integer_map >>> node_map test 1024 24.63 ms 43.36 ms 51.6 ms 6.35 ms 70.92 ms test >>> 2048 10.91 ms 46.71 ms 17.52 ms 3.57 ms 51.62 ms test 4096 1.71 ms 31.52 >>> ms 11.7 ms 1.04 ms 23.59 ms test 8192 2.49 ms 22.88 ms 15.41 ms 1.72 ms >>> 32.5 >>> ms test 16384 5.12 ms 20.23 ms 33.47 ms 2.9 ms 50.62 ms test 32768 8.31 >>> ms 31.54 ms 58.95 ms 5.77 ms 86.86 ms test 65536 20.1 ms 71.39 ms 99.57 >>> ms 13.0 ms 162.09 ms test 131072 48.15 ms 157.27 ms 680.47 ms 33.68 ms >>> 947.9 >>> ms test 262144 99.67 ms 305.88 ms 1132.31 ms 77.85 ms 683.37 ms test >>> 524288 239.43 ms 569.46 ms 1769.53 ms 383.77 ms 2903.24 ms test >>> 1.05E+06 737.79 ms 2485.75 ms 5022.02 ms 1031.71 ms 10204.07 ms >>> >>>
Re: [basex-talk] Alternative to XQDT ?
Thx Christian, It seems nice, I will try this plug-in for the "community" IntelliJ edition. However only the very expensive "ultimate edition" of IntelliJ can integrate with spring framework and SQL databases ? Too bad that XQDT is stalling for xquery 3.0, because Eclipse was capable to integrate JAVA, C++, xquery, SQL and spring framework in the same IDE, very handy for me. 2013/12/11 Christian Grün > Hi Jean-Marc, > > currently, there are no plugin alternatives for Eclipse, and it seems > that the development has pretty much stalled. > > Grzegorz Ligas is currently developing a nice plugin for IntelliJ IDEA > [1]; you may want to have a look it, too. > > Best, > Christian > > [1] http://ligasgr.github.io/intellij-xquery/ > _______ > > On Wed, Dec 11, 2013 at 9:40 AM, jean-marc Mercier > wrote: > > Hello, > > > > I am developing using Eclipse, together with the integrated xquery editor > > provided by XQDT. > > This last editor does not seem to support xquery 3.0 syntax. I can > survive, > > but there are a lot of nice functionality that are not working anymore > (for > > instance the outline windows). > > > > Does someone know any alternative to XQDT editor that could integrate > with > > eclipse and is xquery 3.0 compliant ? > > > > Cheers > > > > Jean-Marc > > > > ___ > > BaseX-Talk mailing list > > BaseX-Talk@mailman.uni-konstanz.de > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
[basex-talk] Alternative to XQDT ?
Hello, I am developing using Eclipse, together with the integrated xquery editor provided by XQDT. This last editor does not seem to support xquery 3.0 syntax. I can survive, but there are a lot of nice functionality that are not working anymore (for instance the outline windows). Does someone know any alternative to XQDT editor that could integrate with eclipse and is xquery 3.0 compliant ? Cheers Jean-Marc ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] map:module. Behavior of maps of nodes
Christian, Hi >thanks a lot for your profiling results and the attached XQuery modules Since I am currently extensively using these files, I spotted and corrected some bugs. I will provide a more robust code as soon as I consider them as stables. Notice that I am also using sets to have a meaningful *except, intersection or union* operators for nodes or functions. Indeed, the definition given in http://www.w3.org/TR/xpath-functions/#union-intersection-except is not very clear for me. Jean-Marc 2013/12/7 Christian Grün > Hi Jean-Marc, > > thanks a lot for your profiling results and the attached XQuery modules, > very appreciated. Your code may serve as good inspiration for other users > getting into the world of higher order functions. We’ll consider your input > once we make existing modules public in a central repository (sth. that has > been planned for a while, but... well, is one of many other projects we are > working on). > > Christian > ___ > > On Fri, Dec 6, 2013 at 4:29 PM, jean-marc Mercier < > jeanmarc.merc...@gmail.com> wrote: > >> Christian, >> >> Hi. >> >> To end this topic, you should find enclosed two files that fulfills now >> my requirements. I found these objects very practical, their purpose are to >> handle set or maps of nodes / functions. >> I am publishing them here very informally, hoping other BaseX users could >> find them as helpful as I do, or maybe yourself, or a standardization >> committee. >> >> - The first one, idmap.xqm, allows to use maps, with nodes or functions >> as keys. >> - The second one, set.xqm, allows to define set of items (I use such an >> object to define a stack). >> >> These files consists in small overload of the BaseX map module. Both >> require an external id function of atomic type, that must be provided by >> the user. >> >> Below are the profile tests for these maps and set. The profile function >> is located into idmap:profile($i,$j) to be run by others. Basically, the >> "idmap" seems to be five time slower than BaseX map for inserting atomic >> types. Set seems to be quite comparable to BaseX map. This overload come >> from perf issues in XQUERY, but I can handle this small overload for my >> purposes. Obviously, for atomic type, it is better to use directly the map >> module, and these objects embed a basic protection to avoid a misusing. >> >> Some remarks : >> >> 1) It means that users are linked to BaseX map module when they are using >> these objects. As you already know, I tried unsuccessfully to write them in >> pure XQUERY, one of the motivation was not to be linked an external module. >> 2) Using a map module to define a set is not really the best way to >> proceed, but it seems that XQUERY 3.0 users don't have any other choice. >> 3) I think that BaseX could provide a simplified access to such objects. >> Indeed, the BaseX interpreter must have a default id for nodes or functions >> that it could expose to users, or use it directly through the BaseX map >> module. >> >> Hope that helps >> >> Cheers, >> >> Jean-Marc >> >> >> Tag name/Text nb integer_set node_set integer_map Basex_integer_map >> node_map test 1024 24.63 ms 43.36 ms 51.6 ms 6.35 ms 70.92 ms test 2048 >> 10.91 >> ms 46.71 ms 17.52 ms 3.57 ms 51.62 ms test 4096 1.71 ms 31.52 ms 11.7 ms >> 1.04 >> ms 23.59 ms test 8192 2.49 ms 22.88 ms 15.41 ms 1.72 ms 32.5 ms test >> 16384 5.12 ms 20.23 ms 33.47 ms 2.9 ms 50.62 ms test 32768 8.31 ms 31.54 >> ms 58.95 ms 5.77 ms 86.86 ms test 65536 20.1 ms 71.39 ms 99.57 ms 13.0 >> ms 162.09 ms test 131072 48.15 ms 157.27 ms 680.47 ms 33.68 ms 947.9 ms >> test 262144 99.67 ms 305.88 ms 1132.31 ms 77.85 ms 683.37 ms test 524288 >> 239.43 >> ms 569.46 ms 1769.53 ms 383.77 ms 2903.24 ms test 1.05E+06 737.79 ms 2485.75 >> ms 5022.02 ms 1031.71 ms 10204.07 ms >> >> >> >> >> >> >> >> >> 2013/12/1 jean-marc Mercier >> >>> Christian, >>> >>> Concerning this topic, it took me a whole week of quite hard work to >>> write a Red / Black removable tree written in XQUERY, together with a map >>> interface that mimic the basex one. However I won't go public, and would >>> advise not to use it, because it suffers from performance issues. >>> >>> I opened a thread concerning this perf issue, that might be deeper that >>> I thought. >>> >>> Cheers, >>> >>> Jean-Marc >>> >>> >>> >>
Re: [basex-talk] BaseX Evaluation for Big Data
Gary, A remark over this topic, since you are worried about performances for 20 Gb of data with BaseX and might be interested by a user experience. Once you will be successful in inserting, simple XPATH queries over such a collection should take time below milliseconds. 2013/12/6 jean-marc Mercier > Hi Gary, > > Maybe you should wait for BaseX team members advises over this topic. > > Telling from my own experience, I can remember also having met some > problem trying to insert 20 Gb of data into BaseX through the GUI. > > I had to achieve this into smaller steps, using batch commands (see > http://docs.basex.org/wiki/Commands), in order to free memory. i.e. I > first inserted the xml into 4 or 5 passes. Then created some indexes, > etc...In that way it was possible to run the insertion, even with 4Gb > machines. > > Hope this helps > > Jean-Marc > > > > > > > 2013/12/6 Huband, Gary W *HS > >> I still get an out of main memory when I try to create attribute indexes. >> >> I originally added the xml data with no indexes. >> >> Gary >> >> >> From: jean-marc Mercier [jeanmarc.merc...@gmail.com] >> Sent: Friday, December 06, 2013 9:37 AM >> To: Huband, Gary W *HS >> Cc: basex-talk@mailman.uni-konstanz.de >> Subject: Re: [basex-talk] BaseX Evaluation for Big Data >> >> I meant I am NOT a baseX member ! sorry for the mistyping ! >> >> >> 2013/12/6 jean-marc Mercier > jeanmarc.merc...@gmail.com>> >> Hi Gary >> >> I am a BaseX team member, but have you sized up your JVM to run "Big >> Data" ? >> >> See the files BaseX/basex*.bat. I am currently using set VM=-Xmx6g >> >> Hope this helps >> >> Jean-Marc >> >> >> 2013/12/6 Huband, Gary W *HS > gwh...@hscmail.mcc.virginia.edu>> >> >> I'm evaluating BaseX to use in a project that involves 10s of TB of xml >> data. Our goal is to be able to return a small amount of data from a query >> within a few seconds. Is BaseX capable of handling this much data? >> >> My initial evaluation of 1GB of data met our requirements. But, when I >> evaluated BaseX using about 20GB of data I get an out of memory message >> when I run a query. >> >> I'm running Windows 7 Pro on a Core i7, 8GB memory, and using the BaseX >> GUI. >> >> Thanks for your time >> >> Gary >> >> ___ >> BaseX-Talk mailing list >> BaseX-Talk@mailman.uni-konstanz.de> BaseX-Talk@mailman.uni-konstanz.de> >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> >> >> >> > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] BaseX Evaluation for Big Data
Hi Gary, Maybe you should wait for BaseX team members advises over this topic. Telling from my own experience, I can remember also having met some problem trying to insert 20 Gb of data into BaseX through the GUI. I had to achieve this into smaller steps, using batch commands (see http://docs.basex.org/wiki/Commands), in order to free memory. i.e. I first inserted the xml into 4 or 5 passes. Then created some indexes, etc...In that way it was possible to run the insertion, even with 4Gb machines. Hope this helps Jean-Marc 2013/12/6 Huband, Gary W *HS > I still get an out of main memory when I try to create attribute indexes. > > I originally added the xml data with no indexes. > > Gary > > ____ > From: jean-marc Mercier [jeanmarc.merc...@gmail.com] > Sent: Friday, December 06, 2013 9:37 AM > To: Huband, Gary W *HS > Cc: basex-talk@mailman.uni-konstanz.de > Subject: Re: [basex-talk] BaseX Evaluation for Big Data > > I meant I am NOT a baseX member ! sorry for the mistyping ! > > > 2013/12/6 jean-marc Mercier jeanmarc.merc...@gmail.com>> > Hi Gary > > I am a BaseX team member, but have you sized up your JVM to run "Big Data" > ? > > See the files BaseX/basex*.bat. I am currently using set VM=-Xmx6g > > Hope this helps > > Jean-Marc > > > 2013/12/6 Huband, Gary W *HS gwh...@hscmail.mcc.virginia.edu>> > > I'm evaluating BaseX to use in a project that involves 10s of TB of xml > data. Our goal is to be able to return a small amount of data from a query > within a few seconds. Is BaseX capable of handling this much data? > > My initial evaluation of 1GB of data met our requirements. But, when I > evaluated BaseX using about 20GB of data I get an out of memory message > when I run a query. > > I'm running Windows 7 Pro on a Core i7, 8GB memory, and using the BaseX > GUI. > > Thanks for your time > > Gary > > ___ > BaseX-Talk mailing list > BaseX-Talk@mailman.uni-konstanz.de BaseX-Talk@mailman.uni-konstanz.de> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > > > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] map:module. Behavior of maps of nodes
Christian, Hi. To end this topic, you should find enclosed two files that fulfills now my requirements. I found these objects very practical, their purpose are to handle set or maps of nodes / functions. I am publishing them here very informally, hoping other BaseX users could find them as helpful as I do, or maybe yourself, or a standardization committee. - The first one, idmap.xqm, allows to use maps, with nodes or functions as keys. - The second one, set.xqm, allows to define set of items (I use such an object to define a stack). These files consists in small overload of the BaseX map module. Both require an external id function of atomic type, that must be provided by the user. Below are the profile tests for these maps and set. The profile function is located into idmap:profile($i,$j) to be run by others. Basically, the "idmap" seems to be five time slower than BaseX map for inserting atomic types. Set seems to be quite comparable to BaseX map. This overload come from perf issues in XQUERY, but I can handle this small overload for my purposes. Obviously, for atomic type, it is better to use directly the map module, and these objects embed a basic protection to avoid a misusing. Some remarks : 1) It means that users are linked to BaseX map module when they are using these objects. As you already know, I tried unsuccessfully to write them in pure XQUERY, one of the motivation was not to be linked an external module. 2) Using a map module to define a set is not really the best way to proceed, but it seems that XQUERY 3.0 users don't have any other choice. 3) I think that BaseX could provide a simplified access to such objects. Indeed, the BaseX interpreter must have a default id for nodes or functions that it could expose to users, or use it directly through the BaseX map module. Hope that helps Cheers, Jean-Marc Tag name/Text nb integer_set node_set integer_map Basex_integer_map node_map test 1024 24.63 ms 43.36 ms 51.6 ms 6.35 ms 70.92 ms test 2048 10.91 ms 46.71 ms 17.52 ms 3.57 ms 51.62 ms test 4096 1.71 ms 31.52 ms 11.7 ms 1.04 ms 23.59 ms test 8192 2.49 ms 22.88 ms 15.41 ms 1.72 ms 32.5 ms test 16384 5.12 ms 20.23 ms 33.47 ms 2.9 ms 50.62 ms test 32768 8.31 ms 31.54 ms 58.95 ms 5.77 ms 86.86 ms test 65536 20.1 ms 71.39 ms 99.57 ms 13.0 ms 162.09 ms test 131072 48.15 ms 157.27 ms 680.47 ms 33.68 ms 947.9 ms test 262144 99.67 ms 305.88 ms 1132.31 ms 77.85 ms 683.37 ms test 524288 239.43 ms 569.46 ms 1769.53 ms 383.77 ms 2903.24 ms test 1.05E+06 737.79 ms 2485.75 ms 5022.02 ms 1031.71 ms 10204.07 ms 2013/12/1 jean-marc Mercier > Christian, > > Concerning this topic, it took me a whole week of quite hard work to write > a Red / Black removable tree written in XQUERY, together with a map > interface that mimic the basex one. However I won't go public, and would > advise not to use it, because it suffers from performance issues. > > I opened a thread concerning this perf issue, that might be deeper that I > thought. > > Cheers, > > Jean-Marc > > > > 2013/11/21 Christian Grün > >> >>Sounds interesting. Have you thought about making it public to get >> >> more feedback? >> > I will, as soon as it will be tested... I need to implement a >> > remove($map,$key) function, meaning that I have to re-implement the >> whole >> > BRTree...This will take some time (2 days works I guess to have a clean >> > implementation), to find into my agenda :) >> >> That would be fast anyway… I’m looking forward to the result! >> Christian >> >> >> >>It’s an implementation of Phil Bagwell’s immutable hash tries (as >> >>mentioned in your previous thread…): >> > Oops..sorry, my linker is slow sometime ;) Thanks for your patience >> > >> > >> > 2013/11/21 Christian Grün >> >> >> >> > 1) I implemented in pure XQUERY 3.0 a map having the desired >> behavior. I >> >> > can >> >> > now define optionally an ordering function, and thus is able to >> insert >> >> > nodes, maps, sequences. It is very useful for a lot of things. I >> would >> >> > not >> >> > recommend to external person to use it at present time (not tested, >> >> > functionalities to add, bugs suspected), but I can provide the file >> on >> >> > demand. >> >> >> >> Sounds interesting. Have you thought about making it public to get >> >> more feedback? >> >> >> >> > 2) A question about the BaseX map module : what kind of tree >> technology >> >> > is >> >> > used behind ? >> >> >> >> It’s an implementation of Phil Bagwell’s immutable hash tries (as >> >> mentioned in your previous thread…): >> >> >> >> >> >> >> https://github.com/BaseXdb/basex/tree/master/basex-core/src/main/java/org/basex/query/value/map >> >> >> >> Christian >> > >> > >> > > idmap.xqm Description: Binary data set.xqm Description: Binary data ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] BaseX Evaluation for Big Data
I meant I am NOT a baseX member ! sorry for the mistyping ! 2013/12/6 jean-marc Mercier > Hi Gary > > I am a BaseX team member, but have you sized up your JVM to run "Big Data" > ? > > See the files BaseX/basex*.bat. I am currently using set VM=-Xmx6g > > Hope this helps > > Jean-Marc > > > 2013/12/6 Huband, Gary W *HS > > I'm evaluating BaseX to use in a project that involves 10s of TB of xml >> data. Our goal is to be able to return a small amount of data from a query >> within a few seconds. Is BaseX capable of handling this much data? >> >> My initial evaluation of 1GB of data met our requirements. But, when I >> evaluated BaseX using about 20GB of data I get an out of memory message >> when I run a query. >> >> I'm running Windows 7 Pro on a Core i7, 8GB memory, and using the BaseX >> GUI. >> >> Thanks for your time >> >> Gary >> >> ___ >> BaseX-Talk mailing list >> BaseX-Talk@mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> > > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] BaseX Evaluation for Big Data
Hi Gary I am a BaseX team member, but have you sized up your JVM to run "Big Data" ? See the files BaseX/basex*.bat. I am currently using set VM=-Xmx6g Hope this helps Jean-Marc 2013/12/6 Huband, Gary W *HS > I'm evaluating BaseX to use in a project that involves 10s of TB of xml > data. Our goal is to be able to return a small amount of data from a query > within a few seconds. Is BaseX capable of handling this much data? > > My initial evaluation of 1GB of data met our requirements. But, when I > evaluated BaseX using about 20GB of data I get an out of memory message > when I run a query. > > I'm running Windows 7 Pro on a Core i7, 8GB memory, and using the BaseX > GUI. > > Thanks for your time > > Gary > > ___ > BaseX-Talk mailing list > BaseX-Talk@mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] some questions over the profile module
Christian, Hello. I was trying to profile code through a client / server as you advised. Thus I installed the module into the basex repository, and launched it through a client. However I can't find the log left by fn:trace, or prof:time. Do you know where these traces append in this configuration ? Thx Jean-Marc 2013/12/5 jean-marc Mercier > Hi Christian, > > Thx for your answer. > > it is advisable to remove all trace() calls from the code before doing > performance comparisons. > > You're right. From another side, the trace functions are called very few > times (68 calls), compared to the time execution (about two minutes to run > the entire test). This should be of small impact over the results. > > A remark over this test : I noticed that the test is running slower when I > disabled the options INLINELIMIT = 0 and TAILCALL = -1. > > > > 2013/12/5 Christian Grün > >> Hi Jean-Marc, >> >> another note (I just spotted your mail on t...@xquery.com; thanks for >> making your code public!): it is advisable to remove all trace() calls >> from the code before doing performance comparisons. >> >> Looking forward to your experiences, >> Christian >> ___ >> >> On Thu, Dec 5, 2013 at 11:39 AM, Christian Grün >> wrote: >> > Hi Jean-Marc, >> > >> >>>Currently no. You can use the prof:current-ns() >> >> I thought about this one, but isn't this method less accurate than >> prof:time >> > >> > If your measurements don’t fall below milliseconds, it should make no >> > difference. >> > >> > Benchmarking is a complex field on its own, however: >> > >> > • In BaseX, it’s advisable to do all measurements with a running >> > server instance. Otherwise, your measurement will also include the >> > starup time for initializing the JVM and doing just-in-time >> > compilations. >> > >> > • If you use the GUI, those effects will also be amortized after a >> > while, but the visual processing of the query results may take some >> > additional time. >> > >> > • You can measure the number of multiple runs via the RUNS option [1]. >> > The following call on command line runs the query "1" for 10,000 >> > times: >> > >> >> basex -o result.txt -r 1 -V 1 >> > ... >> > Parsing: 0.06 ms (avg) >> > Compiling: 0.0 ms (avg) >> > Evaluating: 0.0 ms (avg) >> > Printing: 0.03 ms (avg) >> > Total Time: 0.1 ms (avg) >> > ... >> > Query executed in 0.1 ms (avg). >> > >> > Please note that only the results of the first run will be output, so >> > this option should only be used when the output is small. As you have >> > probably seen, the output can also be “swallowed” via prof:dump(). >> > Some little experiments will soon give you a feeling on what’s >> > comparable and what’s not. >> > >> > If you want to compare the timing with other XQuery implementations, >> > it gets even more complicated, because each engine has its own timing >> > output, optimizations to speed up several runs, etc. In this case, a >> > simple call on command line gives you results that may still be best >> > comparable. >> > >> > Hope this helps, >> > Christian >> > >> > [1] http://docs.basex.org/wiki/Options#RUNS >> > > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] inspect functions and %private
Christian,
Hello.
I just tested the new snapshot. There might still be a small issue there
(maybe there's nothing you can do and we have to careful), since I can now
execute %private functions :
I have a module where there is a %private function.
module namespace common = 'http://www.example.com/common';
declare %private function common:test(){"test"};
Then I switch to a my main project and tries
for $fun in inspect:functions() where fn:data(inspect:function($fun)/@name)
= "common:test" return $fun()
and it returns "test"
Cheers,
Jean-Marc
2013/12/5 Christian Grün
> Hi Jean-Marc,
>
> thanks for the report. Leo has just fixed the issue. Feel free to
> check out the new snapshot [1].
>
> Christian
>
> [1] http://files.basex.org/releases/latest/
> _______
>
> On Mon, Dec 2, 2013 at 1:18 PM, jean-marc Mercier
> wrote:
> > Hi christian, yes this is the point :
> >
> > Stopped at C:/Jiheme/informatique/workspace/Graph
> DataBase/module/common.xq,
> > 31/89:
> > [XPST0017] Function 'local:test' is not visible from this module.
> >
> > If this is desired, it means that we can not use the inspection module as
> > soon as there is %private annotation in any imported module.
> >
> > Maybe an alternative behavior for the inspection module could be to
> ignore
> > %private functions ?
> >
> >
> >
> > 2013/12/2 Christian Grün
> >>
> >> > However, If the function test is located in another module, say test,
> >> > then
> >> > the following code
> >> >
> >> > import module namespace test="http://www.example.com/test"; at
> test.xq";
> >> >
> >> > for $fun in inspect:functions() return "gotcha"
> >> >
> >> > raise an exception.
> >>
> >> If you mean the exception "Function 'test:test' is not visible from
> >> this module.", that’s how it should be. If you mean another exception,
> >> could you please provide us with an SSCCE?
> >>
> >> Christian
> >
> >
>
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] some questions over the profile module
Hi Christian, Thx for your answer. > it is advisable to remove all trace() calls from the code before doing performance comparisons. You're right. From another side, the trace functions are called very few times (68 calls), compared to the time execution (about two minutes to run the entire test). This should be of small impact over the results. A remark over this test : I noticed that the test is running slower when I disabled the options INLINELIMIT = 0 and TAILCALL = -1. 2013/12/5 Christian Grün > Hi Jean-Marc, > > another note (I just spotted your mail on t...@xquery.com; thanks for > making your code public!): it is advisable to remove all trace() calls > from the code before doing performance comparisons. > > Looking forward to your experiences, > Christian > ___ > > On Thu, Dec 5, 2013 at 11:39 AM, Christian Grün > wrote: > > Hi Jean-Marc, > > > >>>Currently no. You can use the prof:current-ns() > >> I thought about this one, but isn't this method less accurate than > prof:time > > > > If your measurements don’t fall below milliseconds, it should make no > > difference. > > > > Benchmarking is a complex field on its own, however: > > > > • In BaseX, it’s advisable to do all measurements with a running > > server instance. Otherwise, your measurement will also include the > > starup time for initializing the JVM and doing just-in-time > > compilations. > > > > • If you use the GUI, those effects will also be amortized after a > > while, but the visual processing of the query results may take some > > additional time. > > > > • You can measure the number of multiple runs via the RUNS option [1]. > > The following call on command line runs the query "1" for 10,000 > > times: > > > >> basex -o result.txt -r 1 -V 1 > > ... > > Parsing: 0.06 ms (avg) > > Compiling: 0.0 ms (avg) > > Evaluating: 0.0 ms (avg) > > Printing: 0.03 ms (avg) > > Total Time: 0.1 ms (avg) > > ... > > Query executed in 0.1 ms (avg). > > > > Please note that only the results of the first run will be output, so > > this option should only be used when the output is small. As you have > > probably seen, the output can also be “swallowed” via prof:dump(). > > Some little experiments will soon give you a feeling on what’s > > comparable and what’s not. > > > > If you want to compare the timing with other XQuery implementations, > > it gets even more complicated, because each engine has its own timing > > output, optimizations to speed up several runs, etc. In this case, a > > simple call on command line gives you results that may still be best > > comparable. > > > > Hope this helps, > > Christian > > > > [1] http://docs.basex.org/wiki/Options#RUNS > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] some questions over the profile module
Christian,
thx for your answers.
> the InfoView will give you some hints what’s going on
oh...I should have thought about it. I still don't have this reflex. thanx
>Currently no. You can use the prof:current-ns()
I thought about this one, but isn't this method less accurate than prof:time
?
2013/12/4 Christian Grün
> Hi Jean-Marc,
>
> > 1) The following code
> > declare function local:sequence($nb){for $i in 1 to xs:integer($nb)
> return
> > $i};
> > prof:time(fn:count(local:sequence(100)))
> > return 4.94 ms.
>
> the InfoView will give you some hints what’s going on. The reason is
> that your code is compiled and simplified to the following query..
>
>prof:time(fn:count((1 to 100)))
>
> ...which can then be evaluated in constant time.
>
>
> > 2) It seems that you somehow call the fn:trace function in the profile
> > module. Is there a way to catch the returned values of these functions?
>
> Currently no. You can use the prof:current-ns() function instead:
>
> let $start := prof:current-ns()
> let $void := prof:void((1 to 1000)[. = 1])
> let $end := prof:current-ns()
> let $ms := ($end - $start) div 100
> return $ms || ' ms'
>
> Hope this helps,
> Christian
>
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
[basex-talk] some questions over the profile module
Hello Christian,
I have some small questions over the profile module.
1) The following code
declare function local:sequence($nb){for $i in 1 to xs:integer($nb) return
$i};
prof:time(fn:count(local:sequence(100)))
return 4.94 ms.
It means that you lazy evaluate the query "{for $i in 1 to xs:integer($nb)
return $i" ?
(I recall that you already mentioned this point that but I would like a
confirmation, since it might be a profiler issue)
2) It seems that you somehow call the fn:trace function in the profile
module. Is there a way to catch the returned values of these functions ?
The motivation is for profiling purposes. To be more specific, I would like
to treat the output results as
{prof:time( "xquery" ) }
to be able to aggregate results at a higher level of call, i.e. something
like
for $output in $prof//output
let $fun := data($output/@function)
group by $fun
return element-name{$fun}{sum(data($output) ) }
Thanks
Jean-Marc
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] inspect functions and %private
Hi christian, yes this is the point : Stopped at C:/Jiheme/informatique/workspace/Graph DataBase/module/common.xq, 31/89: [XPST0017] Function 'local:test' is not visible from this module. If this is desired, it means that we can not use the inspection module as soon as there is %private annotation in any imported module. Maybe an alternative behavior for the inspection module could be to ignore %private functions ? 2013/12/2 Christian Grün > > However, If the function test is located in another module, say test, > then > > the following code > > > > import module namespace test="http://www.example.com/test"; at test.xq"; > > > > for $fun in inspect:functions() return "gotcha" > > > > raise an exception. > > If you mean the exception "Function 'test:test' is not visible from > this module.", that’s how it should be. If you mean another exception, > could you please provide us with an SSCCE? > > Christian > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] Decoupling custom code with restxq
Hi Sebastian,
A quick comment for your code : it is better to use it sparsely, because
inspect:functions is quite slow.
If you want to improve perfs, create first a map. Here is the code that I
am using for this kind of "overloading"
declare variable $common:MapIntrospecting := common:map_inspect_functions();
declare function common:name-function($fun as
function(*)){fn:data(inspect:function($fun)/@name)};
declare function common:map_inspect_functions() { map:new(for $fun in
inspect:functions() return map:entry(common:name-function($fun),$fun)) };
(this one has to be modified accordingly to your %dispatch annotation, i.e.
add where exists(inspect:function($f)[annotation[@name='dispatch:
default'][literal[1]=$id]]))
and the dispatcher becomes simply
declare function common:dispatcher(nodes as node()*,$fun as *function(*)*)
as item()* {
let $disptchedfun := map:get($common:MapIntrospecting,
*common:name-function($fun)*) return
if (empty($disptchedfun)) then $fun($nodes) else
$disptchedfun($nodes) )
};
>By „map of functions“ did you mean what "inspection:functions()“ is doing?
No, I am talking about a map that can take any item a keys, and any item as
values. For instance, in the code above, I would prefer to write the
following dispatcher
declare function common:dispatcher(nodes as node()*,$fun as *function(*)*)
as item()* {
let $disptchedfun := map:get(common:MapIntrospecting(),$fun)
return
if (empty($disptchedfun)) then $fun($nodes) else
$disptchedfun($nodes) )
};
Its is simply a question of performance : in the first mechanism, you have
first to "serialize" the function as a string (calling the
inspect:function), then to look to a map(string, function), to finally get
the correct function call. In the second you directly access to the
function, avoiding an expensive call (inspect:function).
However, most probably this enhancement would be of less impact in your
code, if you don't call this dispatcher trillions of time.
>I don’t get it: how would you be able to overload the query eval function?
exactly as above. It is something a little bit more elaborate than the
following code
declare function common:evil_eval($nodes as node()*,$expr as xs:string) as
item()* {
try {
let $fun := map:get($common:MapIntrospecting,$expr) return
if (empty($fun)) then xquery:eval('$xml'||$expr,map{'$xml' :=
$nodes})
else $fun($nodes)
}
catch *("ouch")
};
Hope this helps
Cheers,
Jean-Marc
2013/12/2 Christian Grün
> Hi Sebastian,
>
> > BTW: I unfortunately missed your and Alexanders presentation at the
> MarkUp
> > Forum: are there any slides available?
>
> The slides should soon be available here:
> http://www.markupforum.de/archiv.html
>
> > This is similar to imported modules in XSLT: If you import template
> rules,
> > which are more specific,
> > they will be applied first. Could this be applied to XQuery as well?
> (But in
> > XSLT you would have to
> > import the custom code and hence change your code. We can do that in
> XQuery
> > as well, but it
> > is not completely decoupled, is it?)
>
> As Jean-Marc pointed out, the Inspection Module may help out. The
> module is fairly new, so we are interested in your (and everyone’s)
> feedback if you would like to have some functions added..
>
> Christian
> ___
> BaseX-Talk mailing list
> BaseX-Talk@mailman.uni-konstanz.de
> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
>
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
[basex-talk] inspect functions and %private
Hi,
To report a small issue in the inspect functions module with %private
declaration.
The following code runs perfectly :
declare %private function local:test(){()};
for $fun in inspect:functions() return "gotcha"
However, If the function test is located in another module, say test, then
the following code
import module namespace test="http://www.example.com/test"; at test.xq";
for $fun in inspect:functions() return "gotcha"
raise an exception.
Cheers,
Jean-Marc
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] map:module. Behavior of maps of nodes
Christian, Concerning this topic, it took me a whole week of quite hard work to write a Red / Black removable tree written in XQUERY, together with a map interface that mimic the basex one. However I won't go public, and would advise not to use it, because it suffers from performance issues. I opened a thread concerning this perf issue, that might be deeper that I thought. Cheers, Jean-Marc 2013/11/21 Christian Grün > >>Sounds interesting. Have you thought about making it public to get > >> more feedback? > > I will, as soon as it will be tested... I need to implement a > > remove($map,$key) function, meaning that I have to re-implement the whole > > BRTree...This will take some time (2 days works I guess to have a clean > > implementation), to find into my agenda :) > > That would be fast anyway… I’m looking forward to the result! > Christian > > > >>It’s an implementation of Phil Bagwell’s immutable hash tries (as > >>mentioned in your previous thread…): > > Oops..sorry, my linker is slow sometime ;) Thanks for your patience > > > > > > 2013/11/21 Christian Grün > >> > >> > 1) I implemented in pure XQUERY 3.0 a map having the desired > behavior. I > >> > can > >> > now define optionally an ordering function, and thus is able to insert > >> > nodes, maps, sequences. It is very useful for a lot of things. I would > >> > not > >> > recommend to external person to use it at present time (not tested, > >> > functionalities to add, bugs suspected), but I can provide the file on > >> > demand. > >> > >> Sounds interesting. Have you thought about making it public to get > >> more feedback? > >> > >> > 2) A question about the BaseX map module : what kind of tree > technology > >> > is > >> > used behind ? > >> > >> It’s an implementation of Phil Bagwell’s immutable hash tries (as > >> mentioned in your previous thread…): > >> > >> > >> > https://github.com/BaseXdb/basex/tree/master/basex-core/src/main/java/org/basex/query/value/map > >> > >> Christian > > > > > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] Perf issue
Leo,
Hi.
Indeed, I misunderstood your last e-mail. The problem seems deeper, as you
stated : it is the concatenation of sequences, that actually consists in
in-memory copying operations. Thus it implies that at present time, the
complexity of creating any sequences with XQUERY is quadratic in sequence
length. Indeed, to illustrate the problem, the following simple query
fn:fold-left(for $i in 1 to $nb return $i,(),
function($map,$entry){($map,$entry )})
takes on my PC (after your yesterday update) 10 sec with $nb = 10, 40
sec with $nb = 200 000, 3 min with $nb = 400 000, and 15 min for 1 000 000
entries. As a comparison, the map module have a linear complexity, and
takes less than a second to create a map of 1 000 000 entries. However, I
can't use this module for nodes or functions...
Maybe the real issue is that XQUERY does not provide any container for
linked lists ?
I can't see any work around here now, apart perhaps trying JAVA embedding
to create sequences, together with travestying the map module, that accept
nodes and functions as values. Any cleaner suggestions ?
2013/11/30 jean-marc Mercier
> Leo,
>
> Thx for your answer and the release. I'll check the release, but even 13 s
> to produce a sequence of 10 integer is a shot-down for my purposes.
>
> I think that there are two quick work-around however, waiting for super
> clojure sequences.
>
> - The first one is on my side, but I don't like it : I must bound my code
> to handle sequence of functors, and can't work with functor of functor.
> This might be a serious limitation to what I wanted to do with XQUERY.
>
> - Another suggestion, that might be quick and efficient, would be to add a
> direct optimized access to the functor
>
> id($el) as function() as item()* {function(){ $el }}
>
> into a BaseX module. For instance it could be pertinent to place it as
> hof:id-functor, since you already introduced hof:id. I think that there are
> two points of view here :
>
> 1) From the user point of view, this would greatly improve the range of
> algorithmic possibilities. Moreover, it is already possible to produce
> functor of functor of Thus we remain XQUERY coherent.
>
> 2) From the BaseX point of view, I don't know... Such a functor would
> become "de facto" a super type for XQUERY type hierarchy, since every
> XQUERY types could be casted into that functor (it would be above item,
> because it contains empty-sequences).
>
>
> Cheers
>
> Jean-Marc
>
>
>
>
>
>
>
>
>
> 2013/11/30 Leo Wörteler
>
>> Dear Jean-Marc,
>>
>> Am 30.11.2013 15:29, schrieb jean-marc Mercier:
>>
>> I am encountering a performance issue with BaseX interpreter,
>>> illustrated by the snippet code above. This code first creates a
>>> sequence of 10 integers, taking more than a minute with my
>>> environment. Then it creates a BaseX map of 10 integers, taking 0.1
>>> sec. This issue seems to be due to a poor performance of the operator ()
>>> (see function local:id below).
>>>
>>
>> one part of the problem is indeed sequence concatenation, which at the
>> moment has costs linear in the length of the result if the sequence really
>> has to be materialized in memory, e.g. as a function result. We are
>> thinking about switching to another representation of sequences (e.g.
>> finger trees or chunked sequences as used in Clojure [1]), but don't hold
>> your breath as that will probably take some time.
>>
>> The other (and bigger) cause of the slowdown was that there were type
>> checks for `item()*` introduced and not eliminated by the optimizer. As
>> everything in XQuery is a sequence of items, these are no-ops by
>> definition, but traversing and checking 10 items takes time.
>>
>> I fixed the latter problem and Christian already uploaded a new snapshot
>> [2]. With that, the execution time for your query drops from 57.9 to 12.4
>> seconds on my notebook.
>>
>> Hope that helps,
>> Leo
>>
>> [1] http://code.google.com/p/clojure/source/browse/trunk/
>> src/jvm/clojure/lang/PersistentVector.java
>> [2] http://files.basex.org/releases/latest/
>>
>
>
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] Decoupling custom code with restxq
Hi Sebastian,
On this problem, a quick comment coming into my mind, since I faced a
comparable problem recently. Without changing XQUERY or BaseX, it is
possible to emulate overloading using the introspection module of BaseX.
But you need a map which keys and values are functions (not possible with
BaseX map module, but possible with XQUERY).
With such a map, keys are computed using the BaseX introspection module for
your prod functions, since values are computed using the BaseX
introspection module for your custom functions.
All calls are then concentrated through a dispatcher : something like
call($function,$argument) { if contains($MapOfFunctions, $function) then
map:get($MapOfFunctions,$function)(argument) else $function($argument) }.
I tested this trick very recently actually, to overload the xquery:eval
BaseX function. I named the resulting function evil_eval (a very dangerous
function) !
Hope this help
Cheers
Jean-Marc
2013/11/30 Wiemer, Sebastian
> Hi Christian,
>
> Hi Sebastian,
>
> thanks for your e-mail; I’m pleased to see you are still working with
> BaseX.
>
> And we still love BaseX :-)
> Thanks for keeping up the good work!
>
> BTW: I unfortunately missed your and Alexanders presentation at the
> MarkUp Forum: are there any slides available?
>
>
>
> So just by dropping a file like „custom.xqm" in the restxq path, we can
> customize the system.
>
>
> …sounds like a nice idea.
>
> The drawback is that we can only use this for HTTPS requests from the
> client, but not from the server, because we use HTTPS. So we cannot use
> RESTXQ requests server side with http:send-request. (Or do we?)
>
>
> I guess I have no answer to that, as I haven’t actually come across
> this challenge yet. This might be completely off the track, but maybe
> it’s also a question of how Jetty is configured? Did you already check
> out the jetty.xml file and possible settings?
>
>
> Yes, I checked jetty configuration (but I gave up), and it seems to me,
> that jetty.xml or web.xml might not be the right place for a solution after
> all.
>
> What Florian and I probably are trying to accomplish here is a way to
> introduce aspect oriented or rule based
> programming to XQuery to decouple custom extensions from product code.
>
> This is similar to imported modules in XSLT: If you import template
> rules, which are more specific,
> they will be applied first. Could this be applied to XQuery as well? (But
> in XSLT you would have to
> import the custom code and hence change your code. We can do that in
> XQuery as well, but it
> is not completely decoupled, is it?)
>
> Florian and I came around with an idea, that does not use HTTP requests
> and maybe clarify
> our goal: how about introducing a „dispatch" annotation to XQuery?
>
> Let’s assume we have a function „age“ that takes an age as integer and a
> name as string and prints a message.
> A customization would be to be more specific for teenagers. But we don’t
> want to change the code.
> the annotation „dispatch:declare“ declares a function to be potentially
> dispatched: if there is not a more specific
> function with a „dispatch:replace“ annotation, than that function is
> called.
> Maybe "dispatch:prepend" and "dispatch:append“ would be nice as well.
> Also it would come in very handy if the replacing function is allowed be
> updating or not updating.
>
> product.xqm
> %dispatch:declare(age($age,$name))
> declare function prod:age($age as xs:integer,$name as xs:string){
> $name||" is "||$age||" years old."
> };
>
> prod:age(14,'Emily’) = „Emily is 14 years old."
>
> custom.xqm
> %dispatch:replace(age(19>$age>13,$name))
> declare function custom:age($age as xs:integer,$name as xs:string){
>$name||" is a "||$age||“ years old teenager."
> };
>
>
> So a use of „prod:age“ would yield the result of „custom:age“:
> *prod*:age(14,'Emily’) = „Emily is a 14 years old *teenager*.“
>
>
> Comments on this idea are very appreciated.
>
> Sebastian
>
>
> Christian
>
>
>
> ___
> BaseX-Talk mailing list
> BaseX-Talk@mailman.uni-konstanz.de
> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
>
>
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] Perf issue
Leo,
Thx for your answer and the release. I'll check the release, but even 13 s
to produce a sequence of 10 integer is a shot-down for my purposes.
I think that there are two quick work-around however, waiting for super
clojure sequences.
- The first one is on my side, but I don't like it : I must bound my code
to handle sequence of functors, and can't work with functor of functor.
This might be a serious limitation to what I wanted to do with XQUERY.
- Another suggestion, that might be quick and efficient, would be to add a
direct optimized access to the functor
id($el) as function() as item()* {function(){ $el }}
into a BaseX module. For instance it could be pertinent to place it as
hof:id-functor, since you already introduced hof:id. I think that there are
two points of view here :
1) From the user point of view, this would greatly improve the range of
algorithmic possibilities. Moreover, it is already possible to produce
functor of functor of Thus we remain XQUERY coherent.
2) From the BaseX point of view, I don't know... Such a functor would
become "de facto" a super type for XQUERY type hierarchy, since every
XQUERY types could be casted into that functor (it would be above item,
because it contains empty-sequences).
Cheers
Jean-Marc
2013/11/30 Leo Wörteler
> Dear Jean-Marc,
>
> Am 30.11.2013 15:29, schrieb jean-marc Mercier:
>
> I am encountering a performance issue with BaseX interpreter,
>> illustrated by the snippet code above. This code first creates a
>> sequence of 10 integers, taking more than a minute with my
>> environment. Then it creates a BaseX map of 10 integers, taking 0.1
>> sec. This issue seems to be due to a poor performance of the operator ()
>> (see function local:id below).
>>
>
> one part of the problem is indeed sequence concatenation, which at the
> moment has costs linear in the length of the result if the sequence really
> has to be materialized in memory, e.g. as a function result. We are
> thinking about switching to another representation of sequences (e.g.
> finger trees or chunked sequences as used in Clojure [1]), but don't hold
> your breath as that will probably take some time.
>
> The other (and bigger) cause of the slowdown was that there were type
> checks for `item()*` introduced and not eliminated by the optimizer. As
> everything in XQuery is a sequence of items, these are no-ops by
> definition, but traversing and checking 10 items takes time.
>
> I fixed the latter problem and Christian already uploaded a new snapshot
> [2]. With that, the execution time for your query drops from 57.9 to 12.4
> seconds on my notebook.
>
> Hope that helps,
> Leo
>
> [1] http://code.google.com/p/clojure/source/browse/trunk/
> src/jvm/clojure/lang/PersistentVector.java
> [2] http://files.basex.org/releases/latest/
>
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
[basex-talk] Perf issue
Hello,
I am encountering a performance issue with BaseX interpreter, illustrated
by the snippet code above. This code first creates a sequence of 10
integers, taking more than a minute with my environment. Then it creates a
BaseX map of 10 integers, taking 0.1 sec. This issue seems to be due to
a poor performance of the operator () (see function local:id below).
This could be an issue for the BaseX interpreter, or a problem with my
environment, but might be also a XQUERY 3.0 "leak". Have you any idea ?
cheers
Jean-Marc
declare function local:id($el) as function() as item()*
{function(){ $el }};
declare function local:new()
{local:id(())};
declare function local:new($map,$item)
{local:id(($map(),$item))};
let $nb := 10 return
let $testid := fn:fold-left(for $i in 1 to $nb return $i,local:new(),
function($map,$entry){local:new( $map,$entry )})
let $basexmap := fn:fold-left(for $i in 1 to $nb return
map:entry($i,$i),map:entry(0,0), function($entry,$map){map:new(
($map,$entry) )})
return (fn:count(prof:time( $testid )),fn:count(prof:time( $basexmap )))
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
[basex-talk] preprocessor instructions for BaseX XQUERY interpreters ?
Hello,
I have some questions about BaseX interpreter.
1) Is there a way to send preprocessor instructions to BaseX interpreter ?
I am looking to a way to tell the interpreter to skip code based on
external instructions, mainly to distinguish the behavior between debug /
release modes.
This is achieved in C++ with the macro set of instructions #define, #if,
#endif etc... I was wondering whether there exists such a mechanism in
XQUERY.
Note : processor instructions might not be relevant for XQuery, since
interpreters are heavily optimized.
2) Could you confirm that BaseX interpreter will not try to interpret
"dead" code ?
To emulate processor instructions with XQUERY, I am presently using dynamic
local variable declarations as (declare function debug(){true()/false()}),
and encapsulating my code with a lot of "if (debug()) then (.1.) else
(.2.)". Can you confirm that BaseX interpreter skip looking at (.1.) or
(.2.) accordingly to the debug function return value ?
Cheers
Jean-Marc
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] BaseX Interpreter / Eclipse integration issue ?
Hi Leo, sorry for the delayed answer. I was looking to something smart to say over this topic, but nothing came :) Indeed, I don't have tested yet patterns like expression template to make any pertinent remark there. > you should probably also set `TAILCALLS` to -1 Thx for pointing this out. >This also means that unconditionally inlining recursive functions can send the compiler into an infinite loop. For C++ (the only interpreter I know a little), this is left under the user responsibility. When such infinite loop occurs, then the user payback its freedom spending a lot of time to blindly understand why its interpreter rose an exception. I don't know whether it is a good idea or not. > This is conceptually simpler, but would bloat the syntax tree and mostly (?) not help much. See remark above. In a first phase, having inlined version of fn:fold-left (maybe naming it as hof:fold-left would be safer ?) should be enough, and safer, since you should know in advance the call depth. Shall (INLINE = ?) control the depth call ? I did not have a look to the code yet, but I was wondering how can you detect recursive calls ? The self recursive case seems easy to detect, but if I glue two functions together, it seems trickier. Cheers Jean-Marc 2013/11/24 Leo Wörteler > Jean-Marc, > > Am 23.11.2013 17:52, schrieb jean-marc Mercier: > > A drawback is that the stack trace is inefficient, hardening the >> debugging. Thus it is better to set INLINE = 0 in dev phase. >> > > yes; you should probably also set `TAILCALLS` to -1 if you want the stack > trace to always be accurate. As BaseX performs tail-call optimization [1], > the stack will otherwise never contain more than a fixed number (currently > 256) of tail-call frames. > > > Could you be kind enough to point me out the link to your code at Basex >> repository ? >> > > Sure, function inlining is triggered in StaticFuncCall [2] and DynFuncCall > [3]. > > > Motivation : I would like to try evaluating whether inlining recursive >> functions is possible or not. You are welcome to suggestions ! >> > > The hard part about inlining recursive functions is figuring out when to > stop. As they have a call to themselves in the function body, each time you > inline them at least one new call is inserted. This also means that > unconditionally inlining recursive functions can send the compiler into an > infinite loop. > > Inlining is indeed possible for all functions for which we can guarantee > that the number of replacements will be finite. I'm currently implementing > this for `fn:fold-left`, `fn:fold-right` and `fn:for-each` (basically loop > unrolling) when the sequence is short and known at compile time. It would > definitely be possible to implement this for user-defined functions, too, > but we'd need to identify (a subset of) the correct ones. > > Another option would be to always inline recursive functions up to a fixed > depth. This is conceptually simpler, but would bloat the syntax tree and > mostly (?) not help much. > > What are your ideas? > Leo > > [1] http://en.wikipedia.org/wiki/Tail_call > [2] https://github.com/BaseXdb/basex/blob/next/basex-core/ > src/main/java/org/basex/query/func/StaticFuncCall.java#L71 > [3] https://github.com/BaseXdb/basex/blob/next/basex-core/ > src/main/java/org/basex/query/func/DynFuncCall.java#L57 > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] BaseX Interpreter / Eclipse integration issue ?
Leo, I have a remark and a question about inlining. - 1) this inlining mechanism is good, no contest about it. A drawback is that the stack trace is inefficient, hardening the debugging. Thus it is better to set INLINE = 0 in dev phase. 2) Could you be kind enough to point me out the link to your code at Basex repository ? Motivation : I would like to try evaluating whether inlining recursive functions is possible or not. You are welcome to suggestions ! Cheers 2013/11/22 jean-marc Mercier > Leo, Christian > > >I fixed the bug, thank you for reporting it. You can get the current > snapshot here: http://files.basex.org/releases/latest/ > I just tested it with my lib code, it is now working fine, thx. You guys > are reactive ! > > >That only sets the setting for the current GUI session, not permanently. > Ok thx, this explains why I couldn't make it works with my environment. > > > BaseX inlines all functions (static ones and function items) etc.. > That's really interesting. Very close to template programming as > expression templating, but without having to write a single line of code. > That could be of great utility if I try to write an algebra module. > > Cheers, > > Jean-Marc > > > > > > 2013/11/22 Leo Wörteler > >> Dear Jean-Marc, >> >> I fixed the bug, thank you for reporting it. You can get the current >> snapshot here: http://files.basex.org/releases/latest/ >> >> Am 22.11.2013 15:06, schrieb jean-marc Mercier: >> >> Thx very much for your workaround. I did not succeed using the GUI and >>> executing »SET INLINELIMIT 0«. >>> >> >> That only sets the setting for the current GUI session, not permanently. >> >> >> - Edit the file .basex, adding the line "INLINELIMIT = 0" after #Local >>> Options. I would prefer this method, would you have no objection. >>> >> >> It should be fine, but globally disables all inlining of functions. So >> just remember to remove it after you have a fixed version of BaseX. >> >> >> I am not sure to understand well this parameter. Does it means that it >>> is possible to inline function in XQUERY ? This is quite close to >>> template mechanism programming ! >>> >> >> BaseX inlines all functions (static ones and function items) that are >> * not recursive (or, for function items, don't contain themself), and >> * below a certain size, which can be changed through the `INLINELIIT`. >> >> This means that introducing abstraction by encapsulating code in many >> small functions should have little to no runtime overhead. Together with >> higher-order functions, this makes quite general and abstract libraries >> feasible. >> >> Cheers, Leo >> > > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] BaseX Interpreter / Eclipse integration issue ?
Leo, Christian >I fixed the bug, thank you for reporting it. You can get the current snapshot here: http://files.basex.org/releases/latest/ I just tested it with my lib code, it is now working fine, thx. You guys are reactive ! >That only sets the setting for the current GUI session, not permanently. Ok thx, this explains why I couldn't make it works with my environment. > BaseX inlines all functions (static ones and function items) etc.. That's really interesting. Very close to template programming as expression templating, but without having to write a single line of code. That could be of great utility if I try to write an algebra module. Cheers, Jean-Marc 2013/11/22 Leo Wörteler > Dear Jean-Marc, > > I fixed the bug, thank you for reporting it. You can get the current > snapshot here: http://files.basex.org/releases/latest/ > > Am 22.11.2013 15:06, schrieb jean-marc Mercier: > > Thx very much for your workaround. I did not succeed using the GUI and >> executing »SET INLINELIMIT 0«. >> > > That only sets the setting for the current GUI session, not permanently. > > > - Edit the file .basex, adding the line "INLINELIMIT = 0" after #Local >> Options. I would prefer this method, would you have no objection. >> > > It should be fine, but globally disables all inlining of functions. So > just remember to remove it after you have a fixed version of BaseX. > > > I am not sure to understand well this parameter. Does it means that it >> is possible to inline function in XQUERY ? This is quite close to >> template mechanism programming ! >> > > BaseX inlines all functions (static ones and function items) that are > * not recursive (or, for function items, don't contain themself), and > * below a certain size, which can be changed through the `INLINELIIT`. > > This means that introducing abstraction by encapsulating code in many > small functions should have little to no runtime overhead. Together with > higher-order functions, this makes quite general and abstract libraries > feasible. > > Cheers, Leo > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] BaseX Interpreter / Eclipse integration issue ?
Christian,
Thx very much for your workaround. I did not succeed using the GUI and
executing »SET INLINELIMIT 0«. However, it seems that there exists several
way to set this parameter without the GUI. I tested successfully the two
following methods:
- Set the parameter before the query as follow : (# db:INLINELIMIT 0 #)
{fn:count($insert())} However, it is somehow tedious to write all queries
that way.
- Edit the file .basex, adding the line "INLINELIMIT = 0" after #Local
Options. I would prefer this method, would you have no objection.
I am not sure to understand well this parameter. Does it means that it is
possible to inline function in XQUERY ? This is quite close to template
mechanism programming !
Cheers,
Jean-Marc
2013/11/22 Christian Grün
> Hi Jean-Marc,
>
> the bug you encountered is due to a newly feature (called “function
> inlining”) we’ve introduced just recently. It will be fixed soon [2].
> Until then, you can simply disable it by setting INLINELIMIT to 0 [1]
> This can e.g. be done by choosing »Command« in the dropdown box of the
> GUI main window, and executing the command »SET INLINELIMIT 0«.
>
> Thanks for the reproducible code,
> Christian
>
> [1] http://docs.basex.org/wiki/Options#INLINELIMIT
> [2] https://github.com/BaseXdb/basex/issues/796
> _______
>
> On Fri, Nov 22, 2013 at 10:28 AM, jean-marc Mercier
> wrote:
> > Hi,
> >
> > Within my environment, I am experiencing some frustrating behavior of the
> > Xquery Basex engine. I've done my best to single out the problem in the
> > following XQUERY code :
> >
> > declare function local:wtf($x as item() ) {function() { $x}};
> > declare function local:dummy($f,$x) {
> > if(fn:empty($f())) then local:wtf($x) else local:wtf(())};
> >
> > declare variable $insert := local:dummy(function(){ () }, function() {
> (),
> > ()} );
> > fn:count($insert())
> >
> > The issues are :
> >
> > 1) executing this code yields an "Improper use? Potential bug?" exception
> > (see details further on).
> > 2) If I change the execution query to be
> > fn:count(local:dummy(function(){ () }, function() { (), ()} )()) ( I just
> > copy and pasted the value of $insert into the fn:count(..) ) then the
> query
> > is executed normally, returning "1".
> > 3) If I change the function declaration local:wtf as follow
> > declare function local:wtf($x) {function() { $x}}; (I just removed "as
> > item()"), then the query is executed normally, returning "1".
> >
> > Can you reproduce it with another environment ?
> >
> > Cheers,
> >
> > Jean-Marc
> >
> >
> > Improper use? Potential bug? Your feedback is welcome:
> > Contact: basex-talk@mailman.uni-konstanz.de
> > Version: BaseX 7.8 beta 6aeaebf
> > Java: Oracle Corporation, 1.7.0_05
> > OS: Windows 7, amd64
> > Stack Trace:
> > java.lang.NullPointerException
> > at org.basex.query.value.item.FuncItem.inlineExpr(FuncItem.java:308)
> > at org.basex.query.func.DynFuncCall.optimize(DynFuncCall.java:58)
> > at org.basex.query.func.DynFuncCall.compile(DynFuncCall.java:39)
> > at org.basex.query.expr.Arr.compile(Arr.java:40)
> > at org.basex.query.func.StandardFunc.compile(StandardFunc.java:60)
> > at org.basex.query.MainModule.compile(MainModule.java:63)
> > at org.basex.query.QueryCompiler.compile(QueryCompiler.java:70)
> > at org.basex.query.QueryCompiler.compile(QueryCompiler.java:61)
> > at org.basex.query.QueryContext.analyze(QueryContext.java:270)
> > at org.basex.query.QueryContext.compile(QueryContext.java:253)
> > at org.basex.query.QueryProcessor.compile(QueryProcessor.java:71)
> > at org.basex.core.cmd.AQuery.query(AQuery.java:81)
> > at org.basex.core.cmd.XQuery.run(XQuery.java:22)
> > at org.basex.core.Command.run(Command.java:323)
> > at org.basex.core.Command.execute(Command.java:92)
> > at org.basex.server.LocalSession.execute(LocalSession.java:121)
> > at org.basex.server.Session.execute(Session.java:37)
> > at org.basex.core.Main.execute(Main.java:146)
> > at org.basex.BaseX.(BaseX.java:119)
> > at org.basex.BaseX.main(BaseX.java:38)
> >
> >
> >
> > ___
> > BaseX-Talk mailing list
> > BaseX-Talk@mailman.uni-konstanz.de
> > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
> >
>
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] search into basex talk archive ?
Hi Christian, The mail archive is just fine for me. Thx for pointing me out his link. Cheers, Jean-Marc 2013/11/21 Christian Grün > Hi Jean-Marc, > > > is there a (convenient) way to search into basex talk archives ? > > Web search engines like DuckDuckGo, Bing and... (I’ve forgotten the > others ;) are probably the best solution for browsing all mailing list > contents. All posting are also archived in the Mail Archive [1,2]. > > Some time ago, we’ve made our list searchable via BaseX and its full > text capabilities. I guess that’s something we should revive. > > Christian > > [1] http://basex.org/open-source/ > [2] > http://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/info.html > > > > > At present time, I downloaded some of the monthly archive to look at, > but it > > is a little bit tedious... > > > > Cheers > > > > ___ > > BaseX-Talk mailing list > > BaseX-Talk@mailman.uni-konstanz.de > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
[basex-talk] BaseX Interpreter / Eclipse integration issue ?
Hi,
Within my environment, I am experiencing some frustrating behavior of the
Xquery Basex engine. I've done my best to single out the problem in the
following XQUERY code :
declare function local:wtf($x as item() ) {function() { $x}};
declare function local:dummy($f,$x) {
if(fn:empty($f())) then local:wtf($x) else local:wtf(())};
declare variable $insert := local:dummy(function(){ () }, function() { (),
()} );
fn:count($insert())
The issues are :
1) executing this code yields an "Improper use? Potential bug?" exception
(see details further on).
2) If I change the execution query to be
fn:count(local:dummy(function(){ () }, function() { (), ()} )()) ( I just
copy and pasted the value of $insert into the fn:count(..) ) then the query
is executed normally, returning "1".
3) If I change the function declaration local:wtf as follow
declare function local:wtf($x) {function() { $x}}; (I just removed "as
item()"), then the query is executed normally, returning "1".
Can you reproduce it with another environment ?
Cheers,
Jean-Marc
Improper use? Potential bug? Your feedback is welcome:
Contact: basex-talk@mailman.uni-konstanz.de
Version: BaseX 7.8 beta 6aeaebf
Java: Oracle Corporation, 1.7.0_05
OS: Windows 7, amd64
Stack Trace:
java.lang.NullPointerException
at org.basex.query.value.item.FuncItem.inlineExpr(FuncItem.java:308)
at org.basex.query.func.DynFuncCall.optimize(DynFuncCall.java:58)
at org.basex.query.func.DynFuncCall.compile(DynFuncCall.java:39)
at org.basex.query.expr.Arr.compile(Arr.java:40)
at org.basex.query.func.StandardFunc.compile(StandardFunc.java:60)
at org.basex.query.MainModule.compile(MainModule.java:63)
at org.basex.query.QueryCompiler.compile(QueryCompiler.java:70)
at org.basex.query.QueryCompiler.compile(QueryCompiler.java:61)
at org.basex.query.QueryContext.analyze(QueryContext.java:270)
at org.basex.query.QueryContext.compile(QueryContext.java:253)
at org.basex.query.QueryProcessor.compile(QueryProcessor.java:71)
at org.basex.core.cmd.AQuery.query(AQuery.java:81)
at org.basex.core.cmd.XQuery.run(XQuery.java:22)
at org.basex.core.Command.run(Command.java:323)
at org.basex.core.Command.execute(Command.java:92)
at org.basex.server.LocalSession.execute(LocalSession.java:121)
at org.basex.server.Session.execute(Session.java:37)
at org.basex.core.Main.execute(Main.java:146)
at org.basex.BaseX.(BaseX.java:119)
at org.basex.BaseX.main(BaseX.java:38)
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Re: [basex-talk] Linear Algebra module in XQUERY ?
>However, this doesn’t necessarily mean that XQuery is the wrong >language for a linear algebra. Even a raytracer has already been >implemented in XQuery (from the developers of XMLPrime; it seems to be >offline). Some other examples on what can be done with XQuery is found >in [1]. Indeed, I think that XQUERY might be very well suited to linear algebra (except for dynamic memory allocation). Most of the linear algebra algorithmic is of "functional" (in the sense recursive then evaluate expression) nature. I bet that a linear algebra module written in xquery could end up with 10% amount lines of code than its C++ or JAVA equivalent. > Direct overloading may be supported in a future version of XQuery. > Many things are happening; for example, arrays will be introduced with > XQuery 3.1. You are welcome to post feature requests in the W3 Bug > Tracker [2]. That's interesting. Specially if further version of xquery support constant-sized arrays, meaning that xquery can allocate contiguous block of memory (I will not have to bind JAVA anymore !). I was trying to look to a site listing the new features of xquery 3.1 but did not succeed. Could you be kind enough to provide a link if you have one in mind ? Thanks for yours answers, cheers, Jean-Marc 2013/11/18 Christian Grün > Hi again, > > > - I don't know how to allocate and address directly contiguous memory > blocks > > in XQUERY. > > Operations on this level cannot be realized in XQuery. Instead, the > query processor is responsible for all memory management. If > operations turn out to really be too cpu/memory consuming, Java > bindings (and annotations like @Deterministic) may be the better > choice. > > However, this doesn’t necessarily mean that XQuery is the wrong > language for a linear algebra. Even a raytracer has already been > implemented in XQuery (from the developers of XMLPrime; it seems to be > offline). Some other examples on what can be done with XQuery is found > in [1]. > > > - Immutability might result in a big overload in memory? This might not > be > > true, see a recent thread over this topic (XQUERY for noobs). > > That’s difficult to say, as memory consumption largely depends on the > specific problem, and the concrete implementation. > > > - I don't know how to overload the basic operators (*,+,-) with XQUERY. > This > > may result in quite heavy notations like > > alg:prod($mat1,alg:minus($mat2,$mat3)) instead of $mat1*($mat2-$mat3). > > Function implementations can be assigned to variables, which makes the > notation more compact: > > $prod($mat1, $minus($mat2, $mat3)) > > Direct overloading may be supported in a future version of XQuery. > Many things are happening; for example, arrays will be introduced with > XQuery 3.1. You are welcome to post feature requests in the W3 Bug > Tracker [2]. > > Christian > > [1] http://cs.uef.fi/~kilpelai/RDK11/exercises/Ex8Files/xqueryProblems.pdf > [2] https://www.w3.org/Bugs/Public/ > ___ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
