Re: [basex-talk] Use of function-lookup in REST-XQ environment
Hi Mauro,
Mi dispiace di averti fatto aspettare.
Your test case was very helpful for reproducing the issue. The problem
was caused by the combination of mixupdates and function-lookup. I
hope that the issue is resolved with the latest stable snapshot [1]
(it’s time to give it a TLA… maybe LSS?).
Ciao,
Christian
[1] http://files.basex.org/releases/latest/
On Mon, Apr 27, 2020 at 1:04 PM Mauro Mugnaini wrote:
>
> Dear Christian,
> About this problem we discover another issue that is causing us some troubles.
> In the same scenario of calling RESTXQ updating functions discovered
> dynamically in a basex with MIXUPDATES turned on: if a reading function is
> called on the same DB while an updating function is storing some content in
> the DB the DB will be corrupted. We performed some tests and some times we
> get an OutOfMemoryError exception, some time an
> ArrayIndexOutOfBoundsException with the result that some “rubbish” data gets
> stored into the DB (e.g. standalone attributes).
> I’ll provide you a SSCE [1] and the inserting function [2] we used to confirm
> the problem. During the save it is sufficient to call the read via rest
> several times with [3] to cause the problem. We tested on both last release
> (adding the “updating” prefix before the looked up function) and latest
> nightly build.
>
> All my best,
>
> Mauro
>
> [1] test.xqm file
>
> module namespace _ = 'urn:test';
>
> declare
> %rest:path("save")
> %output:method("json")
> %output:media-type('application/json')
> %rest:POST("{$resource}")
> function _:create($resource as node()) {
> function-lookup(xs:QName("db:add"), 3)("test", $resource, random:uuid())
> };
>
> declare
> %rest:path("readall")
> %output:method("json")
> %output:media-type('application/json')
> %rest:GET
> function _:readall() {
> {
> for tumbling window $in-window in function-lookup(xs:QName("db:open"),
> 1)("test")
> start $first at $s when $s = 1
> end $last at $e when $e - $s = 10 - 1
> return $in-window
> }
> };
>
> [2]
> distinct-values(for $i in 1 to 1
> let $res :=
> http:send-request(
>
> , "http://localhost:8984/save;, '{
> "resourceType": "Encounter",
> "status" : "finished"
> }')
> return string($res[1]/@status))
>
> [3] http://localhost:8984/readall
>
> > Il giorno 31 mar 2020, alle ore 14:14, Christian Grün
> > ha scritto:
> >
> > Dear Mauro,
> >
> > The combination of higher-order functions and MIXUPDATE led to a wrong
> > assignment of static properties of your updating expression. I have
> > made the compilation step more rigid: If a dynamic function call is
> > found, and if mixing updates is enabled, the function will now always
> > be tagged as an updating function. Feel free to check out the latest
> > stable snapshot [1].
> >
> > If you want to stick with 9.3.2, you can prefix the function lookup
> > call with an additional "updating" keyword:
> >
> > updating fn:function-lookup(fn:QName('urn:ns:impl2', 'store'),
> > 1)($resource)
> >
> > Cari saluti all’ Italia; our hearts are with you,
> > Christian
> >
> > [1] http://files.basex.org/releases/latest/
Re: [basex-talk] Use of function-lookup in REST-XQ environment
Dear Christian,
About this problem we discover another issue that is causing us some troubles.
In the same scenario of calling RESTXQ updating functions discovered
dynamically in a basex with MIXUPDATES turned on: if a reading function is
called on the same DB while an updating function is storing some content in the
DB the DB will be corrupted. We performed some tests and some times we get an
OutOfMemoryError exception, some time an ArrayIndexOutOfBoundsException with
the result that some “rubbish” data gets stored into the DB (e.g. standalone
attributes).
I’ll provide you a SSCE [1] and the inserting function [2] we used to confirm
the problem. During the save it is sufficient to call the read via rest several
times with [3] to cause the problem. We tested on both last release (adding the
“updating” prefix before the looked up function) and latest nightly build.
All my best,
Mauro
[1] test.xqm file
module namespace _ = 'urn:test';
declare
%rest:path("save")
%output:method("json")
%output:media-type('application/json')
%rest:POST("{$resource}")
function _:create($resource as node()) {
function-lookup(xs:QName("db:add"), 3)("test", $resource, random:uuid())
};
declare
%rest:path("readall")
%output:method("json")
%output:media-type('application/json')
%rest:GET
function _:readall() {
{
for tumbling window $in-window in function-lookup(xs:QName("db:open"),
1)("test")
start $first at $s when $s = 1
end $last at $e when $e - $s = 10 - 1
return $in-window
}
};
[2]
distinct-values(for $i in 1 to 1
let $res :=
http:send-request(
, "http://localhost:8984/save;, '{
"resourceType": "Encounter",
"status" : "finished"
}')
return string($res[1]/@status))
[3] http://localhost:8984/readall
> Il giorno 31 mar 2020, alle ore 14:14, Christian Grün
> ha scritto:
>
> Dear Mauro,
>
> The combination of higher-order functions and MIXUPDATE led to a wrong
> assignment of static properties of your updating expression. I have
> made the compilation step more rigid: If a dynamic function call is
> found, and if mixing updates is enabled, the function will now always
> be tagged as an updating function. Feel free to check out the latest
> stable snapshot [1].
>
> If you want to stick with 9.3.2, you can prefix the function lookup
> call with an additional "updating" keyword:
>
> updating fn:function-lookup(fn:QName('urn:ns:impl2', 'store'), 1)($resource)
>
> Cari saluti all’ Italia; our hearts are with you,
> Christian
>
> [1] http://files.basex.org/releases/latest/
Re: [basex-talk] Use of function-lookup in REST-XQ environment
Dear Christian,
Thank you so much, it also worked very well in version 9.3.1 that is the one we
are using at the moment.
We plan to move to the next stable release when it will be available.
Thanks also for your words; all the best to all: to Italy and also to people
all over the world.
Mauro
> Il giorno 31 mar 2020, alle ore 14:14, Christian Grün
> ha scritto:
>
> Dear Mauro,
>
> The combination of higher-order functions and MIXUPDATE led to a wrong
> assignment of static properties of your updating expression. I have
> made the compilation step more rigid: If a dynamic function call is
> found, and if mixing updates is enabled, the function will now always
> be tagged as an updating function. Feel free to check out the latest
> stable snapshot [1].
>
> If you want to stick with 9.3.2, you can prefix the function lookup
> call with an additional "updating" keyword:
>
> updating fn:function-lookup(fn:QName('urn:ns:impl2', 'store'), 1)($resource)
>
> Cari saluti all’ Italia; our hearts are with you,
> Christian
>
> [1] http://files.basex.org/releases/latest/
>
>
>
>> Hi all,
>> In order to have a "dynamic backend” implementation to REST-XQ invocations
>> we used the fn:function-lookup() function but we are experiencing a strange
>> behavior if the target looked-up function is an updating function that for
>> example adds a resource to a database.
>>
>> Putting the provided files in the webapp folder of a fresh new BaseX latest
>> (9.3.2) downloaded instance (adding "MIXUPDATES = true” setting in the
>> .basex and with a database named DB already created inside) and invoking the
>> REST function that calls directly the backend function with [1] the resource
>> is correctly created on the database, by invoking the REST function that
>> performs the dynamic lookup by using the fn:function-lookup() with [2] it
>> completes without errors but the resource is not added to the DB.
>> In our implementation the implementation modules are stored in the repo but
>> the same behavior can be reproduced with “direct” function call as per my
>> example.
>>
>> Is there any limitation in the looked-up function related to updating
>> databases?
>>
>> Regards,
>>
>> Mauro
>>
>> [1]
>> http:send-request(
>>
>>
>> ,
>> "http://localhost:8984/store;,
>>
>> )
>> [2]
>> http:send-request(
>>
>>
>> ,
>> "http://localhost:8984/dyn-store;,
>>
>> )
>>
Re: [basex-talk] Use of function-lookup in REST-XQ environment
Dear Mauro,
The combination of higher-order functions and MIXUPDATE led to a wrong
assignment of static properties of your updating expression. I have
made the compilation step more rigid: If a dynamic function call is
found, and if mixing updates is enabled, the function will now always
be tagged as an updating function. Feel free to check out the latest
stable snapshot [1].
If you want to stick with 9.3.2, you can prefix the function lookup
call with an additional "updating" keyword:
updating fn:function-lookup(fn:QName('urn:ns:impl2', 'store'), 1)($resource)
Cari saluti all’ Italia; our hearts are with you,
Christian
[1] http://files.basex.org/releases/latest/
> Hi all,
> In order to have a "dynamic backend” implementation to REST-XQ invocations we
> used the fn:function-lookup() function but we are experiencing a strange
> behavior if the target looked-up function is an updating function that for
> example adds a resource to a database.
>
> Putting the provided files in the webapp folder of a fresh new BaseX latest
> (9.3.2) downloaded instance (adding "MIXUPDATES = true” setting in the .basex
> and with a database named DB already created inside) and invoking the REST
> function that calls directly the backend function with [1] the resource is
> correctly created on the database, by invoking the REST function that
> performs the dynamic lookup by using the fn:function-lookup() with [2] it
> completes without errors but the resource is not added to the DB.
> In our implementation the implementation modules are stored in the repo but
> the same behavior can be reproduced with “direct” function call as per my
> example.
>
> Is there any limitation in the looked-up function related to updating
> databases?
>
> Regards,
>
> Mauro
>
> [1]
> http:send-request(
>
>
> ,
> "http://localhost:8984/store;,
>
> )
> [2]
> http:send-request(
>
>
> ,
> "http://localhost:8984/dyn-store;,
>
> )
>
[basex-talk] Use of function-lookup in REST-XQ environment
Hi all,In order to have a "dynamic backend” implementation to REST-XQ invocations we used the fn:function-lookup() function but we are experiencing a strange behavior if the target looked-up function is an updating function that for example adds a resource to a database.Putting the provided files in the webapp folder of a fresh new BaseX latest (9.3.2) downloaded instance (adding "MIXUPDATES = true” setting in the .basex and with a database named DB already created inside) and invoking the REST function that calls directly the backend function with [1] the resource is correctly created on the database, by invoking the REST function that performs the dynamic lookup by using the fn:function-lookup() with [2] it completes without errors but the resource is not added to the DB.In our implementation the implementation modules are stored in the repo but the same behavior can be reproduced with “direct” function call as per my example.Is there any limitation in the looked-up function related to updating databases?Regards,Mauro[1]http:send-request( , "http://localhost:8984/store", )[2]http:send-request( , "http://localhost:8984/dyn-store", ) impl2.xqm Description: Binary data restxq.xqm Description: Binary data
