date:20130925

Re: [basex-talk] Set Operator Examples

2013-09-25 Thread Arve Gengelbach

> I had a typo in my example. The return did not call the map2 version. With 
> it, the results were (now on 7.7 and Windows 7)

And I posted the wrong version. Usage of ! within map:new() is not as
fast as a flowr expression (in general).

BUT neither of the “fastest two” functions yields valid results
(e.g. take 2,2,3 for $a and empty sequence for $b)
Arto, I hope replacing one flowr expression with the simple map operator
can improve speed in your use-cases. So hopefully this is fastest for you:

declare function local:difference-maps($a, $b) {
  let $m1 := map:new(for $i in $a return map:entry($i, true()))
  let $m2 := map:new(for $i in $b return map:entry($i, false()))
  let $m3 := map:new(($m1, $m2))
  return map:keys($m3) ! (if ($m3(.)) then . else ())
};

As the speed gain depends on the spreading of the data, rewriting set
operations needs proove of efficency for any given data.
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

Re: [basex-talk] Set Operator Examples

2013-09-25 Thread Arto Viitanen

I had a typo in my example. The return did not call the map2 version. With it, 
the results were (now on 7.7 and Windows 7)

- pred 58278.25 ms
- map 272.68 ms
- map2 74.27 ms
- map2! 92.97 ms
- flowr 43018.3 ms

Hej Arto,

One can even improve your map2 by using the [Simple map operator].

declare function local:difference-map-2-excl($a, $b) {
  let $m2 := map:new($b ! map:entry(., true()))
  return $a ! (if($m2(.)) then () else .) };

- map 299.47 ms
- map2 290.96 ms
- map2 + ! 128.25 ms

..
Arto
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

Re: [basex-talk] Set Operator Examples

2013-09-25 Thread Andy Bunce

Hi,
Nice work. I wonder if the BaseX query optimizer could not spot the
idioms for the set operations and rewrite the query to something
similar for the cases:

distinct-values(($arg1, $arg2))
distinct-values($arg1[.=$arg2])
distinct-values($arg1[not(.=$arg2)])


>From : http://www.xqueryfunctions.com/xq/c0015.html#c0075

Regards
/Andy

On Wed, Sep 25, 2013 at 8:04 PM, Arve Gengelbach  wrote:
> Hej Arto,
>
> One can even improve your map2 by using the [Simple map operator].
>
> declare function local:difference-map-2-excl($a, $b) {
>   let $m2 := map:new($b ! map:entry(., true()))
>   return $a ! (if($m2(.)) then () else .)
> };
>
> - map 299.47 ms
> - map2 290.96 ms
> - map2 + ! 128.25 ms
>
> cheers
> Arve
>
> [Simple map operator] http://www.w3.org/TR/xquery-30/#id-map-operator
>
> Am 25.09.13 10:41 schrieb Arto Viitanen:
>>
>>
>> -Original Message-
>> From: Arve Gengelbach [mailto:a...@basex.org]
>> Sent: 25. syyskuuta 2013 9:08
>> To: Arto Viitanen
>> Cc: basex-talk@mailman.uni-konstanz.de
>> Subject: Re: [basex-talk] Set Operator Examples
>>
>> Hej Arto,
>>
>> just being interested:
>> How does it compare to a FLOWR expression?
>>
>> declare function local:difference-flowr($a, $b) {
>>   for $x in $a
>>   where not($x = $b)
>>   group by $x
>>   return $x[1]
>> };
>>
>> pred 51787.06 ms
>> map 485.16 ms
>> map2 325.09 ms
>> flowr 45275.97 ms
>>
>> --
>> Arto Viitanen
>> Finland
>>
> ___
> BaseX-Talk mailing list
> BaseX-Talk@mailman.uni-konstanz.de
> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

Re: [basex-talk] Set Operator Examples

2013-09-25 Thread Arve Gengelbach

Hej Arto,

One can even improve your map2 by using the [Simple map operator].

declare function local:difference-map-2-excl($a, $b) {
  let $m2 := map:new($b ! map:entry(., true()))
  return $a ! (if($m2(.)) then () else .)
};

- map 299.47 ms
- map2 290.96 ms
- map2 + ! 128.25 ms

cheers
Arve

[Simple map operator] http://www.w3.org/TR/xquery-30/#id-map-operator

Am 25.09.13 10:41 schrieb Arto Viitanen:
> 
> 
> -Original Message-
> From: Arve Gengelbach [mailto:a...@basex.org] 
> Sent: 25. syyskuuta 2013 9:08
> To: Arto Viitanen
> Cc: basex-talk@mailman.uni-konstanz.de
> Subject: Re: [basex-talk] Set Operator Examples
> 
> Hej Arto,
> 
> just being interested:
> How does it compare to a FLOWR expression?
> 
> declare function local:difference-flowr($a, $b) {
>   for $x in $a
>   where not($x = $b)
>   group by $x
>   return $x[1]
> };
> 
> pred 51787.06 ms
> map 485.16 ms
> map2 325.09 ms
> flowr 45275.97 ms
> 
> --
> Arto Viitanen
> Finland
> 
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

Re: [basex-talk] BaseX http running single threaded?

2013-09-25 Thread Joe Templeman

I found that it is locking up because of the database global
lock which
locks during any updating function. On the long updating call that was
happening first, it was grabbing the lock and so every subsequent request
was being blocked until the update had finished. I think the solution for
us is to turn it off since we're never updating any content in place, only
adding content, and that content is static so it doesn't matter if two
requests clobber the data.




On Wed, Sep 25, 2013 at 9:30 AM, France Baril
wrote:

> I'm having similar issues where everything freezes on an http request done
> from inside a restxq function, but only if certain conditions are present.
>
> The conditions are not easy to pinpoint. However, they affect our ability
> to validate content against modular dtds/xsds, to apply modular .xsl  and
> .xsl that were grabbing pieces of content with doc(restxq...) accesses, and
> to create PDF with a FOP module because the fo references images stored as
> raw files and only accessible by http://..rest/ paths.
>
> I'm in the 3rd week of refactoring our application so we can migrate to
> 7.7. I believe I am half way there. The main task is to augment content
> through query before the document is sent to .xsl to remove all needs for
> the use of the doc(restxq) functions. We also find that we can no longer
> have a self-contained application and that we have to create new steps to
>  export pieces of content and applicative files to the file system.
>
> Not having a self-contained application is a huge step back for us where
> we have to start considering that different OS handle different paths
> differently. For example, Windows is killing our capitalization on content
> re-import which multiplies the folder structures (looks like we'll need to
> naming conventions).
>
> I'm hoping someone can provide a case simple enough to be submitted and
> that will help pinpoint the exact source of the problem and resolve the
> issue. Our refactoring for doc(restxq) will be done by there, but not
> having to export applicative files and images for processing would bring us
> back to a neat/clean self-contained application.
>
> Regards,
>
> France
>
>
>
> On Fri, Sep 20, 2013 at 6:13 PM, Joe Templeman  wrote:
>
>> Hi all,
>>
>> I'm having trouble with the running a restxq API, set up as either a
>> stand alone app (using basexhttp) or running the servlet in jetty.
>>
>> When I make a request, the server will lazily load in the requested
>> content using a small java library that I've built which can take quite a
>> while, and unfortunately it completely blocks waiting for this request to
>> complete. This means that if I make a request for content which is already
>> loaded, which should return in a couple of milliseconds, it instead waits
>> for the downloading of the first request to complete before returning.
>>
>> I'm sure it's something simple that I'm missing in the config somewhere,
>> but I have the threadpool parameters set up in my jetty.xml config file but
>> it doesn't seem to make any difference.
>>
>> Is this expected behavior? I can't imagine any HTTP server would ever be
>> designed to be single threaded...
>>
>> Any help is welcome, I've been trying to figure this out for days now.
>>
>> Thanks,
>> Joe
>>
>>
>> ___
>> BaseX-Talk mailing list
>> BaseX-Talk@mailman.uni-konstanz.de
>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
>>
>>
>
>
> --
> France Baril
> Architecte documentaire / Documentation architect
> france.ba...@architextus.com
> (514) 572-0341
>
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

Re: [basex-talk] BaseX http running single threaded?

2013-09-25 Thread France Baril

I'm having similar issues where everything freezes on an http request done
from inside a restxq function, but only if certain conditions are present.

The conditions are not easy to pinpoint. However, they affect our ability
to validate content against modular dtds/xsds, to apply modular .xsl  and
.xsl that were grabbing pieces of content with doc(restxq...) accesses, and
to create PDF with a FOP module because the fo references images stored as
raw files and only accessible by http://..rest/ paths.

I'm in the 3rd week of refactoring our application so we can migrate to
7.7. I believe I am half way there. The main task is to augment content
through query before the document is sent to .xsl to remove all needs for
the use of the doc(restxq) functions. We also find that we can no longer
have a self-contained application and that we have to create new steps to
 export pieces of content and applicative files to the file system.

Not having a self-contained application is a huge step back for us where we
have to start considering that different OS handle different paths
differently. For example, Windows is killing our capitalization on content
re-import which multiplies the folder structures (looks like we'll need to
naming conventions).

I'm hoping someone can provide a case simple enough to be submitted and
that will help pinpoint the exact source of the problem and resolve the
issue. Our refactoring for doc(restxq) will be done by there, but not
having to export applicative files and images for processing would bring us
back to a neat/clean self-contained application.

Regards,

France

On Fri, Sep 20, 2013 at 6:13 PM, Joe Templeman  wrote:

> Hi all,
>
> I'm having trouble with the running a restxq API, set up as either a stand
> alone app (using basexhttp) or running the servlet in jetty.
>
> When I make a request, the server will lazily load in the requested
> content using a small java library that I've built which can take quite a
> while, and unfortunately it completely blocks waiting for this request to
> complete. This means that if I make a request for content which is already
> loaded, which should return in a couple of milliseconds, it instead waits
> for the downloading of the first request to complete before returning.
>
> I'm sure it's something simple that I'm missing in the config somewhere,
> but I have the threadpool parameters set up in my jetty.xml config file but
> it doesn't seem to make any difference.
>
> Is this expected behavior? I can't imagine any HTTP server would ever be
> designed to be single threaded...
>
> Any help is welcome, I've been trying to figure this out for days now.
>
> Thanks,
> Joe
>
>
> ___
> BaseX-Talk mailing list
> BaseX-Talk@mailman.uni-konstanz.de
> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
>
>

-- 
France Baril
Architecte documentaire / Documentation architect
france.ba...@architextus.com
(514) 572-0341
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

Re: [basex-talk] Issue with linebreak using REST since 7.7.1

2013-09-25 Thread Christian Grün

Dear Yoann,

I tried to reproduce your example, using the following query:

$session = new Session("localhost", 1984, "admin", "admin");
$xml = $session->execute("XQUERY ");
print $xml;
$session->execute("XQUERY insert node $xml into ");
$session->close();

This one runs without problems on my machine; can you run it, too?

If it fails, it could well be that you were running the code with an
older snapshot (I remember there was an issue with line breaks). You
could also try the very latest snapshot [2].

It it succeeds, it could well be that I’ve probably misinterpreted the
problem, and it would be great if you could provide us with a little
example that runs out of the box.

Thanks,
Christian

[1] http://files.basex.org/releases/latest/
___

2013/9/24 Yoann Maingon :
> Hi,
>
> I've got a simple node move PHP script (sorry for my terrible code, I'm an
> electronician!)
>
> // get the node
> $xquery = "XQUERY //noteitem[@id='$movedID']";
> $saveNode = $session->execute($xquery);
>
> // delete the node
> $xquery = "XQUERY delete node //noteitem[@id='$movedID']";
> $content = $session->execute($xquery);
> // insert the node
> if ($nextID == 'last') {
>   $xquery = "XQUERY insert node ($saveNode) as last into
> //meeting[@id='$meetingID']/items ";
>   $content = $session->execute($xquery);
> } else {
>   echo $saveNode;
>   $xquery = "XQUERY insert node ($saveNode) before
> //meeting[@id='$meetingID']/items/noteitem[@id='$nextID'] ";
>   $content = $session->execute($xquery);
> }
> $contentFull = "$content";
> echo $contentFull;
>
>
> If I look into the content that is saved, deleted and inserted I have the
> following:
>
>   id="MnNo524180f9587c1" genid="MnNo524180f9587c1" owner="1" type="question">
> ...
>   
>
> It worked fine until the latest update. Where apparently the query breaks
> with the line break.
> It creates an error saying that the closing  tag is missing.
> Just got back to 7.6 and it works well.
>
> any idea?
>
> Yoann Maingon
> CEO - mydatalinx
> +33664324966
>
> ___
> BaseX-Talk mailing list
> BaseX-Talk@mailman.uni-konstanz.de
> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
>
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

Re: [basex-talk] Set Operator Examples

2013-09-25 Thread Arto Viitanen



-Original Message-
From: Arve Gengelbach [mailto:a...@basex.org] 
Sent: 25. syyskuuta 2013 9:08
To: Arto Viitanen
Cc: basex-talk@mailman.uni-konstanz.de
Subject: Re: [basex-talk] Set Operator Examples

Hej Arto,

just being interested:
How does it compare to a FLOWR expression?

declare function local:difference-flowr($a, $b) {
  for $x in $a
  where not($x = $b)
  group by $x
  return $x[1]
};

pred 51787.06 ms
map 485.16 ms
map2 325.09 ms
flowr 45275.97 ms

--
Arto Viitanen
Finland
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

Re: [basex-talk] feature request -- GUI database creation overall progress

2013-09-25 Thread Christian Grün

> Any chance of getting, in addition to the per-file progress, the overall
> progress of creating the database?

...currently no, because we’re not counting all files before
retrieving them (this can take quite a while when e.g. a network
resource is addressed, or if a ZIP file from a remote URL is
retrieved).

Hope this helps,
Christian


>
> Thanks!
> Graydon
>
> ___
> BaseX-Talk mailing list
> BaseX-Talk@mailman.uni-konstanz.de
> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
>
___
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

Re: [basex-talk] Set Operator Examples

Re: [basex-talk] Set Operator Examples

Re: [basex-talk] Set Operator Examples

Re: [basex-talk] Set Operator Examples

Re: [basex-talk] BaseX http running single threaded?

Re: [basex-talk] BaseX http running single threaded?

Re: [basex-talk] Issue with linebreak using REST since 7.7.1

Re: [basex-talk] Set Operator Examples

Re: [basex-talk] feature request -- GUI database creation overall progress

9 matches

Site Navigation

Mail list logo

Footer information