Hie Christian,
You're right : historically, the XQuery code was executed with the Saxon
engine. This is no longer possible without paying a license. In addition to
the cost generated, this limits the replicability of the processing. This is
why we are evaluating the BaseX solution.
I don't see how to profile XQuery code. I will carry out tests with a
database. I will also improve the syntax of some queries. I will keep you
informed of the results.
Thanks a lot,
Antonio
De : Christian Grün <christian.gr...@gmail.com>
Envoyé : lundi 22 avril 2024 13:45
À : ANDRADE Antonio <antonio.andr...@ofb.gouv.fr>
Cc : basex-talk@mailman.uni-konstanz.de
Objet : Re: [basex-talk] Performance issue with BaseX CLI
Hi again,
I had a quick look into the monitoring code, and I noticed two things:
1. It looks to me (correct me if I’m wrong) as if the code of the project
was initially written for Saxon and then ported to BaseX. If you are
interested in using BaseX, you could focus on the slow functions, try
alternative writings and (if you want to run the code with both processors
in the future) ensure that Saxon still gives delivers good performance.
2. Some functions can be noticeably sped up (for both BaseX and Saxon) if
you use XQuery 3.1 features such as maps or group by. For example, the
runtime of #131014 could possibly be reduced with something similar to…
for $ms in $Monitoring/*:MonitoringSite
let $emsc := $ms/*:euMonitoringSiteCode
for $ceqm in $ms/*:ChemicalEcologicalQuantitativeMonitoring
let $V_rech := $ceqm/*:parameterCode || '/' || $ceqm/*:parameterOther ||
'/' || $ceqm/*:chemicalMatrix
group by $group := $emsc || ': ' || $V_rech
where count($ceqm) > 1
return $V_rech
If BaseX turns out to be the way to go, it’s definitely worth taking
advantage of the database aspect. In BaseX, databases are fairly
light-weight, which means you can simply create them before running the
queries (e.g., with a single 'CREATE DB poc
/path/to/poc_rapportage_controle-main/xml' command) and use db:get('poc',
'your-doc.xml') in the queries to access a document (or even stick with
doc('your-doc.xml') if you enable DEFAULTDB [1]).
Hope this helps,
Christian
[1]
<https://antiphishing.vadesecure.com/v4?f=TzBPM05TMWhaUkVuRncweoPMjK2QCEAycDsFPXW7oVXv7fvatzw4hMuVApRk99dY&i=NmhwUWdPbjljNWRxSlVxNUTscW_hVGAeTpWfW3ms-T8&k=Yhxs&r=bGlzcjBDTTd4VWcyWjZtQ73JEaKYtalSlfUg_UlIdniAsQJc8JIQqvObJohKyZTu&s=397bd47452d05252330ef5fc1fa5598015668b2d3656a078232680d7881de307&u=https%3A%2F%2Fdocs.basex.org%2Fwiki%2FOptions%23DEFAULTDB>
https://docs.basex.org/wiki/Options#DEFAULTDB
On Mon, Apr 22, 2024 at 9:32 AM Christian Grün <christian.gr...@gmail.com
<mailto:christian.gr...@gmail.com> > wrote:
Hi Antonio,
As Liam indicated, you may get better performance when adding your documents
to a database.
In general, though, the runtimes of BaseX and Saxon have aligned pretty much
over the years, and I assume there’ll be a trivial reason behind the drastic
difference in the runtime.
Your test setup is probably too complex for us readers to spend more time
with it. Could you possibly share a more basic example with us, ideally with
a single document and query file?
Thanks in advance,
Christian
On Mon, Apr 22, 2024 at 8:54 AM ANDRADE Antonio <antonio.andr...@ofb.gouv.fr
<mailto:antonio.andr...@ofb.gouv.fr> > wrote:
<mailto:l...@fromoldbooks.org> @Liam R. E. Quin : Thanks for your feedback.
The processing time is between 2 minutes and more than 11 hours (see table
below). Thus, the loading time of the Java virtual machine has little
impact. The main XQuery script loads the XML document once at the start of
processing. It is then requested several times as part of more or less
complex quality controls. At this moment, the XML document is not intended
to be stored. This is why it is not loaded into a database before
processing.
Saxon
BaseX
Start
Stop
Elapse time
Start
Stop
Elapse time
Check Monitoring 2022 FRH
06:16:54
06:19:30
00:02:36
06:44:06
10:05:21
03:21:15
Check Multi schéma 2022 FRH
06:25:46
06:31:47
00:06:01
10:05:55
11:39:07
01:33:12
De : Liam R. E. Quin <l...@fromoldbooks.org <mailto:l...@fromoldbooks.org> >
Envoyé : samedi 20 avril 2024 05:00
À : ANDRADE Antonio <antonio.andr...@ofb.gouv.fr
<mailto:antonio.andr...@ofb.gouv.fr> >; basex-talk@mailman.uni-konstanz.de
<mailto:basex-talk@mailman.uni-konstanz.de>
Objet : Re: [basex-talk] Performance issue with BaseX CLI
On Fri, 2024-04-19 at 10:45 +0200, ANDRADE Antonio wrote:
Hie,
For the purposes of European Water Framework Directive reporting, I compared
the performances of the Saxon and BaseX XQuery engines.
First, you should consider (as i think Martin said) the Java runtime startup
time, typically a second or so.
Second, BaseX is a database. If you will process the same document many
times, first load it into a database and then use the Python BaseX client.
This will avoid startup time, and, more importantly, will allow BaseX to
make use of database indexes.
If you will only process any given document once, then Saxon may well be the
appropriate tool.
liam
--
Liam Quin, https://www.delightfulcomputing.com/
<https://antiphishing.vadesecure.com/v4?f=SnpNUUNxek1BTWh6ZFZjaWCyrlumiLHtHmHGdEVdgTGAg0gyDE-v9PTNgKgfV2Nw&i=cHp0TkJvdm11bGhoR250SmgzWyo1rr-iN9AzEpeQLkU&k=6xq5&r=Z1RORVRCV0NEb2hhaDhMZNAVNIuDyvWRZH6WeNsm4siBbLteM10PATfmacXtXZrM&s=4c7cb6bdfca9fe7ddb2b3683dad19c0df1db6cbdc34171049937c0a5791ef479&u=https%3A%2F%2Fwww.delightfulcomputing.com%2F>
Available for XML/Document/Information Architecture/XSLT/
XSL/XQuery/Web/Text Processing/A11Y training, work & consulting.
Barefoot Web-slave, antique illustrations: http://www.fromoldbooks.org
<https://antiphishing.vadesecure.com/v4?f=SnpNUUNxek1BTWh6ZFZjaWCyrlumiLHtHmHGdEVdgTGAg0gyDE-v9PTNgKgfV2Nw&i=cHp0TkJvdm11bGhoR250SmgzWyo1rr-iN9AzEpeQLkU&k=6xq5&r=Z1RORVRCV0NEb2hhaDhMZNAVNIuDyvWRZH6WeNsm4siBbLteM10PATfmacXtXZrM&s=8296c1235680d7664e4428bb2543916368c7e78a7235acc5e660e575e227a9fd&u=http%3A%2F%2Fwww.fromoldbooks.org%2F>
