Hi Bridger,
GNU parallel has --jobs but each BaseX process is then free to spawn
as many processes as it likes/can. The code is simple:
find . -name *tok01.xml | parallel --progress --jobs 10
../basex114/bin/basex -bfile={} myscript.xq
Best,
Giuseppe
Quoting Bridger Dyson-Smith <[email protected]>:
Giuseppe -
Interesting! I'm not sure about the parallelized node operations, but I
also confess that I'm not sure if you're wanting to keep all of your cores
busy, or if you're wanting to restrain BaseX from using too many cores. If
you're leveraging GNU parallel to control a BaseX instance, I assume you're
using `-j|--jobs` to set a limit on the number of BaseXes.. BaseXen?
BaseXeses... :) launched.
Would you be able to share an example of how you're approaching your
problem? Someone may be able to give suggestions based on it.
Best,
Bridger
On Fri, Dec 6, 2024 at 8:01 AM Giuseppe G. A. Celano <
[email protected]> wrote:
Hi Bridger,
I am not using xquery:fork-join. If I understand correctly, some
operations on nodes are parallelized automatically by BaseX. If the
same script is then run in parallel on different files, I get all my
cores busy.
Best,
Giuseppe
Quoting Bridger Dyson-Smith <[email protected]>:
> Hi Giuseppe -
> Hope you're well. Did you try adding the `parallel` option to
> xquery:fork-join? See
> https://docs.basex.org/main/XQuery_Functions#xquery:fork-join - if not
> specified, then BaseX will be greedy :)
>
>
> Best,
>
> Bridger
>
> On Fri, Dec 6, 2024, 7:36 AM <[email protected]> wrote:
>
>> Is there a way to specify how many parallel processes Basex 11 is
>> allowed to spawn (I am not using the database)? In combination with
>> GNU parallel, it becomes difficult to have control on parallelism.
>>
>> Best,
>> Giuseppe
>>
>>
--
Universität Leipzig
Institute of Computer Science
Augustusplatz 10
04109 Leipzig
Deutschland
[email protected]
--
Universität Leipzig
Institute of Computer Science
Augustusplatz 10
04109 Leipzig
Deutschland
[email protected]
