Re: [GRASS-dev] r.in.gdal and xargs

2017-03-07 Thread Blumentrath, Stefan
...@lists.osgeo.org] On Behalf Of Blumentrath, Stefan Sent: mandag 6. mars 2017 15.57 To: Vincent Bain <b...@toraval.fr> Cc: GRASS developers list (grass-dev@lists.osgeo.org) <grass-dev@lists.osgeo.org> Subject: Re: [GRASS-dev] r.in.gdal and xargs I can parallelize on a higher level, so i can set

Re: [GRASS-dev] r.in.gdal and xargs

2017-03-06 Thread Blumentrath, Stefan
Thanks Vincent for your swift reply. In principle the pipe to xargs works, as 99% of the data is imported properly. And also in the case were I get roors, the command is started properly... Thus, I suspect that r.in.gdal can have issues when run in parallel; or the creation of temp files, in

Re: [GRASS-dev] r.in.gdal and xargs

2017-03-06 Thread Vincent Bain
Yes, perhaps something to do with r.in.gdal temp files handling. And what if you try to reduce P value ? well, of course it will slow down the bulk import... Le lundi 06 mars 2017 à 14:02 +, Blumentrath, Stefan a écrit : > Thanks Vincent for your swift reply. > > In principle the pipe to

Re: [GRASS-dev] r.in.gdal and xargs

2017-03-06 Thread Vincent Bain
Hello Stefan, just a suggestion, inspired from how I usually use xargs: did you try to put the arguments list in a file (say my_ringdal_args.txt), then run : xargs -a my_ringdal_args.txt -P10 -n 3 r.in.gdal (-n 3 indicating xargs to read 3 arguments for each call to r.in.gdal, i.e. input=

[GRASS-dev] r.in.gdal and xargs

2017-03-06 Thread Blumentrath, Stefan
Dear all, I am trying to import time series data using a combination of xargs an r.in gdal: cat current_datasets_age.txt | awk -v U="myunits" -v N="name" '{ print "r.in.gdal input=$1 ".bil output=" $2 "_tmp title=\"" N " in " U " at " $3 "\" --o --q -o\0"}' | xargs -P 10 -I {} -0 bash -c {}