Re: [GRASS-dev] pyGRASS: ParallelModuleQueue
2016-11-30 13:59 GMT+01:00 Martin Landa : > Hi, > > 2016-11-30 13:29 GMT+01:00 Martin Landa : > > Wouldn't make sense to add something like _progress=True. Then module > > could call msgr.percent() automatically when it finish. > > > > queue.percent(5, count) # print each 5% > > module = Module(..., _run=False, _progress=5) > > for ... in ...: > > ... > > queue.wait() > > another possibility would be to define own function, eg. > > def process(i, n, count): > Module(..., _run=True, quiet=True) > msgr.percent(i, n, count) > Maybe you can implement a derivative from the Module class and implement the required behaviour there? > > Similar approach could be useful when for example applying mask and > doing some processing for each scene with multiple bands. > > But such function cannot be used together with ParallelModuleQueue. Or > am I wrong? Ma > I am not sure if i understand you correctly. But you can use the MultiModule[1] class to specify a stack of modules that should be run in serial order. Several MultiModule objects can be run in parallel by the ParallelModuleQueue. However, setting a mask in one module will affect all other modules that are running in parallel as well, since they are running in the same mapset. Changing the region in each stack is no problem, since the MultiModule class provides a dedicated region environment that does not affect other parallel running modules in the same mapset. Ciao Sören [1] https://grass.osgeo.org/grass73/manuals/libpython/_modules/pygrass/modules/interface/module.html#MultiModule > > -- > Martin Landa > http://geo.fsv.cvut.cz/gwiki/Landa > http://gismentors.cz/mentors/landa > ___ > grass-dev mailing list > grass-dev@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/grass-dev > ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] pyGRASS: ParallelModuleQueue
Hi, 2016-11-30 13:29 GMT+01:00 Martin Landa : > Wouldn't make sense to add something like _progress=True. Then module > could call msgr.percent() automatically when it finish. > > queue.percent(5, count) # print each 5% > module = Module(..., _run=False, _progress=5) > for ... in ...: > ... > queue.wait() another possibility would be to define own function, eg. def process(i, n, count): Module(..., _run=True, quiet=True) msgr.percent(i, n, count) Similar approach could be useful when for example applying mask and doing some processing for each scene with multiple bands. But such function cannot be used together with ParallelModuleQueue. Or am I wrong? Ma -- Martin Landa http://geo.fsv.cvut.cz/gwiki/Landa http://gismentors.cz/mentors/landa ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] pyGRASS: ParallelModuleQueue
Hi, 2016-11-30 13:00 GMT+01:00 Pietro : > Yes, I think so, if you pass the stderr_=subprocess.PIPE and > stdout_=subprocess.PIPE parameters to the module, then you should be > able to write the current percentage using grass.percent() or > msgr.percent(). yes, but then you get progress info for each module separately, right? Then you need to process stderr/stdout on your own... Wouldn't make sense to add something like _progress=True. Then module could call msgr.percent() automatically when it finish. queue.percent(5, count) # print each 5% module = Module(..., _run=False, _progress=5) for ... in ...: ... queue.wait() ? Ma -- Martin Landa http://geo.fsv.cvut.cz/gwiki/Landa http://gismentors.cz/mentors/landa ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] pyGRASS: ParallelModuleQueue
Hi Martin, On Wed, Nov 30, 2016 at 12:25 PM, Martin Landa wrote: > is it possible to use progress info when using ParallelModuleQueue? > Something similar to grass.percent(). Eg. Yes, I think so, if you pass the stderr_=subprocess.PIPE and stdout_=subprocess.PIPE parameters to the module, then you should be able to write the current percentage using grass.percent() or msgr.percent(). Pietro ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] pyGRASS: ParallelModuleQueue
Hi,
is it possible to use progress info when using ParallelModuleQueue?
Something similar to grass.percent(). Eg.
"""
grass.message("Clipping...")
for name in maps:
new_mapcalc =
copy.deepcopy(mapcalc_module)
queue.put(new_mapcalc(expression='{name}={name}'.format(name=name)))
queue.wait()
"""
->
Clipping
5% <- 5% of jobs done
10%
...
Thanks, Martin
--
Martin Landa
http://geo.fsv.cvut.cz/gwiki/Landa
http://gismentors.cz/mentors/landa
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
