Re: [GRASS-dev] ParallelModuleQueue freezes...
Hi Anna, Thanks for clarification. Would be great if you could backport #1407. I don`t know if the PR depends on others... But if it is safe to backport just this one, I could do that too of course. In core, MultiModule is used just in t.rast.neighbors, it seems. And that module thus does not work in GRASS 7.8 right now. So the current state can be considered a bug... Would be great if it would be possible to get that into 7.8.6 somehow, even if there is already a RC. I am about to finish a new AddOn, where I would like to use the combination of MultiModule and ParallelModuleQueue. The development version of GRASS will not be available in the production environment in my institute for quite some time, so I would have to wait for some month to use the addon... Cheers, Stefan From: Anna Petrášová Sent: fredag 28. mai 2021 16:36 To: Stefan Blumentrath Cc: Luca Delucchi ; GRASS developers list Subject: Re: [GRASS-dev] ParallelModuleQueue freezes... On Fri, May 28, 2021 at 10:02 AM Stefan Blumentrath mailto:stefan.blumentr...@nina.no>> wrote: Some more context. The t.rast.neighbors test seems to work with the current dev version "2048e1016" from 2021-05-19T09:38:49+00:00 It fails however, with 7.8.5 from 2020-12-22T15:55:09+00:00 and an older development version "35edbf7dc" from 2020-06-03T16:03:52+00:00 So, maybe one or more of the recent changes need backporting? https://github.com/OSGeo/grass/commits/master/python/grass/pygrass/modules/interface/module.py<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fcommits%2Fmaster%2Fpython%2Fgrass%2Fpygrass%2Fmodules%2Finterface%2Fmodule.py=04%7C01%7C%7C8fb6cc6211854d8631f708d921e5fa6e%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637578093904701621%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000=HYHd7YUo5l9wQD%2Fdooe3zC%2BYCd%2BL%2B3WRMbX5OeCjI5U%3D=0> This one: https://github.com/OSGeo/grass/pull/1407<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fpull%2F1407=04%7C01%7C%7C8fb6cc6211854d8631f708d921e5fa6e%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637578093904711618%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000=SO3%2FOC9wbs1WXlumRUJYpqn0r5i3LfMcGxk7STcpPII%3D=0> The reason I didn't backport it was there has been a small change in API to make it work. But Python3 support is probably a good reason to backport it. In the 7_8 release branch, the latest change is from May 2019: https://github.com/OSGeo/grass/commits/releasebranch_7_8/lib/python/pygrass/modules/interface/module.py<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fcommits%2Freleasebranch_7_8%2Flib%2Fpython%2Fpygrass%2Fmodules%2Finterface%2Fmodule.py=04%7C01%7C%7C8fb6cc6211854d8631f708d921e5fa6e%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637578093904711618%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000=pm4%2BxQc8G9FSlaad%2FsXEBzKwhs4lxC%2BXSvRWLRqsjWg%3D=0> Hope that helps identifying a solution? Cheers Stefan -Original Message- From: Stefan Blumentrath mailto:stefan.blumentr...@nina.no>> Sent: torsdag 27. mai 2021 23:09 To: Stefan Blumentrath mailto:stefan.blumentr...@nina.no>>; Luca Delucchi mailto:lucadel...@gmail.com>>; Anna Petrášová mailto:kratocha...@gmail.com>> Cc: GRASS developers list mailto:grass-dev@lists.osgeo.org>> Subject: RE: [GRASS-dev] ParallelModuleQueue freezes... See: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fpull%2F1600data=04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=5yfxG5gjlYlzgJn6yTP9zbUk7GE1JdcYPxWIQFwUK%2Fo%3Dreserved=0<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fpull%2F1600=04%7C01%7C%7C8fb6cc6211854d8631f708d921e5fa6e%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637578093904721611%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000=Kw1pcn3mIlEK8hRS70B966zqhgpud9cB3bcKA2KmRwI%3D=0> Please test. Unittests for t.rast.neighbors should work again with this small change... -Original Message- From: grass-dev mailto:grass-dev-boun...@lists.osgeo.org>> On Behalf Of Stefan Blumentrath Sent: torsdag 27. mai 2021 08:19 To: Luca Delucchi mailto:lucadel...@gmail.com>>; Anna Petrášová mailto:kratocha...@gmail.com>> Cc: GRASS developers list mailto:grass-dev@lists.osgeo.org>> Subject: Re: [GRASS-dev] ParallelModuleQueue freezes... The combination ParallelModuleQueue and MultiModule is used in t.rast.neighbors: https://eur01.s
Re: [GRASS-dev] ParallelModuleQueue freezes...
On Fri, May 28, 2021 at 10:02 AM Stefan Blumentrath < stefan.blumentr...@nina.no> wrote: > Some more context. > The t.rast.neighbors test seems to work with the current dev version > "2048e1016" from 2021-05-19T09:38:49+00:00 > It fails however, with 7.8.5 from 2020-12-22T15:55:09+00:00 and an older > development version "35edbf7dc" > from 2020-06-03T16:03:52+00:00 > > So, maybe one or more of the recent changes need backporting? > > https://github.com/OSGeo/grass/commits/master/python/grass/pygrass/modules/interface/module.py This one: https://github.com/OSGeo/grass/pull/1407 The reason I didn't backport it was there has been a small change in API to make it work. But Python3 support is probably a good reason to backport it. > > > In the 7_8 release branch, the latest change is from May 2019: > > https://github.com/OSGeo/grass/commits/releasebranch_7_8/lib/python/pygrass/modules/interface/module.py > > Hope that helps identifying a solution? > > Cheers > Stefan > > -Original Message- > From: Stefan Blumentrath > Sent: torsdag 27. mai 2021 23:09 > To: Stefan Blumentrath ; Luca Delucchi < > lucadel...@gmail.com>; Anna Petrášová > Cc: GRASS developers list > Subject: RE: [GRASS-dev] ParallelModuleQueue freezes... > > See: > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fpull%2F1600data=04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=5yfxG5gjlYlzgJn6yTP9zbUk7GE1JdcYPxWIQFwUK%2Fo%3Dreserved=0 > Please test. > Unittests for t.rast.neighbors should work again with this small change... > > -Original Message- > From: grass-dev On Behalf Of Stefan > Blumentrath > Sent: torsdag 27. mai 2021 08:19 > To: Luca Delucchi ; Anna Petrášová < > kratocha...@gmail.com> > Cc: GRASS developers list > Subject: Re: [GRASS-dev] ParallelModuleQueue freezes... > > The combination ParallelModuleQueue and MultiModule is used in > t.rast.neighbors: > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fblob%2Fmaster%2Ftemporal%2Ft.rast.neighbors%2Ft.rast.neighbors.pydata=04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=5QyN%2FR1cXgoWWGS2lgjKi8%2BeLX77w37bfMQzcP6O4bM%3Dreserved=0 > > The testsuite of that module is deactivated for Python > 2. > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fblob%2Fmaster%2Ftemporal%2Ft.rast.neighbors%2Ftestsuite%2Ftest_neighbors.pydata=04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=nQTdDIGmefI6F37zdjhyMDBYUrIxVaC%2FiA5FzeUK2KU%3Dreserved=0 > > If you activate it by commenting out line 15. You will see the behavior... > > Thanks for looking into this! > > Cheers > Stefan > > > -Original Message- > From: Luca Delucchi > Sent: torsdag 27. mai 2021 06:07 > To: Anna Petrášová > Cc: Stefan Blumentrath ; GRASS developers > list > Subject: Re: [GRASS-dev] ParallelModuleQueue freezes... > > On Thu, 27 May 2021 at 04:07, Anna Petrášová > wrote: > > > > Could you provide an example that is failing? The doctests are running > for me. I spent some time fixing it a couple months ago. I use Python 3.6, > I wonder if newer Python would cause some issues. > > > > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass-addons%2Fpull%2F523%2Ffilesdata=04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=KT5h55J89LLXhK7uEujAmIwfksjPGyXEy2u7gYIc2Ns%3Dreserved=0 > line 544, the commented lines are not working for me.. > If you want to try t.vi just comment line 543 and uncomment the following > lines.. > > > Anna > > > > -- > ciao > Luca > > > https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.lucadelu.org%2Fdata=04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=hM8CpSgjw7SFjGKxM9SFfWI3G8U3nr8eeCqedZORdso%3Dreserved=0 > ___
Re: [GRASS-dev] ParallelModuleQueue freezes...
Which Python do you have? The test is running just fine for me with 3.6. On Thu, May 27, 2021 at 2:19 AM Stefan Blumentrath < stefan.blumentr...@nina.no> wrote: > The combination ParallelModuleQueue and MultiModule is used in > t.rast.neighbors: > https://github.com/OSGeo/grass/blob/master/temporal/t.rast.neighbors/t.rast.neighbors.py > > The testsuite of that module is deactivated for Python > 2. > > https://github.com/OSGeo/grass/blob/master/temporal/t.rast.neighbors/testsuite/test_neighbors.py > > If you activate it by commenting out line 15. You will see the behavior... > > Thanks for looking into this! > > Cheers > Stefan > > > -Original Message- > From: Luca Delucchi > Sent: torsdag 27. mai 2021 06:07 > To: Anna Petrášová > Cc: Stefan Blumentrath ; GRASS developers > list > Subject: Re: [GRASS-dev] ParallelModuleQueue freezes... > > On Thu, 27 May 2021 at 04:07, Anna Petrášová > wrote: > > > > Could you provide an example that is failing? The doctests are running > for me. I spent some time fixing it a couple months ago. I use Python 3.6, > I wonder if newer Python would cause some issues. > > > > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass-addons%2Fpull%2F523%2Ffilesdata=04%7C01%7C%7Cb35d171402fc441dfd7508d920c4edc3%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637576852445926368%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=a8JVAn3N77HwODX5FguDQjxI4PbUbkN4ZicO4RYuHqo%3Dreserved=0 > line 544, the commented lines are not working for me.. > If you want to try t.vi just comment line 543 and uncomment the following > lines.. > > > Anna > > > > -- > ciao > Luca > > > https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.lucadelu.org%2Fdata=04%7C01%7C%7Cb35d171402fc441dfd7508d920c4edc3%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637576852445936369%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=RWkoI4e%2BbY9%2F5Ow%2BOMqPtEgHGS2BqWRL4pes%2B%2B5T%2BNA%3Dreserved=0 > ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] ParallelModuleQueue freezes...
Some more context. The t.rast.neighbors test seems to work with the current dev version "2048e1016" from 2021-05-19T09:38:49+00:00 It fails however, with 7.8.5 from 2020-12-22T15:55:09+00:00 and an older development version "35edbf7dc" from 2020-06-03T16:03:52+00:00 So, maybe one or more of the recent changes need backporting? https://github.com/OSGeo/grass/commits/master/python/grass/pygrass/modules/interface/module.py In the 7_8 release branch, the latest change is from May 2019: https://github.com/OSGeo/grass/commits/releasebranch_7_8/lib/python/pygrass/modules/interface/module.py Hope that helps identifying a solution? Cheers Stefan -Original Message- From: Stefan Blumentrath Sent: torsdag 27. mai 2021 23:09 To: Stefan Blumentrath ; Luca Delucchi ; Anna Petrášová Cc: GRASS developers list Subject: RE: [GRASS-dev] ParallelModuleQueue freezes... See: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fpull%2F1600data=04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=5yfxG5gjlYlzgJn6yTP9zbUk7GE1JdcYPxWIQFwUK%2Fo%3Dreserved=0 Please test. Unittests for t.rast.neighbors should work again with this small change... -Original Message- From: grass-dev On Behalf Of Stefan Blumentrath Sent: torsdag 27. mai 2021 08:19 To: Luca Delucchi ; Anna Petrášová Cc: GRASS developers list Subject: Re: [GRASS-dev] ParallelModuleQueue freezes... The combination ParallelModuleQueue and MultiModule is used in t.rast.neighbors: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fblob%2Fmaster%2Ftemporal%2Ft.rast.neighbors%2Ft.rast.neighbors.pydata=04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=5QyN%2FR1cXgoWWGS2lgjKi8%2BeLX77w37bfMQzcP6O4bM%3Dreserved=0 The testsuite of that module is deactivated for Python > 2. https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fblob%2Fmaster%2Ftemporal%2Ft.rast.neighbors%2Ftestsuite%2Ftest_neighbors.pydata=04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=nQTdDIGmefI6F37zdjhyMDBYUrIxVaC%2FiA5FzeUK2KU%3Dreserved=0 If you activate it by commenting out line 15. You will see the behavior... Thanks for looking into this! Cheers Stefan -Original Message- From: Luca Delucchi Sent: torsdag 27. mai 2021 06:07 To: Anna Petrášová Cc: Stefan Blumentrath ; GRASS developers list Subject: Re: [GRASS-dev] ParallelModuleQueue freezes... On Thu, 27 May 2021 at 04:07, Anna Petrášová wrote: > > Could you provide an example that is failing? The doctests are running for > me. I spent some time fixing it a couple months ago. I use Python 3.6, I > wonder if newer Python would cause some issues. > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass-addons%2Fpull%2F523%2Ffilesdata=04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=KT5h55J89LLXhK7uEujAmIwfksjPGyXEy2u7gYIc2Ns%3Dreserved=0 line 544, the commented lines are not working for me.. If you want to try t.vi just comment line 543 and uncomment the following lines.. > Anna > -- ciao Luca https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.lucadelu.org%2Fdata=04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=hM8CpSgjw7SFjGKxM9SFfWI3G8U3nr8eeCqedZORdso%3Dreserved=0 ___ grass-dev mailing list grass-dev@lists.osgeo.org https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fgrass-devdata=04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=IwP5QxiLzELTGrr393lqAWrlWRhBVg%2FbOhrz9lBHMkw%3Dreserved=0 ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] ParallelModuleQueue freezes...
See: https://github.com/OSGeo/grass/pull/1600 Please test. Unittests for t.rast.neighbors should work again with this small change... -Original Message- From: grass-dev On Behalf Of Stefan Blumentrath Sent: torsdag 27. mai 2021 08:19 To: Luca Delucchi ; Anna Petrášová Cc: GRASS developers list Subject: Re: [GRASS-dev] ParallelModuleQueue freezes... The combination ParallelModuleQueue and MultiModule is used in t.rast.neighbors: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fblob%2Fmaster%2Ftemporal%2Ft.rast.neighbors%2Ft.rast.neighbors.pydata=04%7C01%7C%7C9ed002957eca4be47b7908d920d75b53%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637576931588309077%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=2zjyEI5TAne19FgKv9vZo%2FcNG1SGP0KP7u0G%2FtQdC5Q%3Dreserved=0 The testsuite of that module is deactivated for Python > 2. https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fblob%2Fmaster%2Ftemporal%2Ft.rast.neighbors%2Ftestsuite%2Ftest_neighbors.pydata=04%7C01%7C%7C9ed002957eca4be47b7908d920d75b53%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637576931588319065%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=jHctgXkjIEHaZ%2BHveeBo49RnOvzzB%2BRhWq7ri2PY59g%3Dreserved=0 If you activate it by commenting out line 15. You will see the behavior... Thanks for looking into this! Cheers Stefan -Original Message- From: Luca Delucchi Sent: torsdag 27. mai 2021 06:07 To: Anna Petrášová Cc: Stefan Blumentrath ; GRASS developers list Subject: Re: [GRASS-dev] ParallelModuleQueue freezes... On Thu, 27 May 2021 at 04:07, Anna Petrášová wrote: > > Could you provide an example that is failing? The doctests are running for > me. I spent some time fixing it a couple months ago. I use Python 3.6, I > wonder if newer Python would cause some issues. > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass-addons%2Fpull%2F523%2Ffilesdata=04%7C01%7C%7C9ed002957eca4be47b7908d920d75b53%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637576931588319065%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=e9%2FkUD2TLUp2AHji07wI56Q0pkF%2Bctld3qtTnsb9%2F34%3Dreserved=0 line 544, the commented lines are not working for me.. If you want to try t.vi just comment line 543 and uncomment the following lines.. > Anna > -- ciao Luca https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.lucadelu.org%2Fdata=04%7C01%7C%7C9ed002957eca4be47b7908d920d75b53%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637576931588319065%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=tgQKu%2BWrWIu2PDesyTQ41Hk%2Bp%2FOdeU3itdIrluTwwBA%3Dreserved=0 ___ grass-dev mailing list grass-dev@lists.osgeo.org https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fgrass-devdata=04%7C01%7C%7C9ed002957eca4be47b7908d920d75b53%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637576931588319065%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=mN%2FeOFVJIQ8PZ2P6GEGM903s44%2BHbcGpodK4yxPxwY4%3Dreserved=0 ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] ParallelModuleQueue freezes...
The combination ParallelModuleQueue and MultiModule is used in t.rast.neighbors: https://github.com/OSGeo/grass/blob/master/temporal/t.rast.neighbors/t.rast.neighbors.py The testsuite of that module is deactivated for Python > 2. https://github.com/OSGeo/grass/blob/master/temporal/t.rast.neighbors/testsuite/test_neighbors.py If you activate it by commenting out line 15. You will see the behavior... Thanks for looking into this! Cheers Stefan -Original Message- From: Luca Delucchi Sent: torsdag 27. mai 2021 06:07 To: Anna Petrášová Cc: Stefan Blumentrath ; GRASS developers list Subject: Re: [GRASS-dev] ParallelModuleQueue freezes... On Thu, 27 May 2021 at 04:07, Anna Petrášová wrote: > > Could you provide an example that is failing? The doctests are running for > me. I spent some time fixing it a couple months ago. I use Python 3.6, I > wonder if newer Python would cause some issues. > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass-addons%2Fpull%2F523%2Ffilesdata=04%7C01%7C%7Cb35d171402fc441dfd7508d920c4edc3%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637576852445926368%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=a8JVAn3N77HwODX5FguDQjxI4PbUbkN4ZicO4RYuHqo%3Dreserved=0 line 544, the commented lines are not working for me.. If you want to try t.vi just comment line 543 and uncomment the following lines.. > Anna > -- ciao Luca https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.lucadelu.org%2Fdata=04%7C01%7C%7Cb35d171402fc441dfd7508d920c4edc3%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637576852445936369%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=RWkoI4e%2BbY9%2F5Ow%2BOMqPtEgHGS2BqWRL4pes%2B%2B5T%2BNA%3Dreserved=0 ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] ParallelModuleQueue freezes...
On Thu, 27 May 2021 at 04:07, Anna Petrášová wrote: > > Could you provide an example that is failing? The doctests are running for > me. I spent some time fixing it a couple months ago. I use Python 3.6, I > wonder if newer Python would cause some issues. > https://github.com/OSGeo/grass-addons/pull/523/files line 544, the commented lines are not working for me.. If you want to try t.vi just comment line 543 and uncomment the following lines.. > Anna > -- ciao Luca www.lucadelu.org ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] ParallelModuleQueue freezes...
Could you provide an example that is failing? The doctests are running for me. I spent some time fixing it a couple months ago. I use Python 3.6, I wonder if newer Python would cause some issues. Anna On Tue, May 25, 2021 at 8:33 AM Stefan Blumentrath < stefan.blumentr...@nina.no> wrote: > Dear devs, > > > > Currently, I am trying to run a combination of ParallelModuleQueue and > MultiModule as described here in GRASS 7.8: > > > https://grass.osgeo.org/grass79/manuals/libpython/pygrass.modules.interface.html?highlight=parallel%20module%20qu#pygrass.modules.interface.module.ParallelModuleQueue > > > > It seems – in contrast to the documentation - also the finish_= parameter > needs to be set to True for the modules to run. > > > > However, after finishing the queue, the script stalls and does not seem to > get over queue.wait(). > > Just to be clear, also the code from the doc does not seem to work for me… > > > > Any ideas? > > > > Cheers, > > Stefan > ___ > grass-dev mailing list > grass-dev@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/grass-dev > ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] ParallelModuleQueue freezes...
On Wed, 26 May 2021 at 15:13, Stefan Blumentrath wrote: > > Thanks, Markus. > > Seems that the ParallelModuleQueue itself works if it is fed with Module > objects, but not if MultiModule objects are put into the queue... > > So, I believe I am struggling with the same issue as Luca: > https://github.com/OSGeo/grass-addons/pull/523#issue-629160914 > probably yes > Can it be that returns of the MuliModule should be different? If I understand > the source code [1] correctly, Module returns "self" (a Module object) while > MultiModule returns "None". Can that be the root cause? > it seems to me that MultiModule returns "None" when it finish otherwise return self.p (a Process instance) > Any help is much appreciated... > same for my side :-) > Cheers > Stefan > > 1: > https://grass.osgeo.org/grass79/manuals/libpython/_modules/pygrass/modules/interface/module.html#MultiModule > -- ciao Luca www.lucadelu.org ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] ParallelModuleQueue freezes...
Thanks, Markus. Seems that the ParallelModuleQueue itself works if it is fed with Module objects, but not if MultiModule objects are put into the queue... So, I believe I am struggling with the same issue as Luca: https://github.com/OSGeo/grass-addons/pull/523#issue-629160914 Can it be that returns of the MuliModule should be different? If I understand the source code [1] correctly, Module returns "self" (a Module object) while MultiModule returns "None". Can that be the root cause? Any help is much appreciated... Cheers Stefan 1: https://grass.osgeo.org/grass79/manuals/libpython/_modules/pygrass/modules/interface/module.html#MultiModule -Original Message- From: Markus Neteler Sent: tirsdag 25. mai 2021 20:17 To: Stefan Blumentrath Cc: GRASS developers list Subject: Re: [GRASS-dev] ParallelModuleQueue freezes... Hi Stefan, On Tue, May 25, 2021 at 2:33 PM Stefan Blumentrath wrote: > > Dear devs, > > > > Currently, I am trying to run a combination of ParallelModuleQueue and > MultiModule as described here in GRASS 7.8: > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgras > s.osgeo.org%2Fgrass79%2Fmanuals%2Flibpython%2Fpygrass.modules.interfac > e.html%3Fhighlight%3Dparallel%2520module%2520qu%23pygrass.modules.inte > rface.module.ParallelModuleQueuedata=04%7C01%7C%7C61c3f8df57d44d9 > 0905108d91fa9435d%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C6375756 > 34109191335%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMz > IiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=PC64zLohfEqbxUcNfX4U > hHHNf5VIgRO7x0o%2FbiYHyU4%3Dreserved=0 > > It seems - in contrast to the documentation - also the finish_= parameter > needs to be set to True for the modules to run. > > However, after finishing the queue, the script stalls and does not seem to > get over queue.wait(). > > Just to be clear, also the code from the doc does not seem to work for > me... While I have no direct recommendation you may want to check the working code here, in the addons: - t.rast.mosaic - t.sentinel.import - t.sentinel.mask - t.rast.resample See: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsearch%3Fq%3Dorg%253Amundialis%2BParallelModuleQueue%26type%3Dcodedata=04%7C01%7C%7C61c3f8df57d44d90905108d91fa9435d%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637575634109191335%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=tbzIS%2FhRmi6hWOAuieoNxubEN73Q8MDmfqHK0wyy7eE%3Dreserved=0 Best, Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] ParallelModuleQueue freezes...
Hi Stefan, On Tue, May 25, 2021 at 2:33 PM Stefan Blumentrath wrote: > > Dear devs, > > > > Currently, I am trying to run a combination of ParallelModuleQueue and > MultiModule as described here in GRASS 7.8: > > https://grass.osgeo.org/grass79/manuals/libpython/pygrass.modules.interface.html?highlight=parallel%20module%20qu#pygrass.modules.interface.module.ParallelModuleQueue > > It seems – in contrast to the documentation - also the finish_= parameter > needs to be set to True for the modules to run. > > However, after finishing the queue, the script stalls and does not seem to > get over queue.wait(). > > Just to be clear, also the code from the doc does not seem to work for me… While I have no direct recommendation you may want to check the working code here, in the addons: - t.rast.mosaic - t.sentinel.import - t.sentinel.mask - t.rast.resample See: https://github.com/search?q=org%3Amundialis+ParallelModuleQueue=code Best, Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] ParallelModuleQueue freezes...
Dear devs, Currently, I am trying to run a combination of ParallelModuleQueue and MultiModule as described here in GRASS 7.8: https://grass.osgeo.org/grass79/manuals/libpython/pygrass.modules.interface.html?highlight=parallel%20module%20qu#pygrass.modules.interface.module.ParallelModuleQueue It seems - in contrast to the documentation - also the finish_= parameter needs to be set to True for the modules to run. However, after finishing the queue, the script stalls and does not seem to get over queue.wait(). Just to be clear, also the code from the doc does not seem to work for me... Any ideas? Cheers, Stefan ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
