Hi Moritz, Stefan (and Marcus if you’re around?) Every day is a new day.
Not IOPs, Not memory, Not CPU… Hmm, how about reboot... >> >> As you have suspected, I get no benefit from additional CPUs. > > Are you sure the problem is CPU-bound ? I started by testing v.net (the maintenance module) in parallel - and observed pretty much linear scaling in performance over and above 2 parallel jobs. So I then proceeded to test v.net.distance… and sure enough, it is now scaling. Really different looking CPU performance profiles to yesterday, when it wasn’t scaling. On top, all the v.net CPUs are humming at 50%, the other 50% used by pg processes. Below are some iostat profiles of the different parallel test. So v.net.distance does parallelise nicely. The neatest way is to launch GNU parallel from within a single grass_batch_job. I will write up on the grass wiki in the coming 2 weeks. Mark :-))))) ---------------V.NET --------------- Single job v.net iostat 1 TOTAL SCRIPT TIME: 284 avg-cpu: %user %nice %system %iowait %steal %idle 15.92 0.00 35.32 0.50 0.00 48.26 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvde 18.00 8.00 744.00 8 744 xvdj 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 20.00 0.00 34.00 0.50 0.00 45.50 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvde 9.90 0.00 633.66 0 640 xvdj 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 22.50 0.00 35.50 0.50 0.00 41.50 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvde 9.00 0.00 592.00 0 592 xvdj 0.00 0.00 0.00 0 0 Two parallel jobs v.net iostat 1 TOTAL SCRIPT TIME: 397 avg-cpu: %user %nice %system %iowait %steal %idle 32.16 0.00 67.84 0.00 0.00 0.00 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvde 19.00 0.00 1024.00 0 1024 xvdj 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 39.50 0.00 60.50 0.00 0.00 0.00 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvde 19.00 0.00 912.00 0 912 xvdj 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 35.50 0.00 64.50 0.00 0.00 0.00 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvde 16.00 0.00 1024.00 0 1024 xvdj 0.00 0.00 0.00 0 0 Four parallel jobs v.net iostat 1 TOTAL SCRIPT TIME: 388 avg-cpu: %user %nice %system %iowait %steal %idle 32.92 0.00 67.08 0.00 0.00 0.00 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvde 31.00 0.00 1952.00 0 1952 xvdj 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 33.83 0.00 66.17 0.00 0.00 0.00 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvde 35.00 0.00 2016.00 0 2016 xvdj 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 32.92 0.00 67.08 0.00 0.00 0.00 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvde 61.00 0.00 4032.00 0 4032 xvdj 0.00 0.00 0.00 0 0 ---------------V.NET.DISTANCE --------------- Single job v.net.distance iostat 1 TOTAL SCRIPT TIME: 88 avg-cpu: %user %nice %system %iowait %steal %idle 4.00 0.00 21.00 0.00 0.00 75.00 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvde 0.00 0.00 0.00 0 0 xvdj 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 7.75 0.00 21.75 0.00 0.00 70.50 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvde 0.00 0.00 0.00 0 0 xvdj 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 6.27 0.00 22.56 0.00 0.00 71.18 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvde 4.00 0.00 56.00 0 56 xvdj 0.00 0.00 0.00 0 0 Two parallel jobs v.net.distance iostat 1 TOTAL SCRIPT TIME: 96 avg-cpu: %user %nice %system %iowait %steal %idle 9.02 0.00 40.85 0.00 0.00 50.13 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvde 0.00 0.00 0.00 0 0 xvdj 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 11.75 0.00 43.75 0.00 0.00 44.50 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvde 3.00 0.00 40.00 0 40 xvdj 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 9.50 0.00 40.75 0.00 0.00 49.75 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvde 0.00 0.00 0.00 0 0 xvdj 0.00 0.00 0.00 0 0 Four parallel jobs v.net.distance iostat 1 TOTAL SCRIPT TIME: 124 avg-cpu: %user %nice %system %iowait %steal %idle 17.96 0.00 82.04 0.00 0.00 0.00 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvde 3.00 0.00 32.00 0 32 xvdj 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 18.50 0.00 81.50 0.00 0.00 0.00 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvde 0.00 0.00 0.00 0 0 xvdj 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 21.50 0.00 78.50 0.00 0.00 0.00 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvde 0.00 0.00 0.00 0 0 xvdj 7.00 0.00 64.00 0 64 Eight parallel jobs v.net.distance iostat 1 TOTAL SCRIPT TIME: 122 avg-cpu: %user %nice %system %iowait %steal %idle 19.00 0.00 81.00 0.00 0.00 0.00 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvde 0.00 0.00 0.00 0 0 xvdj 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 19.50 0.00 80.50 0.00 0.00 0.00 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvde 0.00 0.00 0.00 0 0 xvdj 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 18.65 0.00 81.35 0.00 0.00 0.00 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvde 3.00 0.00 48.00 0 48 xvdj 0.00 0.00 0.00 0 0
_______________________________________________ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user