[GRASS-dev] Ignored enhancements
A while ago I made some simple, but significant, enhancements and submitted them to trac. They haven't been picked up, so I wanted to point them out so someone can commit them before they diverge from the trunk. http://trac.osgeo.org/grass/ticket/1624 This has had some activity recently, but it's status is still "new" and it isn't in trunk. http://trac.osgeo.org/grass/ticket/1575 ~Seth via iPhone___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Ignored enhancements
Seth, I finally got a chance to patch your enhancements from the GRASS 70 svn code I pulled last night. I compiled with these options: ./configure --with-gdal --with-freetype --with-freetype-includes=/usr/include/freetype2 --with-proj --with-sqlite --enable-largefile --with-cxx --enable-64bit --with-python --with-blas --with-lapack --with-cairo --with-wxwidgets --with-spatialite --with-tcltk --with-tcltk-includes=/usr/include/tcl8.4 --with-liblas=/usr/local/bin/liblas-config --enable-largefile --with-openmp --with-readline The computer uses an AMD Vishera 8 core cpu with 8GB DDR3 1333 RAM ( freshly updated home machine) I used this 23 million cell raster of 20ft bare earth elevation in the mountains of western North Carolina. Layer:nc_1@PERMANENT Date: Thu Jul 5 20:56:51 2012 | | Mapset: PERMANENT Login of Creator: dnewcomb | | Location: ncstpft | | DataBase: /data1/grass7 | | Title:bilinear interpolation with Tykhonov regularization ( nc_1 ) | | Timestamp: none | || | | | Type of Map: raster Number of Categories: 7000 | | Data Type:DCELL | | Rows: 5328 | | Columns: 4375 | | Total Cells: 2331 | |Projection: Lambert Conformal Conic | |N: 593000S: 486439 Res: 20.00018769 | |E: 492500W: 405000 Res:20 | | Range of data:min = -8269.7102472716 max = 6999.76272107517 | | | | Data Description: | |generated by v.surf.bspline | | | | Comments: | |v.surf.bspline -z input="nc_1@PERMANENT" raster="nc_1" sie=40 sin=40\ | | method="bilinear" lambda_i=0.01 layer="1" solver="cholesky" maxit=1\ | |00 error=0.01 memory=300 | | | ++ First, with the default GRASS 7 svn pull from 2/9 GRASS 7.0.svn (ncstpft):~ > time r.horizon -d elevin=nc_1 horizonstep=90 horizon=test_normal Calculating map 1 of 4 (angle 0.00, raster map ) 100% Calculating map 2 of 4 (angle 90.00, raster map ) 100% Calculating map 3 of 4 (angle 180.00, raster map ) 100% Calculating map 4 of 4 (angle 270.00, raster map ) 100% real 16m14.028s user 16m11.993s sys 0m1.860s Second, patching r.horizon main.c with the diff file from your patch. GRASS 7.0.svn (ncstpft):~ > time r.horizon -d elevin=nc_1 horizonstep=90 horizon=test_price Calculating map 1 of 4 (angle 0.00, raster map ) 100% Calculating map 2 of 4 (angle 90.00, raster map ) 100% Calculating map 3 of 4 (angle 180.00, raster map ) 100% Calculating map 4 of 4 (angle 270.00, raster map ) 100% real 14m22.607s user 14m20.582s sys 0m1.836s Using r.mapcal to subtract the normal r.horizon calculation from the modified calculation shows 0 difference between the two results I usually run r.horizon on a 755 million cell grid and do 24 horizons, I am assuming the the benefits would scale proportionally? Doug On Mon, Feb 4, 2013 at 12:40 PM, Seth Price wrote: > A while ago I made some simple, but significant, enhancements and > submitted them to trac. They haven't been picked up, so I wanted to point > them out so someone can commit them before they diverge from the trunk. > > http://trac.osgeo.org/grass/ticket/1624 > > This has had some activity recently, but it's status is still "new" and it > isn't in trunk. > > http://trac.osgeo.org/grass/ticket/1575 > > ~Seth > > > via iPhone > > ___ > grass-dev mailing list > grass-dev@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/grass-dev > -- Doug Newcomb USFWS Raleigh, NC 919-856-4520 ext. 14 doug_newc...@fws.gov - The opinions I express are my own and are not representative of the official policy of the U.S.Fish and Wildlife Service or Dept. of the Interior. Life is too short for undocumented, proprietary data formats. ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Ignored enhancements
They should. I didn't really make any changes to the calculations, I just rearranged the code to make it more processor and compiler friendly. ~Seth via iPhone On Feb 9, 2013, at 3:27 PM, "Newcomb, Doug" wrote: > Seth, > I finally got a chance to patch your enhancements from the GRASS 70 svn code > I pulled last night. > > I compiled with these options: > ./configure --with-gdal --with-freetype > --with-freetype-includes=/usr/include/freetype2 --with-proj --with-sqlite > --enable-largefile --with-cxx --enable-64bit --with-python --with-blas > --with-lapack --with-cairo --with-wxwidgets --with-spatialite --with-tcltk > --with-tcltk-includes=/usr/include/tcl8.4 > --with-liblas=/usr/local/bin/liblas-config --enable-largefile --with-openmp > --with-readline > > The computer uses an AMD Vishera 8 core cpu with 8GB DDR3 1333 RAM ( freshly > updated home machine) > > > I used this 23 million cell raster of 20ft bare earth elevation in the > mountains of western North Carolina. > > > Layer:nc_1@PERMANENT Date: Thu Jul 5 20:56:51 2012| > | Mapset: PERMANENT Login of Creator: dnewcomb > | > | Location: ncstpft > | > | DataBase: /data1/grass7 > | > | Title:bilinear interpolation with Tykhonov regularization ( nc_1 ) > | > | Timestamp: none > | > > || > | > | > | Type of Map: raster Number of Categories: 7000 > | > | Data Type:DCELL > | > | Rows: 5328 > | > | Columns: 4375 > | > | Total Cells: 2331 > | > |Projection: Lambert Conformal Conic > | > |N: 593000S: 486439 Res: 20.00018769 > | > |E: 492500W: 405000 Res:20 > | > | Range of data:min = -8269.7102472716 max = 6999.76272107517 > | > | > | > | Data Description: > | > |generated by v.surf.bspline > | > | > | > | Comments: > | > |v.surf.bspline -z input="nc_1@PERMANENT" raster="nc_1" sie=40 sin=40\ > | > | method="bilinear" lambda_i=0.01 layer="1" solver="cholesky" maxit=1\ > | > |00 error=0.01 memory=300 > | > | > | > > ++ > > > First, with the default GRASS 7 svn pull from 2/9 > > GRASS 7.0.svn (ncstpft):~ > time r.horizon -d elevin=nc_1 horizonstep=90 > horizon=test_normal > Calculating map 1 of 4 (angle 0.00, raster map ) > 100% > Calculating map 2 of 4 (angle 90.00, raster map ) > 100% > Calculating map 3 of 4 (angle 180.00, raster map ) > 100% > Calculating map 4 of 4 (angle 270.00, raster map ) > 100% > > real 16m14.028s > user 16m11.993s > sys 0m1.860s > > > Second, patching r.horizon main.c with the diff file from your patch. > > GRASS 7.0.svn (ncstpft):~ > time r.horizon -d elevin=nc_1 horizonstep=90 > horizon=test_price > Calculating map 1 of 4 (angle 0.00, raster map ) > 100% > Calculating map 2 of 4 (angle 90.00, raster map ) > 100% > Calculating map 3 of 4 (angle 180.00, raster map ) > 100% > Calculating map 4 of 4 (angle 270.00, raster map ) > 100% > > real 14m22.607s > user 14m20.582s > sys 0m1.836s > > Using r.mapcal to subtract the normal r.horizon calculation from the modified > calculation shows 0 difference between the two results > > I usually run r.horizon on a 755 million cell grid and do 24 horizons, I am > assuming the the benefits would scale proportionally? > > > Doug > > > > On Mon, Feb 4, 2013 at 12:40 PM, Seth Price wrote: >> A while ago I made some simple, but significant, enhancements and submitted >> them to trac. They haven't been picked up, so I wanted to point them out so >> someone can commit them before they diverge from the trunk. >> >> http://trac.osgeo.org/grass/ticket/1624 >> >> This has had some activity recently, but it's status is still "new" and it >> isn't in trunk. >> >> http://trac.osgeo.or
Re: [GRASS-dev] Ignored enhancements
Seth, I appreciate the effort. Working with large rasters, this will make a big difference! Doug On Sun, Feb 10, 2013 at 9:18 PM, Seth Price wrote: > They should. I didn't really make any changes to the calculations, I just > rearranged the code to make it more processor and compiler friendly. > > ~Seth > > > via iPhone > > On Feb 9, 2013, at 3:27 PM, "Newcomb, Doug" wrote: > > Seth, > I finally got a chance to patch your enhancements from the GRASS 70 svn > code I pulled last night. > > I compiled with these options: > ./configure --with-gdal --with-freetype > --with-freetype-includes=/usr/include/freetype2 --with-proj --with-sqlite > --enable-largefile --with-cxx --enable-64bit --with-python --with-blas > --with-lapack --with-cairo --with-wxwidgets --with-spatialite > --with-tcltk --with-tcltk-includes=/usr/include/tcl8.4 > --with-liblas=/usr/local/bin/liblas-config --enable-largefile --with-openmp > --with-readline > > The computer uses an AMD Vishera 8 core cpu with 8GB DDR3 1333 RAM ( > freshly updated home machine) > > > I used this 23 million cell raster of 20ft bare earth elevation in the > mountains of western North Carolina. > > > Layer:nc_1@PERMANENT Date: Thu Jul 5 20:56:51 2012 >| > | Mapset: PERMANENT Login of Creator: dnewcomb >| > | Location: ncstpft >| > | DataBase: /data1/grass7 >| > | Title:bilinear interpolation with Tykhonov regularization ( nc_1 ) > | > | Timestamp: none >| > > > || > | >| > | Type of Map: raster Number of Categories: 7000 >| > | Data Type:DCELL >| > | Rows: 5328 > | > | Columns: 4375 > | > | Total Cells: 2331 > | > |Projection: Lambert Conformal Conic > | > |N: 593000S: 486439 Res: 20.00018769 > | > |E: 492500W: 405000 Res:20 > | > | Range of data:min = -8269.7102472716 max = 6999.76272107517 > | > | >| > | Data Description: >| > |generated by v.surf.bspline > | > | >| > | Comments: >| > |v.surf.bspline -z input="nc_1@PERMANENT" raster="nc_1" sie=40 > sin=40\ | > | method="bilinear" lambda_i=0.01 layer="1" solver="cholesky" > maxit=1\ | > |00 error=0.01 memory=300 >| > | >| > > > ++ > > > First, with the default GRASS 7 svn pull from 2/9 > > GRASS 7.0.svn (ncstpft):~ > time r.horizon -d elevin=nc_1 horizonstep=90 > horizon=test_normal > Calculating map 1 of 4 (angle 0.00, raster map ) > 100% > Calculating map 2 of 4 (angle 90.00, raster map ) > 100% > Calculating map 3 of 4 (angle 180.00, raster map ) > 100% > Calculating map 4 of 4 (angle 270.00, raster map ) > 100% > > real 16m14.028s > user 16m11.993s > sys 0m1.860s > > > Second, patching r.horizon main.c with the diff file from your patch. > > GRASS 7.0.svn (ncstpft):~ > time r.horizon -d elevin=nc_1 horizonstep=90 > horizon=test_price > Calculating map 1 of 4 (angle 0.00, raster map ) > 100% > Calculating map 2 of 4 (angle 90.00, raster map ) > 100% > Calculating map 3 of 4 (angle 180.00, raster map ) > 100% > Calculating map 4 of 4 (angle 270.00, raster map ) > 100% > > real 14m22.607s > user 14m20.582s > sys 0m1.836s > > Using r.mapcal to subtract the normal r.horizon calculation from the > modified calculation shows 0 difference between the two results > > I usually run r.horizon on a 755 million cell grid and do 24 horizons, I > am assuming the the benefits would scale proportionally? > > > Doug > > > > On Mon, Feb 4, 2013 at 12:40 PM, Seth Price wrote: > >> A while ago I made some simple, but significant, enhancements and >> submitted them to trac. They haven't been picked up, so I wanted to point >> them out so someone can commit them before they diverge from the trunk. >> >> http://trac.osgeo.org/grass/ticket/1624 >> >> This has had some activity recently, but it's status is still "new" and >> it isn't in trunk. >> >> http://trac.osgeo.org/grass/ticket/1575 >> >> ~Seth >> >> >> via iPhone >> >> ___ >> grass-dev mailing list >> grass-dev@lists.osgeo.org >> http://lists.osgeo.org/mailman/listinfo/grass-dev >> > > > > -- > Doug Newcomb > USFWS > Raleigh, NC > 919-856-4520 ext. 14 doug_newc...@fws.gov > > - > The opinions I express are my own and are not representative of the > official policy of the U.S.Fish and Wildlife Service or Dept. of the > Interior. Life is too short for undocumented, proprietary data formats. > > -- Doug Newcomb USFWS Raleigh, NC 919-856-4520 ext. 14 doug_newc...@fws.gov
Re: [GRASS-dev] Ignored enhancements
Doug wrote: > I appreciate the effort. (me too :) > Working with large rasters, this will make a big difference! with respect to r.hozion, I'm still trying to understand where the best use case for it is. the idea as I understand it (which is not completely) is that it precomutes some of what r.sun does, so that the r.sun module runs faster. But the r.sun module places the sun exactly where it wants it, while r.horizon must make some compromise and approximate the position. to get any sort of reasonable placement you'd need upwards of 100 seed maps, but since r.sun must open them all at once the performance hit you take from the extra overhead cancels out the speed gain. And so you have more disk I/O with worse positioning -- better not to have used r.horizon at all? see further discussion and benchmarking in "r.sun2 commissioning trials": http://trac.osgeo.org/grass/ticket/498 and on the r.sun page in the wiki: http://grasswiki.osgeo.org/wiki/R.sun#Seed_maps thanks for any light you can shed on this, it has puzzled me for a while. Hamish ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Ignored enhancements
It's been a while, but I think one motivating factor is the limiting GPU RAM. If we can't fit everything on the GPU, then it needs to be tiled. The only way to accurately tile is to make each tile independent of its surroundings. This can be done arbitrarily small with pre computed horizons. I think the other reason is the whole pre computation (but only once) thing you mention. I don't think it bothers me too much at the moment what you decide to do with it, but the shadowing & tiling code becomes more complicated to compensate. ~Seth via iPhone On Feb 12, 2013, at 2:10 AM, Hamish wrote: > Doug wrote: >> I appreciate the effort. > > (me too :) > >> Working with large rasters, this will make a big difference! > > with respect to r.hozion, I'm still trying to understand where > the best use case for it is. > > the idea as I understand it (which is not completely) is that > it precomutes some of what r.sun does, so that the r.sun module > runs faster. But the r.sun module places the sun exactly where > it wants it, while r.horizon must make some compromise and > approximate the position. to get any sort of reasonable placement > you'd need upwards of 100 seed maps, but since r.sun must open > them all at once the performance hit you take from the extra > overhead cancels out the speed gain. And so you have more disk > I/O with worse positioning -- better not to have used r.horizon > at all? > > see further discussion and benchmarking in "r.sun2 commissioning > trials": > http://trac.osgeo.org/grass/ticket/498 > > and on the r.sun page in the wiki: > http://grasswiki.osgeo.org/wiki/R.sun#Seed_maps > > > thanks for any light you can shed on this, it has puzzled me > for a while. > > > Hamish ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
