date:20170704

Re: [GRASS-dev] [GRASS GIS] #3365: r.in.gdal: crash on large dataset

2017-07-04 Thread GRASS GIS

#3365: r.in.gdal: crash on large dataset
--+-
  Reporter:  neteler  |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.2.2
 Component:  Raster   |Version:  svn-releasebranch72
Resolution:   |   Keywords:  r.in.gdal
   CPU:  x86-64   |   Platform:  Linux
--+-

Old description:

> Trying to import a large VRT dataset, I get a crash:
>
> {{{
> GRASS 7.2.2svn (latlong):~/software/grass72_svn_debugmode/bin.x86_64-pc-
> linux-gnu > r.in.gdal
> input=/mnt/kvm7-storage/geodata/global_forest/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015/loss_global_forest_cover_loss_2000_2015.vrt
> output=loss_global_forest_cover_loss_2000_2015
> Proceeding with import of 1 raster bands...
> Importing raster map ...
> Segmentation fault (core dumped)
> }}}
>
> Debugging:
> {{{
> GRASS 7.2.2svn (latlong):~/software/grass72_svn_debugmode/bin.x86_64-pc-
> linux-gnu > gdb r.in.gdal
> GNU gdb (GDB) Fedora 7.12.1-48.fc25
> Copyright (C) 2017 Free Software Foundation, Inc.
> [...]
> Reading symbols from r.in.gdal...done.
> (gdb) r
> input=/mnt//GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015.vrt
> output=loss_global_forest_cover_loss_2000_2015
> Starting program:
> /home/mundialis/software/grass72_svn_debugmode/dist.x86_64-pc-linux-
> gnu/bin/r.in.gdal
> input=/mnt//GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015.vrt
> output=loss_global_forest_cover_loss_2000_2015
> Missing separate debuginfos, use: dnf debuginfo-install
> glibc-2.24-8.fc25.x86_64
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib64/libthread_db.so.1".
> Proceeding with import of 1 raster bands...
> Importing raster map ...
>
> Program received signal SIGSEGV, Segmentation fault.
> 0x777a1ab9 in put_data (fd=0, null_buf=0x7fe9b020 "",
> cell=0x7fffc62ac010, row=0, n=144, zeros_r_nulls=0) at put_row.c:370
> 370 compressed_buf[0] = work_buf[0] = nbytes;
>

>
> (gdb) bt full
> #0  0x777a1ab9 in put_data (fd=0, null_buf=0x7fe9b020 "",
> cell=0x7fffc62ac010, row=0, n=144, zeros_r_nulls=0) at put_row.c:370
> wk = 0x7f91cb31 ""
> nbytes = 1
> compressed_buf = 0x7f7bd220  address 0x7f7bd220>
> total = 144
> fcb = 0x691080
> compressed = 1
> len = 4
> work_buf = 0x7f91cb30 "\001"
> wk = 0x7f91cb31 ""
> nwrite = 0
> #1  0x777a1f98 in put_raster_data (fd=0, null_buf=0x7fe9b020
> "", rast=0x7fffc62ac010, row=0, n=144, zeros_r_nulls=0, map_type=0)
> at put_row.c:476
> fcb = 0x691080
> #2  0x777a2bdb in put_raster_row (fd=0, buf=0x7fffc62ac010,
> data_type=0, zeros_r_nulls=0) at put_row.c:699
> convert_and_write_FtypeOtype = {{0x0, 0x777a23d9
> , 0x777a25f5 }, {
> 0x777a280f , 0x0, 0x777a26f3
> }, {0x777a2928 ,
> 0x777a24d7 , 0x0}}
> fcb = 0x691080
> null_buf = 0x7fe9b020 ""
> #3  0x777a0f8b in Rast_put_row (fd=0, buf=0x7fffc62ac010,
> data_type=0) at put_row.c:67
> No locals.
> #4  0x00406e4d in ImportBand (hBand=0x7dbcb0, output=0x627dc0
> "loss_global_forest_cover_loss_2000_2015", group_ref=0x0) at main.c:1141
> data_type = 0
> eGDT = GDT_Int32
> eRawGDT = GDT_Byte
> row = 1
> nrows = 56
> ncols = 144
> complex = 0
> cf = 0
> cfR = 32767
> cfI = -145625496
> bNoDataEnabled = 0
> indx = 32767
> cell = 0x7fffc62ac010
> cellReal = 0x77
> cellImg = 0x7fffac40
> bufComplex = 0x7fffab40
> dfNoData = -1
> outputReal =
> "0\341\377\367\377\177\000\000\260\254\377\377\377\177\000\000\071\031@\000\000\000\000\000\243\366\251\242\000\000\000\000\377\377\377\377",
> '\000' ,
> "H\017\064\366\377\177\000\000\000\240\374\367\377\177\000\000\000\000\000\000\000\000\360?",
> '\000' , "\360?", '\000' ,
> "\360?\000\000\000\000\000\000\000\000\220\062@\000\000\000\000\000\314\345w\367\377\177\000\000P\255\377\377\377\177\000\000\350\262`\000\000\000\000\000\220\062@\000\000\000\000\000\320\332\377\377\377\177",
> '\000' , "\360\331\377\377\377\177\000\000"...
> outputImg = '\000' ,
> "\377\000\000\000\000\000\377\377\000\000\000\000\000\000\377\377\000\000\000\000\000\000\377\377\377\377\377\377\000\377\377\377",
> '\000' , "\360?", '\000' , "\360?",
> '\000' ,
> "\360?\000\000\000\000\000\000\000\000\210\345w\367\377\177", '\000'
> ,
> "\001\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\200\256\377\377\377\177\000\000\240\224z\367\377\177\000\000\000\273}",
> '\000' ...
> nullFlags = 0x0
> history = {fields = {0x77fca000 "", 0x7fffabe8 "",
> 0x7fffabe4 "", 0x77ff7b78 "\350\343\377\367\

Re: [GRASS-dev] [GRASS GIS] #3365: r.in.gdal: crash on large dataset

2017-07-04 Thread GRASS GIS

#3365: r.in.gdal: crash on large dataset
--+-
  Reporter:  neteler  |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.2.2
 Component:  Raster   |Version:  svn-releasebranch72
Resolution:   |   Keywords:  r.in.gdal
   CPU:  x86-64   |   Platform:  Linux
--+-

Comment (by neteler):

 I got an idea how to more easily reproduce the problem with less data: it
 is sufficient to only download the four corner tiles and create a VRT from
 it:

 {{{
 # download tiles
 wget -c https://storage.googleapis.com/earthenginepartners-
 hansen/GFC-2015-v1.3/Hansen_GFC-2015-v1.3_loss_50S_170E.tif
 wget -c https://storage.googleapis.com/earthenginepartners-
 hansen/GFC-2015-v1.3/Hansen_GFC-2015-v1.3_loss_50S_180W.tif
 wget -c https://storage.googleapis.com/earthenginepartners-
 hansen/GFC-2015-v1.3/Hansen_GFC-2015-v1.3_loss_80N_170E.tif
 wget -c https://storage.googleapis.com/earthenginepartners-
 hansen/GFC-2015-v1.3/Hansen_GFC-2015-v1.3_loss_80N_180W.tif


 # create VRT
 gdalbuildvrt forest_loss_corner_tiles.vrt
 Hansen_GFC-2015-v1.3_loss_50S_170E.tif
 Hansen_GFC-2015-v1.3_loss_50S_180W.tif
 Hansen_GFC-2015-v1.3_loss_80N_170E.tif
 Hansen_GFC-2015-v1.3_loss_80N_180W.tif

 # import
 r.in.gdal input=forest_loss_corner_tiles.vrt
 output=loss_global_forest_corner_tiles
 Segmentation fault (core dumped)
 }}}

 Looks like an overflow with the large amount of rows/cols.

--
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3365: r.in.gdal: crash on large dataset

2017-07-04 Thread GRASS GIS

#3365: r.in.gdal: crash on large dataset
--+-
  Reporter:  neteler  |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.2.2
 Component:  Raster   |Version:  svn-releasebranch72
Resolution:   |   Keywords:  r.in.gdal
   CPU:  x86-64   |   Platform:  Linux
--+-

Comment (by mmetz):

 Replying to [comment:2 neteler]:
 > Using valgrind:
 >
 > {{{
 > GRASS 7.3.svn (latlong):~ > valgrind $CMD
 > ==30211== Memcheck, a memory error detector
 > ==30211== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et
 al.
 > ==30211== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright
 info
 > ==30211== Command: r.in.gdal
 
input=/mnt/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015/loss_global_forest_cover_loss_2000_2015.vrt
 output=loss_global_forest_cover_loss_2000_2015
 > ==30211==
 > Importing raster map ...
 > ==30211== Warning: client switching stacks?  SP change: 0xffee9d140 -->
 0xffe91ed30
 > ==30211==  to suppress, use: --max-stackframe=5760016 or greater
 > ==30211== Invalid write of size 8
 > ==30211==at 0x5273D89: ??? (in
 /home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
 gnu/lib/libgrass_raster.7.3.svn.so)
 > }}}

 Maybe you need to recompile GRASS with -g in order to replace "???" with
 some real information.

 I could reproduce the problem, the segfault is caused by stack overflow
 because alloca allocates memory in the stack, while malloc and calloc
 allocate memory in the heap. I could fix the segfault by replacing
 G_alloca()/G_freea() with G_malloc()/G_free(). Please try the attached
 patch against lib/raster/.

--
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3365: r.in.gdal: crash on large dataset

2017-07-04 Thread GRASS GIS

#3365: r.in.gdal: crash on large dataset
--+-
  Reporter:  neteler  |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.2.2
 Component:  Raster   |Version:  svn-releasebranch72
Resolution:   |   Keywords:  r.in.gdal
   CPU:  x86-64   |   Platform:  Linux
--+-
Changes (by mmetz):

 * Attachment "rasterlib.patch" added.

 patch for rasterlib to replace G_alloca for reading rows (avoids stack
 overflow)

--
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Introduction for GSoC 2017

2017-07-04 Thread Yann


Hi Stefan and all,

Anybody had the opportunity to try the modules ?

Any feed back?

Cheers,

Yann


On 12/05/2017 10:13, Blumentrath, Stefan wrote:

Thanks Yann! Great news!
I will use the modules on tilted digital camera images (from wildlife camera 
traps).
But I can check if we have some good aerial photos somewere in a drawer….

Cheers
Stefan


From: Yann Chemin [mailto:dr.yann.che...@gmail.com]
Sent: fredag 12. mai 2017 09.19
To: Blumentrath, Stefan ; Luca Delucchi 

Cc: GRASS-dev ; Manan Singh 
Subject: Re: [GRASS-dev] Introduction for GSoC 2017

Hi Stefan,
the first throw is there in SVN, just update and recompile.
The two GUI are g.gui.iphoto2image and g.gui.iimage2target.
Finalization of the names are also open to suggestion, so far I just 
concatenated the old module names in v6 and added to g.gui.
Please test and if you have a scanned aerial photo with good info about camera 
and XYZ of fiducials, please throw it my side too so I can validate/finalize 
the output accuracy.
Cheers,
Yann

--
Yann Chemin
Planetary Scientist
Latest: https://peerj.com/preprints/2124/
-
JRC: 
https://www.thefreelibrary.com/3628+-+CANHEMON+Remote+sensing+based+forest+canopy+health+monitoring...-a0453990005
OSGeo: https://wiki.osgeo.org/wiki/Open_Monitoring_Systems_Working_Group

On 21 March 2017 at 08:31, Yann 
mailto:dr.yann.che...@gmail.com>> wrote:
Hi Stefan,

please also launch the main menu: i.ortho.photo at the command line, it tends 
to be erratic.

(careful the i.photo.2target is not yet existing)

Ciao,
Yann



On 21/03/2017 08:04, Blumentrath, Stefan wrote:
Hi Yann,

Thanks so much! Very cool!
I will test the photo2image GUI tool ASAP!

Cheers
Stefan

From: Yann Chemin 
[mailto:dr.yann.che...@gmail.com]
Sent: tirsdag 21. mars 2017 07.56
To: Luca Delucchi mailto:lucadel...@gmail.com>>
Cc: GRASS-dev mailto:grass-dev@lists.osgeo.org>>; Manan Singh 
mailto:mananpa...@gmail.com>>; Blumentrath, Stefan 
mailto:stefan.blumentr...@nina.no>>
Subject: Re: [GRASS-dev] Introduction for GSoC 2017


Hi all,

I have added to SVN trunk one of the two missing gui modules of i.ortho.photo 
recently: g.gui.iphoto2image at the command line launches it. All other modules 
but one are operational (still not fully tested though), I have already started 
working on the last one, which is also GUI based.

Cheers,
Yann

On Mar 20, 2017 5:25 PM, "Luca Delucchi" 
mailto:lucadel...@gmail.com>>>
 wrote:
On 18 March 2017 at 08:30, Blumentrath, Stefan
mailto:stefan.blumentr...@nina.no>>>
 wrote:
Hi,
Hi,
Could also porting (competition) of the i.ortho.photo modules (and here
especially the missing GUI modules around it) to GRASS 7 become scope of
this GSoC idea?
I fully support this idea, GUI for i.ortho.photo is the most important
loss from GRASS 6

Cheers,

Stefan

--
ciao
Luca

www.lucadelu.org
___
grass-dev mailing list
grass-dev@lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/grass-dev




___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3365: r.in.gdal: crash on large dataset

2017-07-04 Thread GRASS GIS

#3365: r.in.gdal: crash on large dataset
--+-
  Reporter:  neteler  |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.2.2
 Component:  Raster   |Version:  svn-releasebranch72
Resolution:   |   Keywords:  r.in.gdal
   CPU:  x86-64   |   Platform:  Linux
--+-

Comment (by neteler):

 Replying to [comment:4 mmetz]:
 ...
 > Maybe you need to recompile GRASS with -g in order to replace "???" with
 some real information.

 My bad, I was calling the wrong grass72 version without debugging symbols
 in...

 > I could reproduce the problem, the segfault is caused by stack overflow
 because alloca allocates memory in the stack, while malloc and calloc
 allocate memory in the heap. I could fix the segfault by replacing
 G_alloca()/G_freea() with G_malloc()/G_free(). Please try the attached
 patch against lib/raster/.

 Thanks! r.in.gdal now runs, no more immediate crash. In some hours the
 dataset may be imported, I'll report.

--
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3365: r.in.gdal: crash on large dataset

2017-07-04 Thread GRASS GIS

#3365: r.in.gdal: crash on large dataset
--+-
  Reporter:  neteler  |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.2.2
 Component:  Raster   |Version:  svn-releasebranch72
Resolution:   |   Keywords:  r.in.gdal
   CPU:  x86-64   |   Platform:  Linux
--+-

Comment (by neteler):

 Works!

 {{{
 # note: mounted disk connected from a different server via NFS
 GRASS 7.2.2svn (latlong) > date ; r.in.gdal
 
input=/mnt/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015/loss_global_forest_cover_loss_2000_2015.vrt
 output=loss_global_forest_cover_loss_2000_2015 ; date
 Wed Jul  5 00:22:36 CEST 2017
 Proceeding with import of 1 raster bands...
 Importing raster map ...
  100%
 Wed Jul  5 03:33:54 CEST 2017

 GRASS 7.2.2svn (latlong): > g.region
 raster=loss_global_forest_cover_loss_2000_2015 -p
 projection: 3 (Latitude-Longitude)
 zone:   0
 datum:  wgs84
 ellipsoid:  wgs84
 north:  80N
 south:  60S
 west:   180W
 east:   180E
 nsres:  0:00:00.9
 ewres:  0:00:00.9
 rows:   56
 cols:   144
 cells:  8064

 # d.rast shows this easily (20sec over ssh connected d.mon wx0)
 }}}

 Looks like a very useful patch :) thanks.

--
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3365: r.in.gdal: crash on large dataset

2017-07-04 Thread GRASS GIS

#3365: r.in.gdal: crash on large dataset
--+-
  Reporter:  neteler  |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.2.2
 Component:  Raster   |Version:  svn-releasebranch72
Resolution:   |   Keywords:  r.in.gdal
   CPU:  x86-64   |   Platform:  Linux
--+-

Comment (by mmetz):

 Replying to [comment:6 neteler]:
 > Works!
 >
 > [...]
 >
 > Looks like a very useful patch :) thanks.

 Patch applied to trunk and relbr72 in r71241,2.

--
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3365: r.in.gdal: crash on large dataset

Re: [GRASS-dev] [GRASS GIS] #3365: r.in.gdal: crash on large dataset

Re: [GRASS-dev] [GRASS GIS] #3365: r.in.gdal: crash on large dataset

Re: [GRASS-dev] [GRASS GIS] #3365: r.in.gdal: crash on large dataset

Re: [GRASS-dev] Introduction for GSoC 2017

Re: [GRASS-dev] [GRASS GIS] #3365: r.in.gdal: crash on large dataset

Re: [GRASS-dev] [GRASS GIS] #3365: r.in.gdal: crash on large dataset

Re: [GRASS-dev] [GRASS GIS] #3365: r.in.gdal: crash on large dataset

8 matches

Site Navigation

Mail list logo

Footer information