Re: [gdal-dev] gtiff with internal mask
So... now I 've got issues building the overviews. Gdaladdo loses the transparency so I get to see black areas where I was supposed to see nothing. When I zoom in black disappears. Any known way to overcome this? Thanks, Duarte -Mensagem original- De: Duarte Carreira Enviada: quarta-feira, 17 de Julho de 2013 19:41 Para: 'Even Rouault' Cc: gdal-dev@lists.osgeo.org Assunto: RE: [gdal-dev] gtiff with internal mask Well, you're right... I messed up recreating the footsteps... So, cleaning up: 1) create a rgba vrt using gdalwarp to cut the original mosaic with a shapefile gdalwarp -multi -wm 480 --config GDAL_CACHEMAX 312 -co alpha=yes -dstalpha -cutline shapes\index_diss.shp -of vrt originalmosaic.vrt test_rgba_uncomp.vrt 2) create a masked uncompressed vrt using gdal_translate (because I couldn't compress right away, with errors of missing StripOffsets field) gdal_translate -b 1 -b 2 -b 3 -mask 4 -co alpha=no -co photometric=rgb -co interleave=pixel -co tiled=yes --config GDAL_TIFF_INTERNAL_MASK YES -of vrt test_rgba_uncomp.vrt testmask_uncomp.vrt 3) create final masked compressed tiff using gdal_translate gdal_translate -co alpha=no -co photometric=ycbcr -co interleave=pixel -co tiled=yes -co compress=jpeg --config GDAL_TIFF_INTERNAL_MASK YES testmask_uncomp.vrt test_finalmask.tif The power of VRT amazes me! Also regarding sizes: I still got the same ratio of 8x smaller. To use this in QGIS you still have to create a rgba vrt: 4) gdal_translate -of VRT -b 1 -b 2 -b 3 -b mask test_finalmask.tif test_finalmask_qgis.vrt (couldn't get the 4ª band to be recognized as an alpha band though) So I think now it's correct... I am going to apply this to a real-case mosaic and report back... Duarte -Mensagem original- De: Even Rouault [mailto:even.roua...@mines-paris.org] Enviada: quarta-feira, 17 de Julho de 2013 13:45 Para: Duarte Carreira Cc: gdal-dev@lists.osgeo.org Assunto: Re: [gdal-dev] gtiff with internal mask Le mercredi 17 juillet 2013 14:23:29, Duarte Carreira a écrit : Hi Even. Thanks so much for your tip! It works. I did have to specify I did not want an alpha band when cutting with the shapefile: gdalwarp -multi -wm 480 --config GDAL_CACHEMAX 256 -co photometric=ycbcr -co compress=jpeg -co alpha=no --config GDAL_TIFF_INTERNAL_MASK YES -cutline shapes\index_diss.shp vrtini.vrt testmask.tif Hum, what I don't understand is how the above will produce a mask band in the output file... Does gdalinfo on testmask.tif show something like : Band 1 Block= Type=Byte, ColorInterp=Red Mask Flags: PER_DATASET Band 2 Block= Type=Byte, ColorInterp=Green Mask Flags: PER_DATASET Band 3 Block=... Type=Byte, ColorInterp=Blue Mask Flags: PER_DATASET I think you would need to do the above in 2 steps: 1) gdalwarp to an uncompressed RGBA TIFF 2) gdal_translate to turn the RGBA into a YCbCr JPEG-compressed GTiff + mask band Seems arcgis recognizes the masked tiff, but is much much slower displaying than using a normal alpha band. Perhaps adding -co TILED=YES will help ? (random guess) QGIS does need the vrt trick but then is as fast as usual. Also, the size is must smaller when using an internal mask: from a 219MB rgba,band interleaved,jpegd image to a 27MB ycbcr,pixel interleaved,jpegd masked tiff (8x smaller). Unless I missed something... Not completely surprising although more important than I would have expected. YCbCr compression usually gives a 2x to 3x compression bonus, and due to the usual nature of mask bands, the 1bit deflate compression of the mask band will give better results (both visually and in size) that the 8bit JPEG compression of the alpha band. Thanks again! Duarte -Mensagem original- De: Even Rouault [mailto:even.roua...@mines-paris.org] Enviada: terça-feira, 16 de Julho de 2013 18:58 Para: gdal-dev@lists.osgeo.org Cc: Duarte Carreira Assunto: Re: [gdal-dev] gtiff with internal mask Le mardi 16 juillet 2013 12:29:37, Duarte Carreira a écrit : I'm trying to create gtiffs with transparent areas where there are voids between my original images. I have successfully done it by using an alpha band but this process created 3x larger images. I think partly because of an additional 4th band and partly because I cannot use ycbcr to compress with jpeg. So I thought of using internal masks expecting to get smaller images but cannot get it to work. When converting from the rgba image using -mask 4 I cannot see the transparency and get black areas: gdal_translate -b 1 -b 2 -b 3 -ma sk 4 --config GDAL_TIFF_INTERNAL_MASK YES test_alpha.tif test_internalmask.tif I tried looking at the images using qgis and paint.net. I know QGIS doesn't use the GDALGetMaskBand() API, so no real surprise (that would likely be a possible QGIS enhancement). And paint.net probably doesn't support this feature of the TIFF format either. MapServer does support mask
Re: [gdal-dev] GRIB2 to NetCDF - Large areas of data missing
Etienne, This workaround seems to have solved my problem, although I needed to increase the value from 1000 to 2000 in some instances (some of the GRIB2 messages worked with 1000, others didn't). So I am now using: gdalwarp -wm 2000 --config GDAL_CACHEMAX 2000 -of netcdf -srcnodata .0 -dstnodata .0 -t_srs EPSG:4326 conus-wspd-01.bin conus-wspd-01.nc Thanks very much - I will update the trac ticket. Jason jason.green...@noaa.gov On Wed, Jul 17, 2013 at 6:49 PM, Etienne Tourigny etourigny@gmail.comwrote: You probably are experiencing a known bug, when the warping operation needs more memory than the warp cache. Try setting adding the following to the gdalwarp command (if it doesn't work try a larger value). --config GDAL_CACHEMAX 1000 -wm 1000 Also please update the ticket with any new information http://trac.osgeo.org/gdal/ticket/4484 cheers Etienne On Wed, Jul 17, 2013 at 5:31 PM, Jason Greenlaw - NOAA Affiliate jason.green...@noaa.gov wrote: I am experiencing strange behavior when reprojecting/converting a GRIB2 file to NetCDF using gdalwarp, where the resulting NetCDF file is missing lots of data (or possibly being set incorrectly to NoData) west of a certain longitude. However, if I instead reproject from GRIB2 to an intermediate format (e.g. GeoTIFF), then convert to NetCDF using gdal_translate, the result is correct. The source projection is the weather service's custom lambert conformal, and the destination is WGS84. I'm using the following command: gdalwarp -wm 256 -of netcdf -srcnodata .0 -dstnodata .0 -t_srs EPSG:4326 conus-wspd-01.bin conus-wspd-01.nc GDAL versions tested were 1.9.2 and 1.10. Screenshot of original GRIB2 file (first GRIB message) as displayed in ArcMap: http://jetstreak.unh.edu/gdal/full.png Screenshot of resulting NetCDF file (variable Band1) as displayed in ncBrowse: http://jetstreak.unh.edu/gdal/cropped.png Original GRIB2 file (6.3mb): http://jetstreak.unh.edu/gdal/conus-wspd-01.bin Resulting NetCDF file (171mb): http://jetstreak.unh.edu/gdal/conus-wspd-01.nc Anyone run into a similar problem or have any ideas for how to resolve this? Thanks, Jason ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] using gdalwarp with a new projection in proj.4
Hi Frank, thanks for the reply. I did as you suggested and gdalsrsinfo acts accordingly. gt;gt;gt; gdalsrsinfo '+proj=rhealpix -I +lon_0=0 +a=1 +ellps=WGS84 +npole=0 +spole=0 +wktext' PROJ.4 : '+proj=rhealpix -I +lon_0=0 +a=1 +ellps=WGS84 +npole=0 +spole=0 +wktext' OGC WKT : PROJCS[unnamed, GEOGCS[WGS 84, DATUM[unknown, SPHEROID[WGS84,6378137,298.257223563]], PRIMEM[Greenwich,0], UNIT[degree,0.0174532925199433]], PROJECTION[custom_proj4], EXTENSION[PROJ4,+proj=rhealpix -I +lon_0=0 +a=1 +ellps=WGS84 +npole=0 +spole=0 +wktext]] gt;gt;gt; I then tried the same with gdalwarp (on one of the examples file that comes with GDAL) gt;gt;gt; gdalwarp -s_srs EPSG:26711 -t_srs '+proj=rhealpix -I +lon_0=0 +a=1 +ellps=WGS84 +npole=0 +spole=0 +wktext' ~/data/utm_orig.tif rhealpix.tif Processing input file /home/markus.mueller/data/utm_orig.tif. ERROR 6: Failed to initialize PROJ.4 with `+proj=rhealpix -I +lon_0=0 +a=1 +ellps=WGS84 +npole=0 +spole=0 +wktext'. 0...10...20...30...40...50...60...70...80...90...100 - done. gt;gt;gt; It produces an output file though. Should I be concerned about the created error or can I ignore it? Many thanks, Markus On Tue, 16 Jul 2013 23:55:05 -0700 Frank Warmerdam lt;warmer...@pobox.comgt; wrote Michael, Healpix is not directly supported by OGRSpatialReference - the core of the coordinate system machinery for GDAL/OGR. However, you can request GDAL to treat this as a PROJ.4 extension to WKT by adding the +wktext flag. I would add that the -E and -f flags are options to the cs2cs commandline program, and not part of a PROJ.4 coordinate system and so should not be included. The result is something like this: gdalsrsinfo '+proj=rhealpix -I +lon_0=0 +a=1 +ellps=WGS84 +npole=0 +spole=0 +wktext' PROJ.4 : '+proj=rhealpix -I +lon_0=0 +a=1 +ellps=WGS84 +npole=0 +spole=0 +wktext' OGC WKT : PROJCS[unnamed, GEOGCS[WGS 84, DATUM[unknown, SPHEROID[WGS84,6378137,298.257223563]], PRIMEM[Greenwich,0], UNIT[degree,0.0174532925199433]], PROJECTION[custom_proj4], EXTENSION[PROJ4,+proj=rhealpix -I +lon_0=0 +a=1 +ellps=WGS84 +npole=0 +spole=0 +wktext]] Best regards, Frank On Tue, Jul 16, 2013 at 6:48 PM, Michael Speth lt;spe...@landcareresearch.co.nzgt; wrote: Greetings, I am unable to use the HEALpix and rHEALPix projections which have been added about a year ago to proj.4 in gdalwarp. Is there a configuration file for gdal that needs to be updated for this projection? Here is the command in question that fails: gdalwarp -s_srs EPSG:26711 -t_srs '+proj=rhealpix -f '%.2f' -I +lon_0=0 +a=1 +ellps=WGS84 +npole=0 +spole=0 -E' ~/data/utm_orig.tif rhealpix.tif The error message “Translating source or target SRS failed:” Thanks -- Michael Speth Research Systems Developer Landcare Research Please consider the environment before printing this email Warning: This electronic message together with any attachments is confidential. If you receive it in error: (i) you must not read, use, disclose, copy or retain it; (ii) please contact the sender immediately by reply email and then delete the emails. The views expressed in this email may not be those of Landcare Research New Zealand Limited. http://www.landcareresearch.co.nz ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev -- ---+-- I set the clouds in motion - turn up | Frank Warmerdam, warmer...@pobox.com light and sound - activate the windows | http://pobox.com/~warmerdam and watch the world go round - Rush| Geospatial Software Developer ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] using gdalwarp with a new projection in proj.4
Le jeudi 18 juillet 2013 23:05:45, Markus Müller a écrit : Hi Frank, thanks for the reply. I did as you suggested and gdalsrsinfo acts accordingly. gt;gt;gt; gdalsrsinfo '+proj=rhealpix -I +lon_0=0 +a=1 +ellps=WGS84 +npole=0 +spole=0 +wktext' PROJ.4 : '+proj=rhealpix -I +lon_0=0 +a=1 +ellps=WGS84 +npole=0 +spole=0 +wktext' OGC WKT : PROJCS[unnamed, GEOGCS[WGS 84, DATUM[unknown, SPHEROID[WGS84,6378137,298.257223563]], PRIMEM[Greenwich,0], UNIT[degree,0.0174532925199433]], PROJECTION[custom_proj4], EXTENSION[PROJ4,+proj=rhealpix -I +lon_0=0 +a=1 +ellps=WGS84 +npole=0 +spole=0 +wktext]] gt;gt;gt; I then tried the same with gdalwarp (on one of the examples file that comes with GDAL) gt;gt;gt; gdalwarp -s_srs EPSG:26711 -t_srs '+proj=rhealpix -I +lon_0=0 +a=1 +ellps=WGS84 +npole=0 +spole=0 +wktext' ~/data/utm_orig.tif rhealpix.tif Processing input file /home/markus.mueller/data/utm_orig.tif. ERROR 6: Failed to initialize PROJ.4 with `+proj=rhealpix -I +lon_0=0 +a=1 +ellps=WGS84 +npole=0 +spole=0 +wktext'. 0...10...20...30...40...50...60...70...80...90...100 - done. gt;gt;gt; It produces an output file though. Should I be concerned about the created error or can I ignore it? Markus, Yes, you shouldn't ignore this error. Ideally gdalwarp shouldn't complete when such an error occurs. I'll look at that. First, you should also remove the '-I' parameter. It is a argument of the proj command line utility, not a parameter of a PROJ.4 projection string. Secondly, looking at PROJ history ( http://trac.osgeo.org/proj/log/trunk/proj/src/PJ_healpix.c and http://trac.osgeo.org/proj/log/tags/4.8.0?rev=2377 ), I see that this projection was only introduced in PROJ 4.8.0, so check your PROJ.4 version. I also see that a few fixes have been pushed after the 4.8.0 release, so if you run into bugs, you might need to compile the trunk version of the svn repository. Best regards, Even -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] using gdalwarp with a new projection in proj.4
Hi Evan, removing the '-I' parameter did the trick. For completeness sake, here the gdalwarp call that worked for me: gdalwarp -s_srs EPSG:26711 -t_srs '+proj=rhealpix +lon_0=0 +a=1 +ellps=WGS84 +npole=0 +spole=0 +wktext' ~/data/utm_orig.tif rhealpix.tif Thanks a lot for the help. Markus On Thu, 18 Jul 2013 14:44:28 -0700 Even Rouaultlt;even.roua...@mines-paris.orggt; wrote Le jeudi 18 juillet 2013 23:05:45, Markus Müller a écrit : gt; Hi Frank, gt; gt; thanks for the reply. I did as you suggested and gdalsrsinfo acts gt; accordingly. gt; gt; gt; amp;gt;amp;gt;amp;gt; gt; gdalsrsinfo '+proj=rhealpix -I +lon_0=0 +a=1 +ellps=WGS84 +npole=0 +spole=0 gt; +wktext' gt; gt; gt; PROJ.4 : '+proj=rhealpix -I +lon_0=0 +a=1 +ellps=WGS84 +npole=0 +spole=0 gt; +wktext' gt; gt; gt; OGC WKT : gt; PROJCS[unnamed, gt; GEOGCS[WGS 84, gt; DATUM[unknown, gt; SPHEROID[WGS84,6378137,298.257223563]], gt; PRIMEM[Greenwich,0], gt; UNIT[degree,0.0174532925199433]], gt; PROJECTION[custom_proj4], gt; EXTENSION[PROJ4,+proj=rhealpix -I +lon_0=0 +a=1 +ellps=WGS84 gt; +npole=0 +spole=0 +wktext]] gt; gt; amp;gt;amp;gt;amp;gt; gt; gt; gt; I then tried the same with gdalwarp (on one of the examples file that comes gt; with GDAL) gt; gt; gt; amp;gt;amp;gt;amp;gt; gt; gdalwarp -s_srs EPSG:26711 -t_srs '+proj=rhealpix -I +lon_0=0 +a=1 gt; +ellps=WGS84 +npole=0 +spole=0 +wktext' ~/data/utm_orig.tif rhealpix.tif gt; Processing input file /home/markus.mueller/data/utm_orig.tif. gt; ERROR 6: Failed to initialize PROJ.4 with `+proj=rhealpix -I +lon_0=0 +a=1 gt; +ellps=WGS84 +npole=0 +spole=0 +wktext'. gt; 0...10...20...30...40...50...60...70...80...90...100 - done. gt; gt; amp;gt;amp;gt;amp;gt; gt; gt; gt; It produces an output file though. Should I be concerned about the created gt; error or can I ignore it? Markus, Yes, you shouldn't ignore this error. Ideally gdalwarp shouldn't complete when such an error occurs. I'll look at that. First, you should also remove the '-I' parameter. It is a argument of the proj command line utility, not a parameter of a PROJ.4 projection string. Secondly, looking at PROJ history ( http://trac.osgeo.org/proj/log/trunk/proj/src/PJ_healpix.c and http://trac.osgeo.org/proj/log/tags/4.8.0?rev=2377 ), I see that this projection was only introduced in PROJ 4.8.0, so check your PROJ.4 version. I also see that a few fixes have been pushed after the 4.8.0 release, so if you run into bugs, you might need to compile the trunk version of the svn repository. Best regards, Even -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] A new ogr driver
Hi all, I'm planning to submit a new driver for accessing Walk files, a geospatial data format developed by Walkinfo Tech. mainly for land surveying, evaluation, planning, checking and data analysis in China. Walkinfo Tech. is a China-based company founded since 1999, origignally focused on land surveying software research and development. Currently we have our own geographic information platform WalkGIS, as well as scores of GIS desktop, mobile products based on it, including WalkISurvey, WalkISurface, WalkCheck, WalkFu and so on. The company is also the largest strategic partner with Esri ArcGIS in China. The WalkGIS applications use GDAL/OGR as part of its base libaries. The purpose of developing the Walk driver on GDAL/OGR is to allow other GIS products such as ArcGIS to access our data more easily and conveniently. The driver I am working on is already able to get the information such as layers, features, geometries, and spatial references from Walk files. Can I get the permission to submit it to GDAL project and share it with other GDAL developers? Best Regards, Xian -- View this message in context: http://osgeo-org.1560.x6.nabble.com/gdal-dev-A-new-ogr-driver-tp5067479.html Sent from the GDAL - Dev mailing list archive at Nabble.com. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
