[gdal-dev] -srcnodata in gdalwarp and gdalbuildvrt

2018-03-28 Thread Richard Greenwood 
I have a question that is probably more "user" than "developer". Is there
different place that I should post my question?

I don't think that I understand -srcnodata. I assume that -srcnodata "0 0
0" on a 3 band image means that all 3 bands have to be 0 to be "no data"
but I think what I'm seeing is that if any to the 3 bands is 0 then that
pixel is considered to be "no data". Am I mis-understanding -srcnodata?

Thanks
-- 
Richard W. Greenwood, PLS
www.greenwoodmap.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] FW: gdal_merge.py

2018-03-28 Thread Even Rouault 
On mercredi 28 mars 2018 17:58:37 CEST Raphael Das Gupta wrote:
> On 28.03.2018 16:31, Marcos Dione wrote:
> > in my opinion, gdal_merge should be used only if the merged version
> > 
> > will be used for more things or if you're going to use it with something
> > that doesn't know how to read the .vrt files (i.e., is not using gdal).
> 
> Even then it seems to be faster to first create a (temporary) .vrt file
> and then convert that to something else (thereby materializing the
> content) with e.g. gdaltranslate. (And then throw away the temporary
> .vrt file.)
> 
> In my (and others', see e.g.
> https://stackoverflow.com/a/22602542/674064) experience, the combined
> runtime of gdalbuildvrt and gdaltranslate will be shorter than that of
> gdal_merge.py would be. (And the quality of the result might also be
> better.)
> 
> Though I don't know whether that was just the case for the examples I
> encountered so far or whether that's always the case. Also, I don't know
> whether there are use-cases of gdal_merge.py that gdalbuildvrt +
> gdaltranslate (or + gdalwarp) can't handle.
> 
> If not, maybe gdal_merge.py should be reimplemented as a wrapper of
> gdalbuildvrt + gdaltranslate, so that the obvious (and most convenient)
> choice is also the best one?

My summary of different solutions:

* gdalbuildvrt + gdaltranslate:
   - pros:
  + efficient regarding compression of output dataset
  + work with arbitrarily large input  datasets.
  + several resampling methods
   - cons:
  + does not handle as one would expect overlapping input datasets 
regarding masking / nodata / alpha.
  + cannot update an output dataset
  + all input datasets must have same projection


* gdalwarp:
   - pros:
+ handles properly nodata / mask / alpha in input datasets (with alpha 
blending for alpha)
+ several resampling methods.
+ can deal with input datasets in different projections
+ can update an existing output dataset
   - cons:
+ can lead to datasets larger than needed with compression methods 
(unless -wo OPTIMIZE_SIZE=TRUE).
+ May be slower than other methods.


* gdal_merge.py:
 - pros:
 + handles nodata / mask in input dataset (alpha seen as binary mask)
 + can update an existing output dataset
- cons:
 + only nearest resampling
 + cannot deal with arbitrarily large input datasets (ingests each 
input dataset fully in memory)
 + not friendly for compressed output
 + all input datasets must have same projection


So indeed gdal_merge.py could potentially be emulated with 
gdalbuildvrt+gdal_translate if there's no
overlap between input datasets or if they don't have nodata/mask, and otherwise 
on gdalwarp.

Exercice left to readers :-)

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] FW: gdal_merge.py

2018-03-28 Thread Raphael Das Gupta 
On 28.03.2018 16:31, Marcos Dione wrote:
> in my opinion, gdal_merge should be used only if the merged version
> will be used for more things or if you're going to use it with something
> that doesn't know how to read the .vrt files (i.e., is not using gdal).
Even then it seems to be faster to first create a (temporary) .vrt file
and then convert that to something else (thereby materializing the
content) with e.g. gdaltranslate. (And then throw away the temporary
.vrt file.)

In my (and others', see e.g.
https://stackoverflow.com/a/22602542/674064) experience, the combined
runtime of gdalbuildvrt and gdaltranslate will be shorter than that of
gdal_merge.py would be. (And the quality of the result might also be
better.)

Though I don't know whether that was just the case for the examples I
encountered so far or whether that's always the case. Also, I don't know
whether there are use-cases of gdal_merge.py that gdalbuildvrt +
gdaltranslate (or + gdalwarp) can't handle.

If not, maybe gdal_merge.py should be reimplemented as a wrapper of
gdalbuildvrt + gdaltranslate, so that the obvious (and most convenient)
choice is also the best one?

Cheers,
Raphael
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL can't open MBTiles sample from the MapBox site

2018-03-28 Thread Rahkonen Jukka (MML) 
Hi Even,

Thanks, after updating my GDAL to a more recent 2.3.0dev version ogrinfo indeed 
shows that information.

-Jukka-

Lähettäjä: Even Rouault 
Lähetetty: 28. maaliskuuta 2018 17:47
Vastaanottaja: gdal-dev@lists.osgeo.org
Kopio: Rahkonen Jukka (MML)
Aihe: Re: [gdal-dev] GDAL can't open MBTiles sample from the MapBox site

On mercredi 28 mars 2018 14:28:14 CEST Rahkonen Jukka (MML) wrote:
> Hi,
> On page https://www.mapbox.com/help/define-mbtiles/ there is a link into
> https://www.mapbox.com/help/data/trails.mbtiles
>
> My GDAL 2.3.0dev does list MBTiles as a supported format but it can't still
> open this database
>
> gdalinfo trails.mbtiles
> ERROR 4: `trails.mbtiles' not recognized as a supported file format.
> gdalinfo failed - unable to open 'trails.mbtiles'.

Jukka,

Yep, there's no raster in that file, but vector instead.
So:

$ ogrinfo trails.mbtiles -al -so
INFO: Open of `trails.mbtiles'
  using driver `MBTiles' successful.
Metadata:
  ZOOM_LEVEL=14
  center=-113.8348,48.6402,9
  format=pbf
  maxzoom=14
  minzoom=0

Layer name: multilinestring-glacier_trails
Geometry: Unknown (any)
Feature Count: 1409
Extent: (-12743581.355705, 6149206.051486) - (-12604160.216112,
6276397.266552)
Layer SRS WKT:
PROJCS["WGS 84 / Pseudo-Mercator",
GEOGCS["WGS 84",
DATUM["WGS_1984",
SPHEROID["WGS 84",6378137,298.257223563,
AUTHORITY["EPSG","7030"]],
AUTHORITY["EPSG","6326"]],
PRIMEM["Greenwich",0,
AUTHORITY["EPSG","8901"]],
UNIT["degree",0.0174532925199433,
AUTHORITY["EPSG","9122"]],
AUTHORITY["EPSG","4326"]],
PROJECTION["Mercator_1SP"],
PARAMETER["central_meridian",0],
PARAMETER["scale_factor",1],
PARAMETER["false_easting",0],
PARAMETER["false_northing",0],
UNIT["metre",1,
AUTHORITY["EPSG","9001"]],
AXIS["X",EAST],
AXIS["Y",NORTH],
EXTENSION["PROJ4","+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0
+x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext  +no_defs"],
AUTHORITY["EPSG","3857"]]
mvt_id: Integer64 (0.0)
Avg_Slope: Real (0.0)
CLASS: Real (0.0)
DESC_SEG: String (0.0)
LENGTH: Real (0.0)
MILES: Real (0.0)
NAME: String (0.0)
ROUTE_NO: Real (0.0)
SEG_NO: Real (0.0)
evaluating: String (0.0)

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL can't open MBTiles sample from the MapBox site

2018-03-28 Thread Even Rouault 
On mercredi 28 mars 2018 14:28:14 CEST Rahkonen Jukka (MML) wrote:
> Hi,
> On page https://www.mapbox.com/help/define-mbtiles/ there is a link into
> https://www.mapbox.com/help/data/trails.mbtiles
> 
> My GDAL 2.3.0dev does list MBTiles as a supported format but it can't still
> open this database
> 
> gdalinfo trails.mbtiles
> ERROR 4: `trails.mbtiles' not recognized as a supported file format.
> gdalinfo failed - unable to open 'trails.mbtiles'.

Jukka,

Yep, there's no raster in that file, but vector instead.
So:

$ ogrinfo trails.mbtiles -al -so
INFO: Open of `trails.mbtiles'
  using driver `MBTiles' successful.
Metadata:
  ZOOM_LEVEL=14
  center=-113.8348,48.6402,9
  format=pbf
  maxzoom=14
  minzoom=0

Layer name: multilinestring-glacier_trails
Geometry: Unknown (any)
Feature Count: 1409
Extent: (-12743581.355705, 6149206.051486) - (-12604160.216112, 
6276397.266552)
Layer SRS WKT:
PROJCS["WGS 84 / Pseudo-Mercator",
GEOGCS["WGS 84",
DATUM["WGS_1984",
SPHEROID["WGS 84",6378137,298.257223563,
AUTHORITY["EPSG","7030"]],
AUTHORITY["EPSG","6326"]],
PRIMEM["Greenwich",0,
AUTHORITY["EPSG","8901"]],
UNIT["degree",0.0174532925199433,
AUTHORITY["EPSG","9122"]],
AUTHORITY["EPSG","4326"]],
PROJECTION["Mercator_1SP"],
PARAMETER["central_meridian",0],
PARAMETER["scale_factor",1],
PARAMETER["false_easting",0],
PARAMETER["false_northing",0],
UNIT["metre",1,
AUTHORITY["EPSG","9001"]],
AXIS["X",EAST],
AXIS["Y",NORTH],
EXTENSION["PROJ4","+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 
+x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext  +no_defs"],
AUTHORITY["EPSG","3857"]]
mvt_id: Integer64 (0.0)
Avg_Slope: Real (0.0)
CLASS: Real (0.0)
DESC_SEG: String (0.0)
LENGTH: Real (0.0)
MILES: Real (0.0)
NAME: String (0.0)
ROUTE_NO: Real (0.0)
SEG_NO: Real (0.0)
evaluating: String (0.0)

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] FW: gdal_merge.py

2018-03-28 Thread Marcos Dione 
On Wed, Mar 28, 2018 at 02:28:57PM +0200, Raphael Das Gupta wrote:
> On 27.03.2018 22:56, Ian Reese wrote:
> > Maybe this is a bug or am I using gdal_merge.py improperly.
> I've recently become unsure whether (and when) gdal_merge.py should be
> used at all, or whether gdalbuildvrt followed by gdaltranslate or
> gdalwarp is always preferable. (Also asked here:
> https://gis.stackexchange.com/q/272344/51574 )

in my opinion, gdal_merge should be used only if the merged version
will be used for more things or if you're going to use it with something
that doesn't know how to read the .vrt files (i.e., is not using gdal).

.vrt files have the advantage that are fats to generate, are easy to
update if one of the sources is changed, and it could be faster if the
sources are compressed and the target is thought to be compressed too (I
have no numbers to back this up, it's just a hunch).

Cheers,

-- Marcos.

-- 
(Not so) Random fortune:
The technology industry sees itself as in rebellion against corporate
America: not corrupt, not buttoned-up, not empty. In fact, a tech company
can be as corrupt, soulless, and empty as any corporation, but being
unprofessional helps us maintain the belief that we are somehow different
from Wall Street.
-- Shanley Kane
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GDAL can't open MBTiles sample from the MapBox site

2018-03-28 Thread Rahkonen Jukka (MML) 
Hi,
On page https://www.mapbox.com/help/define-mbtiles/ there is a link into 
https://www.mapbox.com/help/data/trails.mbtiles

My GDAL 2.3.0dev does list MBTiles as a supported format but it can't still 
open this database

gdalinfo trails.mbtiles
ERROR 4: `trails.mbtiles' not recognized as a supported file format.
gdalinfo failed - unable to open 'trails.mbtiles'.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Extension based file identification in drivers

2018-03-28 Thread Even Rouault 
On mercredi 28 mars 2018 15:05:34 CEST Andrew C Aitchison wrote:
> On Tue, 27 Mar 2018, Even Rouault wrote:
> >  The identification logic of drivers is supposed to operate fast
> > 
> > (extension based, or on the content of the first kilobyte of the header),
> 
> If I understand correctly Frank Warmerdam did not favour extension based
> file identification:
> https://lists.osgeo.org/pipermail/gdal-dev/2013-February/035516.html
>GDAL is based on the idea that file formats are discovered by
>inspecting file contents rather than based on extension and I don't
>want to make it too easy for applications to ignore this principle
>and revert to extensions.
> 
> I have therefore tried to avoid using extension to identify file type.
> 
> Has thinking on this moved on in the last five years ?

No. Identification per file content is still the primary identification 
method. There are a few drivers where this is not possible due to the file 
format (absence of header with unambiguous signature), and thus extension is 
used as a fallback.

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Extension based file identification in drivers

2018-03-28 Thread Andrew C Aitchison 

On Tue, 27 Mar 2018, Even Rouault wrote:

 The identification logic of drivers is supposed to operate fast 
(extension based, or on the content of the first kilobyte of the header),


If I understand correctly Frank Warmerdam did not favour extension based 
file identification:

https://lists.osgeo.org/pipermail/gdal-dev/2013-February/035516.html
  GDAL is based on the idea that file formats are discovered by
  inspecting file contents rather than based on extension and I don't
  want to make it too easy for applications to ignore this principle
  and revert to extensions.

I have therefore tried to avoid using extension to identify file type.

Has thinking on this moved on in the last five years ?

--
Andrew C. Aitchison Cambridge, UK
and...@aitchison.me.uk
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] gdal_merge.py

2018-03-28 Thread Even Rouault 
On lundi 12 mars 2018 05:55:10 CEST Ian Reese wrote:
> No matter which variation of merge I run, those regions that once contained
> 'nodata' or '-' are converted to '0' and the nodata value is not
> retained after the merge.  When I load the test_merge_wnodata.tif into
> QGIS, those regions once containing 'nodata', are now filled with 0 and
> rendered as such.

gdal_merge.py -n - -a_nodata - -o test_merge_wnodata.tif *.tif
or even
gdal_merge.py -a_nodata - -o test_merge_wnodata.tif *.tif
work fine for me with recent enough GDAL (2.2)

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] FW: gdal_merge.py

2018-03-28 Thread Raphael Das Gupta 
Hi Ian, heya GDALers

On 27.03.2018 22:56, Ian Reese wrote:
> Maybe this is a bug or am I using gdal_merge.py improperly.
I've recently become unsure whether (and when) gdal_merge.py should be
used at all, or whether gdalbuildvrt followed by gdaltranslate or
gdalwarp is always preferable. (Also asked here:
https://gis.stackexchange.com/q/272344/51574 )

Regards,
Raphael
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev