[gdal-dev] GDAL install and run with a single command

[Petr Pridal - Klokan Technologies GmbH]

Hi everybody,

If you want to use the latest stable GDAL, for example on a newly installed
Linux machine to do some fast raster data processing with the command-line
utilities, and you need drivers like MrSID, ECW or Kakadu JPEG2000, you may
very probably end up compiling everything yourself. It takes time and you
need to install a lot of dev packages extra to deploy the working
environment.

That's why we created a minimal GDAL in Docker container (about 350MB).

On any computer with Docker installed (see
https://docs.docker.com/installation/#installation) you can download and
run any GDAL utility with a single command.

For example gdalinfo on yourfile.tif stored in local directory:

$ docker run -ti --rm -v $(pwd):/data klokantech/gdal gdalinfo yourfile.tif

and similar with other GDAL tools. For more details see:

https://registry.hub.docker.com/u/klokantech/gdal/

and our blog post:

http://blog.klokantech.com/2015/02/gdal-in-docker-install-run-with-single.html

This Docker container has been made to simplify distribution of MapTiler
Pro, the follower of my GDAL2Tiles, which is now available from
http://www.maptiler.com/.

Thanks everybody for the great work on GDAL!

I hope you find our docker container useful.

Regards,

Petr
-- 
Petr Pridal, Ph.D.
(managing director)

Klokan Technologies GmbH
Hofnerstrasse 96, 6314 Unterageri, Switzerland
Tel: +41 (0)41 511 26 12
Email: i...@klokantech.com
Web: http://www.klokantech.com/
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL install and run with a single command

[Even Rouault]

Petr,

I can see that the libgdal in the Docker image is linked against 
/usr/lib/x86_64-linux-gnu/libtiff.so.4
which happens to be 3.9.6-11. GDAL itself contains a copy of libtiff 4.0.X that 
will get used,
so you can potentially have a mix of libtiff symbols of different ABI that 
could lead to crashes if an application
links against libgdal and this external libtiff in the wrong order (this isn't 
a theoretical concern.
The Orfeo Toolbok folks have suffered from that at a time). This linking 
against system libtiff 3.X
actually comes from libpodofo AFAICS. To be safe in GDAL and avoid any 
potential symbol clashes,
you'd better add to the configure options of GDAL :
"--with-hide-internal-symbols=yes --with-libtiff=internal 
--with-rename-internal-libtiff-symbols"

That's what the UbuntuGIS packagers do.
See for example 
https://launchpad.net/~ubuntugis/+archive/ubuntu/ubuntugis-unstable/+files/gdal_1.11.0%2Bdfsg-1%7Eexp2%7Etrusty2.debian.tar.gz

That doesn't seem to cause problems in that instance, but this is a bit by 
chance...

Even

> Hi everybody,
> 
> If you want to use the latest stable GDAL, for example on a newly installed
> Linux machine to do some fast raster data processing with the command-line
> utilities, and you need drivers like MrSID, ECW or Kakadu JPEG2000, you may
> very probably end up compiling everything yourself. It takes time and you
> need to install a lot of dev packages extra to deploy the working
> environment.
> 
> That's why we created a minimal GDAL in Docker container (about 350MB).
> 
> On any computer with Docker installed (see
> https://docs.docker.com/installation/#installation) you can download and
> run any GDAL utility with a single command.
> 
> For example gdalinfo on yourfile.tif stored in local directory:
> 
> $ docker run -ti --rm -v $(pwd):/data klokantech/gdal gdalinfo yourfile.tif
> 
> and similar with other GDAL tools. For more details see:
> 
> https://registry.hub.docker.com/u/klokantech/gdal/
> 
> and our blog post:
> 
> http://blog.klokantech.com/2015/02/gdal-in-docker-install-run-with-single.h
> tml
> 
> This Docker container has been made to simplify distribution of MapTiler
> Pro, the follower of my GDAL2Tiles, which is now available from
> http://www.maptiler.com/.
> 
> Thanks everybody for the great work on GDAL!
> 
> I hope you find our docker container useful.
> 
> Regards,
> 
> Petr

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL install and run with a single command

[Petr Pridal - Klokan Technologies GmbH]

Even,

thanks for the tip. The binaries are build against the stable debian. We
will adjust the Dockerfile and republish the binaries soon.

Regards,

Petr

On Thu, Feb 5, 2015 at 11:21 AM, Even Rouault 
wrote:

> Petr,
>
> I can see that the libgdal in the Docker image is linked against
> /usr/lib/x86_64-linux-gnu/libtiff.so.4
> which happens to be 3.9.6-11. GDAL itself contains a copy of libtiff 4.0.X
> that will get used,
> so you can potentially have a mix of libtiff symbols of different ABI that
> could lead to crashes if an application
> links against libgdal and this external libtiff in the wrong order (this
> isn't a theoretical concern.
> The Orfeo Toolbok folks have suffered from that at a time). This linking
> against system libtiff 3.X
> actually comes from libpodofo AFAICS. To be safe in GDAL and avoid any
> potential symbol clashes,
> you'd better add to the configure options of GDAL :
> "--with-hide-internal-symbols=yes --with-libtiff=internal
> --with-rename-internal-libtiff-symbols"
>
> That's what the UbuntuGIS packagers do.
> See for example
> https://launchpad.net/~ubuntugis/+archive/ubuntu/ubuntugis-unstable/+files/gdal_1.11.0%2Bdfsg-1%7Eexp2%7Etrusty2.debian.tar.gz
>
> That doesn't seem to cause problems in that instance, but this is a bit by
> chance...
>
> Even
>
> > Hi everybody,
> >
> > If you want to use the latest stable GDAL, for example on a newly
> installed
> > Linux machine to do some fast raster data processing with the
> command-line
> > utilities, and you need drivers like MrSID, ECW or Kakadu JPEG2000, you
> may
> > very probably end up compiling everything yourself. It takes time and you
> > need to install a lot of dev packages extra to deploy the working
> > environment.
> >
> > That's why we created a minimal GDAL in Docker container (about 350MB).
> >
> > On any computer with Docker installed (see
> > https://docs.docker.com/installation/#installation) you can download and
> > run any GDAL utility with a single command.
> >
> > For example gdalinfo on yourfile.tif stored in local directory:
> >
> > $ docker run -ti --rm -v $(pwd):/data klokantech/gdal gdalinfo
> yourfile.tif
> >
> > and similar with other GDAL tools. For more details see:
> >
> > https://registry.hub.docker.com/u/klokantech/gdal/
> >
> > and our blog post:
> >
> >
> http://blog.klokantech.com/2015/02/gdal-in-docker-install-run-with-single.h
> > tml
> >
> > This Docker container has been made to simplify distribution of MapTiler
> > Pro, the follower of my GDAL2Tiles, which is now available from
> > http://www.maptiler.com/.
> >
> > Thanks everybody for the great work on GDAL!
> >
> > I hope you find our docker container useful.
> >
> > Regards,
> >
> > Petr
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
>



-- 
Petr Pridal, Ph.D.
(managing director)

Klokan Technologies GmbH
Hofnerstrasse 96, 6314 Unterageri, Switzerland
Tel: +41 (0)41 511 26 12
Email: i...@klokantech.com
Web: http://www.klokantech.com/
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Can we combine all layers of kml into single GeoJSON

[Jukka Rahkonen]

Even Rouault  spatialys.com> writes:



> Mani,
> 
> you could use a VRT union layer to merge all layers into a single one.
This is 
> documented at http://gdal.org/drv_vrt.html
> 
> In your case this would be creating a .vrt file like :
> 
> 
> 
> 
> source.kml
> name_of_first_kml_layer
> 
> 
> source.kml
> name_of_second_kml_layer
> 
> 
> 
> 
> and then use ogr2ogr to convert this .vrt into geojson.

Hi,

Just thinking, but wouldn't it be nice to have "ogrbuildvrt" tool that would
take list of OGR supported layers and outputs a simple vrt with one  union
layer like the one in your example?

-Jukka Rahkonen-

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Call for discussion on RFC 53: OGR not-null constraints and default values

[Even Rouault]

Hi,

This is a call for discussion on "RFC 53: OGR not-null constraints and default 
values".

http://trac.osgeo.org/gdal/wiki/rfc53_ogr_notnull_default

Summary :

This RFC addresses handling of NOT NULL constraints and DEFAULT values for OGR 
fields. NOT NULL constraints are useful to maintain basic data integrity and 
are handled by most (all?) drivers that have SQL capabilities. Default fields 
values may be used complementary or independantly of NOT NULL constraints to 
specify the value a field must be assigned to if it is not provided when 
inserting a feature into the layer. 

Best regards,

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Can we combine all layers of kml into single GeoJSON

[Manikanta Kondeti]

Yeah that ll be pretty awesome :D

On Thu, Feb 5, 2015 at 4:49 PM, Jukka Rahkonen <
jukka.rahko...@maanmittauslaitos.fi> wrote:

> Even Rouault  spatialys.com> writes:
>
>
>
> > Mani,
> >
> > you could use a VRT union layer to merge all layers into a single one.
> This is
> > documented at http://gdal.org/drv_vrt.html
> >
> > In your case this would be creating a .vrt file like :
> >
> > 
> > 
> > 
> > source.kml
> > name_of_first_kml_layer
> > 
> > 
> > source.kml
> > name_of_second_kml_layer
> > 
> > 
> > 
> >
> > and then use ogr2ogr to convert this .vrt into geojson.
>
> Hi,
>
> Just thinking, but wouldn't it be nice to have "ogrbuildvrt" tool that
> would
> take list of OGR supported layers and outputs a simple vrt with one  union
> layer like the one in your example?
>
> -Jukka Rahkonen-
>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] NetCDF driver improvements (including groups support)

[Julien Demaria]

Hi GDAL team,

I've implemented several improvements to the NetCDF driver and I would like to 
provide them to the community.
Main goal of the changes is to add full support of NetCDF-4 including groups.
NetCDF-4 is the future format of ESA Sentinel-3 products (no groups) and NASA 
Ocean Color team is switching their L2/L3 products to NetCDF-4 with groups 
(VIIRS has already switched to the new format in December).
With the changes NASA L2 products geolocation is automatically handled as 
geolocation arrays and can be reprojected using gdalwarp.

I validated with autotests that nothing is broken in tests netcdf.py (excepting 
test 13 but see my point 5), netcdf_cf.py and hdf5.py, using NetCDF-3 and 4 
libraries.
I've also tested the new functionalities on various NetCDF-4 files.
I think the only possible regression could be for marginal cases where a file 
was seen directly as a dataset and is now seen as multiple subdatasets (for 
example if a file has only one var in the top group and has nested groups 
containing variables), but I think this is not very common.

For the moment I have all these changes in local GIT separated commits on the 
latest gdal-1.11 branch, let me know what changes you want and how can I 
provide them.

Changes :

1) Implement full support for NetCDF-4 groups on reading:
- explore recursively all nested groups to create the subdatasets list
- subdatasets in nested groups use the /group1/group2/.../groupn/var 
standard
  NetCDF-4 convention, excepting for variables in the root group which do 
not
  have a leading slash for backward compatibility
- when accessing a subdataset using NETCDF:$file:$path, the leading slash 
is optional
- global attributes of each nested group are also collected in the GDAL 
dataset
  metadata, using the same convention 
/group1/group2/.../groupn/NC_GLOBAL#attr_name,
  excepting for the root group which do not have a leading slash for 
backward compatibility
- when searching for a variable containing auxiliary information on the 
selected subdataset,
  like coordinate variables or grid_mapping, we now also search in parent 
groups (using NCDFResolveVar).
  I now this is something not specified at this time in the CF convention 
because CF does not know groups,
  but it seems logical to me to support this: NetCDF-4 specifies that 
dimensions of a group are
  shared to its nested groups, so associated coordinate variables could be 
defined as the same level of its
  corresponding dimension.
- reference to coordinate variables using the "coordinates" attribute 
support now also absolute paths,
  this allow for example to specify coordinate variables located outside 
the group of the selected variable
  or its parents. Relative paths could be implemented if needed.
  This feature is used to add support for new NASA Ocean Color L2 products.

2) Implement full read/write support for new NetCDF4 types NC_UBYTE, NC_USHORT, 
NC_UINT and NC_STRING, only if NETCDF_HAS_NC4 is defined (and only if 
format=NC4 for writing).
Support implemented for variables and attributes.
NC_STRING type is supported for reading (scalar and arrays) attributes and 
is used for writing only for array attributes (scalar are still written as 
NC_CHAR).
If NETCDF_HAS_NC4 is not defined or format!=NC4, NC_STRING array attributes 
are written as a single NC_CHAR string using the GDAL {v1,v2,...} convention.
Add missing support for NC_BYTE in CreateBandMetadata() and NC_BYTE/SHORT 
in NCDFPut1DVar().

3) Add support for new NASA Ocean Color L2 products and ESA Sentinel-3 L1 or
L2 products which use the NetCDF-4 format (with groups for NASA, see
http://oceancolor.gsfc.nasa.gov/DOCS/FormatChange.html):
- NASA products: simulate a "coordinates" variable attribute to detect CF
  geolocation arrays, and set bBottomUp to FALSE
- ESA products: set bBottomUp to FALSE and disable warning on missing
  Conventions attribute

4) Fix bug #4554 with a more generic solution by disabling the installation of 
the HDF5 atexit() cleanup routine using H5dont_atexit().
Previous fix was to call GDALExit() (for the moment only defined 
gdalwarp.cpp) at the end of every program, which is more painful.

5) Fix implementation of GetScale/Offset to not always return pbSuccess=TRUE.
Fix CopyMetadata to handle bands with only scale or offset.
==> WARNING this commit breaks the autotest netcdf_13 (check for 
scale/offset = 1.0/0.0 if no scale or offset is available), but for me it is 
not logical to return always pbSuccess=TRUE

6) Optimize IReadBlock() and CheckData() handling of partial blocks in the x 
axis by re-using the GDAL block buffer instead of allocating a new temporary 
buffer for each block.

7) Force block size to 1 scanline for bottom-up datasets if nBlockYSize != 1 
instead of raising a fatal error
==> Solve a recent problem raised on the mailing list

8) Implement Get/SetUnitTyp

[gdal-dev] gdalwarp -crop_to_cutline produces non-square pixels for EPSG:3857 (GDAL 1.10.1, released 2013/08/26)

[Jesse McGraw]

I'm not sure whether this is a real issue or not but I thought I'd bring it
up, at the very least I'll learn something

When using "gdalwarp -cutline  -crop_to_cutline" on an input
raster that is in EPSG:3857 with square-pixels the output raster, while
still EPSG:3857, now has non-square pixels.

They're aren't terribly non-square but aren't they supposed to be
completely square for EPSG:3857?

(FWIW, I see that there are tickets opened that reference similar issues
but they reference the output being shifted or the origin changing, not the
pixel shape changing)

For example:
#Warp our original .tif to EPSG:3857
$gdalwarp \
-t_srs EPSG:3857 \
-dstalpha \
-co TILED=YES \
"ENR_L33.tif" \
"./2.tif"

#See that the output pixels are square
$gdalinfo 2.tif
Origin = (-8577554.996301921084523,5421778.172851986251771)
Pixel Size = (43.677179501975118,-43.677179501975118)


#Now crop the image to a cutline
$gdalwarp \
-crop_to_cutline \
-dstalpha \
-cutline "./ENR_L33.shp" \
-cblend 10 \
-co TILED=YES \
"./2.tif" \
"./3.tif"

#See that output pixels are not square
$ gdalinfo 3.tif
Origin = (-8480047.445924906060100,5366376.137789577245712)
Pixel Size = (43.678439570399853,-43.675457269061347)


I ran some more tests and without the -crop_to_cutline option the output
pixels remain square

Thanks,
  Jesse
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Can we combine all layers of kml into single GeoJSON

[Eli Adam]

>
> Just thinking, but wouldn't it be nice to have "ogrbuildvrt" tool that would
> take list of OGR supported layers and outputs a simple vrt with one  union
> layer like the one in your example?
>

There is this script,
https://svn.osgeo.org/gdal/trunk/gdal/swig/python/samples/ogr2vrt.py,
with details here,
http://lists.osgeo.org/pipermail/gdal-dev/2009-October/022522.html.
It might get you part of the way.

Cheers, Eli
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Can we combine all layers of kml into single GeoJSON

[Even Rouault]

Le jeudi 05 février 2015 17:24:24, Eli Adam a écrit :
> > Just thinking, but wouldn't it be nice to have "ogrbuildvrt" tool that
> > would take list of OGR supported layers and outputs a simple vrt with
> > one  union layer like the one in your example?
> 
> There is this script,
> https://svn.osgeo.org/gdal/trunk/gdal/swig/python/samples/ogr2vrt.py,
> with details here,
> http://lists.osgeo.org/pipermail/gdal-dev/2009-October/022522.html.
> It might get you part of the way.

It doesn't deal with UnionLayers. Just for single datasource to VRT. But a 
ogrbuildvrt shouldn't be very difficult to do if someone wants to tackle it.

> 
> Cheers, Eli
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] gdalwarp -crop_to_cutline produces non-square pixels for EPSG:3857 (GDAL 1.10.1, released 2013/08/26)

[Even Rouault]

Le jeudi 05 février 2015 17:23:06, Jesse McGraw a écrit :
> I'm not sure whether this is a real issue or not but I thought I'd bring it
> up, at the very least I'll learn something
> 
> When using "gdalwarp -cutline  -crop_to_cutline" on an input
> raster that is in EPSG:3857 with square-pixels the output raster, while
> still EPSG:3857, now has non-square pixels.
> 
> They're aren't terribly non-square but aren't they supposed to be
> completely square for EPSG:3857?

Perhaps... Actually the effet of -crop_to_cutline is exactly the same as 
manually passing the target extents with -te with the bounding box of the 
cutline.
It is not possible to both preserve the extent extents and pixel size at the 
same time, due to width and height being integer values.
So, in order to preserve pixel square, the extent should be modified a little 
bit. What is more appropriate is a matter of point of view I think.
You can always re-run "gdalwarp tmp.tif out.tif" where tmp.tif is the output 
of first gdalwarp with -crop_to_cutline. And you should get square pixels I 
believe.

> 
> (FWIW, I see that there are tickets opened that reference similar issues
> but they reference the output being shifted or the origin changing, not the
> pixel shape changing)
> 
> For example:
> #Warp our original .tif to EPSG:3857
> $gdalwarp \
> -t_srs EPSG:3857 \
> -dstalpha \
> -co TILED=YES \
> "ENR_L33.tif" \
> "./2.tif"
> 
> #See that the output pixels are square
> $gdalinfo 2.tif
> Origin = (-8577554.996301921084523,5421778.172851986251771)
> Pixel Size = (43.677179501975118,-43.677179501975118)
> 
> 
> #Now crop the image to a cutline
> $gdalwarp \
> -crop_to_cutline \
> -dstalpha \
> -cutline "./ENR_L33.shp" \
> -cblend 10 \
> -co TILED=YES \
> "./2.tif" \
> "./3.tif"
> 
> #See that output pixels are not square
> $ gdalinfo 3.tif
> Origin = (-8480047.445924906060100,5366376.137789577245712)
> Pixel Size = (43.678439570399853,-43.675457269061347)
> 
> 
> I ran some more tests and without the -crop_to_cutline option the output
> pixels remain square
> 
> Thanks,
>   Jesse

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] gdalwarp -crop_to_cutline produces non-square pixels for EPSG:3857 (GDAL 1.10.1, released 2013/08/26)

[Jesse McGraw]

Thanks Even.

I did try re-warping the cropped output to 3857 again and it does produce
square pixels but at the expense of an additional warping operation

I'm just curious why the warping that is part of the -crop_to_cutline
process is changing the pixel ratio while a regular, non-cropping warping
makes them square

On Thu, Feb 5, 2015 at 11:36 AM, Even Rouault 
wrote:

> Le jeudi 05 février 2015 17:23:06, Jesse McGraw a écrit :
> > I'm not sure whether this is a real issue or not but I thought I'd bring
> it
> > up, at the very least I'll learn something
> >
> > When using "gdalwarp -cutline  -crop_to_cutline" on an input
> > raster that is in EPSG:3857 with square-pixels the output raster, while
> > still EPSG:3857, now has non-square pixels.
> >
> > They're aren't terribly non-square but aren't they supposed to be
> > completely square for EPSG:3857?
>
> Perhaps... Actually the effet of -crop_to_cutline is exactly the same as
> manually passing the target extents with -te with the bounding box of the
> cutline.
> It is not possible to both preserve the extent extents and pixel size at
> the
> same time, due to width and height being integer values.
> So, in order to preserve pixel square, the extent should be modified a
> little
> bit. What is more appropriate is a matter of point of view I think.
> You can always re-run "gdalwarp tmp.tif out.tif" where tmp.tif is the
> output
> of first gdalwarp with -crop_to_cutline. And you should get square pixels I
> believe.
>
> >
> > (FWIW, I see that there are tickets opened that reference similar issues
> > but they reference the output being shifted or the origin changing, not
> the
> > pixel shape changing)
> >
> > For example:
> > #Warp our original .tif to EPSG:3857
> > $gdalwarp \
> > -t_srs EPSG:3857 \
> > -dstalpha \
> > -co TILED=YES \
> > "ENR_L33.tif" \
> > "./2.tif"
> >
> > #See that the output pixels are square
> > $gdalinfo 2.tif
> > Origin = (-8577554.996301921084523,5421778.172851986251771)
> > Pixel Size = (43.677179501975118,-43.677179501975118)
> >
> >
> > #Now crop the image to a cutline
> > $gdalwarp \
> > -crop_to_cutline \
> > -dstalpha \
> > -cutline "./ENR_L33.shp" \
> > -cblend 10 \
> > -co TILED=YES \
> > "./2.tif" \
> > "./3.tif"
> >
> > #See that output pixels are not square
> > $ gdalinfo 3.tif
> > Origin = (-8480047.445924906060100,5366376.137789577245712)
> > Pixel Size = (43.678439570399853,-43.675457269061347)
> >
> >
> > I ran some more tests and without the -crop_to_cutline option the output
> > pixels remain square
> >
> > Thanks,
> >   Jesse
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
>
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] gdalwarp -crop_to_cutline produces non-square pixels for EPSG:3857 (GDAL 1.10.1, released 2013/08/26)

[Even Rouault]

Le jeudi 05 février 2015 17:41:22, Jesse McGraw a écrit :
> Thanks Even.
> 
> I did try re-warping the cropped output to 3857 again and it does produce
> square pixels but at the expense of an additional warping operation
> 
> I'm just curious why the warping that is part of the -crop_to_cutline
> process is changing the pixel ratio while a regular, non-cropping warping
> makes them square

I think I explained it in my previous answer. This is because -crop_to_cutline 
uses -te internally, which in the general case doesn't allow pixel squares.
And non-cropping warping doesn't necessarily preserve extent of the output to 
be the same as extent of the input (even in non-reprojection cases). You can 
check that actually by comparing gdalinfo on the temporary and final datasets.
There's no fundamental reason why -crop_to_cutline couldn't produce exact 
square pixels, if people are annoyed by that behaviour. Just someone has to do 
it...

> 
> On Thu, Feb 5, 2015 at 11:36 AM, Even Rouault 
> 
> wrote:
> > Le jeudi 05 février 2015 17:23:06, Jesse McGraw a écrit :
> > > I'm not sure whether this is a real issue or not but I thought I'd
> > > bring
> > 
> > it
> > 
> > > up, at the very least I'll learn something
> > > 
> > > When using "gdalwarp -cutline  -crop_to_cutline" on an input
> > > raster that is in EPSG:3857 with square-pixels the output raster, while
> > > still EPSG:3857, now has non-square pixels.
> > > 
> > > They're aren't terribly non-square but aren't they supposed to be
> > > completely square for EPSG:3857?
> > 
> > Perhaps... Actually the effet of -crop_to_cutline is exactly the same as
> > manually passing the target extents with -te with the bounding box of the
> > cutline.
> > It is not possible to both preserve the extent extents and pixel size at
> > the
> > same time, due to width and height being integer values.
> > So, in order to preserve pixel square, the extent should be modified a
> > little
> > bit. What is more appropriate is a matter of point of view I think.
> > You can always re-run "gdalwarp tmp.tif out.tif" where tmp.tif is the
> > output
> > of first gdalwarp with -crop_to_cutline. And you should get square pixels
> > I believe.
> > 
> > > (FWIW, I see that there are tickets opened that reference similar
> > > issues but they reference the output being shifted or the origin
> > > changing, not
> > 
> > the
> > 
> > > pixel shape changing)
> > > 
> > > For example:
> > > #Warp our original .tif to EPSG:3857
> > > $gdalwarp \
> > > 
> > > -t_srs EPSG:3857 \
> > > -dstalpha \
> > > -co TILED=YES \
> > > "ENR_L33.tif" \
> > > "./2.tif"
> > > 
> > > #See that the output pixels are square
> > > $gdalinfo 2.tif
> > > Origin = (-8577554.996301921084523,5421778.172851986251771)
> > > Pixel Size = (43.677179501975118,-43.677179501975118)
> > > 
> > > 
> > > #Now crop the image to a cutline
> > > $gdalwarp \
> > > 
> > > -crop_to_cutline \
> > > -dstalpha \
> > > -cutline "./ENR_L33.shp" \
> > > -cblend 10 \
> > > -co TILED=YES \
> > > "./2.tif" \
> > > "./3.tif"
> > > 
> > > #See that output pixels are not square
> > > $ gdalinfo 3.tif
> > > Origin = (-8480047.445924906060100,5366376.137789577245712)
> > > Pixel Size = (43.678439570399853,-43.675457269061347)
> > > 
> > > 
> > > I ran some more tests and without the -crop_to_cutline option the
> > > output pixels remain square
> > > 
> > > Thanks,
> > > 
> > >   Jesse
> > 
> > --
> > Spatialys - Geospatial professional services
> > http://www.spatialys.com

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] gdalwarp -crop_to_cutline produces non-square pixels for EPSG:3857 (GDAL 1.10.1, released 2013/08/26)

[Jesse McGraw]

Thank you, Even.  For now I'll proceed with the additional warping.

Just for my education: are square pixels a requirement for EPSG:3857?

On Thu, Feb 5, 2015 at 11:50 AM, Even Rouault 
wrote:

> Le jeudi 05 février 2015 17:41:22, Jesse McGraw a écrit :
> > Thanks Even.
> >
> > I did try re-warping the cropped output to 3857 again and it does produce
> > square pixels but at the expense of an additional warping operation
> >
> > I'm just curious why the warping that is part of the -crop_to_cutline
> > process is changing the pixel ratio while a regular, non-cropping warping
> > makes them square
>
> I think I explained it in my previous answer. This is because
> -crop_to_cutline
> uses -te internally, which in the general case doesn't allow pixel squares.
> And non-cropping warping doesn't necessarily preserve extent of the output
> to
> be the same as extent of the input (even in non-reprojection cases). You
> can
> check that actually by comparing gdalinfo on the temporary and final
> datasets.
> There's no fundamental reason why -crop_to_cutline couldn't produce exact
> square pixels, if people are annoyed by that behaviour. Just someone has
> to do
> it...
>
> >
> > On Thu, Feb 5, 2015 at 11:36 AM, Even Rouault <
> even.roua...@spatialys.com>
> >
> > wrote:
> > > Le jeudi 05 février 2015 17:23:06, Jesse McGraw a écrit :
> > > > I'm not sure whether this is a real issue or not but I thought I'd
> > > > bring
> > >
> > > it
> > >
> > > > up, at the very least I'll learn something
> > > >
> > > > When using "gdalwarp -cutline  -crop_to_cutline" on an
> input
> > > > raster that is in EPSG:3857 with square-pixels the output raster,
> while
> > > > still EPSG:3857, now has non-square pixels.
> > > >
> > > > They're aren't terribly non-square but aren't they supposed to be
> > > > completely square for EPSG:3857?
> > >
> > > Perhaps... Actually the effet of -crop_to_cutline is exactly the same
> as
> > > manually passing the target extents with -te with the bounding box of
> the
> > > cutline.
> > > It is not possible to both preserve the extent extents and pixel size
> at
> > > the
> > > same time, due to width and height being integer values.
> > > So, in order to preserve pixel square, the extent should be modified a
> > > little
> > > bit. What is more appropriate is a matter of point of view I think.
> > > You can always re-run "gdalwarp tmp.tif out.tif" where tmp.tif is the
> > > output
> > > of first gdalwarp with -crop_to_cutline. And you should get square
> pixels
> > > I believe.
> > >
> > > > (FWIW, I see that there are tickets opened that reference similar
> > > > issues but they reference the output being shifted or the origin
> > > > changing, not
> > >
> > > the
> > >
> > > > pixel shape changing)
> > > >
> > > > For example:
> > > > #Warp our original .tif to EPSG:3857
> > > > $gdalwarp \
> > > >
> > > > -t_srs EPSG:3857 \
> > > > -dstalpha \
> > > > -co TILED=YES \
> > > > "ENR_L33.tif" \
> > > > "./2.tif"
> > > >
> > > > #See that the output pixels are square
> > > > $gdalinfo 2.tif
> > > > Origin = (-8577554.996301921084523,5421778.172851986251771)
> > > > Pixel Size = (43.677179501975118,-43.677179501975118)
> > > >
> > > >
> > > > #Now crop the image to a cutline
> > > > $gdalwarp \
> > > >
> > > > -crop_to_cutline \
> > > > -dstalpha \
> > > > -cutline "./ENR_L33.shp" \
> > > > -cblend 10 \
> > > > -co TILED=YES \
> > > > "./2.tif" \
> > > > "./3.tif"
> > > >
> > > > #See that output pixels are not square
> > > > $ gdalinfo 3.tif
> > > > Origin = (-8480047.445924906060100,5366376.137789577245712)
> > > > Pixel Size = (43.678439570399853,-43.675457269061347)
> > > >
> > > >
> > > > I ran some more tests and without the -crop_to_cutline option the
> > > > output pixels remain square
> > > >
> > > > Thanks,
> > > >
> > > >   Jesse
> > >
> > > --
> > > Spatialys - Geospatial professional services
> > > http://www.spatialys.com
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
>
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] gdalwarp -crop_to_cutline produces non-square pixels for EPSG:3857 (GDAL 1.10.1, released 2013/08/26)

[Even Rouault]

Le jeudi 05 février 2015 17:57:02, Jesse McGraw a écrit :
> Thank you, Even.  For now I'll proceed with the additional warping.
> 
> Just for my education: are square pixels a requirement for EPSG:3857?

In the general case, no. But it might be a requirement (or an optimization) 
depending on what you do with your data afterwards. Difficult to give a 
definitive answer on that...

> 
> On Thu, Feb 5, 2015 at 11:50 AM, Even Rouault 
> 
> wrote:
> > Le jeudi 05 février 2015 17:41:22, Jesse McGraw a écrit :
> > > Thanks Even.
> > > 
> > > I did try re-warping the cropped output to 3857 again and it does
> > > produce square pixels but at the expense of an additional warping
> > > operation
> > > 
> > > I'm just curious why the warping that is part of the -crop_to_cutline
> > > process is changing the pixel ratio while a regular, non-cropping
> > > warping makes them square
> > 
> > I think I explained it in my previous answer. This is because
> > -crop_to_cutline
> > uses -te internally, which in the general case doesn't allow pixel
> > squares. And non-cropping warping doesn't necessarily preserve extent of
> > the output to
> > be the same as extent of the input (even in non-reprojection cases). You
> > can
> > check that actually by comparing gdalinfo on the temporary and final
> > datasets.
> > There's no fundamental reason why -crop_to_cutline couldn't produce exact
> > square pixels, if people are annoyed by that behaviour. Just someone has
> > to do
> > it...
> > 
> > > On Thu, Feb 5, 2015 at 11:36 AM, Even Rouault <
> > 
> > even.roua...@spatialys.com>
> > 
> > > wrote:
> > > > Le jeudi 05 février 2015 17:23:06, Jesse McGraw a écrit :
> > > > > I'm not sure whether this is a real issue or not but I thought I'd
> > > > > bring
> > > > 
> > > > it
> > > > 
> > > > > up, at the very least I'll learn something
> > > > > 
> > > > > When using "gdalwarp -cutline  -crop_to_cutline" on an
> > 
> > input
> > 
> > > > > raster that is in EPSG:3857 with square-pixels the output raster,
> > 
> > while
> > 
> > > > > still EPSG:3857, now has non-square pixels.
> > > > > 
> > > > > They're aren't terribly non-square but aren't they supposed to be
> > > > > completely square for EPSG:3857?
> > > > 
> > > > Perhaps... Actually the effet of -crop_to_cutline is exactly the same
> > 
> > as
> > 
> > > > manually passing the target extents with -te with the bounding box of
> > 
> > the
> > 
> > > > cutline.
> > > > It is not possible to both preserve the extent extents and pixel size
> > 
> > at
> > 
> > > > the
> > > > same time, due to width and height being integer values.
> > > > So, in order to preserve pixel square, the extent should be modified
> > > > a little
> > > > bit. What is more appropriate is a matter of point of view I think.
> > > > You can always re-run "gdalwarp tmp.tif out.tif" where tmp.tif is the
> > > > output
> > > > of first gdalwarp with -crop_to_cutline. And you should get square
> > 
> > pixels
> > 
> > > > I believe.
> > > > 
> > > > > (FWIW, I see that there are tickets opened that reference similar
> > > > > issues but they reference the output being shifted or the origin
> > > > > changing, not
> > > > 
> > > > the
> > > > 
> > > > > pixel shape changing)
> > > > > 
> > > > > For example:
> > > > > #Warp our original .tif to EPSG:3857
> > > > > $gdalwarp \
> > > > > 
> > > > > -t_srs EPSG:3857 \
> > > > > -dstalpha \
> > > > > -co TILED=YES \
> > > > > "ENR_L33.tif" \
> > > > > "./2.tif"
> > > > > 
> > > > > #See that the output pixels are square
> > > > > $gdalinfo 2.tif
> > > > > Origin = (-8577554.996301921084523,5421778.172851986251771)
> > > > > Pixel Size = (43.677179501975118,-43.677179501975118)
> > > > > 
> > > > > 
> > > > > #Now crop the image to a cutline
> > > > > $gdalwarp \
> > > > > 
> > > > > -crop_to_cutline \
> > > > > -dstalpha \
> > > > > -cutline "./ENR_L33.shp" \
> > > > > -cblend 10 \
> > > > > -co TILED=YES \
> > > > > "./2.tif" \
> > > > > "./3.tif"
> > > > > 
> > > > > #See that output pixels are not square
> > > > > $ gdalinfo 3.tif
> > > > > Origin = (-8480047.445924906060100,5366376.137789577245712)
> > > > > Pixel Size = (43.678439570399853,-43.675457269061347)
> > > > > 
> > > > > 
> > > > > I ran some more tests and without the -crop_to_cutline option the
> > > > > output pixels remain square
> > > > > 
> > > > > Thanks,
> > > > > 
> > > > >   Jesse
> > > > 
> > > > --
> > > > Spatialys - Geospatial professional services
> > > > http://www.spatialys.com
> > 
> > --
> > Spatialys - Geospatial professional services
> > http://www.spatialys.com

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Use gdal2tiles to generate XYZ cache

[Jorge Arévalo]

Hello Klokan,

Thanks for the info. Yes, we have a maptiler license. It's a great tool 
:-).


Best regards

Jorge


Klokan Petr Pridal 
February 3, 2015 at 2:20 AM
Hi there,

maptiler generates xyz by default, and back-compatible tms with -tms 
on the command line or in the GUI with switcher in advanced options.


Regards,

Petr




--
http://blog.klokan.cz/
http://www.maptiler.org/
http://www.oldmapsonline.org/
Jorge Arévalo 
February 3, 2015 at 1:02 AM
Hello,

Looks like gdal2tiles is only able to generate TMS caches (OGC 
standard). Or is there any combination of parameters to generate a 
tile cache in XYZ format? (top-left standard). I've tried with all the 
profiles (-p option), but not getting the desired results... Any hints?


Best regards,



--
Sent with Postbox 
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Quick question about gdal2tiles

[Jorge Arévalo]

Hi,

Thanks for the tips, Klokan. Really useful. We already have a maptiler 
license, and we use it, but we can't use that tool this time.


Best regards,

Jorge

Klokan Petr Pridal 
February 3, 2015 at 2:18 AM
Hi Jorge,

rendering of your sample file down to zoomlevel 19 would generate over 
2 billion tiles - probably tens of terabytes of data. I guess it is 
not what you want to do... Typically the overzooming is done on client 
side in a JavaScript viewer or on the tileserver hosting.


The tiling utilities suggests you the native zoom level automatically 
- based on the raster resolution of the input.


In case you want to render large datasets composed of multiple files 
efficiently I would recommend to try our MapTiler Pro command line 
utility.
It is significantly faster then GDAL2Tiles and produces optimised and 
correct tiles, while handling merging of multiple files, solving 
issues with dateline crossing, reprojection and merging of data in 
different srs, direct output to MBTiles, OGC WMTS compatible export, 
and many more features.


The usage on command line is quite easy:

$ maptiler -o merged_output_dir input1.tif input2.tif input3.tif

Input files can be in the original srs - every extra reprojection 
degrades visual quality of the output.
Merging with VRT is not recommend - it slow down the tile rendering, 
as there are artificial blocks introduced - it is better for 
performance to pass the original files directly.


MapTiler is a complete C/C++ reimplementation of my GDAL2Tiles 
utility. There are several significant improvements in the merging and 
internal tiling process including multi-thread CPU parallelization and 
automatic file size optimisation (PNG color quantization and JPEG tweaks).
Installation is possible with the .deb or .rpm packages or binaries 
for Mac OS X or Windows or Docker.


The small file, like your sample mentioned above, can be rendered 
directly with the MapTiler Free in the graphical user interface which 
is available for download at: http://www.maptiler.com/download/


If you drop us an email we send you also the Pro version with command 
line for your free testing.


Best regards,

Petr Pridal, PhD
Klokan Technologies GmbH




--
http://blog.klokan.cz/
http://www.maptiler.org/
http://www.oldmapsonline.org/
Jorge Arévalo 
January 29, 2015 at 1:13 AM


El jueves, 29 de enero de 2015, Even Rouault 
mailto:even.roua...@spatialys.com>> escribió:


Le jeudi 29 janvier 2015 00:05:32, Jorge Arévalo a écrit :
> > Even Rouault 
> > January 28, 2015 at 9:28 PM
> >
> > Le mercredi 28 janvier 2015 20:17:08, Jorge Arévalo a écrit :
> >> Hi,
> >>
> >> I'm working with a patched version of gdal2tiles, which makes
use of
> >> parallelization:
> >>
http://gis.stackexchange.com/questions/7743/performance-of-google-map-ti
> >> le- creation-processes/74446#74446
> >>
> >> I want to create a complete TMS cache from raster imagery. No
> >> assumptions about SRS of data type for input data.
> >>
> >> I want the tiling process to be as fast as possible
(gdal2tiles is
> >> really heavy process), do you have any recomendations about
the data or
> >> the parameters used?
> >>
> >> Right now, I'm doing this
> >>
> >> Step 1: build vrt from input images
> >>
> >> gdal_vrtmerge.py -o merged.vrt
> >>
> >> Step 2: build tiles from vrt file
> >>
> >> gdal2tiles.py -r cubic -s epsg: -z 0-19 -w all merged.vrt
tms_dir
> >>
> >> Even with parallelization, process still feels really slow.
Would it be
> >> faster if, for example, I convert all my input files to
epsg:3857? Or if
> >> I scale them to 8-bit? Or if I use near resampling method
instead of
> >> cubic? (I'm using cubic because I'm working with continuous data:
> >> satellite images, am I doing it right?).
> >>
> >  From a quick look at the source, it seems that there's an
optimization
> >  if the
> >
> > input SRS == output SRS that avoids going through the warped
VRT path.
> >
> > That said, we definitely need one or several maintainers for
gdal2tiles.
> > There are quite a faw patches floating around in Trac that
would need
> > someone to review, test, fix, apply them, as well as writing
tests (no
> > autotest for gdal2tiles yet), etc...
>
> Ok. But the applications is taking hours to generate a complete tile
> cache (zoom levels 0-19) for a 3MB tiff file, in epsg:3857. A 4
cores
> machine with 8GB of RAM. The file is this one
>
>
https://dl.dropboxusercontent.com/u/6599273/gis_data/katrina-3857.tif
>
> Taking so much time for a 3MB file sounds ridiculous. I'm
probably doing
> something wrong. This is the line
>
> gdal2tiles.py  -s epsg:3857 -z 0-19 -r cubic katrina

[gdal-dev] Are those 2 things possible using gdal2tiles?

[Jorge Arévalo]

- Generate JPEG tiles instead of PNG. Tried just replacing the driver's 
name (PNG) with JPEG. Didn't work. Base tiles are generated, but 
overview tiles are not. Getting this error instead: ERROR 1: Buffer too 
small
- Apply a color to nodata value of generated tiles. Tried passing -a 
"200,0,0" to gdal2tiles, but it was ignored.



Log file, with GDAL_DEBUG=On, where you can see the first problem 
(buffer too small): 
https://dl.dropboxusercontent.com/u/6599273/errors/gdal/gdal2tiles_error.log
Modified version of gdal2tiles, to generate xyz caches instead of tms 
ones: 
https://dl.dropboxusercontent.com/u/6599273/errors/gdal/gdal2tiles_modified.py
tif tile for testing (same error with other ones): 
https://dl.dropboxusercontent.com/u/6599273/gis_data/katrina.tif


Many thanks!

--
Jorge Arévalo

CartoDB
http://cartodb.com/
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Gdal 1.11.2 and libtiff

[Even Rouault]

Kurt,

Forwarding this publicly as this is of general interest.

I've created http://trac.osgeo.org/gdal/ticket/5830 and commited :
branches/1.11 r28417 "Internal libtiff: partial upgrade to 4.0.4beta 
(everything, except changes in tif_jpeg.c that are not security related and 
cause differences in output) (#5830)" 

My personal statement would be that people with high security concerns or 
risks should avoid using libtiff, GDAL or more generally most imaging libraries 
on untrusted datasets on non-isolated / non-sandboxed environments. Regarding 
libtiff, disabling codecs that are somewhat esoteric (like NEXT compression 
that has received security fixes in libtiff 4.0.4beta) might be prudent too.
See http://trac.osgeo.org/gdal/wiki/SecurityIssues

Even

Le jeudi 05 février 2015 18:21:59, Kurt Schwehr a écrit :
> Sorry this is so last minute, but I suggest that 1.11.2 be held back until
> libtiff is updated.  e.g. to
> ftp://ftp.remotesensing.org/pub/libtiff/tiff-4.0.4beta.tar.gz or head.
> 
> There are a number of issues out in the wild:
> 
> http://www.conostix.com/pub/adv/CVE-2014-8127-LibTIFF-Out-of-bounds_Reads.t
> xt
> 
> http://www.conostix.com/pub/adv/CVE-2014-8128-LibTIFF-Out-of-bounds_Writes.
> txt
> 
> http://www.conostix.com/pub/adv/CVE-2014-8129-LibTIFF-Out-of-bounds_Reads_a
> nd_Writes.txt
> 
> -kurt

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Gdal 1.11.2 and libtiff

[Kurt Schwehr]

Excellent!  Thanks!

-kurt

On Thu, Feb 5, 2015 at 10:21 AM, Even Rouault 
wrote:

> Kurt,
>
> Forwarding this publicly as this is of general interest.
>
> I've created http://trac.osgeo.org/gdal/ticket/5830 and commited :
> branches/1.11 r28417 "Internal libtiff: partial upgrade to 4.0.4beta
> (everything, except changes in tif_jpeg.c that are not security related and
> cause differences in output) (#5830)"
>
> My personal statement would be that people with high security concerns or
> risks should avoid using libtiff, GDAL or more generally most imaging
> libraries
> on untrusted datasets on non-isolated / non-sandboxed environments.
> Regarding
> libtiff, disabling codecs that are somewhat esoteric (like NEXT compression
> that has received security fixes in libtiff 4.0.4beta) might be prudent
> too.
> See http://trac.osgeo.org/gdal/wiki/SecurityIssues
>
> Even
>
> Le jeudi 05 février 2015 18:21:59, Kurt Schwehr a écrit :
> > Sorry this is so last minute, but I suggest that 1.11.2 be held back
> until
> > libtiff is updated.  e.g. to
> > ftp://ftp.remotesensing.org/pub/libtiff/tiff-4.0.4beta.tar.gz or head.
> >
> > There are a number of issues out in the wild:
> >
> >
> http://www.conostix.com/pub/adv/CVE-2014-8127-LibTIFF-Out-of-bounds_Reads.t
> > xt
> >
> >
> http://www.conostix.com/pub/adv/CVE-2014-8128-LibTIFF-Out-of-bounds_Writes
> .
> > txt
> >
> >
> http://www.conostix.com/pub/adv/CVE-2014-8129-LibTIFF-Out-of-bounds_Reads_a
> > nd_Writes.txt
> >
> > -kurt
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
>



-- 
--
http://schwehr.org
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Are those 2 things possible using gdal2tiles?

[Jorge Arévalo]

Jorge Arévalo 
February 5, 2015 at 7:15 PM
- Generate JPEG tiles instead of PNG. Tried just replacing the 
driver's name (PNG) with JPEG. Didn't work. Base tiles are generated, 
but overview tiles are not. Getting this error instead: ERROR 1: 
Buffer too small
- Apply a color to nodata value of generated tiles. Tried passing -a 
"200,0,0" to gdal2tiles, but it was ignored.



Log file, with GDAL_DEBUG=On, where you can see the first problem 
(buffer too small): 
https://dl.dropboxusercontent.com/u/6599273/errors/gdal/gdal2tiles_error.log
Modified version of gdal2tiles, to generate xyz caches instead of tms 
ones: 
https://dl.dropboxusercontent.com/u/6599273/errors/gdal/gdal2tiles_modified.py
tif tile for testing (same error with other ones): 
https://dl.dropboxusercontent.com/u/6599273/gis_data/katrina.tif


Many thanks!



Hint: The problem is in swig/python/extensions/gdal_wrap.cpp, method 
GDALDatasetShadow_WriteRaster, because Buf len = 196608, min buffer size 
= 262144. So, I know the reason, but still doesn't know how to fix this



--
Jorge Arévalo

CartoDB
http://cartodb.com/
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Are those 2 things possible using gdal2tiles?

[Jorge Arévalo]

Wow, we answered at same time. So, I'll include my findings here


Even Rouault 
February 5, 2015 at 7:30 PM
Jorge,


- Generate JPEG tiles instead of PNG. Tried just replacing the driver's
name (PNG) with JPEG. Didn't work. Base tiles are generated, but
overview tiles are not. Getting this error instead: ERROR 1: Buffer too
small


This messages comes from the WriteRaster() Python bindings, when it is passed
a buffer that isn't large enough regarding the advertized buffer dimensions.
Perhaps due to the fact that JPEG only supports 3 bands and PNG 4, although
this is just a guess without deeper inspection.



The problem is in swig/python/extensions/gdal_wrap.cpp, method 
GDALDatasetShadow_WriteRaster, because Buf len = 196608, min buffer size 
= 262144.


Is there any method to pass a bigger buffer to JPEG driver? Or just 
avoid the 4th band of PNG



- Apply a color to nodata value of generated tiles. Tried passing -a
"200,0,0" to gdal2tiles, but it was ignored.


The doc of -a suggests that it only applies to the input dataset, not the
output
Mmmm... ok. Is there a Python method to specify a color for nodata 
pixels when creating an output dataset?



Thanks again, Even!

Even

Jorge Arévalo 
February 5, 2015 at 7:15 PM
- Generate JPEG tiles instead of PNG. Tried just replacing the 
driver's name (PNG) with JPEG. Didn't work. Base tiles are generated, 
but overview tiles are not. Getting this error instead: ERROR 1: 
Buffer too small
- Apply a color to nodata value of generated tiles. Tried passing -a 
"200,0,0" to gdal2tiles, but it was ignored.



Log file, with GDAL_DEBUG=On, where you can see the first problem 
(buffer too small): 
https://dl.dropboxusercontent.com/u/6599273/errors/gdal/gdal2tiles_error.log
Modified version of gdal2tiles, to generate xyz caches instead of tms 
ones: 
https://dl.dropboxusercontent.com/u/6599273/errors/gdal/gdal2tiles_modified.py
tif tile for testing (same error with other ones): 
https://dl.dropboxusercontent.com/u/6599273/gis_data/katrina.tif


Many thanks!



--
Sent with Postbox 
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Are those 2 things possible using gdal2tiles?

[Even Rouault]

Jorge,

> - Generate JPEG tiles instead of PNG. Tried just replacing the driver's
> name (PNG) with JPEG. Didn't work. Base tiles are generated, but
> overview tiles are not. Getting this error instead: ERROR 1: Buffer too
> small

This messages comes from the WriteRaster() Python bindings, when it is passed 
a buffer that isn't large enough regarding the advertized buffer dimensions. 
Perhaps due to the fact that JPEG only supports 3 bands and PNG 4, although 
this is just a guess without deeper inspection.

> - Apply a color to nodata value of generated tiles. Tried passing -a
> "200,0,0" to gdal2tiles, but it was ignored.

The doc of -a suggests that it only applies to the input dataset, not the 
output

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Are those 2 things possible using gdal2tiles?

[Jukka Rahkonen]

Jorge Arévalo  cartodb.com> writes:

> 
> 
> - Generate JPEG tiles instead of PNG. Tried just replacing the driver's 
> name (PNG) with JPEG. Didn't work. Base tiles are generated, but 
> overview tiles are not. Getting this error instead: ERROR 1: Buffer too 
> small

There is a patch included in http://trac.osgeo.org/gdal/ticket/2052. Try and
tell if it is usable.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Are those 2 things possible using gdal2tiles?

[Even Rouault]

Le jeudi 05 février 2015 20:15:47, Jukka Rahkonen a écrit :
> Jorge Arévalo  cartodb.com> writes:
> > - Generate JPEG tiles instead of PNG. Tried just replacing the driver's
> > name (PNG) with JPEG. Didn't work. Base tiles are generated, but
> > overview tiles are not. Getting this error instead: ERROR 1: Buffer too
> > small
> 
> There is a patch included in http://trac.osgeo.org/gdal/ticket/2052. Try
> and tell if it is usable.

I just came with a quick&dirty quality patch that works in Jorge's case, that 
I've attached to the ticket.

> 
> -Jukka Rahkonen-
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Tags missing on GitHub mirror

[Even Rouault]

Le jeudi 05 février 2015 23:18:30, Leif Gruenwoldt a écrit :
> I happened to be playing around with this more today and I'm fairly certain
> this isn't just a GitHub UI thing. I'm betting it's svn migration related.
> After cloning a copy of the git repo I noticed the tags don't appear as
> normal tags even from the command line.
> 
> $ git clone  https://github.com/OSGeo/gdal.git
> 
> $ git tag
> 
> Returns nothing, but I would expect to see the list of all tags.
> 
> $ git branch -r | grep 1.4.1
> origin/tags/1.4.1
> 
> Ok so the "tag" for the 1.4.1 release as well as all other tags are showing
> up as remote branches, weird.
> 
> $ git checkout 1.4.1
> error: pathspec '1.4.1' did not match any file(s) known to git.
> 
> That's what I expected, it doesn't see it as a normal tag, I wonder if I
> can check it out with the remote branch name.
> 
> $ git checkout origin/tags/1.4.1
> 
> Yup!
> 
> $ git rev-parse HEAD
> 184230238828f0f31aed7335e4c8443963437b66

The firefox thing was just a UI problem. It seems that tags are created as git 
branches and not as git tags and the name is "tags/1.4.1". Must come from the 
git-svn import configuration used. Looking at mapserver, I don't see a similar 
issue, so yes must be something with how git-svn was configured. Something to 
perhaps reconsider if we officially transition to git some day.

> 
> 
> So at least I can work around this and checkout the tag using the remote
> branch name instead. The SHA1 matches up with what I was expecting so
> that's comforting. Something is definitely fishy though with the tags. I've
> migrated CVS -> git but never svn -> git so I'm not sure if this is a known
> issue.
> 
> On Mon Oct 06 2014 at 1:20:35 PM Leif Gruenwoldt  wrote:
> > Oh weird.
> > 
> > I decided to go ahead and clone the gdal repo locally, but I don't see
> > the tags there either. Maybe it's something to do with the svn -> git
> > migration...
> > 
> > Worst case I will use the git SHA1
> > (184230238828f0f31aed7335e4c8443963437b66) from the 1.4.1 github URL
> > you shared for configuring my git submodule.
> > 
> > On Mon, Oct 6, 2014 at 12:20 PM, Even Rouault
> > 
> >  wrote:
> > > Le lundi 06 octobre 2014 18:08:22, Leif Gruenwoldt a écrit :
> > >> I'm trying to add the gdal repo as a git submodule to one
> > >> of my projects. My project requires a specific older revision of gdal
> > >> (1.4.1) so I want to set the git submodule to that point in time, but
> > 
> > it's
> > 
> > >> hard to do that without the tags in the GitHub mirror.
> > >> 
> > >> Can someone push the git tags to the gdal github mirror?
> > > 
> > > It is already there (but it seems there's a UI problem with the github
> > 
> > with
> > 
> > > Firefox, at least my version. had to use Chrome to display
> > > branches/tags)
> > > 
> > > https://github.com/OSGeo/gdal/tree/tags/1.4.1
> > > 
> > >> https://github.com/OSGeo/gdal
> > >> 
> > >> Cheers
> > >> ___
> > >> gdal-dev mailing list
> > >> gdal-dev@lists.osgeo.org
> > >> http://lists.osgeo.org/mailman/listinfo/gdal-dev
> > > 
> > > --
> > > Spatialys - Geospatial professional services
> > > http://www.spatialys.com

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Where is GDAL 2.0?

[Carlos Piury]

Hi, I am trying to follow the tutorials posted in
http://www.gdal.org/ogr_apitut.html , but it's made for GDAL 2.0 and I have
the latest stable release GDAL 1.11.1.
I can rewrite the code to GDAL 1.11.1 but the API for this version shows
some functions like deprecated.

So... Where can I get GDAL 2.0?.

Thanks


Carlos I. Piury
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Where is GDAL 2.0?

[Even Rouault]

Carlos,

> Hi, I am trying to follow the tutorials posted in
> http://www.gdal.org/ogr_apitut.html , but it's made for GDAL 2.0 and I have
> the latest stable release GDAL 1.11.1.

A snapshot of GDAL 1.11.1 doc is available at http://gdal.org/1.11

> I can rewrite the code to GDAL 1.11.1 but the API for this version shows
> some functions like deprecated.
> 
> So... Where can I get GDAL 2.0?.

It is currently in development, so there's no official package yet. If you want 
to use it, you have to work with the "trunk" of the Subversion repository :

https://svn.osgeo.org/gdal/trunk/gdal

> 
> Thanks
> 
> 
> Carlos I. Piury

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev