Re: [gdal-dev] [EXT] Re: Unknown projection

2024-02-01 Thread Matt.Wilkie via gdal-dev
Thanks Even! Matt From: Even Rouault Sent: Wednesday, January 31, 2024 12:54 To: Matt.Wilkie ; gdal-dev@lists.osgeo.org Subject: [EXT] Re: [gdal-dev] Unknown projection Matt, surprisingly or not, the GTiff driver doesn't try to get the SRS from a .prj

[gdal-dev] Unknown projection

2024-01-31 Thread Matt.Wilkie via gdal-dev
Hello gdal, I have a collection of aerial photos for which neither gdal nor arc pro recognize the projection. To my eye the .prj files look ok. Attached txt is gdalinfo v3.8.0 report along with the .prj. I manually formatted the below for readability: PROJCS["UTM_07N", GEOGCS["NAD83-CSRS

Re: [gdal-dev] [EXT] Happy Birthday GDAL!

2023-10-30 Thread Matt.Wilkie via gdal-dev
Oh wow. Happy birthday gdal! And thank you Frank Warmerdam. That means I've been using gdal for just over 20 years, though the oldest extant public record I could find is 2008 (ref). This makes me feel old, in a good way.

Re: [gdal-dev] JPEG compressed COGs nodata

2022-07-25 Thread Matt.Wilkie
For those not wanting or needing to take the processing hit of running nearblack, use gdalwarp and -dstalpha to create the alpha channel, and use intermediate VRT to save space and more time. Something like this: gdalwarp -srcnodata 0 -dstalpha -of vrt source.tif xx-interim.vrt gdal_translate

Re: [gdal-dev] Linux - bash script - Sentinel 1 processing - dark images

2022-05-09 Thread Matt.Wilkie
> Does anybody have some hints for me how to improve the exposure/contrast of > the images? This doesn't help directly with constructing any command lines you might use, but I found the "Gentle Introduction to GDAL" series by Robert Simmon very helpful in determining what to do. Here's Part

Re: [gdal-dev] Hints that a raster band represents a DEM?

2022-05-09 Thread Matt.Wilkie
If you decide to add an elevation metadata keyword for things you generate let us know and I at least will follow suit. ;-) -Matt -Original Message- From: gdal-dev On Behalf Of Nyall Dawson Sent: April 29, 2022 6:11 PM To: gdal dev Subject: [gdal-dev] Hints that a raster band

Re: [gdal-dev] Standardize gdal-utils scripts return code for "no arguments"

2022-04-27 Thread Matt.Wilkie
return 2 is the default for argparse then it seems to we should do the same for all the python utils? -Matt From: Idan Miara On Behalf Of Idan Miara Sent: April 27, 2022 6:24 AM To: Matt.Wilkie Cc: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] Standardize gdal-utils scripts return code for "

Re: [gdal-dev] Standardize gdal-utils scripts return code for "no arguments"

2022-04-26 Thread Matt.Wilkie
on is: where is returncode: 2 coming from? -Matt From: Matt.Wilkie Sent: April 26, 2022 1:03 PM To: 'gdal-dev@lists.osgeo.org' Subject: RE: Standardize gdal-utils scripts return code for "no arguments" Hi Folks, I've converted all the scripts that were using -1 to 1. However when I st

Re: [gdal-dev] Standardize gdal-utils scripts return code for "no arguments"

2022-04-26 Thread Matt.Wilkie
Hi Folks, I've converted all the scripts that were using -1 to 1. However when I started looking at the ones returning 2 it became less clear what to do. Excepting gdal2tiles all of them are using GDALArgumentParser and I don't see where the return value is being set. Thinking this might mean

Re: [gdal-dev] Convert to min containing bit depth?

2022-04-26 Thread Matt.Wilkie
Thank you both for this, very helpful. -Matt From: Idan Miara Sent: April 21, 2022 11:18 PM To: Mike Taves Cc: Matt.Wilkie ; gdal dev Subject: Re: [gdal-dev] Convert to min containing bit depth? Computing the min value is also requited if you have negative values and could also be useful

[gdal-dev] Convert to min containing bit depth?

2022-04-21 Thread Matt.Wilkie
Idea for a small but useful python tool: scan image for min/max values and convert to smallest possible bit depth without losing values. Surely someone has done something like this already. Any suggestions for where to look for prior art? thanks. Matt Wilkie Geomatics Developer & Administrator

[gdal-dev] Standardize gdal-utils scripts return code for "no arguments"

2022-04-04 Thread Matt.Wilkie
Hi folks, I'm working on "Standardize gdal-utils scripts return codes #5561" for all the scripts in swig/python/gdal-utils. Currently the scripts do not return the same status code for "was run without arguments". It would be good for the same code meant the same thing across all the scripts

[gdal-dev] Gitpod workspace with Ubuntu GDAL

2022-03-29 Thread Matt.Wilkie
Gitpod workspace with Ubuntu GDAL How to open a Gitpod machine with the version of GDAL in the Ubuntu software library. 1. Point browser at https://gitpod.io/#https://github.com/OSGeo/gdal 2. Login with Github credentials 3. In a couple of minutes the gitpod linux

[gdal-dev] Guidelines for optimal image dimensions

2022-03-23 Thread Matt.Wilkie
Does anyone have guidelines of what the general purpose optimal image dimensions in pixels are for any given type? (3 band 8bit RGB, 1 band 16bit signed, ...) At some point the time it takes to do anything in memory on a whole image exceeds the time it takes to read the next chunk from storage

[gdal-dev] GDAL Python style guide?

2022-03-22 Thread Matt.Wilkie
Hi All, What's the closest thing to a style guide for GDAL Python code? I found "RFC 69: C/C++ Code Formatting"[0] but haven't discovered anything for python yet. I'm wondering about preferred line length, autopep8 vs black vs ... formatting tools, variable naming patterns, etc. [0]

Re: [gdal-dev] gdal-utils: About GDAL Python Utilities

2022-02-02 Thread Matt.Wilkie
I revised according to your feedback and created an RST document from it. I’m not too experienced with the syntax so there may be some changes needed. Here’s the PR: https://github.com/OSGeo/gdal/pull/5222 -Matt From: Idan Miara Sent: February 1, 2022 12:30 PM To: Matt.Wilkie Cc: gdal-dev

[gdal-dev] gdal-utils: About GDAL Python Utilities

2022-02-01 Thread Matt.Wilkie
Hi, I had to read through RFC 78: gdal-utils package a few times to wrap my head around what the major components are. Here's some writing of what I think I understand. Is it correct? About GDAL Python Utilities The GDAL python

Re: [gdal-dev] Nodata is None, but still has blanks?

2022-01-19 Thread Matt.Wilkie
Warmerdam ; Matt.Wilkie Cc: gdal dev Subject: Re: [gdal-dev] Nodata is None, but still has blanks? > I find that if I do: tiffcp sample-no-mask.tif,0 x0.tif I get an "x0.tif" with just the jpeg image, and not the mask. That may be helpful for you (without uncompressing the jpeg im

Re: [gdal-dev] Potential bug or misunderstanding. Please help

2022-01-19 Thread Matt.Wilkie
Would an in-memory /vsi also work as an intermediate? Expressed as windows CMD batch file I have handy: gdalwarp -of vrt –t_srs epsg:3578 %1 /vsistdout | gdal_translate /vsistdin/ %2 I should add that I use this like: project-to-local.bat input.tif outfile.tif -Matt

Re: [gdal-dev] Potential bug or misunderstanding. Please help

2022-01-19 Thread Matt.Wilkie
Would an in-memory /vsi also work as an intermediate? Expressed as windows CMD batch file I have handy: gdalwarp -of vrt –t_srs epsg:3578 %1 /vsistdout | gdal_translate /vsistdin/ %2 -Matt From: gdal-dev On Behalf Of Even Rouault Sent: January 19, 2022 9:00 AM To: Billy Babis ;

Re: [gdal-dev] Nodata is None, but still has blanks?

2022-01-18 Thread Matt.Wilkie
-4.3.0-h0c97f57_1.tar.bz2 -Matt From: Matt.Wilkie Sent: January 18, 2022 8:35 AM To: gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org> Subject: RE: [gdal-dev] Nodata is None, but still has blanks? Thank you Frank and Even. As is so often the case on this list, I now have a

Re: [gdal-dev] Nodata is None, but still has blanks?

2022-01-18 Thread Matt.Wilkie
? [0]: http://www.libtiff.org/index.html [1]: https://anaconda.org/conda-forge/libtiff/4.3.0/download/win-64/libtiff-4.3.0-h0c97f57_1.tar.bz2 -Matt From: Matt.Wilkie Sent: January 18, 2022 8:35 AM To: gdal-dev@lists.osgeo.org Subject: RE: [gdal-dev] Nodata is None, but still has blanks? Thank you

Re: [gdal-dev] Nodata is None, but still has blanks?

2022-01-18 Thread Matt.Wilkie
Thank you Frank and Even. As is so often the case on this list, I now have answers, and an education. I have been peripherally aware of the tiff tools from libtiff, but not enough so that they came to mind when seeking to puzzle out things. I’m now updated. ;-) -Matt

[gdal-dev] Nodata is None, but still has blanks?

2022-01-17 Thread Matt.Wilkie
I'm confused. The attached image has no Nodata, but when dropped into Qgis and ArcGIS large areas are still drawn transparently, and there is no 4th band for a mask or alpha. How is this happening? (I'm trying to get completely rid of all Nodata and Mask info, so as to start over creating one

Re: [gdal-dev] Edit geotiff in Gimp and don't lose metadata

2022-01-13 Thread Matt.Wilkie
To: Matt.Wilkie Cc: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] Edit geotiff in Gimp and don't lose metadata WoW. This is really awesome! Whish this existed many years ago... I was using GIMP and all that magic tools but without resampling or altering the size of the image, and carefully re-applying

Re: [gdal-dev] Edit geotiff in Gimp and don't lose metadata

2022-01-13 Thread Matt.Wilkie
- take a large multiband band geotiff with a mask side car file, squash the 3+ bands into 1 band Squashing might be unnecessary. If you open the .msk file first the document is set to 1-bit colour mode, then anything dropped on top is converted to the same mode. In effect squashing on

Re: [gdal-dev] Edit geotiff in Gimp and don't lose metadata

2022-01-13 Thread Matt.Wilkie
I discovered that starting with v2.10.24 Gimp knows about and keeps geotiff tags intact.[0] This means we can use tools like Magic Wand fuzzy select and a host of other tools to quickly fix a host of image issues that are difficult to address using command line tools and/or code. It gets

Re: [gdal-dev] Edit geotiff in Gimp and don't lose metadata

2022-01-13 Thread Matt.Wilkie
[5]: https://gis.stackexchange.com/a/115560 Oh look at that. Paint.net has supported geotiff tags since at least 2020. However it dumps a stack trace when I try and open the image I used in the Gimp experiment. The size perhaps. -Matt From: gdal-dev On Behalf Of matt.wil...@yukon.ca Sent:

[gdal-dev] Edit geotiff in Gimp and don't lose metadata

2022-01-13 Thread Matt.Wilkie
I am s happy today! I discovered that starting with v2.10.24 Gimp knows about and keeps geotiff tags intact.[0] This means we can use tools like Magic Wand fuzzy select and a host of other tools to quickly fix a host of image issues that are difficult to address using command line tools

Re: [gdal-dev] How to fix path for gdal .py scripts in Anaconda-Windows?

2022-01-12 Thread Matt.Wilkie
It was something of the sorts. The paths were actually fine, but the "default windows app" is activated when calling a file with a .py extension. So calling ogrmerge.py on the terminal would activate the default app instead of the activated python =/ If you want to manage from the command line

Re: [gdal-dev] How to fix path for gdal .py scripts in Anaconda-Windows?

2022-01-12 Thread Matt.Wilkie
Also see OSGEO4W make-bat-for-py.bat. Old but still works. ;-) ``` for %%g in (*.py) do ( echo @python "%%~dpnxg" %%* > %%~ng.bat ) ``` You should be able to insert settings if needed for environment variables (I didn’t test this particular

Re: [gdal-dev] RPCs in GeoTIFF

2022-01-10 Thread Matt.Wilkie
Thanks for the nudge. I implemented that code as a simple command line script a couple of months ago but neglected to put it anywhere someone else could use it. Fixed that today ;-) https://gist.github.com/maphew/29a73ee7a4517663cef5081c15799fc3 ``` '''Copy RPC metdata from IN raster to OUT

Re: [gdal-dev] follow warp with translate still?

2021-12-30 Thread Matt.Wilkie
There has been an enhancement in gdalwarp to support writing directly to drivers, like COG, that supports only the CreateCopy mode. So you don't need the 2 step process and can directly gdalwarp -of COG (it will more or less do the 2 steps internally) Good news! Thanks Even. I'm glad I took the

[gdal-dev] follow warp with translate still?

2021-12-30 Thread Matt.Wilkie
A few years ago it was best practice to follow a gdal warp process with translate in order to get better compression, and even overall speed although extra disk write is involved. Is this still the case? and when using COG output driver? e.g. gdalwarp infile.tif interim.tif gdal_translate -of

[gdal-dev] DIMAP format questions

2021-12-17 Thread Matt.Wilkie
Hi All, I'm getting conflicting information about the contents of some of the SPOT imagery we've received. Can someone here verify that this archive[0] contains ortho-rectified stereo imagery? If yes can gdal use it? how? The conversions I've tried so far use the RPC info but not the GCP,

Re: [gdal-dev] Transforming Raster Points from GeoTIFF with GCPs

2021-12-10 Thread Matt.Wilkie
and https://gdal.org/programs/gdaltransform.html -Matt From: gdal-dev On Behalf Of Simon Eves Sent: December 10, 2021 12:21 PM To: Yann-Sebastien Tremblay-Johnston Cc: gdal dev Subject: Re: [gdal-dev] Transforming Raster Points from GeoTIFF with GCPs *** External email: Do not click on

Re: [gdal-dev] Access window out of range error with standard utils

2021-12-09 Thread Matt.Wilkie
Thanks Even! I confirm `gdaladdo infile.ext.msk` is a valid workaround on all the problem files I've encountered it so far (about 7). -Matt From: Even Rouault Sent: December 9, 2021 1:28 PM To: Matt.Wilkie ; gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] Access window out of range error

Re: [gdal-dev] Access window out of range error with standard utils

2021-12-08 Thread Matt.Wilkie
Thanks Even, I've narrowed it down to the mask file. When I move it out of the way gdal_translate completes without error. Perhaps my optimization step of using nearblack on only the infrared band instead of all 4 is at fault? I agree it's hard to troubleshoot when the examples are not

[gdal-dev] Access window out of range error with standard utils

2021-12-08 Thread Matt.Wilkie
Hello, Why is gdal off by one for the access window? I'm using standard utilities, no custom code. This is only happening (so far) on one file out of dozens. ``` $ gdal_translate -co compress=zstd -co predictor=yes -co level=17 ^ -co bigtiff=yes -of cog INFILE.rgb-nir.vrt OUTFILE.tif Input

Re: [gdal-dev] georeferenced but not georeferenced?

2021-11-18 Thread Matt.Wilkie
Thank you Even, especially for highlighting a DEM should be used. I 'knew' that but hadn't taken the time to fully ingest the meaning. It's drastically changed how I'm approaching this project. -Matt From: Even Rouault Sent: November 10, 2021 2:16 PM To: Matt.Wilkie ; gdal-dev@lists.osgeo.org

[gdal-dev] georeferenced but not georeferenced?

2021-11-10 Thread Matt.Wilkie
Hi all, I'm creating images from SPOT6/7 archive DIMAP format. The resulting images display in the correct position in the world when combined with other sources, however all the usual tools say the images have no georeferencing. * If the images have no georeferencing how do ArcMap/Pro and

Re: [gdal-dev] Gdal-dev mail archive search sites?

2021-11-02 Thread Matt.Wilkie
> Hi Matt, > > mail-archive seems to be working now: > https://www.mail-archive.com/gdal-dev@lists.osgeo.org/ Oh that's excellent, they picked up the history as far back as April 2020 as well! -Matt ___ gdal-dev mailing list gdal-dev@lists.osgeo.org

Re: [gdal-dev] Gdal-dev mail archive search sites?

2021-11-01 Thread Matt.Wilkie
> What about starting this archive? > https://marc.info/?q=about#Add Thanks Markus, I’ve asked MARC about adding gdal-dev. I’ve done the same with Mail Archive[1] but the expected subscription confirmation doesn’t show up[2], though there is one dating back to 2008[3]. Maybe the old one is

[gdal-dev] Add a list to MARC: gdal-dev

2021-11-01 Thread Matt.Wilkie
Hi, would you please consider adding Gdal-dev to the MARC archives? https://lists.osgeo.org/mailman/listinfo/gdal-dev I'm not an administrator, just someone who relies on the community and is having difficulties consulting historical data now that Nabble has discontinued it's archive. Thank

Re: [gdal-dev] Python gdal.Translate(...) and unicode in Band Descriptions.

2021-10-14 Thread Matt.Wilkie
orking script is attached. -Matt -Original Message- From: Even Rouault Sent: October 14, 2021 12:11 PM To: Matt.Wilkie ; gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] Python gdal.Translate(...) and unicode in Band Descriptions. *** External email: Do not click on links or attachments exc

Re: [gdal-dev] Python gdal.Translate(...) and unicode in Band Descriptions.

2021-10-14 Thread Matt.Wilkie
> For Problem 2 there’s a new wrinkle: in spite of the warning about being > unable to save the .aux.xml info into the source zip sometimes it actually > does. I haven’t sorted out when it can and can’t yet. Solved it: the first time the script is run it saves the Color Interp and Descriptions

Re: [gdal-dev] Python gdal.Translate(...) and unicode in Band Descriptions.

2021-10-14 Thread Matt.Wilkie
Re: Problem 1 - Following a thread “Unicode support in GDAL”[0] I tried adding “ENCODING=UTF-8” but the COG driver doesn’t support it. So I’ve taken the practical route out of just spelling out micron instead of using the symbol. For Problem 2 there’s a new wrinkle: in spite of the warning

Re: [gdal-dev] Gdal-dev mail archive search sites?

2021-10-14 Thread Matt.Wilkie
Thanks Jeff! Here's a bookmarkable link that pre-arms the search box: https://www.google.ca/search?q=site:lists.osgeo.org/pipermail/gdal-dev/ -Matt -Original Message- From: gdal-dev On Behalf Of Jeff McKenna Sent: October 14, 2021 9:28 AM To: gdal-dev@lists.osgeo.org Subject: Re:

[gdal-dev] Gdal-dev mail archive search sites?

2021-10-14 Thread Matt.Wilkie
Hi All, Now that the gdal mail archive on Nabble is gone is what options are there for searching through historical threads? Thanks! Matt Wilkie Geomatics Developer & Administrator Environment | Technology, Innovation and Mapping T 867-667-8133 | Yukon.ca Hours: 08:30-16:30,

Re: [gdal-dev] Enabling default compression for COG ?

2021-10-14 Thread Matt.Wilkie
+1 from me. I think it's a great idea. -Matt ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Python gdal.Translate(...) and unicode in Band Descriptions.

2021-10-13 Thread Matt.Wilkie
Hi, The attached python script that reads from a vendor’s SPOT6 distribution archive and creates a cloud optimized geotiff. Reading and writing works, but the band description does not take properly. Problem 1: The greek letter Mu or

[gdal-dev] Generating data-only footprints of imagery

2021-08-18 Thread Matt.Wilkie
Hi Folks, The standard image footprint mechanism such as that produced by gdaltindex draws a box around the raster but that often includes large nodata areas which we don't care about. Generating polygons of only the data carrying area is something I've often wished for but not devoted the

Re: [gdal-dev] Motion: Approve Even Rouault as a contracted GDAL maintainer

2021-08-17 Thread Matt.Wilkie
As a result of our fundraising activity and development of NumFOCUS as a financial conduit, it is my pleasure to put forward a motion to approve Even Rouault as a contracted GDAL maintainer for the year 2021-2022 beginning on August 1st, 2021 and ending July 31st, 2022. Excellent news! It’s

Re: [gdal-dev] Downscaled geotiff with overviews

2021-07-30 Thread Matt.Wilkie
Watching this thread with interest. I’ve wanted to do the same, but from python. -Matt From: gdal-dev On Behalf Of Javier Jimenez Shaw Sent: July 30, 2021 11:19 AM To: gdal dev Subject: [gdal-dev] Downscaled geotiff with overviews Hi I want to make a (power-of-two) downscaled version of a

Re: [gdal-dev] Making use of SPOT imagery mask files

2021-07-29 Thread Matt.Wilkie
Aha! A small bit of progress: $ gdaltransform -rpc DS_SPOT6_201308032015087_LM1_FR1_FR1_FR1_W137N65_10315\PROD_SPOT6_001\VOL_SPOT6_001_A\IMG_SPOT6_MS_001_A\DIM_SPOT6_MS_201308032015087_SEN_1.XML Enter column line values separated by space, and press Return. 0.0 57083.0 -137.766376889524

[gdal-dev] Making use of SPOT imagery mask files

2021-07-28 Thread Matt.Wilkie
I'm working with SPOT 6 and 7 imagery delivered in DIMAP format. I've figured out how to extract the multispectral and pan-chromatic bands into geotiff with gdal-translate so they're easier to work with. ((cross posted to

[gdal-dev] What is the gdal_translate scale exponent option for?

2021-07-12 Thread Matt.Wilkie
Hi Folks, gdal_translate has the option -exponent to use in conjunction with -scale that will apply non-linear scaling with a power function. exp_val is the exponent of the power function (must be positive).

[gdal-dev] DIMAP driver questions

2021-07-09 Thread Matt.Wilkie
Hi, The DIMAP driver page lists names of the index files to look at. I don't have any of these filenames in our SPOT6 folder tree. Is the driver description page incomplete or perhaps I've been given a different DIMAP version? METADATA.DIM (DIMAP

Re: [gdal-dev] Install GDAL on Arcmap 10.7

2021-07-09 Thread Matt.Wilkie
Hello Sayena, Gdal doesn’t have a graphic interface, it’s all command line utilities and programmer libraries. However Qgis (https://qgis.org/) is very well integrated with Gdal and is similar to ArcMap in a lot of ways and the members of the Qgis

Re: [gdal-dev] Lost band name during merge

2021-07-09 Thread Matt.Wilkie
Hi Lorenzo, Attached is a short python script to copy band info from one file to another. It’s not a solution but might be helpful. Usage is gdal-copy-band-info [in_file] [band #] [out_file] [band #] cheers, -Matt From: gdal-dev On Behalf Of Lorenzo Di Giacomo Sent: July 9, 2021 7:30 AM

Re: [gdal-dev] a flag for translate stop or abort on error?

2021-07-08 Thread Matt.Wilkie
Good to know, thanks. For the rest of the set I'm going to unpack the archives first instead using /vsizip. I'm making several passes over the contents and some testing has shown that is slowing things down a lot. If I manage to find a reproducible set I'll let you know. -Matt From: Even

Re: [gdal-dev] TIFFFetchDirectory:Sanity check on directory count failed

2021-07-08 Thread Matt.Wilkie
Ah, okay. As nothing seems amiss with the translated file I'll just note the name and error and keep going. Thanks for the info Even. -Matt ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] TIFFFetchDirectory:Sanity check on directory count failed

2021-07-08 Thread Matt.Wilkie
Related to my previous question today, what is the nature of "TIFFFetchDirectory:Sanity check on directory count failed" error? :: gdal_translate -co compress=zstd -co predictor=yes -co level=17 -co bigtiff=yes -of cog -a_nodata 0 Finlayson9488_SP7_08Aug2017_150cm.rgb-nir.vrt

[gdal-dev] a flag for translate stop or abort on error?

2021-07-08 Thread Matt.Wilkie
Hi, Is there a flag or something to instruct gdal_translate to stop or abort when it encounters an error? gdal_translate -co compress=zstd -co predictor=yes -co level=17 -co bigtiff=yes -of cog -a_nodata 0 Finlayson9488_SP6_24Aug2016_150cm.rgb-nir.vrt

Re: [gdal-dev] gdaladdo bigtiff err but --config BIGTIFF was used

2021-07-08 Thread Matt.Wilkie
cogger v0.0.5 now supports .ovr external overviews. It will automatically set the output cog to bigtiff if required, so your use-case should now be covered. Oh that’s great, thank you! ___ gdal-dev mailing list gdal-dev@lists.osgeo.org

Re: [gdal-dev] python: set CacheMax to percent, MB, GB?

2021-06-17 Thread Matt.Wilkie
Ohhh. Sorry for being dense. Sometimes the hammer has to strike a couple times... ;-) Thanks! ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] python: set CacheMax to percent, MB, GB?

2021-06-17 Thread Matt.Wilkie
you could also call gdal.SetConfigOption('GDAL_CACHEMAX', string), but you need to do that *before* reading any raster in the process (or any call to gdal.SetCacheMax/GetCacheMax), otherwise it will be ignored. Thanks! ...Later: How to use it? It doesn't seem to work for me: ~~~ >>>

[gdal-dev] python: set CacheMax to percent, MB, GB?

2021-06-17 Thread Matt.Wilkie
I see in https://gdal.org/python/osgeo.gdal-module.html#SetCacheMax we set the value using bytes. Is there a prebuilt method to convert units like percent, megabytes, gigabytes to bytes or is it up to us to build one? Matt ___ gdal-dev mailing list

Re: [gdal-dev] gdaladdo bigtiff err but --config BIGTIFF was used

2021-06-14 Thread Matt.Wilkie
> You can create external overviews with the -ro switch Yeah, I have those. I was trying to create internal ones instead in order to use cogger ;-) (And behind all of this, a weeping and wailing and gnashing of teeth at the upstream person or persons who decided that creating jpegs and

Re: [gdal-dev] gdaladdo bigtiff err but --config BIGTIFF was used

2021-06-14 Thread Matt.Wilkie
> Your initial test.tif file needs to be a bigtiff, i.e. have been > created with -co "BIGTIFF=YES". you can then drop the bigtiff > config arguments of your gdaladdo call. Ahh, thanks Thomas, that indeed is the problem. Unfortunately for me it means in this case I'm stuck, as the original is

[gdal-dev] gdaladdo bigtiff err but --config BIGTIFF was used

2021-06-09 Thread Matt.Wilkie
I am getting tiff file size exceeded error but am using bigtiff creation options. ~~~ $ gdaladdo -r gauss ^ --config COMPRESS_OVERVIEW WEBP ^ --config PHOTOMETRIC_OVERVIEW YCBCR ^ --config INTERLEAVE_OVERVIEW PIXEL ^ --config QUALITY 87 ^ --config BIGTIFF YES ^ --config BIGTIFF_OVERVIEW YES ^

Re: [gdal-dev] cogger output has no overviews?

2021-06-08 Thread Matt.Wilkie
Thanks Thomas. Does cogger need a mask band to be internal as well, and not a sidecar file? -Mat ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] cogger output has no overviews?

2021-06-08 Thread Matt.Wilkie
if they contain overviews ~~~ -Matt From: Matt.Wilkie Sent: June 8, 2021 11:19 AM To: Gdal Dev Subject: cogger output has no overviews? Hi, I took cogger for a first test drive and I'm confused. I'm relatively new to the world of Cloud Optimized Geotiffs, so I don't know if my confusion is about

[gdal-dev] cogger output has no overviews?

2021-06-08 Thread Matt.Wilkie
Hi, I took cogger for a first test drive and I'm confused. I'm relatively new to the world of Cloud Optimized Geotiffs, so I don't know if my confusion is about the COG format itself or cogger: I fed a jpeg-in-geotiff file with external overviews (tif.ovr) and mask (tif.msk) to cogger. The

Re: [gdal-dev] Introducing the cogger and godal projects

2021-06-07 Thread Matt.Wilkie
geotiff?) -Matt From: Even Rouault Sent: June 4, 2021 9:33 AM To: thomas bonfort ; Matt.Wilkie Cc: Gdal Dev Subject: Re: [gdal-dev] Introducing the cogger and godal projects *** External email: Do not click on links or attachments except from trusted senders

Re: [gdal-dev] Introducing the cogger and godal projects

2021-06-04 Thread Matt.Wilkie
Is cogger specifically for the scenario where your converting a large imagery library that already exists, but isn't cloud optimized? i.e. Does it offer any advantages over the one step `gdal_translate -of cog ...` when starting fresh? Cheers, Matt Geomatics Analyst | Environment | T

[gdal-dev] Has gdal_ls.py been replaced?

2021-05-25 Thread Matt.Wilkie
Hi, I only see gdal_ls.py in historical versions: https://github.com/OSGeo/gdal/blob/e346a31f3119f3bb3f77dd7ddc8037b3049c2026/gdal/swig/python/samples/gdal_ls.py I'm picking up on a very old suggestion[0] to use it as part of a chain to help gdaltindex generate footprints for images inside .zip

Re: [gdal-dev] gdaldem color-relief -exact_color_entry results in RGB 000

2021-05-18 Thread Matt.Wilkie
A guess; the tiff is actually stored as 32 bit float so the values have decimals, and thus don’t exactly match the colour table. Maybe the Qgis layer is set to report without decimals? Hmm, maybe not. Something similar was reported in GIS stack a few years ago:

Re: [gdal-dev] Gdal doesn't see NaN values as NoData?

2021-05-05 Thread Matt.Wilkie
>> a) Why does gdalinfo not report the presence of nan? Meaning: how do I know >> when I will need to invoke (b)? > >It doesn't report nan as the nodata value, because the file lacks the >GDAL_NODATA tiff tag set at "nan". So the issue is on the producer side. Ok, noted. I will try trace this

Re: [gdal-dev] Gdal doesn't see NaN values as NoData?

2021-05-05 Thread Matt.Wilkie
Hi Folks, I have a 32bit NDVI image that I'm told was exported from Google Earth Engine, with possibly a trip through R along the way. In ArcGIS desktop and Qgis there are "nan" values reported for the image collar, however gdalinfo doesn't say anything about them. If I project the image in

Re: [gdal-dev] Gdal_translate batch processing

2021-05-04 Thread Matt.Wilkie
Hi Gulnihal, Something like this should work: pushd C:\Users\HP\Desktop\modis for /r %F in (*.hdf) do @echo gdal_translate -sds %F %~dpnF.tif popd This is a dry run to show what it will attempt, it won't actually do anything. Remove the "@echo" part to make it active. Double the %

[gdal-dev] Comprehensive list of values for ColorInterp

2021-03-31 Thread Matt.Wilkie
What are the useful and/or valid values for the `ColorInterp` setting of a raster band? I have seen: Red, Green, Blue, Gray, Undefined. Is this a complete list or can there be values like "Mask", "Infrared", or...? [cid:image001.png@01D7263D.FF7F1CA0] Matt Wilkie Geomatics Analyst Environment

Re: [gdal-dev] More info on gdalwarp resampling methods?

2021-03-31 Thread Matt.Wilkie
#3632 gdalwarp mode resample much more expensive than needed<https://github.com/OSGeo/gdal/issues/3632> -Matt From: Even Rouault Sent: March 31, 2021 10:27 AM To: Matt.Wilkie ; gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] More info on gdalwarp resampling methods? *** External

Re: [gdal-dev] More info on gdalwarp resampling methods?

2021-03-31 Thread Matt.Wilkie
https://sft.gov.yk.ca/link/gV2K9F4PyH1PTJLQH8fy99 until April 14. (500mb). -Matt From: Even Rouault Sent: March 30, 2021 11:39 PM To: Matt.Wilkie ; gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] More info on gdalwarp resampling methods? Matt, (leaving out a bit your general question

[gdal-dev] More info on gdalwarp resampling methods?

2021-03-30 Thread Matt.Wilkie
Hi folks, I'm seeking more information on the various gdalwarp resampling methods, the scenarios where they are most usefully applied, and the converse: where they should not be used. It would be especially helpful if there were also visuals. For example I know what min/max/median are when

Re: [gdal-dev] gdalbuildvrt and band descriptions

2021-03-30 Thread Matt.Wilkie
Done! Enable gdalbuildvrt to retain metadata such as band description https://github.com/OSGeo/gdal/issues/3627 -Matt From: Even Rouault Sent: March 26, 2021 5:09 PM To: Matt.Wilkie ; gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] gdalbuildvrt and band descriptions Matt, copying over band

[gdal-dev] gdalbuildvrt and band descriptions

2021-03-26 Thread Matt.Wilkie
Hi Folks, `gdal_translate -b 1 infile.tif outfile.vrt` will copy the band 1 description to the output file. `gdalbuild.vrt` on the other hand leaves the band descriptions behind. Is there a way to have vrt carry the description across also? The project behind the question is to keep only the

Re: [gdal-dev] Motion: RFC 78: gdal-utils package

2021-03-26 Thread Matt.Wilkie
Thanks, that’s exactly the info I needed to get oriented. -Matt From: Idan Miara Sent: March 26, 2021 5:29 AM To: Matt.Wilkie Cc: gdal dev Subject: Re: [gdal-dev] Motion: RFC 78: gdal-utils package Hi Matt, Point noted. I've updated the Summary section: Idan Summary This RFC suggests

[gdal-dev] interpreting gdalcompare results

2021-03-24 Thread Matt.Wilkie
Hi, How do I find out more info on what the differences are from gdalcompare? What does "New: n" mean? $ gdalcompare source-image.tif xxx-001.tif Files differ at the binary level. Band 1 overview count difference: Golden: 0 New:5 Band 2 overview count difference: Golden: 0 New:

Re: [gdal-dev] SUMMARY RE: What is lost when converting 12 bit imagery to 8 bit?

2021-03-23 Thread Matt.Wilkie
The summary also posted to the GIS Stack Exchange thread https://gis.stackexchange.com/questions/390315/what-is-lost-when-converting-12-bit-imagery-to-8-bit/. Also in the credits I neglected to include @nils-erik-jørgensen. -Matt From: Matt.Wilkie Sent: March 23, 2021 4:27 PM To: 'gdal dev

[gdal-dev] SUMMARY RE: What is lost when converting 12 bit imagery to 8 bit?

2021-03-23 Thread Matt.Wilkie
The position I've moved to after reading through and thinking about all the responses here and gdal-dev (here) is we need to ask

Re: [gdal-dev] What is lost when converting 12 bit imagery to 8 bit?

2021-03-18 Thread Matt.Wilkie
analysis is primary goal, and we do this also. Cheers, -Matt From: Frank Warmerdam Sent: March 17, 2021 8:29 PM To: Patrick Young Cc: Matt.Wilkie ; gdal dev Subject: Re: [gdal-dev] What is lost when converting 12 bit imagery to 8 bit? Patrick, FWIW, Rob's post is on the process he uses

[gdal-dev] What is lost when converting 12 bit imagery to 8 bit?

2021-03-17 Thread Matt.Wilkie
SPOT 6/7 satellite imagery is captured with a dynamic range of 12 bits per pixel per channel (ref). However almost all of the SPOT imagery I have seen in use has been 8 bits per channel, and split into RGB natural colour (Bands-321) and Near-infrared-RG false

[gdal-dev] lossless jpeg crop in gdal?

2021-03-16 Thread Matt.Wilkie
I recently learned that jpegtran from the JPEG library has the ability to crop and save a new jpeg losslessly: These transformations are each completely lossless and reversible. The transformations on the image data comprise: · eliminate non-standard application-specific data inserted

Re: [gdal-dev] gdal_retile: output isn't georeferenced?

2021-03-16 Thread Matt.Wilkie
> Is the "main" or "image" file geo-referenced ? > It may be that the .tif.msk mask files are assumed to be the same as the > matching .tif file. Ahh yes that's it. The tile main image is georeferenced and it's just the mask on it's own which is not. Thanks for pointing that out. -Matt

Re: [gdal-dev] ZSTD compression level 17 smaller than 18+?

2021-03-16 Thread Matt.Wilkie
Oh that’s so weird. Well that puts my take-away as: just use Level=17 and forget the rest exist! -Matt From: Daniel Evans Sent: March 15, 2021 2:36 AM To: Matt.Wilkie ; gdal-dev@lists.osgeo.org Subject: RE: ZSTD compression level 17 smaller than 18+? *** External email: Do not click on links

Re: [gdal-dev] nearblack on multi GB images very slow

2021-03-14 Thread Matt.Wilkie
>You need to count tiles in the row, not pixels. > >ceil(126015/128)*3*8*128*128 = 387 118 080 = half a gigabyte. Thank you! That made a huge difference. With GDAL_CACHEMAX=1024 it finished in about 40 minutes. -Matt ___ gdal-dev mailing list

[gdal-dev] ZSTD compression level 17 smaller than 18+?

2021-03-14 Thread Matt.Wilkie
So this is curious: output to Cloud optimized Geotiff with ZSTD compression to Level 17 is smaller than levels 18-22. Or at least that's my results when testing compression results with Gdal 3.1 via Qgis 3.18 on Windows 10. I expected compression levels to get smaller with higher levels, at

Re: [gdal-dev] vrt guidance: 31 band classified raster

2019-10-03 Thread Matt.Wilkie
> It was quite a bit of work to get this far and I'm not looking forward to > doing this 30 more times. On this point, discovery of the day is Easy XML Editor which allows editing some attributes as if in a spreadsheet. It's alleviated much difficulty, but

[gdal-dev] vrt guidance: 31 band classified raster

2019-10-03 Thread Matt.Wilkie
Hello gdal-dev, it's been a long time! I'm happy to be digging into raster data building again for a change, but could use some nudges in the right direction(s). A few weeks ago Nasa released Landsat-derived Annual Dominant Land Cover Across ABoVE Core Domain,

  1   2   >