Hmm, very tricky situations. Apparently from Julia things remain in memory and
it remembers previous errors.
From a clean Juia REPL start this works (even with authentication)
julia> set_config_option("GDAL_HTTP_COOKIEFILE", joinpath(tempdir(),
"cookies.txt"))
julia> set_config_option("GDAL_HTTP_COOKIEJAR", joinpath(tempdir(),
"cookies.txt"))
julia> set_config_option("GDAL_DISABLE_READDIR_ON_OPEN","YES")
julia> set_config_option("CPL_VSIL_CURL_ALLOWED_EXTENSIONS","TIF")
julia> set_config_option("CPL_VSIL_CURL_USE_HEAD","FALSE")
gdalinfo("/vsicurl/https://lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30.015/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif";)
"Driver: GTiff/GeoTIFF\nFiles:
/vsicurl/https://lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30.015/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif\nSize
is 3660, 3660\nCoordinate System is:\nPROJCRS[\"UTM Zone 10, Northern
Hemisphere\",\n BASEGEOGCRS[\"Unknown datum based upon the WGS 84
ellipsoid\",\n DATUM[\"Not_specified_based_on_WGS_84_spheroid\",\n …..
However, if run the gdalinfo(…) first and set the options after, then any
posterior call to gdalinfo will return that “not supported file format” error.
In fact the errors evolve like this (after another clean start)
julia>
gdalinfo("/vsicurl/https://lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30.015/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif";)
ERROR 11: HTTP response code: 206
julia>
gdalinfo("/vsicurl/https://lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30.015/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif";)
ERROR 4:
`/vsicurl/https://lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30.015/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif'
does not exist in the file system, and is not recognized as a supported
dataset name.
From: Even Rouault <even.roua...@spatialys.com>
Sent: Tuesday, October 5, 2021 8:11 PM
To: Joaquim Manuel Freire Luís <jl...@ualg.pt>; gdal-dev@lists.osgeo.org
Subject: Re: [gdal-dev] How to access remote data that requires authentication?
There are a few historically exported VSIInstaller symbols, but
VSIInstallCurlFileHandler is never exported. None of those symbols need to be
explicitly called as GDAL will automatically register the builtin-handlers.
Perhaps your gdal.dll lacks curl support ? I assumed you pass the needed config
options / env variables for that service. Perhaps you could try with something
simpler that doesn't require authentication.
Le 05/10/2021 à 21:05, Joaquim Manuel Freire Luís a écrit :
Even, sorry to continue this.
Now is from Julia
julia>
gdalinfo("/vsicurl/https://lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30.015/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif";)
ERROR 4:
`/vsicurl/https://lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30.015/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif'
not recognized as a supported file format.
I saw a reference to VSIInstallCurlFileHandler() so I thought “right I need
this guy too” but when I try to install it
julia> GMT.Gdal.VSIInstallCurlFileHandler()
ERROR: could not load symbol "VSIInstallCurlFileHandler":
The specified procedure could not be found.
And indeed my gdal.dll has a couple of VSIInstaller symbols but not this one
(nor zip, gzip and others).
Is the “not supported file format” related to this? Shouldn’t that symbol be
exported to the dll?
From: gdal-dev
<gdal-dev-boun...@lists.osgeo.org><mailto:gdal-dev-boun...@lists.osgeo.org> On
Behalf Of Joaquim Manuel Freire Luís
Sent: Tuesday, October 5, 2021 5:59 PM
To: Even Rouault
<even.roua...@spatialys.com><mailto:even.roua...@spatialys.com>;
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
Subject: Re: [gdal-dev] How to access remote data that requires authentication?
>The mention to _netrc is weird...
On Windows is _netrc, not .netrc
And I finally made it (twisted). The problem was that I detest where MS decides
is my home dir (and all the hidden dirs stuff that it puts there) and always
have had a home at “HOME=c:\j” and it’s there that libcurl (I presume) seeks
for the netrc file.
Thanks for all the tips. But I still wonder how GMT manages to read the file
without anything of this.
For the record if any other Windows user needs it
gdalinfo
/vsicurl/https://lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30.015/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif
--config GDAL_DISABLE_READDIR_ON_OPEN EMPTY_DIR --config
CPL_VSIL_CURL_USE_HEAD FALSE --config GDAL_HTTP_COOKIEFILE c:/TEMP/cookies.txt
--config GDAL_HTTP_COOKIEJAR c:/TEMP/cookies.txt
Driver: GTiff/GeoTIFF
Files:
/vsicurl/https://lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30.015/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif
Size is 3660, 3660
Coordinate System is:
PROJCRS["UTM Zone 10, Northern Hemisphere",
Regarding authentication issues, this service is quite annoying and requires
enabling cookies. See
https://lists.osgeo.org/pipermail/gdal-dev/2021-October/054728.html
You need to add things like --config GDAL_HTTP_COOKIEFILE /tmp/cookies.txt
--config GDAL_HTTP_COOKIEJAR /tmp/cookies.txt
Le 05/10/2021 à 18:20, Joaquim Manuel Freire Luís a écrit :
OK, tried more things from that thread.
gdalinfo
/vsicurl/https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/HLSL30.020/HLS.L30.T10TEK.2021192T184511.v2.0/HLS.L30.T10TEK.2021192T184511.v2.0.B04.tif
--config GDAL_DISABLE_READDIR_ON_OPEN EMPTY_DIR --config CPL_CURL_VERBOSE ON
--config CPL_VSIL_CURL_USE_HEAD FALSE
* Couldn't find host data.lpdaac.earthdatacloud.nasa.gov in the _netrc file;
using defaults ...
OK, right it's not there.
But latter down it says
* Couldn't find host urs.earthdata.nasa.gov in the _netrc file; using defaults
Now this is not right.
< HTTP/1.1 401 Unauthorized
...
< WWW-Authenticate: Basic realm="Please enter your Earthdata Login credentials.
If you do not have a Earthdata Login, create one at
https://urs.earthdata.nasa.gov//users/new<https://urs.earthdata.nasa.gov/users/new>"
...
ERROR 11: HTTP response code: 401
So I'm back to the Authentication problem. I do have an _netrc file (and .netrc
btw) in my home dir as well as current dir but it does seem to find it. Is
there something else that I must to in order to that file be found/used?
-----Original Message-----
From: thomas bonfort <thomas.bonf...@gmail.com><mailto:thomas.bonf...@gmail.com>
Sent: Tuesday, October 5, 2021 4:31 PM
To: Joaquim Manuel Freire Luís <jl...@ualg.pt><mailto:jl...@ualg.pt>
Subject: Re: [gdal-dev] How to access remote data that requires authentication?
for the 206 there seems to be a similar issue posted here a few days ago,
search for "Problem accessing NASA Cloud Optimized GeoTIFF data"
in the archives
On Tue, Oct 5, 2021 at 5:26 PM Joaquim Manuel Freire Luís
<jl...@ualg.pt><mailto:jl...@ualg.pt> wrote:
you should also change your password, now you have posted it on a
public mailing list :/
Shit, thanks for spotting it.
But it doesn't work with /vsicur/ neither (had tried it before). Now
the error is 206
gdalinfo
/vsicurl/https://user:p...@lpdaac.earthdata.nasa.gov/lp-prod-protected
/HLSS30.015/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif
ERROR 11: HTTP response code: 206
On Tue, Oct 5, 2021 at 5:12 PM Joaquim Manuel Freire Luís
<jl...@ualg.pt><mailto:jl...@ualg.pt> wrote:
Hi,
I’ve read a lot of the docs, tried many -co options but can’t get through this
mystery.
I can access the data through GMT, which uses GDAL to do this job,
but can’t do it with GDAL directly
gdalinfo
https://user:p...@lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30
.0 15/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif
ERROR 1: HTTP error code : 401
gdalinfo failed - unable to open
'https://jluis:abaixo0earthd...@lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30.015/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif'.
So passing longin:password via url does not work either. But it does
if indirectly used
grdinfo
https://user:p...@lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30
.0 15/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif
HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: Title: Grid imported via
GDAL
HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: Command:
HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: Remark:
HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: Pixel node registration
used [Cartesian grid]
HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: Grid file format: gd =
Import/export through GDAL
HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: x_min: 499980 x_max:
609780 x_inc: 30 name: x n_columns: 3660
HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: y_min: 4390200 y_max:
4500000 y_inc: 30 name: y n_rows: 3660
HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: v_min: -0.0149 v_max:
0.8833 name: z
HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: scale_factor: 0.0001
add_offset: 0 packed z-range: [-149,8833]
+proj=utm +zone=10 +ellps=WGS84 +units=m +no_defs
Joaquim
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/gdal-dev
--
http://www.spatialys.com
My software is free, but my time generally not.
--
http://www.spatialys.com
My software is free, but my time generally not.
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
