[gdal-dev] accessing zip files by URL
Hi, How do we access to a compressed files by URL? We can do that, can't we? As an example I try this (on Windows) gdalinfo /vsizip/C:\http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.dem.zip ERROR 4: `/vsizip/C:\http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.dem.zip' does not exist in the file system, and is not recognised as a supported dataset name. and this gdalinfo /vsizip//http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.dem.zip but the error message is similar. The idea is to fetch the data directly into GMT, but first I need to understand well how it works. Thanks Joaquim Luis ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] accessing zip files by URL
Joaguim, I think /vsizip/vsicurl/http://path.to/the/file.zip works. You have to initialize the zip file and remote file handlers using VSIInstallCurlFileHandler() and VSIInstallZipFileHandler() Refer: http://trac.osgeo.org/gdal/wiki/UserDocs/ReadInZip On Thu, Mar 24, 2011 at 8:48 PM, Joaquim Luis jl...@ualg.pt wrote: Hi, How do we access to a compressed files by URL? We can do that, can't we? As an example I try this (on Windows) gdalinfo /vsizip/C:\ http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.dem.zip ERROR 4: `/vsizip/C:\ http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.dem.zip' does not exist in the file system, and is not recognised as a supported dataset name. and this gdalinfo /vsizip// http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.dem.zip but the error message is similar. The idea is to fetch the data directly into GMT, but first I need to understand well how it works. Thanks Joaquim Luis ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev -- Best regards, Chaitanya kumar CH. /tʃaɪθənjə/ /kʊmɑr/ +91-9494447584 17.2416N 80.1426E ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] accessing zip files by URL
Joaquim, The correct syntax would be : gdalinfo /vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.dem.zip/W020N90.DEM or just : gdalinfo /vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.dem.zip because the zip file only contains one single file (W020N90.DEM) ... But the W020N90.DEM inside the zip file isn't directly recognized by GDAL (even if you download it and unzip the file). It's just a RAW file, that neads an header to tell the dimension, georeferencing, datatype etc, so the above won't directly work. You can for example create a VRT that refers to the raw file : VRTDataset rasterXSize=4800 rasterYSize=6000 GeoTransform-20, 8.3338e-03, 0, 90, 0, -8.3338e-03/GeoTransform VRTRasterBand dataType=Int16 band=1 subClass=VRTRawRasterBand NoDataValue-/NoDataValue SourceFilename relativetoVRT=0/vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.dem.zip/SourceFilename ByteOrderMSB/ByteOrder /VRTRasterBand /VRTDataset I've deduced this VRT from the http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.hdr.zip file that sits next to the .dem.zip file. It's a shame that they didn't put the .hdr and the .dem file inside the same zip. It would have they been possible to open it directly... With the SRTM3 in HGT format, you can directly do : gdalinfo /vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/Africa/N00E006.hgt.zip Best regards, Even Hi, How do we access to a compressed files by URL? We can do that, can't we? As an example I try this (on Windows) gdalinfo /vsizip/C:\http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.de m.zip ERROR 4: `/vsizip/C:\http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.d em.zip' does not exist in the file system, and is not recognised as a supported dataset name. and this gdalinfo /vsizip//http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.dem. zip but the error message is similar. The idea is to fetch the data directly into GMT, but first I need to understand well how it works. Thanks Joaquim Luis ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] accessing zip files by URL
Le jeudi 24 mars 2011 18:30:57, Chaitanya kumar CH a écrit : Joaguim, I think /vsizip/vsicurl/http://path.to/the/file.zip works. You have to initialize the zip file and remote file handlers using VSIInstallCurlFileHandler() and VSIInstallZipFileHandler() Actually, they are automagically called when needed (i.e. when the first access to the VSI Virtual File API is done), so no need to explicitely call them. Refer: http://trac.osgeo.org/gdal/wiki/UserDocs/ReadInZip On Thu, Mar 24, 2011 at 8:48 PM, Joaquim Luis jl...@ualg.pt wrote: Hi, How do we access to a compressed files by URL? We can do that, can't we? As an example I try this (on Windows) gdalinfo /vsizip/C:\ http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.dem.zip ERROR 4: `/vsizip/C:\ http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.dem.zip' does not exist in the file system, and is not recognised as a supported dataset name. and this gdalinfo /vsizip// http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.dem.zip but the error message is similar. The idea is to fetch the data directly into GMT, but first I need to understand well how it works. Thanks Joaquim Luis ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] accessing zip files by URL
Even, Chaitanya Thanks for hint. I did read the http://trac.osgeo.org/gdal/wiki/UserDocs/ReadInZip but it hasn't any mention to the need of using vsicurl I was about to do the tests you did, so thank you also for that. The point here is not to read that file in particular. I just wanted to use one as example and one that is not a *.tar.gz that the docs say it won't work. Ok, I have enough information to make it possible to read some compressed files sitting somewhere in the web and have the data land directly in the internals of GMT. Thanks Joaquim Joaquim, The correct syntax would be : gdalinfo /vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.dem.zip/W020N90.DEM or just : gdalinfo /vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.dem.zip because the zip file only contains one single file (W020N90.DEM) ... But the W020N90.DEM inside the zip file isn't directly recognized by GDAL (even if you download it and unzip the file). It's just a RAW file, that neads an header to tell the dimension, georeferencing, datatype etc, so the above won't directly work. You can for example create a VRT that refers to the raw file : VRTDataset rasterXSize=4800 rasterYSize=6000 GeoTransform-20, 8.3338e-03, 0, 90, 0, -8.3338e-03/GeoTransform VRTRasterBand dataType=Int16 band=1 subClass=VRTRawRasterBand NoDataValue-/NoDataValue SourceFilename relativetoVRT=0/vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.dem.zip/SourceFilename ByteOrderMSB/ByteOrder /VRTRasterBand /VRTDataset I've deduced this VRT from the http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.hdr.zip file that sits next to the .dem.zip file. It's a shame that they didn't put the .hdr and the .dem file inside the same zip. It would have they been possible to open it directly... With the SRTM3 in HGT format, you can directly do : gdalinfo /vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/Africa/N00E006.hgt.zip Best regards, Even Hi, How do we access to a compressed files by URL? We can do that, can't we? As an example I try this (on Windows) gdalinfo /vsizip/C:\http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.de m.zip ERROR 4: `/vsizip/C:\http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.d em.zip' does not exist in the file system, and is not recognised as a supported dataset name. and this gdalinfo /vsizip//http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.dem. zip but the error message is similar. The idea is to fetch the data directly into GMT, but first I need to understand well how it works. Thanks Joaquim Luis ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] accessing zip files by URL
Le jeudi 24 mars 2011 19:03:57, Joaquim Luis a écrit : Even, Chaitanya Thanks for hint. I did read the http://trac.osgeo.org/gdal/wiki/UserDocs/ReadInZip but it hasn't any mention to the need of using vsicurl Yes I am aware that it is lightly documented. I've added recently a few pointers though : - in the GDALOpen() doc : http://gdal.org/gdal_8h.html#e97be045eb4701183ad332ffce29745b which points to : http://gdal.org/cpl__vsi_8h.html#4f791960f2d86713d16e99e9c0c36258 Which mentions : This special file handler can be combined with other virtual filesystems handlers, such as /vsizip. For example, /vsizip//vsicurl/path/to/remote/file.zip/path/inside/zip Perhaps a more didactic page would be needed in the wiki. Any taker :-) ? I remind that the wiki is open in writing for everyone with an osgeo id... I was about to do the tests you did, so thank you also for that. The point here is not to read that file in particular. I just wanted to use one as example and one that is not a *.tar.gz that the docs say it won't work. Ah... well actually reading in a .tar or a .tar.gz is now supported ( http://gdal.org/cpl__vsi_8h.html#d6dd983338849e7da4eaa88f6458ab64 ). Seems that I have changed my mind since. But seeking will still be very slow of course (especially if you combine with /vsicurl !). Ok, I'm going to rectify the page about that point. Ok, I have enough information to make it possible to read some compressed files sitting somewhere in the web and have the data land directly in the internals of GMT. Thanks Joaquim Joaquim, The correct syntax would be : gdalinfo /vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w02 0n90.dem.zip/W020N90.DEM or just : gdalinfo /vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w02 0n90.dem.zip because the zip file only contains one single file (W020N90.DEM) ... But the W020N90.DEM inside the zip file isn't directly recognized by GDAL (even if you download it and unzip the file). It's just a RAW file, that neads an header to tell the dimension, georeferencing, datatype etc, so the above won't directly work. You can for example create a VRT that refers to the raw file : VRTDataset rasterXSize=4800 rasterYSize=6000 GeoTransform-20, 8.3338e-03, 0, 90, 0, -8.3338e-03/GeoTransform VRTRasterBand dataType=Int16 band=1 subClass=VRTRawRasterBand NoDataValue-/NoDataValue SourceFilename relativetoVRT=0/vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/ SRTM30/w020n90/w020n90.dem.zip/SourceFilename ByteOrderMSB/ByteOrder /VRTRasterBand /VRTDataset I've deduced this VRT from the http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.hdr.zip file that sits next to the .dem.zip file. It's a shame that they didn't put the .hdr and the .dem file inside the same zip. It would have they been possible to open it directly... With the SRTM3 in HGT format, you can directly do : gdalinfo /vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/Africa/N00E0 06.hgt.zip Best regards, Even Hi, How do we access to a compressed files by URL? We can do that, can't we? As an example I try this (on Windows) gdalinfo /vsizip/C:\http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90 .de m.zip ERROR 4: `/vsizip/C:\http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n9 0.d em.zip' does not exist in the file system, and is not recognised as a supported dataset name. and this gdalinfo /vsizip//http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.d em. zip but the error message is similar. The idea is to fetch the data directly into GMT, but first I need to understand well how it works. Thanks Joaquim Luis ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] accessing zip files by URL
Joaquim, For .tar.gz, you can use /vsitar Mike -- Michael Smith Remote Sensing/GIS Center US Army Corps of Engineers Hanover, NH On 3/24/11 2:03 PM, Joaquim Luis jl...@ualg.pt wrote: Even, Chaitanya Thanks for hint. I did read the http://trac.osgeo.org/gdal/wiki/UserDocs/ReadInZip but it hasn't any mention to the need of using vsicurl I was about to do the tests you did, so thank you also for that. The point here is not to read that file in particular. I just wanted to use one as example and one that is not a *.tar.gz that the docs say it won't work. Ok, I have enough information to make it possible to read some compressed files sitting somewhere in the web and have the data land directly in the internals of GMT. Thanks Joaquim Joaquim, The correct syntax would be : gdalinfo /vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90 .dem.zip/W020N90.DEM or just : gdalinfo /vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90 .dem.zip because the zip file only contains one single file (W020N90.DEM) ... But the W020N90.DEM inside the zip file isn't directly recognized by GDAL (even if you download it and unzip the file). It's just a RAW file, that neads an header to tell the dimension, georeferencing, datatype etc, so the above won't directly work. You can for example create a VRT that refers to the raw file : VRTDataset rasterXSize=4800 rasterYSize=6000 GeoTransform-20, 8.3338e-03, 0, 90, 0, -8.3338e-03/GeoTransform VRTRasterBand dataType=Int16 band=1 subClass=VRTRawRasterBand NoDataValue-/NoDataValue SourceFilename relativetoVRT=0/vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/SRTM 30/w020n90/w020n90.dem.zip/SourceFilename ByteOrderMSB/ByteOrder /VRTRasterBand /VRTDataset I've deduced this VRT from the http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.hdr.zip file that sits next to the .dem.zip file. It's a shame that they didn't put the .hdr and the .dem file inside the same zip. It would have they been possible to open it directly... With the SRTM3 in HGT format, you can directly do : gdalinfo /vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/Africa/N00E006.h gt.zip Best regards, Even Hi, How do we access to a compressed files by URL? We can do that, can't we? As an example I try this (on Windows) gdalinfo /vsizip/C:\http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.de m.zip ERROR 4: `/vsizip/C:\http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.d em.zip' does not exist in the file system, and is not recognised as a supported dataset name. and this gdalinfo /vsizip//http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.dem. zip but the error message is similar. The idea is to fetch the data directly into GMT, but first I need to understand well how it works. Thanks Joaquim Luis ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] accessing zip files by URL
Great, thanks Joaquim Le jeudi 24 mars 2011 19:03:57, Joaquim Luis a écrit : Even, Chaitanya Thanks for hint. I did read the http://trac.osgeo.org/gdal/wiki/UserDocs/ReadInZip but it hasn't any mention to the need of using vsicurl Yes I am aware that it is lightly documented. I've added recently a few pointers though : - in the GDALOpen() doc : http://gdal.org/gdal_8h.html#e97be045eb4701183ad332ffce29745b which points to : http://gdal.org/cpl__vsi_8h.html#4f791960f2d86713d16e99e9c0c36258 Which mentions : This special file handler can be combined with other virtual filesystems handlers, such as /vsizip. For example, /vsizip//vsicurl/path/to/remote/file.zip/path/inside/zip Perhaps a more didactic page would be needed in the wiki. Any taker :-) ? I remind that the wiki is open in writing for everyone with an osgeo id... I was about to do the tests you did, so thank you also for that. The point here is not to read that file in particular. I just wanted to use one as example and one that is not a *.tar.gz that the docs say it won't work. Ah... well actually reading in a .tar or a .tar.gz is now supported ( http://gdal.org/cpl__vsi_8h.html#d6dd983338849e7da4eaa88f6458ab64 ). Seems that I have changed my mind since. But seeking will still be very slow of course (especially if you combine with /vsicurl !). Ok, I'm going to rectify the page about that point. Ok, I have enough information to make it possible to read some compressed files sitting somewhere in the web and have the data land directly in the internals of GMT. Thanks Joaquim Joaquim, The correct syntax would be : gdalinfo /vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w02 0n90.dem.zip/W020N90.DEM or just : gdalinfo /vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w02 0n90.dem.zip because the zip file only contains one single file (W020N90.DEM) ... But the W020N90.DEM inside the zip file isn't directly recognized by GDAL (even if you download it and unzip the file). It's just a RAW file, that neads an header to tell the dimension, georeferencing, datatype etc, so the above won't directly work. You can for example create a VRT that refers to the raw file : VRTDataset rasterXSize=4800 rasterYSize=6000 GeoTransform-20, 8.3338e-03, 0, 90, 0, -8.3338e-03/GeoTransform VRTRasterBand dataType=Int16 band=1 subClass=VRTRawRasterBand NoDataValue-/NoDataValue SourceFilename relativetoVRT=0/vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/ SRTM30/w020n90/w020n90.dem.zip/SourceFilename ByteOrderMSB/ByteOrder /VRTRasterBand /VRTDataset I've deduced this VRT from the http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.hdr.zip file that sits next to the .dem.zip file. It's a shame that they didn't put the .hdr and the .dem file inside the same zip. It would have they been possible to open it directly... With the SRTM3 in HGT format, you can directly do : gdalinfo /vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/Africa/N00E0 06.hgt.zip Best regards, Even Hi, How do we access to a compressed files by URL? We can do that, can't we? As an example I try this (on Windows) gdalinfo /vsizip/C:\http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90 .de m.zip ERROR 4: `/vsizip/C:\http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n9 0.d em.zip' does not exist in the file system, and is not recognised as a supported dataset name. and this gdalinfo /vsizip//http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.d em. zip but the error message is similar. The idea is to fetch the data directly into GMT, but first I need to understand well how it works. Thanks Joaquim Luis ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] accessing zip files by URL
Ah... well actually reading in a .tar or a .tar.gz is now supported ( http://gdal.org/cpl__vsi_8h.html#d6dd983338849e7da4eaa88f6458ab64 ). Seems that I have changed my mind since. But seeking will still be very slow of course (especially if you combine with /vsicurl !). Ok, I'm going to rectify the page about that point. Even, Still one further point on this matter. This works fine with the promised slowness (fair enough) gdalinfo /vsitar/vsicurl/http://edcftp.cr.usgs.gov/pub/data/gtopo30/global/w020n90.tar.gz/W020N90.DEM but the file name is case dependent. I mean, this doesn't work gdalinfo /vsitar/vsicurl/http://edcftp.cr.usgs.gov/pub/data/gtopo30/global/w020n90.tar.gz/w020n90.dem I'm not sure what is the right thing to do here but as a Win (and occasional Mac or Linus) user I guess that I was expecting no case dependency. Joaquim ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] accessing zip files by URL
Even, Still one further point on this matter. This works fine with the promised slowness (fair enough) gdalinfo /vsitar/vsicurl/http://edcftp.cr.usgs.gov/pub/data/gtopo30/global/w020n90.t ar.gz/W020N90.DEM but the file name is case dependent. I mean, this doesn't work gdalinfo /vsitar/vsicurl/http://edcftp.cr.usgs.gov/pub/data/gtopo30/global/w020n90.t ar.gz/w020n90.dem I'm not sure what is the right thing to do here but as a Win (and occasional Mac or Linus) user I guess that I was expecting no case dependency. GDAL's behaviour with the case sensitivity in a virtual file path is completely OS independant. Actually filenames in archives are case sensitive. I have just made a .tar.gz with 2 filenames that only differ by their case : $ tar tvzf aa.tar.gz -rw-r--r-- even/even 2 2011-03-24 23:54 Aa -rw-r--r-- even/even 3 2011-03-24 23:54 aa And the same with a zip file : $ unzip -l aa.zip Archive: aa.zip Length DateTimeName - -- - 2 2011-03-24 23:54 Aa 3 2011-03-24 23:54 aa - --- 5 2 files Admitedly a windows user will not be very happy when he uncompresses those archives in a real filesystem ;-) I will also note that URLs are also case sensitive. http://edcftp.cr.usgs.gov/pub/data/gtopo30/global/w020n90.tar.gz is a valid URL, but http://edcftp.cr.usgs.gov/pub/data/gtopo30/global/w020n90.tar.gZ is not So I'm not sure that there's really a point in making the lookup of filenames inside the archive case insensitive. Joaquim ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] accessing zip files by URL
GDAL can not fix the fact that: http://edcftp.cr.usgs.gov/pub/data/gtopo30/global/w020n90.tar.gz/w020n90.dem is a 404. -- Chris Sure, the browsers are not yet GDAL compliant and cannot see directly inside compressed files. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] accessing zip files by URL
I understand all that and even though I'm a win user I always try to be very careful with case names (basically, I avoid upper cases). But while it is easy to see the case of the real file (I know that urls are case sensitive) it's not so easy so clear to know what's inside a compressed file on the internet because if you know it it means one already have it in the local file system. Since is up to gdal to read the contents of the compressed file and get its data (NOT the file itself) it seams to me that there would be no danger in being case insensitive. But this is really a minor point (once you know why it didn't work on the first time), I know. Joaquim Even, Still one further point on this matter. This works fine with the promised slowness (fair enough) gdalinfo /vsitar/vsicurl/http://edcftp.cr.usgs.gov/pub/data/gtopo30/global/w020n90.t ar.gz/W020N90.DEM but the file name is case dependent. I mean, this doesn't work gdalinfo /vsitar/vsicurl/http://edcftp.cr.usgs.gov/pub/data/gtopo30/global/w020n90.t ar.gz/w020n90.dem I'm not sure what is the right thing to do here but as a Win (and occasional Mac or Linus) user I guess that I was expecting no case dependency. GDAL's behaviour with the case sensitivity in a virtual file path is completely OS independant. Actually filenames in archives are case sensitive. I have just made a .tar.gz with 2 filenames that only differ by their case : $ tar tvzf aa.tar.gz -rw-r--r-- even/even 2 2011-03-24 23:54 Aa -rw-r--r-- even/even 3 2011-03-24 23:54 aa And the same with a zip file : $ unzip -l aa.zip Archive: aa.zip Length DateTimeName - -- - 2 2011-03-24 23:54 Aa 3 2011-03-24 23:54 aa - --- 5 2 files Admitedly a windows user will not be very happy when he uncompresses those archives in a real filesystem ;-) I will also note that URLs are also case sensitive. http://edcftp.cr.usgs.gov/pub/data/gtopo30/global/w020n90.tar.gz is a valid URL, but http://edcftp.cr.usgs.gov/pub/data/gtopo30/global/w020n90.tar.gZ is not So I'm not sure that there's really a point in making the lookup of filenames inside the archive case insensitive. Joaquim ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] accessing zip files by URL
Le vendredi 25 mars 2011 00:37:38, Joaquim Luis a écrit :
I understand all that and even though I'm a win user I always try to be
very careful with case names (basically, I avoid upper cases). But while
it is easy to see the case of the real file (I know that urls are case
sensitive) it's not so easy so clear to know what's inside a compressed
file on the internet because if you know it it means one already have it
in the local file system.
Actually you don't need to download the file to know its content.
Just a bit of C/Python/Java/C#/Perl magic.
For example, with python :
from osgeo import gdal
print
gdal.ReadDir('/vsizip/vsicurl/http://www.aprsworld.net/gisdata/world/world.zip')
Answer : ['world.dbf', 'world.shp', 'world.shx']
(For a .tar.gz, unfortunately getting the list of files means in practice that
GDAL will have to download the whole .tar.gz)
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] accessing zip files by URL
On Mar 24, 2011, at 6:47 PM, ext Joaquim Luis wrote: Ah... well actually reading in a .tar or a .tar.gz is now supported ( http://gdal.org/cpl__vsi_8h.html#d6dd983338849e7da4eaa88f6458ab64 ). Seems that I have changed my mind since. But seeking will still be very slow of course (especially if you combine with /vsicurl !). Ok, I'm going to rectify the page about that point. Even, Still one further point on this matter. This works fine with the promised slowness (fair enough) gdalinfo /vsitar/vsicurl/http://edcftp.cr.usgs.gov/pub/data/gtopo30/global/w020n90.tar.gz/W020N90.DEM but the file name is case dependent. I mean, this doesn't work gdalinfo /vsitar/vsicurl/http://edcftp.cr.usgs.gov/pub/data/gtopo30/global/w020n90.tar.gz/w020n90.dem I'm not sure what is the right thing to do here but as a Win (and occasional Mac or Linus) user I guess that I was expecting no case dependency. Tough luck? The world is case dependent. Filenames on Linux (and sometimes on Mac) are case dependent, and HTTP servers are usually case dependent. GDAL can not fix the fact that: http://edcftp.cr.usgs.gov/pub/data/gtopo30/global/w020n90.tar.gz/w020n90.dem is a 404. -- Chris Joaquim ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
