Oh wow, that's fantastic - thanks so much. Lots for me to learn from there too, with the HDF5EOS GRID details.
Cheers, Mike On Tue, Apr 30, 2024 at 6:08 AM Even Rouault <even.roua...@spatialys.com> wrote: > Michael, > > actually support for those products was already there at 95% since they > use the HDF5EOS GRID formalism which we support since 3.7. But that > particular type of products has an oddity. They have an empty > GROUP=Dimension within the GROUP=GridStructure in the HDF5EOS metadata, > which our parser didn't like as we use the information in this group to map > dimension names to dimension index and size. Fixed/worked around in > https://github.com/OSGeo/gdal/pull/9807 . Hard to tell if those products > are in fault given that the HDF5EOS specification at > https://www.earthdata.nasa.gov/s3fs-public/imported/ESDS-RFC-008-v1.1.pdf > , appendix A, is extremely minimum regarding the HDF5EOS metadata itself > (for once, I complain for a spec to be too minimal !), and the example they > give for grids doesn't make sense to me (the declared dimension names and > sizes have nothing to do with the ones use in the grid) > > I now get: > > $ gdalinfo > HDF5:"AMSR_U2_L3_SeaIce12km_B04_20120702.he5"://HDFEOS/GRIDS/SpPolarGrid12km/Data_Fields/SI_12km_SH_18H_ASC > Driver: HDF5Image/HDF5 Dataset > Files: AMSR_U2_L3_SeaIce12km_B04_20120702.he5 > Size is 632, 664 > Coordinate System is: > PROJCRS["unnamed", > BASEGEOGCRS["Unknown datum based upon the custom spheroid", > DATUM["Not specified (based on custom spheroid)", > ELLIPSOID["Custom spheroid",6378273,0, > LENGTHUNIT["metre",1, > ID["EPSG",9001]]]], > PRIMEM["Greenwich",0, > ANGLEUNIT["degree",0.0174532925199433, > ID["EPSG",9122]]]], > CONVERSION["Polar Stereographic (variant B)", > METHOD["Polar Stereographic (variant B)", > ID["EPSG",9829]], > PARAMETER["Latitude of standard parallel",-70, > ANGLEUNIT["degree",0.0174532925199433], > ID["EPSG",8832]], > PARAMETER["Longitude of origin",0, > ANGLEUNIT["degree",0.0174532925199433], > ID["EPSG",8833]], > PARAMETER["False easting",0, > LENGTHUNIT["metre",1], > ID["EPSG",8806]], > PARAMETER["False northing",0, > LENGTHUNIT["metre",1], > ID["EPSG",8807]]], > CS[Cartesian,2], > AXIS["(E)",north, > MERIDIAN[90, > ANGLEUNIT["degree",0.0174532925199433, > ID["EPSG",9122]]], > ORDER[1], > LENGTHUNIT["Meter",1]], > AXIS["(N)",north, > MERIDIAN[0, > ANGLEUNIT["degree",0.0174532925199433, > ID["EPSG",9122]]], > ORDER[2], > LENGTHUNIT["Meter",1]]] > Data axis to CRS axis mapping: 1,2 > Origin = (-3950000.000000000000000,4350000.000000000000000) > Pixel Size = (12500.000000000000000,-12500.000000000000000) > Metadata: > Conventions=CF-1.6 > HDFEOS_INFORMATION_HDFEOSVersion=HDFEOS_5.1.15 > history=This version of the Sea Ice processing code contains updates > provided by the science team on September 16, 2019. For details on these > updates, see the release notes provided in the DAP. > institution=NASA's AMSR Science Investigator-led Processing System (SIPS) > references=Please cite these data as: Markus, T., J. C. Comiso, and W. > N. Meier. 2018. AMSR-E/AMSR2 Unified L3 Daily 12.5 km Brightness > Temperatures, Sea Ice Concentration, Motion & Snow Depth Polar Grids, > Version 1. [Indicate subset used]. Boulder, Colorado USA. NASA National > Snow and Ice Data Center Distributed Active Archive Center. doi: > https://doi.org/10.5067/RA1MIJOYPK3P. > source=satellite observation > title=AMSR-E/AMSR2 Unified L3 Daily 12.5 km Brightness Temperatures, Sea > Ice Concentration, Motion & Snow Depth Polar Grids > Corner Coordinates: > Upper Left (-3950000.000, 4350000.000) ( 42d14'27.21"W, 39d11'27.54"S) > Lower Left (-3950000.000,-3950000.000) (135d 0' 0.00"W, 41d23'59.41"S) > Upper Right ( 3950000.000, 4350000.000) ( 42d14'27.21"E, 39d11'27.54"S) > Lower Right ( 3950000.000,-3950000.000) (135d 0' 0.00"E, 41d23'59.41"S) > Center ( 0.000, 200000.000) ( 0d 0' 0.01"E, 88d 8'51.76"S) > Band 1 Block=632x1 Type=Int32, ColorInterp=Undefined > NoData Value=0 > Metadata: > add_offset=0 > coordinates=lon lat > long_name=18.7 GHz horizontal daily average ascending Tbs > packing_convention=netCDF > packing_convention_description=unpacked = scale_factor x packed + > add_offset > scale_factor=0.1 > standard_name=brightness_temperature > units=degree_kelvin > _FillValue=0 > > Even > Le 29/04/2024 à 21:10, Michael Sumner via gdal-dev a écrit : > > This HDF5 (requires earthdata credentials your "Authorization: Bearer > <token>" in GDAL_HTTP_HEADERS, or equiv) presents without geolocation > arrays. > > gdalinfo "/vsicurl/ > https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5" > -sd 26 > Driver: HDF5Image/HDF5 Dataset > Files: /vsicurl/ > https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5 > Size is 608, 896 > Metadata: > Conventions=CF-1.6 > HDFEOS_INFORMATION_HDFEOSVersion=HDFEOS_5.1.15 > history=This version of the Sea Ice processing code contains updates > provided by the science team on September 16, 2019. For details on these > updates, see the release notes provided in the DAP. > institution=NASA's AMSR Science Investigator-led Processing System (SIPS) > references=Please cite these data as: Markus, T., J. C. Comiso, and W. > N. Meier. 2018. AMSR-E/AMSR2 Unified L3 Daily 12.5 km Brightness > Temperatures, Sea Ice Concentration, Motion & Snow Depth Polar Grids, > Version 1. [Indicate subset used]. Boulder, Colorado USA. NASA National > Snow and Ice Data Center Distributed Active Archive Center. doi: > https://doi.org/10.5067/RA1MIJOYPK3P. > source=satellite observation > title=AMSR-E/AMSR2 Unified L3 Daily 12.5 km Brightness Temperatures, Sea > Ice Concentration, Motion & Snow Depth Polar Grids > Corner Coordinates: > Upper Left ( 0.0, 0.0) > Lower Left ( 0.0, 896.0) > Upper Right ( 608.0, 0.0) > Lower Right ( 608.0, 896.0) > Center ( 304.0, 448.0) > Band 1 Block=608x1 Type=Int32, ColorInterp=Undefined > Metadata: > comment=data value meaning: 0 -- Open Water, 110 -- missing/not > calculated, 120 -- Land > coordinates=lon lat > long_name=Sea ice concentration daily average > units=percent > > > > gdalinfo --version > GDAL 3.9.0dev-cb4d30f56d, released 2024/04/15 > > The geolocation arrays are sds 33 and 32 respectively: > > HDF5:"/vsicurl/ > https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5 > "://HDFEOS/GRIDS/NpPolarGrid12km/lon > > HDF5:"/vsicurl/ > https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5 > "://HDFEOS/GRIDS/NpPolarGrid12km/lat > > And things work when lining those up in VRT with warp. Can the HDF5 driver > be made to auto-detect these geolocation arrays? > > I see that the NETCDF driver actually does: > > gdalinfo "NetCDF:/vsicurl/ > https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5" > -sd 26 > > I'm asking as an email rather than pursuing the fix because, these data > are actually a regular grid on the north and south poles, and so > geolocation by arrays is sub-optimal the specification is listed in > > https://nsidc.org/sites/default/files/au_si12-v001-userguide_1.pdf > > and the two parameter sets are > > Np-north: -te -3850000, -5350000, 3750000, 5850000 -t_srs EPSG:3411 > Sp-south: -te -3950000, -3950000, 3950000, 4350000 -t_srs EPSG:3412 > > Is this generally something we should pursue within GDAL? It seems like an > endless task to detect-on-open exactly this situation and assign the easy > fix, but this is a pretty fundamental data stream and it's very common so > the longlat/arrays might be numerically detectable with other > heuristics hinting that it's polar (??) and there are plenty of other > sources that present equivalents in the right way e.g. this one: > > "/vsicurl/ > https://noaadata.apps.nsidc.org/NOAA/G02135/north/daily/geotiff/2012/07_Jul/N_20120702_concentration_v3.0.tif > " > > The right approach is probably to inform the providers and get the right > metadata baked in ... but there's pros and cons to either. I'm not sure > there's even enough information in these files to clearly detect the > situation, it would be a bit like the NSIDCbin driver with its very strict > requirements. > > Cheers, Mike > > > -- > > Michael Sumner > Software and Database Engineer > Australian Antarctic Division > Hobart, Australia > e-mail: mdsum...@gmail.com > > _______________________________________________ > gdal-dev mailing > listgdal-dev@lists.osgeo.orghttps://lists.osgeo.org/mailman/listinfo/gdal-dev > > -- http://www.spatialys.com > My software is free, but my time generally not. > > -- Michael Sumner Software and Database Engineer Australian Antarctic Division Hobart, Australia e-mail: mdsum...@gmail.com
_______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev