Joseph C. Jacob created SDAP-343:
------------------------------------
Summary: Fix reading of time stamp in GPM data
Key: SDAP-343
URL: https://issues.apache.org/jira/browse/SDAP-343
Project: Apache Science Data Analytics Platform
Issue Type: Task
Components: granule-ingester
Reporter: Joseph C. Jacob
The GPM IMERG Early Precipitation L3 1 day 0.1 degree x 0.1 degree V06
(GPM_3IMERGDE) dataset
([https://disc.gsfc.nasa.gov/datasets/GPM_3IMERGDE_06/summary?keywords=GPM)]
gives an error upon ingest:
{quote}{{granule_ingester.exceptions.Exceptions.TileProcessingError: Could not
generate tiles from the granule because of the following error: unsupported
operand type(s) for /: 'cftime._cftime.DatetimeJulian' and 'float'.}}{quote}
{{This appears to be related to how the time variable is read. The time
variable is given as: }}
double time(time) ;
time:units = "days since 1970-01-01 00:00:00Z" ;
time:standard_name = "time" ;
time:calendar = "julian" ;
time:bounds = "time_bnds" ;
time:origname = "time" ;
time:fullnamepath = "/time" ;
double time_bnds(time, nv) ;
time_bnds:units = "days since 1970-01-01 00:00:00Z" ;
time_bnds:coordinates = "time nv" ;
time_bnds:origname = "time_bnds" ;
time_bnds:fullnamepath = "/time_bnds" ;
{{The new SDAP ingester uses xarray to read the NetCDF files. Xarray tries to
force conversion to a datetime64 object if possible, but seems to have been
unable to do so in this case (maybe related to the Julian calendar being
used?). }}
{{The old legacy ningester was able to read GPM in the past (during the
OceanWorks project). A notable difference is that the old ningester used the
NetCDF4 module instead of xarray.}}
In this ticket we need to determine if xarray can be used correctly for this
dataset, and if not, we need to revert back to using NetCDF4.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
