[ https://issues.apache.org/jira/browse/SDAP-281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas G Loubrieu updated SDAP-281: ----------------------------------- Resolution: Fixed Status: Done (was: In Progress) > bigdata ingestion feedback (reading time) > ----------------------------------------- > > Key: SDAP-281 > URL: https://issues.apache.org/jira/browse/SDAP-281 > Project: Apache Science Data Analytics Platform > Issue Type: Bug > Reporter: Thomas G Loubrieu > Assignee: Eamon Ford > Priority: Major > > While doing the ingestion on bigdata wth configuration file > https://github.com/tloubrieu-jpl/sdap-ingester-config/blob/bigdata/collections.yml > None of the dataset have been ingested so far except mur25 (for 3-4 days). > No tiles are availble in solr for the other datasets. > Some granules have been ingested thought as shown in the solr nexusgranules > collection: > "avhrr-l4-glob-v2-daily-ncei-ghrsst-sstblend-avhrr-oi-glob-v020-fv020",13752, > "smap-l2b-sss-daily-smap-sss",7032, > "mur25-jpl-l4-glob-v41-analysed-sst",6062, > "modis-l3-aqua-11um-v2014-0-4km-daily-sst",6036, > "measures-sla-jpl-1603-regrid1x1-ssh",1805, > > "avhrr-l4-glob-v2-clim-seasonal-ncei-ghrsst-sstblend-avhrr-oi-glob-v020-fv020",365] > Here are some information from the logs and netcdf files regarding the > possible issues: > Granule ingester log: > Traceback (most recent call last): > File > "/opt/conda/lib/python3.7/site-packages/sdap_granule_ingester-1.0.0_SNAPSHOT-py3.7.egg/granule_ingester/consumer/Consumer.py", > line 79, in _received_message > await pipeline.run() > File > "/opt/conda/lib/python3.7/site-packages/sdap_granule_ingester-1.0.0_SNAPSHOT-py3.7.egg/granule_ingester/pipeline/Pipeline.py", > line 187, in run > self._slicer.generate_tiles(dataset, granule_name)] > File > "/opt/conda/lib/python3.7/site-packages/sdap_granule_ingester-1.0.0_SNAPSHOT-py3.7.egg/granule_ingester/slicers/TileSlicer.py", > line 50, in generate_tiles > self._tile_spec_list = self._generate_slices(dimensions) > File > "/opt/conda/lib/python3.7/site-packages/sdap_granule_ingester-1.0.0_SNAPSHOT-py3.7.egg/granule_ingester/slicers/SliceFileByStepSize.py", > line 36, in _generate_slices > raise KeyError('Provided dimension "{}" not found in > dataset'.format(dim_name)) > KeyError: 'Provided dimension "time" not found in dataset' > Dataset measures-sla-jpl-1603-regrid1x1-ssh has dimension Time with Uppercase > T: > float Time(Time) ; > Time:units = "Days since 1985-01-01 00:00:00" ; > Time:long_name = "Time" ; > Time:standard_name = "time" ; > Time:calendar = "gregorian" ; > Time:axis = "T" ; > dataset smap-l2b-sss-daily-smap-sss does not have a time dimension at all. > dataset modis-l3-aqua-11um-v2014-0-4km-daily-sst no time dimension > dataset avhrr-l4-glob-v2-daily-ncei-ghrsst-sstblend-avhrr-oi-glob-v020-fv020 > has a time dimension. > dataset > avhrr-l4-glob-v2-clim-seasonal-ncei-ghrsst-sstblend-avhrr-oi-glob-v020-fv020 > does not have a time dimension. -- This message was sent by Atlassian Jira (v8.3.4#803005)