[
https://issues.apache.org/jira/browse/SDAP-481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stepheny Perez updated SDAP-481:
--------------------------------
Description:
>From Vardis:
{quote}Here is a prior SPURS Saildrone example of the application of netCDF
internal compression ..
It uses the nccopy command with -d 5 parameter (deflate=compress at level 5)
applied to an existing netcdf file.
_$ nccopy -4 -d 5
saildrone-gen_6-arctic_misst_2021-sd1058-20210706T000000-20211020T235959-5_minutes-v1.1638218824631.nc
new.nc_
This reduced the file size from 43mb originally to 6.3mb.
I am sure there is a way of actually writing the compressed netCDF natively
from the outset with -d.
[https://www.unidata.ucar.edu/blogs/developer/en/entry/netcdf_compression]
(some info)
[http://climate-cms.wikis.unsw.edu.au/NetCDF_Compression_Tools] (python
package and general info on nc compression).
[https://docs.xarray.dev/en/latest/generated/xarray.Dataset.to_netcdf.html]
(xarray write netCDF - compression syntax/parameters mentioned)
{quote}
> Enable internal compression for netcdf match up outputs
> -------------------------------------------------------
>
> Key: SDAP-481
> URL: https://issues.apache.org/jira/browse/SDAP-481
> Project: Apache Science Data Analytics Platform
> Issue Type: New Feature
> Components: analysis
> Reporter: Stepheny Perez
> Assignee: Stepheny Perez
> Priority: Minor
>
> From Vardis:
> {quote}Here is a prior SPURS Saildrone example of the application of netCDF
> internal compression ..
> It uses the nccopy command with -d 5 parameter (deflate=compress at level 5)
> applied to an existing netcdf file.
> _$ nccopy -4 -d 5
> saildrone-gen_6-arctic_misst_2021-sd1058-20210706T000000-20211020T235959-5_minutes-v1.1638218824631.nc
> new.nc_
> This reduced the file size from 43mb originally to 6.3mb.
> I am sure there is a way of actually writing the compressed netCDF natively
> from the outset with -d.
> [https://www.unidata.ucar.edu/blogs/developer/en/entry/netcdf_compression]
> (some info)
> [http://climate-cms.wikis.unsw.edu.au/NetCDF_Compression_Tools] (python
> package and general info on nc compression).
> [https://docs.xarray.dev/en/latest/generated/xarray.Dataset.to_netcdf.html]
> (xarray write netCDF - compression syntax/parameters mentioned)
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
