Re: [CF-metadata] JSON representation based on CF

Jon Blower Thu, 27 Jul 2017 00:47:58 -0700

P.S. I’ve created an issue on the CovJSON GitHub site for discussion on the distinction between CovJSON, CF-JSON and NCO-JSON. I thought this might be a more appropriate forum for the discussion than this mailing list.

https://github.com/covjson/specification/issues/86

If you have a GitHub login, please feel free to post comments and replies to this issue. I guess the “exam question” we are trying to answer is – to what extent should we be looking to merge these initiatives?

Cheers,
Jon

From: CF-metadata <cf-metadata-boun...@cgd.ucar.edu> on behalf of Jon Blower <j.d.blo...@reading.ac.uk>
Date: Thursday, 27 July 2017 07:59
To: David Johnson <d.john...@metocean.co.nz>, Christopher Barker <chris.bar...@noaa.gov>
Cc: "cf-metadata@cgd.ucar.edu" <cf-metadata@cgd.ucar.edu>
Subject: Re: [CF-metadata] JSON representation based on CF

Hi David,

Just a quick one as I’m conscious that this list isn’t really for JSON discussions – the reasons that CovJSON flattens the arrays are explained here:

https://www.w3.org/TR/covjson-overview/#datavalues

(Short answer: JSON doesn’t really support multidimensional arrays anyway. They are really “arrays of arrays”, which are harder to reason over and manipulate. Also they are less efficient than flat arrays for a few reasons. The shortcomings of flat arrays are easily overcome at the API level.)

HTH,

Jon

From: David Johnson <d.john...@metocean.co.nz>
Date: Thursday, 27 July 2017 05:33
To: Christopher Barker <chris.bar...@noaa.gov>
Cc: Jon Blower <sgs02...@reading.ac.uk>, Sean Arms <sa...@ucar.edu>, "cf-metadata@cgd.ucar.edu" <cf-metadata@cgd.ucar.edu>
Subject: Re: [CF-metadata] JSON representation based on CF

Hi All,

The CF-JSON structure was generally based on CDL (as produced by ncdump) which stores the data in a separate section. The key requirement is the same as for CDL.

But I do see the logic of just including it in the variable section itself.

I haven't had a chance to properly review NCO JSON, but it sounds like we may be very close. This Is there a published specification anywhere? - so far I have just looked at various actual file dumps.

One thing I prefer about CF-JSON is the multidimensional arrays. Both NCO and CovJSON have flattened arrays (as does CDL). I think web devs would probably prefer the former and there is less chance of an array wrangling mistake. I also like the option to dump time as ISO8601 because again I think web devs appreciate the ease of this rather than having to support time conversions and calendar issues.

We are at an early stage so could easily merge.

Dave

On Thu, Jul 27, 2017 at 12:30 PM, Chris Barker <chris.bar...@noaa.gov> wrote:

Just looked a tiny bit more at CF_JSON, and see an issue right away:
"The data object contains the actual data for each variable as its key:value members. Each data key MUST be the same as it variable ID key."
{
    ...
    "variables": {
        "tmp2m": {
            "dimensions": ["time","latitude","longitude"]
        }
    ...
    "data": {
        "tmp2m": [
            [[1.2,3.4,5.6 ...],
             [2.3,6.5,8.7 ...],
             ...
            ],
        ]
    ...
}
JSON can be arbitrarily nested -- so why store the "data" object separately from the variable objects, using a key to map them?

why not:

{
...
"variables": { "tmp2m": {"dimensions": ["time","latitude","longitude"]
"attributes": {"units": "ms^{-1}",
"long_name": "Easterly component of wind",
"standard_name": "eastward_wind"
},

"data": [[1.2,3.4,5.6 ...],
[2.3,6.5,8.7 ...],
...
],
},
...

}
}

I think that's more or less how NCO's JSON works, and it keeps everything about a variable all in one place -- easier for parsing tools, etc. And maps well to the Python netCDF4 data structure anyway.

-CHB

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA 98115   (206) 526-6317   main reception

chris.bar...@noaa.gov

Dr David Johnson
Technical Director
t: +64 7 825 0540 ext.102
m: +64 21 057 1058
www.metocean.co.nz

_______________________________________________
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] JSON representation based on CF

Reply via email to