Re: [CF-metadata] Provide input on draft of STAC "Datacube" Extension

[Roy Mendelssohn - NOAA Federal]

Before taking this  off-line  (now that I am back at work and can use my work 
email),  can I just add that ERDDAP also has many of these features.  In my 
opinion,  a lot of the new efforts are reinventing the wheel.  Perhaps taking 
time to see what is out there,  what they do,  and why they are doing things in 
certain ways,  would be a worthwhile effort.

-Roy


> On Jan 27, 2019, at 2:02 AM, Ryan Abernathey  
> wrote:
> 
> Hi Sean et al,
> 
> Glad to hear that you folks are interested in continuing the discussion. I 
> personally think there is a lot of value in breaking these catalog / metadata 
> standards apart from the language-specific implementations. As you said, this 
> may already be done, but the online docs don't always make this clear.
> 
> I propose we continue the discussion in
> https://github.com/radiantearth/stac-spec/issues/366#issuecomment-457875806
> in order to avoid further traffic on the cf-metadata mailing list.
> 
> In that thread, Chris Holmes said the following:
> 
> > Hyrax looks pretty awesome and the creators of it definitely sound like a 
> > group we should be talking to. Any chance of introducing us? That's awesome 
> > they have JSON-LD markup, as it's one of our main next goals, see #378 I 
> > think we'd be interested in learning from them, to help make STAC better.
> 
> So they are really looking for input. If you or anyone else wants to chime 
> in, you would be very welcome. We are currently discussing scheduling a call 
> for early February to address THREDDS / OpenDAP / STAC integration.
> 
> Cheers,
> Ryan
> 
> 
> On Tue, Jan 22, 2019 at 4:57 PM Sean Arms  wrote:
> Greetings Ryan!
> 
> I'll be the first to admit that we do not do a good job delineating the 
> different pieces covered by the THREDDS umbrella (e.g. Siphon, the catalog 
> spec, netCDF-java, the THREDDS Data Server (TDS), etc.), many of which are in 
> one monolithic repository (will be tackling this soon), and I totally 
> understand thinking "java" when I hear THREDDS. The TDS does have a RESTful 
> catalog API, which allows for linking catalogs, performing catalog 
> subsetting, and generating client catalogs and their associated html views 
> for ease of browsing. If useful, I could see implementing the STAC catalog 
> API as well to help bridge communities. The TDS is currently capable of 
> generating THREDDS client catalogs for collections stored in more cloud 
> friendly environments as well, such as S3, and we are looking at zarr support 
> at the netCDF-java level. That said, I'd like to hold off a bit to see if 
> STAC begins to align better with schema.org (I see there is an issue open 
> about this, and it looks promising); I
 've recently been looking at extending the THREDDS metadata elements to better 
align with the schema.org Dataset and DataCatalog objects so that we can 
produce a richer mapping.
> 
> I'm happy to take this discussion to another venue, as I don't want to take 
> away from the topic at hand. Perhaps we could meet up at the EarthCube All 
> Hands meeting if you will be in town.
> 
> Cheers!
> 
> Sean
> 
> 
> On Mon, Jan 21, 2019 at 4:16 AM Ryan Abernathey  
> wrote:
> Ryan,
> 
> That's a very good question. I'm not sure there IS a definitive advantage of 
> STAC over THREDDS. My main goal here is just to have some discussion, since 
> STAC is growing quickly in the adjacent world of geospatial imagery. The 
> conclusion may very well be, we don't need STAC.
> 
> Within Pangeo, our interest in STAC is in the context of putting netCDF (or 
> more accurately, netCDF-like Zarr) data into cloud storage as static assets. 
> We are looking for a way to catalog those assets, also in a static file, and 
> STAC is one thing we are considering. TBH, it never even occurred to me that 
> we could generate a static THREDDS catalog.xml file and use this to describe 
> our assets. In my mind, THREDDS catalogs are intimately bound to opendap 
> servers, and the THREDDS server in particular. But after reading the THREDDS 
> spec document you sent, I see that I am wrong about that.
> 
> Comparing the STAC vs. THREDDS specs, the main difference to me is the 
> relative high complexity of the THREDDS spec. I could pretty easy figure out 
> how to write a STAC catalog for my data in a text editor. I can't say the 
> same for THREDDS. But that is not a dealbreaker.
> 
> Beyond that, the STAC principles are pretty appealing:
> 
> - Creation and evolution of specs in Github, using Open Source principles
> - JSON + REST + HTTP at the core 
> - Small Reusable Pieces Loosely Coupled
> - Specify in OpenAPI 3.0 (formerly known as Swagger) specification
> - Focus on the developer. Specifications should aim for implementability 
> - Working code required. Proposed changes should be accompanied by working 
> code (ideally with a link to an online service running the code)
> - Design for scale
> 
> The first one--evolution of specs in Github--is particularly important is we 
> explore new 

Re: [CF-metadata] Sea water temperature anomaly standard name

[Roy Mendelssohn - NOAA Federal]

Hi All:


I will move this over to Github as suggested, but one last comment.  I totally 
disagree with this statement, which was similar to the one I received a couple 
of years ago.  The way names should be defined should be based on what makes 
the most sense and is most consistent with present practice  (such as how are 
climatologies defined  - to me anomalies are the flip side of climatologies), 
not present day convenience.  At the present time it may be easier,  but in the 
long-run it is asking for problems.

What I have suggested may in fact be a bad idea,  but it is not a bad idea 
because at the present time there are only 5-6 names with anomaly in them.

-Roy M.

> On Dec 5, 2018, at 9:28 AM, Jonathan Gregory  
> wrote:
> 
> Dear all
> 
> I too think it's fine to have a standard name for sea water temperature
> anomaly. While I understand the concern about potentially huge numbers of
> anomaly standard names, I don't think we need to deal with it by any other
> means at present because very few have been proposed. There are just five at
> present in the table.
> 
> Best wishes
> 
> Jonathan
> 
> - Forwarded message from John Graybeal  -
> 
>> 1) I support Simon/Roy’s request, it seems straightforward.
>> 
>> 2) Roy M, it would be good to get that discussion topic in a ticket, so we 
>> can continue it there. But, I should warn you, I also went through that 
>> argument about 12 years ago, and I can vouch that the CF metadata community 
>> has been consistent over that time. I can’t remember the precise motivation, 
>> but basically it is almost never the preferred model. (But at least one 
>> time, when the observed thing was a taxonomic entity, I believe it was 
>> agreed to adopt a plug-in approach.)  Perhaps now that SWEET is becoming a 
>> more actively managed resource, and once CSDMS is more in the public realm, 
>> these may serve as  patterns (and vocabulary sources) for using plug-ins in 
>> other semantic models. 
>> 
>> john g.
>> 
>> 
>>> On Dec 5, 2018, at 07:22, Roy Mendelssohn - NOAA Federal 
>>>  wrote:
>>> 
>>> 
>>> 
>>>> On Dec 5, 2018, at 7:18 AM, Lowry, Roy K.  wrote:
>>>> 
>>>> I also feel that this could take some time and do not feel it would be 
>>>> fair to block Simon's request until it is resolved. There are a number of 
>>>> anomaly Standard Names and one more isn't going to make a great deal of 
>>>> difference.
>>> 
>>> never meant to block Simon's request,  apologize if it was taken that way,  
>>> I will look into the Github process.  It's  just that anomalies are common, 
>>>  and we can have an endless series of new names,  or else base standard 
>>> names that then have anomalies added to them,  which tells me what to do in 
>>> every case.
>>> 
>>> -Roy M.
> ___
> CF-metadata mailing list
> CF-metadata@cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

**
"The contents of this message do not reflect any position of the U.S. 
Government or NOAA."
**
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new street address***
110 McAllister Way
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: roy.mendelss...@noaa.gov www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected" 
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Sea water temperature anomaly standard name

[Roy Mendelssohn - NOAA Federal]

> On Dec 5, 2018, at 7:18 AM, Lowry, Roy K.  wrote:
> 
> I also feel that this could take some time and do not feel it would be fair 
> to block Simon's request until it is resolved. There are a number of anomaly 
> Standard Names and one more isn't going to make a great deal of difference.

never meant to block Simon's request,  apologize if it was taken that way,  I 
will look into the Github process.  It's  just that anomalies are common,  and 
we can have an endless series of new names,  or else base standard names that 
then have anomalies added to them,  which tells me what to do in every case.

-Roy M.

**
"The contents of this message do not reflect any position of the U.S. 
Government or NOAA."
**
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new street address***
110 McAllister Way
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: roy.mendelss...@noaa.gov www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected" 
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Sea water temperature anomaly standard name

[Roy Mendelssohn - NOAA Federal]

Interesting.  Look at the archive around April 4, 2016.  I was requesting a 
standard way to do anomalies, not just variable specific,  since anomalies are 
calculated all over the place, and it seems to me it makes little sense to have 
a new name for each specific anomaly,  rather than a generic method based on an 
existing name.  Let us just day it was not greeted with open arms.  Perhaps 
this is worth revisiting.

-Roy M.

> On Dec 5, 2018, at 6:20 AM, Lowry, Roy K.  wrote:
> 
> Hello Simon,
> 
> If you look at the definition for 
> ocean_mixed_layer_thickness_defined_by_temperature you will get a better 
> understanding about what sea_water_temperature_difference is about. It is an 
> auxiliary co-ordinate variable used to specify the temperature change that 
> defines the base of a mixed layer. Consequently, it has nothing to do with 
> your requirements.
> 
> So, you need a new Standard Name. I would suggest:
> 
> sea_water_temperature_anomaly
> Canonical Unit degrees Kelvin (note actual units in data files can be deg C)
> 'anomaly' means difference from climatology. Sea water temperature is the in 
> situ temperature within a water body.
> 
> Cheers, Roy.
> 
> I have now retired but will continue to be active through an Emeritus 
> Fellowship using this e-mail address.
> 
> 
> From: CF-metadata  on behalf of Good, Simon 
> 
> Sent: 05 December 2018 13:12
> To: cf-metadata@cgd.ucar.edu
> Subject: [CF-metadata] Sea water temperature anomaly standard name
>  
> Hello,
>  
> I am generating some data that is the difference between a variable with 
> standard name sea_water_temperature and a climatology and I am not sure what 
> to use for the standard_name. There is an existing standard name 
> sea_water_temperature_difference but I am not sure if this is supposed to be 
> used to represent anomalies as the description of it is very brief. There are 
> a few other variables that have standard names defined for anomalies e.g. 
> air_temperature_anomaly, but there is currently no 
> sea_water_temperature_anomaly. I also checked the standard name modifiers as 
> I thought that anomaly might be included in those, but it is not currently. 
> Please could you advise on what I should use and if necessary request that it 
> be added to the list of standard names?
>  
> Many thanks,
>  
> Simon Good
>  
>  
> 
> 
> This email and any attachments are intended solely for the use of the named 
> recipients. If you are not the intended recipient you must not use, disclose, 
> copy or distribute this email or any of its attachments and should notify the 
> sender immediately and delete this email from your system. 
> UK Research and Innovation has taken every reasonable precaution to minimise 
> risk of this email or any attachments containing viruses or malware but the 
> recipient should carry out its own virus and malware checks before opening 
> the attachments. UK Research and Innovation does not accept any liability for 
> any losses or damages which the recipient may sustain due to presence of any 
> viruses. 
> Opinions, conclusions or other information in this message and attachments 
> that are not related directly to UK Research and Innovation business are 
> solely those of the author and do not represent the views of UK Research and 
> Innovation.
> 
> ___
> CF-metadata mailing list
> CF-metadata@cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

**
"The contents of this message do not reflect any position of the U.S. 
Government or NOAA."
**
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new street address***
110 McAllister Way
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: roy.mendelss...@noaa.gov www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected" 
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Recording "day of year on which something happens"

[Roy Mendelssohn - NOAA Federal]

Julian day is used in several different ways,  as John says:

https://landweb.modaps.eosdis.nasa.gov/browse/calendar.html

-Roy

> On Mar 16, 2017, at 12:49 PM, John Helly  wrote:
> 
> In language, definitions are based on usage. Julian date, modulo the year, is 
> a convention that I have been using for decades to do what you are talking 
> about but I defer to wiser minds.
> 
> J.
> 
> On 3/16/17 9:42 AM, Jim Biard wrote:
>> John,
>> 
>> As best as I understand it, Julian day is a term that is grossly misused. 
>> Julian Day is defined as the elapsed days since January 1, 4713 BCE. Lots of 
>> people use the term to refer to day-in-year, but this doesn't seem to be a 
>> proper usage.
>> 
>> Grace and peace,
>> 
>> Jim
>> 
>> On 3/16/17 3:36 PM, John Helly wrote:
>>> Sorry to jump in here but isn't this just the Julian day? 
>>> 
>>> J. 
>>> 
>>> 
>>> On 3/16/17 8:24 AM, Nan Galbraith wrote: 
 I agree that there's a lot of interest, and I have 2 questions. 
 
 To make the data most useful, shouldn't the time coordinate variable be 
 Jan 1, and shouldn't the 'days since' (data) variable represent the 
 yearday 
 within that year? 
 
 My specific concerns with Jim's approach: 
 
 first_freeze_date:units = "days since 1900-01-01 00:00:00"   - This 
 doesn't seem 
 to me to provide the most easily used data point, wouldn't the year-day be 
 more 
 convenient, for seeing how this value varies over the years? 
 
 And with Antoio's: 
 
 first_freeze_date:coordinates="threshold time"; - I don't see how 
 threshold, 
 which is a temperature, can be a coordinate of this variable. Also, I'd 
 like to know 
 why setting   time:units="days since 2000-6-1"; is preferable to using 
 2000-1-1; 
 doesn't this invite errors in using the time in applications like matlab 
 and python? 
 
 Actually, the metadata doesn't tell me how to interpret the values in 
 first_freeze_date - 
 the short name implies that they're dates, the units implies they're 
 elapsed days, but 
 without a reference date to enable decoding. 
 
 Cheers - Nan 
 
 
 On 3/16/17 8:45 AM, Jim Biard wrote: 
> 
> Hi. 
> 
> There is clearly interest here! I agree that day_in_year is rather 
> generic, and there should probably be a more precise term. I'm not so 
> sure about the cell_methods that were suggested below. In my particular 
> case the values are derived from a daily Tmin product. Each value is the 
> date of the first Tmin < 0 C within the time bounds. If it was a spell 
> length, such as growing season length, then I can see the need for a more 
> climatological cell_method. 
> 
> We can keep this up and work up some standard_name definitions to 
> propose. I'm sure the results will be better if we collaborate compared 
> to what I'd do on my own. 
> 
> Grace and peace, 
> 
> Jim 
> 
> 
> On 3/16/17 7:23 AM, Antonio S. Cofi�o wrote: 
>> Dear all, 
>> There is no standard_name for the concept but there are 2 different ones 
>> which delimit the approach that it could be used as templates for the 
>> new one: 
>> *time_when_flood_water_falls_below_threshold 
>> *(time_when_flood_water_rises_above_threshold and 
>> time_of_maximum_flood_depth are also good examples ) 
>> http://cfconventions.org/Data/cf-standard-names/41/build/cf-standard-name-table.html#time_when_flood_water_falls_below_threshold_tr
>>  
>>> The quantity with standard name 
>>> *time_when_flood_water_falls_below_threshold*: is the time elapsed 
>>> between the breaking of a levee (origin of flood water simulation) and 
>>> the instant when the depth falls below a given threshold for the last 
>>> time, having already risen to its maximum depth, at a given point in 
>>> space. If a threshold is supplied, it should be specified by 
>>> associating a coordinate variable or scalar coordinate variable with 
>>> the data variable and giving the coordinate variable a standard name of 
>>> flood_water_thickness. The values of the coordinate variable are the 
>>> threshold values for the corresponding subarrays of the data variable. 
>>> If no threshold is specified, its value is taken to be zero. Flood 
>>> water is water that covers land which is normally not covered by water. 
>> the problem is the event definition, which is quite different to the one 
>> it's been considered here which is more like a climatological 
>> statistics. The good thing is the CF already has some good definitions 
>> for those climatological statistics, like Example 7.11 on CF1.6 
>> document: 
>> http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#extreme-statistics-and-spell-lengths-ex
>>  
>> 
>> And more convenient definition of this 

Re: [CF-metadata] source, institution, and history

[Roy Mendelssohn - NOAA Federal]

Sorry I hit the send button by accident.  Yes,  this is a very real question 
for us.  If we make a new dataset from say two JPL files,  JPL is no longer the 
creator, first because they didn't create it,  but even more to say they are 
are the creator is to imply that they have put their stamp of approval on the 
product,  which they have not.  At the same time,  we want to give proper 
credit that the new file was created from say two JPL files.

Out of all these fields,  we have been debating what is the most appropriate 
way to do so.

-Roy
> On Jan 20, 2017, at 6:20 PM, John Graybeal  wrote:
> 
> Hi all,
> 
> Over on the ACDD (esip-documentation) list, we started a thread to answer 
> some questions about ACDD attributes. One question related to the 3 CF 
> attribute definitions for source, institution, and history (documented in 
> http://cfconventions.org/Data/cf-conventions/cf-conventions-1.7/build/ch02s06.html#description-of-file-contents);
>  I believe ACDD slavishly follows the CF definitions so this list seems the 
> authority.
> 
> I’ve copied that bit of the ACDD thread below for context. (In that thread 
> JPL provided initial data, and ERD processes those data to create the ‘new’ 
> data set that’s being documented.) But the question boils down to the 
> following.
> 
> Do CF source, institution, and history attributes all document the ‘new’ data 
> set, referred to in CF as the current file? (Please note that in some 
> definitions, like institution, this is called “the original data”.)  This is 
> what the introduction to the section seems to say. But if this ‘new’ data set 
> *is* "the original data”, how can there be any “modifications to the original 
> data” as described in the history definition?
> 
> Thanks very much for your input, and apologies for the cross-posting.
> 
> John
> 
> 
>>> source: Should this be info what JPL did since original data is specified 
>>> in the definition (below)
>>> 'The method of production of the original data. If it was model-generated, 
>>> source should name the model and its version, as specifically as could be 
>>> useful. If it is observational, source should characterize it (e.g., " 
>>> surface observation " or " radiosonde “)'
>> 
>> As ’source’ comes directly from the CF conventions, I went back to those 
>> definitions, and I’m still not sure. (See also the next two items.) If no 
>> one on this list sounds authoritative, I suggest you send a question to the 
>> CF conventions list (mailto:cf-metadata@cgd.ucar.edu, subscribe at 
>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata). Actually, I’ll do 
>> that now, just so the CF folks are aware of the question.
>> 
>>> institution: JPL nice original data is specified
>>> 'Specifies where the original data was produced’
>> 
>> Will be the same answer as source. I would have agreed with you, but then 
>> the history definition (next) seems to suggest otherwise.
>>> 
>>> history: where we describe modifications to the original data. 
>>> 'Provides an audit trail for modifications to the original data. 
>>> Well-behaved generic netCDF filters will automatically append their name 
>>> and the parameters with which they were invoked to the global history 
>>> attribute of an input netCDF file. We recommend that each line begin with a 
>>> timestamp indicating the date and time of day that the program was executed’
>> 
>> So the relevant CF section for source, institution, and history is 
>> http://cfconventions.org/Data/cf-conventions/cf-conventions-1.7/build/ch02s06.html#description-of-file-contents.
>>  While the definitions for institution (’Specifies where the original data 
>> was produced’) is almost certainly describing the current file’s 
>> institution, the  history is ‘an audit trail for modifications to the 
>> original data’. If this file _is_ the original data, I don’t see how there 
>> can be any modifications to it yet, I would call this a CF issue (and we of 
>> ACDD carefully avoided messing with CF definitions, as I recall), so maybe 
>> that group needs to weigh in.
> 
> 
> 
> ---
> John Graybeal
> jbgrayb...@mindspring.com
> linkedin: http://www.linkedin.com/in/johngraybeal/
> 
> ___
> CF-metadata mailing list
> CF-metadata@cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

**
"The contents of this message do not reflect any position of the U.S. 
Government or NOAA."
**
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new street address***
110 McAllister Way
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: roy.mendelss...@noaa.gov www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected" 
"the arc of 

Re: [CF-metadata] source, institution, and history

[Roy Mendelssohn - NOAA Federal]

> On Jan 20, 2017, at 6:20 PM, John Graybeal  wrote:
> 
> Hi all,
> 
> Over on the ACDD (esip-documentation) list, we started a thread to answer 
> some questions about ACDD attributes. One question related to the 3 CF 
> attribute definitions for source, institution, and history (documented in 
> http://cfconventions.org/Data/cf-conventions/cf-conventions-1.7/build/ch02s06.html#description-of-file-contents);
>  I believe ACDD slavishly follows the CF definitions so this list seems the 
> authority.
> 
> I’ve copied that bit of the ACDD thread below for context. (In that thread 
> JPL provided initial data, and ERD processes those data to create the ‘new’ 
> data set that’s being documented.) But the question boils down to the 
> following.
> 
> Do CF source, institution, and history attributes all document the ‘new’ data 
> set, referred to in CF as the current file? (Please note that in some 
> definitions, like institution, this is called “the original data”.)  This is 
> what the introduction to the section seems to say. But if this ‘new’ data set 
> *is* "the original data”, how can there be any “modifications to the original 
> data” as described in the history definition?
> 
> Thanks very much for your input, and apologies for the cross-posting.
> 
> John
> 
> 
>>> source: Should this be info what JPL did since original data is specified 
>>> in the definition (below)
>>> 'The method of production of the original data. If it was model-generated, 
>>> source should name the model and its version, as specifically as could be 
>>> useful. If it is observational, source should characterize it (e.g., " 
>>> surface observation " or " radiosonde “)'
>> 
>> As ’source’ comes directly from the CF conventions, I went back to those 
>> definitions, and I’m still not sure. (See also the next two items.) If no 
>> one on this list sounds authoritative, I suggest you send a question to the 
>> CF conventions list (mailto:cf-metadata@cgd.ucar.edu, subscribe at 
>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata). Actually, I’ll do 
>> that now, just so the CF folks are aware of the question.
>> 
>>> institution: JPL nice original data is specified
>>> 'Specifies where the original data was produced’
>> 
>> Will be the same answer as source. I would have agreed with you, but then 
>> the history definition (next) seems to suggest otherwise.
>>> 
>>> history: where we describe modifications to the original data. 
>>> 'Provides an audit trail for modifications to the original data. 
>>> Well-behaved generic netCDF filters will automatically append their name 
>>> and the parameters with which they were invoked to the global history 
>>> attribute of an input netCDF file. We recommend that each line begin with a 
>>> timestamp indicating the date and time of day that the program was executed’
>> 
>> So the relevant CF section for source, institution, and history is 
>> http://cfconventions.org/Data/cf-conventions/cf-conventions-1.7/build/ch02s06.html#description-of-file-contents.
>>  While the definitions for institution (’Specifies where the original data 
>> was produced’) is almost certainly describing the current file’s 
>> institution, the  history is ‘an audit trail for modifications to the 
>> original data’. If this file _is_ the original data, I don’t see how there 
>> can be any modifications to it yet, I would call this a CF issue (and we of 
>> ACDD carefully avoided messing with CF definitions, as I recall), so maybe 
>> that group needs to weigh in.
> 
> 
> 
> ---
> John Graybeal
> jbgrayb...@mindspring.com
> linkedin: http://www.linkedin.com/in/johngraybeal/
> 
> ___
> CF-metadata mailing list
> CF-metadata@cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

**
"The contents of this message do not reflect any position of the U.S. 
Government or NOAA."
**
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new street address***
110 McAllister Way
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: roy.mendelss...@noaa.gov www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected" 
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May 2016, Boulder, CO, USA

[Roy Mendelssohn - NOAA Federal]

Hi David:

I was unaware of these, thanks.  This makes all the more important to look at 
the new HDF5 virtual datasets, which are basically virtual aggregations, in the 
sense of making sure that there is some harmonization.  This is a new feature, 
and nows the time to make certain, if at all possible, that CF aggregation 
rules can be successfully implemented in the new virtual files.

-Roy


> On Mar 31, 2016, at 4:39 AM, David Hassell  wrote:
> 
> Hi David,
> 
> The proposed CF aggregation rules
> (http://cf-trac.llnl.gov/trac/ticket/78,
> http://www.met.reading.ac.uk/~david/cf_aggregation_rules.html) have
> been designed to aggregate *any* CF-compliant datasets, where
> appropriate. I would be very interested in looking at the metadata of
> some of your files to (hopefully!) confirm this in your use case.
> 
> A netCDF schema for storing the results of such aggregations has also
> bee developed (http://www.met.reading.ac.uk/~david/cfa/0.4/).
> 
> As far as I'm aware, there's not yet much software which understands
> these frameworks, apart from cf-python.
> 
> All the best,
> 
> David
> 
>  Original message from Moroni, David F (398G) (12AM 29 Mar 16)
> 
>> Date: Tue, 29 Mar 2016 00:01:00 +
>> From: "Moroni, David F (398G)" 
>> To: Steve Hankin , Ethan Davis ,
>> CF metadata , netCDF SWG
>> 
>> Subject: Re: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May
>> 2016, Boulder, CO, USA
>> user-agent: Microsoft-MacOutlook/14.6.2.160219
>> 
>> Hi Ethan,
>> 
>> I’m glad Steve brought up the “aggregation” topic. On this topic, it would 
>> be interesting to see if anyone has successfully come up with a standardized 
>> schema to aggregate satellite-derived Level 2 swath grid structures.
>> 
>> I know there’s an aggregation implementation in the latest OPeNDAP release, 
>> but it remains to be seen what this might look like for Level 2 (and perhaps 
>> even Level 1) swath grid structured datasets. I don’t know of anyone who’s 
>> successfully tested and released an operationally working instance of that 
>> part of OPeNDAP for swath grid structured datasets.
>> 
>> Cheers,
>> David
>> 
>> From: CF-metadata 
>> > 
>> on behalf of Steve Hankin 
>> >
>> Date: Monday, March 28, 2016 at 9:25 AM
>> To: Ethan Davis >, CF metadata 
>> >, netCDF SWG 
>> >
>> Subject: Re: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May 
>> 2016, Boulder, CO, USA
>> 
>> Ethan,
>> 
>> Two more topics to consider as additions to your list:
>> 
>> 1. "UGRID" -- Of interest to the coastal ocean modeling community, and stuck 
>> in a holding pattern for quite a long time, is the harmonization of 
>> mesh/unstructured grid coordinates into the main body of CF.  The work is 
>> mostly completed ...
>> 
>> 2. "Aggregation" -- CF structures to link multiple files into larger 
>> conceptual datasets.  Time series aggregations, union aggregations 
>> (associated variables in separate files), and ensemble membership are the 
>> most obvious applications of this.   Another important application is "tiled 
>> grids" (ref. the "gridspec" proposal).
>> 
>>- Steve
>> 
>> 
>> 
>> On 3/27/2016 9:24 PM, Ethan Davis wrote:
>> Hi all,
>> 
>> As some of you have heard, we are holding a meeting to discuss current and 
>> future netCDF-CF efforts and directions. The meeting will be held on 24-26 
>> May 2016 in Boulder, CO, USA at the UCAR Center Green facility.
>> 
>> This meeting is organized by the “Advancing netCDF-CF for Geoscience” 
>> project [1] which has been funded by the US NSF EarthCube program. The 
>> project goals include several specific CF development efforts, both 
>> standards development and software prototyping. (We are just starting to 
>> spin-up community conversations around these specific goals for drafting CF 
>> extensions.) Another aspect of the funded project is community engagement 
>> including with the existing CF community, geoscience domains that have not 
>> been very involved in CF, and other standards bodies (e.g., OGC). These and 
>> other topics are on the current list of possible agenda items. Please let us 
>> know if you have other agenda items in mind.
>> 
>> The current list of possible agenda items includes:
>> 
>>  *   Investigate using features of the netCDF enhanced data model to 
>> improve/simplify the CF Discrete Sampling Geometries (which includes point, 
>> sounding, and trajectory data types)
>>  *   Drafting a CF enhancement that would add support for complex data 

[CF-metadata] Standard attributes for anomaly

[Roy Mendelssohn - NOAA Federal]

Hi All:

We have been given files related o sea surface temperature.  The observed value 
has the attributes:

analysed_sst:long_name = "analysed sea surface temperature" ;
analysed_sst:standard_name = 
"sea_surface_foundation_temperature" ;


and the climatology has the attributes:

mean_sst:long_name = "climatological mean sea surface 
temperature" ;
mean_sst:standard_name = "sea_surface_foundation_temperature" ;

What would be the proper attributes for the anomaly calculated from these?

Thanks,

-Roy

**
"The contents of this message do not reflect any position of the U.S. 
Government or NOAA."
**
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new address and phone***
110 Shaffer Road
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: roy.mendelss...@noaa.gov www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected" 
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] [NetCDF.swg] Meeting invite: Advancing netCDF-CF, 24-26 May 2016, Boulder, CO, USA

[Roy Mendelssohn - NOAA Federal]

> On Mar 28, 2016, at 9:25 AM, Steve Hankin via NetCDF.swg 
>  wrote:
> 
> 2. "Aggregation" -- CF structures to link multiple files into larger 
> conceptual datasets.  Time series aggregations, union aggregations 
> (associated variables in separate files), and ensemble membership are the 
> most obvious applications of this.   Another important application is "tiled 
> grids" (ref. the "gridspec" proposal).
> 
> - Steve


1.  Who is this Steve person??????!!

2.  Speaking of aggregation, this should be included in the discussion somehow:

https://www.hdfgroup.org/HDF5/Tutor/vds.html

and

https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesVirtualDatasetDocs.html

-Roy


**
"The contents of this message do not reflect any position of the U.S. 
Government or NOAA."
**
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new address and phone***
110 Shaffer Road
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: roy.mendelss...@noaa.gov www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected" 
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Reference for GRIDSPEC?

[Roy Mendelssohn - NOAA Federal]

> On Dec 18, 2015, at 10:47 AM, V Balaji - NOAA Affiliate  
> wrote:
> 
> Also, neither I nor any of the original proponents are actively
> developing gridspec at this point. I am happy to support further
> development along the ugrid lines, and treat gridspec's ugrid as a
> historical oddity... the Betamax of unstructured grid specification.

I think Balaji won the internet today with that final comment.

-Roy


**
"The contents of this message do not reflect any position of the U.S. 
Government or NOAA."
**
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new address and phone***
110 Shaffer Road
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: roy.mendelss...@noaa.gov www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected" 
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Fwd: [NetCDF.swg] OGC NetCDF SWG DIscussions

[Roy Mendelssohn - NOAA Federal]

I pass this along to the CF list for obvious reasons.

-Roy

> Begin forwarded message:
> 
> From: Ben Domenico 
> Subject: Re: [NetCDF.swg] OGC NetCDF SWG DIscussions
> Date: October 21, 2015 at 8:16:22 PM PDT
> To: Cox Simon 
> Cc: "netcdf@lists.opengeospatial.org" 
> , Greg Buehler 
> 
> 
> Hi Simon,
> 
> Thanks for the suggestion.That approach would seem to have many 
> advantages, but my sense is that there would be some concerns among some 
> members of the CF conventions community.   But, with Russ Rew and John Caron 
> having left Unidata, I need to figure out a good liaison with whom to broach 
> the topic.
> 
> Have you brought it up with any of the active CF brethren?
> 
> -- Ben
> 
> On Wed, Oct 21, 2015 at 3:53 PM,  > wrote:
> Hi Ben –
> 
>  
> 
> Regarding CF convention terms – would there be any interest in seeing these 
> hosted by OGC with a permanent URI per item?
> 
> As I think you are aware, we are moving the OGC system to a registry in which 
> maintenance of the content can be delegated to appropriate managers. Mark 
> Hedley (who was on the program here) is very familiar with this system too.
> 
>  
> 
> Take a look at our test system here http://registry.it.csiro.au/def 
> 
> We have a bunch of sandboxes here http://registry.it.csiro.au/sandbox 
>  so I could create one there for netCDF 
> to do some testing.
> 
>  
> 
> Simon
> 
>  
> 
>  
> 
> From: NetCDF.swg [mailto:netcdf.swg-bounces+simon.cox 
> =csiro...@lists.opengeospatial.org 
> ] On Behalf Of Ben Domenico
> Sent: Thursday, 22 October 2015 2:21 AM
> To: netcdf@lists.opengeospatial.org 
> 
> Subject: [NetCDF.swg] OGC NetCDF SWG DIscussions
> 
>  
> 
> Hello all:
> 
>  
> 
> First and foremost, I thank Lorenzo Bigagli for chairing the OGC TC NetCDF 
> SWG session at the TC meetings in Nottingham.   Stefano and I were unable to 
> attend those meetings.   The items from his meeting summary for the closing 
> plenary are included below along with discussion topics that have arisen in 
> email and other interactions since the meeting.
> 
> 
> 
> -- Ben
> 
>  
> 
> 
> 
>  
> 
> Nottingham TC NetCDF SWG Session Summary Items
> 
>  
> 
> Agenda:
> 
> 1.   NetCDF-CF – ISO 19115 conceptual model mapping – Stefano Nativi 
> (CNR) [skipped].ISO Spatial referencing for netCDF – Mark Hedley (MetOffice)
> 
> 2.   NetCDF Uncertainty Conventions (netCDF-U) update – Lorenzo Bigagli 
> (CNR)
> 
> 3.   Update on MetOceans interactions
> 
> 4.   Update on WCS interaction
> 
> 5.   AOB
> 
> Issues:
> 
> 1.   How to apply Well Known Text for Coordinate Reference Systems to 
> CF-netCDF
> 
> o Consider OGC standards to map CF-netCDF to ISO terms
> 
> 2.   OGC may investigate a standardised approach to encoding spatial 
> coordinates and spatial referencing in netCDF
> 
> o Possibly applicable also to other formats (HDF, JSON)
> 
> o May be provided back to CF as a basis for a revision of the CF Conventions
> 
> o Equivalent to OGC Coverage Implementation Schema, but for NetCDF
> 
> Actions:
> 
> ·  Ask OGC staff about publishing revised NetCDF-U Conventions DP as 
> corrigendum
> 
> ·  Continue gathering NetCDF-U use-cases and usage examples
> 
> o Aim: NetCDF-U Best Practices document
> 
> 
> 
> Items from follow up discussions
> 
>  
> 
> The items below have come up in follow up discussions after the Nottingham 
> meetings.   Many of them warrant further discussion at the NetCDF session at 
> the next OGC TC meeting in Sydney.
> 
>  
> 
> Issue From the Nottingham Meeting Summary
> 
>  
> 
> "OGC may investigate a standardised approach to encoding spatial coordinates 
> and spatial referencing in netCDF."  In practice, how do we to follow up on 
> this issue?  Is it an item for the netCDF SWG or an ongoing action in some 
> other WG?  Is it related to the more general OGC effort to “standardize” the 
> CRS specification, introducing N-Dimensions? 
> 
>  
> 
> ESA Prod Trees Project Completion
> 
>  
> 
> ESA Prod Trees project is now complete and plans to contribute its work to 
> the OGC NetCDF SWG. CNR has already given of presentations on that work in 
> earlier OGC TC meetings.  It will be useful to continue efforts to harmonize 
> this and other ongoing efforts within the OGC SWG activity and establish 
> extended conventions supporting the Geosciences (Earth Sciences) Community.   
> Likewise it is important to continuing to ensure that these conventions do 
> not conflict with 

Re: [CF-metadata] Are ensembles a compelling use case for group-aware metadata?

[Roy Mendelssohn - NOAA Federal]

Hi All:

Charlie provided me with some sample files.  Below, with but two comments, is  
the one line command in R for several examples, using David Pierce's wonderful 
ncdf4 package, to traverse the complete structure of the file. The results show 
the command and the resulting structure. My two comments are:

1.  As a user, compare this to receiving multiple files with a manifest.

2. Note that the return, as all returns from ncdf4, whether the file is flat or 
has groups, is a hierarchical structure.

-Roy

***   Example 1 *

 testFile-nc_open('cmip5.nc')
 str(testFile)
List of 11
 $ filename  : chr cmip5.nc
 $ writable  : logi FALSE
 $ id: int 65536
 $ format: chr NC_FORMAT_NETCDF4
 $ groups:List of 4
  ..$ :List of 7
  .. ..$ id   : int 65536
  .. ..$ name : chr 
  .. ..$ ndims: int 0
  .. ..$ nvars: int 0
  .. ..$ natts: int 3
  .. ..$ dimid: int[0 (1d)] 
  .. ..$ fqgn : chr 
  .. ..- attr(*, class)= chr ncgroup4
  ..$ :List of 7
  .. ..$ id   : int 65537
  .. ..$ name : chr cesm
  .. ..$ ndims: int 1
  .. ..$ nvars: int 1
  .. ..$ natts: int 1
  .. ..$ dimid: int [1(1d)] 0
  .. ..$ fqgn : chr cesm
  .. ..- attr(*, class)= chr ncgroup4
  ..$ :List of 7
  .. ..$ id   : int 65538
  .. ..$ name : chr ecmwf
  .. ..$ ndims: int 1
  .. ..$ nvars: int 1
  .. ..$ natts: int 1
  .. ..$ dimid: int [1(1d)] 1
  .. ..$ fqgn : chr ecmwf
  .. ..- attr(*, class)= chr ncgroup4
  ..$ :List of 7
  .. ..$ id   : int 65539
  .. ..$ name : chr giss
  .. ..$ ndims: int 1
  .. ..$ nvars: int 1
  .. ..$ natts: int 1
  .. ..$ dimid: int [1(1d)] 2
  .. ..$ fqgn : chr giss
  .. ..- attr(*, class)= chr ncgroup4
 $ ndims : num 3
 $ natts : num 6
 $ dim   :List of 3
  ..$ cesm/time :List of 10
  .. ..$ name : chr cesm/time
  .. ..$ len  : int 4
  .. ..$ unlim: logi TRUE
  .. ..$ group_index  : int 2
  .. ..$ group_id : int 65537
  .. ..$ id   : int 0
  .. ..$ dimvarid :List of 5
  .. .. ..$ id : int -1
  .. .. ..$ group_index: int 2
  .. .. ..$ group_id   : int 65537
  .. .. ..$ list_index : num -1
  .. .. ..$ isdimvar   : logi TRUE
  .. .. ..- attr(*, class)= chr ncid4
  .. ..$ vals : int [1:4] 1 2 3 4
  .. ..$ units: chr 
  .. ..$ create_dimvar: logi FALSE
  .. ..- attr(*, class)= chr ncdim4
  ..$ ecmwf/time:List of 10
  .. ..$ name : chr ecmwf/time
  .. ..$ len  : int 4
  .. ..$ unlim: logi TRUE
  .. ..$ group_index  : int 3
  .. ..$ group_id : int 65538
  .. ..$ id   : int 1
  .. ..$ dimvarid :List of 5
  .. .. ..$ id : int -1
  .. .. ..$ group_index: int 3
  .. .. ..$ group_id   : int 65538
  .. .. ..$ list_index : num -1
  .. .. ..$ isdimvar   : logi TRUE
  .. .. ..- attr(*, class)= chr ncid4
  .. ..$ vals : int [1:4] 1 2 3 4
  .. ..$ units: chr 
  .. ..$ create_dimvar: logi FALSE
  .. ..- attr(*, class)= chr ncdim4
  ..$ giss/time :List of 10
  .. ..$ name : chr giss/time
  .. ..$ len  : int 4
  .. ..$ unlim: logi TRUE
  .. ..$ group_index  : int 4
  .. ..$ group_id : int 65539
  .. ..$ id   : int 2
  .. ..$ dimvarid :List of 5
  .. .. ..$ id : int -1
  .. .. ..$ group_index: int 4
  .. .. ..$ group_id   : int 65539
  .. .. ..$ list_index : num -1
  .. .. ..$ isdimvar   : logi TRUE
  .. .. ..- attr(*, class)= chr ncid4
  .. ..$ vals : int [1:4] 1 2 3 4
  .. ..$ units: chr 
  .. ..$ create_dimvar: logi FALSE
  .. ..- attr(*, class)= chr ncdim4
 $ unlimdimid: num 1
 $ nvars : num 3
 $ var   :List of 3
  ..$ cesm/tas :List of 21
  .. ..$ id  :List of 5
  .. .. ..$ id : num 0
  .. .. ..$ group_index: num -1
  .. .. ..$ group_id   : int 65537
  .. .. ..$ list_index : num 1
  .. .. ..$ isdimvar   : logi FALSE
  .. .. ..- attr(*, class)= chr ncid4
  .. ..$ name: chr cesm/tas
  .. ..$ ndims   : int 1
  .. ..$ natts   : int 0
  .. ..$ size: int 4
  .. ..$ dimids  : int 0
  .. ..$ prec: chr float
  .. ..$ units   : chr 
  .. ..$ longname: chr tas
  .. ..$ group_index : int 2
  .. ..$ chunksizes  : int 1
  .. ..$ storage : int 2
  .. ..$ shuffle : int 0
  .. ..$ compression : logi NA
  .. ..$ dims: list()
  .. ..$ dim :List of 1
  .. .. ..$ :List of 10
  .. .. .. ..$ name : chr cesm/time
  .. .. .. ..$ len  : int 4
  .. .. .. ..$ unlim: logi TRUE
  .. .. .. ..$ group_index  : int 2
  .. .. .. ..$ group_id : int 65537
  .. .. .. ..$ id   : int 0
  .. .. .. ..$ dimvarid :List of 5
  .. .. .. .. ..$ id : int -1
  .. .. .. .. ..$ group_index: int 2
  .. .. .. .. ..$ group_id   : int 65537
  .. .. .. .. ..$ list_index : num -1
  .. .. .. .. ..$ isdimvar   : logi TRUE
  .. .. .. .. ..- attr(*, class)= chr ncid4
  .. .. .. ..$ vals : int [1:4] 1 2 3 4
  .. .. .. ..$ units: chr 
  .. .. .. ..$ create_dimvar: logi FALSE
  .. .. .. ..- attr(*, class)= 

Re: [CF-metadata] Towards recognizing and exploiting hierarchical groups (Charlie Zender - Steve Hankin - Richard Signell)

[Roy Mendelssohn - NOAA Federal]

Hi All:

NASA has used hierarchies for years, and appears committed to them.  So, either 
it is done in an ad hoc way, or through a standard.  That doesn't mean CF is 
the place for the standard, just that it would be nice to have one.

I would point out that every major modern  programming language has structures, 
which are essentially hierarchies.  Matlab was criticized for years about not 
having structures, and finally added them a few years back.  R has them, C has 
them, Python has them, even modern Fortran has them.  So clearly there must be 
situations where hierarchies make sense, and are more efficient than having 
everything flat.  There are clearly situations where flattening everything 
makes sense.

My $0.02.

-Roy



On Sep 18, 2013, at 4:52 AM, Signell, Richard rsign...@usgs.gov wrote:

 All,
 
 I'm glad we are discussing this topic, but the fact that large data
 providers are already distributing data using groups and hierarchies
 is not a compelling reason to endorse this practice through CF.  After
 all, a lot of data providers are currently distributing scientific
 data in any number of forms, and the point of CF (along with OGC
 standards) is to help clean up the mess!
 
 I agree that groups make sense for metadata and for certain types of
 datasets.  For example, the discrete sampling geometry featureTypes
 like profile collection would be easier to understand and deal with as
 a netcdf4 group of profiles rather than as a netcdf3 ragged array.
 But the choice was made for CF 1.6 that backward compatibility was
 more important.
 
 I don't think it's cowardly to belive that the more folks use groups
 to organize their data in an ad hoc way (the suitcase analogy), the
 more it will hinder the remarkable progress that has been made
 recently on finding and utilizing distributed CF data via the catalog
 services (e.g. the geonetwork, gi-cat, geoportal, CKAN instances) that
 many governments are setting up.   When we open the data service
 endpoints that our query returns, we need to have known data
 structures, and that's what the CF featureTypes provide.
 
 To return to the suitcase/clothing analogy again, we are rapidly
 gaining the capability via good metadata and catalog services to find
 all the black socks owned by Jim and Martin that have been washed in
 the last week.  But if our catalog query returns fourteen of Jim's
 suitcases and twelve of Martin's, then we have more work to do.
 Unlike socks, luckily we don't need actual suitcases to organize data,
 we can construct collections on the fly using whatever attributes we
 desire.
 
 I would hope that our job as the CF community would be to identify
 compelling additional specific featureTypes that we should support.
 And if these identified featureTypes demand groups for efficiency or
 some other reason, well, let's have that discussion.
 
 -Rich
 
 On Wed, Sep 18, 2013 at 12:08 AM, Roy Mendelssohn - NOAA Federal
 roy.mendelss...@noaa.gov wrote:
 Hi All:
 
 I am old and slow, and I must be missing something, because at this point 
 most of the discussion has been about the desirability of files with groups 
 and hierarchies.  Again, unless I am missing something, there already are 
 data providers who are distributing data using groups and hierarchies, 
 including at least one very large data provider,  and they obviously feel 
 that there is a benefit to such structures.  I am not arguing whether they 
 are right or wrong, just that is the reality.
 
 If we start from that premise, then the real questions for discussion are 
 should there be conventions on how groups and hierarchies are used in 
 netcdf4 and hdf5 files, so that a user or software provider will know what 
 to expect, and the second question is if it is deemed desirable to have such 
 conventions, is CF the  proper place for them to be developed.
 
 My sense it that this is what the original proposers are after.
 
 -Roy
 
 
 **
 The contents of this message do not reflect any position of the U.S. 
 Government or NOAA.
 **
 Roy Mendelssohn
 Supervisory Operations Research Analyst
 NOAA/NMFS
 Environmental Research Division
 Southwest Fisheries Science Center
 1352 Lighthouse Avenue
 Pacific Grove, CA 93950-2097
 
 e-mail: roy.mendelss...@noaa.gov (Note new e-mail address)
 voice: (831)-648-9029
 fax: (831)-648-8440
 www: http://www.pfeg.noaa.gov/
 
 Old age and treachery will overcome youth and skill.
 From those who have been given much, much will be expected
 the arc of the moral universe is long, but it bends toward justice -MLK Jr.
 
 ___
 CF-metadata mailing list
 CF-metadata@cgd.ucar.edu
 http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
 
 
 
 -- 
 Dr. Richard P. Signell   (508) 457-2229
 USGS, 384 Woods Hole Rd.
 Woods Hole, MA 02543-1598

**
The contents of this message do not reflect any position of the U.S. 
Government or NOAA

Re: [CF-metadata] Towards recognizing and exploiting hierarchical groups (Charlie Zender - Steve Hankin - Richard Signell)

[Roy Mendelssohn - NOAA Federal]

Hi All:

I am old and slow, and I must be missing something, because at this point most 
of the discussion has been about the desirability of files with groups and 
hierarchies.  Again, unless I am missing something, there already are data 
providers who are distributing data using groups and hierarchies, including at 
least one very large data provider,  and they obviously feel that there is a 
benefit to such structures.  I am not arguing whether they are right or wrong, 
just that is the reality.

If we start from that premise, then the real questions for discussion are 
should there be conventions on how groups and hierarchies are used in netcdf4 
and hdf5 files, so that a user or software provider will know what to expect, 
and the second question is if it is deemed desirable to have such conventions, 
is CF the  proper place for them to be developed.

My sense it that this is what the original proposers are after.

-Roy


**
The contents of this message do not reflect any position of the U.S. 
Government or NOAA.
**
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097

e-mail: roy.mendelss...@noaa.gov (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/

Old age and treachery will overcome youth and skill.
From those who have been given much, much will be expected 
the arc of the moral universe is long, but it bends toward justice -MLK Jr.

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata