Re: [CF-metadata] Convention attribute

[Jonathan Gregory]

Dear Dave

> I will be glad to open a trac ticket.  Let me have a day or so draft
> some amendment wording.  My intention is to use Unidata's
> recommendations in the NetCDF Users Guide as a starting point, and
> include notes about conformance and avoiding conflicts.

Thanks for volunteering. Best wishes

Jonathan
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Convention attribute

[Dave Allured]

I will be glad to open a trac ticket.  Let me have a day or so draft
some amendment wording.  My intention is to use Unidata's
recommendations in the NetCDF Users Guide as a starting point, and
include notes about conformance and avoiding conflicts.

--Dave

On Tue, Jan 3, 2012 at 11:49 AM, Jonathan Gregory
 wrote:
> Dear all - such as Roy, John (Graybeal) and Mark (Hedley)
>
> Would someone have time to open a trac ticket to propose a change to the CF
> convention, in order to clarify the Conventions attribute, following all this
> useful email discussion? It seems to me that it would be good to make a
> definite decision. I think it's been recognised that, whatever the syntax, it
> is up to the data-writer to ensure that multiple conventions do not conflict,
> so we ought to state this as well as defining the permissible syntax.
>
> Best wishes
>
> Jonathan
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Convention attribute

[Jonathan Gregory]

Dear all - such as Roy, John (Graybeal) and Mark (Hedley)

Would someone have time to open a trac ticket to propose a change to the CF
convention, in order to clarify the Conventions attribute, following all this
useful email discussion? It seems to me that it would be good to make a
definite decision. I think it's been recognised that, whatever the syntax, it
is up to the data-writer to ensure that multiple conventions do not conflict,
so we ought to state this as well as defining the permissible syntax.

Best wishes

Jonathan

On Mon, Jan 02, 2012 at 07:21:52PM +, Lowry, Roy K. wrote:
> 
> Hi John,
> 
> Guess we're on the same wavelength.  99% of usage of the conventions 
> attribute string will be searches using functions like INSTR (function that 
> locates a substring wihin a string), which make delimiters irrelevant but as 
> Benno pointed out carry a (minimal if we're aware) element of risk.  
> Therefore those of us that care are free to use delimiter conventions to 
> convey semantics on the understanding that their point will be missed by the 
> vast majority of users.
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Convention attribute

[Lowry, Roy K.]

Hi John,

Guess we're on the same wavelength.  99% of usage of the conventions attribute 
string will be searches using functions like INSTR (function that locates a 
substring wihin a string), which make delimiters irrelevant but as Benno 
pointed out carry a (minimal if we're aware) element of risk.  Therefore those 
of us that care are free to use delimiter conventions to convey semantics on 
the understanding that their point will be missed by the vast majority of users.

Cheers, Roy.


From: John Graybeal [jbgrayb...@mindspring.com]
Sent: 02 January 2012 18:47
To: Lowry, Roy K.
Cc: CF Metadata List; sdn2-t...@seadatanet.org
Subject: Re: [CF-metadata] Convention attribute

I wasn't sure how to parse these, I'm a little slow today I guess.  After 
trying a few ways, I decided they mostly use spaces to separate convention 
identifiers, and slashes to designate hierarchy. (Except the first two embed a 
space within "OceanSITES x.x", which I think should be a hyphen.)

Then I finally got your point, that / (slash) is also a delimiter, but one with 
an explicit meaning.

> I guess the $64,000 question is whether any application program cares about 
> such subtleties and I think the answer is probably not.

As of today, certainly no application program cares about such subtleties, 
since no standard or community practice exists to make use of them. But even if 
we state the question as "what is likely to be the most useful in the future, 
while being usable now?" (likely what you meant), I can't find a use case where 
the computer would use the order information, other than for display -- the 
context being of some educational value to users of the data sets, as you say.

But this leads to an implementation question, using the examples (modified with 
hyphens per the above comment):

> (A) CF-x.x OceanSITES-x.x SeaDataNet-x.x -- 3 conventions are listed
> (B) CF-x.x/OceanSITES-x.x/SeaDataNet-x.x -- 1 conventions are listed,  
> with its relationship to two others
> (C) CF-x.x/SeaDataNet-x.x CF-x.x/OceanSITES-x.x-- 2 conventions are 
> listed, each with its relationship to a third

>From these strings, in the first and third case we don't really know either 
>way about some of the relationships -- the fact that SeaDataNet-x.x is listed 
>separately from OceanSITES-x.x may mean it is truly independent, but it could 
>just as well be derivative, unless we explicitly preclude that usage. Stated 
>another way, do I have to list every derivative relationship in my Convention 
>Attribute? That is, if SeaDataNet profiles OceanSites which profiles CF, do I 
>have to use form (B) for a SeaDataNet Convention identifier, or are forms (A) 
>and (C) equally acceptable?

If all agree form (B) is the only acceptable form, then the profiles can 
reflect that explicitly when they define the identifier (the SeaDataNet 
profiler ID will always be of the form (B), and every time CF or OceanSITES 
updates their profile, SeaDataNet needs to double-check that no conflicts have 
been created with their profile, and perhaps put out an updated 'version 
conformance' list each time to reflect that check being successful). Validation 
software could reasonably expect to validate appropriate sequences, and reject 
invalid ones.

If we say any of these forms is OK, then I expect slash will become an 
equivalent to space over time.  Users will start using the (B) form, but get 
things out of order (which software won't have any reason to catch), and soon 
the attribute will just be treated as a list that supports multiple separators.

John










On Dec 31, 2011, at 03:50, Lowry, Roy K. wrote:

> We therefore end up with several possibilities for the Convention Attribute:
>
> CF-x.x OceanSITES x.x SeaDataNet-x.x
> CF-x.x/OceanSITES x.x/SeaDataNet-x.x
> CF-x.x/SeaDataNetx.x CF-x.x/OceanSITES x.x
>
> These have subtly different semantics.  The first says nothing about the 
> convention relationship.  The second states SeaDataNet is a profile of 
> OceanSITES which is in turn a profile of CF.  The third states that the file 
> is conformant to two independent profiles of CF.
>
> I guess the $64,000 question is whether any application program cares about 
> such subtleties and I think the answer is probably not. Most will simply 
> search for the convention required by the application within the attribute 
> string.  Therefore we should be more concerned to ensure that our convention 
> designators avoid including well-known designators - like 'CF' - as 
> substrings than with delimiters. Therefore,having information about the 
> relationship between conventions recorded within the file is useful 
> provenance metadata that could be achieved at virtually no cost.



John Graybeal   <mailto:jgrayb...@ucsd.e

Re: [CF-metadata] Convention attribute

[John Graybeal]

I wasn't sure how to parse these, I'm a little slow today I guess.  After 
trying a few ways, I decided they mostly use spaces to separate convention 
identifiers, and slashes to designate hierarchy. (Except the first two embed a 
space within "OceanSITES x.x", which I think should be a hyphen.)  

Then I finally got your point, that / (slash) is also a delimiter, but one with 
an explicit meaning.

> I guess the $64,000 question is whether any application program cares about 
> such subtleties and I think the answer is probably not. 

As of today, certainly no application program cares about such subtleties, 
since no standard or community practice exists to make use of them. But even if 
we state the question as "what is likely to be the most useful in the future, 
while being usable now?" (likely what you meant), I can't find a use case where 
the computer would use the order information, other than for display -- the 
context being of some educational value to users of the data sets, as you say.

But this leads to an implementation question, using the examples (modified with 
hyphens per the above comment): 

> (A) CF-x.x OceanSITES-x.x SeaDataNet-x.x -- 3 conventions are listed
> (B) CF-x.x/OceanSITES-x.x/SeaDataNet-x.x -- 1 conventions are listed,  
> with its relationship to two others
> (C) CF-x.x/SeaDataNet-x.x CF-x.x/OceanSITES-x.x-- 2 conventions are 
> listed, each with its relationship to a third

>From these strings, in the first and third case we don't really know either 
>way about some of the relationships -- the fact that SeaDataNet-x.x is listed 
>separately from OceanSITES-x.x may mean it is truly independent, but it could 
>just as well be derivative, unless we explicitly preclude that usage. Stated 
>another way, do I have to list every derivative relationship in my Convention 
>Attribute? That is, if SeaDataNet profiles OceanSites which profiles CF, do I 
>have to use form (B) for a SeaDataNet Convention identifier, or are forms (A) 
>and (C) equally acceptable? 

If all agree form (B) is the only acceptable form, then the profiles can 
reflect that explicitly when they define the identifier (the SeaDataNet 
profiler ID will always be of the form (B), and every time CF or OceanSITES 
updates their profile, SeaDataNet needs to double-check that no conflicts have 
been created with their profile, and perhaps put out an updated 'version 
conformance' list each time to reflect that check being successful). Validation 
software could reasonably expect to validate appropriate sequences, and reject 
invalid ones.

If we say any of these forms is OK, then I expect slash will become an 
equivalent to space over time.  Users will start using the (B) form, but get 
things out of order (which software won't have any reason to catch), and soon 
the attribute will just be treated as a list that supports multiple separators.

John







 


On Dec 31, 2011, at 03:50, Lowry, Roy K. wrote:

> We therefore end up with several possibilities for the Convention Attribute:
> 
> CF-x.x OceanSITES x.x SeaDataNet-x.x
> CF-x.x/OceanSITES x.x/SeaDataNet-x.x
> CF-x.x/SeaDataNetx.x CF-x.x/OceanSITES x.x
> 
> These have subtly different semantics.  The first says nothing about the 
> convention relationship.  The second states SeaDataNet is a profile of 
> OceanSITES which is in turn a profile of CF.  The third states that the file 
> is conformant to two independent profiles of CF.
> 
> I guess the $64,000 question is whether any application program cares about 
> such subtleties and I think the answer is probably not. Most will simply 
> search for the convention required by the application within the attribute 
> string.  Therefore we should be more concerned to ensure that our convention 
> designators avoid including well-known designators - like 'CF' - as 
> substrings than with delimiters. Therefore,having information about the 
> relationship between conventions recorded within the file is useful 
> provenance metadata that could be achieved at virtually no cost.



John Graybeal    
phone: 858-534-2162
Product Manager
Ocean Observatories Initiative Cyberinfrastructure Project: 
http://ci.oceanobservatories.org
Marine Metadata Interoperability Project: http://marinemetadata.org   

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Convention attribute

[Lowry, Roy K.]

Hi Nan,

SeaDataNet is based on a whole family of standards.  In the case of the (still 
to be finalised) NetCDF format that will be adopted by SeaDataNet one 
possibility is that it will be CF-1.6 with just a handful of additional SDN 
namespace attributes.  That to me is 'tweaking' (or profiling as I prefer to 
call it).  It has also crossed my mind that OceanSITES might be a profile of CF 
rather than a full-blown convention, but am not sufficiently familiar with the 
details to form a definite opinion.  There is also the (eminently sensible to 
me) possibility that the SeaDataNet profile might be a profile of OceanSITES 
(i.e. OceanSITES with a handful of extra attributes).

We therefore end up with several possibilities for the Convention Attribute:

CF-x.x OceanSITES x.x SeaDataNet-x.x
CF-x.x/OceanSITES x.x/SeaDataNet-x.x
CF-x.x/SeaDataNetx.x CF-x.x/OceanSITES x.x

These have subtly different semantics.  The first says nothing about the 
convention relationship.  The second states SeaDataNet is a profile of 
OceanSITES which is in turn a profile of CF.  The third states that the file is 
conformant to two independent profiles of CF.

I guess the $64,000 question is whether any application program cares about 
such subtleties and I think the answer is probably not. Most will simply search 
for the convention required by the application within the attribute string.  
Therefore we should be more concerned to ensure that our convention designators 
avoid including well-known designators - like 'CF' - as substrings than with 
delimiters. Therefore,having information about the relationship between 
conventions recorded within the file is useful provenance metadata that could 
be achieved at virtually no cost.

Cheers, Roy. 

From: cf-metadata-boun...@cgd.ucar.edu [cf-metadata-boun...@cgd.ucar.edu] On 
Behalf Of Nan Galbraith [ngalbra...@whoi.edu]
Sent: 30 December 2011 17:16
To: cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] Convention attribute

Hmmm, I GUESS it's good that someone noticed that text, and
I'm surprised how much on that page I miss every time I look at
it.  I'm not sure what the advantage is in inserting the '/'  vs using
a space, and I'm very curious if any existing software recognizes
that syntax.

It seems to me that the authors intended the '/' to be used
for specifications that are "tweaked" versions of major standards;
SeaDataNet is a full-blown standard, though, and IMHO should
be listed as a separate entity (blank separated) in this space.

What happens to data that is compliant with both SeaDataNet and
(e.g.) OceanSITES?  It's not a problem if these are each listed separately.

And, regarding the care we need for using multiple conventions, it seems
logical that the authors of standards like SDN, OceanSITES and ACDD
would be responsible for ensuring that their specifications don't conflict
with CF. This is why it's critical for the developers to provide an open
venue
for users to discuss the details of a new standard.  CF certainly provides
this kind of forum, so if we're adding a term, we can be fairly sure that
users of other standards have the opportunity to comment on possible
conflicts.

Cheers - Nan


On 12/29/11 10:14 AM, Lowry, Roy K. wrote:
> Thanks Mark,
>
> It's good to have somebody reading stuff that I should read but never have 
> the time! That certainly works for me. Providing there are no comments to the 
> contrary that would make ''CF-1.6/SeaDataNet' or maybe 
> 'CF-1.6/SeaDataNet-1.0' the value to be specified in the conventions 
> attribute for the SeaDataNet NetCDF profile that will be a part of SeaDataNet 
> II.
>
> I feel it is essential for profile documentation to be published if data 
> conforming to that profile are to be made publicly available.  Further, if 
> the data are to be exposed through INSPIRE then the way in which the profile 
> is documented should follow the ISO conventions (it's one of the ISO19xxx 
> documents, but I forget which).
>
> Cheers, Roy.
> 
> From: cf-metadata-boun...@cgd.ucar.edu [cf-metadata-boun...@cgd.ucar.edu] On 
> Behalf Of Hedley, Mark [mark.hed...@metoffice.gov.uk]
> Sent: 29 December 2011 13:27
> To: cf-metadata@cgd.ucar.edu
> Subject: Re: [CF-metadata] Convention attribute
>
> hello Roy
>
> I wonder if this could be captured by the notation in the Unidata 
> documentation:
>
> '''
> Later, if another group agrees upon some additional conventions for a 
> specific subset of XXX data, for example time series data, the description of 
> the additional conventions might be associated with the name 
> "XXX/Time_series", and files that adhered to these additional conventions 
> would use

Re: [CF-metadata] Convention attribute

[Nan Galbraith]

Hmmm, I GUESS it's good that someone noticed that text, and
I'm surprised how much on that page I miss every time I look at
it.  I'm not sure what the advantage is in inserting the '/'  vs using
a space, and I'm very curious if any existing software recognizes
that syntax.

It seems to me that the authors intended the '/' to be used
for specifications that are "tweaked" versions of major standards;
SeaDataNet is a full-blown standard, though, and IMHO should
be listed as a separate entity (blank separated) in this space.

What happens to data that is compliant with both SeaDataNet and
(e.g.) OceanSITES?  It's not a problem if these are each listed separately.

And, regarding the care we need for using multiple conventions, it seems
logical that the authors of standards like SDN, OceanSITES and ACDD
would be responsible for ensuring that their specifications don't conflict
with CF. This is why it's critical for the developers to provide an open 
venue

for users to discuss the details of a new standard.  CF certainly provides
this kind of forum, so if we're adding a term, we can be fairly sure that
users of other standards have the opportunity to comment on possible
conflicts.

Cheers - Nan


On 12/29/11 10:14 AM, Lowry, Roy K. wrote:

Thanks Mark,

It's good to have somebody reading stuff that I should read but never have the 
time! That certainly works for me. Providing there are no comments to the 
contrary that would make ''CF-1.6/SeaDataNet' or maybe 'CF-1.6/SeaDataNet-1.0' 
the value to be specified in the conventions attribute for the SeaDataNet 
NetCDF profile that will be a part of SeaDataNet II.

I feel it is essential for profile documentation to be published if data 
conforming to that profile are to be made publicly available.  Further, if the 
data are to be exposed through INSPIRE then the way in which the profile is 
documented should follow the ISO conventions (it's one of the ISO19xxx 
documents, but I forget which).

Cheers, Roy.

From: cf-metadata-boun...@cgd.ucar.edu [cf-metadata-boun...@cgd.ucar.edu] On 
Behalf Of Hedley, Mark [mark.hed...@metoffice.gov.uk]
Sent: 29 December 2011 13:27
To: cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] Convention attribute

hello Roy

I wonder if this could be captured by the notation in the Unidata documentation:

'''
Later, if another group agrees upon some additional conventions for a specific subset of 
XXX data, for example time series data, the description of the additional conventions 
might be associated with the name "XXX/Time_series", and files that adhered to 
these additional conventions would use the global attribute

 :Conventions = "XXX/Time_series" ;
'''
(http://www.unidata.ucar.edu/software/netcdf/conventions.html)


Does this provide a mechanism for profiling CF?:

:Conventions = "CF-1.6/mark'sFruityProfile"

Does the profile name "mark'sFruityProfile" get recorded somewhere?

Should this be published, or can I keep it to myself and my friends?

By the nature of the :Conventions declaration I have stated that 
"mark'sFruityProfile" is CF compliant, is that enough information?

cheers
mark

-Original Message-
From: cf-metadata-boun...@cgd.ucar.edu on behalf of Lowry, Roy K.
Sent: Thu 29/12/2011 10:08
To: Jonathan Gregory; cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] Convention attribute

Dear All,

One thought that this debate has brought to mind is what should the practice be 
if the file convention is a profile (in the ISO sense) of CF?  In other words, 
the file conforms to a given version of CF modified by a formally documented 
set of extensions (e.g. optional CF attributes declared as mandatory or 
additional attributes in the profile's namespace).  Should both the CF 
convention and the profile name be included?  My vote would be yes to avoid 
application software having to be aware of all CF profiles, but should there be 
any indication that it is a profile rather than an independent parallel 
standard?

Cheers, Roy.


From: cf-metadata-boun...@cgd.ucar.edu [cf-metadata-boun...@cgd.ucar.edu] On 
Behalf Of Jonathan Gregory [j.m.greg...@reading.ac.uk]
Sent: 28 December 2011 22:22
To: cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] Convention attribute

Dear Mark and Dave

I agree with Dave's answers. If two conventions are used together, it is the
responsibility of the data-writer to guarantee that the metadata supplied is
consistent if there are any overlaps in meaning. A particular case of that is
if the two conventions define attributes with the same names. It has been
suggested that conventions could signal their own name-spaces e.g. CF
attributes could all be prefixed with "cf_" (like the cf_role 

Re: [CF-metadata] Convention attribute

[Lowry, Roy K.]

Thanks Mark,

It's good to have somebody reading stuff that I should read but never have the 
time! That certainly works for me. Providing there are no comments to the 
contrary that would make ''CF-1.6/SeaDataNet' or maybe 'CF-1.6/SeaDataNet-1.0' 
the value to be specified in the conventions attribute for the SeaDataNet 
NetCDF profile that will be a part of SeaDataNet II.

I feel it is essential for profile documentation to be published if data 
conforming to that profile are to be made publicly available.  Further, if the 
data are to be exposed through INSPIRE then the way in which the profile is 
documented should follow the ISO conventions (it's one of the ISO19xxx 
documents, but I forget which).

Cheers, Roy. 

From: cf-metadata-boun...@cgd.ucar.edu [cf-metadata-boun...@cgd.ucar.edu] On 
Behalf Of Hedley, Mark [mark.hed...@metoffice.gov.uk]
Sent: 29 December 2011 13:27
To: cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] Convention attribute

hello Roy

I wonder if this could be captured by the notation in the Unidata documentation:

'''
Later, if another group agrees upon some additional conventions for a specific 
subset of XXX data, for example time series data, the description of the 
additional conventions might be associated with the name "XXX/Time_series", and 
files that adhered to these additional conventions would use the global 
attribute

:Conventions = "XXX/Time_series" ;
'''
(http://www.unidata.ucar.edu/software/netcdf/conventions.html)


Does this provide a mechanism for profiling CF?:

:Conventions = "CF-1.6/mark'sFruityProfile"

Does the profile name "mark'sFruityProfile" get recorded somewhere?

Should this be published, or can I keep it to myself and my friends?

By the nature of the :Conventions declaration I have stated that 
"mark'sFruityProfile" is CF compliant, is that enough information?

cheers
mark

-Original Message-
From: cf-metadata-boun...@cgd.ucar.edu on behalf of Lowry, Roy K.
Sent: Thu 29/12/2011 10:08
To: Jonathan Gregory; cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] Convention attribute

Dear All,

One thought that this debate has brought to mind is what should the practice be 
if the file convention is a profile (in the ISO sense) of CF?  In other words, 
the file conforms to a given version of CF modified by a formally documented 
set of extensions (e.g. optional CF attributes declared as mandatory or 
additional attributes in the profile's namespace).  Should both the CF 
convention and the profile name be included?  My vote would be yes to avoid 
application software having to be aware of all CF profiles, but should there be 
any indication that it is a profile rather than an independent parallel 
standard?

Cheers, Roy.


From: cf-metadata-boun...@cgd.ucar.edu [cf-metadata-boun...@cgd.ucar.edu] On 
Behalf Of Jonathan Gregory [j.m.greg...@reading.ac.uk]
Sent: 28 December 2011 22:22
To: cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] Convention attribute

Dear Mark and Dave

I agree with Dave's answers. If two conventions are used together, it is the
responsibility of the data-writer to guarantee that the metadata supplied is
consistent if there are any overlaps in meaning. A particular case of that is
if the two conventions define attributes with the same names. It has been
suggested that conventions could signal their own name-spaces e.g. CF
attributes could all be prefixed with "cf_" (like the cf_role attribute, which
has been introduced in the new CF section 9). That could help with preventing
collisions of namespaces, but

* it would be cumbersome for writers of files that adhere to only one
convention, which is the usual case, and awkward for programs that read files,
since they would have to check for every attribute by two different names
(with and without the prefix, considering all the data that already exists
without prefixes).

* it doesn't help if the two conventions are inconsistent in their metadata,
whether or not they use similarly named attributes, and this is the more
serious problem, I would argue.

Therefore I don't think this is really a magic solution to get rid of the
potential difficulty. Rather, the writers of conventions have to be aware of
other netCDF conventions that might be used with theirs, and try to use ones
that already exist instead of defining new ones for a given purpose.

Best wishes

Jonathan
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata--
This message (and any attachments) is for the recipient only. NERC
is subject to the Freedom of Information Act 2000 and the contents
of this email and any reply you make

Re: [CF-metadata] Convention attribute

[Hedley, Mark]

hello Roy

I wonder if this could be captured by the notation in the Unidata documentation:

'''
Later, if another group agrees upon some additional conventions for a specific 
subset of XXX data, for example time series data, the description of the 
additional conventions might be associated with the name "XXX/Time_series", and 
files that adhered to these additional conventions would use the global 
attribute

:Conventions = "XXX/Time_series" ;
'''
(http://www.unidata.ucar.edu/software/netcdf/conventions.html)


Does this provide a mechanism for profiling CF?:

:Conventions = "CF-1.6/mark'sFruityProfile"

Does the profile name "mark'sFruityProfile" get recorded somewhere? 

Should this be published, or can I keep it to myself and my friends?

By the nature of the :Conventions declaration I have stated that 
"mark'sFruityProfile" is CF compliant, is that enough information?

cheers
mark

-Original Message-
From: cf-metadata-boun...@cgd.ucar.edu on behalf of Lowry, Roy K.
Sent: Thu 29/12/2011 10:08
To: Jonathan Gregory; cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] Convention attribute
 
Dear All,

One thought that this debate has brought to mind is what should the practice be 
if the file convention is a profile (in the ISO sense) of CF?  In other words, 
the file conforms to a given version of CF modified by a formally documented 
set of extensions (e.g. optional CF attributes declared as mandatory or 
additional attributes in the profile's namespace).  Should both the CF 
convention and the profile name be included?  My vote would be yes to avoid 
application software having to be aware of all CF profiles, but should there be 
any indication that it is a profile rather than an independent parallel 
standard?

Cheers, Roy.


From: cf-metadata-boun...@cgd.ucar.edu [cf-metadata-boun...@cgd.ucar.edu] On 
Behalf Of Jonathan Gregory [j.m.greg...@reading.ac.uk]
Sent: 28 December 2011 22:22
To: cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] Convention attribute

Dear Mark and Dave

I agree with Dave's answers. If two conventions are used together, it is the
responsibility of the data-writer to guarantee that the metadata supplied is
consistent if there are any overlaps in meaning. A particular case of that is
if the two conventions define attributes with the same names. It has been
suggested that conventions could signal their own name-spaces e.g. CF
attributes could all be prefixed with "cf_" (like the cf_role attribute, which
has been introduced in the new CF section 9). That could help with preventing
collisions of namespaces, but

* it would be cumbersome for writers of files that adhere to only one
convention, which is the usual case, and awkward for programs that read files,
since they would have to check for every attribute by two different names
(with and without the prefix, considering all the data that already exists
without prefixes).

* it doesn't help if the two conventions are inconsistent in their metadata,
whether or not they use similarly named attributes, and this is the more
serious problem, I would argue.

Therefore I don't think this is really a magic solution to get rid of the
potential difficulty. Rather, the writers of conventions have to be aware of
other netCDF conventions that might be used with theirs, and try to use ones
that already exist instead of defining new ones for a given purpose.

Best wishes

Jonathan
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata-- 
This message (and any attachments) is for the recipient only. NERC
is subject to the Freedom of Information Act 2000 and the contents
of this email and any reply you make may be disclosed by NERC unless
it is exempt from release under the Act. Any material supplied to
NERC may be stored in an electronic records management system.
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata


___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Convention attribute

[Hedley, Mark]

I think this represents a sensible approach.  I think that the name_spacing is 
implicit and will have to remain so.

I think a strong statement detailing the responsibility placed on the data 
provider to have checked that the metadata is unambiguously consistent with all 
standards stated in the 'Conventions' global attribute would be useful 
alongside any statement that multiple conventions may be defined for a data file

cheers

mark


Dear Mark and Dave

I agree with Dave's answers. If two conventions are used together, it is the
responsibility of the data-writer to guarantee that the metadata supplied is
consistent if there are any overlaps in meaning. A particular case of that is
if the two conventions define attributes with the same names. It has been
suggested that conventions could signal their own name-spaces e.g. CF
attributes could all be prefixed with "cf_" (like the cf_role attribute, which
has been introduced in the new CF section 9). That could help with preventing
collisions of namespaces, but

* it would be cumbersome for writers of files that adhere to only one
convention, which is the usual case, and awkward for programs that read files,
since they would have to check for every attribute by two different names
(with and without the prefix, considering all the data that already exists
without prefixes).

* it doesn't help if the two conventions are inconsistent in their metadata,
whether or not they use similarly named attributes, and this is the more
serious problem, I would argue.

Therefore I don't think this is really a magic solution to get rid of the
potential difficulty. Rather, the writers of conventions have to be aware of
other netCDF conventions that might be used with theirs, and try to use ones
that already exist instead of defining new ones for a given purpose.

Best wishes

Jonathan
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Mark,

That is an important question.  I suggest that there is only one
practical interpretation for multiple conventions: 100 percent
compiance with the requirements of each listed convention.  This leads
to immediate answers for your questions, as follows.

On Wed, Dec 28, 2011 at 9:59 AM, Hedley, Mark
 wrote:
>
> I am interested in the implications for defining multiple conventions in the 
> same file.  As a data creator, what am I asserting by defining my file with 
> multiple conventions?
>
> It could be said that the conventions attribute provides an implicit name 
> space for the controlled terms it defines.  This enables a data consumer to 
> assign meaning to the terms defined by the convention which exist within a 
> particular file.
>
> If I have one convention, I can pattern match all the attributes and 
> explicitly link them to a convention.  All the ones that don't match are user 
> defined, and not part of the convention.  Does this scale to having multiple 
> conventions defined?

Yes, assuming that there are no incompatibilities between attributes
in use in any given file.

> Do conventions maintain mutually exclusive vocabularies? I don't think they 
> do.

Agreed.  Each shared term must be used in a mutually compatible way,
or else it must not be used in the context of two incompatible
definitions.

> Where vocabularies share terms, is there oversight that ensures that shared 
> terms are defined the same way?

Only good judgement on the part of convention designers.  In my
limited experience, there are a few popular conventions that have had
considerable influence on the development of other conventions.  This
helps to engender conforming usage of attributes.  Such conventions
include Unidata's attribute recommendations (mainly User's Guide
appendix B), COARDS, and CF; perhaps others.

> If I assert that two conventions are being used, does that mean that I have 
> checked that my file contains no attributes which are ambiguously defined?

Not necessarily that checks were actually performed, but only that you
are asserting full compliance with each convention.

> If I want to use a term which is ambiguously defined, can I do this 
> effectively?

By "ambiguously defined" do you mean defined incompatibly between two
conventions?  I would say No, this would not be allowed if your goal
is to have the term interpreted unambiguously by all possible readers.

> There seems to be significant potential for confusion here; I think care is 
> required.

Requiring independent conformance with each listed convention should
alleviate confusion.

--Dave
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Convention attribute

[Lowry, Roy K.]

Dear All,

One thought that this debate has brought to mind is what should the practice be 
if the file convention is a profile (in the ISO sense) of CF?  In other words, 
the file conforms to a given version of CF modified by a formally documented 
set of extensions (e.g. optional CF attributes declared as mandatory or 
additional attributes in the profile's namespace).  Should both the CF 
convention and the profile name be included?  My vote would be yes to avoid 
application software having to be aware of all CF profiles, but should there be 
any indication that it is a profile rather than an independent parallel 
standard?

Cheers, Roy.


From: cf-metadata-boun...@cgd.ucar.edu [cf-metadata-boun...@cgd.ucar.edu] On 
Behalf Of Jonathan Gregory [j.m.greg...@reading.ac.uk]
Sent: 28 December 2011 22:22
To: cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] Convention attribute

Dear Mark and Dave

I agree with Dave's answers. If two conventions are used together, it is the
responsibility of the data-writer to guarantee that the metadata supplied is
consistent if there are any overlaps in meaning. A particular case of that is
if the two conventions define attributes with the same names. It has been
suggested that conventions could signal their own name-spaces e.g. CF
attributes could all be prefixed with "cf_" (like the cf_role attribute, which
has been introduced in the new CF section 9). That could help with preventing
collisions of namespaces, but

* it would be cumbersome for writers of files that adhere to only one
convention, which is the usual case, and awkward for programs that read files,
since they would have to check for every attribute by two different names
(with and without the prefix, considering all the data that already exists
without prefixes).

* it doesn't help if the two conventions are inconsistent in their metadata,
whether or not they use similarly named attributes, and this is the more
serious problem, I would argue.

Therefore I don't think this is really a magic solution to get rid of the
potential difficulty. Rather, the writers of conventions have to be aware of
other netCDF conventions that might be used with theirs, and try to use ones
that already exist instead of defining new ones for a given purpose.

Best wishes

Jonathan
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata-- 
This message (and any attachments) is for the recipient only. NERC
is subject to the Freedom of Information Act 2000 and the contents
of this email and any reply you make may be disclosed by NERC unless
it is exempt from release under the Act. Any material supplied to
NERC may be stored in an electronic records management system.
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Convention attribute

[Jonathan Gregory]

Dear Mark and Dave

I agree with Dave's answers. If two conventions are used together, it is the
responsibility of the data-writer to guarantee that the metadata supplied is
consistent if there are any overlaps in meaning. A particular case of that is
if the two conventions define attributes with the same names. It has been
suggested that conventions could signal their own name-spaces e.g. CF
attributes could all be prefixed with "cf_" (like the cf_role attribute, which
has been introduced in the new CF section 9). That could help with preventing
collisions of namespaces, but

* it would be cumbersome for writers of files that adhere to only one
convention, which is the usual case, and awkward for programs that read files,
since they would have to check for every attribute by two different names
(with and without the prefix, considering all the data that already exists
without prefixes).

* it doesn't help if the two conventions are inconsistent in their metadata,
whether or not they use similarly named attributes, and this is the more
serious problem, I would argue.

Therefore I don't think this is really a magic solution to get rid of the
potential difficulty. Rather, the writers of conventions have to be aware of
other netCDF conventions that might be used with theirs, and try to use ones
that already exist instead of defining new ones for a given purpose.

Best wishes

Jonathan
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Convention attribute

[Dave Allured]

Mark,

That is an important question.  I suggest that there is only one
practical interpretation for multiple conventions: 100 percent
compiance with the requirements of each listed convention.  This leads
to immediate answers for your questions, as follows.

On Wed, Dec 28, 2011 at 9:59 AM, Hedley, Mark
 wrote:
>
> I am interested in the implications for defining multiple conventions in the 
> same file.  As a data creator, what am I asserting by defining my file with 
> multiple conventions?
>
> It could be said that the conventions attribute provides an implicit name 
> space for the controlled terms it defines.  This enables a data consumer to 
> assign meaning to the terms defined by the convention which exist within a 
> particular file.
>
> If I have one convention, I can pattern match all the attributes and 
> explicitly link them to a convention.  All the ones that don't match are user 
> defined, and not part of the convention.  Does this scale to having multiple 
> conventions defined?

Yes, assuming that there are no incompatibilities between attributes
in use in any given file.

> Do conventions maintain mutually exclusive vocabularies? I don't think they 
> do.

Agreed.  Each shared term must be used in a mutually compatible way,
or else it must not be used in the context of two incompatible
definitions.

> Where vocabularies share terms, is there oversight that ensures that shared 
> terms are defined the same way?

Only good judgement on the part of convention designers.  In my
limited experience, there are a few popular conventions that have had
considerable influence on the development of other conventions.  This
helps to engender conforming usage of attributes.  Such conventions
include Unidata's attribute recommendations (mainly User's Guide
appendix B), COARDS, and CF; perhaps others.

> If I assert that two conventions are being used, does that mean that I have 
> checked that my file contains no attributes which are ambiguously defined?

Not necessarily that checks were actually performed, but only that you
are asserting full compliance with each convention.

> If I want to use a term which is ambiguously defined, can I do this 
> effectively?

By "ambiguously defined" do you mean defined incompatibly between two
conventions?  I would say No, this would not be allowed if your goal
is to have the term interpreted unambiguously by all possible readers.

> There seems to be significant potential for confusion here; I think care is 
> required.

Requiring independent conformance with each listed convention should
alleviate confusion.

--Dave
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Convention attribute

[John Graybeal]

Perhaps this problem can be resolved if we carefully distinguish between the 
convention *identifier* in the Conventions attribute, and the convention name 
(which can be in any other attribute, like Convention_Names or 
Metadata_Conventions). For the names we could specify either attribute, or 
both, or be silent.

I'd argue that good practices suggest that the identifier should be created to 
support automatic parsing of the netCDF Conventions attribute. Such a good 
practice would eliminate spaces from the identifier, given current guidance.

In this case I'd argue the value of supporting automated and deterministic 
detection of conventions, outweighs the cost of insisting the identifier can be 
free-form text. And although if I ran the world I'd standardize on the 
attribute for the full convention names and how to delimit them, it won't 
bother me too much if by our silence we force a human reader to scan through 
the rest of the attributes to find the full names of all the applicable 
conventions.

John

As in the example I tossed out earlier, OceanSITES 1.1" but that isn't the 
complete name of the convention I'm thinking.
On Dec 22, 2011, at 14:01, Jim Biard wrote:

> Seems a bit extreme to say that a convention is not compatible with CF just 
> because it has a name with spaces in it.  Apart from the Conventions 
> attribute, a file can be completely conformant with both CF and one or more 
> other conventions.  I feel like our goal should be to maximize the ability of 
> people to indicate what conventions are in use in their files.  If we don't 
> allow this attribute to somehow contain the names of all of the conventions, 
> it will mean that other attributes will be used to contain the names of the 
> other conventions.  We will then have a hodgepodge of attributes that need to 
> be checked in order to find out what conventions are applicable.  In the 
> spirit of encouraging cooperation, let's allow this attribute definition to 
> be flexible enough to accommodate non-CF approaches to doing things.  Doesn't 
> mean we have to do it everywhere.
> 
> Grace and peace,
> 
> Jim
> 
> On 12/22/2011 4:03 PM, Jonathan Gregory wrote:
>> Dear all
>> 
>> This is certainly a lively thread! :-)
>> 
>> An array of strings would be nice but I don't think we should do that because
>> it's not compatible with the Unidata convention and it depends on the non-
>> classic netCDF model. In this case we can probably get by without it. While
>> we can't control the names of netCDF conventions in general, and it appears
>> that Unidata have not done so, CF could impose such a rule, as John suggests.
>> That is, we could state that if a convention is to be regarded as compatible
>> with CF, it must have a name with no spaces it. Then we can make the
>> Conventions attribute a blank-separated list safely. Would that be 
>> acceptable?
>> 
>> Cheers
>> 
>> Jonathan
>> ___
>> CF-metadata mailing list
>> CF-metadata@cgd.ucar.edu
>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
> 
> -- 
> Jim Biard
> 
> Government Contractor, STG Inc.
> Remote Sensing and Applications Division (RSAD)
> National Climatic Data Center
> 151 Patton Ave.
> Asheville, NC 28801-5001
> 
> jim.bi...@noaa.gov
> 828-271-4900
> 
> ___
> CF-metadata mailing list
> CF-metadata@cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata



John Graybeal    
phone: 858-534-2162
Product Manager
Ocean Observatories Initiative Cyberinfrastructure Project: 
http://ci.oceanobservatories.org
Marine Metadata Interoperability Project: http://marinemetadata.org   

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Convention attribute

[Jim Biard]

Seems a bit extreme to say that a convention is not compatible with CF 
just because it has a name with spaces in it.  Apart from the 
Conventions attribute, a file can be completely conformant with both CF 
and one or more other conventions.  I feel like our goal should be to 
maximize the ability of people to indicate what conventions are in use 
in their files.  If we don't allow this attribute to somehow contain the 
names of all of the conventions, it will mean that other attributes will 
be used to contain the names of the other conventions.  We will then 
have a hodgepodge of attributes that need to be checked in order to find 
out what conventions are applicable.  In the spirit of encouraging 
cooperation, let's allow this attribute definition to be flexible enough 
to accommodate non-CF approaches to doing things.  Doesn't mean we have 
to do it everywhere.


Grace and peace,

Jim

On 12/22/2011 4:03 PM, Jonathan Gregory wrote:

Dear all

This is certainly a lively thread! :-)

An array of strings would be nice but I don't think we should do that because
it's not compatible with the Unidata convention and it depends on the non-
classic netCDF model. In this case we can probably get by without it. While
we can't control the names of netCDF conventions in general, and it appears
that Unidata have not done so, CF could impose such a rule, as John suggests.
That is, we could state that if a convention is to be regarded as compatible
with CF, it must have a name with no spaces it. Then we can make the
Conventions attribute a blank-separated list safely. Would that be acceptable?

Cheers

Jonathan
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata


--
Jim Biard

Government Contractor, STG Inc.
Remote Sensing and Applications Division (RSAD)
National Climatic Data Center
151 Patton Ave.
Asheville, NC 28801-5001

jim.bi...@noaa.gov
828-271-4900

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Convention attribute

[Jonathan Gregory]

Dear all

This is certainly a lively thread! :-)

An array of strings would be nice but I don't think we should do that because
it's not compatible with the Unidata convention and it depends on the non-
classic netCDF model. In this case we can probably get by without it. While
we can't control the names of netCDF conventions in general, and it appears
that Unidata have not done so, CF could impose such a rule, as John suggests.
That is, we could state that if a convention is to be regarded as compatible
with CF, it must have a name with no spaces it. Then we can make the
Conventions attribute a blank-separated list safely. Would that be acceptable?

Cheers

Jonathan
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Convention attribute

[Mike Grant]

On 22/12/11 18:00, John Graybeal wrote:
> So my vote
> is to create a truly machine-friendly way of handling convention
> identifiers and clearly spell it out in the standard.

Sounds like an array of strings if you're really going for a clean
solution ;)

Delimiters (and anything embedding information in the data stream) will
always cause parsing problems.

Cheers,

Mike.




Plymouth Marine Laboratory
Registered Office: 
Prospect Place
The Hoe
Plymouth  PL1 3DH


Website: http://www.pml.ac.uk";>www.pml.ac.uk

http://www.pml.ac.uk/pdf/PMLAR2010.pdf";>Click here for PML Annual 
Review

Registered Charity No. 1091222
PML is a company limited by guarantee
registered in England & Wales
company number 4178503

Please think before you 
print.



This e-mail, its content and any file 
attachments are confidential.

If you have received this e-mail in error please 
do not copy, disclose it to any third party or use the contents or attachments 
in any way. Please notify the sender by replying to this e-mail or e-mail 
fori...@pml.ac.uk and then delete the email without making any copies or using 
it in any other way.

The content of this message may contain personal 
views which are not the views of Plymouth Marine Laboratory unless specifically 
stated.

You are reminded that e-mail communications are 
not secure and may contain viruses. Plymouth Marine Laboratory accepts no 
liability for any loss or damage which may be caused by viruses.





___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Convention attribute

[John Graybeal]

I'm confused by several comments on this thread.

If netCDF has always been explicit about space-delimited convention 
identifiers, what explains the existence of a convention identifier with a 
space in it (Conventions = "OceanSITES 1.1, CF-1.1")?Does netCDF have a 
review process for netCDF convention identifiers, such that it can and will 
enforce proper convention naming (or really identifier naming)?  If not, is 
there an explicit statement about the rule for not including certain characters 
in the identifier?

If there is a convention identifier with a space in it, how will 
space-delimited parsing produce the desired result? It will look like two 
conventions. This does not seem simple and machine-parseable to me.

As use of netCDF gets  more sophisticated and mature, it becomes likely that 
conventions will come in batches (with extensions and/or versions), so 
anticipating -- allowing -- the scenario of one convention identifier being a 
substring of another seems forward-looking and useful (and ruling it out would 
be unfortunate, not that anyone is arguing for that).

Note that simply replacing the space delimiter with comma doesn't fix the 
problem unless the above principles are addressed. There has to be a rule that 
either precludes the delimiter in the identifier, or allows for it to be 
escaped.  To become a true netCDF convention, the netCDF folks will need to 
ensure a few rules like that are followed. Maybe to become a "CF-compatible 
netCDF convention", the CF folks will need to vet the convention for conflicts.

I do think a convention for managing multiple conventions is essential -- data 
will be passed around more every year, and software will become smarter about 
how to handle it. The applications and cyberinfrastructure systems being built 
now will use whatever information they can to provide a better user experience. 
 Being such a strong part of the 'computable ecosystem', netCDF plays an 
important role in enabling these better user experiences.  So my vote is to 
create a truly machine-friendly way of handling convention identifiers and 
clearly spell it out in the standard.

John




On Dec 22, 2011, at 08:56, Benno Blumenthal wrote:

> Russ (and thus core netcdf) has always been explicit about space-delimited 
> conventions, so really there shouldn't be any conventions with spaces in the 
> names.
> 
> On the other hand, we have adopted the technique of using the convention name 
> as a pattern to match against the convention attribute, so that we do not 
> care about delimiters.
> 
> This, of course, will fail if someone has a convention name that is a 
> substring of another convention name, and that convention is not a subset of 
> the convention with the longer name.
> 
> Benno
> 
> 
> 
> On Thu, Dec 22, 2011 at 10:42 AM, Jim Biard  wrote:
> It is "easier" (not by much, code-wise) to not to allow commas as delimiters, 
> but if you want to allow for machine-recognition of convention names, how are 
> you going to handle conventions that have spaces in their names?  Telling 
> everyone else to get rid of spaces isn't a practical solution, and you have 
> just created a thornier problem for coders who have to figure out which way 
> someone dealt with forbidden spaces.
> 
> 
> On 12/22/2011 10:18 AM, Nan Galbraith wrote:
> Thanks Russ, Dave(s), Jonathan and Lorenzo -
> 
> Thanks for the clarifications. I agree that it makes sense to
> require that convention names not contain spaces, and that
> it's easier (and more CF-like, hence better!) to parse space
> separated terms.
> 
> Cheers - Nan
> 
> The recommendation on the Unidata site for multiple conventions
> 
>   http://www.unidata.ucar.edu/netcdf/conventions.html
> 
> is to use spaces rather than commas:
> 
>   It is possible for a netCDF file to adhere to more than one set of
>   conventions, even when there is no inheritance relationship among the
>   conventions. In this case, the value of the `Conventions' attribute
>   may be a single text string containing a list of the convention names
>   separated by blank space (recommended) or commas (if a convention name
>   contains blanks), for example
> 
> :Conventions = "XXX YYY" ;
> 
> 
>  On Dec/22/2011 6:01 AM, Lorenzo Bigagli wrote:
> Hi all,
> 
> my opinion is to keep with the current recommendation, which better supports 
> automatic parsing and the existing conforming datasets.
> In particular, I would avoid any parsing rule for the conventions attribute, 
> keeping its syntax as simple as possible (as Jonathan points out, 
> blank-separated lists are more CF-like).
> 
> I think it makes sense to require convention identifiers not to contain 
> spaces (as usual in identifiers).
> Those conventions that have not followed Unidata recommendation may be dealt 
> with on a transitional basis (e.g. by means of specific parsing exceptions), 
> while they are aligned in a future revision.
> 
> Best wishes,
>   LB
> 
> Il giorno 22/dic/2011, alle ore 10:11, Jon

Re: [CF-metadata] Convention attribute

[Benno Blumenthal]

Russ (and thus core netcdf) has always been explicit about space-delimited
conventions, so really there shouldn't be any conventions with spaces in
the names.

On the other hand, we have adopted the technique of using the convention
name as a pattern to match against the convention attribute, so that we do
not care about delimiters.

This, of course, will fail if someone has a convention name that is a
substring of another convention name, and that convention is not a subset
of the convention with the longer name.

Benno



On Thu, Dec 22, 2011 at 10:42 AM, Jim Biard  wrote:

> It is "easier" (not by much, code-wise) to not to allow commas as
> delimiters, but if you want to allow for machine-recognition of convention
> names, how are you going to handle conventions that have spaces in their
> names?  Telling everyone else to get rid of spaces isn't a practical
> solution, and you have just created a thornier problem for coders who have
> to figure out which way someone dealt with forbidden spaces.
>
>
> On 12/22/2011 10:18 AM, Nan Galbraith wrote:
>
>> Thanks Russ, Dave(s), Jonathan and Lorenzo -
>>
>> Thanks for the clarifications. I agree that it makes sense to
>> require that convention names not contain spaces, and that
>> it's easier (and more CF-like, hence better!) to parse space
>> separated terms.
>>
>> Cheers - Nan
>>
>>  The recommendation on the Unidata site for multiple conventions
>>>
>>>   
>>> http://www.unidata.ucar.edu/**netcdf/conventions.html
>>>
>>> is to use spaces rather than commas:
>>>
>>>   It is possible for a netCDF file to adhere to more than one set of
>>>   conventions, even when there is no inheritance relationship among the
>>>   conventions. In this case, the value of the `Conventions' attribute
>>>   may be a single text string containing a list of the convention names
>>>   separated by blank space (recommended) or commas (if a convention name
>>>   contains blanks), for example
>>>
>>> :Conventions = "XXX YYY" ;
>>>
>>
>>
>>  On Dec/22/2011 6:01 AM, Lorenzo Bigagli wrote:
>>
>>> Hi all,
>>>
>>> my opinion is to keep with the current recommendation, which better
>>> supports automatic parsing and the existing conforming datasets.
>>> In particular, I would avoid any parsing rule for the conventions
>>> attribute, keeping its syntax as simple as possible (as Jonathan points
>>> out, blank-separated lists are more CF-like).
>>>
>>> I think it makes sense to require convention identifiers not to contain
>>> spaces (as usual in identifiers).
>>> Those conventions that have not followed Unidata recommendation may be
>>> dealt with on a transitional basis (e.g. by means of specific parsing
>>> exceptions), while they are aligned in a future revision.
>>>
>>> Best wishes,
>>>   LB
>>>
>>> Il giorno 22/dic/2011, alle ore 10:11, Jonathan Gregory ha scritto:
>>>
>>>  Dear all

 The existing Unidata recommendation is OK and we could incorporate it
 into
 CF but it would help to be more precise, for instance: If the
 Conventions att
 includes no commas, it is interpreted as a blank-separated list of
 conventions;
 if it contains at least one comma, it is interpreted as a
 comma-separated list.
 Blank-separated lists are more CF-like - many CF attributes use that
 syntax -
 but obviously we can't insist that other conventions don't have blanks
 in their
 names, and it would be simpler therefore to use a comma-separated list
 for
 this attribute, despite the Unidata recommendation.


>>>
>>
> --
> Jim Biard
>
> Government Contractor, STG Inc.
> Remote Sensing and Applications Division (RSAD)
> National Climatic Data Center
> 151 Patton Ave.
> Asheville, NC 28801-5001
>
> jim.bi...@noaa.gov
> 828-271-4900
>
>
> __**_
> CF-metadata mailing list
> CF-metadata@cgd.ucar.edu
> http://mailman.cgd.ucar.edu/**mailman/listinfo/cf-metadata
>



-- 
Dr. M. Benno Blumenthal  be...@iri.columbia.edu
International Research Institute for climate and society
The Earth Institute at Columbia University
Lamont Campus, Palisades NY 10964-8000   (845) 680-4450
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Convention attribute

[john caron]

On 12/22/2011 2:11 AM, Jonathan Gregory wrote:

Dear all

The existing Unidata recommendation is OK and we could incorporate it into
CF but it would help to be more precise, for instance: If the Conventions att
includes no commas, it is interpreted as a blank-separated list of conventions;
if it contains at least one comma, it is interpreted as a comma-separated list.
Blank-separated lists are more CF-like - many CF attributes use that syntax -
but obviously we can't insist that other conventions don't have blanks in their
names, and it would be simpler therefore to use a comma-separated list for
this attribute, despite the Unidata recommendation.

I see no problem with allowing multiple conventions except the important
proviso that if the file follows more that one convention it is the
responsibility of the data-writer to ensure there is no inconsistency between
the metadata following these conventions. That is, they must serve
complementary purposes. It would be impossible to check this automatically so
we have to depend on the data-writer doing it correctly.

Best wishes

Jonathan
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

I think this would solve the problem:

"If the Conventions att
includes no commas, it is interpreted as a blank-separated list of conventions;
if it contains at least one comma, it is interpreted as a comma-separated list."

I could also point out that reading software has a list of conventions it 
recognizes, so in practice one takes the result of this parsing and compares to 
a known list. also, the netcdf-4 data model allows attribute values to be a 1-d 
array of Strings.

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Convention attribute

[Jim Biard]

It is "easier" (not by much, code-wise) to not to allow commas as 
delimiters, but if you want to allow for machine-recognition of 
convention names, how are you going to handle conventions that have 
spaces in their names?  Telling everyone else to get rid of spaces isn't 
a practical solution, and you have just created a thornier problem for 
coders who have to figure out which way someone dealt with forbidden spaces.


On 12/22/2011 10:18 AM, Nan Galbraith wrote:

Thanks Russ, Dave(s), Jonathan and Lorenzo -

Thanks for the clarifications. I agree that it makes sense to
require that convention names not contain spaces, and that
it's easier (and more CF-like, hence better!) to parse space
separated terms.

Cheers - Nan


The recommendation on the Unidata site for multiple conventions

   http://www.unidata.ucar.edu/netcdf/conventions.html

is to use spaces rather than commas:

   It is possible for a netCDF file to adhere to more than one set of
   conventions, even when there is no inheritance relationship among the
   conventions. In this case, the value of the `Conventions' attribute
   may be a single text string containing a list of the convention names
   separated by blank space (recommended) or commas (if a convention 
name

   contains blanks), for example

 :Conventions = "XXX YYY" ;



  On Dec/22/2011 6:01 AM, Lorenzo Bigagli wrote:

Hi all,

my opinion is to keep with the current recommendation, which better 
supports automatic parsing and the existing conforming datasets.
In particular, I would avoid any parsing rule for the conventions 
attribute, keeping its syntax as simple as possible (as Jonathan 
points out, blank-separated lists are more CF-like).


I think it makes sense to require convention identifiers not to 
contain spaces (as usual in identifiers).
Those conventions that have not followed Unidata recommendation may 
be dealt with on a transitional basis (e.g. by means of specific 
parsing exceptions), while they are aligned in a future revision.


Best wishes,
   LB

Il giorno 22/dic/2011, alle ore 10:11, Jonathan Gregory ha scritto:


Dear all

The existing Unidata recommendation is OK and we could incorporate 
it into
CF but it would help to be more precise, for instance: If the 
Conventions att
includes no commas, it is interpreted as a blank-separated list of 
conventions;
if it contains at least one comma, it is interpreted as a 
comma-separated list.
Blank-separated lists are more CF-like - many CF attributes use that 
syntax -
but obviously we can't insist that other conventions don't have 
blanks in their
names, and it would be simpler therefore to use a comma-separated 
list for

this attribute, despite the Unidata recommendation.







--
Jim Biard

Government Contractor, STG Inc.
Remote Sensing and Applications Division (RSAD)
National Climatic Data Center
151 Patton Ave.
Asheville, NC 28801-5001

jim.bi...@noaa.gov
828-271-4900

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Convention attribute

[Nan Galbraith]

Thanks Russ, Dave(s), Jonathan and Lorenzo -

Thanks for the clarifications. I agree that it makes sense to
require that convention names not contain spaces, and that
it's easier (and more CF-like, hence better!) to parse space
separated terms.

Cheers - Nan


The recommendation on the Unidata site for multiple conventions

   http://www.unidata.ucar.edu/netcdf/conventions.html

is to use spaces rather than commas:

   It is possible for a netCDF file to adhere to more than one set of
   conventions, even when there is no inheritance relationship among the
   conventions. In this case, the value of the `Conventions' attribute
   may be a single text string containing a list of the convention names
   separated by blank space (recommended) or commas (if a convention name
   contains blanks), for example

 :Conventions = "XXX YYY" ;



  On Dec/22/2011 6:01 AM, Lorenzo Bigagli wrote:

Hi all,

my opinion is to keep with the current recommendation, which better supports 
automatic parsing and the existing conforming datasets.
In particular, I would avoid any parsing rule for the conventions attribute, 
keeping its syntax as simple as possible (as Jonathan points out, 
blank-separated lists are more CF-like).

I think it makes sense to require convention identifiers not to contain spaces 
(as usual in identifiers).
Those conventions that have not followed Unidata recommendation may be dealt 
with on a transitional basis (e.g. by means of specific parsing exceptions), 
while they are aligned in a future revision.

Best wishes,
   LB

Il giorno 22/dic/2011, alle ore 10:11, Jonathan Gregory ha scritto:


Dear all

The existing Unidata recommendation is OK and we could incorporate it into
CF but it would help to be more precise, for instance: If the Conventions att
includes no commas, it is interpreted as a blank-separated list of conventions;
if it contains at least one comma, it is interpreted as a comma-separated list.
Blank-separated lists are more CF-like - many CF attributes use that syntax -
but obviously we can't insist that other conventions don't have blanks in their
names, and it would be simpler therefore to use a comma-separated list for
this attribute, despite the Unidata recommendation.





--
***
* Nan Galbraith(508) 289-2444 *
* Upper Ocean Processes GroupMail Stop 29 *
* Woods Hole Oceanographic Institution*
* Woods Hole, MA 02543*
***



___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Convention attribute

[Lorenzo Bigagli]

Hi all,

my opinion is to keep with the current recommendation, which better supports 
automatic parsing and the existing conforming datasets.
In particular, I would avoid any parsing rule for the conventions attribute, 
keeping its syntax as simple as possible (as Jonathan points out, 
blank-separated lists are more CF-like).

I think it makes sense to require convention identifiers not to contain spaces 
(as usual in identifiers).
Those conventions that have not followed Unidata recommendation may be dealt 
with on a transitional basis (e.g. by means of specific parsing exceptions), 
while they are aligned in a future revision.

Best wishes,
  LB

Il giorno 22/dic/2011, alle ore 10:11, Jonathan Gregory ha scritto:

> Dear all
> 
> The existing Unidata recommendation is OK and we could incorporate it into
> CF but it would help to be more precise, for instance: If the Conventions att
> includes no commas, it is interpreted as a blank-separated list of 
> conventions;
> if it contains at least one comma, it is interpreted as a comma-separated 
> list.
> Blank-separated lists are more CF-like - many CF attributes use that syntax -
> but obviously we can't insist that other conventions don't have blanks in 
> their
> names, and it would be simpler therefore to use a comma-separated list for
> this attribute, despite the Unidata recommendation.
> 

---
Dott. Lorenzo Bigagli
Consiglio Nazionale delle Ricerche
Istituto di Metodologie per l'Analisi Ambientale (CNR-IMAA)

i: Area della Ricerca di Potenza, Contrada Santa Loja
   Zona Industriale, 85050 Tito Scalo (PZ), Italia
t: +39 0971 427221
f: +39 0971 427222
m: lorenzo.biga...@cnr.it

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Convention attribute

[Jonathan Gregory]

Dear all

The existing Unidata recommendation is OK and we could incorporate it into
CF but it would help to be more precise, for instance: If the Conventions att
includes no commas, it is interpreted as a blank-separated list of conventions;
if it contains at least one comma, it is interpreted as a comma-separated list.
Blank-separated lists are more CF-like - many CF attributes use that syntax -
but obviously we can't insist that other conventions don't have blanks in their
names, and it would be simpler therefore to use a comma-separated list for
this attribute, despite the Unidata recommendation.

I see no problem with allowing multiple conventions except the important
proviso that if the file follows more that one convention it is the
responsibility of the data-writer to ensure there is no inconsistency between
the metadata following these conventions. That is, they must serve
complementary purposes. It would be impossible to check this automatically so
we have to depend on the data-writer doing it correctly.

Best wishes

Jonathan
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Convention attribute

[Dave Allured]

Russ et al,

I suggest that the Unidata recommendation needs to be changed to
commas only.  I recall seeing recent cases where a convention name
includes spaces, but I have never seen a comma in such name.  Case in
point, from Nan's follow-up:

   Conventions = "OceanSITES 1.1, CF-1.1" ;

Also I agree that CF should be amended to allow more than one
convention within the Conventions attribute, comma separation only.
Does anyone see a problem with this?

--Dave A.
NOAA/PSD/CIRES

On Wed, Dec 21, 2011 at 2:46 PM, Russ Rew  wrote:
>
> The recommendation on the Unidata site for multiple conventions
>
>  http://www.unidata.ucar.edu/netcdf/conventions.html
>
> is to use spaces rather than commas:
>
>  It is possible for a netCDF file to adhere to more than one set of
>  conventions, even when there is no inheritance relationship among the
>  conventions. In this case, the value of the `Conventions' attribute
>  may be a single text string containing a list of the convention names
>  separated by blank space (recommended) or commas (if a convention name
>  contains blanks), for example
>
>    :Conventions = "XXX YYY" ;
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Convention attribute

[Nan Galbraith]

  
  
Ah, thanks, I guess that's another solution. 

In OceanSITES, we have added ours to the main Conventions 
attribute, so we have

Conventions = "OceanSITES 1.1, CF-1.1" ;

which might not be ideal... I was fairly sure we'd discussed it 
on this list, but maybe not.

Thanks - Nan


On Dec/21/2011 4:16 PM, Dave Neufeld wrote:

  
  Hi Nan,
  
  Within ACDD the guidance is to use: 
  
  Metadata_Conventions = "Unidata Dataset Discovery v1.0"; 
  
  This is documented at the top of the Unidata page but not within
  the listed HTML tables which I think makes it easy to overlook.
  
  http://www.unidata.ucar.edu/software/netcdf-java/formats/DataDiscoveryAttConvention.html
  
  -Dave
  
  On 12/21/2011 1:53 PM, Nan Galbraith wrote:
  

Hi all - 

I'm certain that this has been discussed, but I can't
find it anywhere in the email archive, or on the trac site.

Don't we allow a compound (comma separated) string
in the global attribute 'Conventions'? 

Because there are new, complimentary  conventions 
available - like the ACDD (discovery attributes) -  it would
be very handy to allow 
  multiple
  conventions to be specified

in a single attribute: "CF-1.6,
  ACDD, OceanSITESv2.1.2".

Here's the text from the current version of the cf-conventions
document:

> 2.6.1. Identification of
  Conventions
  > > We recommend that netCDF files that follow these
  conventions
  > indicate this by setting the NUG defined global attribute
  Conventions
  > to the string value "CF-1.6". The string is interpreted
  as a
  > directory name relative to a directory that is a
  repository
  of
  > documents describing sets of discipline-specific
  conventions.

I'm not sure about the use of the directory structure in the 
quoted section of the standard; is that actually used by a
lot of software? Would a separate attribute providing the
URL for the convention be more flexible/useful?

Thanks - 
Nan
 
-- 
***
* Nan Galbraith(508) 289-2444 *
* Upper Ocean Processes GroupMail Stop 29 *
* Woods Hole Oceanographic Institution*
* Woods Hole, MA 02543*
***



  
  
  
  
  
  This body part will be downloaded on demand.



-- 
***
* Nan Galbraith(508) 289-2444 *
* Upper Ocean Processes GroupMail Stop 29 *
* Woods Hole Oceanographic Institution*
* Woods Hole, MA 02543*
***



  


___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Convention attribute

[Russ Rew]

Hi Nan,

> I'm certain that this has been discussed, but I can't
> find it anywhere in the email archive, or on the trac site.
> 
> Don't we allow a compound (comma separated) string
> in the global attribute 'Conventions'?
> 
> Because there are new, complimentary  conventions
> available - like the ACDD (discovery attributes) -  it would
> be very handy to allow multiple conventions to be specified
> in a single attribute: "CF-1.6, ACDD, OceanSITESv2.1.2".

The recommendation on the Unidata site for multiple conventions

  http://www.unidata.ucar.edu/netcdf/conventions.html

is to use spaces rather than commas:

  It is possible for a netCDF file to adhere to more than one set of
  conventions, even when there is no inheritance relationship among the
  conventions. In this case, the value of the `Conventions' attribute
  may be a single text string containing a list of the convention names
  separated by blank space (recommended) or commas (if a convention name
  contains blanks), for example

:Conventions = "XXX YYY" ;

> Here's the text from the current version of the cf-conventions
> document:
> 
> > 2.6.1. Identification of Conventions > > We recommend that netCDF files 
> > that follow these
> conventions > indicate this by setting the NUG defined global attribute 
> Conventions > to the string
> value "CF-1.6". The string is interpreted as a > directory name relative to a 
> directory that is a
> repository of > documents describing sets of discipline-specific conventions.
> 
> I'm not sure about the use of the directory structure in the
> quoted section of the standard; is that actually used by a
> lot of software? Would a separate attribute providing the
> URL for the convention be more flexible/useful?

We agree and Unidata Conventions web page recommends what you have
described:

  ... Originally, these conventions were named by a string that was
  interpreted as a directory name relative to the directory
  /pub/netcdf/Conventions/ on the host ftp.unidata.ucar.edu.

  This web page is now the preferred and authoritative location for
  registering a URI reference to a set of conventions maintained
  elsewhere. The FTP site will be preserved for compatibility with
  existing references, but authors of new conventions should submit a
  request to support-net...@unidata.ucar.edu for listing on this page.

--Russ
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Convention attribute

[Dave Neufeld]

Hi Nan,

Within ACDD the guidance is to use:

Metadata_Conventions = "Unidata Dataset Discovery v1.0";

This is documented at the top of the Unidata page but not within the 
listed HTML tables which I think makes it easy to overlook.


http://www.unidata.ucar.edu/software/netcdf-java/formats/DataDiscoveryAttConvention.html

-Dave

On 12/21/2011 1:53 PM, Nan Galbraith wrote:

Hi all -

I'm certain that this has been discussed, but I can't
find it anywhere in the email archive, or on the trac site.

Don't we allow a compound (comma separated) string
in the global attribute 'Conventions'?

Because there are new, complimentary  conventions
available - like the ACDD (discovery attributes) -  it would
be very handy to allow multiple
conventions to be specified

in a single attribute: "CF-1.6,
ACDD, OceanSITESv2.1.2".

Here's the text from the current version of the cf-conventions
document:

> 2.6.1. Identification of
Conventions
> > We recommend that netCDF files that follow these
conventions
> indicate this by setting the NUG defined global attribute
Conventions
> to the string value "CF-1.6". The string is interpreted as a
> directory name relative to a directory that is a repository
of
> documents describing sets of discipline-specific conventions.

I'm not sure about the use of the directory structure in the
quoted section of the standard; is that actually used by a
lot of software? Would a separate attribute providing the
URL for the convention be more flexible/useful?

Thanks -
Nan

--
***
* Nan Galbraith(508) 289-2444 *
* Upper Ocean Processes GroupMail Stop 29 *
* Woods Hole Oceanographic Institution*
* Woods Hole, MA 02543*
***




___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Convention attribute

[Nan Galbraith]

  
  
Hi all - 

I'm certain that this has been discussed, but I can't
find it anywhere in the email archive, or on the trac site.

Don't we allow a compound (comma separated) string
in the global attribute 'Conventions'? 

Because there are new, complimentary  conventions 
available - like the ACDD (discovery attributes) -  it would
be very handy to allow  multiple
  conventions to be specified

in a single attribute: "CF-1.6,
  ACDD, OceanSITESv2.1.2".

Here's the text from the current version of the cf-conventions
document:

> 2.6.1. Identification of
  Conventions
  > > We recommend that netCDF files that follow these
  conventions
  > indicate this by setting the NUG defined global attribute
  Conventions
  > to the string value "CF-1.6". The string is interpreted as a
  > directory name relative to a directory that is a repository
  of
  > documents describing sets of discipline-specific conventions.

I'm not sure about the use of the directory structure in the 
quoted section of the standard; is that actually used by a
lot of software? Would a separate attribute providing the
URL for the convention be more flexible/useful?

Thanks - 
Nan
 
-- 
***
* Nan Galbraith(508) 289-2444 *
* Upper Ocean Processes GroupMail Stop 29 *
* Woods Hole Oceanographic Institution*
* Woods Hole, MA 02543*
***



  


___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata