Re: [CF-metadata] Convention attribute
Dear All, One thought that this debate has brought to mind is what should the practice be if the file convention is a profile (in the ISO sense) of CF? In other words, the file conforms to a given version of CF modified by a formally documented set of extensions (e.g. optional CF attributes declared as mandatory or additional attributes in the profile's namespace). Should both the CF convention and the profile name be included? My vote would be yes to avoid application software having to be aware of all CF profiles, but should there be any indication that it is a profile rather than an independent parallel standard? Cheers, Roy. From: cf-metadata-boun...@cgd.ucar.edu [cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Jonathan Gregory [j.m.greg...@reading.ac.uk] Sent: 28 December 2011 22:22 To: cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] Convention attribute Dear Mark and Dave I agree with Dave's answers. If two conventions are used together, it is the responsibility of the data-writer to guarantee that the metadata supplied is consistent if there are any overlaps in meaning. A particular case of that is if the two conventions define attributes with the same names. It has been suggested that conventions could signal their own name-spaces e.g. CF attributes could all be prefixed with "cf_" (like the cf_role attribute, which has been introduced in the new CF section 9). That could help with preventing collisions of namespaces, but * it would be cumbersome for writers of files that adhere to only one convention, which is the usual case, and awkward for programs that read files, since they would have to check for every attribute by two different names (with and without the prefix, considering all the data that already exists without prefixes). * it doesn't help if the two conventions are inconsistent in their metadata, whether or not they use similarly named attributes, and this is the more serious problem, I would argue. Therefore I don't think this is really a magic solution to get rid of the potential difficulty. Rather, the writers of conventions have to be aware of other netCDF conventions that might be used with theirs, and try to use ones that already exist instead of defining new ones for a given purpose. Best wishes Jonathan ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata-- This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Convention attribute
Thanks Mark, It's good to have somebody reading stuff that I should read but never have the time! That certainly works for me. Providing there are no comments to the contrary that would make ''CF-1.6/SeaDataNet' or maybe 'CF-1.6/SeaDataNet-1.0' the value to be specified in the conventions attribute for the SeaDataNet NetCDF profile that will be a part of SeaDataNet II. I feel it is essential for profile documentation to be published if data conforming to that profile are to be made publicly available. Further, if the data are to be exposed through INSPIRE then the way in which the profile is documented should follow the ISO conventions (it's one of the ISO19xxx documents, but I forget which). Cheers, Roy. From: cf-metadata-boun...@cgd.ucar.edu [cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Hedley, Mark [mark.hed...@metoffice.gov.uk] Sent: 29 December 2011 13:27 To: cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] Convention attribute hello Roy I wonder if this could be captured by the notation in the Unidata documentation: ''' Later, if another group agrees upon some additional conventions for a specific subset of XXX data, for example time series data, the description of the additional conventions might be associated with the name "XXX/Time_series", and files that adhered to these additional conventions would use the global attribute :Conventions = "XXX/Time_series" ; ''' (http://www.unidata.ucar.edu/software/netcdf/conventions.html) Does this provide a mechanism for profiling CF?: :Conventions = "CF-1.6/mark'sFruityProfile" Does the profile name "mark'sFruityProfile" get recorded somewhere? Should this be published, or can I keep it to myself and my friends? By the nature of the :Conventions declaration I have stated that "mark'sFruityProfile" is CF compliant, is that enough information? cheers mark -Original Message- From: cf-metadata-boun...@cgd.ucar.edu on behalf of Lowry, Roy K. Sent: Thu 29/12/2011 10:08 To: Jonathan Gregory; cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] Convention attribute Dear All, One thought that this debate has brought to mind is what should the practice be if the file convention is a profile (in the ISO sense) of CF? In other words, the file conforms to a given version of CF modified by a formally documented set of extensions (e.g. optional CF attributes declared as mandatory or additional attributes in the profile's namespace). Should both the CF convention and the profile name be included? My vote would be yes to avoid application software having to be aware of all CF profiles, but should there be any indication that it is a profile rather than an independent parallel standard? Cheers, Roy. From: cf-metadata-boun...@cgd.ucar.edu [cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Jonathan Gregory [j.m.greg...@reading.ac.uk] Sent: 28 December 2011 22:22 To: cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] Convention attribute Dear Mark and Dave I agree with Dave's answers. If two conventions are used together, it is the responsibility of the data-writer to guarantee that the metadata supplied is consistent if there are any overlaps in meaning. A particular case of that is if the two conventions define attributes with the same names. It has been suggested that conventions could signal their own name-spaces e.g. CF attributes could all be prefixed with "cf_" (like the cf_role attribute, which has been introduced in the new CF section 9). That could help with preventing collisions of namespaces, but * it would be cumbersome for writers of files that adhere to only one convention, which is the usual case, and awkward for programs that read files, since they would have to check for every attribute by two different names (with and without the prefix, considering all the data that already exists without prefixes). * it doesn't help if the two conventions are inconsistent in their metadata, whether or not they use similarly named attributes, and this is the more serious problem, I would argue. Therefore I don't think this is really a magic solution to get rid of the potential difficulty. Rather, the writers of conventions have to be aware of other netCDF conventions that might be used with theirs, and try to use ones that already exist instead of defining new ones for a given purpose. Best wishes Jonathan ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata-- This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system.
Re: [CF-metadata] Convention attribute
I think this represents a sensible approach. I think that the name_spacing is implicit and will have to remain so. I think a strong statement detailing the responsibility placed on the data provider to have checked that the metadata is unambiguously consistent with all standards stated in the 'Conventions' global attribute would be useful alongside any statement that multiple conventions may be defined for a data file cheers mark Dear Mark and Dave I agree with Dave's answers. If two conventions are used together, it is the responsibility of the data-writer to guarantee that the metadata supplied is consistent if there are any overlaps in meaning. A particular case of that is if the two conventions define attributes with the same names. It has been suggested that conventions could signal their own name-spaces e.g. CF attributes could all be prefixed with "cf_" (like the cf_role attribute, which has been introduced in the new CF section 9). That could help with preventing collisions of namespaces, but * it would be cumbersome for writers of files that adhere to only one convention, which is the usual case, and awkward for programs that read files, since they would have to check for every attribute by two different names (with and without the prefix, considering all the data that already exists without prefixes). * it doesn't help if the two conventions are inconsistent in their metadata, whether or not they use similarly named attributes, and this is the more serious problem, I would argue. Therefore I don't think this is really a magic solution to get rid of the potential difficulty. Rather, the writers of conventions have to be aware of other netCDF conventions that might be used with theirs, and try to use ones that already exist instead of defining new ones for a given purpose. Best wishes Jonathan ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata Mark, That is an important question. I suggest that there is only one practical interpretation for multiple conventions: 100 percent compiance with the requirements of each listed convention. This leads to immediate answers for your questions, as follows. On Wed, Dec 28, 2011 at 9:59 AM, Hedley, Mark wrote: > > I am interested in the implications for defining multiple conventions in the > same file. As a data creator, what am I asserting by defining my file with > multiple conventions? > > It could be said that the conventions attribute provides an implicit name > space for the controlled terms it defines. This enables a data consumer to > assign meaning to the terms defined by the convention which exist within a > particular file. > > If I have one convention, I can pattern match all the attributes and > explicitly link them to a convention. All the ones that don't match are user > defined, and not part of the convention. Does this scale to having multiple > conventions defined? Yes, assuming that there are no incompatibilities between attributes in use in any given file. > Do conventions maintain mutually exclusive vocabularies? I don't think they > do. Agreed. Each shared term must be used in a mutually compatible way, or else it must not be used in the context of two incompatible definitions. > Where vocabularies share terms, is there oversight that ensures that shared > terms are defined the same way? Only good judgement on the part of convention designers. In my limited experience, there are a few popular conventions that have had considerable influence on the development of other conventions. This helps to engender conforming usage of attributes. Such conventions include Unidata's attribute recommendations (mainly User's Guide appendix B), COARDS, and CF; perhaps others. > If I assert that two conventions are being used, does that mean that I have > checked that my file contains no attributes which are ambiguously defined? Not necessarily that checks were actually performed, but only that you are asserting full compliance with each convention. > If I want to use a term which is ambiguously defined, can I do this > effectively? By "ambiguously defined" do you mean defined incompatibly between two conventions? I would say No, this would not be allowed if your goal is to have the term interpreted unambiguously by all possible readers. > There seems to be significant potential for confusion here; I think care is > required. Requiring independent conformance with each listed convention should alleviate confusion. --Dave ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
[CF-metadata] Discrete Sampling Geometries | sample data
I am pleased to see the chapter on Discrete Sampling Geometries in the 1.6 version of CF Appendix H provides useful examples of datasets using the approach detailed in chapter 9 Is there a package of such sample data files available for software developers to use as test cases for building reading and writing tools for these data types? ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Convention attribute
hello Roy I wonder if this could be captured by the notation in the Unidata documentation: ''' Later, if another group agrees upon some additional conventions for a specific subset of XXX data, for example time series data, the description of the additional conventions might be associated with the name "XXX/Time_series", and files that adhered to these additional conventions would use the global attribute :Conventions = "XXX/Time_series" ; ''' (http://www.unidata.ucar.edu/software/netcdf/conventions.html) Does this provide a mechanism for profiling CF?: :Conventions = "CF-1.6/mark'sFruityProfile" Does the profile name "mark'sFruityProfile" get recorded somewhere? Should this be published, or can I keep it to myself and my friends? By the nature of the :Conventions declaration I have stated that "mark'sFruityProfile" is CF compliant, is that enough information? cheers mark -Original Message- From: cf-metadata-boun...@cgd.ucar.edu on behalf of Lowry, Roy K. Sent: Thu 29/12/2011 10:08 To: Jonathan Gregory; cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] Convention attribute Dear All, One thought that this debate has brought to mind is what should the practice be if the file convention is a profile (in the ISO sense) of CF? In other words, the file conforms to a given version of CF modified by a formally documented set of extensions (e.g. optional CF attributes declared as mandatory or additional attributes in the profile's namespace). Should both the CF convention and the profile name be included? My vote would be yes to avoid application software having to be aware of all CF profiles, but should there be any indication that it is a profile rather than an independent parallel standard? Cheers, Roy. From: cf-metadata-boun...@cgd.ucar.edu [cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Jonathan Gregory [j.m.greg...@reading.ac.uk] Sent: 28 December 2011 22:22 To: cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] Convention attribute Dear Mark and Dave I agree with Dave's answers. If two conventions are used together, it is the responsibility of the data-writer to guarantee that the metadata supplied is consistent if there are any overlaps in meaning. A particular case of that is if the two conventions define attributes with the same names. It has been suggested that conventions could signal their own name-spaces e.g. CF attributes could all be prefixed with "cf_" (like the cf_role attribute, which has been introduced in the new CF section 9). That could help with preventing collisions of namespaces, but * it would be cumbersome for writers of files that adhere to only one convention, which is the usual case, and awkward for programs that read files, since they would have to check for every attribute by two different names (with and without the prefix, considering all the data that already exists without prefixes). * it doesn't help if the two conventions are inconsistent in their metadata, whether or not they use similarly named attributes, and this is the more serious problem, I would argue. Therefore I don't think this is really a magic solution to get rid of the potential difficulty. Rather, the writers of conventions have to be aware of other netCDF conventions that might be used with theirs, and try to use ones that already exist instead of defining new ones for a given purpose. Best wishes Jonathan ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata-- This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
