Re: [CF-metadata] the need to store lat/lon coordinates in a CF-compliant netCDF file
On 8/3/2011 8:19 AM, Jon Blower wrote: Hi all, I've been following this thread with great interest. For me it boils down to this question: - Is the datum always known by the data provider? If the answer is yes then I see no reason to omit the datum (and plenty of reasons to include it). If there *are* situations where the datum is genuinely unknown or undefined then we need to express this clearly too so users can beware. (Previous posts mentioned GCMs as possible situations where the datum is unknown - is this really true? Surely there is always a sphere with a known radius on which coordinates are based?) Cheers, Jon -- Dr Jon Blower Technical Director, Reading e-Science Centre Environmental Systems Science Centre University of Reading, UK Tel: +44 (0)118 378 5213 http://www.resc.reading.ac.uk ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata Just to add to the above, there are 2 issues we are discussing: 1) possible recommendations by CF to always include the datum, and making sure we have the right metadata to do so. 2) possible recommendations by CF as to what to do if the datum is not present, or is only partially specified. John ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
[CF-metadata] [FRANCE] Question about CF Convention
Hi all, I am new on this mailing list and I hope my question is not so obvious My problem is with: At the adress : http://cf-pcmdi.llnl.gov/conformance/requirements-and-recommendations/1.5/ I find « 2.1 Filename Requirements: * Filename must have .nc suffix. But If I read the CF documentation http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/cf-conventions-multi.html I find 2.1. Filename NetCDF files should have the file name extension .nc. For me there is a difference between must and should?. If i have another suffix than nc, can i be CF compliant? Best regards Alain MAZURIER Responsable Unité Technologique Logicielle Technopôle Brest Iroise Site du Vernis - CS 23866 29238 Brest Cedex 3 Tél : + 33 2 98 05 43 21 Tél direct: + 33 2 98 05 71 90 Mobile : 06 08 92 10 11 Fax : + 33 2 98 05 20 34 www.altran.com http://www.altran.com P Before printing, think about ENVIRONMENTAL responsibility image001.jpg___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] the need to store lat/lon coordinates in a CF-compliant netCDF file
Correction to the datum/ellipsoid explanation below. The NAD83 datum does use the GRS80 ellipsoid. The NAD27 datum uses the Clarke 1866 ellipsoid. Woops! Dave On Aug 3, 2011, at 9:07 AM, David Blodgett wrote: Dear Phil, I would have thought that if a particular piece of data analysis is at a resolution that requires a geodetic datum to be specified then, The problem here is that there is no analysis that does not require this information. While some choose to assume that all data uses the same datum and drop those terms of analysis out of their math, that does not mean the information is not there. giving the user the opportunity to select the one s/he believes to be the most appropriate for the task in hand. Having personally been in this situation, it took me two years to track down exactly the right geographic transformation to apply to accurately apply radar data to the landscape. It is unfair to expect that a terrestrial modeler understand the handling of geographic data in climate and forecasting applications to such an extent that they are comfortable making such a decision. Your argument about darwinian evolution of data use would cause a massive set back in interdisciplinary science. This one minor inclusion of necessary metadata would allow a broad community of users to more easily leverage data using the climate and forecasting metadata conventions. Since the CF community does disregard datum metadata, they could continue to silo themselves from the rest of the environmental modeling community. Or, they could extend a bit of an olive branch and recognize that this information is critical and required for most terrestrial applications that consume atmospheric data. Regarding the difference between GRS 80 and WGS 84 ellipsoids, they are different by small fractions of a meter. From wikipedia: The very small difference in the flattening thus results in a—very theoretical—difference of 0.105 mm in the semi polar axis. For most purposes, the differing polar axes can be merged to 6 356 752.3 m, with the inverse flattening rounded to 298.257. You are referring to the NAD83 Datum which uses the Clarke 1866 ellipsoid which does fit the continental united states better than the GRS80/WGS84 ellipsoid. Cheers, Dave B On Aug 3, 2011, at 8:48 AM, Bentley, Philip wrote: Dear Heiko, I hope CF could define a default datum, e.g. the GRS1980 Authalic Sphere, since this matches most closely with existing software (netcdf-java). This would make live easier for the software-developers who have to use something if nothing is given. I'm not sure that defining a default datum for CF is the right way to go in this instance. I would have thought that if a particular piece of data analysis is at a resolution that requires a geodetic datum to be specified then, in absentia the actual one being defined in metadata, it's not clear to me that using some semi-arbitrary, and potentially invalid, default datum is any better than giving the user the opportunity to select the one s/he believes to be the most appropriate for the task in hand. The current CF conventions include a (fairly minimal) set of metadata attributes which can be used to describe the basic properties of the coordinate reference system associated with a given dataset. The onus then is on data producers to utilise those metadata attributes to describe their data to the fullest extent possible. Furthermore, other non-CF attributes may be used to augment the standard set - over time some of these additional attributes would no doubt find their way into the CF specification. Ultimately, if end-users consider that a given dataset has insufficient metadata to justify its use within a particular context, then they can always choose to ignore that dataset. With the passage of time - and in true Darwinian fashion - such datasets (and their producers) will find that they are increasingly disregarded/overlooked in analyses. Hopefully this would galvanise such data producers into improving the quality of their spatial metadata! Regards, Phil PS: if a default datum were to be encoded into the CF conventions, I'd imagine that the WGS84 datum would be the way to go rather than GRS80 which, if I understand correctly, has somewhat more of a bias towards use over the North American continent. That said, I suspect the differences between the 2 datums are sufficiently small as to get lost in the underflow for many metocean research applications. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list
Re: [CF-metadata] CF-metadata Digest, Vol 100, Issue 2
Bonjour Alain, Being new to the world of meteorological/oceangraphic data myself recently, I had the same problem you have when reading the various conventions available. Based upon discussions about this matter, and how others are using and interpretating the conventions, I personally have adopted a position whereby the term should is regarded as a mandatory requirement. That is to say I interpret should as must. While I agree that the general definition of should would imply something being optional, the general definition also implies a strong sense of obligation. That is to say you must do something unless you have a good reason not to, i.e. I should not kill anyone (I use this as an example of how strongly should could be regarded as obligatory, rather than my homicidal tendencies :-) Hope this helps, though I am sure others will also comment. Salut, Glenn -- Message: 6 Date: Wed, 3 Aug 2011 16:31:11 +0200 From: Mazurier Alain alain.mazur...@altran.com Subject: [CF-metadata] [FRANCE] Question about CF Convention To: cf-metadata@cgd.ucar.edu Message-ID: ee5e0b0870aeb74bbf8c9c7bddc9c06004646...@xvs-dcfr-23.europe.corp.altran .com Content-Type: text/plain; charset=iso-8859-1 Hi all, I am new on this mailing list and I hope my question is not so obvious My problem is with: At the adress : http://cf-pcmdi.llnl.gov/conformance/requirements-and-recommendations/1. 5/ I find ? 2.1 Filename Requirements: * Filename must have .nc suffix. But If I read the CF documentation http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/cf-conventions-mul ti.html I find 2.1. Filename NetCDF files should have the file name extension .nc. For me there is a difference between must and should?. If i have another suffix than nc, can i be CF compliant? Best regards Alain MAZURIER Responsable Unit Technologique Logicielle ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] the need to store lat/lon coordinates in a CF-compliant netCDF file
OK, so this is a use case for an unknown (as opposed to a deliberately unspecified) datum I guess? Almost certainly, but by a comparatively small number of individuals within an organisation I would guess. All the more reason to encourage people to encode it in NetCDF files! ;-) Jon -Original Message- From: Bentley, Philip [mailto:philip.bent...@metoffice.gov.uk] Sent: 03 August 2011 16:18 To: Jon Blower; cf-metadata@cgd.ucar.edu Subject: RE: [CF-metadata] the need to store lat/lon coordinates in a CF-compliant netCDF file Hi Jon, I've been following this thread with great interest. For me it boils down to this question: - Is the datum always known by the data provider? Almost certainly, but by a comparatively small number of individuals within an organisation I would guess. However - and here's the blocker - probably not by many/enough of the people who have the (let's be frank) unglamorous job of compiling metadata for large volumes of spatial data. If the answer is yes then I see no reason to omit the datum (and plenty of reasons to include it). If there *are* situations where the datum is genuinely unknown or undefined then we need to express this clearly too so users can beware. (Previous posts mentioned GCMs as possible situations where the datum is unknown - is this really true? Surely there is always a sphere with a known radius on which coordinates are based?) That would be my assumption also - that the raw facts are always known. In my experience the problem tends to be organisational rather than technical. Few organisations seem to employ staff whose lapel badge reads 'Spatial Metadata Operative'. Which is, I realise, a pretty lame excuse. Anyhows, I'm drifting off topic. Cheers, Phil ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
[CF-metadata] [FRANCE] Question about CF Convention
Alain, Further to my earlier comment, and to address your question regarding file name suffix, my company makes data available via THREDDS and ERDDAP neither of which I have found has any problem using files that do not have a .nc suffix. When editing the metadata for files to be published via THREDDS and ERDDAP the use of other file suffix, such as .grb for GRIB files, has not caused any issues or stopped us from state conformance with conventions such as CF. Regards, Glenn -Original Message- From: Comiskey, Glenn Sent: 03 August 2011 16:19 To: 'cf-metadata@cgd.ucar.edu' Cc: 'alain.mazur...@altran.com' Subject: RE: CF-metadata Digest, Vol 100, Issue 2 Bonjour Alain, Being new to the world of meteorological/oceangraphic data myself recently, I had the same problem you have when reading the various conventions available. Based upon discussions about this matter, and how others are using and interpretating the conventions, I personally have adopted a position whereby the term should is regarded as a mandatory requirement. That is to say I interpret should as must. While I agree that the general definition of should would imply something being optional, the general definition also implies a strong sense of obligation. That is to say you must do something unless you have a good reason not to, i.e. I should not kill anyone (I use this as an example of how strongly should could be regarded as obligatory, rather than my homicidal tendencies :-) Hope this helps, though I am sure others will also comment. Salut, Glenn -- Message: 6 Date: Wed, 3 Aug 2011 16:31:11 +0200 From: Mazurier Alain alain.mazur...@altran.com Subject: [CF-metadata] [FRANCE] Question about CF Convention To: cf-metadata@cgd.ucar.edu Message-ID: ee5e0b0870aeb74bbf8c9c7bddc9c06004646...@xvs-dcfr-23.europe.corp.altran .com Content-Type: text/plain; charset=iso-8859-1 Hi all, I am new on this mailing list and I hope my question is not so obvious My problem is with: At the adress : http://cf-pcmdi.llnl.gov/conformance/requirements-and-recommendations/1. 5/ I find ? 2.1 Filename Requirements: * Filename must have .nc suffix. But If I read the CF documentation http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/cf-conventions-mul ti.html I find 2.1. Filename NetCDF files should have the file name extension .nc. For me there is a difference between must and should?. If i have another suffix than nc, can i be CF compliant? Best regards Alain MAZURIER Responsable Unit Technologique Logicielle ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] [FRANCE] Question about CF Convention
Glenn Thanks for your comments. The nc suffix was an example, but I found other cases in the documentation. I found a lot of differences between the convention and the list of requirements and recommendations. My work is to highlight the differences between a netcdf file (bathymetric data) and the CF convention And what is the way to make these files CF compliant... Best regards Alain MAZURIER Responsable Unité Technologique Logicielle Technopôle Brest Iroise Site du Vernis – CS 23866 29238 Brest Cedex 3 Tél : + 33 2 98 05 43 21 Tél direct: + 33 2 98 05 71 90 Mobile : 06 08 92 10 11 Fax : + 33 2 98 05 20 34 www.altran.com Before printing, think about ENVIRONMENTAL responsibility -Message d'origine- De : Comiskey, Glenn [mailto:g.comis...@geos.com] Envoyé : mercredi 3 août 2011 17:48 À : cf-metadata@cgd.ucar.edu Cc : Mazurier Alain Objet : [FRANCE] Question about CF Convention Alain, Further to my earlier comment, and to address your question regarding file name suffix, my company makes data available via THREDDS and ERDDAP neither of which I have found has any problem using files that do not have a .nc suffix. When editing the metadata for files to be published via THREDDS and ERDDAP the use of other file suffix, such as .grb for GRIB files, has not caused any issues or stopped us from state conformance with conventions such as CF. Regards, Glenn -Original Message- From: Comiskey, Glenn Sent: 03 August 2011 16:19 To: 'cf-metadata@cgd.ucar.edu' Cc: 'alain.mazur...@altran.com' Subject: RE: CF-metadata Digest, Vol 100, Issue 2 Bonjour Alain, Being new to the world of meteorological/oceangraphic data myself recently, I had the same problem you have when reading the various conventions available. Based upon discussions about this matter, and how others are using and interpretating the conventions, I personally have adopted a position whereby the term should is regarded as a mandatory requirement. That is to say I interpret should as must. While I agree that the general definition of should would imply something being optional, the general definition also implies a strong sense of obligation. That is to say you must do something unless you have a good reason not to, i.e. I should not kill anyone (I use this as an example of how strongly should could be regarded as obligatory, rather than my homicidal tendencies :-) Hope this helps, though I am sure others will also comment. Salut, Glenn -- Message: 6 Date: Wed, 3 Aug 2011 16:31:11 +0200 From: Mazurier Alain alain.mazur...@altran.com Subject: [CF-metadata] [FRANCE] Question about CF Convention To: cf-metadata@cgd.ucar.edu Message-ID: ee5e0b0870aeb74bbf8c9c7bddc9c06004646...@xvs-dcfr-23.europe.corp.altran .com Content-Type: text/plain; charset=iso-8859-1 Hi all, I am new on this mailing list and I hope my question is not so obvious My problem is with: At the adress : http://cf-pcmdi.llnl.gov/conformance/requirements-and-recommendations/1. 5/ I find ? 2.1 Filename Requirements: * Filename must have .nc suffix. But If I read the CF documentation http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/cf-conventions-mul ti.html I find 2.1. Filename NetCDF files should have the file name extension .nc. For me there is a difference between must and should?. If i have another suffix than nc, can i be CF compliant? Best regards Alain MAZURIER Responsable Unit Technologique Logicielle ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] the need to store lat/lon coordinates in a CF-compliant netCDF file
Hi, - Original Message - From: Bentley, Philip philip.bent...@metoffice.gov.uk Date: Wednesday, August 3, 2011 11:17 am Subject: Re: [CF-metadata] the need to store lat/lon coordinates in a CF-compliant netCDF file Hi Jon, I've been following this thread with great interest. For me it boils down to this question: - Is the datum always known by the data provider? Tthe World Ocean Database ( http://www.nodc.noaa.gov/OC5/WOD09/pr_wod09.html ), which is creating by aggregating and standardizing datasets collected over a long period of time, has some datasets which go all the way back to the time of James Cook. Some of these datasets, I am certain, have datum which is not known. But, I think the provider could still give a best guess for what the datum could have been with adequate qualification instead of leaving it to the user to select arbitrary datum. Upendra Almost certainly, but by a comparatively small number of individuals within an organisation I would guess. However - and here's the blocker - probably not by many/enough of the people who have the (let's be frank)unglamorous job of compiling metadata for large volumes of spatial data. If the answer is yes then I see no reason to omit the datum (and plenty of reasons to include it). If there *are* situations where the datum is genuinely unknown or undefined then we need to express this clearly too so users can beware. (Previous posts mentioned GCMs as possible situations where the datum is unknown - is this really true? Surely there is always a sphere with a known radius on which coordinates are based?) That would be my assumption also - that the raw facts are always known. In my experience the problem tends to be organisational rather than technical. Few organisations seem to employ staff whose lapel badge reads 'Spatial Metadata Operative'. Which is, I realise, a pretty lame excuse. Anyhows, I'm drifting off topic. Cheers, Phil ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
