Re: [CF-metadata] standard_name modifiers
Dear Christina, As this debate unfolds I am coming to the realisation that there might be a significant difference between what you mean by 'chlorophyll count' (the unmodified reading from a fluorometer analogue-to-digital converter that is a function of chlorophyll concentration) and what CF understands by 'chlorophyll count' (the number of individual measurements used to determine a chlorophyll value). Am I correct? If so, apologies for not having realised this sooner. Cheers, Roy. -Original Message- From: cf-metadata-boun...@cgd.ucar.edu [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Jonathan Gregory Sent: 24 February 2011 18:08 To: Schultz, Martin Cc: cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] standard_name modifiers Dear Martin The idea of the modifiers was to provide standard names for ancillary data, such as count of obs, standard error, and so on. The other kinds of thing you mention, such as means over periods and other statistics, can often be described by cell_methods, which is more flexible because it refers to the dimensions of the data. Best wishes Jonathan ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- This message (and any attachments) is for the recipient only NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] standard_name modifiers
Dear John, there is a distinct difference between cell_methods and the kind of operators I am talking about (or I misunderstand the cell_methods description): cell_methods describe operations that are done with respect to variable dimensions (averaging over time and/or lon or lat, etc.). From the CF1.5 document: Each name: method pair indicates that for an axis identified by name, the cell values representing the field have been determined or derived by the specified method. The modifier or operator describes operations that are done to the variable values themselves without changing the dimensions. Let me clarify: Take a global model output of temperature, for example. How do you describe temperature differences between this model and another one? The resulting quantity is still an air_temperature (OK, actually air_temperature_difference) with units of K. Yet, it would be nice to know that this field is a result of differencing two models. Difference could be accomodated relatively easily with the standard_name modifier (but how do you describe what has been differenced?). More complicated operations, such as normalized mean bias (X-Y/(X+Y)) will at some point be impossible to maintain through standard_name modifiers, I believe. Reading through the CF1.5 description of cell_methods again, I see that this is probably the way to go, but one would need to define a way of expressing such methods that are not associated with a dimension. For example, this could be done with standard_name:difference, but this might be rather clumsy (think of atmosphere_absorption_optical_thickness_due_to_particulate_organic_matter_ambient_aerosol:difference). self:difference could be another option, with additional information in paranthesis (e.g. self:difference (ERA-interim, CRU)). Writing only difference is a third option - however this complicates the syntax again, because you parse for colon in some cases but not always. Cheers, Martin -Original Message- From: Jonathan Gregory [mailto:j.m.greg...@reading.ac.uk] Sent: Thursday, February 24, 2011 7:08 PM To: Schultz, Martin Cc: cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] standard_name modifiers Dear Martin The idea of the modifiers was to provide standard names for ancillary data, such as count of obs, standard error, and so on. The other kinds of thing you mention, such as means over periods and other statistics, can often be described by cell_methods, which is more flexible because it refers to the dimensions of the data. Best wishes Jonathan Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Proposal for new standard names
Dear Tomoo I agree that the word sedimentation needs more clarification. (1') tendency_of_mass_fraction_of_stratiform_cloud_liquid_water_in_air_ due_to_sedimentation_of_cloud_liquid_water . In this proposal, I hope to supplement the standard names which describe the cloud budget of various GCMs, and the above option (1') is exactly what we need for this purpose (i.e., it fits the variable name in a GCM). It looks repetitive, but it appears that this clarification is necessary since you also have (2) tendency_of_mass_fraction_of_stratiform_cloud_condensed_water_in_air_ _due_to_sedimentation_of_cloud_liquid_water Can sedimentation in this context only refer to cloud liquid water, or could the same word be applied to cloud ice as well? (Please excuse my ignorance - I am not a cloud microphysicist!) Best wishes Jonathan ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] standard_name modifiers
Dear Jonathan, I don't quite agree with the implication you derive from : In general, CF metadata describes what a quantity *is* and not how it was calculated from other quantities. -- a temperature difference is a temperature, but you don't want to confuse it with a temperature (pun intended). Suppose you have this wonderful search engine that finds out that NOAA, UK Met Office, JMA, ECMWF and many others provide daily fields of sea surface temperatures. Then you load them all and compute a mean value. What happens if the NOAA values are SST anomalies rather than actual temperatures? As far as I can tell, the software would have no way of telling. OK: you can argue this is why we still need humans, but in my view it defeats the GEOSS concept of interoperability. In particular, because it should be avoidable. But perhaps a simple solution yould be to add a standard_name modifier called derived_quantity which would mean that it has similar properties (grid, units, etc.) like the ori ginal, but values should not be compared with the original. Best, Martin It would probably help if we focussed on some real use-cases where it is essential to provide *systematic* metadata i.e. which can be processed by programs. It is always possible to provide descriptive metadata, useful to humans, in non-standardised attributes such as long_name and history, and this can explain how the quantity is obtained. Best wishes Jonathan Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] standard_name modifiers
I agree that it's misleading (even to humans, who might not be as thorough as some software) to use a common standard name for a quantity that's not actually the measurement of that observable, as in your example of a temperature anomaly. The term derived_quantity doesn't seem especially helpful as a standard name modifier, though; many observed phenomena (salinity, longwave radiation) are, technically, derived quantities. In our data sets, for secondary variables like temperature differences, we wouldn't usually supply a standard name at all. That's perfectly legit in CF, and would prevent users from mistakenly using the data as a primary variable, but it doesn't help make the data discoverable. For that reason, it could be worthwhile to introduce more standard name modifiers, e.g. anomaly, even if more attributes are needed to describe the meaning of the data. Regards - Nan On 2/25/11 8:56 AM, Schultz, Martin wrote: Dear Jonathan, I don't quite agree with the implication you derive from : In general, CF metadata describes what a quantity *is* and not how it was calculated from other quantities. -- a temperature difference is a temperature, but you don't want to confuse it with a temperature (pun intended). Suppose you have this wonderful search engine that finds out that NOAA, UK Met Office, JMA, ECMWF and many others provide daily fields of sea surface temperatures. Then you load them all and compute a mean value. What happens if the NOAA values are SST anomalies rather than actual temperatures? As far as I can tell, the software would have no way of telling. OK: you can argue this is why we still need humans, but in my view it defeats the GEOSS concept of interoperability. In particular, because it should be avoidable. But perhaps a simple solution yould be to add a standard_name modifier called derived_quantity which would mean that it has similar properties (grid, units, etc.) like the ori ginal, but values should not be compared with the original. Best, Martin It would probably help if we focussed on some real use-cases where it is essential to provide *systematic* metadata i.e. which can be processed by programs. It is always possible to provide descriptive metadata, useful to humans, in non-standardised attributes such as long_name and history, and this can explain how the quantity is obtained. Best wishes Jonathan -- *** * Nan Galbraith(508) 289-2444 * * Upper Ocean Processes GroupMail Stop 29 * * Woods Hole Oceanographic Institution* * Woods Hole, MA 02543* *** ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] CF and ISO19115
On 2/24/2011 8:06 AM, Ted Habermann wrote: Hello all, There was some recent discussion about challenges related to adding ISO metadata to netCDF files. Rich Signell mentioned some work my group has been doing recently and John Graybeal pointed out that initially that work has focused on the NcML ISO side of the equation. We have also focused on the Data Discovery Conventions more than the more use oriented CF-Conventions. As Martin Schultz pointed out initially, the problem is meshing the well developed ISO structure with the parameter-value tradition of netCDF. The current approach I am experimenting with involves creating components of ISO documentation that are identified by UUID's and embedding those UUID's as values for attributes with standard names in the netCDF, possibly in a special group called ISO_Metadata (whatever). The names of these attributes would indicate their location in the full ISO record and they would be inserted into the record during a translation step. This would create a record with references that could then be resolved at the users request. There is a discussion of this general approach to the ISO at https://geo-ide.noaa.gov/wiki/index.php?title=ISO_Components. It may be a step in the right direction...? Hi Ted, I think this is definitely a step in the right direction. By its nature metadata records need in to have a level of independence from the data files, themselves. The metadata must evolve, as knowledge of and documentation of a dataset continues to grow after the dataset has been published. My personal 2 cents: embedding use metadata and some discovery metadata feels like the right level of detail to include in the netCDF file, itself ... augmented by the linkages you are proposing, which would connect the data file to full-detail (ISO), evolving metadata records. When it is ready, I hope you'll send a specific proposal on how to achieve these linkages for comments/discussion on this list. - Steve BTW, there are also some revisions to 19115 that are in the pipeline and will help with the CF-ISO translation. These are described at https://geo-ide.noaa.gov/wiki/index.php?title=Coverages_and_ISO_Metadata Ted Habermann ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] standard_name modifiers
Dear Martin The case of anomaly is a good use case. It could be indicated by a standard name modifier, as you say, but I agree with Nan that it should be a specific one i.e. anomaly rather than generic. However in that case we have so far been adding new standard_names instead of using a modifier, viz air_pressure_anomaly air_temperature_anomaly geopotential_height_anomaly surface_temperature_anomaly As you see, we have not so far been flooded with requests for these, even though in principle it could be relevant for any quantity. Following the usual pragmatic approach, I would argue that we don't yet need a general solution for this particular case; we can just add more standard names if you need some. Best wishes and happy weekend Jonathan ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Proposal for new standard names
-Original Message- From: cf-metadata-boun...@cgd.ucar.edu [mailto:cf-metadata- boun...@cgd.ucar.edu] On Behalf Of Jonathan Gregory Sent: Friday, February 25, 2011 5:25 AM To: Tomoo Ogura Cc: Jennifer Kay; Yoko Tsushima; cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] Proposal for new standard names Dear Tomoo I agree that the word sedimentation needs more clarification. (1') tendency_of_mass_fraction_of_stratiform_cloud_liquid_water_in_air_ due_to_sedimentation_of_cloud_liquid_water . In this proposal, I hope to supplement the standard names which describe the cloud budget of various GCMs, and the above option (1') is exactly what we need for this purpose (i.e., it fits the variable name in a GCM). It looks repetitive, but it appears that this clarification is necessary since you also have (2) tendency_of_mass_fraction_of_stratiform_cloud_condensed_water_in_air_ _due_to_sedimentation_of_cloud_liquid_water Can sedimentation in this context only refer to cloud liquid water, or could the same word be applied to cloud ice as well? (Please excuse my ignorance - I am not a cloud microphysicist!) Best wishes Jonathan Hi Jonathan, I think one could apply 'sedimentation' to each of aerosols, liquid_water, and ice. That said, I would usually use 'precipitation' instead of 'sedimentation' for liquid_water and ice, and 'sedimentation' for aerosols. I'm not sure if there is a formal distinction between 'precipitation' and 'sedimentation' for the atmosphere. The only distinction I can think of is that 'precipitation' makes me think of *fast* fall speeds and 'sedimentation' makes me think of *slow* fall speeds. I do note that 'sedimentation' is currently only used in CF for ocean names (eg, tendency_of_ocean_mole_content_of_carbon_due_to_sedimentation), whereas 'precipitation' is already used for atmospheric terms (eg, tendency_of_specific_humidity_due_to_stratiform_precipitation). This suggests to me CF should use 'precipitation' in the proposal above, unless one wants to make some distinction on the basis of fall speed. However, 'precipitation' will sound odd if/when extended to aerosols, so I suggest that there be some mention in the description/help text to help with discovery (ie whether precipitation includes sedimentation, or not). Best wishes, Philip --- Dr Philip Cameron-Smith, p...@llnl.gov, Lawrence Livermore National Lab. --- ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
