Re: [CF-metadata] standard names for stations
Dear Nan, John, Jeff I think variable attributes are generally better than global attributes, because it's possible or indeed likely that you might have data from different sources in the same file. I prefer data variables to describe themselves. We can provide global attributes as a default for the whole file, but it seems safer to me to repeat the attributes per variable. Following John's point, I agree that this station ID information will not be usable by machine or guaranteed to be unique unless we standardise it in some way. I don't know about this subject, but I am sure there are people who are well-informed about it! If we want the station ID attribute(s) to be machine- readable, we can propose a standard format for them. If we want an attr where the data-writer can informally record ID information that another human could interpret, we don't need a standard format. Which do we want - any views? Best wishes Jonathan ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] standard names for stations
Hi Jonathan, My vote would go to something that is both human and machine-readable. There are standards designed for this such as JSON. Some might consider this comes down on the side of the machine more than the human, but I find once I get my in I can read JSON much easier than XML. If this is considered to be going over the top then we need to at least specify the order in which the component parts of the string are included and preferably we should specify a delimiter to separate the components. Even an ultra-light standard such as this makes the life of any programmer writing code to parse the string infinitely easier. Cheers, Roy. -Original Message- From: cf-metadata-boun...@cgd.ucar.edu [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Jonathan Gregory Sent: 16 September 2011 15:32 To: cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] standard names for stations Dear Nan, John, Jeff I think variable attributes are generally better than global attributes, because it's possible or indeed likely that you might have data from different sources in the same file. I prefer data variables to describe themselves. We can provide global attributes as a default for the whole file, but it seems safer to me to repeat the attributes per variable. Following John's point, I agree that this station ID information will not be usable by machine or guaranteed to be unique unless we standardise it in some way. I don't know about this subject, but I am sure there are people who are well-informed about it! If we want the station ID attribute(s) to be machine- readable, we can propose a standard format for them. If we want an attr where the data-writer can informally record ID information that another human could interpret, we don't need a standard format. Which do we want - any views? Best wishes Jonathan ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] standard names for stations
For what it is worth, we migrated a lot of our systems metadata and config to JSON and it works like a charm. The primary reason was to enable humans and computers to co exist better. We tried XML but JSON won the day for us. Regards Pk - Original Message - From: cf-metadata-boun...@cgd.ucar.edu cf-metadata-boun...@cgd.ucar.edu To: Jonathan Gregory j.m.greg...@reading.ac.uk; cf-metadata@cgd.ucar.edu cf-metadata@cgd.ucar.edu Sent: Fri Sep 16 23:13:56 2011 Subject: Re: [CF-metadata] standard names for stations Hi Jonathan, My vote would go to something that is both human and machine-readable. There are standards designed for this such as JSON. Some might consider this comes down on the side of the machine more than the human, but I find once I get my in I can read JSON much easier than XML. If this is considered to be going over the top then we need to at least specify the order in which the component parts of the string are included and preferably we should specify a delimiter to separate the components. Even an ultra-light standard such as this makes the life of any programmer writing code to parse the string infinitely easier. Cheers, Roy. -Original Message- From: cf-metadata-boun...@cgd.ucar.edu [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Jonathan Gregory Sent: 16 September 2011 15:32 To: cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] standard names for stations Dear Nan, John, Jeff I think variable attributes are generally better than global attributes, because it's possible or indeed likely that you might have data from different sources in the same file. I prefer data variables to describe themselves. We can provide global attributes as a default for the whole file, but it seems safer to me to repeat the attributes per variable. Following John's point, I agree that this station ID information will not be usable by machine or guaranteed to be unique unless we standardise it in some way. I don't know about this subject, but I am sure there are people who are well-informed about it! If we want the station ID attribute(s) to be machine- readable, we can propose a standard format for them. If we want an attr where the data-writer can informally record ID information that another human could interpret, we don't need a standard format. Which do we want - any views? Best wishes Jonathan ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Request for a region standard_name
Dear Jim Following Don's comment, if you'd be happy with a standard region name of contiguous_united_states, and since no-one else has objected, I think we should agree to that. (We can omit of_america; I would expect people to know where the united_kingdom is without of_great_britain_and_northern_ireland!) I expect Alison will comment and/or add it to the table when she is able to. Cheers Jonathan ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] standard names for stations
Heres a few comments on this discussion from my POV: 1) to summarize whats already in CF1.6: section A9.2: It is strongly recommended that there should be a station variable (which may be of any type) with the attribute cf_role=”timeseries_id”, whose values uniquely identify the stations. It is recommended that there should be station variables with standard_name attributes station_description, surface_altitude and “station_wmo_id” when applicable. Since surface_altitude already exists, the other two are called out at the end: New standard names to be added to the standard name table - station_description : variable of character type containing a description of a time series station - station_wmo_id : variable of character or integer type, containing the WMO identifier of an observing station (i dont see this last part on the web site at http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.6 so here is the final version in pdf for reference: http://www.unidata.ucar.edu/staff/caron/public/CFch9-may10.pdf note that this is not a draft, but been accepted for 1.6. However, we can always amend and extend it for 1.7.) 2) the NetCDF Attribute Convention for Dataset Discovery is at http://www.unidata.ucar.edu/software/netcdf-java/formats/DataDiscoveryAttConvention.html but doesnt have anything about stations. it does have a naming authority which was intended to create globally unique dataset ids 3) the attribute cf_role=”timeseries_id” has the same effect as a standard name. our intention was to start to separate structural meatdata vs naming physical quantities via standard names. so cf_role=”timeseries_id” indicates a unique identifier for the station. 4) There is an important wrinkle introduced in 1.6 wrt the global vs variable attributes. The info for a particular station is associated by way of the station dimension, and all variables with just that dimension are station variables. The set of variables for a station are also associated by various mechanism involving dimensions. So: 1. any metadata intended to describe the station should be a station variable or an attribute on a station variable. 2. if the data, for example, came from multiple instruments, you might want to annotate the variables with that info, understanding that the variable is already associated with a specific station and must be consistent. 5) Generally i like the idea of richer metadata for stations and platforms etc, and a naming authority is a really good idea. In service of Getting Things Done, i would recommend that we agree on something that works for human readable metadata, and then start to experiment with machine readable versions, eg JSON. whether the naming authority is part of the name or not is a bit of style, but ill say that i like it. 6) So what would be helpful would be to start with the existing new things in 1.6: 1) station variable (which may be of any type) with the attribute cf_role=”timeseries_id”, whose values uniquely identify the stations. 2) station variable with standard_name station_description 3) station variable with standard_name “station_wmo_id” and propose clarification and extensions to that. The concrete proposal has come from Jeffery, so perhaps he wants to revise it based on feedback so far and propose another reversion? ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Ragged Arrays?
Dear Chris I was hoping that someone else might respond to this, because there was a discussion not so long ago on this email list about a similar issue. I'm working on a format for storing the output from a particle tracking model. Once we're happy with it, we'll post here for feedback, but for the moment a question: As many particle tracking models create and destroy particles as the run goes on, we have decided that a ragged array representation is the way to go. But I'm having trouble figure out the standard way to represent ragged arrays. You refer in your email to the discrete sampling geometries proposal and I think that should be the way to do it, and I wonder if this is topologically the same as one of the existing feature_types. The final agreed version of that proposal, which will be in the next version of CF now being compiled, is at http://www.unidata.ucar.edu/staff/caron/public/CFch9-feb25_jg.pdf (see track ticket 37). Does it do what you need? Cheers Jonathan ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] use of volume_* and *_optical_thickness in variable names
Dear Markus Welcome to CF. 1) To what degree are qualifications part of the standard_name? In the guidelines for constructing standard_names, qualifications such as _in_air or _due_to_dry_aerosol are separated from the standard name, while they are included with some standard_names in the table. Would a suitable standard_name be surface_volume_scattering_coefficient_at_stp_in_air_due_to_pm1_dry_aerosol or just volume_scattering_coefficient with the qualifications optional? Would the qualifications I used in the example be correct? The qualifications are part of the standard name. So far, we have decided to approve each qualified standard name individually, although if they follow existing patterns it is often easy to reach agreement. 2) How are the terms _optical_depth and _optical_thickness used? I know there are deep ideological divides about the correct use of these terms, and I don't plan to re-open any possible previous discussions. I only would like to know how these terms are used in the CF convention. I found the standard name atmosphere_optical_thickness_due_to_ambient_aerosol in the standard_name table, and the explanation The optical thickness is the integral along the path of radiation of a volume scattering/absorption/attenuation coefficient. Is this path meant as the slant path pointing, e.g., from the surface at the sun, or along the vertical axis from the surface through the atmosphere? I don't recall a previous discussion about this. Perhaps someone with relevant expertise could comment. It might be that more precise names are needed if you wish to draw the distinction. 3) I found the standard_name volume_extinction_coefficient_in_air_due_to_ambient_aerosol in the table. In what sense qualifying is the term volume_, i.e. how would the volume_extinction_coefficient be different from the extinction_coefficient? If I remember correctly, the terminology volume_*_coefficient was adopted for those with units of m-1, in contrast with those for scattering by a surface, which are dimensionless, and volume_*_function for those in m-1 sr-1. Best wishes Jonathan ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Request for a region standard_name
Jonathan, I'm fine with contiguous_united_states_of_america or contiguous_united_states. Jim On 9/16/2011 11:54 AM, Jonathan Gregory wrote: Dear Jim Following Don's comment, if you'd be happy with a standard region name of contiguous_united_states, and since no-one else has objected, I think we should agree to that. (We can omit of_america; I would expect people to know where the united_kingdom is without of_great_britain_and_northern_ireland!) I expect Alison will comment and/or add it to the table when she is able to. Cheers Jonathan ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- Jim Biard Government Contractor, STG Inc. Remote Sensing and Applications Division (RSAD) National Climatic Data Center 151 Patton Ave. Asheville, NC 28801-5001 jim.bi...@noaa.gov 828-271-4900 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
