Re: [CF-metadata] udunits time unit question
Dear All, This debate between Chris and Benno hinges upon how one determines the sampling interval for a time series. Benno depends upon semantics encoded in the units of measure field - something I have done in the past although I am now convinced that it is not best practice. Chris advocates obtaining it by analysis of the time channel. Although I see Benno's usage as not best practice, it is established practice and therefore I for one feel that it should continue to be supported. Cheers, Roy. -Original Message- From: cf-metadata-boun...@cgd.ucar.edu [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Christopher Barker Sent: 30 March 2011 23:27 To: cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] udunits time unit question On 3/29/11 12:24 PM, Benno Blumenthal wrote: Well I thought I was helping, but maybe not. you certainly are helping -- we all need to know what folks' use cases are to determine a good solution. -- no one who uses months since date-time is using the udunits interpretation -- all of us who use it are using the calendar 360_day interpretation, which is part of the standard, and beyond udunits. uhm, how can you be sure what all of us are doing? Not to pick on Chris, You aren't picking on me, you're picking on my understanding of the situation -- which is fair game. but I think it is clear over the course of this conversation that many of the participants do not understand the calendar attribute, which is causing a great deal of pain (or at least conversation). yes, i think you are right. I just went to read the CF standard about this again, and it helps, but I'm still confused about how Calendars like 360_day can be used in practice, and I don't see why anyone needs: 5 days since 1960-01-01 calendar 365_days rather than: days since 1960-01-01 calendar 365_days and multiply the time variable by 5. So I think the use of units like 5 days, and use of various calendars are orthogonal. on to my question about calenders. Form the docs: 360_day All years are 360 days divided into 30 day months. OK, so I take it that a day is 24 hours, so a year is 24 * 360 hours long. I can see the utility in this, but then i guess 1960-01-31 is illegal? And if you do years since or even months since, the meaning is going to get very offset after a few year from the usual calendar. I'm also not sure what 1960-01-01 means in a 360_day calendar -- what is year 0? Or do you use the Gregorian calendar for the point in time part? Maybe that doesn't matter, as long as we aren't concerned about absolute dates. This does satisfy Steve's point about having a meaningful axis for doing math -- every month and year is the same length. However, at the end of the day, I don't see the use -- you really can't talk about dates in this calendar anyway, so why not pick an arbitrary point in time and use hours? When if comes down to it, I can certainly see why one might want to do calculations and plotting, etc, in those calendars, but I don't see why your netcdf file has to store the data that way. And there is a lot of data out there with year being the right sense of things -- the fuzziness is real -- tree ring data, coral data, ice cores, all count years more-or-less, yup. many of us are modeling the earth, which means that the earth-bound descriptions of time are what we need to concisely describe what we are doing. right -- and I think that the calendar_year concept makes sense for this -- but you'd better use something like the Gregorian calendar, or your years won't line up with the seasons for long! -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- This message (and any attachments) is for the recipient only NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
[CF-metadata] physical vs dimensional units
udunits is a dimensional units library, for manipulating powers of the fundamental dimensions (length, mass, time, charge, temperature). this is necessary but not complete for capturing the meaning of the physical units of our data. We also need units like kg/kg, for which the udunit canonical string is the empty string. But even more difficult is atoms CO2 / atoms air or grams CO2 / grams air, or count of phytoplankton. the simple thing is to just introduce another attribute unit labels (or something) for this, to be displayed to the user. but such an opaque string limits the amount of automatic inferencing that can be done. better would be a grammar from which both the dimensional units and the substance we are talking about can be understood. also, all of our space/time units arent dimensional units, they are all referenced to a datum. we include the datum in the udunit string for time, but not for vertical or horizontal coordinates. thats not a particular problem, but it does point out that these units are not the same as dimensional units. we need to include the datum in the physical units representation, which could be one string or several strings . what are other solutions that are being used for physical units? Im wondering how Roy Lowry's software deals with this? Or the MMI or SWEET ontologies , etc? Is there something nice and compact like udunits strings, but with more semantics, without getting into the complexity of RDF triples? ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] physical vs dimensional units
Hi John, We label Units of Measure with a URI (e.g. SDN:P061::ULAA represents metres), which represents a term in the P061 vocabulary (http://vocab.ndg.nerc.ac.uk/P061/list/current). The vocab (born in the late 70s) is a work of pragmatism that contains a mixture of 'dimensional units' plus 'physical units' carrying semantics like 'mg/g dry sediment'. Work in progress is to migrate semantics into our equivalent to Standard Names deprecating P061 terms as we go with the medium term objective of getting as close to udunits as possible. However, I feel complete harmonisation will never be possible. All our current software does with the P061 URIs is to use them to create text labels for plots, ASCII listings and so on. The resulting consistent labelling is the primary objective of P061. Of course, my ambition is to develop UoM interconversions, probably built on udunits, but that isn't even at the vapourware stage yet. So, the answer to your question is that we use 'physical units' as labels, but don't try and do any processing based on them. I'm starting to wonder whether the 'labelling' and 'interconversion' use cases have become a little confused in the discussions of the past few weeks. Cheers, Roy. -Original Message- From: cf-metadata-boun...@cgd.ucar.edu [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of John Caron Sent: 31 March 2011 14:00 To: cf-metadata@cgd.ucar.edu Subject: [CF-metadata] physical vs dimensional units udunits is a dimensional units library, for manipulating powers of the fundamental dimensions (length, mass, time, charge, temperature). this is necessary but not complete for capturing the meaning of the physical units of our data. We also need units like kg/kg, for which the udunit canonical string is the empty string. But even more difficult is atoms CO2 / atoms air or grams CO2 / grams air, or count of phytoplankton. the simple thing is to just introduce another attribute unit labels (or something) for this, to be displayed to the user. but such an opaque string limits the amount of automatic inferencing that can be done. better would be a grammar from which both the dimensional units and the substance we are talking about can be understood. also, all of our space/time units arent dimensional units, they are all referenced to a datum. we include the datum in the udunit string for time, but not for vertical or horizontal coordinates. thats not a particular problem, but it does point out that these units are not the same as dimensional units. we need to include the datum in the physical units representation, which could be one string or several strings . what are other solutions that are being used for physical units? Im wondering how Roy Lowry's software deals with this? Or the MMI or SWEET ontologies , etc? Is there something nice and compact like udunits strings, but with more semantics, without getting into the complexity of RDF triples? ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- This message (and any attachments) is for the recipient only NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] udunits time unit question
On 3/31/11 12:42 AM, Lowry, Roy K. wrote: Dear All, This debate between Chris and Benno hinges upon how one determines the sampling interval for a time series. Benno depends upon semantics encoded in the units of measure field - something I have done in the past although I am now convinced that it is not best practice. Chris advocates obtaining it by analysis of the time channel. Yes, I think that does sum it up well. If nothing else, there is a clear need for a little more detail (or a reference) for the calendar definition in the CF docs: http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.1/ch04s04.html Perhaps things like 360_day calendar are standard enough in the climate modeling community that they don't nee definition, but these brief descriptions are terrible unclear/imprecise for folks like me. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] udunits time unit question
For those who have never encountered a non-standard calendar in the wild, here's one place they show up: 360-day and 365-day calendars are used in climate modeling. It requires a lot of extra work to write code that can handle leap years by making the year length variable, and the payoff for doing so is minimal. So in most (maybe all) cases, nobody's ever done it. Even the irregularity of variable-length months is too much work for some model developers. Yes, 1960-01-31 is an illegal date in a 360-day calendar. The year 1960 would generally mean the year when prescribed model conditions correspond to the real-world conditions of 1960. You wouldn't get a cumulative offset in the dates using years since because there's no one-to-one mapping between model days and historical days -- if you want to compare the model output with data that uses a different calendar, the first thing you do is aggregate it to a monthly or longer timescale where there is a one-to-one correspondence. (And usually it doesn't make much sense to compare things day-by-day anyway.) The point of storing the data using the 360-day calendar is that it faithfully reflects the structure of the model output. You could massage the data to fit a Gregorian calendar, but it would be extra work, it would introduce gaps in the time coordinate, and the result might not accurately reflect the descriptions of what went into the model. For example, if some land-cover boundary condition was prescribed on a monthly basis, you'd infer the wrong value for day 60 if the calendar was Gregorian (March 1st) instead of 360-day (Feb. 30th). So it's just better all ways round to define a non-standard calendar that matches how the model works. Does that help clarify what's going on with these strange beasts? Cheers, --Seth 360_day All years are 360 days divided into 30 day months. OK, so I take it that a day is 24 hours, so a year is 24 * 360 hours long. I can see the utility in this, but then i guess 1960-01-31 is illegal? And if you do years since or even months since, the meaning is going to get very offset after a few year from the usual calendar. I'm also not sure what 1960-01-01 means in a 360_day calendar -- what is year 0? Or do you use the Gregorian calendar for the point in time part? Maybe that doesn't matter, as long as we aren't concerned about absolute dates. This does satisfy Steve's point about having a meaningful axis for doing math -- every month and year is the same length. However, at the end of the day, I don't see the use -- you really can't talk about dates in this calendar anyway, so why not pick an arbitrary point in time and use hours? When if comes down to it, I can certainly see why one might want to do calculations and plotting, etc, in those calendars, but I don't see why your netcdf file has to store the data that way. - Seth McGinnis NARCCAP Data Manager Associate Scientist IMAGe / NCAR -- ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
