Dear Lars, Maybe my point is lost in the reference to an outside source. I think CF should support more than udunits. It should support an extended units attribute for a variable meant to represent a calendar/time axis.
That extension should be prepending "calendar" to a udunits date string. This is the second option in the NetCDF-Java Common Data Model <https://www.unidata.ucar.edu/software/thredds/current/netcdf-java/#home> which was adopted from the Environmental Data Abstraction Library <https://www.semanticscholar.org/paper/The-Environmental-Data-Abstraction-Library-(-Edal-)-Griffiths-Haines/77c1018b473dbcee44068075a6458c5ef2c2f5a4>, both of which support CF but also support non-cf data that exists in the community. If the developers of udunits want to support the extension, that would be great -- they would be coming into line with practices implemented by others in the community. Regarding the analogy to length units. Let's dig in a bit. Units declarations define the function that has to be applied to take one quantity and transform it to a reference datum or some other arbitrary datum. Time is no different -- the function is just a little different. There are plenty of "transformed" quantities that require a specialized function to go from the units they are stored in to units that make the data comparable or useful in some other context (sigma coordinates for example) <https://www.phy.ornl.gov/csep/om/node24.html>. In this case, what we are saying is that we have a specialized function for time that has real utility and isn't supported by the assumptions in udunits. We even have implementations of our specialized function and a way to describe it!! All the best, - Dave > On Oct 20, 2018, at 5:39 AM, Bärring Lars <lars.barr...@smhi.se> wrote: > > Hi all, > > @Jim, I think that even within the domain of months and years, a month is not > necessarily a well behaved unit. Think of precipitation height (mm) or > sunshine hours (h); these quantities depends on the length of the month. > Without taking the month length into account one cannot really compare > precipitation height during January with that of February (10 % difference in > accumulation period) despite that the difference in the time coordinate is 1. > If one on the other hand factor out the the length of the months by using > precipitation flux (kg m-2 s-1), or precipitation rate (mm s-1, mm h-1, mm > day-1, but not mm month-1), and fraction_of_sunshine_hours (1), things > suddenly are OK. > > @David, I am not familiar with the software you are pointing at, but from > your description (and my rather limited software engineering skills) it > certainly sounds like a good way forward/solution. But what I am after is > that the CF Conventions as such should be more specific and not back away > from the issue by just referring to Udunits, especially as CF has otherwise > done such an excellent work in sorting out the sometimes hairy issues > pertaining to geophysical modelling data and observations. After all, the > complications from having different calendars is specific to the community > that CF sets out to support (and sort out...). > > @Chris (your last para in earlier post): > "I know that CF refers to udunits for unit definitions, but we really need to > either allow exceptions or periodically update udunits to meet the needs of > CF." > Yes and Yes to this suggestion ! > > > Lars > > Från: CF-metadata [cf-metadata-boun...@cgd.ucar.edu > <mailto:cf-metadata-boun...@cgd.ucar.edu>] för Jim Biard [jbi...@cicsnc.org > <mailto:jbi...@cicsnc.org>] > Skickat: den 19 oktober 2018 22:32 > Kopia: cf-metadata@cgd.ucar.edu > Ämne: Re: [CF-metadata] 'months since' and 'years since' time units > > Chris, > I see what you are saying about category, yet it is metric. Within the domain > of months and years (without days, leap days, etc), it is completely possible > to do exact math, just as we can within the domain of seconds, minutes, > hours, and days. The problem arises when you try to bridge between these two > domains. The current UDUNITS tries to keep it all in one domain, with the > result that values in months and years become problematic. > Grace and peace, > Jim > > On 10/19/18 3:53 PM, Chris Barker - NOAA Federal wrote: >> I remember this coming up on this list a while back. >> >> Yes, the concept of e.g. monthly averaged data is useful. >> >> But in that case, a “month” is really a category, not a continuous time >> variable. >> >> So we need a different way if describing it than a time access with units of >> month... >> >> -CHB >> >> Sent from my iPhone >> >> On Oct 19, 2018, at 8:45 AM, Jim Biard <jbi...@cicsnc.org >> <mailto:jbi...@cicsnc.org>> wrote: >> >>> I like how you have described the issue, Chris. >>> Using month in anything except a 360-day calendar (assuming the month is >>> defined correctly for that calendar) produces erroneous results if you try >>> to do anything but math that remains in those units - such as convert a >>> month count to a date. The same goes for physical years. >>> And yet... If we are writing data collected on monthly, seasonal, or other >>> large-scale time binnings (when doing climatologies, for example), month >>> and year become regularized - that is, they become metric within the >>> context of that data, and it would be so much more human-friendly to be >>> able to work directly in months, seasons, and years, regardless of whether >>> that is a real year, a 365 day year, or a 360 day year. >>> It seems to me that time recorded in "month and larger" terms really >>> represents a different thing, separate from 'time' as defined by CF. I see >>> two different solutions that could be based around a new standard name of >>> 'date', as Martin Juckes suggested. >>> Declare that the values for a standard name of 'date' are date strings >>> following a YYYY-MM-DD format (and perhaps something for megayears, etc >>> that would be useful for paleo people). >>> Declare that the values for a standard name of 'date' would use units of >>> calendar_month, calendar_season, and calendar_year (or whatever names >>> people like best). These units are entirely convertible between each other, >>> and would need to be added to UDUNITS. UDUNITS would not convert between >>> these units and time units. People would be free to write code that could >>> do the conversion between time and date, but there wouldn't be any >>> particular way that would be considered standard. >>> Grace and peace, >>> >>> Jim >>> >>> On 10/19/18 11:13 AM, Chris Barker - NOAA Federal wrote: >>>> Calendars are a mess— both because the earth’s rotation is not >>>> human-frendly, and because of human legacy. >>>> >>>> So we need to accept that, and not try to use calendars as though they >>>> are logical units for time. >>>> >>>> I like to think about it this way: there are time operations: “this >>>> much time has passed”, and there are calendar operations: “this day >>>> next month”. >>>> >>>> Calendar operations require a lot of machinations and a well defined >>>> calendar. Time operations are straightforward and behave “sensibly” >>>> with the usual math. >>>> >>>> Also — operations belong in libraries, not data files. CF should use >>>> only well defined units. >>>> >>>> Personally, I think udunits should never have defined “months” or >>>> “years” as a time unit. But in any case, CF can at least highly >>>> discourage, if not ban, their use. >>>> >>>> Possible exception: for “artificial” Calendars, a month can be well >>>> defined, but then the unit should be called something like >>>> “30_day_month”, other well defined name. >>>> >>>> I know that CF refers to udunits for unit definitions, but we really >>>> need to either allow exceptions or periodically update udunits to meet >>>> the needs of CF. >>>> >>>> -CHB >>>> >>>> >>>>> On Oct 19, 2018, at 5:13 AM, Bärring Lars <lars.barr...@smhi.se> >>>>> <mailto:lars.barr...@smhi.se> wrote: >>>>> >>>>> Dear all, >>>>> >>>>> I agree with Jonathan's wish for a more well-behaved Earth in the >>>>> planetary system :-) >>>>> >>>>> However, awaiting this I think that we have two issues before us: >>>>> >>>>> 1. The fact that different datasets fundamentally are based on different >>>>> length of a year, while Udunits defines a year to be the real world >>>>> tropical year. >>>>> >>>>> 2. The common language notion of months of different length (and for >>>>> February varying), while Udunits defines the length of a month to be one >>>>> twelfth of the real world tropical year. >>>>> >>>>> >>>>> I fully agree that for datasets from one source (having one calendar) the >>>>> warning against using "month_since..." and "year_since..." is very well >>>>> motivated. But, as Martin points out, when combining data based on >>>>> different calendars the easiest /best/most relevant way is to use one of >>>>> these. Having said that I will in turn go into some detail with the two >>>>> issues. >>>>> >>>>> >>>>> 1. Regarding the length of the year, which is what my previous posts >>>>> concerned, I think that we have three options: >>>>> A) Convince Udunits to change the behaviour of the year unit to depend >>>>> on the calendar. To my mind this would actually solve the problem stated >>>>> in the initial post of this thread. Whether this option s possible, have >>>>> a chance to fly, or is something that the CF community actually want I do >>>>> not now. But here is the idea for you to consider. If this solution is >>>>> implemented, it would mean changing the text in Section 4.4. >>>>> B) In the CF Conventions specify that the definition of the length of a >>>>> year deviates from the one used in Udunits. This would mean changing the >>>>> text in Section 4.4. Again, once this change has penetrated into software >>>>> libraries, this would solve the initial poster's problem. >>>>> C) Leave things as they are, preferably with some additional language in >>>>> Section 4.4 to explain when unit year might be useful and when not to use >>>>> it. >>>>> >>>>> Of course, there might be other options that I have not though of. >>>>> Personally I advocate A) or B), because to my mind the impact of having >>>>> defined different calendars is not fully implemented in CF because the >>>>> different model calendar _do_ imply different length of the year. >>>>> Moreover, at least for the 360_day calendar the Udunits definition of a >>>>> month will then coincide with what is expected from that calendar. >>>>> >>>>> 2. The obvious root of the problem here is of course the different >>>>> months' lengths in the real world calendar. Having said that, what adds >>>>> to the confusion is that Udunits has chosen the name "month" for the unit >>>>> year/12. I will put aside the problem of different month lengths and >>>>> only consider the Udunits months, which I here will call "month12" for >>>>> clarity, and focus on the implications of having data from different >>>>> calendars being merged (think ensemble summary) and described using >>>>> "month12_since" (where the time coordinate takes on integer values). For >>>>> all calendars, but 360_day, this implies that a fractional day goes into >>>>> the month12ly statistic. If this is a problem one may one may postulate >>>>> that the fractional day belongs to the month12 where the larger part >>>>> belongs. If this is complemented with the notion of calendar_month it >>>>> may be reasonable to restrict at least the latter, or both, to only take >>>>> on integer values, to avoid hairy questions like ones that Jonathan asked >>>>> "What does 1 calendar_month since 31 January mean? What does 7.23 >>>>> calendar_months since 31 January mean?" >>>>> >>>>> >>>>> Kind regards, >>>>> Lars >>>>> >>>>> PS. For those of you without a Scandinavian keyboard; you might relieved >>>>> to know that my first name is Lars (and Bärring is family name, despite >>>>> what the email alias might suggest) >>>>> >>>>> >>>>> ________________________________________ >>>>> Från: CF-metadata [cf-metadata-boun...@cgd.ucar.edu] för Taylor, >>>>> Karl E. [taylo...@llnl.gov] >>>>> Skickat: den 19 oktober 2018 06:14 >>>>> Till: cf-metadata@cgd.ucar.edu >>>>> Ämne: Re: [CF-metadata] 'months since' and 'years since' time units >>>>> >>>>> Dear all, >>>>> >>>>> I won't make any recommendations for udunits, but I will comment on the >>>>> CF-conventions. >>>>> >>>>> In general, I think we should discourage usage of calendar month as a >>>>> unit of measurement because for the real world, these are only defined >>>>> for 12 special periods each year (the beginning to the end of each >>>>> calendar month) and the " >>>>> <mailto:Kindregards,LarsPS.ForthoseofyouwithoutaScandinaviankeyboard;youmightrelievedtoknowthatmyfirstnameisLars%28andB%C3%A4rringisfamilyname,despitewhattheemailaliasmightsuggest%29________________________________________Fr%C3%A5n:CF-metadata[cf-metadata-boun...@cgd.ucar.edu]förTaylor,KarlE.[taylo...@llnl.gov]Skickat:den19oktober201806:14Till:cf-metadata@cgd.ucar.edu%C3%84mne:Re:[CF-metadata]%27monthssince%27and%27yearssince%27timeunitsDearall,Iwon%27tmakeanyrecommendationsforudunits,butIwillcommentontheCF-conventions.Ingeneral,Ithinkweshoulddiscourageusageofcalendarmonthasaunitofmeasurementbecausefortherealworld,theseareonlydefinedfor12specialperiodseachyear%28thebeginningtotheendofeachcalendarmonth%29andthe>unit" >>>>> is not constant throughout the year. >>>>> Nevertheless there are some good arguments for considering adopting a >>>>> special calendar month unit by the CF conventions, but only for limited >>>>> and very specific purposes. (I'll refer to these new units as >>>>> "calendar_months" in the following, with the understanding that the unit >>>>> will depend on the calendar adopted and will in general vary for >>>>> different months of the year and depend on whether or not the year is a >>>>> leap year.) Here are reasons (already articulated by others) why we >>>>> might want to adopt "calendar_months" as a unit: >>>>> >>>>> 1) there are existing data sets with monthly-mean data and a >>>>> time-coordinate that is supposed to indicate how many calendar months >>>>> have passed since some base month/year. Such data sets are not easily >>>>> accommodated by CF. >>>>> 2) for some judiciously selected reference times, coordinates expressed >>>>> in "calendar_months since" can be easily generated without the help of >>>>> calendar-aware time-translation algorithms. For example, given units of >>>>> "calendar_months since 2001-01-01" we can trivially generate the >>>>> coordinate values for the middle of each month: 0.5, 1.5, 2.5, etc. It >>>>> would be much more difficult to generate the coordinate values in units >>>>> of "days since ...". >>>>> 3) monthly mean data sets with time coordinates based on different >>>>> calendars can more easily be compared because if all the data sets >>>>> adopted the same reference time, then comparable months would have the >>>>> same coordinate values, independent of the actual month length defined >>>>> by different calendars. [In contrast, if the time coordinate were given >>>>> in units of "days since ... ", the coordinate values would depend on the >>>>> calendar.] >>>>> >>>>> If we define a unit of "calendar_months since ..." we would need to >>>>> address Jonathan's point that it is not immediately obvious how to >>>>> handle fractions of months and reference times different from the >>>>> beginning of a month. One approach we should consider is to restrict >>>>> use of calendar_month units to datasets reporting monthly values only >>>>> (not, daily, annual, hourly, etc.) Also for simplicity we could >>>>> restrict the reference time to be the beginning of a calendar year >>>>> (i.e., January 1 at 0:0:0 for some specified year). If we did this, it >>>>> would be relatively easy to define what we mean by fractions of months >>>>> and it would also be easy to generate the values needed to define the >>>>> time-coordinates and the cell_bounds or the bounds needed to define >>>>> climatologies. [It would be almost as easy if we allowed the reference >>>>> time to be the beginning of *any* month, but let's consider the more >>>>> restrictive "beginning of a year" option first.] >>>>> >>>>> I note that using calendar_month units to report data at intervals other >>>>> than monthly intervals offers no advantages. For example different >>>>> fractional month increments would have to be used to report data at an >>>>> invariant daily interval. This would seem to introduce complexity to a >>>>> simple time-coordinate and is why I would restrict use of calendar_month >>>>> units to monthly data. >>>>> >>>>> Since months are not all equal in length, the interval of times >>>>> represented by the same fraction may differ between months. For a >>>>> Gregorian calendar, half-way through the month of January would be noon >>>>> on the 16th of January (15.5 days from the beginning of the month), >>>>> whereas half-way through the month of June would be the 16th of June at >>>>> 00:00:00 (15.0 days from the beginning of the month. Thus 0.5 months >>>>> since 2001-01-01 would be 2001-01-16 12:00:00 and 5.5 months since >>>>> 2001-01-01 would be 2001-06-16 00:00:00. Also note that the date >>>>> corresponding to the middle of February depends on whether the year is a >>>>> leap year or not. >>>>> >>>>> Of course for a different calendar (e.g., 360_day), the dates >>>>> corresponding to 0.5 months since 2001-01-01 and 5.5 months since >>>>> 2001-01-01 would be different from those for the Gregorian calendar. >>>>> >>>>> The bottom line is that under the above restrictions, it is easy to >>>>> convert fractions of months to dates (and also to alternative units such >>>>> as "days since ..."). >>>>> >>>>> Common types of "monthly" data sets are: >>>>> >>>>> 1) reports of monthly statistics computed from multiple samples within >>>>> each month (e.g., means, standard deviation of daily values, maximum >>>>> daily mean; requires cell_bounds) >>>>> >>>>> 2) reports of monthly accumulations (e.g. monthly precipitation amount; >>>>> requires cell_bounds) >>>>> >>>>> 3) monthly climatologies (requiring climatology attribute pointing to >>>>> climatology bounds) >>>>> >>>>> For all of the above the bounds coincide with the beginning and end of >>>>> each month so with the reference time restricted to the beginning of a >>>>> year, the bounds will all be integer values of "months since". The >>>>> coordinate value for any of the above can be assigned any value in the >>>>> interval defined by the bounds. For monthly statistics one might choose >>>>> the middle of each month so coordinate values would be numbers like 0.5, >>>>> 1.5, 2.5, etc. For monthly accumulations one might prefer to use the >>>>> time at the end of each interval to represent the coordinate value >>>>> (e.g., 1.0, 2.0, 3.0, etc.). >>>>> >>>>> I understand that defining a unit of calendar_months is not compatible >>>>> with udunits, but I think we can rely on tools other than udunits to >>>>> handle more generally this new unit and the various CF conventions >>>>> calendars to convert coordinate values to other time units like "days >>>>> since ...". >>>>> >>>>> Perhaps the biggest argument in favor of introducing a calendar_month >>>>> unit is that it should make it much easier for data providers to >>>>> generate correct time coordinates for data reported at monthly >>>>> intervals. Regardless of calendar, I think it is easy to generate >>>>> monthly time coordinates under the current CF standard that are simply >>>>> wrong. In contrast, everyone should be able to trivially create a >>>>> coordinate axis with values like (1, 2, 3, .... ) or (0.5, 1.5, 2.5, >>>>> ...) without making a mistake. >>>>> >>>>> best regards, >>>>> Karl >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>> On 10/18/18 10:58 AM, Jonathan Gregory wrote: >>>>>> Dear Martin >>>>>> >>>>>> The definition of a calendar_month unit would also need rules about >>>>>> calendar >>>>>> arithmetic e.g. What does 1 calendar_month since 31 January mean? What >>>>>> does >>>>>> 7.23 calendar_months since 31 January mean? >>>>>> >>>>>> Best wishes >>>>>> >>>>>> Jonathan >>>>>> >>>>>> >>>>>> ----- Forwarded message from Martin Juckes - UKRI STFC >>>>>> <martin.juc...@stfc.ac.uk> <mailto:martin.juc...@stfc.ac.uk> ----- >>>>>> >>>>>>> Date: Thu, 18 Oct 2018 16:33:28 +0000 >>>>>>> From: Martin Juckes - UKRI STFC <martin.juc...@stfc.ac.uk> >>>>>>> <mailto:martin.juc...@stfc.ac.uk> >>>>>>> To: Jonathan Gregory <j.m.greg...@reading.ac.uk> >>>>>>> <mailto:j.m.greg...@reading.ac.uk>, "cf-metadata@cgd.ucar.edu" >>>>>>> <mailto:cf-metadata@cgd.ucar.edu> >>>>>>> <cf-metadata@cgd.ucar.edu> <mailto:cf-metadata@cgd.ucar.edu> >>>>>>> Subject: Re: [CF-metadata] 'months since' and 'years since' time units >>>>>>> >>>>>>> Dear Jonathan, >>>>>>> >>>>>>> >>>>>>> I think you could go further and disallow the use of "month" or "year" >>>>>>> as a time unit when the calendar is not standard. >>>>>>> >>>>>>> >>>>>>> If the "ncdump -t" option produces what the user expects when he has >>>>>>> units "months since 1900-01-01" and a 360 day calendar, then it is >>>>>>> going to be inconsistent with the current convention. >>>>>>> >>>>>>> >>>>>>> I still feel that there is an argument for enabling the storage of >>>>>>> information in user months in the files. E.g. I wish to compare monthly >>>>>>> mean data from 20 climate models which use a range of different >>>>>>> calendars. The mean across the models is not on any specific calendar >>>>>>> ... I could pretend it is and use units of "days since ...", but the >>>>>>> mappings from input time coordinates to output time coordinates then >>>>>>> become rather complex, when they should be trivial. Having a "date" >>>>>>> standard name which allowed the input data to have a "calendar_month" >>>>>>> coordinate would solve this (and I think Klaus's suggestion would also >>>>>>> solve it), >>>>>>> >>>>>>> >>>>>>> regards, >>>>>>> >>>>>>> Martin >>>>>>> >>>>>>> ________________________________ >>>>>>> From: CF-metadata <cf-metadata-boun...@cgd.ucar.edu> >>>>>>> <mailto:cf-metadata-boun...@cgd.ucar.edu> on behalf of Jonathan Gregory >>>>>>> <j.m.greg...@reading.ac.uk> <mailto:j.m.greg...@reading.ac.uk> >>>>>>> Sent: 18 October 2018 17:10:46 >>>>>>> To: cf-metadata@cgd.ucar.edu <mailto:cf-metadata@cgd.ucar.edu> >>>>>>> Subject: Re: [CF-metadata] 'months since' and 'years since' time units >>>>>>> >>>>>>> Dear all >>>>>>> >>>>>>> This is an interesting discussion, and I agree that's a tricky subject. >>>>>>> If only >>>>>>> we could have a well-behaved Earth which orbited the sun in an integral >>>>>>> and >>>>>>> easily factorisable number of days! >>>>>>> >>>>>>> So far I still think that we should not change the way we interpret the >>>>>>> units >>>>>>> string. It's in udunits format, and should be interpreted according to >>>>>>> the >>>>>>> calendar attribute. I would suggest that it's helpful to regard time >>>>>>> coords as >>>>>>> *encoded* and not necessarily easy for humans to read. It's certainly >>>>>>> nice to >>>>>>> see "time=1, 2, 3, ..." months since a reference date - that is easy to >>>>>>> read - >>>>>>> but when you get to 747 or 4689 months since a reference date, you >>>>>>> don't know >>>>>>> what it means any more (unless you're extremely good at mental >>>>>>> arithmetic), and >>>>>>> you might as well encode it as days. >>>>>>> >>>>>>> The antidote to inconvenient encoding is convenient software. For >>>>>>> example, >>>>>>> could cftime allow the user to construct a time coordinate variable >>>>>>> with a >>>>>>> spacing of calendar months, but encode it in the netCDF file in days? >>>>>>> Then it's >>>>>>> transparent. Similarly, time coordinate variables can be decoded into >>>>>>> human- >>>>>>> readable strings by calendar-aware software. It seems to me that this >>>>>>> isn't >>>>>>> different in principle from using ncdump to read a netCDF file, rather >>>>>>> than >>>>>>> insisting it should be intelligible when read in hexadecimal. In fact, >>>>>>> ncdump >>>>>>> itself has a -t option, which should help, according to the man page: >>>>>>> >>>>>>> "-t controls display of time data, if stored in a variable that uses a >>>>>>> udunits >>>>>>> compliant time representation such as `days since 1970-01-01' or >>>>>>> `seconds since >>>>>>> 2009-03-15 12:01:17' .... If this option is specified, time data values >>>>>>> are >>>>>>> displayed as human-readable date-time strings rather than numerical >>>>>>> values, >>>>>>> interpreted in terms of a `calendar' variable attribute, if specified. >>>>>>> ... >>>>>>> Calendar attribute values interpreted with this option include the CF >>>>>>> Conventions values `gregorian' or `standard', `proleptic_gregorian', >>>>>>> `noleap' >>>>>>> or `365_day', `all_leap' or `366_day', `360_day', and `julian'." >>>>>>> >>>>>>> I agree with comments that if we introduced new units such as >>>>>>> calendar_month >>>>>>> or 30day_month, people might well not use them, and would still be >>>>>>> disappointed >>>>>>> that "month" wasn't what they expected. >>>>>>> >>>>>>> The CF conformance document has a recommendation that "year" and >>>>>>> "month" should >>>>>>> be used "with caution". I don't what the CF checker currently does with >>>>>>> this. >>>>>>> We could change it to a recommendation that they should *not* be used, >>>>>>> in which >>>>>>> case the checker would give a warning if they were. >>>>>>> >>>>>>> Best wishes >>>>>>> >>>>>>> Jonathan >>>>>>> >>>>>>> ----- Forwarded message from Bärring Lars <lars.barr...@smhi.se> >>>>>>> <mailto:lars.barr...@smhi.se> ----- >>>>>>> >>>>>>>> Date: Thu, 18 Oct 2018 13:31:10 +0000 >>>>>>>> From: Bärring Lars <lars.barr...@smhi.se> <mailto:lars.barr...@smhi.se> >>>>>>>> To: Martin Juckes - UKRI STFC <martin.juc...@stfc.ac.uk> >>>>>>>> <mailto:martin.juc...@stfc.ac.uk>, David Blodgett >>>>>>>> <dblodg...@usgs.gov> <mailto:dblodg...@usgs.gov>, Ryan >>>>>>>> Abernathey <ryan.abernat...@gmail.com> >>>>>>>> <mailto:ryan.abernat...@gmail.com> >>>>>>>> Cc: "cf-metadata@cgd.ucar.edu" <mailto:cf-metadata@cgd.ucar.edu> >>>>>>>> <cf-metadata@cgd.ucar.edu> <mailto:cf-metadata@cgd.ucar.edu> >>>>>>>> Subject: Re: [CF-metadata] 'months since' and 'years since' time units >>>>>>>> >>>>>>>> Dear Martin, David, all, >>>>>>>> >>>>>>>> As Klaus points out, the aim of my suggestion is to make software >>>>>>>> using CF aware of the fact that the unit "year" is different depending >>>>>>>> on which calendar the model is implementing. To give an example: >>>>>>>> If I want to know when the global average temperature has increased by >>>>>>>> 1.5C, or 4C, above pre-industrial time in the CMIP5 ensemble I will >>>>>>>> get answers as a timedelta in days. As this is not really helpful I >>>>>>>> might feel inclined to convert this to years, but now UDUNITS >>>>>>>> definition of year is not helpful for those models having a 360_day or >>>>>>>> 365_day calendar. However, with the calendar-aware definition of year >>>>>>>> such a calculation would be supported without having to deal with it >>>>>>>> manually. >>>>>>>> >>>>>>>> Now, on to the discussion about months. Before my previous post I >>>>>>>> quickly read through extensive exchange on this list back in 2011, so >>>>>>>> I really appreciate David's comment that it is a complex subject. And >>>>>>>> that is the reason for why I suggested is always month is always a >>>>>>>> year / 12. So, here is an attempt to summarise the suggestion in a >>>>>>>> different way: >>>>>>>> >>>>>>>> * standard and proleptic_gregorian calendars (status quo): >>>>>>>> o the number of days in a month is not an integer >>>>>>>> o same issue with respect to ordinary (western) world months. >>>>>>>> >>>>>>>> * 365_day calendar: >>>>>>>> + the number of seconds in a month would change from being >>>>>>>> "ill-defined (?)" as 84600 * 365.242198781 / 12 = 2574957.50141, to >>>>>>>> more properly 84600 * 365 / 12 = 2573250 >>>>>>>> o same issue with respect to ordinary (western) world months. >>>>>>>> >>>>>>>> * 360_calendar: >>>>>>>> + the number of seconds in a month would change from being "very >>>>>>>> ill-defined (?)" as 84600 * 365.242198781 / 12 = 2574957.50141, to >>>>>>>> more properly 84600 * 360 / 12 = 2538000 >>>>>>>> + the number of days in a month is an integer; 12 * 30 * 84600 = >>>>>>>> 2538000 >>>>>>>> + the definition of a month is consistent with what is expected in >>>>>>>> the "360_day world" >>>>>>>> o same issue with respect to ordinary (western) world months. >>>>>>>> >>>>>>>> That is, even though the suggestion certainly do not solve everything >>>>>>>> (of course!), the only argument against it, that I can see, is the >>>>>>>> work to tease out the details and implement it in software packages. >>>>>>>> As was extensively discussed in the 2011 threads, the real problem is >>>>>>>> the varying length of the western world calendar months. But that is >>>>>>>> the topic for another thread. >>>>>>>> >>>>>>>> >>>>>>>> Kind regards, >>>>>>>> Lars >>>>>>>> >>>>>>>> ________________________________ >>>>>>>> Från: CF-metadata [cf-metadata-boun...@cgd.ucar.edu >>>>>>>> <mailto:cf-metadata-boun...@cgd.ucar.edu>] för David Blodgett >>>>>>>> [dblodg...@usgs.gov <mailto:dblodg...@usgs.gov>] >>>>>>>> Skickat: den 18 oktober 2018 13:58 >>>>>>>> Till: Ryan Abernathey >>>>>>>> Kopia: cf-metadata@cgd.ucar.edu <mailto:cf-metadata@cgd.ucar.edu> >>>>>>>> Ämne: Re: [CF-metadata] 'months since' and 'years since' time units >>>>>>>> >>>>>>>> Dear Ryan, All, >>>>>>>> >>>>>>>> I hesitate to chime in on this thread as I know just how fraught this >>>>>>>> topic can be, but then I think I know how fraught it can be so may >>>>>>>> have something to offer. My experience is at the intersection of >>>>>>>> climatological models and landscape models that are calibrated with >>>>>>>> "real" data. I've worked with daily and monthly time series model >>>>>>>> output and interpolated weather products that needs to match up to >>>>>>>> observations but uses a noleap or 360 calendar. It's an enormous pain >>>>>>>> and we as a community should do better. -- so the business case for >>>>>>>> taking this complexity head on is there! >>>>>>>> >>>>>>>> One resource I've found useful over the years is the [CDM >>>>>>>> implementation](https://www.unidata.ucar.edu/software/thredds/current/netcdf-java/CDM/CalendarDateTime.html >>>>>>>> >>>>>>>> <https://www.unidata.ucar.edu/software/thredds/current/netcdf-java/CDM/CalendarDateTime.html>) >>>>>>>> ut this does not >>>>>>>> There are two factors at play. >>>>>>>> >>>>>>>> 1) Adding "calendar" to a udunits string avoids conversion to a number >>>>>>>> of shorter time increments for long time increments (e.g. seconds per >>>>>>>> month). It keeps things in the declared time units so you hit the >>>>>>>> precise date boundaries you would expect. >>>>>>>> 2) The "calendar" attribute gets you to how to interpret the datum of >>>>>>>> the time axis. >>>>>>>> >>>>>>>> Especially relevant to this thread is: >>>>>>>> >>>>>>>> * uniform30day or 360_day = All years are 360 days divided into 30 >>>>>>>> day months. >>>>>>>> >>>>>>>> With these two, I think the problems here are solved. However, >>>>>>>> inevitably, people will omit the addition attribute for calendar or >>>>>>>> fall back on normal "months since ..." when they actually mean >>>>>>>> "calendar months since ..." and tell us 'why would you interpret my >>>>>>>> data that way it makes no sense?!?' This is perfectly parallel to >>>>>>>> spatial coordinates where people don't declare a datum for their >>>>>>>> latitude/longidute coordinates. Without that information one can not >>>>>>>> use the information with a level of precision that some use cases >>>>>>>> require. >>>>>>>> >>>>>>>> What I'm getting at is that CF should probably: >>>>>>>> 1) adopt enough attribute precision to fully describe what we are >>>>>>>> trying to convey >>>>>>>> 2) make said attributes required or declare sensible defaults that >>>>>>>> reduce ambiguity when users come knocking. >>>>>>>> >>>>>>>> That said, I've had no success pushing the community to accept that >>>>>>>> there should be a default lat/lon datum for software developers to go >>>>>>>> on and I would not doubt that the same will be true here as ambiguity >>>>>>>> and uncertainty is better than dead wrong in many cases. My stance is >>>>>>>> that we should all be dead wrong for the same reason rather than each >>>>>>>> implementor making an arbitrary decision so we all get different >>>>>>>> answers (more ambiguity) from our software du-jour. >>>>>>>> >>>>>>>> All the best, >>>>>>>> >>>>>>>> Dave >>>>>>>> >>>>>>>> >>>>>>>> On Oct 18, 2018, at 6:08 AM, Martin Juckes - UKRI STFC >>>>>>>> <martin.juc...@stfc.ac.uk >>>>>>>> <mailto:martin.juc...@stfc.ac.uk><mailto:martin.juc...@stfc.ac.uk> >>>>>>>> <mailto:martin.juc...@stfc.ac.uk>> wrote: >>>>>>>> >>>>>>>> Hello All, >>>>>>>> >>>>>>>> >>>>>>>> I think the UNIDATA pull request referenced Jeff >>>>>>>> (https://github.com/Unidata/cftime/pull/69 >>>>>>>> <https://github.com/Unidata/cftime/pull/69>) is mis-quoting the CF >>>>>>>> Convention. As far as I can see, Unidata says that a month is exactly >>>>>>>> one 12th of a year, and CF inherits this -- with the statement "For >>>>>>>> similar reasons the unit month, which is defined in >>>>>>>> udunits.dat<http://www.unidata.ucar.edu/software/udunits/> >>>>>>>> <http://www.unidata.ucar.edu/software/udunits/> to be exactly year/12, >>>>>>>> should also be used with caution." >>>>>>>> >>>>>>>> >>>>>>>> I can't see the difference between Lars's suggestion and the status >>>>>>>> quo. In UNIDATA a day is clearly defined as "period of time equal to >>>>>>>> 24 hours", which gives 84600 seconds. >>>>>>>> >>>>>>>> regards, >>>>>>>> Martin >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ________________________________ >>>>>>>> From: CF-metadata <cf-metadata-boun...@cgd.ucar.edu >>>>>>>> <mailto:cf-metadata-boun...@cgd.ucar.edu><mailto:cf-metadata-boun...@cgd.ucar.edu> >>>>>>>> <mailto:cf-metadata-boun...@cgd.ucar.edu>> on behalf of Bärring Lars >>>>>>>> <lars.barr...@smhi.se >>>>>>>> <mailto:lars.barr...@smhi.se><mailto:lars.barr...@smhi.se> >>>>>>>> <mailto:lars.barr...@smhi.se>> >>>>>>>> Sent: 18 October 2018 09:29:50 >>>>>>>> To: Ryan Abernathey; whitaker.jeff...@gmail.com >>>>>>>> <mailto:whitaker.jeff...@gmail.com><mailto:whitaker.jeff...@gmail.com> >>>>>>>> <mailto:whitaker.jeff...@gmail.com>; cf-metadata@cgd.ucar.edu >>>>>>>> <mailto:cf-metadata@cgd.ucar.edu><mailto:cf-metadata@cgd.ucar.edu> >>>>>>>> <mailto:cf-metadata@cgd.ucar.edu> >>>>>>>> Subject: Re: [CF-metadata] 'months since' and 'years since' time units >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I have have come to think about this from a somewhat different >>>>>>>> perspective. For some analyses, as well as when calculating certain >>>>>>>> derived climatological statistics (aka climate indices), using >>>>>>>> datasets based on different calendars the problem becomes obvious. >>>>>>>> >>>>>>>> In the model world of a 365-day GCM one year _is_ 365 days, and in a >>>>>>>> 360_day GCM a year _is_ 360 days. In the case of a gregorian/standard >>>>>>>> calendar GCM I am not sure whether it is 365.25 or 365.242198781 but >>>>>>>> this is in practice less of a problem. >>>>>>>> >>>>>>>> For datasets based non-standard calendars imposing the current UDUNITS >>>>>>>> definition of a year leads to complications that require workarounds >>>>>>>> if one is interested in for example the time elapsed until something >>>>>>>> happens or the duration of some (long-lasting) events. One way to >>>>>>>> partly mitigate these issues would be to use the time unit of >>>>>>>> years_since_START or months_since_START, but this is warned against in >>>>>>>> the CF Conventions and may software tools do not support it . >>>>>>>> >>>>>>>> The fundamental issue is the inconsistency between the GCM year and >>>>>>>> the UDUNITS year. So I would like to call on the wisdom of this list >>>>>>>> to see whether the CF Convention could include a modification to the >>>>>>>> definition of a year and month: >>>>>>>> >>>>>>>> * standard calendar (no change) >>>>>>>> 1 day = 84600 seconds >>>>>>>> 1 year = 365.242198781 days >>>>>>>> 1 month = 365.242198781 / 12 days >>>>>>>> >>>>>>>> * 365_day calendar >>>>>>>> 1 day = 84600 seconds >>>>>>>> 1 year = 365 days >>>>>>>> 1 month = 365 / 12 days >>>>>>>> >>>>>>>> * 360_day calendar >>>>>>>> 1 day = 84600 seconds >>>>>>>> 1 year = 360 days >>>>>>>> 1 month = 360 / 12 days >>>>>>>> >>>>>>>> That is, the seconds per day ratio is not changed thus maintaining the >>>>>>>> consistency to other SI units. And, for the 360_day calendar month >>>>>>>> follows the suggestion by Ryan and Jeffrey. >>>>>>>> >>>>>>>> >>>>>>>> Kind regards, >>>>>>>> Lars >>>>>>>> >>>>>>>> -- >>>>>>>> Lars Bärring >>>>>>>> >>>>>>>> FDr, Forskare >>>>>>>> PhD, Research Scientist >>>>>>>> >>>>>>>> SMHI / Swedish Meteorological and Hydrological Institute >>>>>>>> Rossby Centre >>>>>>>> SE - 601 76 NORRKÖPING >>>>>>>> Tel / Phone: +46 (0)11 495 8604 >>>>>>>> Fax: +46 (0)11 495 8001 >>>>>>>> Besöksadress / Visiting address: Folkborgsvägen 17 >>>>>>>> ________________________________ >>>>>>>> Från: CF-metadata [cf-metadata-boun...@cgd.ucar.edu >>>>>>>> <mailto:cf-metadata-boun...@cgd.ucar.edu><mailto:cf-metadata-boun...@cgd.ucar.edu> >>>>>>>> <mailto:cf-metadata-boun...@cgd.ucar.edu>] för Ryan Abernathey >>>>>>>> [ryan.abernat...@gmail.com >>>>>>>> <mailto:ryan.abernat...@gmail.com><mailto:ryan.abernat...@gmail.com> >>>>>>>> <mailto:ryan.abernat...@gmail.com>] >>>>>>>> Skickat: den 17 oktober 2018 21:22 >>>>>>>> Till: whitaker.jeff...@gmail.com >>>>>>>> <mailto:whitaker.jeff...@gmail.com><mailto:whitaker.jeff...@gmail.com> >>>>>>>> <mailto:whitaker.jeff...@gmail.com> >>>>>>>> Kopia: cf-metadata@cgd.ucar.edu >>>>>>>> <mailto:cf-metadata@cgd.ucar.edu><mailto:cf-metadata@cgd.ucar.edu> >>>>>>>> <mailto:cf-metadata@cgd.ucar.edu> >>>>>>>> Ämne: Re: [CF-metadata] 'months since' and 'years since' time units >>>>>>>> >>>>>>>> Hi everyone, >>>>>>>> >>>>>>>> I am that user, and I'm new to this mailing list. Thank you all for >>>>>>>> your work on CF conventions. It's such a valuable effort! >>>>>>>> >>>>>>>> I want to note that this was inspired by the proliferation of datasets >>>>>>>> in the wild that use "month" as their units. For example, nearly all >>>>>>>> of the IRI Data Library does this, in conjunction with a 3"60_day" >>>>>>>> calendar (example: >>>>>>>> https://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP-NCAR/.CDAS-1/.MONTHLY/.Diagnostic/.surface/.temp/ >>>>>>>> >>>>>>>> <https://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP-NCAR/.CDAS-1/.MONTHLY/.Diagnostic/.surface/.temp/>). >>>>>>>> >>>>>>>> My impression from talking to data providers is that no one is using >>>>>>>> "360_day" calendar and "months" as units, and then expecting "months" >>>>>>>> to be interpreted as 365.242198781/12 days. They all expect it to be >>>>>>>> interpreted as 30 days. While there are various workarounds that can >>>>>>>> be used at different levels of the software stack, the best solution, >>>>>>>> IMHO, is to explicitly allow in CF conventions what Jeff proposed: >>>>>>>> "months and years be interpreted as calendar months and years for >>>>>>>> those calendars where they have a fixed length". I don't think this >>>>>>>> will break existing applications. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Ryan >>>>>>>> >>>>>>>> On Wed, Oct 17, 2018 at 3:06 PM Jeffrey Whitaker >>>>>>>> <whitaker.jeff...@gmail.com >>>>>>>> <mailto:whitaker.jeff...@gmail.com><mailto:whitaker.jeff...@gmail.com> >>>>>>>> <mailto:whitaker.jeff...@gmail.com><mailto:whitaker.jeff...@gmail.com> >>>>>>>> <mailto:whitaker.jeff...@gmail.com>> wrote: >>>>>>>> Hi: I'm a developer of the 'cftime' python package >>>>>>>> (https://github.com/Unidata/cftime >>>>>>>> <https://github.com/Unidata/cftime>). A user submitted a pull request >>>>>>>> (https://github.com/Unidata/cftime/pull/69 >>>>>>>> <https://github.com/Unidata/cftime/pull/69>) that implements support >>>>>>>> for a 30-day calendar month time unit for the '360_day' CF calendar. >>>>>>>> Although using a 'month' time unit is a tricky proposition in general, >>>>>>>> for this calendar it seems straightforward since every month has the >>>>>>>> same length. However, in the discussion of the pull request it was >>>>>>>> pointed out that CF expects that "the value of the units attribute is >>>>>>>> a string that can be recognized by UNIDATA's Udunits package", and >>>>>>>> that UDUNITS defines a month as 365.242198781/12 days. My question >>>>>>>> is this - is it reasonable for our python package to make an exception >>>>>>>> to this rule for the 360_day calendar? More generally, can months and >>>>>>>> years be interpreted as calendar months and years for those calendars >>>>>>>> where they have a fixed length, or will this deviate from the existing >>>>>>>> CF conventions and break existing applications? >>>>>>>> >>>>>>>> Regards, Jeff >>>>>>>> >>>>>>>> -- >>>>>>>> Jeffrey S. Whitaker >>>>>>>> NOAA/OAR/PSD R/PSD1 >>>>>>>> 325 Broadway, Boulder, CO, 80305-3328 >>>>>>>> Phone: (303)497-6313 >>>>>>>> FAX: (303)497-6449 >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> CF-metadata mailing list >>>>>>>> CF-metadata@cgd.ucar.edu >>>>>>>> <mailto:CF-metadata@cgd.ucar.edu><mailto:CF-metadata@cgd.ucar.edu> >>>>>>>> <mailto:CF-metadata@cgd.ucar.edu><mailto:CF-metadata@cgd.ucar.edu> >>>>>>>> <mailto:CF-metadata@cgd.ucar.edu> >>>>>>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata >>>>>>>> <http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata> >>>>>>>> _______________________________________________ >>>>>>>> CF-metadata mailing list >>>>>>>> CF-metadata@cgd.ucar.edu >>>>>>>> <mailto:CF-metadata@cgd.ucar.edu><mailto:CF-metadata@cgd.ucar.edu> >>>>>>>> <mailto:CF-metadata@cgd.ucar.edu> >>>>>>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata >>>>>>>> <http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> CF-metadata mailing list >>>>>>>> CF-metadata@cgd.ucar.edu <mailto:CF-metadata@cgd.ucar.edu> >>>>>>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata >>>>>>>> <http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata> >>>>>>> ----- End forwarded message ----- >>>>>>> _______________________________________________ >>>>>>> CF-metadata mailing list >>>>>>> CF-metadata@cgd.ucar.edu <mailto:CF-metadata@cgd.ucar.edu> >>>>>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata >>>>>>> <http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata> >>>>>> ----- End forwarded message ----- >>>>>> _______________________________________________ >>>>>> CF-metadata mailing list >>>>>> CF-metadata@cgd.ucar.edu <mailto:CF-metadata@cgd.ucar.edu> >>>>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata >>>>>> <http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata> >>>>> _______________________________________________ >>>>> CF-metadata mailing list >>>>> CF-metadata@cgd.ucar.edu <mailto:CF-metadata@cgd.ucar.edu> >>>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata >>>>> <http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata> >>>>> _______________________________________________ >>>>> CF-metadata mailing list >>>>> CF-metadata@cgd.ucar.edu <mailto:CF-metadata@cgd.ucar.edu> >>>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata >>>>> <http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata> >>>> _______________________________________________ >>>> CF-metadata mailing list >>>> CF-metadata@cgd.ucar.edu <mailto:CF-metadata@cgd.ucar.edu> >>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata >>>> <http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata> >>> >>> -- >>> <http://www.cicsnc.org/>Visit us on >>> Facebook <http://www.facebook.com/cicsnc> Jim Biard >>> Research Scholar >>> Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/> >>> North Carolina State University <http://ncsu.edu/> >>> NOAA National Centers for Environmental Information <http://ncdc.noaa.gov/> >>> formerly NOAA’s National Climatic Data Center >>> 151 Patton Ave, Asheville, NC 28801 >>> e: jbi...@cicsnc.org <mailto:jbi...@cicsnc.org> >>> o: +1 828 271 4900 >>> >>> Connect with us on Facebook for climate >>> <https://www.facebook.com/NOAANCEIclimate> and ocean and geophysics >>> <https://www.facebook.com/NOAANCEIoceangeo> information, and follow us on >>> Twitter at @NOAANCEIclimate <https://twitter.com/NOAANCEIclimate> and >>> @NOAANCEIocngeo <https://twitter.com/NOAANCEIocngeo>. >>> >>> _______________________________________________ >>> CF-metadata mailing list >>> CF-metadata@cgd.ucar.edu <mailto:CF-metadata@cgd.ucar.edu> >>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata >>> <http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata> > > -- > <http://www.cicsnc.org/>Visit us on > Facebook <http://www.facebook.com/cicsnc> Jim Biard > Research Scholar > Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/> > North Carolina State University <http://ncsu.edu/> > NOAA National Centers for Environmental Information <http://ncdc.noaa.gov/> > formerly NOAA’s National Climatic Data Center > 151 Patton Ave, Asheville, NC 28801 > e: jbi...@cicsnc.org <mailto:jbi...@cicsnc.org> > o: +1 828 271 4900 > > Connect with us on Facebook for climate > <https://www.facebook.com/NOAANCEIclimate> and ocean and geophysics > <https://www.facebook.com/NOAANCEIoceangeo> information, and follow us on > Twitter at @NOAANCEIclimate <https://twitter.com/NOAANCEIclimate> and > @NOAANCEIocngeo <https://twitter.com/NOAANCEIocngeo>. > > _______________________________________________ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
_______________________________________________ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
