Re: [CF-metadata] Convention attribute
Russ (and thus core netcdf) has always been explicit about space-delimited conventions, so really there shouldn't be any conventions with spaces in the names. On the other hand, we have adopted the technique of using the convention name as a pattern to match against the convention attribute, so that we do not care about delimiters. This, of course, will fail if someone has a convention name that is a substring of another convention name, and that convention is not a subset of the convention with the longer name. Benno On Thu, Dec 22, 2011 at 10:42 AM, Jim Biard wrote: > It is "easier" (not by much, code-wise) to not to allow commas as > delimiters, but if you want to allow for machine-recognition of convention > names, how are you going to handle conventions that have spaces in their > names? Telling everyone else to get rid of spaces isn't a practical > solution, and you have just created a thornier problem for coders who have > to figure out which way someone dealt with forbidden spaces. > > > On 12/22/2011 10:18 AM, Nan Galbraith wrote: > >> Thanks Russ, Dave(s), Jonathan and Lorenzo - >> >> Thanks for the clarifications. I agree that it makes sense to >> require that convention names not contain spaces, and that >> it's easier (and more CF-like, hence better!) to parse space >> separated terms. >> >> Cheers - Nan >> >> The recommendation on the Unidata site for multiple conventions >>> >>> >>> http://www.unidata.ucar.edu/**netcdf/conventions.html<http://www.unidata.ucar.edu/netcdf/conventions.html> >>> >>> is to use spaces rather than commas: >>> >>> It is possible for a netCDF file to adhere to more than one set of >>> conventions, even when there is no inheritance relationship among the >>> conventions. In this case, the value of the `Conventions' attribute >>> may be a single text string containing a list of the convention names >>> separated by blank space (recommended) or commas (if a convention name >>> contains blanks), for example >>> >>> :Conventions = "XXX YYY" ; >>> >> >> >> On Dec/22/2011 6:01 AM, Lorenzo Bigagli wrote: >> >>> Hi all, >>> >>> my opinion is to keep with the current recommendation, which better >>> supports automatic parsing and the existing conforming datasets. >>> In particular, I would avoid any parsing rule for the conventions >>> attribute, keeping its syntax as simple as possible (as Jonathan points >>> out, blank-separated lists are more CF-like). >>> >>> I think it makes sense to require convention identifiers not to contain >>> spaces (as usual in identifiers). >>> Those conventions that have not followed Unidata recommendation may be >>> dealt with on a transitional basis (e.g. by means of specific parsing >>> exceptions), while they are aligned in a future revision. >>> >>> Best wishes, >>> LB >>> >>> Il giorno 22/dic/2011, alle ore 10:11, Jonathan Gregory ha scritto: >>> >>> Dear all >>>> >>>> The existing Unidata recommendation is OK and we could incorporate it >>>> into >>>> CF but it would help to be more precise, for instance: If the >>>> Conventions att >>>> includes no commas, it is interpreted as a blank-separated list of >>>> conventions; >>>> if it contains at least one comma, it is interpreted as a >>>> comma-separated list. >>>> Blank-separated lists are more CF-like - many CF attributes use that >>>> syntax - >>>> but obviously we can't insist that other conventions don't have blanks >>>> in their >>>> names, and it would be simpler therefore to use a comma-separated list >>>> for >>>> this attribute, despite the Unidata recommendation. >>>> >>>> >>> >> > -- > Jim Biard > > Government Contractor, STG Inc. > Remote Sensing and Applications Division (RSAD) > National Climatic Data Center > 151 Patton Ave. > Asheville, NC 28801-5001 > > jim.bi...@noaa.gov > 828-271-4900 > > > __**_ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/**mailman/listinfo/cf-metadata<http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata> > -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] generalizing forecast_reference_time and forecast_period
Sorry, I was not clear, I think. I was proposing time(reftime,leadtime) time:standard_name="time"; time:units = "days since 1970-01-01 00:00:00"; a 2D variable which gives the actual time corresponding to each reftime, leadtime pair -- it can be computed as the unit-corrected sum of reftime and leadtime, but that is irrelevant, it is explicitly in the file. On Tue, May 17, 2011 at 4:29 PM, Karl Taylor wrote: > I wonder what the units for "time" (=reftime+leadtime) would be, since > reftime is different for each trajectory? > Karl > > On 5/17/11 1:24 PM, Benno Blumenthal wrote: > > Aside from avoiding the use of the word "time" for anything that is not > "days since" or equivalent, I would do it as you described. Instead of > time I would use "leadtime" or "L", particularly so that I could define > "time" (reftime + leadtime) (with standard_name "time") and include that as > well in the variable list. > > On Tue, May 17, 2011 at 3:02 PM, Karl Taylor wrote: > >> Hi Benno, >> >> This isn't really a comment on your email, but a question (only somewhat >> related) occurred to me. When you store multiple trajectories (or >> forecasts), each started from a difference reference, but sampled in the >> same way (at equal intervals), how should this be done? Suppose, for >> example, you want to store ozone data from 10 freely moving balloons, with >> samples taken hourly following release (for 24 hours), but each balloon >> released at a different time. Would you do this as follows? >> >> dimensions: >> time = 24 ; >> ref_time=10 >> variables: >> float O3(time,ref_time) ; >> O3:long_name = "mole_fraction_of_ozone_in_air" ; >> O3:units = "1e-9" ; >> O3:coordinates = "lon lat z" ; >> double time(time) >> time:standard_name = "elapsed_time??? or relative_time???" >> time:long_name = "elapsed time since the beginning of the trajectory" >> time:units = "hr" >> double ref_time(ref_time) ; >> ref_time:standard_name = "reference_time???" ; >> ref_time:long_name = "time when balloon was released and the reference >> for elapsed_time (relative_time)" >> ref_time:units = "days since 1970-01-01 00:00:00" ; >> float lon(time,ref_time) ; >> lon:standard_name = "longitude" ; >> lon:units = "degrees_east" ; >> float lat(time,ref_time) ; >> lat:standard_name = "latitude" ; >> lat:units = "degrees_north" ; >> float z(time,ref_time) ; >> z:standard_name = "height_above_reference_ellipsoid" ; >> z:units = "km" ; >> z:positive = "up" ; >> >> Note the possible standard_names (I think, suggested by others). >> >> Best regards, >> Karl >> >> >> >> >> On 5/17/11 8:46 AM, Benno Blumenthal wrote: >> >> CF has standard names forecast_reference_time, forecast_period and >> time which are interrelated in a particular way. >> >> I have a trajectory dataset which also has reference_time, period, >> and time which are interrelated in the same way, but forecast is not >> an appropriate descriptor: the reference_time is the start of the >> trajectory, the period is the time relative to the start_time along >> the trajectory. >> >> I am wondering how important "forecast" is in the semantics of these >> particular standard_names -- does it really have to be a forecast? >> After all, these are the standard names for the time coordinates, e.g. >> independent variables, while forecast is a property of the dependent >> variables, i.e. how they were computed. >> >> Do we need more general names? Am I taking the current names too literally? >> >> >> >> ___ >> CF-metadata mailing list >> CF-metadata@cgd.ucar.edu >> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata >> >> > > > -- > Dr. M. Benno Blumenthal be...@iri.columbia.edu > International Research Institute for climate and society > The Earth Institute at Columbia University > Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 > > > ___ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > > -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] generalizing forecast_reference_time and forecast_period
Aside from avoiding the use of the word "time" for anything that is not "days since" or equivalent, I would do it as you described. Instead of time I would use "leadtime" or "L", particularly so that I could define "time" (reftime + leadtime) (with standard_name "time") and include that as well in the variable list. On Tue, May 17, 2011 at 3:02 PM, Karl Taylor wrote: > Hi Benno, > > This isn't really a comment on your email, but a question (only somewhat > related) occurred to me. When you store multiple trajectories (or > forecasts), each started from a difference reference, but sampled in the > same way (at equal intervals), how should this be done? Suppose, for > example, you want to store ozone data from 10 freely moving balloons, with > samples taken hourly following release (for 24 hours), but each balloon > released at a different time. Would you do this as follows? > > dimensions: > time = 24 ; > ref_time=10 > variables: > float O3(time,ref_time) ; > O3:long_name = "mole_fraction_of_ozone_in_air" ; > O3:units = "1e-9" ; > O3:coordinates = "lon lat z" ; > double time(time) > time:standard_name = "elapsed_time??? or relative_time???" > time:long_name = "elapsed time since the beginning of the trajectory" > time:units = "hr" > double ref_time(ref_time) ; > ref_time:standard_name = "reference_time???" ; > ref_time:long_name = "time when balloon was released and the reference > for elapsed_time (relative_time)" > ref_time:units = "days since 1970-01-01 00:00:00" ; > float lon(time,ref_time) ; > lon:standard_name = "longitude" ; > lon:units = "degrees_east" ; > float lat(time,ref_time) ; > lat:standard_name = "latitude" ; > lat:units = "degrees_north" ; > float z(time,ref_time) ; > z:standard_name = "height_above_reference_ellipsoid" ; > z:units = "km" ; > z:positive = "up" ; > > Note the possible standard_names (I think, suggested by others). > > Best regards, > Karl > > > > > On 5/17/11 8:46 AM, Benno Blumenthal wrote: > > CF has standard names forecast_reference_time, forecast_period and > time which are interrelated in a particular way. > > I have a trajectory dataset which also has reference_time, period, > and time which are interrelated in the same way, but forecast is not > an appropriate descriptor: the reference_time is the start of the > trajectory, the period is the time relative to the start_time along > the trajectory. > > I am wondering how important "forecast" is in the semantics of these > particular standard_names -- does it really have to be a forecast? > After all, these are the standard names for the time coordinates, e.g. > independent variables, while forecast is a property of the dependent > variables, i.e. how they were computed. > > Do we need more general names? Am I taking the current names too literally? > > > > ___ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > > -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] generalizing forecast_reference_time and forecast_period
I think what Jonathan said Therefore I think it would be sensible to add some more general names which > you could use, such as reference_time and elapsed_time. You might say that > a "time" is expressed by udunits as an elapsed_time since a reference_time. > is pretty much on the mark: I was using the word "period" because the standard_name's grammar suggests using period for a time interval and saving "time" for temporal with reference time, though "elapsed_period" sounds a bit strange to my ears. As for my using "forecast_reference_time", I am willing to accept the judgement that it is not what I want, but I can't say that it is all that clear. Forecasts are integrations from initial conditions a.k.a the analysis, which is precisely what happens in a trajectory -- the terms used in defining forecast_reference_time are mostly ill-defined, particularly what is the "time of the analysis" -- in practice it is the sampling time of the most recent data in the forecast, plus the processing time it takes to get the data to the forecast, which means essentially that the forecaster gets to define it -- it does not have an absolute (standardized) meaning. Benno On Tue, May 17, 2011 at 12:31 PM, Christopher Barker wrote: > On 5/17/11 9:17 AM, Jonathan Gregory wrote: >> >> Dear Benno >> >>> CF has standard names forecast_reference_time, forecast_period and >>> time which are interrelated in a particular way. >>> > > note, from the standard name table: > > """ > The forecast reference time in NWP is the "data time", the time of the > analysis from which the forecast was made. It is not the time for which the > forecast is valid; the standard name of time should be used for that time. > """ > > So this really is a concept specific to forecasts, and not at all what you > want. Similarly for forecast_period. > >>> I have a trajectory dataset which also has reference_time, period, >>> and time which are interrelated in the same way, but forecast is not >>> an appropriate descriptor: > > I think you have the really standard time here, usually simply the "time" > array (expressed in time-since-a-reference-datetime). What does "period" > mean in your case? > > -Chris > > > -- > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R(206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > chris.bar...@noaa.gov > ___ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
[CF-metadata] generalizing forecast_reference_time and forecast_period
CF has standard names forecast_reference_time, forecast_period and time which are interrelated in a particular way. I have a trajectory dataset which also has reference_time, period, and time which are interrelated in the same way, but forecast is not an appropriate descriptor: the reference_time is the start of the trajectory, the period is the time relative to the start_time along the trajectory. I am wondering how important "forecast" is in the semantics of these particular standard_names -- does it really have to be a forecast? After all, these are the standard names for the time coordinates, e.g. independent variables, while forecast is a property of the dependent variables, i.e. how they were computed. Do we need more general names? Am I taking the current names too literally? -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] physical vs dimensional units
Yes, but it also says the sample was annealed at a temperature of 955 K for 12 h*but not:*the sample was annealed at a temperature of 955 kelvins for 12 hours but I guess I am old enough to not expect perfection from my authorities. Benno On Sat, Apr 2, 2011 at 1:06 PM, Steve Hankin wrote: > good sleuthing! > > > On 4/2/2011 9:32 AM, Steve Emmerson wrote: > >> Dear CF'ers, >> >> On 04/02/2011 05:42 AM, Jonathan Gregory wrote: >> >> I don't really agree with this. Units are units, not descriptins of >>> quantities. >>> >> The National Institute of Science and Technology (NIST) agrees that >> information shouldn't be attached to unit specifications. See >> <http://physics.nist.gov/Pubs/SP811/sec07.html#7.4> and >> <http://physics.nist.gov/Pubs/SP811/sec07.html#7.5> for more >> information. >> >> Regards, >> Steve Emmerson >> ___ >> CF-metadata mailing list >> CF-metadata@cgd.ucar.edu >> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata >> > _______ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] udunits time unit question
On Tue, Mar 29, 2011 at 5:16 PM, Jon Blower wrote: > > I tend to agree with Chris’s points by the way – I am not sure that Benno’s > formulations make things clearer. If Chris has misunderstood the use of the > calendar attribute then so have I and so have many others – IMHO this points > to a lack of clarity in the standard rather than a failure of the community > to “get it”. Just my viewpoint. > > > I did not say that there was a failure of the community to "get it". What I meant was that the calendar attribute controls the interpretation of time, and many of the e-mails ignored it, making the conversation hard to understand. I'll not participate in this conversation again, Benno > > > > *From:* cf-metadata-boun...@cgd.ucar.edu [mailto: > cf-metadata-boun...@cgd.ucar.edu] *On Behalf Of *Benno Blumenthal > *Sent:* 29 March 2011 20:25 > *To:* Christopher Barker > *Cc:* cf-metadata@cgd.ucar.edu > *Subject:* Re: [CF-metadata] udunits time unit question > > > > Well I thought I was helping, but maybe not. > > > > > > On Tue, Mar 29, 2011 at 12:35 PM, Christopher Barker < > chris.bar...@noaa.gov> wrote: > > > > There are data sets that use "months since date-time" in existence. > However, anyone using those data sets with the udunits lib is interpreting > that data in a way that may not be what the data creator intended -- this is > simply broken, It can not be enshrined as a standard. > > > > > > and I agree. I would take it a bit farther -- no one who uses "months > since date-time" is using the udunits interpretation -- all of use who use > it are using the calendar 360_day interpretation, which is part of the > standard, and beyond udunits. > > > > Not to pick on Chris, but I think it is clear over the course of this > conversation that many of the participants do not understand the calendar > attribute, which is causing a great deal of pain (or at least conversation). > > > > My examples indirectly show how I handled the problem before the calendar > attribute existed -- I defined two fundamental time units within udunits > (seconds *and* years), and thus did not let udunits convert between them > (outside code, which understands "calendar", does convert between them). > This is not a major code change, but a configuration change (udunits.dat in > my day). Anyway, calendar is now part of the standard, which makes things > clearer (or so we would like to think). > > > > Anyway, we have a basic problem: while "since" seems to let us convert > between duration (which has physical units), and an instant in time, there > is a precision problem which calendar partly addresses/papers over: sometime > we just talk about years (or much much longer), sometimes we ignore the > variation in month length (calendar 360_days), sometimes we ignore the leap > years (calendar 365_days or calendar 366_days), and we almost always ignore > leap_seconds (at least I do, anyway). The standard has to come to grips > with this aspect of the conversion, which is always there, physical units > being what they are. > > > > And there is a lot of data out there with year being the right sense of > things -- the fuzziness is real -- tree ring data, coral data, ice cores, > all count years more-or-less, though one wonders if a year is catastrophic > enough whether it messes up the count. Not to mention the data where year > seems a bit small. Yes, we gave up on the passing of the seasons for > defining "time" a while ago, but many of us are modelling the earth, which > means that the earth-bound descriptions of time are what we need to > concisely describe what we are doing. > > > > Benno > > > > > > On 3/26/11 6:07 PM, Benno Blumenthal wrote: > > 1/365 years since 1960-01-01 calendar 365_days (equivalent to days in > 365_day calendar) > > > > Is it? what varies here, the length of the year or the length of the day? > if the length of the year is well defined (what is it here?), the 1/365 > years is well defined, and it very well may not be 24 hours. If if is, then > why the heck do you not use "days since"? > > > > 1/360 years since 1960-01-01 calendar 360_days (equivalent to days in > 360_day calendar) > 1/12 years since 1960-01-01 calendar 360_days (the much maligned months) > > > > but which month? one with a defined number of hours, all the same, or one > that varies depending on where in the calendar year the data is? > > As I read these, I'm quite confused -- I can certainly see why one might > what to collect or store data at such frequencies, but that's not what the > C
Re: [CF-metadata] udunits time unit question
Well I thought I was helping, but maybe not. On Tue, Mar 29, 2011 at 12:35 PM, Christopher Barker wrote: There are data sets that use "months since date-time" in existence. However, > anyone using those data sets with the udunits lib is interpreting that data > in a way that may not be what the data creator intended -- this is simply > broken, It can not be enshrined as a standard. and I agree. I would take it a bit farther -- no one who uses "months since date-time" is using the udunits interpretation -- all of use who use it are using the calendar 360_day interpretation, which is part of the standard, and beyond udunits. Not to pick on Chris, but I think it is clear over the course of this conversation that many of the participants do not understand the calendar attribute, which is causing a great deal of pain (or at least conversation). My examples indirectly show how I handled the problem before the calendar attribute existed -- I defined two fundamental time units within udunits (seconds *and* years), and thus did not let udunits convert between them (outside code, which understands "calendar", does convert between them). This is not a major code change, but a configuration change (udunits.dat in my day). Anyway, calendar is now part of the standard, which makes things clearer (or so we would like to think). Anyway, we have a basic problem: while "since" seems to let us convert between duration (which has physical units), and an instant in time, there is a precision problem which calendar partly addresses/papers over: sometime we just talk about years (or much much longer), sometimes we ignore the variation in month length (calendar 360_days), sometimes we ignore the leap years (calendar 365_days or calendar 366_days), and we almost always ignore leap_seconds (at least I do, anyway). The standard has to come to grips with this aspect of the conversion, which is always there, physical units being what they are. And there is a lot of data out there with year being the right sense of things -- the fuzziness is real -- tree ring data, coral data, ice cores, all count years more-or-less, though one wonders if a year is catastrophic enough whether it messes up the count. Not to mention the data where year seems a bit small. Yes, we gave up on the passing of the seasons for defining "time" a while ago, but many of us are modelling the earth, which means that the earth-bound descriptions of time are what we need to concisely describe what we are doing. Benno On 3/26/11 6:07 PM, Benno Blumenthal wrote: > >> 1/365 years since 1960-01-01 calendar 365_days (equivalent to days in >> 365_day calendar) >> > > Is it? what varies here, the length of the year or the length of the day? > if the length of the year is well defined (what is it here?), the 1/365 > years is well defined, and it very well may not be 24 hours. If if is, then > why the heck do you not use "days since"? > > > 1/360 years since 1960-01-01 calendar 360_days (equivalent to days in >> 360_day calendar) >> 1/12 years since 1960-01-01 calendar 360_days (the much maligned months) >> > > but which month? one with a defined number of hours, all the same, or one > that varies depending on where in the calendar year the data is? > > As I read these, I'm quite confused -- I can certainly see why one might > what to collect or store data at such frequencies, but that's not what the > CF standard is about. It is about clearly defining the data when stored, > transmitted and shared -- ALL of the above examples are ripe for confusion, > and could just as easily be expressed as "hours since" or "days since". > > > months since 1960-01-01 calendar 360_days >> > > what is a "calendar 360_day"? > > > (just as a reminder -- it is >> important to us, no matter how many people write to say months are >> meaningless) >> > > I don't think any of us think months are meaningless -- simple that they > are not a good choice for well-defined time durations. > > It seems to me that there are two cases: > > 1) The data is defined on some kind of regular interval (or irregular, but > on clearly defined points in the time continuum) - in which case use an > appropriate unit: seconds, hours, days. (days is open to debate, I suppose) > > or > > 2) the data correspond to calendar months, i.e. monthly averaged data, etc > -- a way to specify "calendar unit" makes sense: "calendar months since > 1989-01" > > but using all these peculiar combination of units, so that the time > variable can be: (1,2,3,4), rather than say, (0, 5, 10, 15) makes no sense > to me. > > And is it used in any other context? If you have an instrument tha
Re: [CF-metadata] udunits time unit question
examples that we use 5 days since 1960-01-01 calendar 365_days (to represent pentads, though we understand pentads too) 1/365 years since 1960-01-01 calendar 365_days (equivalent to days in 365_day calendar) 1/360 years since 1960-01-01 calendar 360_days (equivalent to days in 360_day calendar) 1/12 years since 1960-01-01 calendar 360_days (the much maligned months) months since 1960-01-01 calendar 360_days (just as a reminder -- it is important to us, no matter how many people write to say months are meaningless) Thanks for asking, Benno On Fri, Mar 25, 2011 at 9:10 PM, John Caron wrote: > On 3/25/2011 6:36 PM, Benno Blumenthal wrote: > > > > On Fri, Mar 25, 2011 at 6:54 PM, John Caron wrote: > >> On 3/22/2011 6:53 AM, John Caron wrote: >> >> Consider: >> >>int time(sample=1001); >> :long_name = "Measurement time"; >> :standard_name = "time"; >> :units = "days since 1970-01-01"; >> >> vs >> >>int time(sample=1001); >> :long_name = "Measurement time"; >> :standard_name = "time"; >> :units = "3 days since 1970-01-01"; >> >> values = 1, 2, 3, ... >> >> are these equivalent or does the second one mean every 3 days ? Is the >> second one illegal ? >> >> >> Im am going to assume that the second form is illegal, that is, you may >> not have a number in front of the unit in a "time coordinate unit" (CF >> section 4.4) >> > > I wish you wouldn't. We use that all the time, though mostly for 365_day > calendars. > > Benno > > > could you give an example. do you ever use non-integers ? > > ___ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > > -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] udunits time unit question
On Fri, Mar 25, 2011 at 6:54 PM, John Caron wrote: > On 3/22/2011 6:53 AM, John Caron wrote: > > Consider: > >int time(sample=1001); > :long_name = "Measurement time"; > :standard_name = "time"; > :units = "days since 1970-01-01"; > > vs > >int time(sample=1001); > :long_name = "Measurement time"; > :standard_name = "time"; > :units = "3 days since 1970-01-01"; > > values = 1, 2, 3, ... > > are these equivalent or does the second one mean every 3 days ? Is the > second one illegal ? > > > Im am going to assume that the second form is illegal, that is, you may not > have a number in front of the unit in a "time coordinate unit" (CF section > 4.4) > I wish you wouldn't. We use that all the time, though mostly for 365_day calendars. Benno > _______ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > > -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] udunits handling of fuzzy time units
Time is an issue, but temperature is not a "dimensional unit" either --
there is differential temperature (degree_Celsius/degree_Kevin, appropriate
for anomalies), and temperature scale, i.e. Celsius_scale and Kelvin_scale
differ only in the offset. Yes, the offset part does not "work" as well as
pure dimensional units. Yes, udunits does dimensional units best, but that
is not enough to do science data.
Benno
On Tue, Mar 22, 2011 at 8:30 AM, John Caron wrote:
> On 3/21/2011 11:14 AM, Steve Hankin wrote:
>
>
>
> On 3/17/2011 5:20 PM, John Caron wrote:
>
> On 3/17/2011 12:19 PM, Steve Hankin wrote:
>
>
>
> On 3/17/2011 9:50 AM, Christopher Barker wrote:
>
> On 3/16/11 8:47 AM, John Caron wrote:
>
> 1. time instants vs time duration
> - one must distinguish between dimensional time ("time duration",
> units="secs"), and calendar time ("time instant", or "point on the time
> continuum") *which is not dimensional. *
>
>
> yup -- key clarification in all this.
>
>
> I think we are leading ourselves astray here. *A point in time has a
> dimension.* It has dimensions of "time". Whether the units happen to be
> days, months, years or whatever depends upon the encoding. But it
> definitely has dimensions of time. The date-time string loses its meaning
> if we see it as detached from the axis that gives it a dimensionality.
>
>
> in "dimensional units", "secs" is a base dimensional unit, and it means
> "duration", eg watts = joules/sec, the sec is a time duration, not an
> instant of time.
>
> "time" is not a dimensional unit, it refers to a point on the time
> continuum. it does not have dimensional units of "secs", that is, it cannot
> be converted to a duration in "secs".
>
>
> Hi John,
>
> Beg to differ on these most fundamental of issues. *All times* (all
> "points on the time continuum") indicate intervals. Typical date-time
> strings (e.g. "21-MAR-2011:10:10") are an artfully contrived way of stating
> an interval of time relative to a precise zero reference that is 2011+ years
> ago, while still retaining high resolution (fractions of seconds) in that
> interval measurement. But perhaps this is not a point to pursue much deeper
> without beer in hand.
>
> To me the most important point is just this: before proposing new
> libraries and new data models, there should be an effort to see whether
> there is any functionality that cannot be very satisfactorily handled by
> adding some convenience methods to the encodings that CF and udunits have
> already standardized.
>
> - Steve
>
>
> Hi Steve:
>
> Well I havent had a beer yet today (6:30 am), but ill try to simulate the
> vibe ;^)
>
> I think Im using "units" in a more restrictive sense then you are. Im
> thinking about "dimensional units", which udunits handles well. Dimensional
> units are powers of base units, m^2/sec^2 for example. A "calendar time
> unit" is different, in that one never combines it with base units or takes
> powers of it. m^2 / (2011-11-01)^2 wouldnt make sense.
>
> Whether one wants to consider an instant of calendar time or an interval of
> calendar time doesnt matter so much as the fact that its not a dimensional
> unit.
>
> à votre santé!
>
> John
>
>
> ___
> CF-metadata mailing list
> CF-metadata@cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>
>
--
Dr. M. Benno Blumenthal be...@iri.columbia.edu
International Research Institute for climate and society
The Earth Institute at Columbia University
Lamont Campus, Palisades NY 10964-8000 (845) 680-4450
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] udunits handling of fuzzy time units
Quoting from http://www.iso.org/iso/support/faqs/faqs_widely_used_standards/widely_used_standards_other/date_and_time_format.htm which says things like *Example: *2003-04-01 represents the first day of April in 2003. *Example:* 2003-04-01T13:01:02 represents one minute and two seconds after one o'clock in the afternoon of 2003-04-01. Of course, it goes on to say ISO 8601 does not specify the exact meaning of its representations. This means for example that the standard does not define whether 09:00 refers to the exact end of the ninth hour of the day, any other point in the following minute or the minute as a whole. Users of the standard must agree on a more exact interpretation of the representation if this is relevant. So in the end they abdicate. So I'll back off, and say IMNSHO one must always specify both aspects of time, and the ISO8601 is a lovely string format for cleanly doing precisely that -- any ISO8601 string should be interpreted "as a whole" Benno On Fri, Mar 18, 2011 at 3:05 PM, John Caron wrote: > hmm, where do you see that part about "the iso8601 string implies a moment > and a width a.k.a resolution". i was reading it that the duration = width > spec was separate. > > in any case, this probably highlights the need to be specific about what CF > semantics are. > > > On 3/18/2011 12:27 PM, Benno Blumenthal wrote: > > It is worse than that -- though the local time of the standard creates a > problem for us -- according to the iso standard, if time is omitted, the > string corresponds to the whole day, not the start of the day -- ISO8601 is > a time *interval* standard -- the iso8601 string implies a moment and a > width a.k.a resolution. For example, 2001-01-01/07 is the first week of > 2001 is an ISO8601 string. This is an "excellent" time representation, and > an essential characteristic of a good time representation, but does not map > well into software that only holds an instantaneous time, or even XML Schema > for that matter. CF, of course, has bounds, so that the edges of the time > interval can be specified. > > Benno > > On Fri, Mar 18, 2011 at 1:23 PM, Jon Blower wrote: > >> > Just so you know, the UDUNITS package does assume the first day of the >> > year at 00:00:00 UTC if additional resolution time-fields are omitted. >> > This conforms to the ISO standard. >> >> Actually (according to Wikipedia at least) the ISO8601 standard assumes >> local time if the time zone is omitted. I'm not sure what time information >> (if any) is inferred if the ISO string omits hours/minutes/seconds (i.e. I >> don't think 2008-01-01 is inferred to be the same as 2008-01-01T00:00:00Z). >> >> But it's important to highlight that UDUNITS does assume midnight UTC if >> no time is provided. It implies that the temporal resolution is not to be >> inferred from the length of the UDUNITS time string. >> >> Jon >> >> -Original Message- >> From: cf-metadata-boun...@cgd.ucar.edu [mailto: >> cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Steve Emmerson >> Sent: 18 March 2011 17:14 >> To: cf-metadata@cgd.ucar.edu >> Subject: Re: [CF-metadata] udunits handling of fuzzy time units >> >> Martin, >> >> On 03/18/2011 04:11 AM, Schultz, Martin wrote: >> > PS: I do disagree with Christopher when he says ''"30 days since 31 Jan >> 2008" is perfectly well defined.'' - do you refer to 00 UTC or 12 UTC on 31 >> Jan 2008? Or even 00:00 UTC or 01:02:30.3625132 h UTC? OK: if you define an >> "oceanographic calendar" (where anything shorter than a day doesn't matter), >> you could have a rule that all hours, minutes, seconds, milliseconds, etc. >> are mapped onto one value (say 00:00:00 h UTC). But you will need to define >> this rule in order to give a meaning to your calendar. >> >> Just so you know, the UDUNITS package does assume the first day of the >> year at 00:00:00 UTC if additional resolution time-fields are omitted. >> This conforms to the ISO standard. >> >> Regards, >> Steve Emmerson >> ___ >> CF-metadata mailing list >> CF-metadata@cgd.ucar.edu >> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata >> ___ >> CF-metadata mailing list >> CF-metadata@cgd.ucar.edu >> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata >> > > > > -- > Dr. M. Benno Blumenthal be...@iri.columbia.edu > International Research Institute for climate and society > The Earth Institute at Columbia University >
Re: [CF-metadata] udunits handling of fuzzy time units
It is worse than that -- though the local time of the standard creates a problem for us -- according to the iso standard, if time is omitted, the string corresponds to the whole day, not the start of the day -- ISO8601 is a time *interval* standard -- the iso8601 string implies a moment and a width a.k.a resolution. For example, 2001-01-01/07 is the first week of 2001 is an ISO8601 string. This is an "excellent" time representation, and an essential characteristic of a good time representation, but does not map well into software that only holds an instantaneous time, or even XML Schema for that matter. CF, of course, has bounds, so that the edges of the time interval can be specified. Benno On Fri, Mar 18, 2011 at 1:23 PM, Jon Blower wrote: > > Just so you know, the UDUNITS package does assume the first day of the > > year at 00:00:00 UTC if additional resolution time-fields are omitted. > > This conforms to the ISO standard. > > Actually (according to Wikipedia at least) the ISO8601 standard assumes > local time if the time zone is omitted. I'm not sure what time information > (if any) is inferred if the ISO string omits hours/minutes/seconds (i.e. I > don't think 2008-01-01 is inferred to be the same as 2008-01-01T00:00:00Z). > > But it's important to highlight that UDUNITS does assume midnight UTC if no > time is provided. It implies that the temporal resolution is not to be > inferred from the length of the UDUNITS time string. > > Jon > > -Original Message- > From: cf-metadata-boun...@cgd.ucar.edu [mailto: > cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Steve Emmerson > Sent: 18 March 2011 17:14 > To: cf-metadata@cgd.ucar.edu > Subject: Re: [CF-metadata] udunits handling of fuzzy time units > > Martin, > > On 03/18/2011 04:11 AM, Schultz, Martin wrote: > > PS: I do disagree with Christopher when he says ''"30 days since 31 Jan > 2008" is perfectly well defined.'' - do you refer to 00 UTC or 12 UTC on 31 > Jan 2008? Or even 00:00 UTC or 01:02:30.3625132 h UTC? OK: if you define an > "oceanographic calendar" (where anything shorter than a day doesn't matter), > you could have a rule that all hours, minutes, seconds, milliseconds, etc. > are mapped onto one value (say 00:00:00 h UTC). But you will need to define > this rule in order to give a meaning to your calendar. > > Just so you know, the UDUNITS package does assume the first day of the > year at 00:00:00 UTC if additional resolution time-fields are omitted. > This conforms to the ISO standard. > > Regards, > Steve Emmerson > ___ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > ___ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] udunits handling of fuzzy time units
On Wed, Mar 16, 2011 at 5:06 PM, Steve Hankin wrote: > > > > P.S. Together with enhancing udunits to handle the formatting of > date-time strings,it would be natural to add the same functionality for > longitudes, too. Ditto degrees of temperature -- degrees F, C and K. > Again -- the challenges of handling times are found in the other units, too > (admittedly with less complexity and MUCH less baggage attached). > > I have to admit I extended udunits for perhaps these problems a long time ago, but there was no way to get my changes back in the original code. I added another component to the units structure so that it would keep track of the direction, so that degrees_north and degrees_east were distinguishable, and I was careful to distinguish between degree_Celsius (no origin) and Celsius_scale (origin) (which is not a code change, just a change in the .dat file that holds all the unit definitions to define terms with clear meanings). As for time, I did define both seconds and years as different physical units, but that was long before multiple calendars which is the better solution. I agree with Steve that we still need to discuss and debate the fundementals -- the discussion so far would invalidate a lot of our current data files for no particular reason. Benno -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] udunits handling of fuzzy time units
While wikipedia does use the word "accuracy", it is clear in the context that it does not really mean it, and that "resolution" is more appropriate. Benno On Wed, Mar 16, 2011 at 11:52 AM, Nan Galbraith wrote: > > On 3/16/11 10:21 AM, Hedley, Mark wrote: > >> I think one the important factors, implied if not stated is what is >> meant by accuracy. >> > Sorry, it should really be "resolution"; we also use accuracy with our > time variable (for real-time data) to describe clock drift. > > In our previous discussion on this, we did use the term accuracy: > mailman.cgd.ucar.edu/pipermail/cf-metadata/2010/thread.html, > 'New standard names for satellite obs data (time as ISO strings)' > Jon quoted the wikipedia description of ISO 8601: > > /> From Wikipedia (http://en.wikipedia.org/wiki/ISO_8601): > />/ > />/ "For reduced accuracy, any number of values may be dropped from any of > />/ the date and time representations, but in the order from the least to > />/ the most significant. For example, "2004-05" is a valid ISO 8601 date, > />/ which indicates May (the fifth month) 2004.// > / > Udunits should really treat the reference date this way. > > Nan > // > > > >> -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] udunits handling of fuzzy time units
On Tue, Mar 15, 2011 at 12:45 PM, Jon Blower wrote: > Benno, > > > > So are you suggesting that CF should allow “years since” for 360, 365, and > 366-day calendars and “months since” for 360 day calendars (and continue to > deprecate both of these for other systems)? > > > > If so, I can see how this would be convenient, but it means that > effectively we’re not using UDUNITS to define the unit string in these > special cases. > > > > I can also see the benefit in disassociating the time units string from > UDUNITS, which was one of John’s suggestions, but I would worry about > backward compatibility (haven’t thought this through). > > > > Jon > Since the standard supports calendars and udunits does not, we have very nearly already disassociated the time units string from udunits for most calendars (and if udunits is "fixed" w.r.t. calendars, we are OK there as well). There is not really a backwards compatibility problem if months and years are restricted to the explicit calendars for which they are meaningful. As for the comment that There are various date-time libraries that will let you do math like: > > a_date_time + 3 months. > > But in that case, the actual length of "3 months" depends on the date-time > it is being added to. > That is the whole point of calendars -- by converting from standard_calendar to 360, added 3 months, and converting back to the standard calendar, you do in-fact get months that depend on the date-time being added to. Those calendar-to-calendar conversions need to be done carefully, we do it by matching up the edges of each time interval, e.g. [ 1 Jan 1960, 1 Feb 1960) corresponds to Jan 1960 in all three calendars. But it is not sloppy per se. Steve's point about sloppy science is well taken, but I am not sure we can solve that by pretending that we cannot understand months when we can handle them fairly precisely. Personally I am more upset about the year meaning climatology, always a bad idea, but a particularly bad choice now that proleptic-gregorian is the ISO8601 standard calendar. In fact, the only systematic way to represent a monthly climatology (average year) is calendar or 360/365 with a modulus 360 or 365 -- days and standard calendar just do not cut it. In this context in particular months/years make a lot of sense, and I am not going to be able to persuade my uses that monthly climatologies should not be used. And sooner-or-later we are going to get into trouble for saving climatologies without the target period, anyway (1981-2010 anyone?). > > > *From:* bennoblument...@gmail.com [mailto:bennoblument...@gmail.com] *On > Behalf Of *Benno Blumenthal > *Sent:* 15 March 2011 15:20 > *To:* Jon Blower > *Cc:* John Caron; cf-metadata@cgd.ucar.edu > > *Subject:* Re: [CF-metadata] udunits handling of fuzzy time units > > > > I am sorry, but this conversation is more confusing that it needs to be -- > once calendar 360_day is chosen, there is nothing "fuzzy" about month or > year, and once calendar 365_day or 366_day is chosen, there is nothing > "fuzzy" about year. udunits does not support calendar, so its poor choice > of month/year support is not an issue -- if it did support calendar (which > is in the standard), then it would handle year/month correctly for these > choices of calendar. > > Benno > > On Tue, Mar 15, 2011 at 9:14 AM, Jon Blower > wrote: > > I think it's good to remove the dependence on UDUNITS from the CDM for date > handling. > > However, although "date" is not a unit of measure, "seconds" is, and so is > "month" in the definition of UDUNITS. Since CF defines that we use the > UDUNITS interpretation of month/year, it would seem dangerous to change this > assumption for backward compatibility? > > (It's not just that months are of variable lengths within a year, but also > that there are different definitions of a "month". UDUNITS uses a fixed > year-length (not a calendar year length) and a month is year/12.) > > BTW, the various calendars are implemented in ncWMS at > http://www.resc.rdg.ac.uk/trac/ncWMS/browser/trunk/src/java/uk/ac/rdg/resc/edal/time > . > > I even wrote half-decent unit tests - aren't I a good boy? ;-) > > > Jon > > -Original Message- > From: cf-metadata-boun...@cgd.ucar.edu [mailto: > cf-metadata-boun...@cgd.ucar.edu] On Behalf Of John Caron > > Sent: 15 March 2011 13:02 > To: cf-metadata@cgd.ucar.edu > > Subject: Re: [CF-metadata] udunits handling of fuzzy time units > > On 3/15/2011 5:03 AM, Karl Taylor wrote: > > I agree with Jon. > > > > By definition, I think, a "unit of measure" must not vary
Re: [CF-metadata] udunits handling of fuzzy time units
I am sorry, but this conversation is more confusing that it needs to be -- once calendar 360_day is chosen, there is nothing "fuzzy" about month or year, and once calendar 365_day or 366_day is chosen, there is nothing "fuzzy" about year. udunits does not support calendar, so its poor choice of month/year support is not an issue -- if it did support calendar (which is in the standard), then it would handle year/month correctly for these choices of calendar. Benno On Tue, Mar 15, 2011 at 9:14 AM, Jon Blower wrote: > I think it's good to remove the dependence on UDUNITS from the CDM for date > handling. > > However, although "date" is not a unit of measure, "seconds" is, and so is > "month" in the definition of UDUNITS. Since CF defines that we use the > UDUNITS interpretation of month/year, it would seem dangerous to change this > assumption for backward compatibility? > > (It's not just that months are of variable lengths within a year, but also > that there are different definitions of a "month". UDUNITS uses a fixed > year-length (not a calendar year length) and a month is year/12.) > > BTW, the various calendars are implemented in ncWMS at > http://www.resc.rdg.ac.uk/trac/ncWMS/browser/trunk/src/java/uk/ac/rdg/resc/edal/time > . > > I even wrote half-decent unit tests - aren't I a good boy? ;-) > > Jon > > -Original Message- > From: cf-metadata-boun...@cgd.ucar.edu [mailto: > cf-metadata-boun...@cgd.ucar.edu] On Behalf Of John Caron > Sent: 15 March 2011 13:02 > To: cf-metadata@cgd.ucar.edu > Subject: Re: [CF-metadata] udunits handling of fuzzy time units > > On 3/15/2011 5:03 AM, Karl Taylor wrote: > > I agree with Jon. > > > > By definition, I think, a "unit of measure" must not vary; hence month > > is not a proper unit and not only depends on month of year, but also > > on assumed calendar (and similarly for year). Therefore, I think > > "months since" and "years since" should not be allowed in CF. > > > > Karl > > Hi Karl: > > so if currently we cant actually use months and years, because of the > way udunits handles them, why not redefine how they should be understood > when you do use them, namely as setting the month or year field in a > date calculation. > > this eases the burden on data writers, and makes the metadata human > readable, at the cost of a small increase in the complexity of libraries > that read data. > > one more comment: a date is not a unit of measure, and therein lies all > the trouble. IMO, date handling should be removed from the udunits > package, which is what im doing now in the CDM (not removing date > handling from udunits, just not using udunits anymore to handle dates). > > John > ___ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > ___ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] staggered data
One defines edge and face dimensions, e.g. longitude as X and X_01 and latitude as Y and Y_01, and put the appropriate variables on the appropriate dimension, e.g. temp(X,Y) and u(X_01,Y) and v(X,Y_01). Benno On Mon, Jan 24, 2011 at 11:53 AM, Alexander Pletzer wrote: > Hi, > > What is the best way to store staggered data, ie face/edge centered data, > in CF? I know cell_methods can be used for cell centered data but I don't > know of an attribute for representing vector fields arising in Arakawa C and > D meshes. The vector field components are not co-located. Thanks in advance. > > Best, > > --Alex > ___ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Web reference to a standard name?
a-boun...@cgd.ucar.edu] On Behalf Of Jeff deLaBeaujardiere > > > >>> [jeff.delabeaujardi...@noaa.gov] > >>> Sent: 16 December 2010 19:40 > >>> To: John Graybeal > >>> Cc: cf-metadata@cgd.ucar.edu > >>> Subject: Re: [CF-metadata] Web reference to a standard name? > >>> > >>> On 2010-12-14 12:56, John Graybeal wrote: > >>>> Just to be crystal clear, the places where you have '16' could also > > have 'current' (if I understand correctly what Roy was saying about > > their server), and the mmisw one could also be served with a particular > > version ID (analogous to the NERC example). > >>> > >>> I think it is of the utmost importance to have a URI that does not > >>> include a version number and always provides the latest answer. > >>> Otherwise you have a proliferation of identifiers mean the same thing > > > >>> but appear to change every time the overall vocabulary is updated. > > You can also have a version-specific entry if desired. > >>> > >>> There were similar discussions regarding identifiers for coordinate > >>> reference system identifiers from EPSG (European Petroleum Survey > >>> Group), and it was fortunately recognized that a version-less URI was > > essential. > >>> > >>> -Jeff > >>> > >>> > >>> ___ > >>> CF-metadata mailing list > >>> CF-metadata@cgd.ucar.edu > >>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata-- > >>> This message (and any attachments) is for the recipient only NERC is > >>> subject to the Freedom of Information Act 2000 and the contents of > >>> this email and any reply you make may be disclosed by NERC unless it > >>> is exempt from release under the Act. Any material supplied to NERC > >>> may be stored in an electronic records management system. > >> ___ > >> CF-metadata mailing list > >> CF-metadata@cgd.ucar.edu > >> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > > > > > > > > John Graybeal <mailto:jgrayb...@ucsd.edu> > > phone: 858-534-2162 > > System Development Manager > > Ocean Observatories Initiative Cyberinfrastructure Project: > > http://ci.oceanobservatories.org > > Marine Metadata Interoperability Project: http://marinemetadata.org > > > > ___ > > CF-metadata mailing list > > CF-metadata@cgd.ucar.edu > > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > > ___ > > CF-metadata mailing list > > CF-metadata@cgd.ucar.edu > > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > > -- > > This message (and any attachments) is for the recipient only NERC > > is subject to the Freedom of Information Act 2000 and the contents > > of this email and any reply you make may be disclosed by NERC unless > > it is exempt from release under the Act. Any material supplied to > > NERC may be stored in an electronic records management system. > > ___ > > CF-metadata mailing list > > CF-metadata@cgd.ucar.edu > > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > > > > John Graybeal <mailto:jgrayb...@ucsd.edu> > phone: 858-534-2162 > System Development Manager > Ocean Observatories Initiative Cyberinfrastructure Project: > http://ci.oceanobservatories.org > Marine Metadata Interoperability Project: http://marinemetadata.org > > ___ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Web reference to a standard name?
stand correctly what Roy was saying about > > their server), and the mmisw one could also be served with a > > particular version ID (analogous to the NERC example). > > > > If there is someone on the CF (or other) team that wants to play with > > us on the 'simple matter of programming' part, that would be a > > collaboration we'd be happy to encourage. > > > > John > > > > On Dec 14, 2010, at 08:16, Steve Hankin wrote: > > > I second that thought. > > > > > > On 12/14/2010 3:12 AM, Bryan Lawrence wrote: > > >> Hi Dom > > >> > > >> Actually, I think we really want: > > >> > > >> http://cfconventions.org/something > > >> > > >> (We own cfconventions.org, so we can point it where we want!) > > >> > > >> And I would like in 2011 to have settled on a permanent answer to > > >> that question! > > >> > > >> Cheers > > >> Bryan > > >> > > >>> Thanks all, > > >>> > > >>> Interesting to see the new MMI ontology work. > > >>> I take from this discussion then that there is not currently a CF > > >>> 'blessed' method to reference a term? > > >>> > > >>> E.g. for air_temperature I can choose from any of these: > > >>> > > >>> http://mmisw.org/ont/cf/parameter/air_temperature > > >>> http://vocab.ndg.nerc.ac.uk/term/P071/16/CFSN0023 > > >>> http://cf-pcmdi.llnl.gov/documents/cf-standard-names/standard-nam > > >>> e-ta ble/16/cf-standard-name-table.html#air_temperature plus: > > >>> http://cf-pcmdi.llnl.gov/documents/cf-standard-names/standard-na > > >>> me-ta ble/16/cf-standard-name-table.xml > > >>> > > >>> > > >>> Cheers, > > >>> Dom > > >>> > > >>> On 14/12/10 08:18, Lowry, Roy K. wrote: > > >>>> Dear All, > > >>>> > > >>>> Working to find a way of harmonising the two sources of > > >>>> web-served Standard Names is on by ‘To-Do’ list for next year > > >>>> (part of an issue list fed into the NETMAR project by John G.) > > >>>> > > >>>> Cheers, Roy. > > >>>> > > >>>> *From:* cf-metadata-boun...@cgd.ucar.edu > > >>>> [mailto:cf-metadata-boun...@cgd.ucar.edu] *On Behalf Of *John > > >>>> Graybeal *Sent:* 13 December 2010 22:28 > > >>>> *To:* Benno Blumenthal > > >>>> *Cc:* cf-metadata@cgd.ucar.edu > > >>>> *Subject:* Re: [CF-metadata] Web reference to a standard name? > > >>>> > > >>>> A minor clarification as to how MMI is doing things: > > >>>> > > >>>> Benno, all, > > >>>> > > >>>> On Dec 13, 2010, at 07:25, Benno Blumenthal wrote: > > >>>> We have tried to get standard URIs for CF concepts, but the > > >>>> effort bogged down. > > >>>> > > >>>> As for standard names, MMI established a set of URIs at > > >>>> > > >>>> http://marinemetadata.org/cf > > >>>> > > >>>> but has not been keeping up to date. > > >>>> > > >>>> Thanks for the mention. This is our older URL, sorry; we are now > > >>>> maintaining CF standard names in our ontology repository, using > > >>>> resolvable URLs. [1] > > >>>> > > >>>> There was a period when MMI was not keeping up to date with CF > > >>>> changes, but in the recent past we have been continuously > > >>>> updating the SKOS ontology for the parameter names at the site > > >>>> Carlos mentioned [1] (We will forward the misleading /cf URL > > >>>> above, or fix it to make clear in some other way the current > > >>>> status; thanks for calling that out.) > > >>>> > > >>>> As with Roy's handling, we provide URLs both for the current > > >>>> version at the reference URL, and for individual releases of the > > >>>> vocabulary. > > >>>> > > >>>> John > > >>>> > > >>>> [1] http://mmisw.org/ont/cf/parameter > > >>>> > > >>>> >
Re: [CF-metadata] potential temperature
I am a bit rusty at this, but in the ocean potential temperature and salinity are preserved in an adiabatic transition, "potential density" is not because the eq-of-state is non-linear. So potential density is spoken of in the ocean, but it gets one into trouble because if parcels are moved far enough to the reference pressure, the relative potential_density of the two can switch. So I would think potential_temperature is the better choice of variable for a modeler, and reference_pressure_for_potential_temperature is in fact an accurate description of the situation even for potential_density (salinity does not have a reference pressure). Benno On Mon, Dec 13, 2010 at 3:11 PM, Karl Taylor wrote: > Dear Jonathan, > > I support this proposal, except I'd suggest defining also > "reference_pressure_for_potential_density". Hopefully, an ocean modeler > will weigh in on this, but I think potential density may be directly > calculated in ocean models, in which case it would be odd to assume that one > should use the reference pressure for temperature in your definition of > potential_density. > > Best regards, > Karl > > On 12/3/10 1:19 PM, Jonathan Gregory wrote: > > Dear Karl > > I would suggest that we change the definition of potential temperature so that > it says *by default* the reference pressure is 1e5 Pa, but that the data var > could also have a size-one coordinate variable or a scalar coordinate variable > with the standard_name of reference_pressure_for_potential_temperature, which > I am proposing as an addition to the standard_name table, that would specify > the reference pressure. This would be backward-compatible because any existing > use of sea_water_potential_temperature would be with the default reference > pressure by definition. I agree that the definition should have said 1e5 Pa, > not sea level pressure, but I am sure that people will have used it as 1e5 Pa > and not worried about the difference. If anyone had noticed the definition and > been concerned, they would have queried it before. > > The same issue arises for potential density. Is it OK to use a > reference_pressure_for_potential_temperature > to define potential density? I think so. It is the temperature which changes; > the potential density is computed from the potential temperature. > > Cheers > > Jonathan > ___ > CF-metadata mailing > listcf-metad...@cgd.ucar.eduhttp://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > > > _______ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > > -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Web reference to a standard name?
On Mon, Dec 13, 2010 at 5:27 PM, John Graybeal wrote: > A minor clarification as to how MMI is doing things: > > Benno, all, > > On Dec 13, 2010, at 07:25, Benno Blumenthal wrote: > > We have tried to get standard URIs for CF concepts, but the effort bogged >> down. >> > > As for standard names, MMI established a set of URIs at > > http://marinemetadata.org/cf > > but has not been keeping up to date. > > > Thanks for the mention. This is our older URL, sorry; we are now > maintaining CF standard names in our ontology repository, using resolvable > URLs. [1] > > There was a period when MMI was not keeping up to date with CF changes, but > in the recent past we have been continuously updating the SKOS ontology for > the parameter names at the site Carlos mentioned [1] (We will forward the > misleading /cf URL above, or fix it to make clear in some other way the > current status; thanks for calling that out.) > Glad you have a new system that can need up-to-date. Let me point out two things, very important to me at least. 1) you changed the namespace of the standard names, so your new document is not a direct replacement of the old, you would need to provide a mapping to make the new replace the old. 2) you changed the ontology. SKOS is good and fine, but it makes it impossible to connect to CF -- that is external to what you are providing. Your old ontology, because it was a complete representation of what was in the XML file, gave a handle to map from a XML representation of a netcdf file to the semantic representation of standard names. You cannot get that by a redirect to your SKOS representation. To be more concrete, in my system, I have cfatt:standard_name which is string_valued, and is one of many attributes a variable in a netcdf file can have. If a variable has that string valued attribute, it can be connected to the SKOS term corresponding to the standard name. Since SKOS is deprecating that property, (or at least was last time I looked), I use my own property iriterm:isDescribedBy So my variable has var iriterm:isDescribedBy cfsn:air_temperature. and var cfatt:standard_name "air_temperature". when the crosswalking is done. The crosswalk uses the standard_name property of the old Standard_Name objects to connect the objects to the cfatt property of the netcdf variable. SKOS is important for interrelating concepts, but we need to relate to the CF structure as well. Please do not throw away this key piece of the problem in an effort to "standardize". Benno > > As with Roy's handling, we provide URLs both for the current version at the > reference URL, and for individual releases of the vocabulary. > > John > > [1] http://mmisw.org/ont/cf/parameter > > > > I have written a XSL transform at > > http://iridl.ldeo.columbia.edu/ontologies/xslt/.cfsn2rdf.xsl > > that can be applied to > > > http://cf-pcmdi.llnl.gov/documents/cf-standard-names/standard-name-table/16/cf-standard-name-table.xml > > to generate an up-to-date version, e.g. > > xsltproc http://iridl.ldeo.columbia.edu/ontologies/xslt/.cfsn2rdf.xsl > http://cf-pcmdi.llnl.gov/documents/cf-standard-names/standard-name-table/16/cf-standard-name-table.xml > > On the other hand, Roy has created opaque URIs for standard names at > > http://vocab.ndg.nerc.ac.uk/list/P07/9 contains all the Standard Names >> that have ever been published, including names that have been aliased >> >> http://vocab.ndg.nerc.ac.uk/list/P071/9 contains the Standard Names that >> are currently valid (i.e. those that have not been deprecated through >> aliasing) >> >> http://vocab.ndg.nerc.ac.uk/list/P072/9 contains the Standard Names that >> have been deprecated through aliasing >> >> The mapping between deprecates and their replacement aliases my be >> obtained through the following API call >> >> >> http://vocab.ndg.nerc.ac.uk/axis2/services/vocab/getMap?subjectList=http://vocab.ndg.nerc.ac.uk/list/P072/current&predicate=255&objectList=http://vocab.ndg.nerc.ac.uk/list/P071/current&inference=false >> > > > he changes them with each version, so it is both more definitive and harder > to use. It is also pure SKOS, so one needs to add structure to actually > connect it with a CF-described dataset. I plan to write out a map between > the two, so that I can use his relationships between CF standard_names, but > I do not have it yet. > > Additional information and ontologies describing the CF structure both as > attributes and conceptually are available at > http://iridl.ldeo.columbia.edu/ontologies/, including the connections > between the standard_name ontologies and the CF structure. > > > > Benno > >
Re: [CF-metadata] Web reference to a standard name?
> > We have tried to get standard URIs for CF concepts, but the effort bogged > down. > As for standard names, MMI established a set of URIs at http://marinemetadata.org/cf but has not been keeping up to date. I have written a XSL transform at http://iridl.ldeo.columbia.edu/ontologies/xslt/.cfsn2rdf.xsl that can be applied to http://cf-pcmdi.llnl.gov/documents/cf-standard-names/standard-name-table/16/cf-standard-name-table.xml to generate an up-to-date version, e.g. xsltproc http://iridl.ldeo.columbia.edu/ontologies/xslt/.cfsn2rdf.xsl http://cf-pcmdi.llnl.gov/documents/cf-standard-names/standard-name-table/16/cf-standard-name-table.xml On the other hand, Roy has created opaque URIs for standard names at http://vocab.ndg.nerc.ac.uk/list/P07/9 contains all the Standard Names that > have ever been published, including names that have been aliased > > http://vocab.ndg.nerc.ac.uk/list/P071/9 contains the Standard Names that > are currently valid (i.e. those that have not been deprecated through > aliasing) > > http://vocab.ndg.nerc.ac.uk/list/P072/9 contains the Standard Names that > have been deprecated through aliasing > > The mapping between deprecates and their replacement aliases my be obtained > through the following API call > > > http://vocab.ndg.nerc.ac.uk/axis2/services/vocab/getMap?subjectList=http://vocab.ndg.nerc.ac.uk/list/P072/current&predicate=255&objectList=http://vocab.ndg.nerc.ac.uk/list/P071/current&inference=false > he changes them with each version, so it is both more definitive and harder to use. It is also pure SKOS, so one needs to add structure to actually connect it with a CF-described dataset. I plan to write out a map between the two, so that I can use his relationships between CF standard_names, but I do not have it yet. Additional information and ontologies describing the CF structure both as attributes and conceptually are available at http://iridl.ldeo.columbia.edu/ontologies/, including the connections between the standard_name ontologies and the CF structure. Benno On Mon, Dec 13, 2010 at 9:22 AM, Dominic Lowe wrote: > Hello all, > > What's the current CF recommended scheme for referencing a standard name on > the web? (either by URL or URN). > > I know about the vocab server and the xml/html online versions and could > choose to use URLs to point to items in any of these but is there a > definitive pattern that should be used? (or at least a best practice that > should be encouraged?). > > Much appreciated, > > Dominic > -- > Scanned by iCritical. > ___ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] time as ISO strings
Hi Ben, First of all, you are spending a lot of time talking about what many of us think as the parsing of the offset in the units string when it is passed to udunits, which makes it seem like a side issue, probably unfairly. ISO8601, as pointed out before, is not designed as a science standard, it is a business exchange standard, so they can get away with 1) first deciding to skip year , then changing their minds and insisting on year zero (Note that xSchema dateTime was based on the earlier choice, so it continues to skip year zero). 2) ISO8601, according to http://en.wikipedia.org/wiki/ISO_8601, is proleptic_gregorian, by mutual agreement. Also, one can put more digits in the year, again b y mutual agreement. That is not exactly what one expects from a time representation standard for science, which needs things stately clearly or left undetermined. They do say, however, that time should be continuous at the day granularity, which forces the conclusion, mutual agreement or not. Note that XML Schema merely says there is no year http://www.w3.org/TR/2004/REC-xmlschema-2-20041028/. Whether they mean by this is that earlier times should be represented by year -2 when ISO8601 says year -1, or they mean you cannot represent just year , or you cannot represent times before year , is hard to say. Aren't commercial standards wonderful? So it would seem that conforming to ISO8601 would require proleptic gregorian calendar support, not in the concrete sense of parsing a udunits string but in the more general sense of a continuous numeric representation of time corresponding to ISO8601. And the "standard" calendar in udunits is not actually the "standard" calendar of ISO8601. So it would seem that at least proleptic_gregorian support in udunits is essential for future ease of use. Benno On Mon, Nov 1, 2010 at 1:36 PM, Ben Hetland wrote: > On 30.10.2010 01:35, John Caron wrote: >> 1) no one is suggesting to replace the udunits convention; the >> original suggestion a few years back was to allow ISO 8601 >> formatted time strings in addition. >> >> 2) Udunits accepts ISO Strings for the date part, AFAIU, but >> allows more variations I think. > > Yes, or at least this would be a good principle to follow. Alas, Udunits > lacks a bit to fully accept any ISO 8601 timestamp, even if only the > date part is considered. For example, a date like "20101101" is accepted > neither as DATE nor as TIMSTAMP (sic) according to the referenced > Udunits-2 unit grammar, AFAICT. > > I suppose this is also a question of amending the Udunits library itself > so that full ISO 8601 compliance is achieved. This is something that > would also make it more suitable in a number of other applications > besides the netCDF-related ones, I suppose. > > >> The formal grammar is here: >> >> http://www.unidata.ucar.edu/software/udunits/udunits-2/udunits2lib.html#Grammar > > Thanks for the link! > (I was aware of it, but wasn't sure whether it applied to Udunits-2 > only, or just Udunits in general...) > > >> 3) The problem with udunits time handling is that it doesnt handle >> anything other than the default Calendar. So, it cant be used as the >> reference implementation for non-standard Calendars. > > I believe the same "problem" exists with ISO 8601 as well. > > I also think that adding calendar support is an issue that should be > introduced with great caution. Calendars are location and culture > specific, and even when dealing with historical or culture-specific > dates (how applicable is that to CF?) one needs quite a bit of > additional meta information to clearly deal with such a "calendar time > scale". > > For scientific usage, I would claim that while any such time scale can > definitely be useful, one would still sooner or later need to convert > that "time unit" into something that can be related to _other_ time > scales, at least if sharing and reuse of data is a concern (which it > should, IMHO!). To me, it seems that the only internationally accepted > common standard, or de facto "reference time scale", is the current > Gregorian calendar. It is often used in parallel with other calendars. > If data in _other_ calendars (i.e. "time scales") were to be specifiable > in CF, then at least some means of unambiguously mapping them to > "Gregorian ISO 8601" would need to be settled. > > -- > Regards, > -+-Ben-+- > > Opinions expressed are my own, not necessarily those of my employer. > ___ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > -- Dr. M. Benno Blument
Re: [CF-metadata] Interpretation of unspecified time zone (was: New standard names for satellite obs data (timeas ISO strings))
It is not really a problem to represent this in CF, just less than pretty. time becomes a function of more than one dimension. So, if by "local time" you mean that time is such that noon local time has the sun directly overhead, i.e. local time before the railroads established time zones, then we have time(longitude,time) i.e. two dimensions time, longitude, and a variable time which gives UTC for each raster point. (Unless, of course, my physics is a bit off). On the other hand, if you mean local time as in local time zone, then time(longitude,latitude,time) because time zones have a big political component to them (which is time dependent). You can, of course, use two different names for time dimension and time variable, would make things easier to understand. Given that this analyis, while valid, is a bit unusual, seems appropriate that CF can represent it, though not as simply as an image which is uniform in UTC. Benno On Fri, Oct 22, 2010 at 12:32 PM, Julia Collins wrote: > Hi, > > On Fri, 22 Oct 2010, Lowry, Roy K wrote: >> >> I wonder how many existing CF data files would have the meaning of their >> time channel changed were this suggestion to be adopted. > > I am sure many. In some cases it might even make them accurate. :-) At some > level, it's a tradeoff between "breaking" existing data descriptions and > creating a more robust standard. Whether my suggestion creates a more > robust standard or not is up for debate. It does align it more closely with > the ISO standard, and I think there's something to be said for that. > >> If I were Julia I would be reworking my data so that the time channel was >> true UT. I've had so many problems in the past with local time >> co-ordinates > > It's not data I'm generating, but rather data provided by a PI who has > determined that this is how he wants to present his data. I believe the > idea is to be able to view a wide geographic area over the same effective > local time. It's a composite image. If you can tell me how to represent > these multiple UTC times for one variable using the existing CF > conventions, then I'll try to do so. I believe CF limits me to specifying > one time zone for a particular time dimension. Re-working the data so that > it's broken up into separate time zones would, as I understand it, mean > splitting up the data variable into multiple variables, each with their own > time dimension, which is not what the data provider wants to do. Splitting > the variable would require the user to put the pieces back together in > order to see the image as the data provider intended. As far as I can tell, > following the ISO interpretation of the time zone indicator does exactly > what I need without ambiguity. I'd like to describe the data in a way > that's perfectly compliant with the CF convention, but in this case I don't > see a way to do so. > > Thanks, > Julia > -- > Julia Collins > National Snow and Ice Data Center > http://nsidc.org/ > colli...@nsidc.org > +1 303.492.6405 > ___ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] New standard names for satellite obs data (timeas ISO strings)
On Fri, Oct 22, 2010 at 6:27 AM, Jon Blower wrote: > Hi Roy, > > Yes, but *only* if the ISO string contains time information. If it only > contains a date (e.g. "2009-09-01") you can't add a time zone as I understand > it. Furthermore, there's no default to UTC - in fact the default is local > time. This is true, but one can always write out the time in explicit intervals 2009-9-01T00:00Z/24:00Z One does, of course lose all the elegance of the ISO8601 abbreviation, particularly with respect to months, rendering it somewhat pointless. Given that ISO defaults to "local time", it behooves CF to specify what that local time is (if CF is "where" we are). As it currently stands, time zone can be encoded in the units. Also the CF document states Note: if the time zone is omitted the default is UTC, and if both time and time zone are omitted the default is 00:00:00 UTC. If we take that to constrain ISO8601 as well, we have made progress. If we can do such a thing. Benno > > (And I'm not sure your syntax is right - I think you use "+5:00", not "Z5". > I think "Z" is only used on its own to indicate Zulu/UTC. Might be wrong > though.) > > Cheers, Jon > > -Original Message- > From: Lowry, Roy K [mailto:r...@bodc.ac.uk] > Sent: 22 October 2010 09:06 > To: Jon Blower; Benno Blumenthal > Cc: cf-metadata@cgd.ucar.edu > Subject: RE: [CF-metadata] New standard names for satellite obs data (timeas > ISO strings) > > Hi Jon, > > Full ISO8601 does carry time zone expressed in hours relative to UT in the > syntax Zx where x is the offset from Zulu at the right-hand end of the string. > > Cheers, Roy. > > -Original Message- > From: cf-metadata-boun...@cgd.ucar.edu > [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Jon Blower > Sent: 21 October 2010 23:28 > To: Benno Blumenthal > Cc: cf-metadata@cgd.ucar.edu > Subject: Re: [CF-metadata] New standard names for satellite obs data (time as > ISO strings) > > Hi Benno, > >> No one I know beyond the age of four thinks Sep 2009 is ambiguous > > Do you mean "beyond the age of precisely 4.000... years" or "beyond the age > of 4.999... years"? Or is the ambiguous temporal metadata concept of "the > age of four" sufficient? > > ;-) > > All ISO8601 dates (year, month or day resolution) are inherently ambiguous > because they carry no time zone information (your precise bounds are likely > to be 5 hours different from mine, or something more complex if daylight > saving is involved). So with ISO8601 alone I don't think there's such a > thing as the "preciseSep2009" in your argument below, unless I've > misunderstood what you mean, in which case apologies. > > Hmm... come to think of it, this might actually argue *against* using ISO8601 > strings alone as indicators of time resolution. If we really *do* mean that > data are representative of a 24-hour period starting at midnight UTC on the > first of September, we can't represent this unambiguously as "2009-01-01" > because of the time zone problem. (I think that "2009-01-01Z" is illegal.) > In this case we would be better off representing this period as a nominal > value plus explicit bounds, or a nominal value plus time zone plus some > additional information that we can discard any precision greater than 1 day. > > *However*, it's still very useful to know the resolution of the time axis by > some means other than inspecting the coordinate bounds. An application (e.g. > automatically generating a time selector widget in a GUI) will probably not > want to look at all the bounds of all the time coordinates to infer the time > resolution: apart from being generally tedious, it would be very difficult > for monthly data (because months are different lengths, and vary between > calendars). Additionally, this inference would be complicated if > floating-point numbers were used to represent time coordinates, since these > are usually slightly inaccurate (dangerous to compare floats for equality, > etc. etc.) > > > So, I'm starting to like the idea of an additional (and optional) CF > attribute to specify time coordinate resolution. This could be specified in > precise numeric terms (e.g. instrument precision of 0.12 ms) or in > less-precise human calendar terms for certain kinds of data (e.g. "P1M" for > monthly resolution). It would be additional to the coordinate bounds: data > providers could specify one or the other, or both if they are consistent (or > neither if appropriate?) > > This would not require or preclude the use of I
Re: [CF-metadata] New standard names for satellite obs data (time as ISO strings)
Hi Jon, Sorry, I am not buying it. No one I know beyond the age of four thinks Sep 2009 is ambiguous, and I don't read your examples as needing vagueness on the time specifically. Suppose, for a moment, that you succeed beyond your wildest dreams, and it is possible to express in CF some relationship to a vague notion of Sep2009, i.e. data hasADataRelationshipWith vagueSep2009. I would say there is another relationship vagueSep2009 isAVagueVersionOf preciseSep2009 And you could have just as easily coded in CF data hasADataRelationshipWithAVaugeVersionOf preciseSep2009 i.e. there is no reason why the vaugeness cannot be coded as a dependent data property. Which is what CF is currently set up to do, with a possible extension of the cell_methods vocabulary Futhermore, you said > You *could* modify CF so that to represent data that are "representative of > September 2010", you specify a nominal date half-way through September and > set the bounds to the first and last instants of September. And perhaps use > a new cell_methods of "representative". But the half-way point and the > bounds would be quite (very) tedious to compute in the general case (months > and years are of variable length for example and depend on the calendar > system). > That is not a modification of CF -- that is the way it is currently encoded in CF (though there is no meaning to the nominal value, so you can set it to whatever). And yes, you have to generate the edges, which you have to do anyway if you are going to sensibly handle computations with the data. And let me repeat my main original point, so that it does not get completely buried -- CF software really needs to render time bounds as ISO8601 conveniently and universally (both directions seems to be essential, i.e. reading and writing), so the the CF convention can be easily used in this regard. Sorry I couldn't be more helpful, Benno On Thu, Oct 21, 2010 at 11:57 AM, Jon Blower wrote: > Hi Benno, > > 2010-09 is not necessarily a precise specification of a month - time zones > make it a little fuzzy for one thing. Separate to this, there are parallel > conversations going on in the ISO/OGC community about what time strings > actually mean. A metadata person might say that "2010-09" is simply a > shorthand for the fuzzy concept of "September 2010" and does not represent a > precise interval (i.e. a square-wave function that is 1 during September and > 0 outside). Apart from the time zone issue which blurs the boundaries, this > square-wave is simply not what humans mean when, for example, they tag a > report as having been written in September 2010. It just distinguishes it > from version 2 of the report, which was written in November. In this > context, it's just a label with some temporal meaning. > > These "metadata guys" are in discussion with the "positioning guys" who view > date/times as precisely-defined positions within a temporal CRS. You may (or > may not!) like to look at the GeoAPI mailing list, in which we are trying to > figure out whether we can actually use the same Java types for both of these > subtly-different views of date/times (we hope we can but haven't agreed). > One might think that they are obviously the same thing, but I don't think so. > > You *could* modify CF so that to represent data that are "representative of > September 2010", you specify a nominal date half-way through September and > set the bounds to the first and last instants of September. And perhaps use > a new cell_methods of "representative". But the half-way point and the > bounds would be quite (very) tedious to compute in the general case (months > and years are of variable length for example and depend on the calendar > system). > >> Of course, how the data is actually related to that interval is where the >> notion of precision might come in > > Actually, you've probably gathered that I also consider the notion of > precision to apply to the interval itself, not just how the data relates to > it. > > This discussion repeats a bit of the previous discussion on this list > entitled "bounds/precision for time axis". I like Jonathan's distinction > between the concepts of temporal resolution and representivity: > http://www.mail-archive.com/cf-metadata@cgd.ucar.edu/msg01341.html. > > And just for completeness we should not that ISO8601 strings are not > fixed-length, nor do they have a maximum length (in contrast to what I said > before, sorry). So I can see some implementation challenges in NetCDF. > > Cheers, Jon > > > -Original Message- > From: bennoblument...@gmail.com [mailto:bennoblument...@gmail.c
Re: [CF-metadata] New standard names for satellite obs data (time as ISO strings)
f Of Ben Hetland > Sent: 20 October 2010 09:14 > To: cf-metadata@cgd.ucar.edu > Subject: Re: [CF-metadata] New standard names for satellite obs data > > On 19.10.2010 16:27, Seth McGinnis wrote: > > What about using 'date' for string-valued times? That was my homebrew > solution when I was considering a similar problem. > > If I may butt in and contribute here, I usually prefer names like > 'datetime' or 'timestamp' in cases like this, because 'date' is > potentially confusing. It may not be immediately obvious to a future > reader (or programmer) that a variable called 'date' supports points in > time down to for example seconds of accuracy. > > > (Note that string data is a big pain to deal with in NetCDF-3, because > you're limited to fixed-length character arrays. You need to use > NetCDF-4 / HDF5 to get Strings as a data type.) > > (It may not be such a practical issue with ISO 8601 strings, as a > reasonable max. length can be determined, I presume.) > > > ___ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > > -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] proposed change to CF calendar
Sorry, I don't understand why you simply label your datasets calendar: proleptic_gregorian if they go before 1582, and support gregorian with the restriction that the values correspond to post 1582. Much better than changing the calendar attribute standard (again!), and messing up existing code that does support the standard, and existing data that is correctly labeled with the current standard. Benno On Wed, Oct 13, 2010 at 8:55 PM, Earl Schwab wrote: > > Hi, > > As Tony indicated, the ESMF Team also supports this proposal to make > the CF default/standard Gregorian calendar proleptic. Like CCSM, our > Gregorian calendar is proleptic, and, also like CCSM, we do not > support a mixed Julian/Gregorian historical calendar, as we've had > no such requirement to-date from our users. > > Regards, > Earl Schwab > ESMF Core Team > > > Original Message > Subject: proposed change to calendar definitions > Date: Fri, 08 Oct 2010 11:00:57 -0700 > From: Tony Craig > To: cf-metadata@cgd.ucar.edu > CC: Mariana Vertenstein , eschwab.e...@gmail.com > > The NCAR CCSM climate model tries to follow the CF standard > as much as possible with regard to input and output datasets. > However, we are finding it nearly impossible to adhere to the > calendaring standard. We do not have access to time managers > either internally or through external sources, such as ESMF, to > support the default CF "gregorian/standard" calendar as documented here, > > http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.4/cf-conventions.html#calendar > > And at the end of that section, the CF convention notes in particular > the challenges associated with "the discontinuity" in the mixed > gregorian/julian calendar. We have discussed calendaring issues with the > ESMF developers, and they also have no immediate plans to support > the mixed gregorian/julian calendar. ESMF is one of our primary tools > for supporting infrastructure in climate models, especially time managers. > > As a result, in our implementation, we treat the calendar naming > conventions differently. > > gregorian == proleptic_gregorian > > and mixed gregorian/julian is not supported. > > We would like to propose a change to the CF calendar conventions to > reflect what we believe the community is already doing. We propose > > 1. continued support of the name proleptic_calendar for backwards > compatability. > 2. changing the definition of gregorian and standard to be the proleptic > gregorian calendar instead of the mixed gregorian/julian. > 3. make the proleptic gregorian the default calendar > 4. add a new calendar name, such a gregorian_julian to provide the > ability to define a mixed calendar in CF conventions. > > that would change the definition in the above CF standards document to the > following > > gregorian or proleptic_gregorian or standard > A gregorian calendar extended to dates before 1582... > This is the default. > > gregorian_julian > Mixed Gregorian/Julian calendar as defined by Udunits. > > We've had discussions about this with ESMF and they are supportive > of this proposal. Their gregorian calendar is also proleptic and this > proposed change would make both their and our implementation more > closely aligned with the CF convention. > > If you have any questions, please feel free to contact me. thanks, > > tony.. > > > ___ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] add_offset for time dimension
ingrid allows it as well On Thu, Sep 16, 2010 at 2:36 PM, John Caron wrote: > On 9/16/2010 11:04 AM, Jonathan Gregory wrote: >> >> Dear Upendra >> >> add_offset is included in CF for data variables, but not coordinate >> variables. >> Its intention is to help with packing data (CF section 8.1). >> >> For time coordinate variables, however, an offset can effectively be >> included >> in the units, which have the form "UNITS since TIME". You can specify any >> reference TIME you like. Why would you like an offset for X and Y >> coordinates? >> >> Best wishes >> >> Jonathan >> ___ >> CF-metadata mailing list >> CF-metadata@cgd.ucar.edu >> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > > Does CF explicitly exclude using scale/offset on coordinate variables? The > CDM allows this. > ___ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] standard_name for coordinate variable corresponding to complex parts (real and imaginary)
This proposal is for a standard_name for the real/imaginary component, i.e. a way of coding complex data in a standard way -- talking about amplitude/phase is a total red herring, and fourier transform was merely an example, again, not the proposal. The fact remains that all software that is treating the data as complex will want both components -- if one is only interested in one part, then we are not talking about complex data any more. This discussion should go on to spherical harmonics, and EOFs, as Bryan suggested, not repeating the same Fourier example over and over. Maybe we could pick up on Fourier after talking about some of the other examples. I have non-standard structures for both spherical harmonics and EOFs. EOFs are fairly straightforward, a dataset with a number of related variables, e.g. http://iridl.ldeo.columbia.edu/expert/SOURCES/.CAC/.ssta/%7BY/cosd%7D%5BX/Y%5D%5BT%5Dsvd/ as opendap http://iridl.ldeo.columbia.edu/expert/SOURCES/.CAC/.ssta/%7BY/cosd%7D%5BX/Y%5D%5BT%5Dsvd/dods as netcdf http://iridl.ldeo.columbia.edu/expert/SOURCES/.CAC/.ssta/%7BY/cosd%7D%5BX/Y%5D%5BT%5Dsvd/data.cdf In this case the time series retain the original units, and the structures are normalized to r.m.s. 1. So the time series can be tagged by inheriting the tags of the original data, since all we have done is exchanged the spacial dependence (X,Y) for a modal dependence (ev). This would suggest new standard_names for the eigenvalue independent variable, and the normalized structures. As for spherical harmonics, it is an important example, because I am pretty sure almost everyone who stores them thinks of them as a compression of the data, or, as I put it earlier, a basis transformation. On the other hand, my experience is limited to triangular truncation and the velocity/temperature basis distinction, and the structure I use essentially tags it as such -- a modeler would be better suited to discussing what would be representations of the entire family of spherical harmonics. Benno On Fri, Jul 9, 2010 at 2:58 AM, Jonathan Gregory wrote: > Dear Bryan > > I may have overemphasised the point that you might be interested in only one > of the amplitude and phase. My main point is you might want to express a FT > as amplitude and phase, rather than real and imaginary. Since amplitude and > phase don't have the same units, I think it would be against CF conventions > to give them the same standard name and put them in the same data variable. > The same treatment should apply to real and imaginary. But you might not agree > with that argument on grounds of consistency. > > Although it may look clunky, I think using standard names to distinguish the > two parts of the FT is actually clearer - more self-describing - than using an > index coordinate variable with arbitrary values (such as 0=real 1=imaginary - > or would it be the other way round - and what if they are amplitude and > phase?) > to do it. That's another reason for my preference. > > Best wishes > > Jonathan > ___ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] standard_name for coordinate variable corresponding to complex parts (real and imaginary)
Hi Bryan, Thanks for chiming in -- your comments are quite helpful. As for On Thu, Jun 24, 2010 at 6:45 AM, Bryan Lawrence wrote: > However, despite the discussion thus far, I'm not entirely sure I > understand exactly what Benno is proposing, not least because there are > different ways of arranging FT components in an array - or exactly what > Jonathan is implying for spherical harmonics. My original proposal was Add a standard_name complex_parts to "tag" the dimension that corresponds to complex parts. so that we could tag a coordinate variable of a real-valued netcdf variable as corresponding to the real/imaginary direction, e.g. complex sst(C,X,Y,F) would have dimensions: C = 2; Y = 30 ; X = 84 ; F = 399 ; variables: integer C(C) ; C:standard_name = "complex_parts" ; float Y(Y) ; Y:standard_name = "latitude" ; Y:units = "degree_north" ; float X(X) ; X:standard_name = "longitude" ; X:units = "degree_east" ; float F(F) ; F:standard_name = "frequency" ; TFunits = "1/sec" ; float ssta(Y, X, F, C) ; ssta:long_name = "Sea Surface Temperature Anomaly" ; The conversation then got rapidly more complicated. -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] standard_name for coordinate variable corresponding to complex parts (real and imaginary)
Clearly stated, wish I could return the favor. However, 1) Fourier transform is a change of basis, not physical variable -- it is invertible as a matter of fact, as long as you keep the real and imaginary parts. Because it is a change of basis, it is much more analagous to a projection transformation than a transformation of the physical variable. There is a change of units, you are right, but that is misleading if it leads you to think that more than a change of basis has occured. I suppose if you were really stubborn about it, we could transmit discrete Fourier transforms, but that seems a little silly. Besides, the units part of standard_name is the weakest part of the framework -- relatively little data with the scales I am interested in atm/ocean is exchanged with time in seconds, for example, despite the standard_name constraint on time. As it should be. 2) Software that manipulates complex numbers, (including FFT), needs both the real and imaginary parts -- putting them in separate variables is a highly artificial reconstruction of the data which has to be undone in order to proceed with any manipulation that treats it as complex numbers. Also, the software needs to detect the components of the complex numbers -- you are making it much more difficult. The point of metadata is to make the necessary information available clearly and unambiguously -- having a dimension clearly labelled as corresponding to real and imaginary is much closer to the information needed to write software to correctly handle complex data. 3) CF should handle spectral harmonics as well. Benno On Tue, Jun 15, 2010 at 3:24 AM, Jonathan Gregory wrote: > Dear Benno > >> This time you have understood > Good. > >> I believe my proposal is a lot more useful. > > Our proposals differ in two ways: > > * I suggest the standard name should not be air_pressure, but should include > a phrase like fourier_transform_of_air_pressure_wrt_time. Basically that's > because if you do a Fourier transform, it's not the same physical quantity and > doesn't have the same units as air pressure, so it needs a new standard name. > > * I suggest that the imaginary and real components should be in different > data variables, distinguished by the standard name. That's just the same as > we do for spatial components of vectors and tensors. When you search a file > to find the quantity you want, you can identify the real or imaginary part > by looking at the standard name. I think that's as easy, or easier, than > finding a dimension for real/imaginary and checking the coordinates. > > Cheers > > Jonathan > >> On Mon, Jun 14, 2010 at 1:42 PM, Jonathan Gregory >> wrote: >> > Dear Benno >> > >> > You mean C is either "real" or "imaginary"? I think the CF-like way to do >> > this would be to have two different data variables for it, one for the real >> > part, one for the imaginary. That's like having different standard names >> > for spatial components, as we do, rather than dimensions for components. >> > As for spatial components, where "eastward", "upward", etc. are in the >> > standard name, here you would put "real" or "imaginary" in the standard >> > name. >> > The standard name would then be not air_pressure but something like >> > real|imaginary_part_of_fourier_transform_of_air_pressure_wrt_time >> > I'd suggest. >> > >> > I hope I have understood now, but maybe not! Best wishes >> > >> > Jonathan > -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] standard_name for coordinate variable corresponding to complex parts (real and imaginary)
This time you have understood, and I believe my proposal is a lot more useful. Benno On Mon, Jun 14, 2010 at 1:42 PM, Jonathan Gregory wrote: > Dear Benno > > You mean C is either "real" or "imaginary"? I think the CF-like way to do > this would be to have two different data variables for it, one for the real > part, one for the imaginary. That's like having different standard names > for spatial components, as we do, rather than dimensions for components. > As for spatial components, where "eastward", "upward", etc. are in the > standard name, here you would put "real" or "imaginary" in the standard name. > The standard name would then be not air_pressure but something like > real|imaginary_part_of_fourier_transform_of_air_pressure_wrt_time > I'd suggest. > > I hope I have understood now, but maybe not! Best wishes > > Jonathan > -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] standard_name for coordinate variable corresponding to complex parts (real and imaginary)
I am sorry, you misunderstand me: I am looking for the standard_name to label the coordinate variable that corresponds to real and imaginary parts, not the non-coordinate-variable that has real and imaginary parts. For example, If I had a Fourier transform of 3d air_pressure, it would be a real function of C,freq,lon,lat, where air_pressure has standard_name air_pressure, lon has standard_name longitude, lat has standard_name latitude, freq has standard_name frequency, i.e. inverse time, and C has standard_name complex_parts Benno On Mon, Jun 14, 2010 at 3:21 AM, Jonathan Gregory wrote: > Dear Benno > > In a standard name, more information is needed, to identify the quantity whose > real and imaginary parts you want to name. > > Best wishes > > Jonathan > -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
[CF-metadata] standard_name for coordinate variable corresponding to complex parts (real and imaginary)
Hello All, We have made some use of having a coordinate-variable that corresponds to real and imaginary parts, for outputs of Fourier transforms, for example. We usually tag this name "C" and long_name="complex", and give it values of "R" and "I", but it would be appropriate to tag it with a standard_name as well. Perhaps "complex" or "complex_parts"? Benno -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Cell bounds associated with coordinate variable rather than data variable
On Fri, Nov 13, 2009 at 9:11 AM, John Caron wrote: > > > > I think another possibility is to put the bounds information on the > "continuous integral" data variables, rather than on the time coordinate. > Otherwise we have this proliferation of time coordinates, which confuses > things. > > I am sorry -- the proliferation of time coordinates describes things, it does not confuse things. Mislabelling the data w.r.t. time confuses things. This is particularly true of models which put variables of vastly different time meanings on the same grid, and never explain what they have done. An instantaneous sample is considerably different from an integration over the interval (careful time analysis will undoubtably reveal a sub-gridpoint time lag). This is not just a model problem -- current meters once upon a time calculated average speed and instantaneous direction, which messed up calculations of component velocities. Also, thermisters have thermal mass, also introducing a lag with respect to other "simultaneously" measured quantities. We are just beginning to describe data better -- this is no time to perpetuate mistakes. -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Cell bounds associated with coordinate variable rather than data variable
I have to vote for multiple dimensions here -- these are different, parallel dimensions which happen to have the same number of points. Also, it is not true that a generic program cannot use the bounds. Ingrid, in fact, prints time almost always as an interval, e.g. Jan 1980 if the time cell goes from 1 Jan 1980 to 1 Feb 1980, so that cell boundaries are critical for time, much more important that spacially. This is because in common usage when we specify a time we almost always implicitly specify an interval. Also, it is not particularly natural to use the end point as the label for the data, certainly not so natural that a program would assume that not knowing anything else about the data. It makes sense from a data collection point of view, maybe, but stops there. Most of us use center of interval as default, particularly since our semantics are not good enough to detect accumulation (which is the only case where end of interval is natural). Benno On Thu, Oct 22, 2009 at 6:08 PM, Ethan Davis wrote: > Hi John, > > John Caron wrote: > > > > The time coordinate here means forecast time, and you are trying to > > capture "interval of accumulation". > > I'm not sure I understand your point here. The forecast time is the only > time dimension we have. What other time coordinate would the "interval > of accumulation" be over? > > > As jonathan says, you would have to create separate time coordinates for > > each variable which has a distinct bounds. > > Yes, CF is currently written so only one boundary variable can be > associate with one coordinate variable so this situation would require > multiple time coordinates. I'm wondering if CF should be extended to > allow multiple boundary variables to be associated with a single > coordinate variable. > > > theoretically theres no > > problem with that, practically it may be more confusing than just > > documenting the bounds on the variable in a non-standard but > > human-readable way. it seems unlikely that a generic program could do > > anything useful with those coordinate bounds. > > I agree that multiple time coordinates would do nothing for clarity. > Which is why I'm wondering about extending CF to allow for a clearer, > programmatically useful representation of this data. > > Ethan > > > > > _______ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > -- Dr. M. Benno Blumenthal be...@iri.columbia.edu International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] a different (but perhaps unoriginal) approach to standard name construction
On Mon, Nov 3, 2008 at 9:34 AM, Jonathan Gregory <[EMAIL PROTECTED]> wrote: > Dear Bryan > > > > What we disagree on is where the effort is mostly spent. I know for sure that > most of the large amount of time I have myself spent on them has been on > understanding the concepts and how they relate to each other, in order to > describe them clearly and consistently. It seems to me that the majority of > the time is required on the new names which don't conform to existing syntax > or > vocabulary. Perhaps you do agree with that, though? > I think the issue is the above effort is not written down in a way that the rest of us can utilize. Precise identifiers are find for data providers to tag their data, but in order for those precise tags to be useful for search/interface building they need to be related to each other and their component concepts. This is precisely way I have been pushing to establish URIs for the concepts in CF, including standard names, so that machine-readable documents can be produced that write down that structure. Benno -- Dr. M. Benno Blumenthal [EMAIL PROTECTED] International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] CF standard names for chemical constituents and aerosols (resulting from a GRIB2 p
Let me just add that a URI datatype is useful not just for external references, but it is also useful for local references as well, e.g. instead of cf:grid_mapping being a string that is the name of a local variable, it would be a URI that points to a local variable (i.e. probably still that same variable name). Seemingly a minor distinction, but it allows a clean mapping into an object-oriented framework for information in netcdf files. LIke RDF, or Java Beans, or ... Benno On Tue, Oct 21, 2008 at 11:24 AM, Benno Blumenthal <[EMAIL PROTECTED]> wrote: > Hello All, > > This is heading directly towards where I was trying to steer > common_concept in particular (Trac Ticket #24), and where the > namespace discussion went more generally (Trac Ticket #27). There is > a need for a framework for using namespaces in netcdf, both with the > attributes (so attributes can be explicitly placed in different > conventions, not the issue here per se), and attribute values (so that > attributes can point to URI's identifying concepts). My short > version of this is that cf:standard_name is a property that points to > a single controlled vocabulary (i.e. a string chosen from a list), > while the proposed cf:common_concept is a property that points to > concepts identified with URI's (e.g. namespace plus value). > > standard_name's could have URI's as well (Roy Lowry has assigned > opaque URI's to all the standard names, MMI has assigned more readable > URIs to many of the standard_names (e.g. cf:air_temperature), CF could > very well get a URN space of its own which will allow it to create > some definitive URIs, which lets us relate cf:standard_name's to other > vocabularies (Roy has done this as well), and would let us write down > relationships between standard_names (Roy is working on this). > > The attribute namespace problem is easier, and we discussed a couple > of possible solutions which we hope to sort out by actually > implementing them. namespaces for values seems trickier at the > moment, because netcdf (unlike DAP) does not have a URI datatype, and > so even if one had a list of namespaces specified as id #24, one would > not be sure which string-valued properties have been > namespace-abbreviated without reading through some document that > describes each property in the convention, a level of sophistication > in the reading software we would like to avoid. Of course, one does > not need to abbreviate -- a URN in particular should be quite readable > in its entirety. > > Benno > > On Mon, Oct 20, 2008 at 3:22 PM, John Graybeal <[EMAIL PROTECTED]> wrote: >> Seth, >> >> That's an entertainingly close description of the semantic interoperability >> framework [1] and workshop [2] that MMI is proposing and offering for marine >> and related environmental sciences. (apologies for the plug, but it sure >> seemed relevant) So you'll have my full support for that concept. >> >> The interesting and subtle distinction between MMI's approach, and what I >> think you're implying is that there would be a single authority (CF) tying >> together all those names (e.g., using the CF aliases to bring the names from >> different namespaces together). There are two ways to do that, but I don't >> think aliases or a 'single authority' (in the way you imply it) is the right >> model. >> >> I think a better model is to consider the CF standard names one of the major >> namespaces, to which many other vocabularies can be mapped. Then, modify >> the CF standard to allow the specification of a term from any namespace >> (since many terms may not be appropriate to the CF vocabulary; see my >> comment on CF scope in the last post); this is a proposal I've been >> back-burnering for about 9 months now. Finally, CF can accept any or all >> well-framed names that are in any of these namespaces, or can even alias to >> them, through manual or semi-automated processes. >> >> Under this model, CF has a mind share on solving this problem with a >> particular approach, but CF users can take advantage of other approaches. CF >> doesn't have to take on the 'vocabulary publication and mediation' task for >> the entire community, but can pick and choose its own targets for inclusion >> in the CF vocabulary (and the targets will be much more conspicuous and >> contextually defined). >> >> There are multiple complexities that arise as you start trying to make this >> concept (mine, yours, or CF's) work -- I like to think our MMI plans know >> about 85% of the issues in a fully operational but basic system -- and it >> isn&#x
Re: [CF-metadata] CF standard names for chemical constituents and aerosols (resulting from a GRIB2 p
rg/semanticframeworkconcept (many of the planned > concepts are not fully documented yet) > [2] http://marinemetadata.org/events/oossi (sign up soon if you want good > hotel rates!) > > On Oct 20, 2008, at 11:40 AM, Seth McGinnis wrote: > >>> I don't know if this is supportive or not, but the "sheer number of >>> groups bringing new requirements" is an issue that I don't think CF has >>> realistically addressed. >> >> >> With regard to the standard names problem, here's a perhaps-mad idea from >> someone standing on the sidelines. I don't think this idea has been floated >> before, but I may have missed it. >> >> Where I'm coming from: I'm the data manager for a modeling project >> (NARCCAP, http://narccap.ucar.edu) that's using the CF standard. I don't >> generate or use any of the project's data, I'm just in charge of checking it >> to make sure it's properly formatted and trying to make things usable for >> the community that will use the data. From my perspective, the really >> important thing about standard names is that if two different files have a >> variable with the same standard name, it means (1) they're talking about the >> same thing, and (2) a user can go somewhere and look up the definition and >> the canonical units. The fact that the name has passed through an approval >> process is entirely incidental. >> >> So... why not just let any group that wants to define a set of standard >> names for their community do so, as long as they're willing to also take >> charge of publishing a Standard Name Table for it? >> >> Essentially, it would be creating namespaces. The only mechanism that >> would need to be added would be some way of noting which namespace the name >> comes from and the URL of the corresponding Standard Name Table. So, for >> example, maybe for chemical names, all the standard names would all start >> with "iupac:" and somewhere in the global attributes for the file you'd have >> "standard_namespaces = (default: >> cf-pcmdi.llnl.gov/cf-standard-name-table.xml, chem: >> www.iupac.org/cf/standard-names-3.0.xml)". Or something like that. >> >> Each community can manage its own namespace. There's already a mechanism >> in place (aliases) that could be extended to handle cases where there are >> two different names for one thing. And if you end up with two names for the >> same thing with different dimensions or slightly different definitions, >> that's fine, because that situation already exists in the single table. >> (Rain, for example, can be defined as a mass flux per unit area >> (precipitation_flux) or as an accumulated depth >> (lwe_thickness_of_precipitation_amount). They're different ways of talking >> about the same physical thing. So this is not a new problem for the user.) >> >> Put some bounds on the representational domain that the default standard >> name table will handle, delegate everything else to whoever in that >> community wants to deal with it, and no matter how many groups come to CF >> with new standard names, the system can handle it. Right? >> >> Cheers, >> >> --Seth >> >> >> Seth McGinnis >> [EMAIL PROTECTED] >> NARCCAP Data & User Community Manager >> Associate Scientist >> ISSE / NCAR >> >> ___ >> CF-metadata mailing list >> CF-metadata@cgd.ucar.edu >> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > > > John > > -- > John Graybeal <mailto:[EMAIL PROTECTED]> -- 831-775-1956 > Monterey Bay Aquarium Research Institute > Marine Metadata Interoperability Project: http://marinemetadata.org > > ___ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > -- Dr. M. Benno Blumenthal [EMAIL PROTECTED] International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
