Re: [CF-metadata] CF point observation Conventions ready for review
Hi Sara:
Thanks for adding your example to the mix. This appears to be a timeseries at a
single x,y location with all the measurements on single, but different levels
(?)
Because these are single level, I would be inclined to use a variation of the station
representation. One possibility is to put each z coordinate in its own coordinate using a
"scalar dimension" (dimension with length 1) :
dimension
z_inst1 = 1;
z_inst2 = 1;
z_inst3 = 1;
float lon;
float lat;
float z_inst1(z_inst1);
float z_inst2(z_inst2);
float z_inst3(z_inst3);
long time(time);
float wspd(time,z_inst1);
float wdir(time,z_inst1);
float atemp(time,z_inst2);
float SST(time,z_inst3);
Sara Haines wrote:
Dear Nan and John:
I've been monitoring this conversation for just a short period. So I
apologize ahead for chiming in so late in the game.
It is very important for observational data model to represent the
actual system that is deployed. The point that Nan and others have
raised is a need for a container to represent multiple measurement
heights for several sensors at a station (e.g. met above the ocean
surface, near-surface and near-bottom ocean temperature and salinity).
You all seem in agreement on this and that it is not a simple side-bar
case. Similar to what Nan described was kludged in the SEACOOS CDL v2.0
netcdf format because no case like moorings and buoys with multiple
point measurements was covered in CF-conventions back in 2004. We
defined the requirement that the point-measurement sensors be associated
with the same location or site. SEACOOS CDL v2.0 was adopted by SECOORA
but most definitely needs revisiting. It looks like this:
netcdf fixed-point {
// This is a fixed-point station with dependent variables
// measured by sensors at different heights.
//
// All sensors are sampling at the same time.
//(1) x and y and sensor-specific z are repeated for each time.
//(2) Assign z dimension equal to number of z measurement levels
//(3) Assign values for z levels in the data section
//(4) Use variable attribute called "z" under dependent variable to
// associate specific z of point measurement
//(5) Use z variable to describe all the required attributes
// without having to repeat under each dependent variable
dimensions:
time = 4;
lon = 1;
lat = 1;
z = 3;
variables:
// INDEPENDENT VARIABLES
long time(time);
time: standard_name = "time";
time: units = "sec since 1970-1-1 00:00:00";
time: axis = "T";
float lon(lon);
lon: standard_name = "longitude";
lon: units = "degrees_east" ;
lon: reference = "geographical coordinates" ;
lon: axis = "X";
float lat(lat);
lat: standard_name = "latitude";
lat: units = "degrees_east";
lat: reference = "geographical coordinates";
lat: axis = "Y";
float z(z);
z: standard_name = "height";
z: units = "m";
z: reference = "mean sea level (MSL)";
z: positive = "up";
z: axis = "Z";
// DEPENDENT VARIABLES
float wspd(time);
wspd: long_name = "Wind Speed";
wspd: standard_name = "wind_speed";
wspd: units = "m s-1";
wspd: z = 45;
float wdir(time);
wdir: long_name= "Wind Direction (from)";
wdir: standard_name = "wind_from_direction";
wdir: units = "degrees";
wdir: reference = "clockwise from true north";
wdir: valid_range = 0.,360.;
wdir: z = 45;
float atemp(time);
atemp: long_name = "Air temperature";
atemp: standard_name = "air_temperature";
atemp: units = "degrees Celsius";
atemp: z = 24;
float SST(time);
SST: long_name = "Sea Surface Temperature";
SST: standard_name = "sea_surface_temperature";
SST: units = "degrees Celsius";
SST: z = -5;
// global attributes:
// (use all for adopted SEACOOS CDL v2.0)
:format_category = "fixed-point";
data:
time = 1062804600,1062808200,106281,1062811800;
lon = -82.543;
lat = 18.462702;
z = 45, 24, -5;
wspd = 9.80, 12.63, 16.52, 18.97;
wdir = 88.8,103.6,112.7,105.0;
atemp = 27.00, 27.23, 27.05, 26.97;
SST = 28.11, 28.17, 28.17, 28.35;
}
I understand your points about making lat and lon scalars. So ignore
that for just now. (Remember: This was developed 5 years ago!)
The variable attribute above ("SST: z = -5") should probably be an index
to the z variable to be less confusing. But I think you get the idea.
There is something workable for conventions of Point Observation Data
without being cobbling something together. I like many of the ideas in
this proposed convention, for example, multiple deployments at the same
site. It addresses most short-comings in previous CF-conventions for
non-gridded data and what is kludged in the Southeast Region.
I would most enjoy discussing this further. It will help me understand
what is being proposed. Right now, I'm mostly chiming in to underscore
Nan's need and the need to address it in this round of the review.
Thanks,
Sara
___
CF-metad
Re: [CF-metadata] CF point observation Conventions ready for review
Hi Roy, Nan: Let me try to see where the mapping between the two classifications is currently at: Lowry, Roy K wrote: Dear All, I come from Nan's community with the added complication of exposure to CSML through working with NDG. From this position in BODC we have developed a collection of feature type names that my intention is to map to the CF feature type names once John's work is complete. As I watch the development of the proposal I'm starting to get uneasy feelings that this mapping is getting harder as the feature type definitions become more fluid, which echoes Nan's feelings that the proposal is drifting away from her domain. The feature terms we use for observational data in BODC are: Profile - single set of measurements with single (by assumption) x, y, t values but varying spatial z. An example is a single, fully processed (i.e. binned) CTD cast. this is a single profile; a single feature in a file is considered a special case of multiple features in a file, with some special handling to allow scalar coordinate variables. Profile collection - an aggregation of profiles into a single data object. An example is all the CTDs from a section or a cruise. sounds like a "section", a connected set of profiles along a trajectory. If you dont care about the "along a trajectory" then its just a collection of profiles. Profile series - a set of measurements with single x,y a fixed set of spatial z values and progessing t. An example is a single moored ADCP deployment record. a single "stationProfile". station is synonomous with timeSeries, so we could just as well call it timeSeriesProfiles. A special case of a collection of stationProfile. Point - a set of measurements with single x, y, and z and progressing t. Example is a single moored recording current meter record. a special case of a stationTimeSeries collection, with only one station in the file. A station is a named x,y (optional z) location. Point collection - an aggregation of point features in a single container. Example is all the records from all the current meters on a mooring or deployed on a cruise. possibly a point collection, if you think of these as unconnected. Spectrum series - a set of measurements with single x,y a fixed set of non-spatial z values and progessing t. An example is a power spectrum time series from a wave recorder. a special case of a stationTimeSeries collection, with a non-georeferencing dimension. we would not refer to the wavelength dimension as z just to keep things clear. 2D-trajectory - a set of measurements with variable x, y, t and a single spatial z. Example is the thermosalinograph record from a cruise. A trajectory with a scalar z coordinate. 3D-trajectory - set of measurements with variable x, y, t and a variable spatial z. Example is the thermosalinograph record from an AUV mission. It is also applicable to a yo-yo CTD station, mirroring Chris's comments on atmospheric "profiles" with variant x,y. A trajectory with a z coordinate. I think that Nan and most of the observational oceanographic community recognise these concepts and consequently, if a mapping to them to your feature definitions is maintained then it will help keep us on board. Note that the difference between 'point' and 'point collection' is important to me as on observational data manager, which is a different perspective to an observational data ingestor. Im not sure if I have captured the difference correctly. What do you think? Cheers, Roy. Im not particularly attached to the names, as long as they are clear and can be explained easily. Nan suggested: You could change the list of types that cover profiles and time series from: * stationTimeSeries: a time-series of data points at the same location, with varying time * profile: a set of data points along a vertical line * stationProfile: a time-series of profiles at a named location to something like: * timeseries: a time-series of data points at one X,Y,Z with varying T * stationTimeseries: a time-series of data points at one X,Y, with multiple Z and T * profile: a set of data points along a vertical line * stationProfile: a time-series of profiles at a named location which i think amounts to renaming "stationTimeSeries" to "timeseries" (ok with me, anyone else care?). however "stationTimeseries: a time-series of data points at one X,Y, with multiple Z and T" probably needs a bit more thinking. Im not all that clear how the two differ. Is "timeseries" intended to be only for surface data? I think the main case thats not clear is station data with different z coordinates for different measurements. I still have to generate an example of that and see if we need to add something to the proposal. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] CF point observation Conventions ready for review
Roy, I come from this community also and agree that the feature types you mention need be representable as feature types in the CF Data Model to be effective for the oceangraphic community. Two minor clarifications: 1. > 3D-trajectory - set of measurements with variable x, y, t and a single spatial z. Example is the thermosalinograph record from an AUV mission. It is also applicable to a yo-yo CTD station, mirroring Chris's comments on atmospheric "profiles" with variant x,y. Did you mean variable z here? 2. So you don't distinguish between time series as a point and just scattered x,y,z points with no specific time, or each with single time? -Rich On Sun, Nov 15, 2009 at 7:23 AM, Lowry, Roy K wrote: > Dear All, > > I come from Nan's community with the added complication of exposure to CSML > through working with NDG. From this position in BODC we have developed a > collection of feature type names that my intention is to map to the CF > feature type names once John's work is complete. As I watch the development > of the proposal I'm starting to get uneasy feelings that this mapping is > getting harder as the feature type definitions become more fluid, which > echoes Nan's feelings that the proposal is drifting away from her domain. > > The feature terms we use for observational data in BODC are: > > Profile - single set of measurements with single (by assumption) x, y, t > values but varying spatial z. An example is a single, fully processed (i.e. > binned) CTD cast. > > Profile collection - an aggregation of profiles into a single data object. > An example is all the CTDs from a section or a cruise. > > Profile series - a set of measurements with single x,y a fixed set of spatial > z values and progessing t. An example is a single moored ADCP deployment > record. > > Point - a set of measurements with single x, y, and z and progressing t. > Example is a single moored recording current meter record. > > Point collection - an aggregation of point features in a single container. > Example is all the records from all the current meters on a mooring or > deployed on a cruise. > > Spectrum series - a set of measurements with single x,y a fixed set of > non-spatial z values and progessing t. An example is a power spectrum time > series from a wave recorder. > > 2D-trajectory - a set of measurements with variable x, y, t and a single > spatial z. Example is the thermosalinograph record from a cruise. > > 3D-trajectory - set of measurements with variable x, y, t and a single > spatial z. Example is the thermosalinograph record from an AUV mission. It > is also applicable to a yo-yo CTD station, mirroring Chris's comments on > atmospheric "profiles" with variant x,y. > > I think that Nan and most of the observational oceanographic community > recognise these concepts and consequently, if a mapping to them to your > feature definitions is maintained then it will help keep us on board. > > Note that the difference between 'point' and 'point collection' is important > to me as on observational data manager, which is a different perspective to > an observational data ingestor. > > Cheers, Roy. > > ____________ > From: cf-metadata-boun...@cgd.ucar.edu [cf-metadata-boun...@cgd.ucar.edu] On > Behalf Of Nan Galbraith [ngalbra...@whoi.edu] > Sent: 13 November 2009 17:42 > To: cf-metadata@cgd.ucar.edu > Subject: Re: [CF-metadata] CF point observation Conventions ready for review > > Hmm, I think this is an unfortunate choice of terms - we make a fairly > clear distinction between profile data and station data - but I don't think > it's just a local semantic problem. > > While I realize that the storage for multiple profiles and multiple-depth > time series data are similar, I still don't think this is a workable system > for us. If we are to treat each time stamp as a profile, then > float alt(profile, z) > indicates that we need a T-dimensioned depth, which is not just > inefficient, > but misleading - our depth variable is not measured and we don't KNOW > how it varies with time, for the most part. > > Going back to the semantics, moored station data are different from > profiles (even agreeing with your statement that profiles can have a time > that varies with Z) because measurements are not taken at evenly > distributed > depth and because, except in the special case of ADCPs or similar > profilers, > mooring data from each depth is taken with an independent instrument. These > may be of different types, have different sample rates, different output > parameters. > The parameters that do coincide may have
Re: [CF-metadata] CF point observation Conventions ready for review
Thanks Rich, (1) Perils of copy and paste after travel hell. Definitely meant variable z. Thanks for picking it up. (2) It's not that we don't distinguish, it's that measurements at fixed t,z with variable x,y is something we've never encountered and therefore have never given it a name. The closest we come is grab samples or surface water samples from a cruise, but we consider that as a 2-D trajectory - it's just lower frequency sampling than a TSG. If I did handle data like you describe I would ceratinly be looking for a different feature type label for them. Cheers, Roy. From: Richard Signell [rsign...@gmail.com] Sent: 15 November 2009 15:00 To: Lowry, Roy K Cc: ngalbra...@whoi.edu; cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] CF point observation Conventions ready for review Roy, I come from this community also and agree that the feature types you mention need be representable as feature types in the CF Data Model to be effective for the oceangraphic community. Two minor clarifications: 1. > 3D-trajectory - set of measurements with variable x, y, t and a single spatial z. Example is the thermosalinograph record from an AUV mission. It is also applicable to a yo-yo CTD station, mirroring Chris's comments on atmospheric "profiles" with variant x,y. Did you mean variable z here? 2. So you don't distinguish between time series as a point and just scattered x,y,z points with no specific time, or each with single time? -Rich On Sun, Nov 15, 2009 at 7:23 AM, Lowry, Roy K wrote: > Dear All, > > I come from Nan's community with the added complication of exposure to CSML > through working with NDG. From this position in BODC we have developed a > collection of feature type names that my intention is to map to the CF > feature type names once John's work is complete. As I watch the development > of the proposal I'm starting to get uneasy feelings that this mapping is > getting harder as the feature type definitions become more fluid, which > echoes Nan's feelings that the proposal is drifting away from her domain. > > The feature terms we use for observational data in BODC are: > > Profile - single set of measurements with single (by assumption) x, y, t > values but varying spatial z. An example is a single, fully processed (i.e. > binned) CTD cast. > > Profile collection - an aggregation of profiles into a single data object. > An example is all the CTDs from a section or a cruise. > > Profile series - a set of measurements with single x,y a fixed set of spatial > z values and progessing t. An example is a single moored ADCP deployment > record. > > Point - a set of measurements with single x, y, and z and progressing t. > Example is a single moored recording current meter record. > > Point collection - an aggregation of point features in a single container. > Example is all the records from all the current meters on a mooring or > deployed on a cruise. > > Spectrum series - a set of measurements with single x,y a fixed set of > non-spatial z values and progessing t. An example is a power spectrum time > series from a wave recorder. > > 2D-trajectory - a set of measurements with variable x, y, t and a single > spatial z. Example is the thermosalinograph record from a cruise. > > 3D-trajectory - set of measurements with variable x, y, t and a single > spatial z. Example is the thermosalinograph record from an AUV mission. It > is also applicable to a yo-yo CTD station, mirroring Chris's comments on > atmospheric "profiles" with variant x,y. > > I think that Nan and most of the observational oceanographic community > recognise these concepts and consequently, if a mapping to them to your > feature definitions is maintained then it will help keep us on board. > > Note that the difference between 'point' and 'point collection' is important > to me as on observational data manager, which is a different perspective to > an observational data ingestor. > > Cheers, Roy. > > ________________ > From: cf-metadata-boun...@cgd.ucar.edu [cf-metadata-boun...@cgd.ucar.edu] On > Behalf Of Nan Galbraith [ngalbra...@whoi.edu] > Sent: 13 November 2009 17:42 > To: cf-metadata@cgd.ucar.edu > Subject: Re: [CF-metadata] CF point observation Conventions ready for review > > Hmm, I think this is an unfortunate choice of terms - we make a fairly > clear distinction between profile data and station data - but I don't think > it's just a local semantic problem. > > While I realize that the storage for multiple profiles and multiple-depth > time series data are similar, I still don't think this is a
Re: [CF-metadata] CF point observation Conventions ready for review
Dear All, I come from Nan's community with the added complication of exposure to CSML through working with NDG. From this position in BODC we have developed a collection of feature type names that my intention is to map to the CF feature type names once John's work is complete. As I watch the development of the proposal I'm starting to get uneasy feelings that this mapping is getting harder as the feature type definitions become more fluid, which echoes Nan's feelings that the proposal is drifting away from her domain. The feature terms we use for observational data in BODC are: Profile - single set of measurements with single (by assumption) x, y, t values but varying spatial z. An example is a single, fully processed (i.e. binned) CTD cast. Profile collection - an aggregation of profiles into a single data object. An example is all the CTDs from a section or a cruise. Profile series - a set of measurements with single x,y a fixed set of spatial z values and progessing t. An example is a single moored ADCP deployment record. Point - a set of measurements with single x, y, and z and progressing t. Example is a single moored recording current meter record. Point collection - an aggregation of point features in a single container. Example is all the records from all the current meters on a mooring or deployed on a cruise. Spectrum series - a set of measurements with single x,y a fixed set of non-spatial z values and progessing t. An example is a power spectrum time series from a wave recorder. 2D-trajectory - a set of measurements with variable x, y, t and a single spatial z. Example is the thermosalinograph record from a cruise. 3D-trajectory - set of measurements with variable x, y, t and a single spatial z. Example is the thermosalinograph record from an AUV mission. It is also applicable to a yo-yo CTD station, mirroring Chris's comments on atmospheric "profiles" with variant x,y. I think that Nan and most of the observational oceanographic community recognise these concepts and consequently, if a mapping to them to your feature definitions is maintained then it will help keep us on board. Note that the difference between 'point' and 'point collection' is important to me as on observational data manager, which is a different perspective to an observational data ingestor. Cheers, Roy. From: cf-metadata-boun...@cgd.ucar.edu [cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Nan Galbraith [ngalbra...@whoi.edu] Sent: 13 November 2009 17:42 To: cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] CF point observation Conventions ready for review Hmm, I think this is an unfortunate choice of terms - we make a fairly clear distinction between profile data and station data - but I don't think it's just a local semantic problem. While I realize that the storage for multiple profiles and multiple-depth time series data are similar, I still don't think this is a workable system for us. If we are to treat each time stamp as a profile, then float alt(profile, z) indicates that we need a T-dimensioned depth, which is not just inefficient, but misleading - our depth variable is not measured and we don't KNOW how it varies with time, for the most part. Going back to the semantics, moored station data are different from profiles (even agreeing with your statement that profiles can have a time that varies with Z) because measurements are not taken at evenly distributed depth and because, except in the special case of ADCPs or similar profilers, mooring data from each depth is taken with an independent instrument. These may be of different types, have different sample rates, different output parameters. The parameters that do coincide may have different attributes and be measured and processed differently. A moored buoy may have only surface and bottom data, and can't be used to characterize the water column between its Z limits - which is the purpose of a profile. One domain scientist commented, when shown this proposal, that a very good time series would be turned into a very bad set of profiles. If you'd like to include moored stations in this convention, you could change the list of types that cover profiles and time series from: * stationTimeSeries: a time-series of data points at the same location, with varying time * profile: a set of data points along a vertical line * stationProfile: a time-series of profiles at a named location to something like: * timeseries: a time-series of data points at one X,Y,Z with varying T * stationTimeseries: a time-series of data points at one X,Y, with multiple Z and T * profile: a set of data points along a vertical line * stationProfile: a time-series of profiles at a named location although these terms for the types are not very exact and are actually confusing, with "
Re: [CF-metadata] CF point observation Conventions ready for review
Dear Nan and John:
I've been monitoring this conversation for just a short period. So I
apologize ahead for chiming in so late in the game.
It is very important for observational data model to represent the
actual system that is deployed. The point that Nan and others have
raised is a need for a container to represent multiple measurement
heights for several sensors at a station (e.g. met above the ocean
surface, near-surface and near-bottom ocean temperature and salinity).
You all seem in agreement on this and that it is not a simple side-bar
case. Similar to what Nan described was kludged in the SEACOOS CDL v2.0
netcdf format because no case like moorings and buoys with multiple
point measurements was covered in CF-conventions back in 2004. We
defined the requirement that the point-measurement sensors be associated
with the same location or site. SEACOOS CDL v2.0 was adopted by SECOORA
but most definitely needs revisiting. It looks like this:
netcdf fixed-point {
// This is a fixed-point station with dependent variables
// measured by sensors at different heights.
//
// All sensors are sampling at the same time.
//(1) x and y and sensor-specific z are repeated for each time.
//(2) Assign z dimension equal to number of z measurement levels
//(3) Assign values for z levels in the data section
//(4) Use variable attribute called "z" under dependent variable to
// associate specific z of point measurement
//(5) Use z variable to describe all the required attributes
// without having to repeat under each dependent variable
dimensions:
time = 4;
lon = 1;
lat = 1;
z = 3;
variables:
// INDEPENDENT VARIABLES
long time(time);
time: standard_name = "time";
time: units = "sec since 1970-1-1 00:00:00";
time: axis = "T";
float lon(lon);
lon: standard_name = "longitude";
lon: units = "degrees_east" ;
lon: reference = "geographical coordinates" ;
lon: axis = "X";
float lat(lat);
lat: standard_name = "latitude";
lat: units = "degrees_east";
lat: reference = "geographical coordinates";
lat: axis = "Y";
float z(z);
z: standard_name = "height";
z: units = "m";
z: reference = "mean sea level (MSL)";
z: positive = "up";
z: axis = "Z";
// DEPENDENT VARIABLES
float wspd(time);
wspd: long_name = "Wind Speed";
wspd: standard_name = "wind_speed";
wspd: units = "m s-1";
wspd: z = 45;
float wdir(time);
wdir: long_name= "Wind Direction (from)";
wdir: standard_name = "wind_from_direction";
wdir: units = "degrees";
wdir: reference = "clockwise from true north";
wdir: valid_range = 0.,360.;
wdir: z = 45;
float atemp(time);
atemp: long_name = "Air temperature";
atemp: standard_name = "air_temperature";
atemp: units = "degrees Celsius";
atemp: z = 24;
float SST(time);
SST: long_name = "Sea Surface Temperature";
SST: standard_name = "sea_surface_temperature";
SST: units = "degrees Celsius";
SST: z = -5;
// global attributes:
// (use all for adopted SEACOOS CDL v2.0)
:format_category = "fixed-point";
data:
time = 1062804600,1062808200,106281,1062811800;
lon = -82.543;
lat = 18.462702;
z = 45, 24, -5;
wspd = 9.80, 12.63, 16.52, 18.97;
wdir = 88.8,103.6,112.7,105.0;
atemp = 27.00, 27.23, 27.05, 26.97;
SST = 28.11, 28.17, 28.17, 28.35;
}
I understand your points about making lat and lon scalars. So ignore
that for just now. (Remember: This was developed 5 years ago!)
The variable attribute above ("SST: z = -5") should probably be an index
to the z variable to be less confusing. But I think you get the idea.
There is something workable for conventions of Point Observation Data
without being cobbling something together. I like many of the ideas in
this proposed convention, for example, multiple deployments at the same
site. It addresses most short-comings in previous CF-conventions for
non-gridded data and what is kludged in the Southeast Region.
I would most enjoy discussing this further. It will help me understand
what is being proposed. Right now, I'm mostly chiming in to underscore
Nan's need and the need to address it in this round of the review.
Thanks,
Sara
--
***
* Sara Haines * 919-962-1253 (off) *
* Department of Marine Sciences * 919-962-1254 (fax) *
* University of North Carolina * sara_hai...@unc.edu *
* Chapel Hill, NC 27599-3300* *
***
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] CF point observation Conventions ready for review
Nan Galbraith wrote: Hi John - This file has a single variable from a single mooring deployment. A complete mooring deployment file would have 3 (or 4) depth coordinate variables, one each for temperature, velocity, and salinity measurements, since we measure temperature everywhere and add velocity or salinity at selected depths; it would also have met data, which would either have a singleton depth dimension or none - but would have sensor heights provided as variable attributes. For the OceanSITES project, we're limited to a single coordinate variable for Z, which is why I'm building multiple files for each deployment - at least at the moment. The sample file also represents a single deployment at a site; a nearly identical mooring is deployed every year at nominally the same location, with some changes to the depths or to the types of instruments (and so, different variables) at these depths. Being able to combine all the deployments for a single site could be a big help in organizing this data, although at present it's not possible to put even one full deployment into a single file in the OceanSITES system. Hope this isn't too much information... Cheers - Nan Yes, we see this same issue in wind profiler data - several depth coordinates for the same instrument. This should work in this CF proposal, though I confess that I dont yet deal with this case in my CDM / IDV implementation. Ill send out a sample of this ASAP. Also, taking the lat/lon dimension out of the data variables allows multiple buoys to be in the same file. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] CF point observation Conventions ready for review
Hi John - This file has a single variable from a single mooring deployment. A complete mooring deployment file would have 3 (or 4) depth coordinate variables, one each for temperature, velocity, and salinity measurements, since we measure temperature everywhere and add velocity or salinity at selected depths; it would also have met data, which would either have a singleton depth dimension or none - but would have sensor heights provided as variable attributes. For the OceanSITES project, we're limited to a single coordinate variable for Z, which is why I'm building multiple files for each deployment - at least at the moment. The sample file also represents a single deployment at a site; a nearly identical mooring is deployed every year at nominally the same location, with some changes to the depths or to the types of instruments (and so, different variables) at these depths. Being able to combine all the deployments for a single site could be a big help in organizing this data, although at present it's not possible to put even one full deployment into a single file in the OceanSITES system. Hope this isn't too much information... Cheers - Nan Hi Han: Ok, let me understand what you have before I comment on whether it fits in: ... It looks like you have a single moored buoy in this file. The 1D lat, lon I think are clearer as scalars : float latitude ; latitude:axis = "Y" ; float longitude ; longitude:axis = "X" ; float time(time) ; time:axis = "T" ; float depth(depth) ; depth:axis = "Z" ; float STMP(time, depth) ; STMP:standard_name = "sea_water_temperature" ; short INST_SN(depth) ; INST_SN:long_name = "instrument_serial_number" ; So it appears that you have the same depth coordinate for each measurement, and some data which is independent of the time, but z dependent. Am I missing anything? -- *** * Nan Galbraith(508) 289-2444 * * Upper Ocean Processes GroupMail Stop 29 * * Woods Hole Oceanographic Institution* * Woods Hole, MA 02543* *** ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] CF point observation Conventions ready for review
Hi Han:
Ok, let me understand what you have before I comment on whether it fits in:
netcdf OS_NTAS-2009_T_R {
dimensions:
time = UNLIMITED ; // (3033 currently)
depth = 5 ;
latitude = 1 ;
longitude = 1 ;
variables:
float time(time) ;
time:axis = "T" ;
float depth(depth) ;
depth:axis = "Z" ;
float latitude(latitude) ;
latitude:axis = "Y" ;
float longitude(longitude) ;
longitude:axis = "X" ;
float STMP(time, depth,latitude,longitude) ;
STMP:standard_name = "sea_water_temperature" ;
short INST_SN(depth) ;
INST_SN:long_name = "instrument_serial_number" ;
It looks like you have a single moored buoy in this file.
The 1D lat, lon I think are clearer as scalars :
float latitude ;
latitude:axis = "Y" ;
float longitude ;
longitude:axis = "X" ;
float time(time) ;
time:axis = "T" ;
float depth(depth) ;
depth:axis = "Z" ;
float STMP(time, depth) ;
STMP:standard_name = "sea_water_temperature" ;
short INST_SN(depth) ;
INST_SN:long_name = "instrument_serial_number" ;
So it appears that you have the same depth coordinate for each measurement, and
some data which is independent of the time, but z dependent. Am I missing
anything?
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] CF point observation Conventions ready for review
Hmm, I think this is an unfortunate choice of terms - we make a fairly
clear distinction between profile data and station data - but I don't think
it's just a local semantic problem.
While I realize that the storage for multiple profiles and multiple-depth
time series data are similar, I still don't think this is a workable system
for us. If we are to treat each time stamp as a profile, then
float alt(profile, z)
indicates that we need a T-dimensioned depth, which is not just
inefficient,
but misleading - our depth variable is not measured and we don't KNOW
how it varies with time, for the most part.
Going back to the semantics, moored station data are different from
profiles (even agreeing with your statement that profiles can have a time
that varies with Z) because measurements are not taken at evenly
distributed
depth and because, except in the special case of ADCPs or similar
profilers,
mooring data from each depth is taken with an independent instrument. These
may be of different types, have different sample rates, different output
parameters.
The parameters that do coincide may have different attributes and be
measured
and processed differently.
A moored buoy may have only surface and bottom data, and can't be used
to characterize the water column between its Z limits - which is the
purpose
of a profile. One domain scientist commented, when shown this proposal,
that a very good time series would be turned into a very bad set of
profiles.
If you'd like to include moored stations in this convention, you could
change
the list of types that cover profiles and time series from:
* stationTimeSeries: a time-series of data points at the same
location, with varying time
* profile: a set of data points along a vertical line
* stationProfile: a time-series of profiles at a named location
to something like:
* timeseries: a time-series of data points at one X,Y,Z with varying T
* stationTimeseries: a time-series of data points at one X,Y, with
multiple Z and T
* profile: a set of data points along a vertical line
* stationProfile: a time-series of profiles at a named location
although these terms for the types are not very exact and are actually
confusing,
with "station" just indicating "multiple" - not too meaningful.
Here's the structure of one of our OceanSITES files, with unrelated
fields omitted -
netcdf OS_NTAS-2009_T_R {
dimensions:
time = UNLIMITED ; // (3033 currently)
depth = 5 ;
latitude = 1 ;
longitude = 1 ;
variables:
float time(time) ;
time:axis = "T" ;
float depth(depth) ;
depth:axis = "Z" ;
float latitude(latitude) ;
latitude:axis = "Y" ;
float longitude(longitude) ;
longitude:axis = "X" ;
float STMP(time, depth,latitude,longitude) ;
STMP:standard_name = "sea_water_temperature" ;
short INST_SN(depth) ;
INST_SN:long_name = "instrument_serial_number" ;
Cheers - Nan
Alternatively, it seems like you could use a stationTimeSeries and
just have different locations (stations) to indicate the different
depths. The x/y would be the same, but the z would be different for
each location. It sounds like you are interested in a time series at
each depth rather than looking at a timeseries of the entire column.
In the latter, I think the stationProfile would fit the bill.
Don
John Caron wrote:
Nan Galbraith wrote:
IF this is intended to be used for time series at multiple depths,
then the
definition of "stationTimeSeries" would need to be changed from "a
time-series of data points at the same location" to "a time-series
of data
points at the same X-Y location" - to make it more clear that this type
can hold time series data at multiple depths. The example would also
need to show how a dependent variable would be dimensioned to hold
2d data at each station.
In recent discussions on this list, it looks almost like there's
some idea
that the "profile" feature type should be used for this kind of
data. That
would be a very hard sell. The term profile, as used in the
observational
community, is very different from time series - most importantly, it
indicates that Z varies with T. If moored station data is meant to be
covered by this specification, and if it won't fit into the
"station" feature
type, a term other than profile should be used.
Hi Nan:
The proposal is intended to work with surface moorings, so if it
doesnt work, lets understand why.
I think your data would probably be classified as stationProfile
data: "a time-series of profiles at a named location". A single file
can contain one or a collection of surface moorings data. See section
9.6 for representations.
I think you are being misled by the word "profile", which apparently
has a different meaning in your community. In this proposal, a
profile is a series of point measurements at different z, with the
same x,y. There may be a single time for each profile, or time may
depend on z. It
Re: [CF-metadata] CF point observation Conventions ready for review
Nan- Alternatively, it seems like you could use a stationTimeSeries and just have different locations (stations) to indicate the different depths. The x/y would be the same, but the z would be different for each location. It sounds like you are interested in a time series at each depth rather than looking at a timeseries of the entire column. In the latter, I think the stationProfile would fit the bill. Don John Caron wrote: Nan Galbraith wrote: Steve Hankin wrote: Its approaching two weeks (Oct 27) since a revised "CF point observation Conventions" proposal was made: https://cf-pcmdi.llnl.gov/trac/wiki/PointObservationConventions https://cf-pcmdi.llnl.gov/trac/ticket/37 Given the complexity of the proposal, it would be helpful if discussion happened sooner than later, including statements of support or issues to be resolved. Input is especially important from data providers who actually write point observation files. I haven't commented on this new version because, as far as I understand it, it will not work for data from surface moorings, and therefore won't apply to most of the datasets in projects I'm involved with. IF this is intended to be used for time series at multiple depths, then the definition of "stationTimeSeries" would need to be changed from "a time-series of data points at the same location" to "a time-series of data points at the same X-Y location" - to make it more clear that this type can hold time series data at multiple depths. The example would also need to show how a dependent variable would be dimensioned to hold 2d data at each station. In recent discussions on this list, it looks almost like there's some idea that the "profile" feature type should be used for this kind of data. That would be a very hard sell. The term profile, as used in the observational community, is very different from time series - most importantly, it indicates that Z varies with T. If moored station data is meant to be covered by this specification, and if it won't fit into the "station" feature type, a term other than profile should be used. Cheers - Nan Hi Nan: The proposal is intended to work with surface moorings, so if it doesnt work, lets understand why. I think your data would probably be classified as stationProfile data: "a time-series of profiles at a named location". A single file can contain one or a collection of surface moorings data. See section 9.6 for representations. I think you are being misled by the word "profile", which apparently has a different meaning in your community. In this proposal, a profile is a series of point measurements at different z, with the same x,y. There may be a single time for each profile, or time may depend on z. It sounds like the former is true in your case. Assuming that the semantics fits, i wouldnt be overly concerned if the name "stationProfile" is a bit misleading. You will have to add a global attribute :CF:featureType = "stationProfile"; to your file, and follow one of the representations in section 9.6, but otherwise the variable and dimension names can use whatever terminology makes sense to your community. If you want to send a sample (perhaps simplified) CDL, we could see what it would look like under this proposal. Regards, John ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- * Don Murray UCAR Unidata Program dmur...@unidata.ucar.eduP.O. Box 3000 (303) 497-8628 Boulder, CO 80307 http://www.unidata.ucar.edu/staff/donm * ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] CF point observation Conventions ready for review
Nan Galbraith wrote: Steve Hankin wrote: Its approaching two weeks (Oct 27) since a revised "CF point observation Conventions" proposal was made: https://cf-pcmdi.llnl.gov/trac/wiki/PointObservationConventions https://cf-pcmdi.llnl.gov/trac/ticket/37 Given the complexity of the proposal, it would be helpful if discussion happened sooner than later, including statements of support or issues to be resolved. Input is especially important from data providers who actually write point observation files. I haven't commented on this new version because, as far as I understand it, it will not work for data from surface moorings, and therefore won't apply to most of the datasets in projects I'm involved with. IF this is intended to be used for time series at multiple depths, then the definition of "stationTimeSeries" would need to be changed from "a time-series of data points at the same location" to "a time-series of data points at the same X-Y location" - to make it more clear that this type can hold time series data at multiple depths. The example would also need to show how a dependent variable would be dimensioned to hold 2d data at each station. In recent discussions on this list, it looks almost like there's some idea that the "profile" feature type should be used for this kind of data. That would be a very hard sell. The term profile, as used in the observational community, is very different from time series - most importantly, it indicates that Z varies with T. If moored station data is meant to be covered by this specification, and if it won't fit into the "station" feature type, a term other than profile should be used. Cheers - Nan Hi Nan: The proposal is intended to work with surface moorings, so if it doesnt work, lets understand why. I think your data would probably be classified as stationProfile data: "a time-series of profiles at a named location". A single file can contain one or a collection of surface moorings data. See section 9.6 for representations. I think you are being misled by the word "profile", which apparently has a different meaning in your community. In this proposal, a profile is a series of point measurements at different z, with the same x,y. There may be a single time for each profile, or time may depend on z. It sounds like the former is true in your case. Assuming that the semantics fits, i wouldnt be overly concerned if the name "stationProfile" is a bit misleading. You will have to add a global attribute :CF:featureType = "stationProfile"; to your file, and follow one of the representations in section 9.6, but otherwise the variable and dimension names can use whatever terminology makes sense to your community. If you want to send a sample (perhaps simplified) CDL, we could see what it would look like under this proposal. Regards, John ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] CF point observation Conventions ready for review
Steve Hankin wrote: Its approaching two weeks (Oct 27) since a revised "CF point observation Conventions" proposal was made: https://cf-pcmdi.llnl.gov/trac/wiki/PointObservationConventions https://cf-pcmdi.llnl.gov/trac/ticket/37 Given the complexity of the proposal, it would be helpful if discussion happened sooner than later, including statements of support or issues to be resolved. Input is especially important from data providers who actually write point observation files. I haven't commented on this new version because, as far as I understand it, it will not work for data from surface moorings, and therefore won't apply to most of the datasets in projects I'm involved with. IF this is intended to be used for time series at multiple depths, then the definition of "stationTimeSeries" would need to be changed from "a time-series of data points at the same location" to "a time-series of data points at the same X-Y location" - to make it more clear that this type can hold time series data at multiple depths. The example would also need to show how a dependent variable would be dimensioned to hold 2d data at each station. In recent discussions on this list, it looks almost like there's some idea that the "profile" feature type should be used for this kind of data. That would be a very hard sell. The term profile, as used in the observational community, is very different from time series - most importantly, it indicates that Z varies with T. If moored station data is meant to be covered by this specification, and if it won't fit into the "station" feature type, a term other than profile should be used. Cheers - Nan -- *** * Nan Galbraith(508) 289-2444 * * Upper Ocean Processes GroupMail Stop 29 * * Woods Hole Oceanographic Institution* * Woods Hole, MA 02543* *** ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] CF point observation Conventions ready for review
Hi All, Its approaching two weeks (Oct 27) since a revised "CF point observation Conventions" proposal was made: https://cf-pcmdi.llnl.gov/trac/wiki/PointObservationConventions https://cf-pcmdi.llnl.gov/trac/ticket/37 Given the complexity of the proposal, it would be helpful if discussion happened sooner than later, including statements of support or issues to be resolved. Input is especially important from data providers who actually write point observation files. The ideas have been floated for more than a year and a number of projects need a resolution. - Steve John Caron wrote: I have complete a new version of the CF point observation Conventions at: https://cf-pcmdi.llnl.gov/trac/wiki/PointObservationConventions Discussion is at: https://cf-pcmdi.llnl.gov/trac/ticket/37 I have incorporated various feedback from the past year, and made a preliminary implementation to be sure that a generic application can distinguish the various cases without human intervention. I have tried to simplify, esp in the station profile and section feature types, where the combinations of options got too complex. The document is now explicit about all possible representations. The use of missing values is also clarified. While the document is rather long, if you manage to wade through it you'll see that the patterns of use keep repeating and are mostly regular. Showing full examples I think is the best way to prevent misunderstandings. I did make enough changes that anyone who wrote files using the previous version should check to see whats changed. Apologies for that. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
[CF-metadata] CF point observation Conventions ready for review
I have complete a new version of the CF point observation Conventions at: https://cf-pcmdi.llnl.gov/trac/wiki/PointObservationConventions Discussion is at: https://cf-pcmdi.llnl.gov/trac/ticket/37 I have incorporated various feedback from the past year, and made a preliminary implementation to be sure that a generic application can distinguish the various cases without human intervention. I have tried to simplify, esp in the station profile and section feature types, where the combinations of options got too complex. The document is now explicit about all possible representations. The use of missing values is also clarified. While the document is rather long, if you manage to wade through it you'll see that the patterns of use keep repeating and are mostly regular. Showing full examples I think is the best way to prevent misunderstandings. I did make enough changes that anyone who wrote files using the previous version should check to see whats changed. Apologies for that. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
