Re: [CF-metadata] per-variable metadata?
Hi Nan, I admit that my thinking is driven by coming from a community (SeaDataNet/Geo-Seas) where 99% of data are currently in formats other than netCDF (hopefully something that will change in SeaDataNet II) and therefore need a usage metadata solution that will initially at least operate across all formats in use. This is why I'm looking at O&M/SensorML. I have real concerns that a scenario will develop where SDN II partners build XML usage metadata stock for their ODV (the ASCII format holding most SeaDataNet data) data holdings and then resist migration to CF netCDF if this XML stock cannot be utilised. Whilst admittedly this isn't 100% relevant to a discussion on CF, I would argue that if a universal standard requiring XML encoding is developed CF should buy into that rather going its own way. Note that a solution where a specified binding to external resources is defined that can sit side-by-side with internal encodings for a usage metadata subset would make me happy and shut me up (on this issue at least). Cheers, Roy. From: Nan Galbraith [ngalbra...@whoi.edu] Sent: 05 August 2011 20:58 To: Lowry, Roy K. Cc: John Graybeal; cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] per-variable metadata? I don't think the metadata structure needs to be terribly complex, in most cases. Changes to instruments (either a new instrument, a change to the sample scheme, or some kind of change to calibrations, etc) partway through could be a problem, but that's true in xml too. John G is right, we often lose access to the network at sea; but even without that consideration, NetCDF is supposed to be self-documenting, and we should be able to come up with some strategies for including fairly complex metadata in the files. I like the ancillary variable route because it seems straightforward and flexible. * Data variables can share one or more ancillary variables, very useful for something like an ADCP record. * Ancillary variables can have data types. * Ancillary variables can have dimensions that make them self-explanatory. Cheers - Nan On Aug/05/2011 2:08 PM, Lowry, Roy K. wrote: Hi John, We seem to have a difference of perception in the fragility of Web access to the average scientist. Certainly, I was assuming ships had virtually the same level of internet access as I have at home (if not in the lab) based on my experience of the NERC ships. However, if lack of internet access becomes an issue then we need to look at URI resolution to physical XML files sitting with the NetCDF. Cheers, Roy. From: John Graybeal [jbgrayb...@mindspring.com<mailto:jbgrayb...@mindspring.com>] Sent: 05 August 2011 17:57 To: Lowry, Roy K. Cc: ngalbra...@whoi.edu<mailto:ngalbra...@whoi.edu>; cf-metadata@cgd.ucar.edu<mailto:cf-metadata@cgd.ucar.edu> Subject: Re: [CF-metadata] per-variable metadata? On Aug 5, 2011, at 09:27, Lowry, Roy K. wrote: Hi Nan, At the risk of being shot for herecy, I maintain the belief that in the current technological environonment packing everything inside the straightjacket container of a physical file is an anachronism.So, I would have an attribute in the CF file holding a permanent identifier that is guaranteed to be resolvable from now until eternity into an XML file delivering usage metadata. I agree with the first statement, because 'everything' is a long list indeed. The world of linked open data agrees too, I expect. But with respect to *usage* metadata, I wonder. Because the storage of complex data for perpetual access on the web is not a solved problem, and not everyone has access to the entire web, or at least every metadata provider for every file they are working with, all the time. So then you're stuck, until the ship gets back or the DSL line is up or the hosting site finishes its weekly maintenance, or someone goes back and resuscitates some old system that had the metadata years or decades before. John-- This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system. -- *** * Nan Galbraith(508) 289-2444 * * Upper Ocean Processes GroupMail Stop 29 * * Woods Hole Oceanographic Institution* * Woods Hole, MA 02543* *** ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] per-variable metadata?
I don't think the metadata structure needs to be terribly complex, in most cases. Changes to instruments (either a new instrument, a change to the sample scheme, or some kind of change to calibrations, etc) partway through could be a problem, but that's true in xml too. John G is right, we often lose access to the network at sea; but even without that consideration, NetCDF is supposed to be self-documenting, and we should be able to come up with some strategies for including fairly complex metadata in the files. I like the ancillary variable route because it seems straightforward and flexible. Data variables can share one or more ancillary variables, very useful for something like an ADCP record. Ancillary variables can have data types. Ancillary variables can have dimensions that make them self-explanatory. Cheers - Nan On Aug/05/2011 2:08 PM, Lowry, Roy K. wrote: Hi John, We seem to have a difference of perception in the fragility of Web access to the average scientist. Certainly, I was assuming ships had virtually the same level of internet access as I have at home (if not in the lab) based on my experience of the NERC ships. However, if lack of internet access becomes an issue then we need to look at URI resolution to physical XML files sitting with the NetCDF. Cheers, Roy. From: John Graybeal [jbgrayb...@mindspring.com] Sent: 05 August 2011 17:57 To: Lowry, Roy K. Cc: ngalbra...@whoi.edu; cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] per-variable metadata? On Aug 5, 2011, at 09:27, Lowry, Roy K. wrote: Hi Nan, At the risk of being shot for herecy, I maintain the belief that in the current technological environonment packing everything inside the straightjacket container of a physical file is an anachronism.So, I would have an attribute in the CF file holding a permanent identifier that is guaranteed to be resolvable from now until eternity into an XML file delivering usage metadata. I agree with the first statement, because 'everything' is a long list indeed. The world of linked open data agrees too, I expect. But with respect to *usage* metadata, I wonder. Because the storage of complex data for perpetual access on the web is not a solved problem, and not everyone has access to the entire web, or at least every metadata provider for every file they are working with, all the time. So then you're stuck, until the ship gets back or the DSL line is up or the hosting site finishes its weekly maintenance, or someone goes back and resuscitates some old system that had the metadata years or decades before. John-- This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system. -- *** * Nan Galbraith(508) 289-2444 * * Upper Ocean Processes GroupMail Stop 29 * * Woods Hole Oceanographic Institution* * Woods Hole, MA 02543* *** ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] per-variable metadata?
Hi John, We seem to have a difference of perception in the fragility of Web access to the average scientist. Certainly, I was assuming ships had virtually the same level of internet access as I have at home (if not in the lab) based on my experience of the NERC ships. However, if lack of internet access becomes an issue then we need to look at URI resolution to physical XML files sitting with the NetCDF. Cheers, Roy. From: John Graybeal [jbgrayb...@mindspring.com] Sent: 05 August 2011 17:57 To: Lowry, Roy K. Cc: ngalbra...@whoi.edu; cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] per-variable metadata? On Aug 5, 2011, at 09:27, Lowry, Roy K. wrote: > Hi Nan, > > At the risk of being shot for herecy, I maintain the belief that in the > current technological environonment packing everything inside the > straightjacket container of a physical file is an anachronism.So, I > would have an attribute in the CF file holding a permanent identifier that is > guaranteed to be resolvable from now until eternity into an XML file > delivering usage metadata. I agree with the first statement, because 'everything' is a long list indeed. The world of linked open data agrees too, I expect. But with respect to *usage* metadata, I wonder. Because the storage of complex data for perpetual access on the web is not a solved problem, and not everyone has access to the entire web, or at least every metadata provider for every file they are working with, all the time. So then you're stuck, until the ship gets back or the DSL line is up or the hosting site finishes its weekly maintenance, or someone goes back and resuscitates some old system that had the metadata years or decades before. John-- This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] per-variable metadata?
On Aug 5, 2011, at 09:27, Lowry, Roy K. wrote: > Hi Nan, > > At the risk of being shot for herecy, I maintain the belief that in the > current technological environonment packing everything inside the > straightjacket container of a physical file is an anachronism.So, I > would have an attribute in the CF file holding a permanent identifier that is > guaranteed to be resolvable from now until eternity into an XML file > delivering usage metadata. I agree with the first statement, because 'everything' is a long list indeed. The world of linked open data agrees too, I expect. But with respect to *usage* metadata, I wonder. Because the storage of complex data for perpetual access on the web is not a solved problem, and not everyone has access to the entire web, or at least every metadata provider for every file they are working with, all the time. So then you're stuck, until the ship gets back or the DSL line is up or the hosting site finishes its weekly maintenance, or someone goes back and resuscitates some old system that had the metadata years or decades before. John ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] per-variable metadata?
Hi Nan, At the risk of being shot for herecy, I maintain the belief that in the current technological environonment packing everything inside the straightjacket container of a physical file is an anachronism. As you've probably guessed my view is that metadata and its associated semantics have data structures that incorporate complex interrelationships, which can only be properly encoded in formats like XML. So, I would have an attribute in the CF file holding a permanent identifier that is guaranteed to be resolvable from now until eternity into an XML file delivering usage metadata. Cheers, Roy. From: Nan Galbraith [ngalbra...@whoi.edu] Sent: 05 August 2011 16:30 To: Lowry, Roy K.; cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] per-variable metadata? Hi Roy and all - Do you see a possible place for this metadata in a more structured attribute system, or somewhere else in the NetCDF file structure? Or are you thinking that we need metadata outside our CF files? Since CF has the precedent of putting other quality-related information (flag values, uncertainty, etc.) in ancillary variables, it seems (or seemed) reasonable to me to include more complete provenance info there. It might blur the line between data and metadata, but I don't see another solution without higher costs, at the moment. I'm looking forward to someone solving this, and I'm glad to hear that SeadataNet may be looking at it. Cheers - Nan On Aug/04/2011 4:13 PM, Lowry, Roy K. wrote: Hi Nan, Encoding what I would regard as usage metadata into ancilliary variables is one solution, but I would much rather see a formalised metadata model for doing this. As you say a standard way of doing this is long overdue. My hope is that SeadataNet II starting later this year will take on and address this challenge. However, if anybody knows of anything already in existence that fits the bill it would be good to prevent yet another wheel being reinvented. Cheers, Roy. From: cf-metadata-boun...@cgd.ucar.edu<mailto:cf-metadata-boun...@cgd.ucar.edu> [cf-metadata-boun...@cgd.ucar.edu<mailto:cf-metadata-boun...@cgd.ucar.edu>] On Behalf Of Nan Galbraith [ngalbra...@whoi.edu<mailto:ngalbra...@whoi.edu>] Sent: 04 August 2011 19:17 To: Steve Hankin; cf-metadata@cgd.ucar.edu<mailto:cf-metadata@cgd.ucar.edu> Cc: Jeff deLaBeaujardiere Subject: Re: [CF-metadata] per-variable metadata? Hi Steve - I'm very interested in the background discussion on this - any chance of bringing it into the foreground? I'm using ancillary variables in 2-D in situ data files to describe instruments and things like precision, accuracy, sample scheme, etc.. For temperature files from moorings where different sensor types are at different depths, I'd like to use something like TEMP:ancillary_variables = "Instrument_manufacturer Instrument_model Instrument_sample_scheme Instrument_serial_number TEMP_qc_procedure TEMP_accuracy TEMP_precision TEMP_resolution"; and then short INST_SN(depth) ; INST_SN:long_name = "instrument_serial_number" ; ... etc., etc. If there's going to be a standard way to do this, I'd really like to know about it - sooner rather than than later. Thanks - Nan On 8/4/11 11:35 AM, Steve Hankin wrote: Hi Jeff, Each variable in a CF file may possess an |ancillary_variables| attribute, that points to variables that have relationships (http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/cf-conventions.html#ancillary-data). To attach flags to a variable, use |ancillary_variables| to point to a variable that has |flag_values||| and |flag_meanings |attributes (http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/cf-conventions.html#flags). We have started a discussion in the background, whether an example that illustrates this should be included in the CF documentation. - Steve = On 7/14/2010 6:21 AM, Jeff deLaBeaujardiere wrote: In another discussion, Steve Hankin wrote: > CF generally favors attributes attached to variables over attributes attached > to files This reminds me of a question I wanted to ask: does CF have any conventions regarding how to handle data that contains multiple observed quantities with different quality flags, comment fields or other attributes for each quantity? -Jeff DLB -- *** * Nan Galbraith(508) 289-2444 * * Upper Ocean Processes GroupMail Stop 29 * * Woods Hole Oceanographic Institution* * Woods Hole, MA 02543* *** -- This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents
Re: [CF-metadata] per-variable metadata?
Hi Roy and all - Do you see a possible place for this metadata in a more structured attribute system, or somewhere else in the NetCDF file structure? Or are you thinking that we need metadata outside our CF files? Since CF has the precedent of putting other quality-related information (flag values, uncertainty, etc.) in ancillary variables, it seems (or seemed) reasonable to me to include more complete provenance info there. It might blur the line between data and metadata, but I don't see another solution without higher costs, at the moment. I'm looking forward to someone solving this, and I'm glad to hear that SeadataNet may be looking at it. Cheers - Nan On Aug/04/2011 4:13 PM, Lowry, Roy K. wrote: Hi Nan, Encoding what I would regard as usage metadata into ancilliary variables is one solution, but I would much rather see a formalised metadata model for doing this. As you say a standard way of doing this is long overdue. My hope is that SeadataNet II starting later this year will take on and address this challenge. However, if anybody knows of anything already in existence that fits the bill it would be good to prevent yet another wheel being reinvented. Cheers, Roy. From: cf-metadata-boun...@cgd.ucar.edu [cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Nan Galbraith [ngalbra...@whoi.edu] Sent: 04 August 2011 19:17 To: Steve Hankin; cf-metadata@cgd.ucar.edu Cc: Jeff deLaBeaujardiere Subject: Re: [CF-metadata] per-variable metadata? Hi Steve - I'm very interested in the background discussion on this - any chance of bringing it into the foreground? I'm using ancillary variables in 2-D in situ data files to describe instruments and things like precision, accuracy, sample scheme, etc.. For temperature files from moorings where different sensor types are at different depths, I'd like to use something like TEMP:ancillary_variables = "Instrument_manufacturer Instrument_model Instrument_sample_scheme Instrument_serial_number TEMP_qc_procedure TEMP_accuracy TEMP_precision TEMP_resolution"; and then short INST_SN(depth) ; INST_SN:long_name = "instrument_serial_number" ; ... etc., etc. If there's going to be a standard way to do this, I'd really like to know about it - sooner rather than than later. Thanks - Nan On 8/4/11 11:35 AM, Steve Hankin wrote: Hi Jeff, Each variable in a CF file may possess an |ancillary_variables| attribute, that points to variables that have relationships (http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/cf-conventions.html#ancillary-data). To attach flags to a variable, use |ancillary_variables| to point to a variable that has |flag_values||| and |flag_meanings |attributes (http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/cf-conventions.html#flags). We have started a discussion in the background, whether an example that illustrates this should be included in the CF documentation. - Steve = On 7/14/2010 6:21 AM, Jeff deLaBeaujardiere wrote: In another discussion, Steve Hankin wrote: > CF generally favors attributes attached to variables over attributes attached to files This reminds me of a question I wanted to ask: does CF have any conventions regarding how to handle data that contains multiple observed quantities with different quality flags, comment fields or other attributes for each quantity? -Jeff DLB -- *** * Nan Galbraith(508) 289-2444 * * Upper Ocean Processes GroupMail Stop 29 * * Woods Hole Oceanographic Institution* * Woods Hole, MA 02543* *** ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] per-variable metadata?
On 8/4/2011 1:30 PM, Upendra Dadi wrote: Hi Steve & Nan, Is this allowed in CF? Isn't ancillary_variable meant to be used for per value metadata i.e. metadata for each and every value in the variable it is referring to? If so, shouldn't the ancillary_variable have the same set of dimensions and in the same order as the variable it is referring to? Hi Upendra, Nan, Roy, First off just to comment that these topics seems appropriate as a logical next step. The new Discrete Geometries chapter largely focused on the file structure aspects -- storing the numbers. Community by community the metadata needs vary, of course. Defining "standard profiles" (e.g. OceanSites) is the natural approach to standardizing metadata contents. So I think we are all asking the question, what are the general rules and structures that should be followed so that generic applications are best able to access and utilize the specialized metadata encoded per a standardized CF profile? A second point is to confess that I wasn't personally involved in the discussions that lead to the ancillary_variables machinery. I'm ready to stand corrected if I misinterpret the words found in CF 1.5. Lets just get the ideas out on the table and let others comment. For the case that Nan has described, if one were using the techniques of chapter 9 (https://cf-pcmdi.llnl.gov/trac/attachment/ticket/37/CFch9-may4.docx?format=raw) the metadata would be tied to the variable "TEMP" by its station index, rather than by an ancillary_variable attribute. In this example the "station_info" variable is a model model for "Instrument_manufacturer", "Instrument_model", etc. A9.2.4 Contiguous ragged array representation of timeSeries dimensions: station = 23 ; obs = 1234 ; variables: float lon(station) ; lon:standard_name = "longitude"; lon:long_name = "station longitude"; lon:units = "degrees_east"; float lat(station) ; lat:standard_name = "latitude"; lat:long_name = "station latitude" ; lat:units = "degrees_north" ; char station_name(station, name_strlen) ; station_name:long_name = "station name" ; station_name:cf_role = "station_idtimeseries_id"; * int station_info(station) ;* * station_info:long_name = "some kind of station info" ;* int row_size(station) ; row_size:long_name = "number of observations for this station " ; row_size:ample_dimension = "obs" ; double time(obs) ; time:standard_name = "time"; time:long_name = "time of measurement" ; time:units = "days since 1970-01-01 00:00:00" ; float humidity(obs) ; humidity:standard_name = "specific_humidity" ; Nan, I think in your example the "depth" dimension is effectively the same as the "station" dimension in A9.2.4 (or 9.2.1) -- independent instruments deployed at a list of depths (stations) with metadata describing each depth. So the question is whether the the association of metadata through the station (or depth) dimension is sufficient? (I think it is.) Or is there a use case that demonstrates that the ancillary_variable machinery is needed, as well? Upendra, your point, "/ancillary_variable meant to be used for per value metadata i.e. metadata for each and every value in the variable it is referring to/" is a strict interpretation of the opening sentence of 3.4. Ancillary Data (http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/cf-conventions.html#ancillary-data), "/one data variable provides metadata about the individual values of another data variable/". That interpretation would rule out cases like the following, which seem desirable to encode (imagining an instrument such as a Doppler profiler, where the uncertainty in a velocity measurement is a function of depth) float q(time, depth) ; q:standard_name = "||upward_sea_water_velocity" ; q:ancillary_variables = "q_uncertainty" ; float q_uncertainty(depth) Some word-smithing seems to be in order to clarify that opening sentence of section 3.4. - Steve Upendra On 8/4/2011 2:17 PM, Nan Galbraith wrote: Hi Steve - I'm very interested in the background discussion on this - any chance of bringing it into the foreground? I'm using ancillary variables in 2-D in situ data files to describe instruments and things like precision, accuracy, sample scheme, etc.. For temperature files from moorings where different sensor types are at different depths, I'd like to use something like TEMP:ancillary_variables = "Instrument_manufacturer Instrument_model Instrument_sample_scheme Instrument_serial_number TEMP_qc_procedure TEMP_accuracy TEMP_precision TEMP_resolution"; and then short INST_SN(
Re: [CF-metadata] per-variable metadata?
Hi Steve & Nan, Is this allowed in CF? Isn't ancillary_variable meant to be used for per value metadata i.e. metadata for each and every value in the variable it is referring to? If so, shouldn't the ancillary_variable have the same set of dimensions and in the same order as the variable it is referring to? Upendra On 8/4/2011 2:17 PM, Nan Galbraith wrote: Hi Steve - I'm very interested in the background discussion on this - any chance of bringing it into the foreground? I'm using ancillary variables in 2-D in situ data files to describe instruments and things like precision, accuracy, sample scheme, etc.. For temperature files from moorings where different sensor types are at different depths, I'd like to use something like TEMP:ancillary_variables = "Instrument_manufacturer Instrument_model Instrument_sample_scheme Instrument_serial_number TEMP_qc_procedure TEMP_accuracy TEMP_precision TEMP_resolution"; and then short INST_SN(depth) ; INST_SN:long_name = "instrument_serial_number" ; ... etc., etc. If there's going to be a standard way to do this, I'd really like to know about it - sooner rather than than later. Thanks - Nan On 8/4/11 11:35 AM, Steve Hankin wrote: Hi Jeff, Each variable in a CF file may possess an |ancillary_variables| attribute, that points to variables that have relationships (http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/cf-conventions.html#ancillary-data). To attach flags to a variable, use |ancillary_variables| to point to a variable that has |flag_values||| and |flag_meanings |attributes (http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/cf-conventions.html#flags). We have started a discussion in the background, whether an example that illustrates this should be included in the CF documentation. - Steve = On 7/14/2010 6:21 AM, Jeff deLaBeaujardiere wrote: In another discussion, Steve Hankin wrote: > CF generally favors attributes attached to variables over attributes attached to files This reminds me of a question I wanted to ask: does CF have any conventions regarding how to handle data that contains multiple observed quantities with different quality flags, comment fields or other attributes for each quantity? -Jeff DLB ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- *** * Nan Galbraith(508) 289-2444 * * Upper Ocean Processes GroupMail Stop 29 * * Woods Hole Oceanographic Institution* * Woods Hole, MA 02543* *** ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] per-variable metadata?
Hi Nan, Encoding what I would regard as usage metadata into ancilliary variables is one solution, but I would much rather see a formalised metadata model for doing this. As you say a standard way of doing this is long overdue. My hope is that SeadataNet II starting later this year will take on and address this challenge. However, if anybody knows of anything already in existence that fits the bill it would be good to prevent yet another wheel being reinvented. Cheers, Roy. From: cf-metadata-boun...@cgd.ucar.edu [cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Nan Galbraith [ngalbra...@whoi.edu] Sent: 04 August 2011 19:17 To: Steve Hankin; cf-metadata@cgd.ucar.edu Cc: Jeff deLaBeaujardiere Subject: Re: [CF-metadata] per-variable metadata? Hi Steve - I'm very interested in the background discussion on this - any chance of bringing it into the foreground? I'm using ancillary variables in 2-D in situ data files to describe instruments and things like precision, accuracy, sample scheme, etc.. For temperature files from moorings where different sensor types are at different depths, I'd like to use something like TEMP:ancillary_variables = "Instrument_manufacturer Instrument_model Instrument_sample_scheme Instrument_serial_number TEMP_qc_procedure TEMP_accuracy TEMP_precision TEMP_resolution"; and then short INST_SN(depth) ; INST_SN:long_name = "instrument_serial_number" ; ... etc., etc. If there's going to be a standard way to do this, I'd really like to know about it - sooner rather than than later. Thanks - Nan On 8/4/11 11:35 AM, Steve Hankin wrote: Hi Jeff, Each variable in a CF file may possess an |ancillary_variables| attribute, that points to variables that have relationships (http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/cf-conventions.html#ancillary-data). To attach flags to a variable, use |ancillary_variables| to point to a variable that has |flag_values||| and |flag_meanings |attributes (http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/cf-conventions.html#flags). We have started a discussion in the background, whether an example that illustrates this should be included in the CF documentation. - Steve = On 7/14/2010 6:21 AM, Jeff deLaBeaujardiere wrote: In another discussion, Steve Hankin wrote: > CF generally favors attributes attached to variables over attributes attached > to files This reminds me of a question I wanted to ask: does CF have any conventions regarding how to handle data that contains multiple observed quantities with different quality flags, comment fields or other attributes for each quantity? -Jeff DLB ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu<mailto:CF-metadata@cgd.ucar.edu> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu<mailto:CF-metadata@cgd.ucar.edu> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- *** * Nan Galbraith(508) 289-2444 * * Upper Ocean Processes GroupMail Stop 29 * * Woods Hole Oceanographic Institution* * Woods Hole, MA 02543* *** -- This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system.___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] per-variable metadata?
Hi Steve - I'm very interested in the background discussion on this - any chance of bringing it into the foreground? I'm using ancillary variables in 2-D in situ data files to describe instruments and things like precision, accuracy, sample scheme, etc.. For temperature files from moorings where different sensor types are at different depths, I'd like to use something like TEMP:ancillary_variables = "Instrument_manufacturer Instrument_model Instrument_sample_scheme Instrument_serial_number TEMP_qc_procedure TEMP_accuracy TEMP_precision TEMP_resolution"; and then short INST_SN(depth) ; INST_SN:long_name = "instrument_serial_number" ; ... etc., etc. If there's going to be a standard way to do this, I'd really like to know about it - sooner rather than than later. Thanks - Nan On 8/4/11 11:35 AM, Steve Hankin wrote: Hi Jeff, Each variable in a CF file may possess an |ancillary_variables| attribute, that points to variables that have relationships (http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/cf-conventions.html#ancillary-data). To attach flags to a variable, use |ancillary_variables| to point to a variable that has |flag_values||| and |flag_meanings |attributes (http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/cf-conventions.html#flags). We have started a discussion in the background, whether an example that illustrates this should be included in the CF documentation. - Steve = On 7/14/2010 6:21 AM, Jeff deLaBeaujardiere wrote: In another discussion, Steve Hankin wrote: > CF generally favors attributes attached to variables over attributes attached to files This reminds me of a question I wanted to ask: does CF have any conventions regarding how to handle data that contains multiple observed quantities with different quality flags, comment fields or other attributes for each quantity? -Jeff DLB ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- *** * Nan Galbraith(508) 289-2444 * * Upper Ocean Processes GroupMail Stop 29 * * Woods Hole Oceanographic Institution* * Woods Hole, MA 02543* *** ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] per-variable metadata?
Hi Jeff, Each variable in a CF file may possess an |ancillary_variables| attribute, that points to variables that have relationships (http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/cf-conventions.html#ancillary-data). To attach flags to a variable, use |ancillary_variables| to point to a variable that has |flag_values||| and |flag_meanings |attributes (http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/cf-conventions.html#flags). We have started a discussion in the background, whether an example that illustrates this should be included in the CF documentation. - Steve = On 7/14/2010 6:21 AM, Jeff deLaBeaujardiere wrote: In another discussion, Steve Hankin wrote: > CF generally favors attributes attached to variables over attributes attached to files This reminds me of a question I wanted to ask: does CF have any conventions regarding how to handle data that contains multiple observed quantities with different quality flags, comment fields or other attributes for each quantity? -Jeff DLB ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
[CF-metadata] per-variable metadata?
Dear Jeff > This reminds me of a question I wanted to ask: does CF have any > conventions regarding how to handle data that contains multiple observed > quantities with different quality flags, comment fields or other > attributes for each quantity? Yes. Ancillary variables such as flags are pointed to from data variables using the ancillary_variables attribute (CF 3.4 and 3.5). There is discussion of global and per-variable attributes in 2.6.2 and the attrs are tabulated in Appendix A. Cheers Jonathan ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
[CF-metadata] per-variable metadata?
In another discussion, Steve Hankin wrote: > CF generally favors attributes attached to variables over attributes attached to files This reminds me of a question I wanted to ask: does CF have any conventions regarding how to handle data that contains multiple observed quantities with different quality flags, comment fields or other attributes for each quantity? -Jeff DLB ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
