Re: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May 2016, Boulder, CO, USA
Ethan, Two more topics to consider as additions to your list: 1. "UGRID" -- Of interest to the coastal ocean modeling community, and stuck in a holding pattern for quite a long time, is the harmonization of mesh/unstructured grid coordinates into the main body of CF. The work is mostly completed ... 2. "Aggregation" -- CF structures to link multiple files into larger conceptual datasets. Time series aggregations, union aggregations (associated variables in separate files), and ensemble membership are the most obvious applications of this. Another important application is "tiled grids" (ref. the "gridspec" proposal). - Steve On 3/27/2016 9:24 PM, Ethan Davis wrote: Hi all, As some of you have heard, we are holding a meeting to discuss current and future netCDF-CF efforts and directions. The meeting will be held on 24-26 May 2016 in Boulder, CO, USA at the UCAR Center Green facility. This meeting is organized by the “Advancing netCDF-CF for Geoscience” project [1] which has been funded by the US NSF EarthCube program. The project goals include several specific CF development efforts, both standards development and software prototyping. (We are just starting to spin-up community conversations around these specific goals for drafting CF extensions.) Another aspect of the funded project is community engagement including with the existing CF community, geoscience domains that have not been very involved in CF, and other standards bodies (e.g., OGC). These and other topics are on the current list of possible agenda items. Please let us know if you have other agenda items in mind. The current list of possible agenda items includes: * Investigate using features of the netCDF enhanced data model to improve/simplify the CF Discrete Sampling Geometries (which includes point, sounding, and trajectory data types) * Drafting a CF enhancement that would add support for complex data footprints (e.g., river segments and drainage basins) * Further develop the CFRadial proposal for representing Radar data * Adding features to better support satellite data * Adding CF support for hierarchical groups for organizing data and metadata * Enhancing CF to support Linked Data / semantic technology concepts * Harmonizing CF grid mapping with OGC CRS (coordinate reference systems), possibly with WKT * Adding CF support for attribute namespaces and the use of URI prefixes in attribute values * Improved support for dimensionless data variables, logarithmic scales, differences, etc. While we wish we could host all those interested in attending, given the available meeting space and the number of project participants, we are asking members of the CF community that are interested in attending to contact us directly and let us know which aspects of CF are of interest to you and your goals for the meeting. Please submit your expressions of interest by Monday, 4 April 2016. We plan to invite participants by Friday, 8 April 2016. Travel and other information will be on the meeting page [2] later this week. Also, for anyone attending EGU in Vienna in April, we will have a splinter meeting (SMP32) to discuss netCDF-CF. The splinter meeting will be held on Thursday, 21 April 2016 at 15:30 in room 2.96. Thank you, Meeting Organizers Ethan Davis, UCAR Unidata Charlie Zender, Univ of CA, Irvine David Arctur, Univ of TX, Austin Dave Santek, Univ of WI, Madison / SSEC Kevin O’Brien, Univ of WA/JISAO and NOAA/PMEL Aleksandar Jelenak, The HDF Group Mike Dixon, NCAR/EOL [1] http://www.nsf.gov/awardsearch/showAward?AWD_ID=1541031 [2] http://www.unidata.ucar.edu/events/2016CFWorkshop/ ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May 2016, Boulder, CO, USA
On Mon, Mar 28, 2016 at 9:25 AM, Steve Hankin wrote: > Two more topics to consider as additions to your list: > > 1. "UGRID" -- Of interest to the coastal ocean modeling community, and > stuck in a holding pattern for quite a long time, is the harmonization of > mesh/unstructured grid coordinates into the main body of CF. The work is > mostly completed ... > Thanks Steve -- I was hoping someone would print that out. Though it's not in a holding pattern, we actually just pushed our a version 1.0, and it's seeign some use: http://ugrid-conventions.github.io/ugrid-conventions/ Though yes, it's been pretty darn quiet, and it would be great to solidify it's relationship to CF. I can't attend the meeting, but hopefully someone from the UGRID community could. I've also been involved in trying to standardize a convention to storing the results of particle tracking models: https://github.com/NOAA-ORR-ERD/nc_particles/blob/master/nc_particle_standard.md Though again, I can't go and that one may still be too fresh for the larger community. > 2. "Aggregation" -- CF structures to link multiple files into larger > conceptual datasets. > +1 on this one too! -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May 2016, Boulder, CO, USA
Hi Ethan, I’m glad Steve brought up the “aggregation” topic. On this topic, it would be interesting to see if anyone has successfully come up with a standardized schema to aggregate satellite-derived Level 2 swath grid structures. I know there’s an aggregation implementation in the latest OPeNDAP release, but it remains to be seen what this might look like for Level 2 (and perhaps even Level 1) swath grid structured datasets. I don’t know of anyone who’s successfully tested and released an operationally working instance of that part of OPeNDAP for swath grid structured datasets. Cheers, David From: CF-metadata mailto:cf-metadata-boun...@cgd.ucar.edu>> on behalf of Steve Hankin mailto:steven.c.han...@noaa.gov>> Date: Monday, March 28, 2016 at 9:25 AM To: Ethan Davis mailto:eda...@ucar.edu>>, CF metadata mailto:cf-metadata@cgd.ucar.edu>>, netCDF SWG mailto:netcdf@lists.opengeospatial.org>> Subject: Re: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May 2016, Boulder, CO, USA Ethan, Two more topics to consider as additions to your list: 1. "UGRID" -- Of interest to the coastal ocean modeling community, and stuck in a holding pattern for quite a long time, is the harmonization of mesh/unstructured grid coordinates into the main body of CF. The work is mostly completed ... 2. "Aggregation" -- CF structures to link multiple files into larger conceptual datasets. Time series aggregations, union aggregations (associated variables in separate files), and ensemble membership are the most obvious applications of this. Another important application is "tiled grids" (ref. the "gridspec" proposal). - Steve On 3/27/2016 9:24 PM, Ethan Davis wrote: Hi all, As some of you have heard, we are holding a meeting to discuss current and future netCDF-CF efforts and directions. The meeting will be held on 24-26 May 2016 in Boulder, CO, USA at the UCAR Center Green facility. This meeting is organized by the “Advancing netCDF-CF for Geoscience” project [1] which has been funded by the US NSF EarthCube program. The project goals include several specific CF development efforts, both standards development and software prototyping. (We are just starting to spin-up community conversations around these specific goals for drafting CF extensions.) Another aspect of the funded project is community engagement including with the existing CF community, geoscience domains that have not been very involved in CF, and other standards bodies (e.g., OGC). These and other topics are on the current list of possible agenda items. Please let us know if you have other agenda items in mind. The current list of possible agenda items includes: * Investigate using features of the netCDF enhanced data model to improve/simplify the CF Discrete Sampling Geometries (which includes point, sounding, and trajectory data types) * Drafting a CF enhancement that would add support for complex data footprints (e.g., river segments and drainage basins) * Further develop the CFRadial proposal for representing Radar data * Adding features to better support satellite data * Adding CF support for hierarchical groups for organizing data and metadata * Enhancing CF to support Linked Data / semantic technology concepts * Harmonizing CF grid mapping with OGC CRS (coordinate reference systems), possibly with WKT * Adding CF support for attribute namespaces and the use of URI prefixes in attribute values * Improved support for dimensionless data variables, logarithmic scales, differences, etc. While we wish we could host all those interested in attending, given the available meeting space and the number of project participants, we are asking members of the CF community that are interested in attending to contact us directly and let us know which aspects of CF are of interest to you and your goals for the meeting. Please submit your expressions of interest by Monday, 4 April 2016. We plan to invite participants by Friday, 8 April 2016. Travel and other information will be on the meeting page [2] later this week. Also, for anyone attending EGU in Vienna in April, we will have a splinter meeting (SMP32) to discuss netCDF-CF. The splinter meeting will be held on Thursday, 21 April 2016 at 15:30 in room 2.96. Thank you, Meeting Organizers Ethan Davis, UCAR Unidata Charlie Zender, Univ of CA, Irvine David Arctur, Univ of TX, Austin Dave Santek, Univ of WI, Madison / SSEC Kevin O’Brien, Univ of WA/JISAO and NOAA/PMEL Aleksandar Jelenak, The HDF Group Mike Dixon, NCAR/EOL [1] http://www.nsf.gov/awardsearch/showAward?AWD_ID=1541031 [2] http://www.unidata.ucar.edu/events/2016CFWorkshop/ ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu<mailto:CF-metadata@cgd.ucar.edu>http://mailman.cgd.ucar.edu/mail
Re: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May 2016, Boulder, CO, USA
Hi Randy, Sorry you won't be able to join us in May. Thanks for the GOES-R / CF document! I'm sure it will be a good resource in discussions about CF and satellite data. The project folks focused on satellite data are planning to use the existing cf-satell...@unidata.ucar.edu email list [1] to make sure the broader community can easily get involved in that conversation. Also, we will have some form of remote participation available. Let me know if you want to participate in any parts of the meeting that way. Cheers, Ethan [1] http://www.unidata.ucar.edu/support/index.html#mailinglists On Mon, Mar 28, 2016 at 5:50 AM, Randy Horne wrote: > Ethan: > > > RE: > > >- Adding features to better support satellite data > > > > See attached white paper for how to extend CF conventions for GOES-R level > 1b space weather data. These conventions will work for polar orbiting > satellite level 1b space weather data also. Level 2 space weather products > ill require additional constructs, > > Business has been lousy. Otherwise, I would love to come ! > > > very respectfully, > > randy > > > > > > > > On Mar 28, 2016, at 12:24 AM, Ethan Davis wrote: > > Hi all, > > As some of you have heard, we are holding a meeting to discuss current and > future netCDF-CF efforts and directions. The meeting will be held on 24-26 > May 2016 in Boulder, CO, USA at the UCAR Center Green facility. > > This meeting is organized by the “Advancing netCDF-CF for Geoscience” > project [1] which has been funded by the US NSF EarthCube program. The > project goals include several specific CF development efforts, both > standards development and software prototyping. (We are just starting to > spin-up community conversations around these specific goals for drafting CF > extensions.) Another aspect of the funded project is community engagement > including with the existing CF community, geoscience domains that have not > been very involved in CF, and other standards bodies (e.g., OGC). These and > other topics are on the current list of possible agenda items. Please let > us know if you have other agenda items in mind. > > The current list of possible agenda items includes: > >- Investigate using features of the netCDF enhanced data model to >improve/simplify the CF Discrete Sampling Geometries (which includes point, >sounding, and trajectory data types) >- Drafting a CF enhancement that would add support for complex data >footprints (e.g., river segments and drainage basins) >- Further develop the CFRadial proposal for representing Radar data >- Adding features to better support satellite data >- Adding CF support for hierarchical groups for organizing data and >metadata >- Enhancing CF to support Linked Data / semantic technology concepts >- Harmonizing CF grid mapping with OGC CRS (coordinate reference >systems), possibly with WKT >- Adding CF support for attribute namespaces and the use of URI >prefixes in attribute values >- Improved support for dimensionless data variables, logarithmic >scales, differences, etc. > > While we wish we could host all those interested in attending, given the > available meeting space and the number of project participants, we are > asking members of the CF community that are interested in attending to > contact us directly and let us know which aspects of CF are of interest to > you and your goals for the meeting. Please submit your expressions of > interest by Monday, 4 April 2016. We plan to invite participants by Friday, > 8 April 2016. > > Travel and other information will be on the meeting page [2] later this > week. > > Also, for anyone attending EGU in Vienna in April, we will have a splinter > meeting (SMP32) to discuss netCDF-CF. The splinter meeting will be held on > Thursday, 21 April 2016 at 15:30 in room 2.96. > > Thank you, > > Meeting Organizers > > Ethan Davis, UCAR Unidata > Charlie Zender, Univ of CA, Irvine > David Arctur, Univ of TX, Austin > Dave Santek, Univ of WI, Madison / SSEC > Kevin O’Brien, Univ of WA/JISAO and NOAA/PMEL > Aleksandar Jelenak, The HDF Group > Mike Dixon, NCAR/EOL > > [1] http://www.nsf.gov/awardsearch/showAward?AWD_ID=1541031 > > [2] http://www.unidata.ucar.edu/events/2016CFWorkshop/ > ___ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > > > _ > > Randy C Horne (rho...@excaliburlabs.com) > Principal Engineer, Excalibur Laboratories Inc. > voice & fax: (321) 952.5100 > cell: (321) 693.1074 > url: http://www.excaliburlabs.com > > > > > ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May 2016, Boulder, CO, USA
Hi Steve, Thanks for those introducing those topics. I've added them to our list of topics and made note of Chris, David, and Roy's comments. I will get our list of possible agenda topics and a rough schedule up on the meeting page [1] today or tomorrow that we will update as the agenda solidifies. Cheers, Ethan [1] http://www.unidata.ucar.edu/events/2016CFWorkshop/ On Mon, Mar 28, 2016 at 10:25 AM, Steve Hankin wrote: > Ethan, > > Two more topics to consider as additions to your list: > > 1. "UGRID" -- Of interest to the coastal ocean modeling community, and > stuck in a holding pattern for quite a long time, is the harmonization of > mesh/unstructured grid coordinates into the main body of CF. The work is > mostly completed ... > > 2. "Aggregation" -- CF structures to link multiple files into larger > conceptual datasets. Time series aggregations, union aggregations > (associated variables in separate files), and ensemble membership are the > most obvious applications of this. Another important application is > "tiled grids" (ref. the "gridspec" proposal). > > - Steve > > > > > On 3/27/2016 9:24 PM, Ethan Davis wrote: > > Hi all, > > As some of you have heard, we are holding a meeting to discuss current and > future netCDF-CF efforts and directions. The meeting will be held on 24-26 > May 2016 in Boulder, CO, USA at the UCAR Center Green facility. > > This meeting is organized by the “Advancing netCDF-CF for Geoscience” > project [1] which has been funded by the US NSF EarthCube program. The > project goals include several specific CF development efforts, both > standards development and software prototyping. (We are just starting to > spin-up community conversations around these specific goals for drafting CF > extensions.) Another aspect of the funded project is community engagement > including with the existing CF community, geoscience domains that have not > been very involved in CF, and other standards bodies (e.g., OGC). These and > other topics are on the current list of possible agenda items. Please let > us know if you have other agenda items in mind. > > The current list of possible agenda items includes: > >- Investigate using features of the netCDF enhanced data model to >improve/simplify the CF Discrete Sampling Geometries (which includes point, >sounding, and trajectory data types) >- Drafting a CF enhancement that would add support for complex data >footprints (e.g., river segments and drainage basins) >- Further develop the CFRadial proposal for representing Radar data >- Adding features to better support satellite data >- Adding CF support for hierarchical groups for organizing data and >metadata >- Enhancing CF to support Linked Data / semantic technology concepts >- Harmonizing CF grid mapping with OGC CRS (coordinate reference >systems), possibly with WKT >- Adding CF support for attribute namespaces and the use of URI >prefixes in attribute values >- Improved support for dimensionless data variables, logarithmic >scales, differences, etc. > > While we wish we could host all those interested in attending, given the > available meeting space and the number of project participants, we are > asking members of the CF community that are interested in attending to > contact us directly and let us know which aspects of CF are of interest to > you and your goals for the meeting. Please submit your expressions of > interest by Monday, 4 April 2016. We plan to invite participants by Friday, > 8 April 2016. > > Travel and other information will be on the meeting page [2] later this > week. > > Also, for anyone attending EGU in Vienna in April, we will have a splinter > meeting (SMP32) to discuss netCDF-CF. The splinter meeting will be held on > Thursday, 21 April 2016 at 15:30 in room 2.96. > > Thank you, > > Meeting Organizers > > Ethan Davis, UCAR Unidata > Charlie Zender, Univ of CA, Irvine > David Arctur, Univ of TX, Austin > Dave Santek, Univ of WI, Madison / SSEC > Kevin O’Brien, Univ of WA/JISAO and NOAA/PMEL > Aleksandar Jelenak, The HDF Group > Mike Dixon, NCAR/EOL > > [1] http://www.nsf.gov/awardsearch/showAward?AWD_ID=1541031 > > [2] http://www.unidata.ucar.edu/events/2016CFWorkshop/ > > > ___ > CF-metadata mailing > listcf-metad...@cgd.ucar.eduhttp://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > > > ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May 2016, Boulder, CO, USA
Ethan:I would very much appreciate participating remotely when the discussions of space weather products came up.I am already a member of cf_satellite so no issue.The problem with my tech memo is there is SO MUCH THERE. Discussions on the message board usually attack one “construct” at a time. It will be overwhelming without an overarching introduction (which I would be glad to do). Note that I have had a quick email exchange with W. Kent Tobiska ("W. Kent Tobiska" ) who is a top authority on space weather in general, but I don’t think he is plugged into netCDF/cf. I think he has been laying the groundwork/foundation for many types of level 2 and level 3 space weather products and using ISO as a means to standardize. The ASCII product files that his efforts generate are self-describing, but they are not done in a standards-based way like CF requires, and he is not using netCDF. I have attached an ISO working draft of the kind of work he is doing. I suspect it would be a formidable activity to get everybody on the same page.I wish I could spend more time on it, but I have got to pay the bills. Thanks !randy
ISO_WD_Std for Ops_201504.docx
Description: MS-Word 2007 document
_Randy C Horne (rho...@excaliburlabs.com)Principal Engineer, Excalibur Laboratories Inc.voice & fax: (321) 952.5100cell: (321) 693.1074url: http://www.excaliburlabs.comOn Mar 30, 2016, at 3:29 PM, Ethan Davis wrote:Hi Randy,Sorry you won't be able to join us in May.Thanks for the GOES-R / CF document! I'm sure it will be a good resource in discussions about CF and satellite data. The project folks focused on satellite data are planning to use the existing cf-satell...@unidata.ucar.edu email list [1] to make sure the broader community can easily get involved in that conversation.Also, we will have some form of remote participation available. Let me know if you want to participate in any parts of the meeting that way.Cheers,Ethan[1] http://www.unidata.ucar.edu/support/index.html#mailinglistsOn Mon, Mar 28, 2016 at 5:50 AM, Randy Horne wrote:Ethan:RE:Adding features to better support satellite dataSee attached white paper for how to extend CF conventions for GOES-R level 1b space weather data. These conventions will work for polar orbiting satellite level 1b space weather data also. Level 2 space weather products ill require additional constructs,Business has been lousy. Otherwise, I would love to come !very respectfully,randyOn Mar 28, 2016, at 12:24 AM, Ethan Davis wrote:Hi all,As some of you have heard, we are holding a meeting to discuss current and future netCDF-CF efforts and directions. The meeting will be held on 24-26 May 2016 in Boulder, CO, USA at the UCAR Center Green facility.This meeting is organized by the “Advancing netCDF-CF for Geoscience” project [1] which has been funded by the US NSF EarthCube program. The project goals include several specific CF development efforts, both standards development and software prototyping. (We are just starting to spin-up community conversations around these specific goals for drafting CF extensions.) Another aspect of the funded project is community engagement including with the existing CF community, geoscience domains that have not been very involved in CF, and other standards bodies (e.g., OGC). These and other topics are on the current list of possible agenda items. Please let us know if you have other agenda items in mind.The current list of possible agenda items includes:Investigate using features of the netCDF enhanced data model to improve/simplify the CF Discrete Sampling Geometries (which includes point, sounding, and trajectory data types)Drafting a CF enhancement that would add support for complex data footprints (e.g., river segments and drainage basins)Further develop the CFRadial proposal for representing Radar dataAdding features to better support satellite dataAdding CF support for hierarchical groups for organizing data and metadataEnhancing CF to support Linked Data / semantic technology conceptsHarmonizing CF grid mapping with OGC CRS (coordinate reference systems), possibly with WKTAdding CF support for attribute namespaces and the use of URI prefixes in attribute valuesImproved support for dimensionless data variables, logarithmic scales, differences, etc.While we wish we could host all those interested in attending, given the available meeting space and the number of project participants, we are asking members of the CF community that are interested in attending to contact us directly and let us know which aspects of CF are of interest to you and your goals for the meeting. Please submit your expressions of interest by Monday, 4 April 2016. We plan to invite participants by Friday, 8 April 2016.Travel and other information will be on the meeting page [2] later this week.Also, for anyone attending EGU in Vienna in April, we will have
Re: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May 2016, Boulder, CO, USA
Hi Jonathan, Thanks for your comments. I've added them to our agenda planning notes. We'll work to get all this integrated in some form for the meeting and future discussions. Yes, though I'm not that familiar with the details of UGRID, it does (now that you mention it) seem related in some ways to the river/basin topic. I've been thinking of the river/basin issue as an extension to cell bounds and the footprint of a data value. But, at least in the case of river segments and drainage basins, capturing the connections between those 1-D and 2-D objects seems much more appropriate. It would also be nice to use similar mechanisms so that hydrological model outputs could interoperate nicely with estuary and coastal modeling. Cheers, Ethan On Tue, Mar 29, 2016 at 6:43 AM, Jonathan Gregory wrote: > Dear Ethan > > I agree with Steve about ugrid and aggregation being important topics. > > > 1. "UGRID" > > The ugrid design is rather CF-compatible and it would be good if they > could be > cemented together somehow. ugrid could be a proposed as a chapter of CF > but it > doesn't have to be, of course. I suppose the advantage of doing so would be > that CF and ugrid development would subsequently remain compatible. > > Could ugrid deal with this one of your agenda items: > > > * Drafting a CF enhancement that would add support for complex data > > footprints (e.g., river segments and drainage basins) > > I'm not sure what you have in mind but it sounds possibly related. > > > 2. "Aggregation" -- CF structures to link multiple files into larger > > conceptual datasets. Time series aggregations, union aggregations > > (associated variables in separate files), and ensemble membership > > are the most obvious applications of this. Another important > > application is "tiled grids" (ref. the "gridspec" proposal). > > There are various types of aggregation which should be distinguished, I > think. > > * Aggregation of variables. This is needed when one variable is split among > several files (often along the time axis) but it can also apply to > variables in > the same file (for instance combining ensemble members or time series with > the > same time coordinates - Steve's other examples). In general it means > combining > several variables to make one variable, by concatenating one or more of > their > axes. This can be done entirely using CF metadata and David Hassell and I > wrote > a CF proposal for it http://cf-trac.llnl.gov/trac/ticket/78 a long time > ago, > which David has since implemented in cf-python software. I believe that > this > approach is fine for CMIP6 purposes, for example. > > * Aggregation of domains. By this I mean regarding domains which have their > own axes as part of a larger domain, like gridspec does. It's not > aggregation > of variables in the same sense as above because the axes can't be > concatenated. > It is similar to ugrid, though, and I think it would be good if an > extension > to CF like ugrid could be developed for this. > > * Aggregation of files. By this I mean regarding several files as one file > *without* associating or joining variables together, just putting them into > one container. This needs rules for dealing with collisions of identically > named variables, like NCO ncks has, or groups could be used, which are > also on > Ethan's agenda. > > > Harmonizing CF grid mapping with OGC CRS (coordinate reference > > systems), possibly with WKT > > It should be possible to translate between CF and OGC CRS descriptions. > Some > extensions to help with this are in http://cf-trac.llnl.gov/trac/ticket/80 > , > which will be in CF 1.7, as will http://cf-trac.llnl.gov/trac/ticket/69, > which > allows WKT to be stored in a CF attribute. CF and OGC CRS have different > views > of the data space, so I don't think we can expect to make them look the > same, > but we should be able to define a mapping for all those concepts which are > in > both. > > Best wishes > > Jonathan > > - Forwarded message from Ethan Davis - > > > Date: Sun, 27 Mar 2016 22:24:56 -0600 > > From: Ethan Davis > > To: CF metadata , netCDF SWG > > > > Subject: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May > 2016, > > Boulder, CO, USA > > > > Hi all, > > > > As some of you have heard, we are holding a meeting to discuss current > and > > future netCDF-CF efforts and directions. The meeting will be held on > 24-26 > > May 2016 in Boulder, CO, USA at the UCAR Center Green facility. > > > > This meeting is organized by the “Advancing netCDF-CF for Geoscience” > > project [1] which has been funded by the US NSF EarthCube program. The > > project goals include several specific CF development efforts, both > > standards development and software prototyping. (We are just starting to > > spin-up community conversations around these specific goals for drafting > CF > > extensions.) Another aspect of the funded project is community engagement > > including with the existing CF co
Re: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May 2016, Boulder, CO, USA
Hi David, The proposed CF aggregation rules (http://cf-trac.llnl.gov/trac/ticket/78, http://www.met.reading.ac.uk/~david/cf_aggregation_rules.html) have been designed to aggregate *any* CF-compliant datasets, where appropriate. I would be very interested in looking at the metadata of some of your files to (hopefully!) confirm this in your use case. A netCDF schema for storing the results of such aggregations has also bee developed (http://www.met.reading.ac.uk/~david/cfa/0.4/). As far as I'm aware, there's not yet much software which understands these frameworks, apart from cf-python. All the best, David Original message from Moroni, David F (398G) (12AM 29 Mar 16) > Date: Tue, 29 Mar 2016 00:01:00 + > From: "Moroni, David F (398G)" > To: Steve Hankin , Ethan Davis , > CF metadata , netCDF SWG > > Subject: Re: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May > 2016, Boulder, CO, USA > user-agent: Microsoft-MacOutlook/14.6.2.160219 > > Hi Ethan, > > I’m glad Steve brought up the “aggregation” topic. On this topic, it would be > interesting to see if anyone has successfully come up with a standardized > schema to aggregate satellite-derived Level 2 swath grid structures. > > I know there’s an aggregation implementation in the latest OPeNDAP release, > but it remains to be seen what this might look like for Level 2 (and perhaps > even Level 1) swath grid structured datasets. I don’t know of anyone who’s > successfully tested and released an operationally working instance of that > part of OPeNDAP for swath grid structured datasets. > > Cheers, > David > > From: CF-metadata > mailto:cf-metadata-boun...@cgd.ucar.edu>> > on behalf of Steve Hankin > mailto:steven.c.han...@noaa.gov>> > Date: Monday, March 28, 2016 at 9:25 AM > To: Ethan Davis mailto:eda...@ucar.edu>>, CF metadata > mailto:cf-metadata@cgd.ucar.edu>>, netCDF SWG > mailto:netcdf@lists.opengeospatial.org>> > Subject: Re: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May > 2016, Boulder, CO, USA > > Ethan, > > Two more topics to consider as additions to your list: > > 1. "UGRID" -- Of interest to the coastal ocean modeling community, and stuck > in a holding pattern for quite a long time, is the harmonization of > mesh/unstructured grid coordinates into the main body of CF. The work is > mostly completed ... > > 2. "Aggregation" -- CF structures to link multiple files into larger > conceptual datasets. Time series aggregations, union aggregations > (associated variables in separate files), and ensemble membership are the > most obvious applications of this. Another important application is "tiled > grids" (ref. the "gridspec" proposal). > > - Steve > > > > On 3/27/2016 9:24 PM, Ethan Davis wrote: > Hi all, > > As some of you have heard, we are holding a meeting to discuss current and > future netCDF-CF efforts and directions. The meeting will be held on 24-26 > May 2016 in Boulder, CO, USA at the UCAR Center Green facility. > > This meeting is organized by the “Advancing netCDF-CF for Geoscience” project > [1] which has been funded by the US NSF EarthCube program. The project goals > include several specific CF development efforts, both standards development > and software prototyping. (We are just starting to spin-up community > conversations around these specific goals for drafting CF extensions.) > Another aspect of the funded project is community engagement including with > the existing CF community, geoscience domains that have not been very > involved in CF, and other standards bodies (e.g., OGC). These and other > topics are on the current list of possible agenda items. Please let us know > if you have other agenda items in mind. > > The current list of possible agenda items includes: > > * Investigate using features of the netCDF enhanced data model to > improve/simplify the CF Discrete Sampling Geometries (which includes point, > sounding, and trajectory data types) > * Drafting a CF enhancement that would add support for complex data > footprints (e.g., river segments and drainage basins) > * Further develop the CFRadial proposal for representing Radar data > * Adding features to better support satellite data > * Adding CF support for hierarchical groups for organizing data and > metadata > * Enhancing CF to support Linked Data / semantic technology concepts > * Harmonizing CF grid mapping with OGC CRS (coordinate reference > systems), possibly with WKT > * Adding CF support for attribute namespaces and the use
Re: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May 2016, Boulder, CO, USA
Hi David: I was unaware of these, thanks. This makes all the more important to look at the new HDF5 virtual datasets, which are basically virtual aggregations, in the sense of making sure that there is some harmonization. This is a new feature, and nows the time to make certain, if at all possible, that CF aggregation rules can be successfully implemented in the new virtual files. -Roy > On Mar 31, 2016, at 4:39 AM, David Hassell wrote: > > Hi David, > > The proposed CF aggregation rules > (http://cf-trac.llnl.gov/trac/ticket/78, > http://www.met.reading.ac.uk/~david/cf_aggregation_rules.html) have > been designed to aggregate *any* CF-compliant datasets, where > appropriate. I would be very interested in looking at the metadata of > some of your files to (hopefully!) confirm this in your use case. > > A netCDF schema for storing the results of such aggregations has also > bee developed (http://www.met.reading.ac.uk/~david/cfa/0.4/). > > As far as I'm aware, there's not yet much software which understands > these frameworks, apart from cf-python. > > All the best, > > David > > Original message from Moroni, David F (398G) (12AM 29 Mar 16) > >> Date: Tue, 29 Mar 2016 00:01:00 + >> From: "Moroni, David F (398G)" >> To: Steve Hankin , Ethan Davis , >> CF metadata , netCDF SWG >> >> Subject: Re: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May >> 2016, Boulder, CO, USA >> user-agent: Microsoft-MacOutlook/14.6.2.160219 >> >> Hi Ethan, >> >> I’m glad Steve brought up the “aggregation” topic. On this topic, it would >> be interesting to see if anyone has successfully come up with a standardized >> schema to aggregate satellite-derived Level 2 swath grid structures. >> >> I know there’s an aggregation implementation in the latest OPeNDAP release, >> but it remains to be seen what this might look like for Level 2 (and perhaps >> even Level 1) swath grid structured datasets. I don’t know of anyone who’s >> successfully tested and released an operationally working instance of that >> part of OPeNDAP for swath grid structured datasets. >> >> Cheers, >> David >> >> From: CF-metadata >> mailto:cf-metadata-boun...@cgd.ucar.edu>> >> on behalf of Steve Hankin >> mailto:steven.c.han...@noaa.gov>> >> Date: Monday, March 28, 2016 at 9:25 AM >> To: Ethan Davis mailto:eda...@ucar.edu>>, CF metadata >> mailto:cf-metadata@cgd.ucar.edu>>, netCDF SWG >> mailto:netcdf@lists.opengeospatial.org>> >> Subject: Re: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May >> 2016, Boulder, CO, USA >> >> Ethan, >> >> Two more topics to consider as additions to your list: >> >> 1. "UGRID" -- Of interest to the coastal ocean modeling community, and stuck >> in a holding pattern for quite a long time, is the harmonization of >> mesh/unstructured grid coordinates into the main body of CF. The work is >> mostly completed ... >> >> 2. "Aggregation" -- CF structures to link multiple files into larger >> conceptual datasets. Time series aggregations, union aggregations >> (associated variables in separate files), and ensemble membership are the >> most obvious applications of this. Another important application is "tiled >> grids" (ref. the "gridspec" proposal). >> >>- Steve >> >> >> >> On 3/27/2016 9:24 PM, Ethan Davis wrote: >> Hi all, >> >> As some of you have heard, we are holding a meeting to discuss current and >> future netCDF-CF efforts and directions. The meeting will be held on 24-26 >> May 2016 in Boulder, CO, USA at the UCAR Center Green facility. >> >> This meeting is organized by the “Advancing netCDF-CF for Geoscience” >> project [1] which has been funded by the US NSF EarthCube program. The >> project goals include several specific CF development efforts, both >> standards development and software prototyping. (We are just starting to >> spin-up community conversations around these specific goals for drafting CF >> extensions.) Another aspect of the funded project is community engagement >> including with the existing CF community, geoscience domains that have not >> been very involved in CF, and other standards bodies (e.g., OGC). These and >> other topics are on the current list of possible agenda items. Please let us >> know if you have other agenda items in mind. >> >> The current list of possi
Re: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May 2016, Boulder, CO, USA
Hi Ethan, It occurred to me that we should extend this invitation to a couple of PO.DAAC's key software developers that are leading the development effort for our Metadata Compliance Checker (MCC), Mike Gangl and Josh Rodriguez respectively. I’ve cc’d them here to make the introduction. For those who are unaware, the MCC tests for both ACDD and CF compliance, and it is available on our UAT server: http://podaac-uat.jpl.nasa.gov/mcc/ Please welcome Mike and Josh to this community. Cheers, David From: CF-metadata mailto:cf-metadata-boun...@cgd.ucar.edu>> on behalf of Ethan Davis mailto:eda...@ucar.edu>> Date: Sunday, March 27, 2016 at 9:24 PM To: CF metadata mailto:cf-metadata@cgd.ucar.edu>>, netCDF SWG mailto:netcdf@lists.opengeospatial.org>> Subject: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May 2016, Boulder, CO, USA Hi all, As some of you have heard, we are holding a meeting to discuss current and future netCDF-CF efforts and directions. The meeting will be held on 24-26 May 2016 in Boulder, CO, USA at the UCAR Center Green facility. This meeting is organized by the “Advancing netCDF-CF for Geoscience” project [1] which has been funded by the US NSF EarthCube program. The project goals include several specific CF development efforts, both standards development and software prototyping. (We are just starting to spin-up community conversations around these specific goals for drafting CF extensions.) Another aspect of the funded project is community engagement including with the existing CF community, geoscience domains that have not been very involved in CF, and other standards bodies (e.g., OGC). These and other topics are on the current list of possible agenda items. Please let us know if you have other agenda items in mind. The current list of possible agenda items includes: * Investigate using features of the netCDF enhanced data model to improve/simplify the CF Discrete Sampling Geometries (which includes point, sounding, and trajectory data types) * Drafting a CF enhancement that would add support for complex data footprints (e.g., river segments and drainage basins) * Further develop the CFRadial proposal for representing Radar data * Adding features to better support satellite data * Adding CF support for hierarchical groups for organizing data and metadata * Enhancing CF to support Linked Data / semantic technology concepts * Harmonizing CF grid mapping with OGC CRS (coordinate reference systems), possibly with WKT * Adding CF support for attribute namespaces and the use of URI prefixes in attribute values * Improved support for dimensionless data variables, logarithmic scales, differences, etc. While we wish we could host all those interested in attending, given the available meeting space and the number of project participants, we are asking members of the CF community that are interested in attending to contact us directly and let us know which aspects of CF are of interest to you and your goals for the meeting. Please submit your expressions of interest by Monday, 4 April 2016. We plan to invite participants by Friday, 8 April 2016. Travel and other information will be on the meeting page [2] later this week. Also, for anyone attending EGU in Vienna in April, we will have a splinter meeting (SMP32) to discuss netCDF-CF. The splinter meeting will be held on Thursday, 21 April 2016 at 15:30 in room 2.96. Thank you, Meeting Organizers Ethan Davis, UCAR Unidata Charlie Zender, Univ of CA, Irvine David Arctur, Univ of TX, Austin Dave Santek, Univ of WI, Madison / SSEC Kevin O’Brien, Univ of WA/JISAO and NOAA/PMEL Aleksandar Jelenak, The HDF Group Mike Dixon, NCAR/EOL [1] http://www.nsf.gov/awardsearch/showAward?AWD_ID=1541031 [2] http://www.unidata.ucar.edu/events/2016CFWorkshop/ ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May 2016, Boulder, CO, USA
Hi Randy, Great, I'll add you to the list of folks planning to join remotely. Yes, there is a lot in that tech memo. An overview/introduction sounds very useful. Thanks for the information on the ISO work dealing with space weather. I've had some conversations with others interested in this work. I'll send a separate email shortly with more on that. Cheers, Ethan On Wed, Mar 30, 2016 at 1:49 PM, Randy Horne wrote: > Ethan: > > I would very much appreciate participating remotely when the discussions > of space weather products came up. > > I am already a member of cf_satellite so no issue. > > The problem with my tech memo is there is SO MUCH THERE. Discussions on > the message board usually attack one “construct” at a time. It will be > overwhelming without an overarching introduction (which I would be glad to > do). > > Note that I have had a quick email exchange with *W. Kent Tobiska (**"W. > Kent Tobiska" >) **who is a top authority on space > weather in general, but I don’t think he is plugged into netCDF/cf. I > think he has been laying the groundwork/foundation for many types of level > 2 and level 3 space weather products and using ISO as a means > to standardize. The ASCII product files that his efforts generate are > self-describing, but they are not done in a standards-based way like CF > requires, and he is not using netCDF. ** I have attached an ISO working > draft of the kind of work he is doing. I suspect it would be a **formidable > activity to get everybody on the same page.* > > *I wish I could spend more time on it, but I have got to pay the bills. * > > Thanks ! > > randy > > > > _ > > Randy C Horne (rho...@excaliburlabs.com) > Principal Engineer, Excalibur Laboratories Inc. > voice & fax: (321) 952.5100 > cell: (321) 693.1074 > url: http://www.excaliburlabs.com > > > > On Mar 30, 2016, at 3:29 PM, Ethan Davis wrote: > > Hi Randy, > > Sorry you won't be able to join us in May. > > Thanks for the GOES-R / CF document! I'm sure it will be a good resource > in discussions about CF and satellite data. The project folks focused on > satellite data are planning to use the existing > cf-satell...@unidata.ucar.edu email list [1] to make sure the broader > community can easily get involved in that conversation. > > Also, we will have some form of remote participation available. Let me > know if you want to participate in any parts of the meeting that way. > > Cheers, > > Ethan > > [1] http://www.unidata.ucar.edu/support/index.html#mailinglists > > On Mon, Mar 28, 2016 at 5:50 AM, Randy Horne > wrote: > >> Ethan: >> >> >> RE: >> >> >>- Adding features to better support satellite data >> >> >> >> See attached white paper for how to extend CF conventions for GOES-R >> level 1b space weather data. These conventions will work for polar >> orbiting satellite level 1b space weather data also. Level 2 space weather >> products ill require additional constructs, >> >> Business has been lousy. Otherwise, I would love to come ! >> >> >> very respectfully, >> >> randy >> >> >> >> >> >> >> >> On Mar 28, 2016, at 12:24 AM, Ethan Davis wrote: >> >> Hi all, >> >> As some of you have heard, we are holding a meeting to discuss current >> and future netCDF-CF efforts and directions. The meeting will be held on >> 24-26 May 2016 in Boulder, CO, USA at the UCAR Center Green facility. >> >> This meeting is organized by the “Advancing netCDF-CF for Geoscience” >> project [1] which has been funded by the US NSF EarthCube program. The >> project goals include several specific CF development efforts, both >> standards development and software prototyping. (We are just starting to >> spin-up community conversations around these specific goals for drafting CF >> extensions.) Another aspect of the funded project is community engagement >> including with the existing CF community, geoscience domains that have not >> been very involved in CF, and other standards bodies (e.g., OGC). These and >> other topics are on the current list of possible agenda items. Please let >> us know if you have other agenda items in mind. >> >> The current list of possible agenda items includes: >> >>- Investigate using features of the netCDF enhanced data model to >>improve/simplify the CF Discrete Sampling Geometries (which includes >> point, >>sounding, and trajectory data types) >>- Drafting a CF enhancement that would add support for complex data >>footprints (e.g., river segments and drainage basins) >>- Further develop the CFRadial proposal for representing Radar data >>- Adding features to better support satellite data >>- Adding CF support for hierarchical groups for organizing data and >>metadata >>- Enhancing CF to support Linked Data / semantic technology concepts >>- Harmonizing CF grid mapping with OGC CRS (coordinate reference >>systems), possibly with WKT >>- Adding CF support for attribute namespaces and the use of URI >>
Re: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May 2016, Boulder, CO, USA
On Wed, Mar 30, 2016 at 1:09 PM, Ethan Davis wrote: > Yes, though I'm not that familiar with the details of UGRID, it does (now > that you mention it) seem related in some ways to the river/basin topic. > well, maybe -- but while UGRIDs are "unstructured", there is SOME structure to them -- i.e. they are usually made up of cells with the same number of sides (triangles, quads, etc), or at least a small number (six or less). And the UGRID data structure does assume this. > I've been thinking of the river/basin > Rivers can be a 1-d "mesh" which there is support in UGRID for, but I think of basins as being arbitrary often high order polygons. Is there any standard for supporting polygons (i.e. what you might put in shape files) in netcdf? -Chris > On Tue, Mar 29, 2016 at 6:43 AM, Jonathan Gregory < > j.m.greg...@reading.ac.uk> wrote: > >> Dear Ethan >> >> I agree with Steve about ugrid and aggregation being important topics. >> >> > 1. "UGRID" >> >> The ugrid design is rather CF-compatible and it would be good if they >> could be >> cemented together somehow. ugrid could be a proposed as a chapter of CF >> but it >> doesn't have to be, of course. I suppose the advantage of doing so would >> be >> that CF and ugrid development would subsequently remain compatible. >> >> Could ugrid deal with this one of your agenda items: >> >> > * Drafting a CF enhancement that would add support for complex data >> > footprints (e.g., river segments and drainage basins) >> >> I'm not sure what you have in mind but it sounds possibly related. >> >> > 2. "Aggregation" -- CF structures to link multiple files into larger >> > conceptual datasets. Time series aggregations, union aggregations >> > (associated variables in separate files), and ensemble membership >> > are the most obvious applications of this. Another important >> > application is "tiled grids" (ref. the "gridspec" proposal). >> >> There are various types of aggregation which should be distinguished, I >> think. >> >> * Aggregation of variables. This is needed when one variable is split >> among >> several files (often along the time axis) but it can also apply to >> variables in >> the same file (for instance combining ensemble members or time series >> with the >> same time coordinates - Steve's other examples). In general it means >> combining >> several variables to make one variable, by concatenating one or more of >> their >> axes. This can be done entirely using CF metadata and David Hassell and I >> wrote >> a CF proposal for it http://cf-trac.llnl.gov/trac/ticket/78 a long time >> ago, >> which David has since implemented in cf-python software. I believe that >> this >> approach is fine for CMIP6 purposes, for example. >> >> * Aggregation of domains. By this I mean regarding domains which have >> their >> own axes as part of a larger domain, like gridspec does. It's not >> aggregation >> of variables in the same sense as above because the axes can't be >> concatenated. >> It is similar to ugrid, though, and I think it would be good if an >> extension >> to CF like ugrid could be developed for this. >> >> * Aggregation of files. By this I mean regarding several files as one file >> *without* associating or joining variables together, just putting them >> into >> one container. This needs rules for dealing with collisions of identically >> named variables, like NCO ncks has, or groups could be used, which are >> also on >> Ethan's agenda. >> >> > Harmonizing CF grid mapping with OGC CRS (coordinate reference >> > systems), possibly with WKT >> >> It should be possible to translate between CF and OGC CRS descriptions. >> Some >> extensions to help with this are in >> http://cf-trac.llnl.gov/trac/ticket/80, >> which will be in CF 1.7, as will http://cf-trac.llnl.gov/trac/ticket/69, >> which >> allows WKT to be stored in a CF attribute. CF and OGC CRS have different >> views >> of the data space, so I don't think we can expect to make them look the >> same, >> but we should be able to define a mapping for all those concepts which >> are in >> both. >> >> Best wishes >> >> Jonathan >> >> - Forwarded message from Ethan Davis - >> >> > Date: Sun, 27 Mar 2016 22:24:56 -0600 >> > From: Ethan Davis >> > To: CF metadata , netCDF SWG >> > >> > Subject: [CF-metadata] Meeting invite: Advancing netCDF-CF, 24-26 May >> 2016, >> > Boulder, CO, USA >> > >> > Hi all, >> > >> > As some of you have heard, we are holding a meeting to discuss current >> and >> > future netCDF-CF efforts and directions. The meeting will be held on >> 24-26 >> > May 2016 in Boulder, CO, USA at the UCAR Center Green facility. >> > >> > This meeting is organized by the “Advancing netCDF-CF for Geoscience” >> > project [1] which has been funded by the US NSF EarthCube program. The >> > project goals include several specific CF development efforts, both >> > standards development and software prototyping. (We are just starting to >> > spin-up community conversati
