Hi Philip,
Please read my response to Bryan Lawrence's post on another thread.
It pertains to many of your points. And I may duplicate parts below...
> Hi All,
>
> I like Steve Hankin's point (below) about 'powerful' versus
> 'interoperable' . I hadn't thought about it quite that way before :-).
Philip Cameron-Smith, p...@llnl.gov, Lawrence Livermore National Lab.
---
From: CF-metadata [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Steve
Hankin
Sent: Thursday, September 19, 2013 10:03 AM
To: Corey Bettenhausen
Cc: CF Metadata List
Subject: Re: [CF-metada
Hi Corey,
Le 19/09/2013 09:05, Corey Bettenhausen a écrit :
>
> On Sep 19, 2013, at 11:29 AM, Karl Taylor wrote:
>
>> Hi all,
>>
>> Again, I may be unaware of all the possible uses of hierarchies, but here's
>> our experience with CMIP.
>>
>> It seems to me if hierarchies are for the purpose of
Hello Nan,
Le 19/09/2013 12:16, Nan Galbraith a écrit :
> Hi all -
>> Did I misunderstand the original proposal?
>
> Could we possibly have a proposal written up as a ticket on the Trac
> system?
I'd prefer not to submit anything to the Trac system
until/unless there is consensus from many doubt
On Sep 19, 2013, at 13:58, Charlie Zender wrote:
> Nothing in our proposal mandates a single way of representing hierarchies. Au
> contraire.
The concern by the hierarchy-wary is not that you'd mandate a single way, but
that by providing an open mechanism, you would encourage a raft of compet
Hi Steve,
Your stern but steady caution about the tension between flexibility
and interoperability is well-taken here.
> What have been the down sides to the
> use of groups and hierarchies?
Those who respond to this please be careful to distinguish between
HDF5 _allowed_ hierarchies and netCDF4
Hi Karl,
Please see new thread
Are ensembles a compelling use case for "group-aware" metadata?
It is a thinly veiled reference to CMIP5, which I don't mention
by name so as not to associate it/you with our proposal.
Nevertheless, many on the list know what CMIP5 is and it has helped
climate mode
Hello Steve,
Responses interleaved.
Best,
cz
Le 18/09/2013 09:32, Steve Hankin a écrit :
>
>
> On 9/18/2013 7:56 AM, Roy Mendelssohn - NOAA Federal wrote:
>> Hi All:
>>
>> NASA has used hierarchies for years, and appears committed to them. So,
>> either it is done in an ad hoc way, or throug
Hi Roy,
Thanks for your latest 3 posts.
They all make the point that hierarchical files
are widely used now. In this post you make the point that maybe
having conventions on how to use/interpret hierarchies would
be helpful, regardless of its desirability for CF.
Yes, part of our motivation is be
Hi Bryan,
Responses interleaved.
Best,
cz
Le 18/09/2013 05:57, Bryan Lawrence a écrit :
> Hi Charlie
>
> (Before I disagree with you, like most on the list, I'm glad we're
> having the conversation on this topic, it needs to be had, so thanks!)
>
> I find this particular example completely unh
Hi all,
No need to respond further to my post. I missed that the groups were
*within* the file. I have now looked at the netCDF documentation about
groups and will hopefully contribute more cogent remarks later.
cheers,
Karl
On 9/19/13 9:05 AM, Corey Bettenhausen wrote:
On Sep 19, 2013, a
Hi all -
Did I misunderstand the original proposal?
Could we possibly have a proposal written up as a ticket on the Trac
system?
I'm finding it difficult to find a description of what's being
suggested, in
the many emails in this thread.
I realize we're very far from a detailed description
On 9/19/2013 9:05 AM, Corey Bettenhausen wrote:
On Sep 19, 2013, at 11:29 AM, Karl Taylor wrote:
Hi all,
Again, I may be unaware of all the possible uses of hierarchies, but here's our
experience with CMIP.
It seems to me if hierarchies are for the purpose of "organizing" datasets (or
orga
On Sep 19, 2013, at 11:29 AM, Karl Taylor wrote:
> Hi all,
>
> Again, I may be unaware of all the possible uses of hierarchies, but here's
> our experience with CMIP.
>
> It seems to me if hierarchies are for the purpose of "organizing" datasets
> (or organizing a bunch of files), this should
Hi all,
Again, I may be unaware of all the possible uses of hierarchies, but
here's our experience with CMIP.
It seems to me if hierarchies are for the purpose of "organizing"
datasets (or organizing a bunch of files), this should fall outside CF's
purview because a single hierarchy is rarel
On Sep 18, 2013, at 12:32 PM, Steve Hankin wrote:
>
>
> On 9/18/2013 7:56 AM, Roy Mendelssohn - NOAA Federal wrote:
>> Hi All:
>>
>> NASA has used hierarchies for years, and appears committed to them. So,
>> either it is done in an ad hoc way, or through a standard. That doesn't
>> mean CF i
Hi all,
I would like to second Steve's suggestion calling for "some compelling
use cases that will help us to understand the costs and benefits".
He is also right that although for CMIP5, we recommended a certain
directory structure (which I guess implies some hierarchy), that
structure is l
CF may well not be the proper place. I am not arguing that. All I am arguing
is that the history of computer science shows that hierarchies are often
beneficial compared to flat structures, whether it be the b-trees we use in our
directories to structures in programming languages, while a lot
On 9/18/2013 7:56 AM, Roy Mendelssohn - NOAA Federal wrote:
Hi All:
NASA has used hierarchies for years, and appears committed to them. So, either
it is done in an ad hoc way, or through a standard. That doesn't mean CF is
the place for the standard, just that it would be nice to have one.
Hi All:
NASA has used hierarchies for years, and appears committed to them. So, either
it is done in an ad hoc way, or through a standard. That doesn't mean CF is
the place for the standard, just that it would be nice to have one.
I would point out that every major modern programming languag
Hi Charlie
(Before I disagree with you, like most on the list, I'm glad we're having
the conversation on this topic, it needs to be had, so thanks!)
I find this particular example completely unhelpful, not least because I
don't see the utility for doing so. However, I can see that others might
se
All,
I'm glad we are discussing this topic, but the fact that large data
providers are already distributing data using groups and hierarchies
is not a compelling reason to endorse this practice through CF. After
all, a lot of data providers are currently distributing scientific
data in any number
Hi All:
I am old and slow, and I must be missing something, because at this point most
of the discussion has been about the desirability of files with groups and
hierarchies. Again, unless I am missing something, there already are data
providers who are distributing data using groups and hiera
Hello John,
> As I understand it, your starting concern is to be able to put
> things into folder-equivalents, pure and simple.
Our original post describes my goals. Groups are a logical place to
start because they must exist for inheritance to be meaningful.
I am for having 1. groups as containe
> The simplest support imaginable is that groups should be completely
> self-contained without attribute inheritance. This would go in the right
> direction,
My value proposition is rather the opposite. In preparing well-documented
netCDF files, I need some kind of inherited attributes, because
Hi Stephen,
I also advocate keeping support for groups simple.
The simplest support imaginable is that groups should be completely
self-contained without attribute inheritance. This would go in the
right direction, yet I think attribute inheritance, at least of global
metadata, ought to be strongl
Dear Jonathan,
Thanks for your input. I expect many here share your views.
Directories full of flat files are often a sensible way to organize.
Easy to search for and ingest what with ls or grep and options.
What limits such searchability within hierarchical files?
Hierarchical files look just li
Hello Bryan,
Thanks for your perspective/counterexample about whether CMIP5 is in
fact a natural candidate for hierarchies. I agree with your points
about the utility of flat systems of objects, while retaining my urge
to hierarchically organize some objects sometimes. Everyone packs
their suitcas
Bryan has beaten me to the points I would have made. I think hierarchies are
over rated at the interface level. Examples abound of where they have been
abandoned: hierarchal vs relational DBs, XML databases and tools (save us from
xquery for Netcdf!).
Under the hood hierarchies are often necess
Hi Folks
CMIP5 is illuminating in a number of ways ... not least because it is
impossible to come up with a *natural* hierarchy for consumers of the data
(as opposed to the producers). But even the producers have different ways
of organising their material (running members of different ensembles a
Dear Charlie
Thank you for your interesting post and the discussion.
As a data analyst, I have a different view from NASA. I dislike hierarchies
and directories. I prefer things to be as flat as possible, with each item
thoroughly described by its own independent metadata, using tools to identify
Hi Russ,
Thanks for your input and link to an earlier presentation of yours.
Agree that the proposal only applies to group hierarchies, i.e., to
groups representable by the Common Data Model 2/extended/enhanced
which for practical purposes means groups exposed by the netCDF4 API.
Your way of putt
Hi Martin,
When NCO's ncecat aggregates multiple files into a single file, it
does as you describe and puts each into its own group with all the
global metadata now as group metadata. What should be the new global
metadata of the group file? neceat duplicates the first file's
metadata for that, an
Dear all,
I very much welcome the initiative by Charlie, Ted and Peter on
hierarchical data structures. Without wanting to offend anyone, the arguments
against this brought forward by Steve and Richard sound to me a bit cowardly -
but of course they do have a point in trying to ensure backw
> Dear all,
I'm also glad to see this discussion surface. Since I first presented
"Developing Conventions for netCDF-4" at the 2007 GO-ESSP meeting:
http://www.unidata.ucar.edu/presentations/Rew/nc4-conventions.pdf
I've been hoping that netCDF-4 feature adoption would begin to gain
traction i
35 matches
Mail list logo
