[petsc-dev] Writing rich state

2010-02-24 Thread Jed Brown
On Wed, 24 Feb 2010 12:58:46 -0600, Dmitry Karpeev wrote: > It seems like optional attributes can be relatively easily implemented > by adding just two > extra columns to the table: "key" and "value" and then constructing > the corresponding > SQL queries something like (I haven't looked at SQL i

[petsc-dev] Writing rich state

2010-02-24 Thread Jed Brown
On Wed, 24 Feb 2010 09:25:39 -0600, Dmitry Karpeev wrote: > Here's the main problem that I see: are file attributes stored in the database > fixed? That could be somewhat inflexible (what if some attributes don't apply > to a given file? what if I want to add another attribute?). The alternati

[petsc-dev] Writing rich state

2010-02-24 Thread Jed Brown
On Wed, 24 Feb 2010 08:31:32 -0600, Dmitry Karpeev wrote: > Yes, I think SQL or some such approach would be a good solution. > I don't even think the actual file format matters too much: we can just > create collections of files that share keys. The database is needed only > to manage file names

[petsc-dev] Writing rich state

2010-02-24 Thread Matthew Knepley
On Wed, Feb 24, 2010 at 9:09 AM, Jed Brown wrote: > On Wed, 24 Feb 2010 08:31:32 -0600, Dmitry Karpeev > wrote: > > Yes, I think SQL or some such approach would be a good solution. > > I don't even think the actual file format matters too much: we can just > > create collections of files that sh

[petsc-dev] Writing rich state

2010-02-24 Thread Dmitry Karpeev
It seems like optional attributes can be relatively easily implemented by adding just two extra columns to the table: "key" and "value" and then constructing the corresponding SQL queries something like (I haven't looked at SQL in a long time) (WHERE(key="precision", value="double")), etc. I think

[petsc-dev] Writing rich state

2010-02-24 Thread Dmitry Karpeev
On Wed, Feb 24, 2010 at 9:09 AM, Jed Brown wrote: > On Wed, 24 Feb 2010 08:31:32 -0600, Dmitry Karpeev > wrote: >> Yes, I think SQL or some such approach would be a good solution. >> I don't even think the actual file format matters too much: we can just >> create collections of files that share

[petsc-dev] Writing rich state

2010-02-24 Thread Dmitry Karpeev
On Wed, Feb 24, 2010 at 12:59 AM, Jed Brown wrote: > On Tue, 23 Feb 2010 13:44:55 -0600, Dmitry Karpeev > wrote: >> Yes, but what about using Spotlight programmatically (e.g., from >> PETSc) to store rich state, checkpointing, etc? ?For example, I want >> to store a Vec. ?How do I label it? ?The

[petsc-dev] Writing rich state

2010-02-24 Thread Jed Brown
On Tue, 23 Feb 2010 13:44:55 -0600, Dmitry Karpeev wrote: > Yes, but what about using Spotlight programmatically (e.g., from > PETSc) to store rich state, checkpointing, etc? For example, I want > to store a Vec. How do I label it? There maybe various user contexts > that share it, so I'd like

[petsc-dev] Writing rich state

2010-02-23 Thread Jed Brown
Matt and I talked about this a couple months ago, but I'd like to also mention it here. It seems to me that data formats like HDF5 are really a pain to use for generic purposes, because you end up trying to map a directed graph of object relations (composition) into a hierarchical data format, and

[petsc-dev] Writing rich state

2010-02-23 Thread Barry Smith
On Feb 23, 2010, at 1:44 PM, Dmitry Karpeev wrote: > Yes, but what about using Spotlight programmatically (e.g., from > PETSc) to store rich state, > checkpointing, etc? > For example, I want to store a Vec. How do I label it? There maybe > various user contexts > that share it, so I'd like to

[petsc-dev] Writing rich state

2010-02-23 Thread Dmitry Karpeev
Yes, but what about using Spotlight programmatically (e.g., from PETSc) to store rich state, checkpointing, etc? For example, I want to store a Vec. How do I label it? There maybe various user contexts that share it, so I'd like to label it with all of them. In a way, I don't to have to look at

[petsc-dev] Writing rich state

2010-02-23 Thread Barry Smith
With google (and Spotlight on the Mac) is there any need to organize anything anymore? Just burp down the data any way you please anywhere you want it and then have smart search tools find it for you and format it the way you need it at the time you need it? This does mean you need dec

[petsc-dev] Writing rich state

2010-02-23 Thread Dmitry Karpeev
This takes the discussion in a somewhat tangential direction, but consider this: We use hierarchical file systems, which are also a pain. Say, I'm working on project PETSc and I'm writing a DOE proposal for it. Should I put it in ~/PETSc/Proposals/DOE/proposal or ~/Proposals/DOE/PETSc/proposal or

[petsc-dev] Writing rich state

2010-02-23 Thread Barry Smith
I've thought about this be never done anything, I think it is worth investigating. BTW: My long term goal is also that all PETSc source code lives in an appropriate database with appropriate relationships and meta-data stored there. The fact that we (meaning HPC and OpenSource in