On Mon, 15 Mar 2010 12:09:39 -0500 (CDT), Roy Stogner <[email protected]> wrote: > I think Jed recently mentioned inventing his own file format in a very > apologetic tone; at the time I was thinking that although reinventing > the wheel is a bad idea in general, it might be justified when your > selection of existing wheels is limited to a hexagonal design, an oval > design, a design made from concrete...
Of course my format is lame and nobody should use it. It's somewhat painful to extend because it puts everything into a hierarchical data model, which I think is fundamentally wrong because object relations are actually *relational* instead of hierarchical. My vis plugin needed to do a fair amount of walking hierarchies to pick up all the necessary relations in order to answer simple queries, and to maintain data integrity. One could write a general purpose query language on top of HDF5, but this seems seriously wrong. I think the only sane long-term solution, if we want to have a cleanly extensible format supporting unstructured multi-domain simulations in a way that we can write generic software for continuous/discrete adjoints and other problems of comparable sophistication, is to actually represent the relations using a relational database (the heavy data can be written to arbitrary files, named by SHA1 or whatever). Each piece of HDF5-walking code that I wrote would be 1-liners in SQL, and integrity would be far easier to assure. The database performance isn't critical since it's just used for metadata, and thus unlikely to have more than a few million records. Some day I'll do a SQLite prototype of this for PETSc Vecs based on DMComposite (very generic since it can contain any combination of structured, unstructured, and non-geometric function spaces). Jed ------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev _______________________________________________ Libmesh-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/libmesh-users
