Dav, A Dimecres 22 Març 2006 21:04, Dav Clark va escriure: > I'm using pytables (1.3beta2) with mod_python (3.2.8) to manage a brain > imaging data repository. > > Right now I break things a lot, and I managed to corrupt a file such that > pytables cannot read it. Matlab, however, can read it.
That's strange. It seems like the action log got corrupted. The action log is implemented as a table living in: /_p_transactions/actionlog; can you read it with Matlab? > > I am very interested in best-practices for maintaining integrity of my > files. I was using the undo feature for this, but suspect this was part of > the corruption... Well, best practices are basically: """Do not exit the program without closing first your files""" For the forthcoming PyTables 1.3 I've added some protection against hitting the ^C on a running program (by using the atexit module), so that the close() method is called on every opened PyTables file. However, I don't think this will offer protection on situations where the program is aborted unexpectedly due to other reasons. Another thing can be to call File.flush() regularly, specially after doing a writing operation. Be careful, because this may drastically reduce the writing performance. > > More details: > > 1) I have undo enabled on this file > > 2) Matlab 7.1 and 7.2 use HDF5 v1.6.2 and v1.6.4 (so far as I can tell), > and both can read the file > > 3) PyTables error claims 'HDF5-DIAG: Error detected in HDF5 library > version: 1.6.5 thread 3086534336.' (full error message below my sig) > > 4) The file was created with (non-pytables) matlab/hdf5 and later updated / > modified with pytables - mostly copying groups from other hdf5 files > recursively. > > 5) There were occassions (though I'm not sure about the one right before > the corruption) where mod_python would bomb out and didn't execute a close > on the file that was opened with mode 'a'. My guess is that the problem here should be the mod_python bomb out. Maybe you can try a recent snapshot of PyTables: http://pytables.carabos.com/download/snapshot/ and see if the "atexit" support would make your files more stable. If this does not help, I'd recommend you to flush from time to time. Cheers, -- >0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-" ------------------------------------------------------- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnk&kid0944&bid$1720&dat1642 _______________________________________________ Pytables-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/pytables-users
