Hello,
I am trying to track down a problem I'm having with pytables segfaulting. The
basic application involves a python thread that asynchronously posts data to a
Queue, and then another thread reads data from the Queue and writes it to an H5
file. The main thread opens a new H5 file, then closes it after a few seconds of
data has been written, then repeats opening and closing like this forever. I
have flags in place to ensure that the writing thread does not write to the h5
file while it's in the process of being closed. However, I am still getting seg
faults when closing the h5 file. Specifically I have traced it to closing the
data Table. I have found that if I add the lines
import gc
gc.collect()
the seg faulting seems to go away, but I imagine this is just hiding the
problem.
The output for print_versions on this machine is:
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
PyTables version: 2.0.4
HDF5 version: 1.8.0
NumPy version: 1.1.0
Zlib version: 1.2.3.3
LZO version: 2.02 (Oct 17 2005)
BZIP2 version: 1.0.4 (20-Dec-2006)
Python version: 2.5.2 (r252:60911, Jul 23 2008, 23:54:29)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)]
Platform: linux2-x86_64
Byte-ordering: little
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Just now, instead of a seg fault, I got a slightly useful error message:
HDF5-DIAG: Error detected in HDF5 (1.8.0) thread 0:
#000: H5Dio.c line 441 in H5Dwrite(): can't write data
major: Dataset
minor: Write failed
#001: H5Dio.c line 680 in H5D_write(): src and dest data spaces have different
sizes
major: Invalid arguments to routine
minor: Bad value
Traceback (most recent call last):
File "testh5.py", line 56, in <module>
myHandler.stop_recording()
File "/home/gej/spec/spec/spectroscopy/backend_components/datahandler.py",
line 192, in stop_recording
self.ts.close()
File "/home/gej/spec/spec/spectroscopy/data/timeseries.py", line 197, in close
self.h5.close()
File "/usr/local/lib/python2.5/site-packages/tables/file.py", line 2051, in
close
lambda path: aliveNodes[path])
File "/usr/local/lib/python2.5/site-packages/tables/file.py", line 2024, in _c
loseNodes
node._f_close()
File "/usr/local/lib/python2.5/site-packages/tables/table.py", line 2342, in
_f_close
self.flush()
File "/usr/local/lib/python2.5/site-packages/tables/table.py", line 2280, in
flush
self.row._flushBufferedRows()
File "tableExtension.pyx", line 1084, in tableExtension.Row._flushBufferedRows
File "/usr/local/lib/python2.5/site-packages/tables/table.py", line 1769, in
_saveBufferedRows
self._append_records(lenrows)
File "tableExtension.pyx", line 460, in tableExtension.Table._append_records
tables.exceptions.HDF5ExtError: Problems appending the records.
Running through gdb gives this back trace, which I don't imagine is very useful.
#0 0x00002b5ef7423765 in raise () from /lib/libc.so.6
#1 0x00002b5ef74251c0 in abort () from /lib/libc.so.6
#2 0x00002b5ef745c60b in ?? () from /lib/libc.so.6
#3 0x00002b5ef7467826 in free () from /lib/libc.so.6
#4 0x00002b5ef7a9d77e in H5MM_xfree () from /usr/lib/libhdf5.so.5
#5 0x00002b5ef7a111f2 in H5D_istore_flush_entry () from /usr/lib/libhdf5.so.5
#6 0x00002b5ef7a12148 in H5D_istore_flush () from /usr/lib/libhdf5.so.5
#7 0x00002b5ef79fe3c5 in H5D_flush_real () from /usr/lib/libhdf5.so.5
#8 0x00002b5ef79fe5d4 in H5D_flush_cb () from /usr/lib/libhdf5.so.5
#9 0x00002b5ef7a912cc in H5I_search () from /usr/lib/libhdf5.so.5
#10 0x00002b5ef79fe076 in H5D_flush () from /usr/lib/libhdf5.so.5
#11 0x00002b5ef7a253cd in H5F_flush () from /usr/lib/libhdf5.so.5
#12 0x00002b5ef7a2922c in H5Fflush () from /usr/lib/libhdf5.so.5
#13 0x00002b5efe7d31e0 in __pyx_f_13hdf5Extension_4Leaf__g_flush (
__pyx_v_self=0x1e295f0, __pyx_args=<value optimized out>,
__pyx_kwds=<value optimized out>) at src/hdf5Extension.c:2598
#14 0x0000000000488de0 in PyEval_EvalFrameEx (f=0x74ac4f0,
throwflag=<value optimized out>) at Python/ceval.c:3573
#15 0x00000000004887c0 in PyEval_EvalFrameEx (f=0x1f8dc60,
throwflag=<value optimized out>) at Python/ceval.c:3659
#16 0x00000000004887c0 in PyEval_EvalFrameEx (f=0x1f8da80,
throwflag=<value optimized out>) at Python/ceval.c:3659
#17 0x0000000000489f9f in PyEval_EvalCodeEx (co=0xeaacd8,
globals=<value optimized out>, locals=<value optimized out>, args=0x2,
argcount=1, kws=0x2eb1f80, kwcount=0, defs=0xf297e8, defcount=1,
closure=0x0) at Python/ceval.c:2836
#18 0x0000000000488366 in PyEval_EvalFrameEx (f=0x2eb1dd0,
throwflag=<value optimized out>) at Python/ceval.c:3669
#19 0x0000000000489f9f in PyEval_EvalCodeEx (co=0xf63eb8,
globals=<value optimized out>, locals=<value optimized out>,
args=0x2eb1cf0, argcount=3, kws=0x2eb1d08, kwcount=0, defs=0xf38368,
defcount=1, closure=0x0) at Python/ceval.c:2836
#20 0x0000000000488366 in PyEval_EvalFrameEx (f=0x2eb1b30,
throwflag=<value optimized out>) at Python/ceval.c:3669
#21 0x0000000000489f9f in PyEval_EvalCodeEx (co=0xf68198,
globals=<value optimized out>, locals=<value optimized out>, args=0x0,
argcount=1, kws=0x2eb1ae8, kwcount=0, defs=0x0, defcount=0, closure=0x0)
at Python/ceval.c:2836
#22 0x0000000000488366 in PyEval_EvalFrameEx (f=0x2eb1960,
throwflag=<value optimized out>) at Python/ceval.c:3669
#23 0x00000000004887c0 in PyEval_EvalFrameEx (f=0x2eb17a0,
throwflag=<value optimized out>) at Python/ceval.c:3659
#24 0x00000000004887c0 in PyEval_EvalFrameEx (f=0x77bd10,
throwflag=<value optimized out>) at Python/ceval.c:3659
#25 0x0000000000489f9f in PyEval_EvalCodeEx (co=0x2b5ef69c9288,
globals=<value optimized out>, locals=<value optimized out>, args=0x0,
argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0)
at Python/ceval.c:2836
#26 0x0000000000489fe2 in PyEval_EvalCode (co=0x3130, globals=0x3130,
locals=0x6) at Python/ceval.c:494
#27 0x00000000004ab4ce in PyRun_FileExFlags (fp=0x759010,
filename=0x7fffb4178897 "testh5.py", start=<value optimized out>,
globals=0x77c640, locals=0x77c640, closeit=1, flags=0x7fffb4176650)
at Python/pythonrun.c:1273
#28 0x00000000004ab760 in PyRun_SimpleFileExFlags (fp=0x759010,
filename=0x7fffb4178897 "testh5.py", closeit=1, flags=0x7fffb4176650)
at Python/pythonrun.c:879
#29 0x0000000000414645 in Py_Main (argc=<value optimized out>,
argv=0x7fffb4176778) at Modules/main.c:523
#30 0x00002b5ef740fb44 in __libc_start_main () from /lib/libc.so.6
#31 0x0000000000413b89 in _start ()
Thanks for any help you can offer,
Glenn
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Pytables-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/pytables-users
