On 5/24/2010 12:31 AM, Francesc Alted wrote:
> 2010/5/24 Christoph Gohlke <[email protected] <mailto:[email protected]>>
>
> 2.- In order to load DLLs, PyTables uses a the `LoadLibrary`
> function to
> check whether the DLL is available or not (see `getLibrary`
> function in
> src/utils.c for details). Perhaps `LoadLibrary` Win call does
> not work
> the same in Python 64/Win 64?
>
>
>
>
> Thanks for the hint. Turns out that the 64 bit DLL name is "libbz2"
> on my system, while utilsExtension.pyx hard-codes the name "bzip2".
> Also, while "import tables._comp_lzo" succeeds, LoadLibrary("lzo2")
> does not because the Python\Lib\site-packages\tables directory,
> where I place the DLLs, is not in the system path. Easy to fix...
>
>
> Great. But I don't know why LZO DLL is not searched by default in
> Python\Lib\site-packages\tables as this path should be searched by
> default by the Python interpreter. Or perhaps this is valid only for
> extensions (.pyd) and not DLLs (.dll)?
That is a common issue on Python for Windows: sys.path !=
os.environ['PATH']. No directory in the Python tree is in
os.environ['PATH'] by default. As a workaround to LoadLibrary and
PyImport_ImportModule problems I sometimes add the following code to a
package __init__.py file.
import sys, os
if os.name == 'nt':
module_path = os.path.abspath(os.path.dirname(__file__))
os.environ['PATH'] = ';'.join((os.environ['PATH'], module_path))
sys.path.append(module_path)
>
> I ran examples/table1.py with no compression, zlib, lzo, blosc, and
> bzip2 just fine.
>
>
> This means that zlib (or any other compression library) is not the problem.
>
> I have recompiled everything using the following version, but the
> tests still fail as before. Zlib shows as version 1.2.3 (that's what
> Python is using internally) event though I am now linking against 1.2.5.
>
>
> Do not pay too much attention to the Zlib version stated by
> tables.print_versions(). As HDF5 does not allow to query the version of
> their compression libraries, print_versions() only prints the zlib
> version available via the Python interpreter. Mmh, this is misleading,
> and perhaps it would be better if I remove this info (I've added a
> ticket about this: http://www.pytables.org/trac/ticket/277).
>
>
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
> PyTables version: 2.2rc1
> HDF5 version: 1.8.5-pre1
> NumPy version: 2.0.0.dev8426
>
> Numexpr version: 1.3.1 (not using Intel's VML/MKL)
> Zlib version: 1.2.3
> LZO version: 2.03 (Apr 30 2008)
> BZIP2 version: 1.0.5 (10-Dec-2007)
>
> Blosc version: 0.9.0 (2010-05-04)
> Python version: 2.7b2 (r27b2:81019, May 9 2010, 10:33:25) [MSC
> v.1500 64 bit (AMD64)]
> Byte-ordering: little
>
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
>
> The following examples fail/crash:
>
> array1.py
>
>
> Hey, array1.py does not use compression at all, so compression is
> definitely not the issue. Moreover, it is small enough to allow a guess
> on what is failing. With the next stripped out version or array1.py:
>
> """
> from numpy import *
> from tables import *
>
> fileh = openFile("array1.h5", mode = "w")
> # Get the root group
> root = fileh.root
>
> # Create an Array
> a = array([-1, 2, 4], int16)
> # Save it on the HDF5 file
> hdfarray = fileh.createArray(root, 'array_1', a, "Signed short array")
>
> # Create a scalar Array
> a = array(4, int16)
> # Save it on the HDF5 file
> hdfarray = fileh.createArray(root, 'array_s', a, "Scalar signed short
> array")
>
> # Create a 3-d array of floats
> a = arange(120, dtype=float64).reshape(20,3,2)
> # Save it on the HDF5 file
> hdfarray = fileh.createArray(root, 'array_f', a, "3-D float array")
>
> # Close the file
> fileh.close()
> """
>
> could you try to comment out ``createArray()`` lines until it works
> well? Knowing the which line triggers the error would put us into the
> good track (I suspect of internal handling of ints, but let's see).
>
OK. The result depends on whether I run your stripped down code or the
original array1.py file.
Original array1.py:
1, s, f: crash, no traceback or console output
1, s: crash with traceback #1
1, f: crash with traceback #1
s, f: crash with traceback #2
1: OK
s: OK
f: crash with traceback #3
Stripped down version of array1.py:
1, s, f: crash with traceback #1 <- different
1, s: crash with traceback #1
1, f: crash with traceback #1
s, f: crash with traceback #2
1: crash with traceback #1 <- different
s: OK
f: crash with traceback #2 <- different
1 -> hdfarray = createArray(root, 'array_1' ...
s -> hdfarray = createArray(root, 'array_s' ...
f -> hdfarray = createArray(root, 'array_f' ...
Thanks for your help!
--
Christoph
Traceback #1
HDF5-DIAG: Error detected in HDF5 (1.8.5-pre1) thread 0:
#000: ..\..\hdf5-1.8.5-svn\src\H5Dio.c line 266 in H5Dwrite(): can't
write data
major: Dataset
minor: Write failed
#001: ..\..\hdf5-1.8.5-svn\src\H5Dio.c line 578 in H5D_write(): can't
write data
major: Dataset
minor: Write failed
#002: ..\..\hdf5-1.8.5-svn\src\H5Dcontig.c line 557 in
H5D_contig_write(): contiguous write failed
major: Dataset
minor: Write failed
#003: ..\..\hdf5-1.8.5-svn\src\H5Dselect.c line 306 in
H5D_select_write(): write error
major: Dataspace
minor: Write failed
#004: ..\..\hdf5-1.8.5-svn\src\H5Dselect.c line 217 in
H5D_select_io(): write error
major: Dataspace
minor: Write failed
#005: ..\..\hdf5-1.8.5-svn\src\H5Dcontig.c line 949 in
H5D_contig_writevv(): block write failed
major: Low-level I/O
minor: Write failed
#006: ..\..\hdf5-1.8.5-svn\src\H5Fio.c line 167 in H5F_block_write():
file write failed
major: Low-level I/O
minor: Write failed
#007: ..\..\hdf5-1.8.5-svn\src\H5FDint.c line 185 in H5FD_write():
driver write request failed
major: Virtual File Layer
minor: Write failed
#008: ..\..\hdf5-1.8.5-svn\src\H5FDwindows.c line 944 in
H5FD_windows_write(): file write failed,
errno = 22, error message = 'Invalid argument'
major: Low-level I/O
minor: Write failed
Traceback (most recent call last):
File "array1.py", line 13, in <module>
hdfarray = fileh.createArray(root, 'array_1', a, "Signed short array")
File "X:\Python27-x64\lib\site-packages\tables\file.py", line 785, in
createArray
object=object, title=title, byteorder=byteorder)
File "X:\Python27-x64\lib\site-packages\tables\array.py", line 227,
in __init__
byteorder, _log)
File "X:\Python27-x64\lib\site-packages\tables\leaf.py", line 291, in
__init__
super(Leaf, self).__init__(parentNode, name, _log)
File "X:\Python27-x64\lib\site-packages\tables\node.py", line 296, in
__init__
self._v_objectID = self._g_create()
File "X:\Python27-x64\lib\site-packages\tables\array.py", line 260,
in _g_create
nparr, self._v_new_title, self.atom)
File "hdf5Extension.pyx", line 851, in
tables.hdf5Extension.Array._createArray (tables\hdf5Extensi
on.c:7551)
tables.exceptions.HDF5ExtError: Problems creating the Array.
Closing remaining open files: array1.h5...HDF5-DIAG: Error detected in
HDF5 (1.8.5-pre1) thread 0:
#000: ..\..\hdf5-1.8.5-svn\src\H5F.c line 1957 in H5Fclose():
decrementing file ID failed
major: Object atom
minor: Unable to close file
#001: ..\..\hdf5-1.8.5-svn\src\H5F.c line 1748 in H5F_close(): can't
close file
major: File accessability
minor: Unable to close file
#002: ..\..\hdf5-1.8.5-svn\src\H5F.c line 1896 in H5F_try_close():
problems closing file
major: File accessability
minor: Unable to close file
#003: ..\..\hdf5-1.8.5-svn\src\H5F.c line 1069 in H5F_dest(): low
level truncate failed
major: File accessability
minor: Write failed
#004: ..\..\hdf5-1.8.5-svn\src\H5FD.c line 1965 in H5FD_truncate():
driver truncate request failed
major: Virtual File Layer
minor: Can't update object
#005: ..\..\hdf5-1.8.5-svn\src\H5FDwindows.c line 1058 in
H5FD_windows_truncate(): unable to exten
d file properly
major: Low-level I/O
minor: Seek failed
Traceback # 2
HDF5-DIAG: Error detected in HDF5 (1.8.5-pre1) thread 0:
#000: ..\..\hdf5-1.8.5-svn\src\H5Dio.c line 266 in H5Dwrite(): can't
write data
major: Dataset
minor: Write failed
#001: ..\..\hdf5-1.8.5-svn\src\H5Dio.c line 578 in H5D_write(): can't
write data
major: Dataset
minor: Write failed
#002: ..\..\hdf5-1.8.5-svn\src\H5Dcontig.c line 557 in
H5D_contig_write(): contiguous write failed
major: Dataset
minor: Write failed
#003: ..\..\hdf5-1.8.5-svn\src\H5Dselect.c line 306 in
H5D_select_write(): write error
major: Dataspace
minor: Write failed
#004: ..\..\hdf5-1.8.5-svn\src\H5Dselect.c line 217 in
H5D_select_io(): write error
major: Dataspace
minor: Write failed
#005: ..\..\hdf5-1.8.5-svn\src\H5Dcontig.c line 949 in
H5D_contig_writevv(): block write failed
major: Low-level I/O
minor: Write failed
#006: ..\..\hdf5-1.8.5-svn\src\H5Fio.c line 167 in H5F_block_write():
file write failed
major: Low-level I/O
minor: Write failed
#007: ..\..\hdf5-1.8.5-svn\src\H5FDint.c line 185 in H5FD_write():
driver write request failed
major: Virtual File Layer
minor: Write failed
#008: ..\..\hdf5-1.8.5-svn\src\H5FDwindows.c line 944 in
H5FD_windows_write(): file write failed,
errno = 22, error message = 'Invalid argument'
major: Low-level I/O
minor: Write failed
Traceback (most recent call last):
File "array1.py", line 23, in <module>
hdfarray = fileh.createArray(root, 'array_f', a, "3-D float array")
File "X:\Python27-x64\lib\site-packages\tables\file.py", line 785, in
createArray
object=object, title=title, byteorder=byteorder)
File "X:\Python27-x64\lib\site-packages\tables\array.py", line 227,
in __init__
byteorder, _log)
File "X:\Python27-x64\lib\site-packages\tables\leaf.py", line 291, in
__init__
super(Leaf, self).__init__(parentNode, name, _log)
File "X:\Python27-x64\lib\site-packages\tables\node.py", line 296, in
__init__
self._v_objectID = self._g_create()
File "X:\Python27-x64\lib\site-packages\tables\array.py", line 260,
in _g_create
nparr, self._v_new_title, self.atom)
File "hdf5Extension.pyx", line 851, in
tables.hdf5Extension.Array._createArray (tables\hdf5Extensi
on.c:7551)
tables.exceptions.HDF5ExtError: Problems creating the Array.
Closing remaining open files: array1.h5...HDF5-DIAG: Error detected in
HDF5 (1.8.5-pre1) thread 0:
#000: ..\..\hdf5-1.8.5-svn\src\H5F.c line 1957 in H5Fclose():
decrementing file ID failed
major: Object atom
minor: Unable to close file
#001: ..\..\hdf5-1.8.5-svn\src\H5F.c line 1748 in H5F_close(): can't
close file
major: File accessability
minor: Unable to close file
#002: ..\..\hdf5-1.8.5-svn\src\H5F.c line 1896 in H5F_try_close():
problems closing file
major: File accessability
minor: Unable to close file
#003: ..\..\hdf5-1.8.5-svn\src\H5F.c line 1069 in H5F_dest(): low
level truncate failed
major: File accessability
minor: Write failed
#004: ..\..\hdf5-1.8.5-svn\src\H5FD.c line 1965 in H5FD_truncate():
driver truncate request failed
major: Virtual File Layer
minor: Can't update object
#005: ..\..\hdf5-1.8.5-svn\src\H5FDwindows.c line 1058 in
H5FD_windows_truncate(): unable to exten
d file properly
major: Low-level I/O
minor: Seek failed
done
Traceback # 3
HDF5-DIAG: Error detected in HDF5 (1.8.5-pre1) thread 0:
#000: ..\..\hdf5-1.8.5-svn\src\H5Ddeprec.c line 169 in H5Dcreate1():
unable to create dataset
major: Dataset
minor: Unable to initialize object
#001: ..\..\hdf5-1.8.5-svn\src\H5Dint.c line 431 in
H5D_create_named(): unable to create and link
to dataset
major: Dataset
minor: Unable to initialize object
#002: ..\..\hdf5-1.8.5-svn\src\H5L.c line 1640 in H5L_link_object():
unable to create new link to
object
major: Links
minor: Unable to initialize object
#003: ..\..\hdf5-1.8.5-svn\src\H5L.c line 1863 in H5L_create_real():
can't insert link
major: Symbol table
minor: Unable to insert object
#004: ..\..\hdf5-1.8.5-svn\src\H5Gtraverse.c line 952 in
H5G_traverse(): internal path traversal f
ailed
major: Symbol table
minor: Object not found
#005: ..\..\hdf5-1.8.5-svn\src\H5Gtraverse.c line 759 in
H5G_traverse_real(): traversal operator f
ailed
major: Symbol table
minor: Callback failed
#006: ..\..\hdf5-1.8.5-svn\src\H5L.c line 1686 in H5L_link_cb():
unable to create object
major: Object header
minor: Unable to initialize object
#007: ..\..\hdf5-1.8.5-svn\src\H5O.c line 2976 in H5O_obj_create():
unable to open object
major: Object header
minor: Can't open object
#008: ..\..\hdf5-1.8.5-svn\src\H5Doh.c line 295 in H5O_dset_create():
unable to create dataset
major: Dataset
minor: Unable to initialize object
#009: ..\..\hdf5-1.8.5-svn\src\H5Dint.c line 1035 in H5D_create():
unable to construct layout info
rmation
major: Dataset
minor: Unable to initialize object
#010: ..\..\hdf5-1.8.5-svn\src\H5Dcontig.c line 410 in
H5D_contig_construct(): unable to retrieve
number of elements in dataspace
major: Dataset
minor: Can't get value
HDF5-DIAG: Error detected in HDF5 (1.8.5-pre1) thread 0:
#000: ..\..\hdf5-1.8.5-svn\src\H5D.c line 377 in H5Dclose(): not a
dataset
major: Invalid arguments to routine
minor: Inappropriate type
Traceback (most recent call last):
File "array1.py", line 23, in <module>
hdfarray = fileh.createArray(root, 'array_f', a, "3-D float array")
File "X:\Python27-x64\lib\site-packages\tables\file.py", line 785, in
createArray
object=object, title=title, byteorder=byteorder)
File "X:\Python27-x64\lib\site-packages\tables\array.py", line 227,
in __init__
byteorder, _log)
File "X:\Python27-x64\lib\site-packages\tables\leaf.py", line 291, in
__init__
super(Leaf, self).__init__(parentNode, name, _log)
File "X:\Python27-x64\lib\site-packages\tables\node.py", line 296, in
__init__
self._v_objectID = self._g_create()
File "X:\Python27-x64\lib\site-packages\tables\array.py", line 260,
in _g_create
nparr, self._v_new_title, self.atom)
File "hdf5Extension.pyx", line 851, in
tables.hdf5Extension.Array._createArray (tables\hdf5Extensi
on.c:7551)
tables.exceptions.HDF5ExtError: Problems creating the Array.
Closing remaining open files: array1.h5... done
------------------------------------------------------------------------------
_______________________________________________
Pytables-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/pytables-users
