Re: [Pytables-users] where() with start/stop args returning incorrect result set
Hi Anthony,
It doesn't happen if I set start=0 or seemingly any number below 3257
(though I didn't try them *all*). I am new to PyTables and hdf5, so
I'm not sure about the chunksize or if I'm at a boundary. I did
however notice that the table's chunkshape is 203, and this happens
for exactly 203 sequential records, so I doubt that's a coincidence.
The table description is below.
Thanks,
Derek
/events (Table(5988,)) ''
description := {
client_id: StringCol(itemsize=24, shape=(), dflt='', pos=0),
data_01: StringCol(itemsize=36, shape=(), dflt='', pos=1),
data_02: StringCol(itemsize=36, shape=(), dflt='', pos=2),
data_03: StringCol(itemsize=36, shape=(), dflt='', pos=3),
data_04: StringCol(itemsize=36, shape=(), dflt='', pos=4),
data_05: StringCol(itemsize=36, shape=(), dflt='', pos=5),
device_id: StringCol(itemsize=36, shape=(), dflt='', pos=6),
id: StringCol(itemsize=36, shape=(), dflt='', pos=7),
timestamp: Time64Col(shape=(), dflt=0.0, pos=8),
type: UInt16Col(shape=(), dflt=0, pos=9),
user_id: StringCol(itemsize=36, shape=(), dflt='', pos=10)}
byteorder := 'little'
chunkshape := (203,)
autoIndex := True
colindexes := {
timestamp: Index(9, full, shuffle, zlib(1)).is_CSI=True,
type: Index(9, full, shuffle, zlib(1)).is_CSI=True,
id: Index(9, full, shuffle, zlib(1)).is_CSI=True,
user_id: Index(9, full, shuffle, zlib(1)).is_CSI=True}
On Tue, Sep 25, 2012 at 9:32 AM, Anthony Scopatz scop...@gmail.com wrote:
Hi Derek,
Ok That is very strange. I cannot reproduce this on any of my data. A
quick couple of extra questions:
1) Does this still happen when you set start=0?
2) What is the chunksize of this data set (are you at a boundary)?
3) Could you send us the full table information, ie repr(table).
Be Well
Anthony
On Tue, Sep 25, 2012 at 12:42 AM, Derek Shockey derek.shoc...@gmail.com
wrote:
I ran the tests. All 4988 passed. The information it output is:
PyTables version: 2.4.0
HDF5 version: 1.8.9
NumPy version: 1.6.2
Numexpr version: 2.0.1 (not using Intel's VML/MKL)
Zlib version: 1.2.5 (in Python interpreter)
LZO version: 2.06 (Aug 12 2011)
BZIP2 version: 1.0.6 (6-Sept-2010)
Blosc version: 1.1.3 (2010-11-16)
Cython version:0.16
Python version:2.7.3 (default, Jul 6 2012, 00:17:51)
[GCC 4.2.1 Compatible Apple Clang 3.1 (tags/Apple/clang-318.0.58)]
Platform: darwin-x86_64
Byte-ordering: little
Detected cores:4
-Derek
On Mon, Sep 24, 2012 at 9:09 PM, Anthony Scopatz scop...@gmail.com
wrote:
Hi Derek,
Can you please run the following command and report back what you see?
python -c import tables; tables.test()
Be Well
Anthony
On Mon, Sep 24, 2012 at 10:56 PM, Derek Shockey
derek.shoc...@gmail.com
wrote:
Hello,
I'm hoping someone can help me. When I specify start and stop values
for calls to where() and readWhere(), it is returning blatantly
incorrect results:
table.readWhere(id == 'ceec536a-394e-4dd7-a182-eea557f3bb93',
start=3257, stop=table.nrows)[0]['id']
'7f589d3e-a0e1-4882-b69b-0223a7de3801'
table.where(id == 'ceec536a-394e-4dd7-a182-eea557f3bb93',
start=3257, stop=table.nrows).next()['id']
'7f589d3e-a0e1-4882-b69b-0223a7de3801'
This happens with a sequential block of about 150 rows of data, and
each time it seems to be 8 rows off (i.e. the row it returns is 8 rows
ahead of the row it should be returning). If I remove the start and
stop args, it behaves correctly. This seems to be a bug, unless I am
misunderstanding something. I'm using Python 2.7.3, PyTables 2.4.0,
and hdf5 1.8.9 on OS X 10.8.2.
Any ideas?
Thanks,
Derek
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond.
Discussions
will include endpoint security, mobile security and the latest in
malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond.
Discussions
will include endpoint security, mobile security and the latest in
malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
--
Live Security
[Pytables-users] where() with start/stop args returning incorrect result set
Hello, I'm hoping someone can help me. When I specify start and stop values for calls to where() and readWhere(), it is returning blatantly incorrect results: table.readWhere(id == 'ceec536a-394e-4dd7-a182-eea557f3bb93', start=3257, stop=table.nrows)[0]['id'] '7f589d3e-a0e1-4882-b69b-0223a7de3801' table.where(id == 'ceec536a-394e-4dd7-a182-eea557f3bb93', start=3257, stop=table.nrows).next()['id'] '7f589d3e-a0e1-4882-b69b-0223a7de3801' This happens with a sequential block of about 150 rows of data, and each time it seems to be 8 rows off (i.e. the row it returns is 8 rows ahead of the row it should be returning). If I remove the start and stop args, it behaves correctly. This seems to be a bug, unless I am misunderstanding something. I'm using Python 2.7.3, PyTables 2.4.0, and hdf5 1.8.9 on OS X 10.8.2. Any ideas? Thanks, Derek -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
Re: [Pytables-users] where() with start/stop args returning incorrect result set
I ran the tests. All 4988 passed. The information it output is: PyTables version: 2.4.0 HDF5 version: 1.8.9 NumPy version: 1.6.2 Numexpr version: 2.0.1 (not using Intel's VML/MKL) Zlib version: 1.2.5 (in Python interpreter) LZO version: 2.06 (Aug 12 2011) BZIP2 version: 1.0.6 (6-Sept-2010) Blosc version: 1.1.3 (2010-11-16) Cython version:0.16 Python version:2.7.3 (default, Jul 6 2012, 00:17:51) [GCC 4.2.1 Compatible Apple Clang 3.1 (tags/Apple/clang-318.0.58)] Platform: darwin-x86_64 Byte-ordering: little Detected cores:4 -Derek On Mon, Sep 24, 2012 at 9:09 PM, Anthony Scopatz scop...@gmail.com wrote: Hi Derek, Can you please run the following command and report back what you see? python -c import tables; tables.test() Be Well Anthony On Mon, Sep 24, 2012 at 10:56 PM, Derek Shockey derek.shoc...@gmail.com wrote: Hello, I'm hoping someone can help me. When I specify start and stop values for calls to where() and readWhere(), it is returning blatantly incorrect results: table.readWhere(id == 'ceec536a-394e-4dd7-a182-eea557f3bb93', start=3257, stop=table.nrows)[0]['id'] '7f589d3e-a0e1-4882-b69b-0223a7de3801' table.where(id == 'ceec536a-394e-4dd7-a182-eea557f3bb93', start=3257, stop=table.nrows).next()['id'] '7f589d3e-a0e1-4882-b69b-0223a7de3801' This happens with a sequential block of about 150 rows of data, and each time it seems to be 8 rows off (i.e. the row it returns is 8 rows ahead of the row it should be returning). If I remove the start and stop args, it behaves correctly. This seems to be a bug, unless I am misunderstanding something. I'm using Python 2.7.3, PyTables 2.4.0, and hdf5 1.8.9 on OS X 10.8.2. Any ideas? Thanks, Derek -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
