-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 05/25/2013 11:49 AM, Tres Seaver wrote: > On 05/25/2013 02:50 AM, Marius Gedminas wrote: >> On Fri, May 24, 2013 at 09:31:55PM -0400, Tres Seaver wrote: >>> On 05/24/2013 01:28 PM, Marius Gedminas wrote: >>>> I was trying to debug a failing ZEO test case >>>> (testConcurrentUpdates), and found out it had nothing to do with >>>> ZEO. There's a bug either in BTrees or in ZODB itself that >>>> causes data corruption under Python 3.x. >>>> >>>> Here's a test case: >>>> https://gist.github.com/mgedmin/5644876#file-zodbfail_simple-py >>> >>> Thanks for the detective work! I can confirm the bug, and that >>> it is in the BTrees C extensions (the tests pass after 'rm >>> .tox/py32/lib/python3.2/site-packages/BTrees/*.so'). > >> It's the COMPARE macro in _compat.h. This patch fixes the data >> corruption: > >> diff --git a/BTrees/_compat.h b/BTrees/_compat.h index >> e004d54..4765df1 100644 --- a/BTrees/_compat.h +++ >> b/BTrees/_compat.h @@ -28,8 +28,8 @@ #define TEXT_FORMAT >> PyUnicode_Format > >> #define COMPARE(lhs, rhs) \ - PyObject_RichCompareBool((lhs), >> (rhs), Py_LT) > 0 ? -1 : \ - (PyObject_RichCompareBool((lhs), >> (rhs), Py_EQ) > 0 ? 0 : 1) + (PyObject_RichCompareBool((lhs), >> (rhs), Py_LT) > 0 ? -1 : \ + (PyObject_RichCompareBool((lhs), >> (rhs), Py_EQ) > 0 ? 0 : 1)) > > >> #else > > >> I'm now trying to write a failing unit test. It's a bit difficult >> to get access to C macros from Python tests ;-) > >> COMPARE is used in two places: > >> BTrees/objectkeymacros.h-#define TEST_KEY_SET_OR(V, KEY, TARGET) \ >> BTrees/objectkeymacros.h:if ( ( (V) = COMPARE((KEY),(TARGET)) ), >> PyErr_Occurred() ) > >> and > >> BTrees/objectvaluemacros.h:#define TEST_VALUE(VALUE, TARGET) >> COMPARE((VALUE),(TARGET)) > >> We can ignore TEST_KEY_SET_OR, as it works fine there. > >> TEST_VALUE is used to implement BTree.byValue and bucket merging. >> The bug I was chasing down was caused by bucket mismerges, I think. > >> It should be easier to write a test for byValue. In fact there is >> one, in test__base.py, but it's not run against real C-implemented >> BTrees. I copied it over into test_OOBTree.py and discovered that >> it fails unexpectedly:
After adding your new test, and wrapping the 'COMPARE()' macro inside extra parens within 'TEST_VALUE' (in 'objectvaluemacros.h'), the new 'byValue' test passes on Py3k. I am trying to reproduce the ZEO test failure now, and then against the BTrees trunk, before making a BTrees release. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlGg9rIACgkQ+gerLs4ltQ6M5ACeIyIgCC9/jWwew1qSFyJNg+eb EDIAnRORMJ2XN7xm8kjd+lZciEaJKkA0 =iXx3 -----END PGP SIGNATURE----- _______________________________________________ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
