[issue11277] test_zlib crashes under Snow Leopard buildbot
STINNER Victor victor.stin...@haypocalc.com added the comment: This issue is not dead: test_zlib failed twice on AMD64 Snow Leopard 3.x buildbot: build 30 (024967cdc2f0e850f0b338e7593a12d965017a6a, Mar 31 01:40:00 2011) and 44 (ebc03d7e711052c0b196aacdbec6778c0a6d5c0c, Apr 4 10:11:20 2011). Build 44 has a traceback thanks to faulthandler: ... [ 79/354] test_time [ 80/354] test_zlib Fatal Python error: Bus error Traceback (most recent call first): File /Users/pythonbuildbot/buildarea/3.x.hansen-osx-x86/build/Lib/test/test_zlib.py, line 85 in test_big_buffer File /Users/pythonbuildbot/buildarea/3.x.hansen-osx-x86/build/Lib/unittest/case.py, line 387 in _executeTestPart File /Users/pythonbuildbot/buildarea/3.x.hansen-osx-x86/build/Lib/unittest/case.py, line 442 in run File /Users/pythonbuildbot/buildarea/3.x.hansen-osx-x86/build/Lib/unittest/case.py, line 494 in __call__ File /Users/pythonbuildbot/buildarea/3.x.hansen-osx-x86/build/Lib/unittest/suite.py, line 105 in run File /Users/pythonbuildbot/buildarea/3.x.hansen-osx-x86/build/Lib/unittest/suite.py, line 67 in __call__ File /Users/pythonbuildbot/buildarea/3.x.hansen-osx-x86/build/Lib/unittest/suite.py, line 105 in run File /Users/pythonbuildbot/buildarea/3.x.hansen-osx-x86/build/Lib/unittest/suite.py, line 67 in __call__ File /Users/pythonbuildbot/buildarea/3.x.hansen-osx-x86/build/Lib/test/support.py, line 1078 in run File /Users/pythonbuildbot/buildarea/3.x.hansen-osx-x86/build/Lib/test/support.py, line 1166 in _run_suite File /Users/pythonbuildbot/buildarea/3.x.hansen-osx-x86/build/Lib/test/support.py, line 1192 in run_unittest File /Users/pythonbuildbot/buildarea/3.x.hansen-osx-x86/build/Lib/test/test_zlib.py, line 611 in test_main File ./Lib/test/regrtest.py, line 1032 in runtest_inner File ./Lib/test/regrtest.py, line 826 in runtest File ./Lib/test/regrtest.py, line 650 in main File ./Lib/test/regrtest.py, line 1607 in module make: *** [buildbottest] Bus error program finished with exit code 2 elapsedTime=1400.363321 http://www.python.org/dev/buildbot/all/builders/AMD64%20Snow%20Leopard%203.x/builds/44/steps/test/logs/stdio test_zlib.py:85 is the crc32(+4 GB) test: -- # Issue #10276 - check that inputs =4GB are handled correctly. class ChecksumBigBufferTestCase(unittest.TestCase): def setUp(self): with open(support.TESTFN, wb+) as f: f.seek(_4G) f.write(basdf) with open(support.TESTFN, rb) as f: self.mapping = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) def tearDown(self): self.mapping.close() support.unlink(support.TESTFN) @unittest.skipUnless(mmap, mmap() is not available.) @unittest.skipUnless(sys.maxsize _4G, Can't run on a 32-bit system.) @unittest.skipUnless(support.is_resource_enabled(largefile), May use lots of disk space.) def test_big_buffer(self): self.assertEqual(zlib.crc32(self.mapping), 3058686908) ~~~ HERE self.assertEqual(zlib.adler32(self.mapping), 82837919) -- -- resolution: fixed - status: closed - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
STINNER Victor victor.stin...@haypocalc.com added the comment: Issue #11760 has been marked as a duplicate of this issue. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Charles-Francois Natali neolo...@free.fr added the comment: Is the SIGBUS generated on the first page access ? How much memory does this buildbot have ? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Antoine Pitrou pit...@free.fr added the comment: The new FreeBSD buildbot had a sporadic SIGKILL in http://www.python.org/dev/buildbot/all/builders/AMD64%20FreeBSD%208.2%203.x/builds/1/steps/test/logs/stdio (apparently, faulthandler didn't dump a traceback) By the way, we can be fairly certain now that the problem is on the OS side rather than on our (Python) side, so I'm lowering the priority. -- nosy: +skrah priority: critical - high ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Antoine Pitrou pit...@free.fr added the comment: By the way, at this point I think we could simply skip the test on BSDs and OS X. The tested functionality is cross-platform, so testing under a limited set of systems should be ok. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Stefan Krah stefan-use...@bytereef.org added the comment: For the new FreeBSD bot, the issue was simply insufficient swap space. With 1GB of memory and 2GB of swap test_zlib runs fine. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment: 11:12 ~/tmp $ python3 ~/usr/opt/py3k/lib/python3.3/test_zlib.py Bus error Your code snippet: 11:21 ~/tmp $ /usr/bin/time -lp python3 test.py posix.stat_result(st_mode=33184, st_ino=10066605, st_dev=234881025, st_nlink=1, st_uid=502, st_gid=20, st_size=4294967300, st_atime=1298715813, st_mtime=1298715813, st_ctime=1298715813) posix.stat_result(st_mode=33184, st_ino=10066605, st_dev=234881025, st_nlink=1, st_uid=502, st_gid=20, st_size=4294967300, st_atime=1298715813, st_mtime=1298715813, st_ctime=1298715813) real71.66 user 0.06 sys 3.71 0 maximum resident set size 0 average shared memory size 0 average unshared data size 0 average unshared stack size 0 page reclaims 0 page faults 0 swaps 0 block input operations 57 block output operations 0 messages sent 0 messages received 0 signals received 2112 voluntary context switches 0 involuntary context switches On Fri, Feb 25, 2011 at 05:05:19PM +, Charles-Francois Natali wrote: Charles-Francois Natali neolo...@free.fr added the comment: Could you try with this: def setUp(self): with open(support.TESTFN, wb+) as f: f.seek(_4G) f.write(basdf) f.flush() + os.fsync(f.fileno()) self.mapping = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) HFS+ doesn't seem to support sparse files, so the file is actually zero-filled asynchronously. Maybe the mapping gets done before the blocks have been allocated, which triggers a segfault when the first page is accessed. I'm not sure it'll make any difference, but I'm curious... Also, I'd be curious to see the result of import os name = '/tmp/foo' f = open(name, 'wb') f.seek(1 32) f.write(b'asdf') f.flush() print(os.fstat(f.fileno())) f.close() print(os.stat(name)) Thanks ! -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment:
I'll give you the same result again but with additional clock(),
just for a heart's pleasure:
clock(): 0.100958 , fstat(): posix.stat_result(st_mode=33184, st_ino=10075508,
st_dev=234881025, st_nlink=1, st_uid=502, st_gid=20, st_size=4294967300,
st_atime=1298719201, st_mtime=1298719305, st_ctime=1298719305)
f.close()
print('clock():', time.clock(), ', stat():', os.stat(name))
clock(): 3.75792 , stat(): posix.stat_result(st_mode=33184, st_ino=10075508,
st_dev=234881025, st_nlink=1, st_uid=502, st_gid=20, st_size=4294967300,
st_atime=1298719201, st_mtime=1298719305, st_ctime=1298719305)
Please don't assume i go for Mac OS X ...
In the end you *always* need to implement an expensive state
machine to get around long-known bugs, mis-implementations or
other poops there.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11277
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Changes by Brett Cannon br...@python.org: -- nosy: -brett.cannon ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Charles-Francois Natali neolo...@free.fr added the comment: Could you try with this: def setUp(self): with open(support.TESTFN, wb+) as f: f.seek(_4G) f.write(basdf) f.flush() + os.fsync(f.fileno()) self.mapping = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) HFS+ doesn't seem to support sparse files, so the file is actually zero-filled asynchronously. Maybe the mapping gets done before the blocks have been allocated, which triggers a segfault when the first page is accessed. I'm not sure it'll make any difference, but I'm curious... Also, I'd be curious to see the result of import os name = '/tmp/foo' f = open(name, 'wb') f.seek(1 32) f.write(b'asdf') f.flush() print(os.fstat(f.fileno())) f.close() print(os.stat(name)) Thanks ! -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment: I append a doc_lib_mmap.patch which may be helpful for those poor creatures who plan to write Python scripts for Mac OS X. (It may be a useful add-on anyway.) -- Added file: http://bugs.python.org/file20859/doc_lib_mmap.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Changes by Steffen Daode Nurpmeso sdao...@googlemail.com: Removed file: http://bugs.python.org/file20859/doc_lib_mmap.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment: Sorry, i've got that kid running around which sometimes doesn't know what it is doing. But this documentation patch may really be a help. It's my first doc-patch, so it surely needs to be revised, if interest exists in such a patch for mmap at all, say. Thanks for your understanding. -- Added file: http://bugs.python.org/file20861/doc_lib_mmap.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Antoine Pitrou pit...@free.fr added the comment: Does it matter that _4G UINT_MAX? You mean _4G UINT_MAX, right? Yes, it matters, otherwise that defeats the point of the test :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment: 'Have no glue, but Ned Daily's patch (msg129011) seems to be required for adler, too. (You know...) -- nosy: +sdaoden ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Antoine Pitrou pit...@free.fr added the comment: Well, it's not a patch, just a traceback :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment: Wait a few minutes, i'll write this simple patch for adler and crc. But excessive testing and such is beyond my current capabilities. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment: Sorry - that was a mess. -- Added file: http://bugs.python.org/file20837/issue11277.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Antoine Pitrou pit...@free.fr added the comment: File: issue11277.patch. Hmm. Two non-register constants and equal code on 32 and 64 bit. Does Python has a '64 bit' switch or the like - PY_SSIZE_T_MAX is not preprocessor-clean, i would guess. Er, how is this patch different from r88460? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment: I guess not at all. Well. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment: test_zlib.py (with my patch but that's somewhat identical in the end, say) does .s... -- Ran 37 tests in 1.809s OK (skipped=1) This is on Snow Leopard 64 bit, 02b70cb59701 (r88451) - Python 3.3a0. Is there a switch i must trigger? Just pulled 24 changesets, recompiling and trying again with r88460. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Antoine Pitrou pit...@free.fr added the comment: This is on Snow Leopard 64 bit, 02b70cb59701 (r88451) - Python 3.3a0. Is there a switch i must trigger? Just pulled 24 changesets, recompiling and trying again with r88460. Have you tried ./python -m test -v -uall test_zlib ? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment: No, i've got no idea of this framework... Just did 'python3 test_zlib.py' directly. Thanks for the switch. But i can't test your thing due to issue11285, so this may take a while (others have more knowledge anyway).. (P.S.: your constant-folding stack patch is a great thing, just wanted to say this once..) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment: So here is this (with my patch, but this is for real: issue11277.2.patch): == CPython 3.3a0 (py3k, Feb 22 2011, 14:00:52) [GCC 4.2.1 (Apple Inc. build 5664)] == Darwin-10.6.0-i386-64bit little-endian == /private/var/folders/Da/DaZX3-k5G8a57zw6MSmjJTM/-Tmp-/test_python_89365 Testing with flags: sys.flags(debug=0, division_warning=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0) [1/1] test_zlib test_adler32empty (test.test_zlib.ChecksumTestCase) ... ok test_adler32start (test.test_zlib.ChecksumTestCase) ... ok test_crc32_adler32_unsigned (test.test_zlib.ChecksumTestCase) ... ok test_crc32empty (test.test_zlib.ChecksumTestCase) ... ok test_crc32start (test.test_zlib.ChecksumTestCase) ... ok test_penguins (test.test_zlib.ChecksumTestCase) ... ok test_same_as_binascii_crc32 (test.test_zlib.ChecksumTestCase) ... ok test_badargs (test.test_zlib.ExceptionTestCase) ... ok test_badcompressobj (test.test_zlib.ExceptionTestCase) ... ok test_baddecompressobj (test.test_zlib.ExceptionTestCase) ... ok test_badlevel (test.test_zlib.ExceptionTestCase) ... ok test_decompressobj_badflush (test.test_zlib.ExceptionTestCase) ... ok test_big_compress_buffer (test.test_zlib.CompressTestCase) ... ok test_big_decompress_buffer (test.test_zlib.CompressTestCase) ... ok test_incomplete_stream (test.test_zlib.CompressTestCase) ... ok test_length_overflow (test.test_zlib.CompressTestCase) ... skipped 'not enough free memory, need at least 4 GB' test_speech (test.test_zlib.CompressTestCase) ... ok test_speech128 (test.test_zlib.CompressTestCase) ... ok test_badcompresscopy (test.test_zlib.CompressObjectTestCase) ... ok test_baddecompresscopy (test.test_zlib.CompressObjectTestCase) ... ok test_big_compress_buffer (test.test_zlib.CompressObjectTestCase) ... ok test_big_decompress_buffer (test.test_zlib.CompressObjectTestCase) ... ok test_compresscopy (test.test_zlib.CompressObjectTestCase) ... ok test_compressincremental (test.test_zlib.CompressObjectTestCase) ... ok test_compressoptions (test.test_zlib.CompressObjectTestCase) ... ok test_decompimax (test.test_zlib.CompressObjectTestCase) ... ok test_decompinc (test.test_zlib.CompressObjectTestCase) ... ok test_decompincflush (test.test_zlib.CompressObjectTestCase) ... ok test_decompress_incomplete_stream (test.test_zlib.CompressObjectTestCase) ... ok test_decompresscopy (test.test_zlib.CompressObjectTestCase) ... ok test_decompressmaxlen (test.test_zlib.CompressObjectTestCase) ... ok test_decompressmaxlenflush (test.test_zlib.CompressObjectTestCase) ... ok test_empty_flush (test.test_zlib.CompressObjectTestCase) ... ok test_flushes (test.test_zlib.CompressObjectTestCase) ... ok test_maxlenmisc (test.test_zlib.CompressObjectTestCase) ... ok test_odd_flush (test.test_zlib.CompressObjectTestCase) ... ok test_pair (test.test_zlib.CompressObjectTestCase) ... ok -- Ran 37 tests in 1.789s OK (skipped=1) 1 test OK. -- Added file: http://bugs.python.org/file20838/issue11277.2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Changes by Steffen Daode Nurpmeso sdao...@googlemail.com: Removed file: http://bugs.python.org/file20837/issue11277.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment: (Is not that much help for a 4GB error, huh?) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment: Just stepping ... with c8d1f99f25eb/r88476: == CPython 3.3a0 (py3k, Feb 22 2011, 14:18:19) [GCC 4.2.1 (Apple Inc. build 5664)] == Darwin-10.6.0-i386-64bit little-endian == /private/var/folders/Da/DaZX3-k5G8a57zw6MSmjJTM/-Tmp-/test_python_5126 Testing with flags: sys.flags(debug=0, division_warning=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0) [1/1] test_zlib test_adler32empty (test.test_zlib.ChecksumTestCase) ... ok test_adler32start (test.test_zlib.ChecksumTestCase) ... ok test_crc32_adler32_unsigned (test.test_zlib.ChecksumTestCase) ... ok test_crc32empty (test.test_zlib.ChecksumTestCase) ... ok test_crc32start (test.test_zlib.ChecksumTestCase) ... ok test_penguins (test.test_zlib.ChecksumTestCase) ... ok test_same_as_binascii_crc32 (test.test_zlib.ChecksumTestCase) ... ok test_big_buffer (test.test_zlib.ChecksumBigBufferTestCase) ... ^C ^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C Bus error -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Antoine Pitrou pit...@free.fr added the comment: Just stepping ... with c8d1f99f25eb/r88476: Right, that's what we should investigate :) Could try to diagnose the crash? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment:
.. even with a self-compiled 1.2.3, INT_MAX/1000 ... nothing.
The problem is not crc32(), but the buffer itself:
if (pbuf.len 1024*5) {
unsigned char *buf = pbuf.buf;
Py_ssize_t len = pbuf.len;
Py_ssize_t i;
fprintf(stderr, CRC 32 2.1\n);
for(i=0; (size_t)i (size_t)len;++i)
*buf++ = 1;
fprintf(stderr, CRC 32 2.2\n);
2.2 is never reached (in fact accessing buf[1] already causes fault).
Thus the problem is not zlib, but PyArg_ParseTuple().
But just don't ask me more on that!
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11277
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment: (P.S.: of course talking about ChecksumBigBufferTestCase and the 4GB, say.) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Antoine Pitrou pit...@free.fr added the comment:
.. even with a self-compiled 1.2.3, INT_MAX/1000 ... nothing.
The problem is not crc32(), but the buffer itself:
if (pbuf.len 1024*5) {
unsigned char *buf = pbuf.buf;
Py_ssize_t len = pbuf.len;
Py_ssize_t i;
fprintf(stderr, CRC 32 2.1\n);
for(i=0; (size_t)i (size_t)len;++i)
*buf++ = 1;
fprintf(stderr, CRC 32 2.2\n);
Thank you! So it's perhaps a bug in mmap on Snow Leopard.
Could you try to debug a bit more precisely and see at which buffer
offset (from the start) the fault occurs?
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11277
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment:
Snippet
if (pbuf.len 1024*5) {
volatile unsigned char *buf = pbuf.buf;
Py_ssize_t len = pbuf.len;
Py_ssize_t i = 0;
volatile unsigned char au[100];
volatile unsigned char*x = au;
fprintf(stderr, CRC ENTER, buffer=%p\n, buf);
for (i=0; (size_t)i (size_t)len; ++i) {
fprintf(stderr, %ld, buf=%p\n, (signed long)i, buf);
*x = *buf++;
}
results in
test_big_buffer (test.test_zlib.ChecksumBigBufferTestCase) ... CRC ENTER,
buffer=0x1014ab000
0, buf=0x1014ab000
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11277
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Antoine Pitrou pit...@free.fr added the comment: Out of curiosity, could you try the following patch? Index: Lib/test/test_zlib.py === --- Lib/test/test_zlib.py (révision 88500) +++ Lib/test/test_zlib.py (copie de travail) @@ -70,7 +70,7 @@ with open(support.TESTFN, wb+) as f: f.seek(_4G) f.write(basdf) -f.flush() +with open(support.TESTFN, rb) as f: self.mapping = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) def tearDown(self): -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Charles-Francois Natali neolo...@free.fr added the comment:
.. even with a self-compiled 1.2.3, INT_MAX/1000 ... nothing.
The problem is not crc32(), but the buffer itself:
if (pbuf.len 1024*5) {
unsigned char *buf = pbuf.buf;
Py_ssize_t len = pbuf.len;
Py_ssize_t i;
fprintf(stderr, CRC 32 2.1\n);
for(i=0; (size_t)i (size_t)len;++i)
*buf++ = 1;
fprintf(stderr, CRC 32 2.2\n);
Unless I'm mistaken, in the test the file is mapped with PROT_READ, so it's
normal to get SIGSEGV when writting to it:
def setUp(self):
with open(support.TESTFN, wb+) as f:
f.seek(_4G)
f.write(basdf)
f.flush()
self.mapping = mmap.mmap(f.fileno(), 0,
access=mmap.ACCESS_READ)
for(i=0; (size_t)i (size_t)len;++i)
*buf++ = 1;
But it seems you're also getting segfaults when only reading it, right ?
I've got a stupid question: how much memory do you have ?
Cause there seems to be some issues with page cache when reading mmaped files
on OS-X:
http://lists.apple.com/archives/darwin-development/2003/Jun/msg00141.html
On Linux, the page cache won't fill forever, so you don't need to have enough
free memory to accomodate the whole file (the page cache should grow, but not
forever). But on OS-X, it seems that the page replacement algorithm seems to
retain mmaped pages in the page cache much longer, which could potentially
trigger an OOM later (because of overcommitting, mmap can very well return a
valid address range which leads to a segfault when accessed later).
I'm not sure why it would segfault on the first page, though.
--
nosy: +neologix
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11277
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment: I have a MacBook with 2 GB RAM. Of course i'm a little bit messy, so an entry is half written before it comes to an ... end. msg129091 is real life, though. Antoine, your msg129093 patch of test_zlib.py does it (with and without fprintf(3)s). CRC ok etc., it just works. (Seems mmap(2) has a problem here, i would say; the mentioned bug report is from 2003, so the golden sunset watchers may need some more additional time, if you allow me that comment.) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment: (That is to say: i think it's better not to assume that these boys plan to *ever* fix it. (Though mmap(2) is not CoreAudio/AudioUnit.)) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment: (neologix: SIGBUS is not the same as SIGSEGV. You know. Thanks for this nice bug report. Eight years is a .. time in computer programming - unbelievable, thinking of all these nervous wrecks who ever reported a bug to Apple! Man!!!) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment: neologix: even with 2 GB RAM top(1) shows more than 600 MB free memory with the 4 GB test up and running ... in an Mac OS X environment ... Lucky me, i don't believe them a single word... -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Antoine Pitrou pit...@free.fr added the comment: Antoine, your msg129093 patch of test_zlib.py does it (with and without fprintf(3)s). CRC ok etc., it just works. Indeed, and it also seems to work on the buildbot. I will commit the patch soon. Thanks for your help! (Seems mmap(2) has a problem here, i would say; the mentioned bug report is from 2003, so the golden sunset watchers may need some more additional time, if you allow me that comment.) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Antoine Pitrou pit...@free.fr added the comment: Committed in r88511 (3.3) and r88514 (3.2). -- resolution: - fixed stage: needs patch - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
New submission from Antoine Pitrou pit...@free.fr: Following r88460 (issue10276), test_zlib crashes on the Snow Leopard buildbot (apparently in the new test_big_buffer test case). -- messages: 129002 nosy: ixokai, ned.deily, pitrou priority: critical severity: normal status: open title: test_zlib crashes under Snow Leopard buildbot type: crash versions: Python 3.2, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Antoine Pitrou pit...@free.fr added the comment: I've tried INT_MAX and it didn't change anything. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
STINNER Victor victor.stin...@haypocalc.com added the comment: Do adler32() and crc32() support length up to UINT32_MAX? Or should we maybe limit the length to INT32_MAX? -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Ned Deily n...@acm.org added the comment: Current OS X zlib is 1.2.3. Test crashes with most recently released zlib, 1.2.5, as well. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Ned Deily n...@acm.org added the comment:
Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: 10 at address: 0x00010170e000
0x0001016eeaa0 in crc32 ()
(gdb) backtrace
#0 0x0001016eeaa0 in crc32 ()
#1 0x0001016e806d in PyZlib_crc32 (self=0x1016aa588, args=0x1016bf220) at
/private/tmp/a/py3k/Modules/zlibmodule.c:993
PyZlib_crc32(PyObject *self, PyObject *args)
...
while (len (size_t) UINT_MAX) {
crc32val = crc32(crc32val, buf, UINT_MAX);
...
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11277
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Changes by Brett Cannon br...@python.org: -- nosy: +brett.cannon ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Ned Deily n...@acm.org added the comment: from test.support import _4G _4G 4294967296 mapping.size() 4294967300 pbuf.len = 4294967300, len = 4294967300 UINT_MAX = 4294967295 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib crashes under Snow Leopard buildbot
Brett Cannon br...@python.org added the comment: Does it matter that _4G UINT_MAX? -- stage: - needs patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11277 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
