[issue17667] Windows: build with "build_pgo.bat -2" fails to optimize python.dll
New submission from Anselm Kruis: Version: 2.7.4 (and any prior 2.7-version. not tested with 3.x) Compiler: VS 2008 Professional If I compile Python using the batch > PCbuild\build_pgo.bat -2 Visual Studio fails to correctly collect profile information for the project "python27.dll". As a result, no real optimisation takes place. The command "build_pgo.bat -2" runs the full test suite (..\lib\test\regrtest.py) to collect profile data. Experiments show, that several test cases (i.e. test_os and test_subprocess) break the collection of profile data. Probably the failure is related to the creation of sub-processes. I consider it a Visual Studio bug. To work around this issue I created a list of good test cases and patched build_pgo.bat to run only the tests from this list. -- components: Build, Windows files: build_pgo-2.patch keywords: patch messages: 186322 nosy: akruis priority: normal severity: normal status: open title: Windows: build with "build_pgo.bat -2" fails to optimize python.dll type: compile error versions: Python 2.7 Added file: http://bugs.python.org/file29739/build_pgo-2.patch ___ Python tracker <http://bugs.python.org/issue17667> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5773] Crash on shutdown after os.fdopen(2) in debug builds
Anselm Kruis added the comment: Hi, I was faced with a very similar problem also caused by an invalid file descriptor. My solution is to set an invalid parameter handler, that does nothing. This effectively disables Dr. Watson. Perhaps this is a suitable solution for other users too. And it does not require a patch. def set_invalid_parameter_handler(flag): """ Set the MSVCRT invalid parameter handler. If flag is True, this function sets an invalid parameter handler, that does nothing. This effectively disables Dr. Watson. If flag is an integer number, it must be the address of an invalid parameter handler function. If flag is None, this function removes the invalid parameter handler. This effectively enables Dr. Watson. The return value is the address of the current handler or None, if no handler is installed. Example:: old = set_invalid_parameter_handler(True) try: do_something_nasty finally: set_invalid_parameter_handler(old) """ try: # get the msvcrt library import ctypes.util libc = ctypes.util.find_msvcrt() if not libc: # probably not windows return None libc = getattr(ctypes.cdll, libc) siph = libc._set_invalid_parameter_handler siph.restype = ctypes.c_void_p siph.argtypes = [ ctypes.c_void_p ] # now we need a suitable handler. # The handler must simply return without performing any actions. # Of course there is none. # But if we look at the calling convention (cdecl), and # at the fact, that we don't need the argument values # we find, that we can call any function, as long as the function # does not harm. A suitable function is "int abs(abs)". null_handler = libc.abs except Exception: # probably not the correct windows version return None if flag is True: flag = null_handler return siph(flag) -- nosy: +anselm.kruis ___ Python tracker <http://bugs.python.org/issue5773> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18004] test_list.test_overflow crashes Win64
New submission from Anselm Kruis: I installed Python 2.7.5 including tests using the MSI installer from http://www.python.org/ftp/python/2.7.5/python-2.7.5.amd64.msi Running python.exe -m test.regrtest -v test_list consumes all available memory and renders the my system completely unusable. I had to reboot. The problem is caused by the test case "test_overflow". If you run the test with an imposed memory limit, e.g. by using ulimitnt (http://code.google.com/p/ulimitnt/) , test_overflow passes. I propose to skip this test on Win64. -- components: Tests messages: 189491 nosy: anselm.kruis priority: normal severity: normal status: open title: test_list.test_overflow crashes Win64 type: resource usage versions: Python 2.7 ___ Python tracker <http://bugs.python.org/issue18004> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15535] Fix pickling efficiency of named tuples in 2.7.3
Anselm Kruis added the comment: >> why the different fix for 3.3 > > I reverted the 2.7.4 addition of __dict__ rather than introduce more > differences between point releases with possible unintended effects. __dict__ was a 2.7.3 addition (changeset 26d5f022eb1a). Now unpickling of named tuples created by 2.7.3 and 2.7.4 fails. -- nosy: +anselm.kruis ___ Python tracker <http://bugs.python.org/issue15535> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18015] python 2.7.5 fails to unpickle namedtuple pickled by 2.7.3 or 2.7.4
New submission from Anselm Kruis: Change 18303391b981 breaks unpickling named tuples pickled by 2.7.3 and 2.7.4. See closed issue #15535 for the full story. Unfortunately Raymond was wrong, when he wrote that the addition of __dict__ was a 2.7.4 change. It was added by changeset 26d5f022eb1a in 2.7.3. Now 2.7.5 can't unpickle any named tuples pickled by 2.7.3, which is probably one of the most widely used python versions. Example: Pickle a namd tuple using 2.7.3 and unpickle it using 2.7.5. anselm@Emmy:~$ python2.7 Python 2.7.3 (default, Sep 16 2012, 21:46:37) [GCC 4.4.5] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import collections >>> import pickletools >>> import pickle >>> N=collections.namedtuple("N","a") >>> n=N(1) >>> p=pickle.dumps(n, 2) >>> p2=pickletools.optimize(p) >>> pickletools.dis(p2) 0: \x80 PROTO 2 2: cGLOBAL '__main__ N' 14: KBININT11 16: \x85 TUPLE1 17: \x81 NEWOBJ 18: cGLOBAL 'collections OrderedDict' 43: ]EMPTY_LIST 44: ]EMPTY_LIST 45: (MARK 46: USHORT_BINSTRING 'a' 49: KBININT11 51: eAPPENDS(MARK at 45) 52: aAPPEND 53: \x85 TUPLE1 54: RREDUCE 55: bBUILD 56: .STOP highest protocol among opcodes = 2 >>> print repr(p2) '\x80\x02c__main__\nN\nK\x01\x85\x81ccollections\nOrderedDict\n]](U\x01aK\x01ea\x85Rb.' anselm@Emmy:~/sc/eclipsews/fg2py$ fg2python Python 2.7.5 (default, May 18 2013, 17:02:17) [GCC 4.1.2 20080704 (Red Hat 4.1.2-44)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import pickle >>> import collections >>> N=collections.namedtuple("N","a") >>> pickle.loads('\x80\x02c__main__\nN\nK\x01\x85\x81ccollections\nOrderedDict\n]](U\x01aK\x01ea\x85Rb.') Traceback (most recent call last): File "", line 1, in File "/home/anselm/sc/eclipsews/fg2py/arch/rhel4u4-x86_64/lib/python2.7/pickle.py", line 1419, in loads return Unpickler(file).load() File "/home/anselm/sc/eclipsews/fg2py/arch/rhel4u4-x86_64/lib/python2.7/pickle.py", line 895, in load dispatch[key](self) File "/home/anselm/sc/eclipsews/fg2py/arch/rhel4u4-x86_64/lib/python2.7/pickle.py", line 1261, in load_build d = inst.__dict__ AttributeError: 'N' object has no attribute '__dict__' As we can see from the trace back, the problem arises from the pickle op-code 'BUILD'. BUILD requires that the object to be build either has a method __setstate__ or has an attribute __dict__. Therefore I propose: - Revert change 18303391b981 and add a __getstate__ method This is the Python 3 fix for the problem. or - Add a method __setstate__: def __setstate__(self, state): """For compatibility with Python 2.7.3 and 2.7.4""" pass -- components: Library (Lib) messages: 189614 nosy: anselm.kruis, rhettinger priority: normal severity: normal status: open title: python 2.7.5 fails to unpickle namedtuple pickled by 2.7.3 or 2.7.4 type: behavior versions: Python 2.7 ___ Python tracker <http://bugs.python.org/issue18015> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18023] msi product code for 2.7.5150 not in Tools/msi/uuids.py
New submission from Anselm Kruis: The file Tools/msi/uuids.py contains the product codes for all recently released Python 2.x versions except 2.7.5. Without this code it is not possible to recreate the MSI installer using Tools\msi\msi.py. The product code of http://www.python.org/ftp/python/2.7.5/python-2.7.5.msi is '{DBDD570E-0952-475F-9453-AB88F3DD5659}'. -- components: Build, Windows messages: 189686 nosy: anselm.kruis priority: normal severity: normal status: open title: msi product code for 2.7.5150 not in Tools/msi/uuids.py type: compile error versions: Python 2.7 ___ Python tracker <http://bugs.python.org/issue18023> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18004] test_list.test_overflow crashes Win64
Anselm Kruis added the comment: > I take it you have more than 16GB of RAM? I used a system with 16GB Ram. > What happens if you replace "sys.maxint" with "sys.maxsize" in test_overflow? The test passes. Both mul and imul raise MemoryError. -- ___ Python tracker <http://bugs.python.org/issue18004> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18015] python 2.7.5 fails to unpickle namedtuple pickled by 2.7.3 or 2.7.4
Anselm Kruis added the comment: Just for the records: the patch works as expected. -- ___ Python tracker <http://bugs.python.org/issue18015> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18015] python 2.7.5 fails to unpickle namedtuple pickled by 2.7.3 or 2.7.4
Anselm Kruis added the comment: I created a small *.pth to monkey patch collections.py until 2.7.6 gets released. Maybe this is useful for someone else. Therefore I attach it here. The pth file runs the following code during Python startup: import collections def _fix_issue_18015(collections): try: template = collections._class_template except AttributeError: # prior to 2.7.4 _class_template didn't exists return if not isinstance(template, basestring): return # strange if "__dict__" in template or "__getstate__" in template: return # already patched lines = template.splitlines() indent = -1 for i,l in enumerate(lines): if indent < 0: indent = l.find('def _asdict') continue if l.startswith(' '*indent + 'def '): lines.insert(i, ' '*indent + 'def __getstate__(self): pass') lines.insert(i, ' '*indent + '__dict__ = _property(_asdict)') break collections._class_template = '''\n'''.join(lines) _fix_issue_18015(collections) -- Added file: http://bugs.python.org/file30338/fix_python_275_issue18015.pth ___ Python tracker <http://bugs.python.org/issue18015> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com