[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)
Χρήστος Γεωργίου (Christos Georgiou) added the comment: First, let's make sure we're on the same page. - These files are created during the `$(MAKE) run_profile_task` stage. - They get removed during the `$(MAKE) clean` stage, along with the build directory. - The build directory gets recreated during the `$(MAKE) build_all_use_profile` stage, without any .gcda files this time. So you won't see these files after a successful build *if* you haven't taken measures to ensure they are saved during the build process. I save these files to a cpio file *before* `make clean` runs and restore them right afterwards. I suggest you modify `Makefile.pre.in` to include similar commands to the ones I mention in issue #23607 to verify whether your system creates these files or not. Now, for the info you required: It's a system running Ubuntu 14.04 64-bit with gcc 4.8.4. It's a build of Python with the `-mx32` flag, along with all required libraries for the needs of a specific project. (I think that the `-mx32` flag is not important to our discussion here though.) It isn't a cross-compilation. -- ___ Python tracker <http://bugs.python.org/issue24915> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)
Χρήστος Γεωργίου (Christos Georgiou) added the comment: There are. (Check issue #26307 that explains this cpio file. This is a x32 build of Python, because the memory savings are very welcome for the multiple worker processes of a project I work on.) $ cpio -it <_modules.gcda.cpio build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/_multiprocessing/semaphore.gcda build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/_multiprocessing/multiprocessing.gcda build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/_posixsubprocess.gcda build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/_struct.gcda build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/_heapqmodule.gcda build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/_opcode.gcda build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/binascii.gcda build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/_ssl.gcda build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/arraymodule.gcda build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/_pickle.gcda build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/_randommodule.gcda build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/_hashopenssl.gcda build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/_lzmamodule.gcda build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/grpmodule.gcda build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/socketmodule.gcda build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/selectmodule.gcda build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/_math.gcda build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/mathmodule.gcda build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/_datetimemodule.gcda build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/resource.gcda build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/zlibmodule.gcda build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/unicodedata.gcda build/temp.linux-x86_64-3.5/opt/x32/src/Python-3.5.1/Modules/_testcapimodule.gcda -- ___ Python tracker <http://bugs.python.org/issue24915> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26307] no PGO for built-in modules with `make profile-opt`
New submission from Χρήστος Γεωργίου (Christos Georgiou): (related to issue #24915) I discovered that `make profile-opt` does not use the profile information for the builtin-modules (e.g. arraymodule or _pickle) because in the `profile-opt` target there is the following sequence of actions: … $(MAKE) build_all_merge_profile @echo "Rebuilding with profile guided optimizations:" $(MAKE) clean $(MAKE) build_all_use_profile … The action `$(MAKE) clean` performs an `rm -rf build`, destroying among other things all the *.gcda files generated for the built-in modules. On my Linux system with gcc, a kludge to `Makefile.pre.in` that works is: … @echo "Rebuilding with profile guided optimizations:" find build -name \*.gcda -print | cpio -o >_modules.gcda.cpio # XXX $(MAKE) clean cpio -id <_modules.gcda.cpio # XXX $(MAKE) build_all_use_profile … but, like I said, it's a kludge and it's POSIX-only (-print0 can be avoided since I believe it's guaranteed that there will be no whitespace-containing-filenames). Now, if this road is to be taken, I believe the most cross-platform method available to save the profile-generated files is to use a custom python script (at this point, the local profile-generating python executable is functional) and make a tar file, which will be untared back. However, I don't like it much. I'm willing to provide any necessary patches if someone gives me a "proper" roadmap as to how to handle this issue. -- components: Build messages: 259842 nosy: tzot priority: normal severity: normal status: open title: no PGO for built-in modules with `make profile-opt` type: performance versions: Python 2.7, Python 3.5, Python 3.6 ___ Python tracker <http://bugs.python.org/issue26307> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24915] Profile Guided Optimization improvements (better training, llvm support, etc)
Χρήστος Γεωργίου (Christos Georgiou) added the comment: Perhaps I'm missing something obvious here, but… … $(MAKE) build_all_merge_profile @echo "Rebuilding with profile guided optimizations:" $(MAKE) clean $(MAKE) build_all_use_profile … the `$(MAKE) clean` does an `rm -rf build`, so it also removes the .gcda for the builtin modules. -- nosy: +tzot ___ Python tracker <http://bugs.python.org/issue24915> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25393] 'resource' module documentation error
New submission from Χρήστος Γεωργίου (Christos Georgiou): https://docs.python.org/3.5/library/resource.html https://docs.python.org/3.5/library/resource.html#resource.RLIMIT_FSIZE ends with the sentence "This only affects the stack of the main thread in a multi-threaded process." I believe this sentence should be appended to https://docs.python.org/3.5/library/resource.html#resource.RLIMIT_STACK This error also exists in previous versions of python documentation. -- assignee: docs@python components: Documentation messages: 252935 nosy: docs@python, tzot priority: normal severity: normal status: open title: 'resource' module documentation error type: enhancement versions: Python 3.5 ___ Python tracker <http://bugs.python.org/issue25393> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10160] operator.attrgetter slower than lambda after adding dotted names ability
Χρήστος Γεωργίου (Christos Georgiou) added the comment: This is not the proper place for it, but in the 3.2 and 2.7 news it is reported that “The multi-argument form of operator.attrgetter() function now runs slightly faster” while it should be “The multi-argument form of operator.attrgetter() function now runs slightly faster and the single-argument form runs much faster”. -- ___ Python tracker <http://bugs.python.org/issue10160> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5069] Use sets instead of list in posixpath._resolve_link
New submission from Χρήστος Γεωργίου (Christos Georgiou) : The paths_seen object is a list; a set is more appropriate, since its main use is a lookup as in "path in paths_seen" -- components: Library (Lib) files: posixpath.diff keywords: patch messages: 80570 nosy: tzot severity: normal status: open title: Use sets instead of list in posixpath._resolve_link type: performance versions: Python 2.7 Added file: http://bugs.python.org/file12867/posixpath.diff ___ Python tracker <http://bugs.python.org/issue5069> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup for / while / if with better bytecode
Χρήστος Γεωργίου (Christos Georgiou) <[EMAIL PROTECTED]> added the comment: A pointer to previous (minor) research: http://groups.google.com/group/comp.lang.python/browse_frm/thread/72505e3cb6d9cb1a/e486759f06ec4ee5 esp. after Terry Reedy's post -- nosy: +tzot ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2459> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2195] urlparse() does not handle URLs with port numbers properly
Χρήστος Γεωργίου (Christos Georgiou) added the comment: RFC1808 §2.1 suggests a generic RL syntax that specifies '://' as the separator, so Gawain's suggestion makes practical sense. However, also as Gawain says, the RFC specifies that '//' is considered as the first part of a "net_path" and is not necessarily included (example: "mailto:[EMAIL PROTECTED]" (and yes, I actually welcome spammers :) ). I believe that urlparse should stay as-is when not called with a default_scheme argument, and fixed as-suggested when called with a default_scheme argument (that's the point for providing default_scheme). -- nosy: +tzot __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2195> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1860] traceback.print_last fails
Χρήστος Γεωργίου (Christos Georgiou) added the comment: I haven't submitted a patch since the transition from sf.net to bugs.python.org; I assume that I don't have to open a new patch for this, but if I have to, please let me know and I will gladly do it. The unified diff is attached; the test example I issued works with the patched version. Added file: http://bugs.python.org/file9194/traceback_patch.diff __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1860> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1860] traceback.print_last fails
New submission from Χρήστος Γεωργίου (Christos Georgiou): traceback.print_last() depends on the existence of sys.last_type, sys.last_value, sys.last_traceback, which don't always exist when called. See attached example file. I will shortly send the patch for Lib/traceback.py -- components: Library (Lib) files: test_tb_print_last.py messages: 60028 nosy: tzot severity: normal status: open title: traceback.print_last fails type: behavior versions: Python 2.5, Python 2.6 Added file: http://bugs.python.org/file9193/test_tb_print_last.py __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1860> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
