Alecsandru Patrascu added the comment:
Serhiy, I am curious why did you have chosen to compute the CRC32 table
everytime? It is standard (the generator polynomial does not change) and always
will output the same values. And it is also less computational intensive to
loading a precomputed
Alecsandru Patrascu added the comment:
The patches in discussions are the v02 ones.
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Alecsandru Patrascu added the comment:
I've updated the patch for 2.7 branch to contain the LTO part merged by Gregory.
--
Added file:
http://bugs.python.org/file43142/cpython2_with_optimizations_v02.patch
___
Python tracker <rep...@bugs.python.
Alecsandru Patrascu added the comment:
Hello Brett, I saw that there was no other activity on this issue so far. Can
you please update this, as I do not have rights?
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Alecsandru Patrascu added the comment:
Brett, I tried to add you in the "assigned to", but I do not have any rights to
update those :(
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.py
Alecsandru Patrascu added the comment:
I've attached a cleaner version for CPython 3 and one for CPython 2. The
problem with CPython 2 is that Gregory did not yet merged the LTO on this
branch (#25702), but as soon as he does it, I'll post updated patches.
Regards,
Alecsandru
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Added file:
http://bugs.python.org/file42872/cpython3_with_optimizations_v02.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Added file:
http://bugs.python.org/file42871/cpython2_with_optimizations_v01.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Alecsandru Patrascu added the comment:
I've added the --with-optimizations option to configure, for CPython3 only at
this point. If it looks good I'll create a version also for CPython2.
--
Added file:
http://bugs.python.org/file42576/cpython3_with_optimizations_v01.patch
Alecsandru Patrascu added the comment:
I uploaded a patch that skips these tests if optimizations are available.
--
keywords: +patch
Added file: http://bugs.python.org/file42574/issue26788.patch
___
Python tracker <rep...@bugs.python.org>
Alecsandru Patrascu added the comment:
This patch is for CPython 3, and if it looks good I will post also the CPython
2 version.
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Alecsandru Patrascu added the comment:
Hello,
You can find a patch for this issue attached. Basically, since LTO needs LD
flags when used, I modified the makefile template file and the test to push the
flags up to the sysconfig module.
--
keywords: +patch
Added file: http
Alecsandru Patrascu added the comment:
I will investigate this issue to understand what is happening there and submit
a fix. Thank you for pointing out the exact OS and toolchain used.
--
___
Python tracker <rep...@bugs.python.org>
Alecsandru Patrascu added the comment:
I will investigate this and submit a fix. At a first glance, it seems the test
is failing because it does not have the knowledge of the LTO flags.
--
___
Python tracker <rep...@bugs.python.org>
Alecsandru Patrascu added the comment:
That sounds like a good idea and plan, to have everything enabled with just one
dedicated configure flag.
I can work on this and post the patches as soon as they are done.
Thank you
--
___
Python tracker
Alecsandru Patrascu added the comment:
Maybe an workflow like the one proposed in issue #26359 can be helpful in these
development phases.
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Alecsandru Patrascu added the comment:
@Stefan and @Marc, you say that people don't want to wait for PGO to build when
running ./configure && make, but why? Even though many developers use it, this
mode is not intended for development, it is production level and should be run
once (or
Alecsandru Patrascu added the comment:
Maybe that end users are lazy and want to type just `make` and let things
happen magically behind the scenes :-)
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Alecsandru Patrascu added the comment:
Hello Victor,
Indeed, the best outcome is to have PGO, LTO, etc enabled by default when
running ./configure && make, for production level. I also feel that they should
be on by default, if only developers would use the debug version.
Do y
Alecsandru Patrascu added the comment:
You are doing measurements on a virtual machine... For sure you are not the
only user that has active workloads on the physical machine while you do
benchmarks :)
On the other hand, the path you are going with just LTO is nice for
experiments
Alecsandru Patrascu added the comment:
>From our experience, pybench only is not a representative benchmark. Instead,
>if you like to measure performance close to real workloads, you can run the
>Grand Unified Python Benchmark suite, that is more complete.
Also, you nee
Alecsandru Patrascu added the comment:
I understand now your question. LTO is not enabled when running just `make`,
only in `make profile-opt`
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Alecsandru Patrascu added the comment:
Yes, you can use LTO without PGO, but the proposed ways it's more efficient and
makes more sense for CPython builds.
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
--
nosy: +gregory.p.smith, lemburg, scoder, steve.dower, zach.ware
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
--
nosy: +brett.cannon, gregory.p.smith, lemburg, pitrou, r.david.murray, scoder,
skrah, steve.dower, zach.ware
___
Python tracker <rep...@bugs.python.org>
<http://
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Added file: http://bugs.python.org/file42000/pgofix-cpython3.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Alecsandru Patrascu added the comment:
I've added the patch here also.
--
keywords: +patch
Added file: http://bugs.python.org/file41999/pgofix-cpython2.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
--
nosy: +alecsandru.patrascu
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
--
nosy: +alecsandru.patrascu
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
--
resolution: fixed ->
status: closed -> open
___
Python tracker <rep...@bugs.python.org>
<http://bugs.
Alecsandru Patrascu added the comment:
I've added a fix for the PGO builds after the issue pointed out by you. Can you
please test it also?
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
New submission from Alecsandru Patrascu:
Hello,
This is Alecsandru from the Dynamic Scripting Languages Optimization Team at
Intel Corporation. I would like to submit a patch as a response to the
discussion thread opened in Python-Dev
(https://mail.python.org/pipermail/python-dev/2016
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Added file: http://bugs.python.org/file41919/cpython3_pgo_default-v02.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Added file: http://bugs.python.org/file41917/pgofix-cpython3.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Alecsandru Patrascu added the comment:
I've added a fix for the PGO builds after a issue pointed out in #26307. Thank
you Christos for your observation!
--
Added file: http://bugs.python.org/file41916/pgofix-cpython2.patch
___
Python tracker <
Alecsandru Patrascu added the comment:
Hello and thank you for your feedback. For CPython this does not apply because
due to the structure of the build system, inside the "build" directory there
are no PGO profiles saved. You can run find . -name '*.gc
Alecsandru Patrascu added the comment:
Thank you for noticing and signaling this issue. Since I proposed the PGO
patches, I will fix it in another patch, as I don't want it to break any builds.
--
___
Python tracker <rep...@bugs.python.org>
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
--
nosy: +alecsandru.patrascu
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Alecsandru Patrascu added the comment:
That's interesting. Even on CPython3, I still don't see any gcda's inside the
build directory, nor the tree structure you are seeing there. Can you please
give me a couple of details regarding your environment (os, distribution, gcc
version, 32/64 bit
Alecsandru Patrascu added the comment:
For my responses, I modified locally the Makefile so that it will not remove
the build directory and any of the gcda files. I will make some more tests
tomorrow, but i think that this problem will solve simpler if the removal of
the build directory
Alecsandru Patrascu added the comment:
I've done again the experiments on larger workloads, such as our OpenStack
Swift cluster, and it works without any issues.
Also, I've attached an archive with a simple external module in CPython3 that
uses PyMem_RawMalloc. The output is ok, and it's
Alecsandru Patrascu added the comment:
Sure, I attached them as files because they have a lot of lines for posting
here (~90 in total).
The linker offers the possibility to show what piece of data/functions was
removed, but I intentionally omitted it in order not to clutter the build
trace
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Added file: http://bugs.python.org/file41809/gc-removed-cpython3.txt
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Added file: http://bugs.python.org/file41806/cpython3-deadcode-v01.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
New submission from Alecsandru Patrascu:
Hi all,
This is Alecsandru from the Dynamic Scripting Languages Optimization Team at
Intel Corporation. I would like to submit a patch that enables garbage
collection of unused input sections from the CPython2 and CPython3 binaries, by
using the "
Alecsandru Patrascu added the comment:
I realize now that I should have explained a bit more the background of this
patch. I'll do this now, for everyone to be clear of what is the effect of
those flags.
This issue was revealed after running the coverage target over various
workloads
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
--
nosy: +alecsandru.patrascu
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Alecsandru Patrascu added the comment:
Yury, thank you for the heads up! Here at Intel, in the Dynamic Scripting
Languages Optimization Team, we can help the community with reviewing and
measuring this patch in our quiet and stable environment, the same one that we
use to provide public
Alecsandru Patrascu added the comment:
As Steve mentioned, the Microsoft compiler uses LTO (they call it Link-Time
Code Generation) and the flags are used when compiling CPython on Windows
systems. Thus our proposal to enable it on GCC and CLANG also
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Added file: http://bugs.python.org/file41666/lto-cpython3-v04.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Alecsandru Patrascu added the comment:
Thank you for your feedback, I've updated the patches and now LTO flags are
used only when building with PGO (v04). CFLAGS/LDFLAGS remain untouched, as
Antoine and Victor suggested is better.
--
Added file: http://bugs.python.org/file41665/lto
Alecsandru Patrascu added the comment:
Hello, I've added an updated set of patches (v03) for the current CPython2 and
CPython3 codebase. Also made some small changes to reduce the number of places
where the flags are set.
--
Added file: http://bugs.python.org/file41493/lto-cpython2
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Added file: http://bugs.python.org/file41494/lto-cpython3-v03.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Alecsandru Patrascu added the comment:
I'm adding Brett, Gregory, Stefan and Victor as nosy because this issue might
be interesting for them also.
--
nosy: +brett.cannon, gregory.p.smith, haypo, scoder, skrah
___
Python tracker <
Alecsandru Patrascu added the comment:
Thank you for the reviews. I've added a new patch (v02) to correct the issues.
--
Added file: http://bugs.python.org/file41386/coverage-2.7-v02.patch
___
Python tracker <rep...@bugs.python.org>
New submission from Alecsandru Patrascu:
Hi All,
This is Alecsandru from Server Scripting Languages Optimization team at Intel
Corporation. I would like to submit a patch that adds extended support for
coverage, in the same way that it is used in CPython3. Used in conjunction
Alecsandru Patrascu added the comment:
Hi, I added a dedicated issue just for LTO only when using GCC and CLANG
(http://bugs.python.org/issue25702), that works well with PGO also.
--
nosy: +alecsandru.patrascu
___
Python tracker <
Alecsandru Patrascu added the comment:
Thank you David for including me in this issue.
On ICC, when executing the instrumented applications, it will generate dynamic
information file that has a unique name and .dyn suffix. From my previous
experience with GCC and CLANG PGO patch (issue24915
Alecsandru Patrascu added the comment:
LTO exists in GCC since version 4.5, but it is true that only recent versions
(>=4.8) perform it in good conditions. It is not enabled by default in this
patch, it is only available when building with PGO support. Running just "make"
will
New submission from Alecsandru Patrascu:
Title: Link Time Optimizations support for GCC and CLANG
Hi All,
This is Alecsandru from Server Scripting Languages Optimization team at Intel
Corporation. I would like to submit a patch that adds support for Link Time
Optimization (LTO) when using
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Added file: http://bugs.python.org/file41131/lto-cpython3-v01.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
--
keywords: +patch
Added file: http://bugs.python.org/file41130/lto-cpython2-v01.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Added file: http://bugs.python.org/file41138/lto-cpython3-v02.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Alecsandru Patrascu added the comment:
Meanwhile I've added the patches (v02) for LTO enabled only if the "./configure
--with-lto" command is issued.
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.py
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Added file: http://bugs.python.org/file41137/lto-cpython2-v02.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Alecsandru Patrascu added the comment:
Thank you Brett for your effort to merge all the changes made to regrtest
despite the changes appeared on the way!
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Alecsandru Patrascu added the comment:
Thank you for the tweaks. I tested the patch and it is all working fine.
Without failure count the PGO run is even more transparent to the user.
I tested your final version of the patch on several Linux server and regular
desktop distributions
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Added file: http://bugs.python.org/file40547/regrtest_36_v02.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Alecsandru Patrascu added the comment:
The patch does not display dots, as they are not that intuitive and users
cannot make an estimate of the time left or the position in the training phase.
Instead, the regrtest.py, will display the progress as [X/Y], like the default
version, but without
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
--
keywords: +patch
Added file: http://bugs.python.org/file40532/regrtest_27.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Added file: http://bugs.python.org/file40533/regrtest_36.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Alecsandru Patrascu added the comment:
I've added the patches for Python default and 2.7. To apply them please use the
command "hg import --no-commit". PGO flag is implemented for both variants
(single and multi process). Default in profile-opt will be sing
Alecsandru Patrascu added the comment:
Thank you for upstreaming this in both branches of Python!
Do you think that a different version of regrtest.py, that will be used only
for PGO training, should be better in this case? I mean, by implementing a
custom version, I think we can control
New submission from Alecsandru Patrascu:
This issue adds improved support for Profile Guided Optimization builds in the
regression test suite.
Mainly, we keep the visual progress status in this process and suppress errors
or any other unnecessary output that can alarm users
Alecsandru Patrascu added the comment:
I can work on modifying the existing regrtest and adding a distinct flag, --pgo
for example, as Antoine suggested. Indeed, it will not be trivial as regrtest
has a dual approach (single process and multi process), but I will give it a
try and post
Alecsandru Patrascu added the comment:
Please hold this patch for a while, while I analyze the received feedback.
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Added file: http://bugs.python.org/file40460/hash8-v02.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Alecsandru Patrascu added the comment:
I've submitted a more compact version of the previous code (hash8-v02.patch)
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/i
Alecsandru Patrascu added the comment:
Yes, it doesn't break the compatibility with existing applications. To make
sure, we tested it in various scenarios and workloads.
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.p
New submission from Alecsandru Patrascu:
Hi All,
This is Alecsandru from Server Scripting Languages Optimization team at Intel
Corporation.
I would like to submit a patch that improves the performance of the hash
computation code on stringobject, bufferobject and unicodeobject. As can
Alecsandru Patrascu added the comment:
If you are talking just about the 3.6 patch, it is called this way to emphasize
the fact that it is intended for the development branch. It is perfectly
compatible with 3.5, therefore it is not needed for packagers to maintain two
distinct versions. I've
Alecsandru Patrascu added the comment:
I've updated the patches with redirect to /dev/null, as is it is more clearer
to the user what is our intent, without having him to necessarily read the
regrtest documentation. I've also added a warning message regarding the output
and ported all
Alecsandru Patrascu added the comment:
I named this task PROFILE_TASK_X86 because it is rigorously tested and we have
proven that regrtest performs better on this architecture. Until any other
clear evidence and solid measurements that regrtest is performing better on
other architectures
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Added file: http://bugs.python.org/file40387/python2.7-pgo-v07.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Added file: http://bugs.python.org/file40388/python3.6-pgo-v07.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Added file: http://bugs.python.org/file40389/README2.7-pgo-v02.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Added file: http://bugs.python.org/file40390/README3.6-pgo-v02.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Removed file: http://bugs.python.org/file40387/python2.7-pgo-v07.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Added file: http://bugs.python.org/file40392/python3.6-pgo-v07.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Removed file: http://bugs.python.org/file40388/python3.6-pgo-v07.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Changes by Alecsandru Patrascu <alecsandru.patra...@intel.com>:
Added file: http://bugs.python.org/file40391/python2.7-pgo-v07.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python
Changes by Alecsandru Patrascu alecsandru.patra...@intel.com:
Added file: http://bugs.python.org/file40299/README2.7-pgo-v01.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24915
Changes by Alecsandru Patrascu alecsandru.patra...@intel.com:
Added file: http://bugs.python.org/file40300/README3.6-pgo-v01.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24915
Alecsandru Patrascu added the comment:
That's a good point Skip. I added another set of patches, just for the README
files, explaining the entire procedure, so now anyone reading it will see that
PGO is available, what are the steps involved and a brief comment about the
warning
Changes by Alecsandru Patrascu alecsandru.patra...@intel.com:
Added file: http://bugs.python.org/file40289/python3.6-pgo-v06.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24915
Changes by Alecsandru Patrascu alecsandru.patra...@intel.com:
Added file: http://bugs.python.org/file40288/python2.7-pgo-v06.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24915
Alecsandru Patrascu added the comment:
I've updated the patches after review and implemented the checkup for
llvm-profdata for both Linux and OSX.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24915
Alecsandru Patrascu added the comment:
The profile merging is necessary in case you want to use a pure clang compiler
or you use GCC in OSX. For example, a general profiling action using clang will
result in at least one binary profile. For our case, when using regrtest, we
will have multiple
Changes by Alecsandru Patrascu alecsandru.patra...@intel.com:
Added file: http://bugs.python.org/file40274/python3.6-pgo-v05.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24915
Changes by Alecsandru Patrascu alecsandru.patra...@intel.com:
Added file: http://bugs.python.org/file40273/python2.7-pgo-v05.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24915
1 - 100 of 123 matches
Mail list logo
