Roundup Robot added the comment:
New changeset 4927899bea8d by Serhiy Storchaka in branch '2.7':
Issue #1285086: Get rid of the refcounting hack and speed up urllib.unquote().
http://hg.python.org/cpython/rev/4927899bea8d
New changeset 3cb07925fcb9 by Serhiy Storchaka in branch '3.2':
Issue
Serhiy Storchaka added the comment:
Sorry, I perhaps missed your response, Senthil. Now committed and closed again.
--
resolution: - fixed
stage: patch review - committed/rejected
status: open - closed
___
Python tracker rep...@bugs.python.org
Senthil Kumaran added the comment:
Serhiy - Is there any technical issue that is holding up this patch? (I dont
see any). If nothing is holding up and you are busy, I shall go ahead with
committing this one. /cc flox
--
___
Python tracker
Senthil Kumaran added the comment:
Serhiy Storchaka added the comment:
+ append = res.append
And then use 'append'?
This speed up unquote_to_bytes by 15%.
Thanks for the response. In fact, writing the whole _hextobyte
verbatim would further increase the speed, but that would be huge
Changes by Ezio Melotti ezio.melo...@gmail.com:
--
nosy: +ezio.melotti
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1285086
___
___
Serhiy Storchaka added the comment:
Serhiy: One question. Is there a need to have -
+ append = res.append
And then use 'append'?
This speed up unquote_to_bytes by 15%.
--
___
Python tracker rep...@bugs.python.org
Maciej Fijalkowski added the comment:
As per discussion on python-dev, this bug should probably be reopened and the
patch maybe reverted as relying on the refcounting hack is both dodgy and hurts
other implementations, like PyPy.
--
nosy: +fijall
Changes by Serhiy Storchaka storch...@gmail.com:
--
resolution: fixed -
stage: committed/rejected - needs patch
status: closed - open
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1285086
Serhiy Storchaka added the comment:
Proposed patches not only get rid of the refcount hack, but make unquote() and
unquote_to_bytes() even significant faster for large strings.
--
nosy: +serhiy.storchaka
Added file: http://bugs.python.org/file29064/urllib_faster_unquote.patch
Changes by Serhiy Storchaka storch...@gmail.com:
--
stage: needs patch - patch review
versions: +Python 3.3, Python 3.4
Added file: http://bugs.python.org/file29065/urllib_faster_unquote-2.7.patch
___
Python tracker rep...@bugs.python.org
Applying this patch - I tried this hypothetical test.
$ ./python.exe -m timeit -s import urllib.parse; x='a%20' * 10
urllib.parse.unquote(x)
10 loops, best of 3: 205 msec per loop
Without the patch, the above test will run for minutes!
This creates a significant difference for extremely
Senthil Kumaran added the comment:
I wrongly minutes. Here is actual data of execution speeds. There is
magnitude of difference (almost 130x faster). Measured on macbook pro
with 2 cores and 4 Gig mem.
Before Patch:
$ ./python.exe -m timeit -s import urllib.parse; x='a%20' * 10
Barry A. Warsaw ba...@python.org added the comment:
Why was this merged to 2.6 after 2.6.6rc1 without approval?
--
nosy: +barry
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1285086
___
Changes by Barry A. Warsaw ba...@python.org:
--
priority: low - release blocker
status: closed - open
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1285086
___
Barry A. Warsaw ba...@python.org added the comment:
flox reverted in r83967 for 2.6.6.
--
priority: release blocker -
status: open - closed
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1285086
Florent Xicluna florent.xicl...@gmail.com added the comment:
Committed in r81265 for 2.7.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1285086
___
Changes by Florent Xicluna florent.xicl...@gmail.com:
Removed file: http://bugs.python.org/file17366/issue1285086_using_rstrip_v2.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1285086
___
Senthil Kumaran orsent...@gmail.com added the comment:
On Mon, May 17, 2010 at 01:38:57PM +, Florent Xicluna wrote:
Committed in r81265 for 2.7.
Thanks. That was interesting. Without resorting to any drastic changes like
use of regex, interesting speed-up seems to have been achieved by
Changes by Senthil Kumaran orsent...@gmail.com:
--
assignee: orsenthil - flox
resolution: - fixed
stage: patch review - committed/rejected
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1285086
Florent Xicluna florent.xicl...@gmail.com added the comment:
Committed to 3.2 in r81271, after some additional tuning.
Btw, I kept list comprehension in 3.2, because it is faster for small strings
(even if the gain is ~10%).
--
status: open - closed
Changes by Florent Xicluna florent.xicl...@gmail.com:
Removed file: http://bugs.python.org/file17291/issue1285086_fast_quote.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1285086
___
Changes by Florent Xicluna florent.xicl...@gmail.com:
Removed file: http://bugs.python.org/file17295/issue1285086_using_translate.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1285086
___
Changes by Florent Xicluna florent.xicl...@gmail.com:
Removed file: http://bugs.python.org/file17298/issue1285086_using_rstrip.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1285086
___
Changes by Florent Xicluna florent.xicl...@gmail.com:
Removed file: http://bugs.python.org/file17299/urllib_quote_speed_test.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1285086
___
Florent Xicluna florent.xicl...@gmail.com added the comment:
Updated script for benchmarks (on 2.x and 3.x).
Inspired by the Tools/iobench script.
It benchmarks various quote/unquote implementations on 2.x and 3.x.
On 2.7 the fastest implementation is something like:
def quote(s):
Changes by Florent Xicluna florent.xicl...@gmail.com:
Added file: http://bugs.python.org/file17366/issue1285086_using_rstrip_v2.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1285086
___
Florent Xicluna florent.xicl...@gmail.com added the comment:
Proposed patches for quote and unquote on 2.7 and 3.2.
--
Added file: http://bugs.python.org/file17367/issue1285086_using_rstrip_py3k.diff
___
Python tracker rep...@bugs.python.org
Florent Xicluna florent.xicl...@gmail.com added the comment:
I've tested a variant of the previous patch.
On my laptop, it gives good performance for simple cases, and the penalty for
real-quoting case is very low.
I've tested a short-circuit for the unquote() function too.
--
nosy:
Changes by Florent Xicluna florent.xicl...@gmail.com:
Removed file: http://bugs.python.org/file17290/issue1285086_fast_quote.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1285086
___
Changes by Florent Xicluna florent.xicl...@gmail.com:
Added file: http://bugs.python.org/file17291/issue1285086_fast_quote.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1285086
___
Senthil Kumaran orsent...@gmail.com added the comment:
Lets also see how this fares in py3k (where quote function takes an encoding )
and possibly push it in.
If there is any hesitation we can consult python-dev or wsgi groups where
frameworks developers might review and voice concerns, if
Florent Xicluna florent.xicl...@gmail.com added the comment:
New patch, using str.translate instead of regexp.
It is faster for normal cases (85% less time than stdlib quote), and the
penalty for the real-quoting case is less than 5%.
It should apply to 3.x with some adaptation.
--
Florent Xicluna florent.xicl...@gmail.com added the comment:
The speed test script did not work on 2.5 (because timeit.Timer does not accept
a callable).
Fixed version, which benchmarks the str.translate(...) version.
Change the '_new_quote_setup' assignment to test other variants.
--
Changes by Florent Xicluna florent.xicl...@gmail.com:
Removed file: http://bugs.python.org/file17297/urllib_quote_speed_test.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1285086
___
Florent Xicluna florent.xicl...@gmail.com added the comment:
actually, there's a simpler implementation, using s.rstrip(always_safe + safe).
It is as fast as the previous one.
--
Added file: http://bugs.python.org/file17298/issue1285086_using_rstrip.diff
Changes by Florent Xicluna florent.xicl...@gmail.com:
Added file: http://bugs.python.org/file17299/urllib_quote_speed_test.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1285086
___
Tres Seaver tsea...@agendaless.com added the comment:
I can only reiterate that Zope apps can call 'urllib.quote' dozens,
hundreds, even thousands of times on a single request: the
reason for the original bug report was that 'urllib.quote' was
showing up frequently on profiling output for such
Antoine Pitrou pit...@free.fr added the comment:
Tres, can you update your patch against SVN trunk?
Also, please do an unified diff, we are mostly used to this format.
--
stage: unit test needed - patch review
versions: +Python 2.7, Python 3.2
___
Tres Seaver tsea...@agendaless.com added the comment:
Updated patch against the trunk attached.
Note that I prefer unidiff myself, but was following a bit of guidance that
Guido (used to, anyway) prefer context diffs.
The updated speed test run against the stdlib with this patch shows only a
Tres Seaver tsea...@agendaless.com added the comment:
I'm uploading a saner version of the speed test which uses timeit's support for
passing a callable as a statement: it is much easier to
see what the test is actually doing.
On my machine, running against the release26-maint branch, my
Changes by Daniel Diniz aja...@gmail.com:
--
nosy: +orsenthil
stage: - test needed
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1285086
___
___
Antoine Pitrou pit...@free.fr added the comment:
If someone needs a faster quote(), it's probably easy to write an
insanely fast C version...
--
nosy: +pitrou
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1285086
Daniel Diniz [EMAIL PROTECTED] added the comment:
This is what I found doing some timings:
For the short-circuit path, the regexp can make quote 10x as fast in
exceptional cases, even comparing to the faster version in trunk. The
average win for short-circuit seems to be twice as fast as the
Changes by Georg Brandl [EMAIL PROTECTED]:
--
type: feature request - performance
_
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue1285086
_
___
Python-bugs-list
44 matches
Mail list logo
