[issue25863] ISO-2022 seeking forgets state
Changes by John Walker <publicjohnwal...@gmail.com>: -- nosy: +johnwalker ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25863> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25977] Typo fixes in Lib/tokenize.py
John Walker added the comment: You're welcome, have a happy new year. :) -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25977> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25977] Typo fixes in Lib/tokenize.py
New submission from John Walker: Minor fixes to comments and a docstring in Lib/tokenize.py I found while looking for more important bugs. afer -> after alternately -> alternatively intput -> input argment -> argument -- assignee: docs@python components: Documentation files: tokenize-fixes.patch keywords: patch messages: 257217 nosy: docs@python, johnwalker priority: normal severity: normal status: open title: Typo fixes in Lib/tokenize.py type: enhancement versions: Python 3.5, Python 3.6 Added file: http://bugs.python.org/file41449/tokenize-fixes.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25977> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25863] ISO-2022 seeking forgets state
John Walker added the comment: Here is Martin's message as a unit test. It checks utf-8 and the iso-2022 family except iso-2022-cn and iso-2022-cn-ext because they are not supported. The errors occur with all iso-2022 charsets. -- keywords: +patch Added file: http://bugs.python.org/file41451/25863-unittest.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25863> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25928] Improve performance of statistics._decimal_to_ratio and fractions.from_decimal
John Walker added the comment:
> I guess there's some version mixup here: From Python 3.3 on
> the integrated C version of decimal does not store the digits
> as a string and does not have the private _int method.
Stefan, _int is a slot in Lib/_pydecimal.py. It should be defined on python 3.5
and tip, unsure about other versions.
Python 3.5.1 (default, Dec 7 2015, 12:58:09)
[GCC 5.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from decimal import Decimal
>>> Decimal("100.00")
Decimal('100.00')
>>> Decimal("100.00")._int
'1'
--
versions: +Python 3.5
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25928>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25928] Improve performance of statistics._decimal_to_ratio and fractions.from_decimal
John Walker added the comment: > No, the regular build uses the libmpdec that is shipped with > Python. The external libmpdec.so only comes into play if you > compile --with-system-libmpdec. Oh, OK. I see whats happening. My distro deletes the shipped version and compiles --with-system-libmpdec. We're on the same page now, thanks. https://projects.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/python -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25928> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25928] Improve performance of statistics._decimal_to_ratio and fractions.from_decimal
John Walker added the comment: > That should only happen if the C version did not build for some reason: Ahh, gotcha. I assume one instance where this happens is when the machine doesn't have libmpdec.so -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25928> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25928] Improve performance of statistics._decimal_to_ratio and fractions.from_decimal
John Walker added the comment: Heres the output of running the benchmark on my machine: Testing proposed implementation number = 1 0.07098613299967838 number = 10 0.6952260910002224 number = 100 6.948197601999709 Testing current implementation number = 1 0.141816276996 number = 10 1.350394603001405 number = 100 13.625065807000283 -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25928> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25928] Improve performance of statistics._decimal_to_ratio and fractions.from_decimal
Changes by John Walker <publicjohnwal...@gmail.com>: -- type: -> performance ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25928> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25928] Improve performance of statistics._decimal_to_ratio and fractions.from_decimal
New submission from John Walker:
In statistics, there is a FIXME on Line 250 above _decimal_to_ratio that says:
# FIXME This is faster than Fraction.from_decimal, but still too slow.
Half of the time is spent in a conversion in d.as_tuple(). Decimal internally
stores the digits as a string, but in d.as_tuple(), the digits are individually
cast to integers and returned as a tuple of integers.
This is OK, but _decimal_to_ratio undoes the work that was done in d.as_tuple()
by adding them all back into an integer. A similar, but slightly different
approach is taken in Fractions.from_decimal, where the tuple is cast into a
string and then parsed into an integer. We can be a lot faster if we use the
_int instance variable directly.
In the case of _decimal_to_ratio, the new code seems to be twice as fast with
usage _decimal_to_ratio(Decimal(str(random.random(:
def _decimal_to_ratio(d):
sign, exp = d._sign, d._exp
if exp in ('F', 'n', 'N'): # INF, NAN, sNAN
assert not d.is_finite()
return (d, None)
num = int(d._int)
if exp < 0:
den = 10**-exp
else:
num *= 10**exp
den = 1
if sign:
num = -num
return (num, den)
If the performance improvement is considered worthwhile, here are a few
solutions I see.
1) Use _int directly in fractions and statistics.
2) Add a digits_as_str method to Decimal. This prevents exposing _int as an
implementation detail, and makes sense to me since I suspect there is a lot of
code casting the tuple of int to a string anyway.
3) Add a parameter to as_tuple for determining whether digits should be
returned as a string or a tuple.
4) Deprecate _int in Decimal and add a new reference str_digits.
There are probably more solutions. I lean towards 4, because it makes usage
easier and avoids cluttering Decimal with methods.
Here is what I used for benchmarks:
import timeit
old_setup = """
import random
from decimal import Decimal
def _decimal_to_ratio(d):
sign, digits, exp = d.as_tuple()
if exp in ('F', 'n', 'N'): # INF, NAN, sNAN
assert not d.is_finite()
return (d, None)
num = 0
for digit in digits:
num = num*10 + digit
if exp < 0:
den = 10**-exp
else:
num *= 10**exp
den = 1
if sign:
num = -num
return (num, den)
def run_it():
dec = Decimal(str(random.random()))
_decimal_to_ratio(dec)
"""
new_setup = """
import random
from decimal import Decimal
def _decimal_to_ratio(d):
sign, exp = d._sign, d._exp
if exp in ('F', 'n', 'N'): # INF, NAN, sNAN
assert not d.is_finite()
return (d, None)
num = int(d._int)
if exp < 0:
den = 10**-exp
else:
num *= 10**exp
den = 1
if sign:
num = -num
return (num, den)
def run_it():
dec = Decimal(str(random.random()))
_decimal_to_ratio(dec)
"""
if __name__ == '__main__':
print("Testing proposed implementation")
print("number = 1")
print(timeit.Timer(stmt='run_it()', setup=new_setup).timeit(number=1))
print("number = 10")
print(timeit.Timer(stmt='run_it()', setup=new_setup).timeit(number=10))
print("number = 100")
print(timeit.Timer(stmt='run_it()', setup=new_setup).timeit(number=100))
print("Testing old implementation")
print("number = 1")
print(timeit.Timer(stmt='run_it()', setup=old_setup).timeit(number=1))
print("number = 10")
print(timeit.Timer(stmt='run_it()', setup=old_setup).timeit(number=10))
print("number = 100")
print(timeit.Timer(stmt='run_it()', setup=old_setup).timeit(number=100))
--
components: Library (Lib)
messages: 256873
nosy: johnwalker
priority: normal
severity: normal
status: open
title: Improve performance of statistics._decimal_to_ratio and
fractions.from_decimal
versions: Python 3.5
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25928>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
