[issue2486] Decimal slowdown in 3.0 due to str/unicode changes
Mark Dickinson dicki...@gmail.com added the comment: (Adding Raymond to the nosy list.) In r68494, I've checked in some work towards rewriting Decimal (or parts of Decimal) in C. I reused Facundo's old /decimal directory in the sandbox, so the URL is: http://svn.python.org/view/sandbox/trunk/decimal/decimal_in_c/ What's there is a fully functional hybrid Python and C implementation of decimal. All the existing decimal tests pass, at least on my machine. There's very little performance increase to boast of at the moment, but that may change. To try it, check out the directory and follow the README instructions. This probably doesn't work on Windows; help fixing that would be appreciated. The code is for py3k only; I started out maintaining versions for both 2.x and 3.x, but that was just too painful. I'd rather get the py3k version working, then backport. The main files are: deccoeff.c: contains classes Deccoeff and _Decimal. Deccoeff implements base 10 nonnegative integer arithmetic: its instances can be treated as integers, but also sliced as though they were strings, which makes rounding and shifting operations simple. _Decimal is a minimal base class for Decimal; the idea is to move functions from Decimal to _Decimal over time. decimal.py: version of Lib/decimal.py, fixed to use Deccoeff instead of string for the coefficient. As I see it, there are two goals for this work: (1) Speed decimal up. (2) Bring proposals to add a decimal literal to C closer to realizability. By the way, presumably it's not a prerequisite for (2) that *all* of the decimal.py functionality be recoded in C? I could imagine a situation where the decimal literals are present in the core, but an 'import decimal' was necessary to get complete functionality. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2486 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2486] Decimal slowdown in 3.0 due to str/unicode changes
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +rhettinger ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2486 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2486] Decimal slowdown in 3.0 due to str/unicode changes
Mark Dickinson dicki...@gmail.com added the comment: Changing title and assignee. (But all help is appreciated: if anyone wants to commit changes to the sandbox directory, please go ahead!) ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2486 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2486] Decimal slowdown in 3.0 due to str/unicode changes
Mark Dickinson [EMAIL PROTECTED] added the comment: Just a quick note to say I am still working on this. I'll post some new code soon. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue2486 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2486] Decimal slowdown in 3.0 due to str/unicode changes
Mark Dickinson [EMAIL PROTECTED] added the comment: So I would suggest either a new directory in the sandbox, or re-using Facundo's original directory (which includes the telco benchmark) Makes sense---thanks for the suggestion! But since it seems I don't have any open ssh ports it looks like I'll just have to post a tarball for now. The module is written to work with Python 3.x, on the basis that it's probably easier to backport to 2.x later than to try to maintain two separate versions. On Unix, untar, edit the top line of the Makefile to point to a Python 3.0 executable, and do ./configure make make test runs the decimal test-suite. Added file: http://bugs.python.org/file11570/deccoeff.tar.gz ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue2486 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2486] Decimal slowdown in 3.0 due to str/unicode changes
Mark Dickinson [EMAIL PROTECTED] added the comment: Sorry for the silence: new country + new job + no internet connection outside work = not many opportunities to spend time on this (or any other part of Python) at the moment. I do now have a version 0.2 of the deccoeff type, and more importantly a version of decimal.py that's adapted to use it; all tests in test_decimal.py pass, which is a start. But there's significant work to be done tidying up the code, identifying speed-critical bits, and moving those bits from Python to C. I'm wondering where to post the code; I could post a tarball here but it's a bit unwieldy. Perhaps it would be worth creating a new branch to work on this? This definitely seems like a (=) two-person task; any help would be much appreciated! Thanks Facundo and Nick for comments so far. I also wonder whether there are alternative approaches that might do better. If we follow this approach, I'm doubtful about doing this for 3.0.1; even though the API is unchanged, this seems like too big a change for a bugfix release, with too great a risk of breakage. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue2486 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2486] Decimal slowdown in 3.0 due to str/unicode changes
Nick Coghlan [EMAIL PROTECTED] added the comment: This is the kind of project where the sandbox is useful - Facundo's original decimal work was done there, as was the attempt at a complete rewrite of the decimal module in C (which turned out to be a less than optimal approach to the speed problem). So I would suggest either a new directory in the sandbox, or re-using Facundo's original directory (which includes the telco benchmark) http://svn.python.org/view/sandbox/trunk/decimal And I agree that it is far more sensible to target 2.7/3.1 at this stage - the 3.0 slowdown, while real, actually isn't as bad as I expected, and even if it's large enough to be unacceptable to heavy users of Decimal, the only consequence is that they will have to hold off on migrating to 3.x for 12-18 months. Should we add something specific to the 3.0 release notes pointing out that there is approximately a 25% slowdown in the decimal module relative to 2.x? ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue2486 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2486] Decimal slowdown in 3.0 due to str/unicode changes
Facundo Batista [EMAIL PROTECTED] added the comment: Nick said: So I would suggest either a new directory in the sandbox, or re-using Facundo's original directory (which includes the telco benchmark) +1 And I agree that it is far more sensible to target 2.7/3.1 at this stage - the 3.0 slowdown, while real, actually isn't as bad as I expected, and even if it's large enough to be unacceptable to heavy users of Decimal, the only consequence is that they will I'm not seeing this job as a way to solve the 3.0 slowdown, but more as the way to slowly, but steadily, replace parts of Decimal from Python to C as needed. So, I'm +1 to target this to 3.1, and I'm -0 to register the slowdown in the releases notes (those who use Decimal aren't really in it for speed...). Thank you!! ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue2486 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2486] Decimal slowdown in 3.0 due to str/unicode changes
Nick Coghlan [EMAIL PROTECTED] added the comment: Looks like we'll be living with the slowdown for 3.0, so marking this as a high priority item for 3.1. (Given that the API doesn't change, I wonder if this could be included in a 3.0.1 release?) -- priority: - high versions: +Python 3.1 -Python 3.0 ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue2486 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2486] Decimal slowdown in 3.0 due to str/unicode changes
Facundo Batista [EMAIL PROTECTED] added the comment: Can not get into this now, but I'll be able to deal with this in some weeks... -- assignee: - facundobatista ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue2486 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2486] Decimal slowdown in 3.0 due to str/unicode changes
Facundo Batista [EMAIL PROTECTED] added the comment: I like this path, also. Beyond Deccoeff, this module could hold in the future other functions that are speed critical. Great work, Mark! __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2486 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2486] Decimal slowdown in 3.0 due to str/unicode changes
Nick Coghlan [EMAIL PROTECTED] added the comment: (I changed the issue title to better reflect where the discussion has moved to) I really like the approach of a custom internal type for the mantissa storage (the module containing would probably best be called _decimal). Then it should be possible to focus on letting C do what it does best (high speed arithmetic) and Python do what it does best (provide the nice API and context management features) -- title: Consider using bytes type instead of str to store Decimal coefficients - Decimal slowdown in 3.0 due to str/unicode changes __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2486 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
