[issue46558] Quadratic time internal base conversions
Carl Friedrich Bolz-Tereick added the comment: Somebody pointed me to V8's implementation of str(bigint) today: https://github.com/v8/v8/blob/main/src/bigint/tostring.cc They say that they can compute str(factorial(1_000_000)) (which is 5.5 million decimal digits) in 1.5s: https://twitter.com/JakobKummerow/status/1487872478076620800 As far as I understand the code (I suck at C++) they recursively split the bigint into halves using % 10^n at each recursion step, but pre-compute and cache the divisors' inverses. -- nosy: +Carl.Friedrich.Bolz ___ Python tracker <https://bugs.python.org/issue46558> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46555] Unicode-mangled names refer inconsistently to constants
Carl Friedrich Bolz-Tereick added the comment:
Ok, I can definitely agree with Serhiy pov: "True" is a keyword that always
evaluates to the object that you get when you call bool(1). There is usually no
name "True" and directly assigning to it is forbidden. But there are various
other ways to assign a name "True". One is eg globals("True") = 5, another one
(discussed in this issue) is using identifiers that NFKC-normalize to the
string "True".
--
___
Python tracker
<https://bugs.python.org/issue46555>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46555] Unicode-mangled names refer inconsistently to constants
Carl Friedrich Bolz-Tereick added the comment:
hah, this is "great":
>>> 핋핣핦핖 = 1
>>> globals()
{'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__':
, '__spec__': None,
'__annotations__': {}, '__builtins__': , 'True':
1}
The problem is that the lexer assumes that anything that is not ASCII cannot be
a keyword and lexes 핋핣핦핖 as an identifier.
--
nosy: +Carl.Friedrich.Bolz
___
Python tracker
<https://bugs.python.org/issue46555>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38085] Interrupting class creation in __init_subclass__ may lead to incorrect isinstance() and issubclass() results
Carl Friedrich Bolz-Tereick added the comment: Or, in other words, in my opinion this is the root cause of the bug: class Base: def __init_subclass__(cls): global broken_class broken_class = cls assert 0 try: class Broken(Base): pass except: pass assert broken_class not in Base.__subclasses__() The assert fails, which imo it shouldn't. -- ___ Python tracker <https://bugs.python.org/issue38085> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38085] Interrupting class creation in __init_subclass__ may lead to incorrect isinstance() and issubclass() results
Carl Friedrich Bolz-Tereick added the comment: hm, I think I figured it out. The root cause is that even though the creation of the class Triffid fails, it can still be found via Animal.__subclasses__(), which the special subclass logic for ABCs is looking at. Triffid fills its _abc_impl data with some content, but Triffid._abc_impl was never successfully initialized, therefore it mutates the _abc_impl of its first base class Animal. My conclusion would be that if a class is not successfully created, it shouldn't appear in the .__subclasses__() list of its bases. See attached script for some illuminating prints. -- nosy: +Carl.Friedrich.Bolz Added file: https://bugs.python.org/file50515/x.py ___ Python tracker <https://bugs.python.org/issue38085> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46042] Error range of "duplicate argument" SyntaxErrors is too big
Carl Friedrich Bolz-Tereick added the comment: Oh, don't worry, it's all good! It got fixed and I learned something. -- ___ Python tracker <https://bugs.python.org/issue46042> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46042] Error range of "duplicate argument" SyntaxErrors is too big
Carl Friedrich Bolz-Tereick added the comment: ah, confused, seems you fixed them both too. will take a closer look! -- ___ Python tracker <https://bugs.python.org/issue46042> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46042] Error range of "duplicate argument" SyntaxErrors is too big
Carl Friedrich Bolz-Tereick added the comment: Oh no, I was about to open mine ;-) https://github.com/python/cpython/compare/main...cfbolz:bpo-46042-syntax-error-range-duplicate-argument?expand=1 Basically equivalent, but I fixed the second bug too (would be very easy to add to yours) -- ___ Python tracker <https://bugs.python.org/issue46042> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46042] Error range of "duplicate argument" SyntaxErrors is too big
Carl Friedrich Bolz-Tereick added the comment:
let's see whether I promised too much, I don't know CPython's symtable.c too
well yet ;-). Will shout for help when I get stuck.
Anyway, here is a related bug, coming from the same symtable function
symtable_add_def_helper, also with an imprecise error location:
$ cat x.py
{i for i in range(5)
if (j := 0)
for j in range(5)}
$ ./python x.py
File "/home/cfbolz/projects/cpython/x.py", line 1
{i for i in range(5)
SyntaxError: comprehension inner loop cannot rebind assignment expression
target 'j'
--
___
Python tracker
<https://bugs.python.org/issue46042>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46042] Error range of "duplicate argument" SyntaxErrors is too big
New submission from Carl Friedrich Bolz-Tereick : The error range for the "duplicate argument in function definition" SyntaxError is too large: $ cat x.py def f(a, b, c, d, e, f, g, a): pass $ python x.py File "/home/cfbolz/projects/cpython/x.py", line 1 def f(a, b, c, d, e, f, g, a): pass ^^^ SyntaxError: duplicate argument 'a' in function definition I would expect only the second 'a' to be underlined. I can try to fix this. -- messages: 408248 nosy: Carl.Friedrich.Bolz, pablogsal priority: normal severity: normal status: open title: Error range of "duplicate argument" SyntaxErrors is too big ___ Python tracker <https://bugs.python.org/issue46042> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37971] Wrong trace with multiple decorators (linenumber wrong in frame)
Change by Carl Friedrich Bolz-Tereick : -- pull_requests: +28252 stage: -> patch review pull_request: https://github.com/python/cpython/pull/30027 ___ Python tracker <https://bugs.python.org/issue37971> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37971] Wrong trace with multiple decorators (linenumber wrong in frame)
Carl Friedrich Bolz-Tereick added the comment: I ran into this problem in PyPy today, preparing a patch for CPython too (without looking at the old one). -- nosy: +Carl.Friedrich.Bolz ___ Python tracker <https://bugs.python.org/issue37971> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45859] test_collections has a wrong test in case _itemgetter is not available
Change by Carl Friedrich Bolz-Tereick : -- keywords: +patch pull_requests: +27929 stage: -> patch review pull_request: https://github.com/python/cpython/pull/29691 ___ Python tracker <https://bugs.python.org/issue45859> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45859] test_collections has a wrong test in case _itemgetter is not available
New submission from Carl Friedrich Bolz-Tereick : test_field_descriptor in test_collections tries to pickle the descriptors of a namedtuple's fields, which is _collections._itemgetter on CPython. However, on PyPy that class doesn't exist. The code in collections deals fine with that fact, but the above-mentioned test does not make sense in that situation, since you can't pickle properties. To test this behaviour, you can replace "from _collections import _tuplegetter" in collections/__init__.py with raise ImportError and see the test fail on CPython too. -- messages: 406738 nosy: Carl.Friedrich.Bolz priority: normal severity: normal status: open title: test_collections has a wrong test in case _itemgetter is not available versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue45859> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45781] Deleting __debug__ should be an SyntaxError
Carl Friedrich Bolz-Tereick added the comment: ouch, apologies for not checking that! -- ___ Python tracker <https://bugs.python.org/issue45781> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45781] Deleting __debug__ should be an SyntaxError
New submission from Carl Friedrich Bolz-Tereick : Right now, deleting __debug__ is not prevented: >>> def f(): ... del __debug__ ... Of course actually executing it doesn't work: >>> del __debug__ Traceback (most recent call last): File "", line 1, in NameError: name '__debug__' is not defined Compare with assigning to __debug__: >>> def f(): ... __debug__ = 1 ... File "", line 2 SyntaxError: cannot assign to __debug__ -- messages: 406149 nosy: Carl.Friedrich.Bolz priority: normal severity: normal status: open title: Deleting __debug__ should be an SyntaxError ___ Python tracker <https://bugs.python.org/issue45781> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45764] Parse error improvement forgetting ( after def
Change by Carl Friedrich Bolz-Tereick : -- keywords: +patch pull_requests: +27735 stage: -> patch review pull_request: https://github.com/python/cpython/pull/29484 ___ Python tracker <https://bugs.python.org/issue45764> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45764] Parse error improvement forgetting ( after def
New submission from Carl Friedrich Bolz-Tereick :
Something I see beginners make occasionally when defining functions without
arguments is this:
def f:
...
Right now it just gives an "invalid syntax", would be nice to get an "expected
'('".
I will try to give this a go! Should be a matter of making the '(' token an
expected one.
--
messages: 406010
nosy: Carl.Friedrich.Bolz
priority: normal
severity: normal
status: open
title: Parse error improvement forgetting ( after def
___
Python tracker
<https://bugs.python.org/issue45764>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45727] Parse error when missing commas is inconsistent
New submission from Carl Friedrich Bolz-Tereick : I found following inconsistency in the error message when there's a missing comma (it behaves that way both on main and 3.10). Here's what happens with numbers, as expected: Python 3.11.0a1+ (heads/main:32f55d1a5d, Nov 5 2021, 13:18:52) [GCC 11.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> 1 2 3 4 File "", line 1 1 2 3 4 ^^^ SyntaxError: invalid syntax. Perhaps you forgot a comma? But with names the error is further right in the lines: >>> a b c d File "", line 1 a b c d ^^^ SyntaxError: invalid syntax. Perhaps you forgot a comma? >>> a b c d e f g File "", line 1 a b c d e f g ^^^ SyntaxError: invalid syntax. Perhaps you forgot a comma? That looks potentially quite confusing to me? (I don't know if these nit-picky parsing issues are too annoying, if they are please tell me to stop filing them). -- messages: 405792 nosy: Carl.Friedrich.Bolz, pablogsal priority: normal severity: normal status: open title: Parse error when missing commas is inconsistent versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue45727> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45716] Confusing parsing error message when trying to use True as keyword argument
New submission from Carl Friedrich Bolz-Tereick : A bit of a nitpick, but the following SyntaxError message is a bit confusing: >>> f(True=1) File "", line 1 f(True=1) ^ SyntaxError: expression cannot contain assignment, perhaps you meant "=="? The problem with that line is not that it contains an assignment, it's almost a valid keyword argument after all. The problem is that the name of the keyword is True, which is no longer a name you can assign to. It would be better to produce the same error as with __debug__: >>> f(__debug__=1) File "", line 1 SyntaxError: cannot assign to __debug__ The latter error message is however produced by the compiler, not the parser I think? -- messages: 405741 nosy: Carl.Friedrich.Bolz priority: normal severity: normal status: open title: Confusing parsing error message when trying to use True as keyword argument ___ Python tracker <https://bugs.python.org/issue45716> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45624] test_graphlib.py depends on iteration order of sets
Change by Carl Friedrich Bolz-Tereick : -- keywords: +patch pull_requests: +27496 stage: -> patch review pull_request: https://github.com/python/cpython/pull/29233 ___ Python tracker <https://bugs.python.org/issue45624> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45624] test_graphlib.py depends on iteration order of sets
Carl Friedrich Bolz-Tereick added the comment: here's the traceback running on pypy3.9-alpha: == FAIL: test_simple_cases (test.test_graphlib.TestTopologicalSort) -- Traceback (most recent call last): File "/home/cfbolz/projects/small-commits-pypy/lib-python/3/test/test_graphlib.py", line 39, in test_simple_cases [(3, 5, 7), (11, 8), (2, 10, 9)], File "/home/cfbolz/projects/small-commits-pypy/lib-python/3/test/test_graphlib.py", line 19, in _test_graph self.assertEqual(list(static_order_with_groups(ts)), list(expected)) AssertionError: Lists differ: [(3, 7, 5), (8, 11), (2, 9, 10)] != [(3, 5, 7), (11, 8), (2, 10, 9)] First differing element 0: (3, 7, 5) (3, 5, 7) - [(3, 7, 5), (8, 11), (2, 9, 10)] + [(3, 5, 7), (11, 8), (2, 10, 9)] -- ___ Python tracker <https://bugs.python.org/issue45624> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45624] test_graphlib.py depends on iteration order of sets
New submission from Carl Friedrich Bolz-Tereick : test_graphlib fails on PyPy because it depends on the iteration order of sets. Will open a PR soon. -- messages: 405084 nosy: Carl.Friedrich.Bolz priority: normal severity: normal status: open title: test_graphlib.py depends on iteration order of sets ___ Python tracker <https://bugs.python.org/issue45624> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30570] issubclass segfaults on objects with weird __getattr__
Carl Friedrich Bolz-Tereick added the comment: PyPy raises a RecursionError here, which sounds like an ok outcome. So simply checking for the recursion would also be a way of fixing this... -- nosy: +Carl.Friedrich.Bolz ___ Python tracker <https://bugs.python.org/issue30570> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45417] Enum creation non-linear in the number of values
Carl Friedrich Bolz-Tereick added the comment: I fixed the reliance of set being insertion ordered in pypy and opened a pull request. -- ___ Python tracker <https://bugs.python.org/issue45417> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45417] Enum creation non-linear in the number of values
Change by Carl Friedrich Bolz-Tereick : -- nosy: +Carl.Friedrich.Bolz nosy_count: 6.0 -> 7.0 pull_requests: +27198 stage: -> patch review pull_request: https://github.com/python/cpython/pull/28907 ___ Python tracker <https://bugs.python.org/issue45417> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34561] Replace list sorting merge_collapse()?
Carl Friedrich Bolz-Tereick added the comment: Thanks for your work Tim, just adapted the changes to PyPy's Timsort, using bits of runstack.py! -- nosy: +Carl.Friedrich.Bolz ___ Python tracker <https://bugs.python.org/issue34561> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25130] Make tests more PyPy compatible
Change by Carl Friedrich Bolz-Tereick : -- nosy: +Carl.Friedrich.Bolz nosy_count: 6.0 -> 7.0 pull_requests: +26460 pull_request: https://github.com/python/cpython/pull/28002 ___ Python tracker <https://bugs.python.org/issue25130> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43907] pickle.py bytearray memoization bug with protocol 5
Change by Carl Friedrich Bolz-Tereick : -- keywords: +patch pull_requests: +24221 stage: -> patch review pull_request: https://github.com/python/cpython/pull/25501 ___ Python tracker <https://bugs.python.org/issue43907> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43907] pickle.py bytearray memoization bug with protocol 5
New submission from Carl Friedrich Bolz-Tereick : The new codepath for the BYTEARRAY8 bytecode is missing memoization: >>> import pickletools, pickle >>> b = (bytearray(b"abc"), ) * 2 >>> b1, b2 = pickle.loads(pickle.dumps(b, 5)) # C version >>> b1 is b2 True (bytearray(b'abc'), bytearray(b'abc')) >>> b1, b2 = pickle.loads(pickle._dumps(b, 5)) # python version >>> b1 is b2 # :-( False Found it because PyPy is using pickle.py with no C implementation. I'm preparing a patch. -- messages: 391537 nosy: Carl.Friedrich.Bolz priority: normal severity: normal status: open title: pickle.py bytearray memoization bug with protocol 5 ___ Python tracker <https://bugs.python.org/issue43907> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43833] Unexpected Parsing of Numeric Literals Concatenated with Boolean Operators
Carl Friedrich Bolz-Tereick added the comment: @shreyanavigyan This is a bit off-topic, but it's called "short-circuiting", described here: https://docs.python.org/3/library/stdtypes.html#boolean-operations-and-or-not (or/and aren't really "operators", like +/- etc, they cannot be overridden, they evaluate their components lazily and are therefore almost control flow) -- ___ Python tracker <https://bugs.python.org/issue43833> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43833] Unexpected Parsing of Numeric Literals Concatenated with Boolean Operators
Carl Friedrich Bolz-Tereick added the comment: It's not just about keywords. Eg '1x' tokenizes too but then produces a syntax error in the parser. Keywords are only special in that they can be used to write syntactically meaningful things with these concatenated numbers. -- nosy: +Carl.Friedrich.Bolz ___ Python tracker <https://bugs.python.org/issue43833> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3451] Asymptotically faster divmod and str(long)
Carl Friedrich Bolz-Tereick added the comment: yes, that sounds fair. In PyPy we improve things occasionally if somebody feels like working on it, but in general competing against GMP is a fools errand. -- ___ Python tracker <https://bugs.python.org/issue3451> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3451] Asymptotically faster divmod and str(long)
Carl Friedrich Bolz-Tereick added the comment: FWIW, we have implemented a faster algorithm for divmod for big numbers using Mark's fast_div.py in PyPy. In particular, this speeds up str(long) for large numbers significantly (eg calling str on the result of math.factorial(2**17) is now 15x faster than CPython ;-)). -- nosy: +Carl.Friedrich.Bolz ___ Python tracker <https://bugs.python.org/issue3451> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43753] [C API] Add Py_Is(x, y) and Py_IsNone(x) functions
Carl Friedrich Bolz-Tereick added the comment: Just chiming in to say that for PyPy this API would be extremely useful, because PyPy's "is" is not implementable with a pointer comparison on the C level (due to unboxing we need to compare integers, floats, etc by value). Right now, C extension code that compares pointers is subtly broken and cannot be fixed by us. -- nosy: +Carl.Friedrich.Bolz ___ Python tracker <https://bugs.python.org/issue43753> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41972] bytes.find consistently hangs in a particular scenario
Carl Friedrich Bolz-Tereick added the comment: > BTW, this initialization in the FASTSEARCH code appears to me to be a > mistake: >skip = mlast - 1; Thanks for pointing that out Tim! Turns out PyPy had copied that mindlessly and I just fixed it. (I'm also generally following along with this issue, I plan to implement the two-way algorithm for PyPy as well, once you all have decided on a heuristic. We are occasionally in a slightly easier situation, because for constant-enough needles we can have the JIT do the pre-work on the needle during code generation.) -- nosy: +Carl.Friedrich.Bolz ___ Python tracker <https://bugs.python.org/issue41972> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43154] code.InteractiveConsole can crash if sys.excepthook is broken
New submission from Carl Friedrich Bolz-Tereick : When using code.InteractiveConsole to implement a Python shell (like PyPy is doing), having a broken sys.excepthook set can crash the console (see attached terminal log). Instead, it should catch errors and report then ignore them (using sys.unraisablehook I would think, but right now it's not that simple to call unraisablehook from python code). Related to https://bugs.python.org/issue43148 -- files: crash.log messages: 386593 nosy: Carl.Friedrich.Bolz priority: normal severity: normal status: open title: code.InteractiveConsole can crash if sys.excepthook is broken Added file: https://bugs.python.org/file49797/crash.log ___ Python tracker <https://bugs.python.org/issue43154> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43148] Call sys.unraisablehook in the REPL when sys.excepthook is broken
Change by Carl Friedrich Bolz-Tereick : -- nosy: +Carl.Friedrich.Bolz ___ Python tracker <https://bugs.python.org/issue43148> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38197] Meaning of tracebacklimit differs between sys.tracebacklimit and traceback module
Carl Friedrich Bolz-Tereick added the comment: It's still inconsistent between the two ways to get a traceback, and the inconsistency is not documented. -- ___ Python tracker <https://bugs.python.org/issue38197> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39486] bug in %-formatting in Python, related to escaped %-characters
Carl Friedrich Bolz-Tereick added the comment: Ok, that means it's intentional. I still think it's missing a documentation change and consistent error messages. -- ___ Python tracker <https://bugs.python.org/issue39486> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39486] bug in %-formatting in Python, related to escaped %-characters
New submission from Carl Friedrich Bolz-Tereick :
The following behaviour of %-formatting changed between Python3.6 and
Python3.7, and is in my opinion a bug that was introduced.
So far, it has been possible to add conversion flags to a conversion specifier
in %-formatting, even if the conversion is '%' (meaning a literal % is emitted
and no argument consumed).
Eg this works in Python3.6:
>>>> "%+%abc% %" % ()
'%abc%'
The conversion flags '+' and ' ' are ignored.
Was it discussed and documented anywhere that this is now an error? Because
Python3.7 has the following strange behaviour instead:
>>> "%+%abc% %" % ()
Traceback (most recent call last):
File "", line 1, in
TypeError: not enough arguments for format string
That error message is just confusing, because the amount of arguments is not
the problem here. If I pass a dict (thus making the number of arguments
irrelevant) I get instead:
>>> "%+%abc% %" % {}
Traceback (most recent call last):
File "", line 1, in
ValueError: unsupported format character '%' (0x25) at index 2
(also a confusing message, because '%' is a perfectly fine format character)
In my opinion this behaviour should either be reverted to how Python3.6 worked,
or the new restrictions should be documented and the error messages improved.
--
messages: 360965
nosy: Carl.Friedrich.Bolz
priority: normal
severity: normal
status: open
title: bug in %-formatting in Python, related to escaped %-characters
versions: Python 3.7
___
Python tracker
<https://bugs.python.org/issue39486>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39485] Bug in mock running on PyPy3
Change by Carl Friedrich Bolz-Tereick : -- keywords: +patch pull_requests: +17629 stage: -> patch review pull_request: https://github.com/python/cpython/pull/18252 ___ Python tracker <https://bugs.python.org/issue39485> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39485] Bug in mock running on PyPy3
New submission from Carl Friedrich Bolz-Tereick :
One of the new-in-3.8 tests for unittest.mock,
test_spec_has_descriptor_returning_function, is failing on PyPy. This exposes a
bug in unittest.mock. The bug is most noticeable on PyPy, where it can be
triggered by simply writing a slightly weird descriptor (CrazyDescriptor in the
test). Getting it to trigger on CPython would be possible too, by implementing
the same descriptor in C, but I did not actually do that.
The relevant part of the test looks like this:
from unittest.mock import create_autospec
class CrazyDescriptor(object):
def __get__(self, obj, type_):
if obj is None:
return lambda x: None
class MyClass(object):
some_attr = CrazyDescriptor()
mock = create_autospec(MyClass)
mock.some_attr(1)
On CPython this just works, on PyPy it fails with:
Traceback (most recent call last):
File "x.py", line 13, in
mock.some_attr(1)
File
"/home/cfbolz/bin/.pyenv/versions/pypy3.6-7.2.0/lib-python/3/unittest/mock.py",
line 938, in __call__
_mock_self._mock_check_sig(*args, **kwargs)
File
"/home/cfbolz/bin/.pyenv/versions/pypy3.6-7.2.0/lib-python/3/unittest/mock.py",
line 101, in checksig
sig.bind(*args, **kwargs)
File
"/home/cfbolz/bin/.pyenv/versions/pypy3.6-7.2.0/lib-python/3/inspect.py", line
3034, in bind
return args[0]._bind(args[1:], kwargs)
File
"/home/cfbolz/bin/.pyenv/versions/pypy3.6-7.2.0/lib-python/3/inspect.py", line
2955, in _bind
raise TypeError('too many positional arguments') from None
TypeError: too many positional arguments
The reason for this problem is that mock deduced that MyClass.some_attr is a
method on PyPy. Since mock thinks the lambda returned by the descriptor is a
method, it adds self as an argument, which leads to the TypeError.
Checking whether something is a method is done by _must_skip in mock.py. The
relevant condition is this one:
elif isinstance(getattr(result, '__get__', None), MethodWrapperTypes):
# Normal method => skip if looked up on type
# (if looked up on instance, self is already skipped)
return is_type
else:
return False
MethodWrapperTypes is defined as:
MethodWrapperTypes = (
type(ANY.__eq__.__get__),
)
which is just types.MethodType on PyPy, because there is no such thing as a
method wrapper (the builtin types look pretty much like python-defined types in
PyPy).
On PyPy the condition isinstance(getattr...) is thus True for all descriptors!
so as soon as result has a __get__, it counts as a method, even in the above
case where it's a custom descriptor.
Now even on CPython the condition makes no sense to me. It would be True for a
C-defined version of CrazyDescriptor, it's just not a good way to check whether
result is a method.
I would propose to replace the condition with the much more straightforward
check:
elif isinstance(result, FunctionTypes):
...
something is a method if it's a function on the class. Doing that change makes
the test pass on PyPy, and doesn't introduce any test failures on CPython
either.
Will open a pull request.
--
messages: 360961
nosy: Carl.Friedrich.Bolz, cjw296
priority: normal
severity: normal
status: open
title: Bug in mock running on PyPy3
___
Python tracker
<https://bugs.python.org/issue39485>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39220] constant folding affects annotations despite 'from __future__ import annotations'
Carl Friedrich Bolz-Tereick added the comment: I don't have a particularly deep opinion on what should be done, just a bit of weirdness I hit upon while implementing the PEP in PyPy. fwiw, we implement it as an AST transformer that the compiler runs before running the optimizer to make sure that the AST optimizations don't get applied to annotions. The transformer replaces all annotations with a Constant ast node, containing the unparsed string. -- ___ Python tracker <https://bugs.python.org/issue39220> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39220] constant folding affects annotations despite 'from __future__ import annotations'
New submission from Carl Friedrich Bolz-Tereick :
PEP 563 interacts in weird ways with constant folding. running the following
code:
```
from __future__ import annotations
def f(a: 5 + 7) -> a ** 39:
return 12
print(f.__annotations__)
```
I would expect this output:
```
{'a': '5 + 7', 'return': 'a ** 39'}
```
But I get:
```
{'a': '12', 'return': 'a ** 39'}
```
--
components: Interpreter Core
files: x.py
messages: 359341
nosy: Carl.Friedrich.Bolz
priority: normal
severity: normal
status: open
title: constant folding affects annotations despite 'from __future__ import
annotations'
versions: Python 3.7
Added file: https://bugs.python.org/file48827/x.py
___
Python tracker
<https://bugs.python.org/issue39220>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38197] Meaning of tracebacklimit differs between sys.tracebacklimit and traceback module
New submission from Carl Friedrich Bolz-Tereick : The meaning of sys.tracebacklimit seems to be different than the meaning of the various limit parameters in the traceback module. One shows the top n stack frames, the other the bottom n. Is this intentional, and if yes, is that difference documented somewhere? (it came up because PyPy just uses the traceback module and has no equivalent of PyTraceBack_Print). See the attached script to understand the problem. The script formats the same exception twice, once with the traceback module, once by the interpreter. I would have expected them to look the same for all limits, but instead: $ ./python /tmp/x.py 3 limit 3 from traceback module: Traceback (most recent call last): File "/tmp/x.py", line 19, in main() File "/tmp/x.py", line 16, in main x3() File "/tmp/x.py", line 14, in x3 x2() ZeroDivisionError: division by zero from interpreter: Traceback (most recent call last): File "/tmp/x.py", line 14, in x3 x2() File "/tmp/x.py", line 12, in x2 x1() File "/tmp/x.py", line 10, in x1 1 / 0 ZeroDivisionError: division by zero -- files: x.py messages: 352628 nosy: Carl.Friedrich.Bolz priority: normal severity: normal status: open title: Meaning of tracebacklimit differs between sys.tracebacklimit and traceback module versions: Python 3.7 Added file: https://bugs.python.org/file48610/x.py ___ Python tracker <https://bugs.python.org/issue38197> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18374] ast.parse gives wrong position (col_offset) for some BinOp-s
Change by Carl Friedrich Bolz-Tereick : -- pull_requests: +14466 pull_request: https://github.com/python/cpython/pull/14659 ___ Python tracker <https://bugs.python.org/issue18374> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18374] ast.parse gives wrong position (col_offset) for some BinOp-s
Change by Carl Friedrich Bolz-Tereick : -- pull_requests: +14423 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/14607 ___ Python tracker <https://bugs.python.org/issue18374> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18374] ast.parse gives wrong position (col_offset) for some BinOp-s
Carl Friedrich Bolz-Tereick added the comment: I think the attached patch fixes the problem. Will create a pull request soon. -- versions: +Python 3.5 -Python 3.7, Python 3.8, Python 3.9 Added file: https://bugs.python.org/file48458/binop-offset.patch ___ Python tracker <https://bugs.python.org/issue18374> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18374] ast.parse gives wrong position (col_offset) for some BinOp-s
Carl Friedrich Bolz added the comment: OK, fair enough. That means right now there is no way to find the position of the operator using the ast module at the moment, correct? -- ___ Python tracker <https://bugs.python.org/issue18374> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18374] ast.parse gives wrong position (col_offset) for some BinOp-s
Carl Friedrich Bolz added the comment: FWIW, in my opinion the col_offsets of the two nodes should be 1 and 3, respectively (the positions of the operators). -- nosy: +Carl.Friedrich.Bolz ___ Python tracker <https://bugs.python.org/issue18374> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
SPLASH-13 CfPs: OOPSLA | Onward! | DLS | Wavefront
ACM Conference on Systems, Programming, Languages, and Applications: Software for Humanity SPLASH-13 Indiana, Indianapolis October 26-31, 2013 http://www.splashcon.org http://twitter.com/splashcon http://www.facebook.com/SPLASHCon http://www.linkedin.com/groups/SPLASH-Conference-2487082 Sponsored by ACM SIGPLAN SPLASH is the ACM conference on Systems, Programming, Languages and Applications: Software for Humanity. SPLASH is an annual conference that embraces all aspects of software construction and delivery. This is the premier conference at the intersection of programming languages, programming, and software engineering. /** *COMBINED CALL FOR PAPERS * **/ * Overview http://splashcon.org/2013/cfp * OOPSLA research papers http://splashcon.org/2013/cfp/618 Abstracts due: March 22, 2013 Submissions due: March 28, 2013 * Onward! research papers http://onward-conference.org http://splashcon.org/2013/cfp/647 Abstracts due: March 29, 2013 Submissions due: April 5, 2013 * Onward! essays http://splashcon.org/2013/cfp/662 Submissions due: April 10, 2013 * Dynamic Languages Symposium (DLS) http://www.splashcon.org/2013/cfp/651 http://www.dynamic-languages-symposium.org/dls-13/cfp/ Submissions due: June 8, 2013 * Wavefront papers and experience reports http://splashcon.org/2013/cfp/656 Submissions due: April 5, 2013 /*/ -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
[issue13340] list.index does not accept None as start or stop
New submission from Carl Friedrich Bolz cfb...@gmx.de: The list.index method does not accept None as start and stop, which makes the error message quite confusing: [1, 2, 3].index(2, None, None) Traceback (most recent call last): File stdin, line 1, in module TypeError: slice indices must be integers or None or have an __index__ method I checked this in 3.2.2 and 2.7.2. Seems similar to #12163. -- messages: 147000 nosy: Carl.Friedrich.Bolz priority: normal severity: normal status: open title: list.index does not accept None as start or stop type: behavior versions: Python 2.7, Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13340 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12422] When deepcopying, don't store immutable objects in the memo dict
Changes by Carl Friedrich Bolz cfb...@gmx.de: -- nosy: +Carl.Friedrich.Bolz ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12422 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11477] Bug in code dispatching based on internal slots
Changes by Carl Friedrich Bolz cfb...@gmx.de: -- nosy: +cfbolz ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11477 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Standardizing RPython - it's time.
Hi John, John Nagle nagle at animats.com writes: All attempts to make the dialect defined by CPython significantly faster have failed. PyPy did not achieve much of a speed improvement over CPython, and is sometimes slower. This is not true. While PyPy is indeed sometimes slower than CPython, there is a significant speedup for many synthetic and real-life benchmarks. PyPy with a JIT is up to 20 times faster than CPython for synthetic benchmarks like richards, but still something like 6 times faster for Django. It also beats Psyco in almost all benchmarks. For detailed measurements, please visit our benchmarking site: http://speed.pypy.org All these numbers are also only the current state, PyPy is still continuously improving. Cheers, Carl Friedrich -- http://mail.python.org/mailman/listinfo/python-list
S3 2010 deadline extension
The S3 2010 Paper deadline was moved forward by two weeks, and is now August 13, 2010. *** Workshop on Self-sustaining Systems (S3) 2010 *** September 27-28, 2010 The University of Tokyo, Japan http://www.hpi.uni-potsdam.de/swa/s3/s3-10/ In cooperation with ACM SIGPLAN === Call for papers === The Workshop on Self-sustaining Systems (S3) is a forum for discussion of topics relating to computer systems and languages that are able to bootstrap, implement, modify, and maintain themselves. One property of these systems is that their implementation is based on small but powerful abstractions; examples include (amongst others) Squeak/Smalltalk, COLA, Klein/Self, PyPy/Python, Rubinius/Ruby, and Lisp. Such systems are the engines of their own replacement, giving researchers and developers great power to experiment with, and explore future directions from within, their own small language kernels. S3 will be take place September 27-28, 2010 at The University of Tokyo, Japan. It is an exciting opportunity for researchers and practitioners interested in self-sustaining systems to meet and share their knowledge, experience, and ideas for future research and development. --- Submissions and proceedings --- S3 invites submissions of high-quality papers reporting original research, or describing innovative contributions to, or experience with, self-sustaining systems, their implementation, and their application. Papers that depart significantly from established ideas and practices are particularly welcome. Submissions must not have been published previously and must not be under review for any another refereed event or publication. The program committee will evaluate each contributed paper based on its relevance, significance, clarity, and originality. Revised papers will be published as post-proceedings in the ACM Digital Library. Papers should be submitted electronically via EasyChair at http://www.easychair.org/conferences/?conf=s32010 in PDF format. Submissions must be written in English (the official language of the workshop) and must not exceed 10 pages. They should use the ACM SIGPLAN 10 point format, templates for which are available at http://www.acm.org/sigs/sigplan/authorInformation.htm. --- Venue --- The University of Tokyo, Komaba Campus, Japan --- Important dates --- Submission of papers: *EXTENDED* August 13, 2010 Author notification: August 27, 2010 Early registration: September 3, 2010 Revised papers: September 10, 2010 S3 workshop: September 27-28, 2010 Final papers for ACM-DL post-proceedings: October 15, 2010 --- Invited talks --- Yukihiro Matsumoto: From Lisp to Ruby to Rubinius Takashi Ikegami: Sustainable Autonomy and Designing Mind Time --- Chairs --- Robert Hirschfeld (Hasso-Plattner-Institut Potsdam, Germany) hirschf...@hpi.uni-potsdam.de Hidehiko Masuhara (The University of Tokyo, Japan) masuh...@graco.c.u-tokyo.ac.jp Kim Rose (Viewpoints Research Institute, USA) kim.r...@vpri.org --- Program committee --- Carl Friedrich Bolz, University of Duesseldorf, Germany Johan Brichau, Universite Catholique de Louvain, Belgium Shigeru Chiba, Tokyo Institute of Technology, Japan Brian Demsky, University of California, Irvine, USA Marcus Denker, INRIA Lille, France Richard P. Gabriel, IBM Research, USA Michael Haupt, Hasso-Plattner-Institut, Germany Robert Hirschfeld, Hasso-Plattner-Institut, Germany (co-chair) Atsushi Igarashi, University of Kyoto, Japan David Lorenz, The Open University, Israel Hidehiko Masuhara, University of Tokyo, Japan (co-chair) Eliot Miranda, Teleplace, USA Ian Piumarta, Viewpoints Research Institute, USA Martin Rinard, MIT, USA Antero Taivalsaari, Nokia, Finland David Ungar, IBM, USA ___ fonc mailing list f...@vpri.org http://vpri.org/mailman/listinfo/fonc -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Call for Papers: Workshop on Self-sustaining Systems (S3) 2010
*** Workshop on Self-sustaining Systems (S3) 2010 *** September 27-28, 2010 The University of Tokyo, Japan http://www.hpi.uni-potsdam.de/swa/s3/s3-10/ In cooperation with ACM SIGPLAN === Call for papers === The Workshop on Self-sustaining Systems (S3) is a forum for discussion of topics relating to computer systems and languages that are able to bootstrap, implement, modify, and maintain themselves. One property of these systems is that their implementation is based on small but powerful abstractions; examples include (amongst others) Squeak/Smalltalk, COLA, Klein/Self, PyPy/Python, Rubinius/Ruby, and Lisp. Such systems are the engines of their own replacement, giving researchers and developers great power to experiment with, and explore future directions from within, their own small language kernels. S3 will be take place September 27-28, 2010 at The University of Tokyo, Japan. It is an exciting opportunity for researchers and practitioners interested in self-sustaining systems to meet and share their knowledge, experience, and ideas for future research and development. --- Submissions and proceedings --- S3 invites submissions of high-quality papers reporting original research, or describing innovative contributions to, or experience with, self-sustaining systems, their implementation, and their application. Papers that depart significantly from established ideas and practices are particularly welcome. Submissions must not have been published previously and must not be under review for any another refereed event or publication. The program committee will evaluate each contributed paper based on its relevance, significance, clarity, and originality. Revised papers will be published as post-proceedings in the ACM Digital Library. Papers should be submitted electronically via EasyChair at http://www.easychair.org/conferences/?conf=s32010 in PDF format. Submissions must be written in English (the official language of the workshop) and must not exceed 10 pages. They should use the ACM SIGPLAN 10 point format, templates for which are available at http://www.acm.org/sigs/sigplan/authorInformation.htm. --- Venue --- The University of Tokyo, Komaba Campus, Japan --- Important dates --- Submission of papers: July 30, 2010 Author notification: August 27, 2010 Early registration: September 3, 2010 Revised papers: September 10, 2010 S3 workshop: September 27-28, 2010 Final papers for ACM-DL post-proceedings: October 15, 2010 --- Chairs --- Robert Hirschfeld (Hasso-Plattner-Institut Potsdam, Germany) hirschf...@hpi.uni-potsdam.de Hidehiko Masuhara (The University of Tokyo, Japan) masuh...@graco.c.u-tokyo.ac.jp Kim Rose (Viewpoints Research Institute, USA) kim.r...@vpri.org --- Program committee --- Carl Friedrich Bolz, University of Duesseldorf, Germany Johan Brichau, Universite Catholique de Louvain, Belgium Shigeru Chiba, Tokyo Institute of Technology, Japan Brian Demsky, University of California, Irvine, USA Marcus Denker, INRIA Lille, France Richard P. Gabriel, IBM Research, USA Michael Haupt, Hasso-Plattner-Institut, Germany Robert Hirschfeld, Hasso-Plattner-Institut, Germany (co-chair) Atsushi Igarashi, University of Kyoto, Japan David Lorenz, The Open University, Israel Hidehiko Masuhara, University of Tokyo, Japan (co-chair) Eliot Miranda, Teleplace, USA Ian Piumarta, Viewpoints Research Institute, USA Martin Rinard, MIT, USA Antero Taivalsaari, Nokia, Finland David Ungar, IBM, USA -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
[issue5377] Strange behavior when performing int on a Decimal made from -sys.maxint-1
Carl Friedrich Bolz cfb...@gmx.de added the comment: PyPy is a bit of a special case, because it cares about the distinction of int and long in the translation toolchain. Nevertheless, this behavior has been annoying to us. -- nosy: +cfbolz ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5377 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5377] Strange behavior when performing int on a Decimal made from -sys.maxint-1
Carl Friedrich Bolz cfb...@gmx.de added the comment: [...] Would the bool/int distinction matter to PyPy? No, it's really mostly about longs and ints, because RPython does not have automatic overflowing of ints to longs (the goal is really to translate ints them to C longs with normal C overflow behaviour). I would understand if you decide for wontfix, because you are not supposed to care about int/long and as I said, PyPy is a special case. Thanks, Carl Friedrich -- title: Strange behavior when performing int on a Decimal made from -sys.maxint-1 - Strange behavior when performing int on a Decimal made from -sys.maxint-1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5377 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7019] unmarshaling of artificial strings can produce funny longs.
Carl Friedrich Bolz cfb...@gmx.de added the comment: [...] How did you encounter this in the first place? I was working on PyPy's marshaler, broke it in such a way that it was producing the bad input that I gave above. Then I fixed it, but didn't kill my .pyc files. So a few days later I got a long zero with the properties above in PyPy (which was quite annoying to debug). Then I figured that CPython likely has the same problem. Thanks a lot for fixing this! Carl Friedrich -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7019 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
PyPy Europython Sprint Announcement
== Birmingham (UK) EuroPython PyPy Sprints 28-29 June/ 3-4 July 2009 == The PyPy team is sprinting at EuroPython again. This year there are `sprint days`_ before (28-29 June) and after (3-4 July) the conference. Some PyPy core people should be present during both periods. .. _`sprint days`: http://wiki.europython.eu/Sprints If you plan to attend the sprints after the conference we recommend you to listen to the PyPy technical talk (`EuroPython schedule`_) during the conference since it will give you a good overview of the status of development. Goals and topics of the sprint -- There are many possible and interesting sprint topics to work on - here we list some possible task areas: - trying out software on PyPy's Python interpreter: the CPython test suite is not all that complete, therefore the fact that we pass most tests is no real indication of bug-freeness. We have tried and know that frameworks like Django and Twisted work with PyPy. Therefore we would like to try running more real applications on top of the Python interpreter (ideally ones that have a good test suite themselves and that don't need unusual extension modules). Running things on Windows is also interesting, we know our coverage there is not as good as on Linux. - check and improve Mac OS X support - starting to work on porting 2.6 features to PyPy's Python interpreter - ongoing JIT generator work - of course we are open to other ideas for what to work on. Examples could be working on other language interpreters, sandboxing, ... Registration If you'd like to come, please subscribe to the `pypy-sprint mailing list`_ and drop a note about your interests and post any questions. More organisational information will be sent to that list. Please register by adding yourself on the following list (via svn): http://codespeak.net/svn/pypy/extradoc/sprintinfo/ep2009/people.txt or on the pypy-sprint mailing list if you do not yet have check-in rights: http://codespeak.net/mailman/listinfo/pypy-sprint --- Preparation (if you feel it is needed): --- - read the `getting-started`_ pages on http://codespeak.net/pypy, especially also the `development of PyPy itself part`_ . - for inspiration, overview and technical status you are welcome to read `the technical reports available and other relevant documentation`_ - please direct any technical and/or development oriented questions to pypy-dev at codespeak.net and any sprint organizing/logistical questions to pypy-sprint at codespeak.net - if you need information about the conference, potential hotels, directions etc we recommend to look at http://www.europython.eu. We are looking forward to meet you at the EuroPython PyPy sprints! The PyPy team .. See also .. .. _getting-started: http://codespeak.net/pypy/dist/pypy/doc/getting-started.html .. _`development of PyPy itself part`: http://codespeak.net/pypy/dist/pypy/doc/getting-started-dev.html .. _`pypy-sprint mailing list`: http://codespeak.net/mailman/listinfo/pypy-sprint .. _`the technical reports available and other relevant documentation`: http://codespeak.net/pypy/dist/pypy/doc/docindex.html .. _`EuroPython schedule`: http://europython.eu/talks/timetable -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
PyPy Europython Sprint Announcement
== Birmingham (UK) EuroPython PyPy Sprints 28-29 June/ 3-4 July 2009 == The PyPy team is sprinting at EuroPython again. This year there are `sprint days`_ before (28-29 June) and after (3-4 July) the conference. Some PyPy core people should be present during both periods. .. _`sprint days`: http://wiki.europython.eu/Sprints If you plan to attend the sprints after the conference we recommend you to listen to the PyPy technical talk (`EuroPython schedule`_) during the conference since it will give you a good overview of the status of development. Goals and topics of the sprint -- There are many possible and interesting sprint topics to work on - here we list some possible task areas: - trying out software on PyPy's Python interpreter: the CPython test suite is not all that complete, therefore the fact that we pass most tests is no real indication of bug-freeness. We have tried and know that frameworks like Django and Twisted work with PyPy. Therefore we would like to try running more real applications on top of the Python interpreter (ideally ones that have a good test suite themselves and that don't need unusual extension modules). Running things on Windows is also interesting, we know our coverage there is not as good as on Linux. - check and improve Mac OS X support - starting to work on porting 2.6 features to PyPy's Python interpreter - ongoing JIT generator work - of course we are open to other ideas for what to work on. Examples could be working on other language interpreters, sandboxing, ... Registration If you'd like to come, please subscribe to the `pypy-sprint mailing list`_ and drop a note about your interests and post any questions. More organisational information will be sent to that list. Please register by adding yourself on the following list (via svn): http://codespeak.net/svn/pypy/extradoc/sprintinfo/ep2009/people.txt or on the pypy-sprint mailing list if you do not yet have check-in rights: http://codespeak.net/mailman/listinfo/pypy-sprint --- Preparation (if you feel it is needed): --- - read the `getting-started`_ pages on http://codespeak.net/pypy, especially also the `development of PyPy itself part`_ . - for inspiration, overview and technical status you are welcome to read `the technical reports available and other relevant documentation`_ - please direct any technical and/or development oriented questions to pypy-dev at codespeak.net and any sprint organizing/logistical questions to pypy-sprint at codespeak.net - if you need information about the conference, potential hotels, directions etc we recommend to look at http://www.europython.eu. We are looking forward to meet you at the EuroPython PyPy sprints! The PyPy team .. See also .. .. _getting-started: http://codespeak.net/pypy/dist/pypy/doc/getting-started.html .. _`development of PyPy itself part`: http://codespeak.net/pypy/dist/pypy/doc/getting-started-dev.html .. _`pypy-sprint mailing list`: http://codespeak.net/mailman/listinfo/pypy-sprint .. _`the technical reports available and other relevant documentation`: http://codespeak.net/pypy/dist/pypy/doc/docindex.html .. _`EuroPython schedule`: http://europython.eu/talks/timetable -- http://mail.python.org/mailman/listinfo/python-list
PyPy 1.1 final released!
. For our main target C, we can mix in different garbage collectors and threading models, including micro-threads aka Stackless. The inherent complexity that arises from this ambitious approach is mostly kept away from the Python interpreter implementation, our main frontend. Socially, PyPy is a collaborative effort of many individuals working together in a distributed and sprint-driven way since 2003. PyPy would not have gotten as far as it has without the coding, feedback and general support from numerous people. Have fun, the PyPy release team, [in alphabetical order] Amaury Forgeot d'Arc, Anders Hammerquist, Antonio Cuni, Armin Rigo, Carl Friedrich Bolz, Christian Tismer, Holger Krekel, Maciek Fijalkowski, Samuele Pedroni and many others: http://codespeak.net/pypy/dist/pypy/doc/contributor.html -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
[issue5791] title information of unicodedata is wrong in some cases
New submission from Carl Friedrich Bolz cfb...@gmx.de: There seems to be a problem with some unicode character's title information: $ python2.6 Python 2.6.2c1 (release26-maint, Apr 14 2009, 08:02:48) [GCC 4.3.3] on linux2 Type help, copyright, credits or license for more information. unichr(453) u'\u01c5' unichr(453).title() u'\u01c4' But the title should return the same character, according to this: http://www.fileformat.info/info/unicode/char/01c5/index.htm (I also checked the files that unicode.org provides). I tried to follow the problem a bit, it seems to come from _PyUnicode_ToTitlecase in unicodetype.c. The unicode record contains the offset of the character to its titled version. If the character is its own titled version, then the offset is zero. But zero is also used for when there is no information available, so the offset to the upper-case version of the character is used. If this is a different character (as for the example above), the result of .title() is wrong. -- components: Interpreter Core messages: 86163 nosy: cfbolz severity: normal status: open title: title information of unicodedata is wrong in some cases type: behavior versions: Python 2.4, Python 2.5, Python 2.6, Python 3.0 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5791 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4069] set.remove raises confusing KeyError
New submission from Carl Friedrich Bolz [EMAIL PROTECTED]: When trying to remove a set from a set, the KeyError that is raised is confusing: Python 2.6 (r26:66714, Oct 7 2008, 13:23:57) [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2 Type help, copyright, credits or license for more information. s = set([frozenset([1, 2]), frozenset([2, 3])]) s.remove(set([3, 4])) Traceback (most recent call last): File stdin, line 1, in module KeyError: frozenset([]) I understand that s.remove(set(...)) turns the set into a frozenset, but I was expecting this converted frozenset to be attached to the KeyError, not an empty set. -- components: Interpreter Core messages: 74461 nosy: cfbolz severity: normal status: open title: set.remove raises confusing KeyError type: behavior versions: Python 2.5, Python 2.6 ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue4069 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: How solid is PyPy?
Hi, [EMAIL PROTECTED] wrote: I've been looking at PyPy recently, and I see it's reached version 1.0 (and supports language version 2.4). Given that, I was wondering what level of backwards-compatibility one can expect from future versions, i.e. if I run code on, say, a translated stackless PyPy now, what is the probability that it will run unmodified on PyPy 1.x, 2.x, etc.? That's hard to say. Nobody is using PyPy in any sort of production system yet, so the PyPy team is currently focused mostly on actually becoming useful. Nobody really has a clue what sort of solutions for backward compatibility we will have. Note also that the 1.0 release is sort of oldish already (but no new release has been made). Most people just use SVN head, which is quite stable. Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
PyPy Berlin Sprint, 17th - 22nd May 2008
= PyPy Berlin Sprint (17-22nd May 2008) = The next PyPy sprint will be in the crashed `c-base space station`_, Berlin, Germany, Earth, Solar System. This is a fully public sprint: newcomers (from all planets) and topics other than those proposed below are welcome. .. _`c-base space station`: http://www.c-base.org/ -- Goals and topics of the sprint -- - work on PyPy's JIT generator: we are refactoring parts of the compiling logic, in ways that may also allow generating better machine code for loops (people or aliens with knowledge on compilers and SSA, welcome) - work on the SPy VM, PyPy's Squeak implementation, particularly the graphics capabilities - work on PyPy's GameBoy emulator, which also needs graphics support - trying some large pure-Python applications or libraries on PyPy and fixing the resulting bugs. Possibilities are Zope 3, Django and others. * We are open to all sorts of other tasks during the sprint, just propose something. --- Location Accomodation --- The sprint will take place in the c-base in Berlin. The address is:: c-base e.V. Rungestrasse 20 10179 Berlin To get there, take the U-Bahn or S-Bahn to the station Jannowitzbrücke. See here_ for a map to get to c-base from there. .. _here: http://maps.google.com/maps?f=qhl=engeocode=q=c-base+berlinie=UTF8ll=52.515464,13.408985spn=0.020449,0.057335z=15iwloc=A If you want to come, please register via by svn: http://codespeak.net/svn/pypy/extradoc/sprintinfo/berlin-2008/people.txt or write a mail to the pypy-sprint mailing list if you do not yet have check-in rights: http://codespeak.net/mailman/listinfo/pypy-sprint Of course, especially given the location, it's ok to show up even if you didn't register. (The c-base has probably got the plans of Guido's famous time machine anyway, so you can register last week.) --- Exact times --- The sprint will be from 17th to 22nd of May 2008. We will start sprinting every day at 10. An introduction will be given on the first day, if there is interest. ___ pypy-sprint mailing list [EMAIL PROTECTED] http://codespeak.net/mailman/listinfo/pypy-sprint -- http://mail.python.org/mailman/listinfo/python-list
ANN: Py-Lib 0.9.1 released
py lib 0.9.1: bugfix release = The py lib team has just released version 0.9.1 of the py lib - a library aiming to support agile and test-driven python development on various levels. This is mostly a bugfix release, with a couple of new features sneaked in. Most important changes: * reduced the number of threads used in py.execnet * some new functionality (authentication, export, locking) in py.path's Subversion APIs * stability and segfault fixes in execnet * numerous small fixes in py.test's rsession (experimental pluggable session) and generative test features * some fixes in the py.test core * added py.misc.killproc, which allows killing processes on (some flavours of) Windows and UNIX For a complete list of changes, see doc/changes-0.9.1.txt in the source package. Download/Install: http://codespeak.net/py/0.9.1/download.html Documentation/API: http://codespeak.net/py/0.9.1/index.html Work on the py lib has been partially funded by the European Union IST programme and by http://merlinux.de within the PyPy project. best, have fun and let us know what you think! holger krekel, Maciej Fijalkowski, Carl Friedrich Bolz, Guido Wesdorp -- http://mail.python.org/mailman/listinfo/python-list
Re: Is crawling the stack bad? Why?
Paul Rubin wrote: Russell Warren [EMAIL PROTECTED] writes: That is exactly where I started (creating my own request handler, snagging the IP address and stashing it), but I couldn't come up with a stash location that would work for a threaded server. How about a dictionary indexed by by the thread name. It's pretty lame, though, that the rpc server module itself doesn't make the request available to the rpc responder. Maybe you should submit a patch. My biggest specific fear at the moment is that sys._frame will do funky things with multiple threads, You should not rely on anything that implementation specific at all. What happens if you want to switch to pypy? Apart from the fact that the idea of walking the stack to get info is indeed rather crazy, PyPy supports sys._getframe and friends perfectly fine (I think even Jython does, but I am not quite sure). In general PyPy tries to implement all these internals of CPython as closely as it is sane to do so. Stuff like inspecting code, function, frame, method objects is very closely mirrored but of course small differences exist: [EMAIL PROTECTED]:~$ python Python 2.5.1 (r251:54863, Oct 5 2007, 13:36:32) [GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2 Type help, copyright, credits or license for more information. dir(dir) ['__call__', '__class__', '__cmp__', '__delattr__', '__doc__', '__getattribute__', '__hash__', '__init__', '__module__', '__name__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__self__', '__setattr__', '__str__'] [EMAIL PROTECTED]:~$ pypy-c-46371-faassen Python 2.4.1 (pypy 1.0.0 build 46371) on linux2 Type help, copyright, credits or license for more information. dir(dir) ['__call__', '__class__', '__delattr__', '__dict__', '__doc__', '__getattribute__', '__hash__', '__init__', '__module__', '__name__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__self__', '__setattr__', '__setstate__', '__str__', '__weakref__', 'func_closure', 'func_code', 'func_defaults', 'func_dict', 'func_doc', 'func_globals', 'func_name'] dir.func_code internal-code object at 0x0826e460 dir.func_name 'dir' Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
Re: Current Fastest Python Implementation?
[EMAIL PROTECTED] wrote: On Feb 18, 9:37 am, Stefan Behnel [EMAIL PROTECTED] wrote: samuraisam wrote: Has anyone done any recent testing as to which current python implementation is the quickest? Search for a recent thread on CPython and IronPython. Perhaps for Django development - though the current one certainly is fast (and I doubt micro optimizations would make much difference in overall performance). Regardless - have those pypy folks made a faster implementation, or the jython folks? Or the .NET implementation? Depends on your use case. Take your application, do some benchmarking and use the implementation that turns out to be a) most reliable and b) the fastest. In that order. That's very good advice. Basically all four of those Python implementations have situations where they are faster than all the others. I guess CPython (possibly using Psyco) is still faster in many cases, but it really depends on your application. PyPy [http://codespeak.net/pypy/dist/pypy/doc/home.html] is getting progressively faster. This is true – PyPy is slowly getting faster. We have two students working explicitly on speed right now: Anto Cuni is doing a phd thesis on speeding up PyPy's Python interpreter when compiled to .NET and I am doing a Master thesis on improving the JIT of PyPy. In fact for certain things it can be faster than C [http:// morepypy.blogspot.com/2008/01/rpython-can-be-faster-than-c.html]! This link is misleading, since it is about the speed of RPython when translated to C, not normal Python programs. Normal Python programs tend to be not RPython, so that figure is hardly interesting. However it seems it still has a way to go to be fully operational! Still looks like the future to me. Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
Re: Why not a Python compiler?
Reedick, Andrew wrote: -Original Message- From: [EMAIL PROTECTED] [mailto:python- [EMAIL PROTECTED] On Behalf Of Luis M. González Sent: Tuesday, February 05, 2008 6:44 PM To: python-list@python.org Subject: Re: Why not a Python compiler? Pypy is a very ambitious project and it aims, amongst many other goals, to provide a fast just-in-time python implementation. They even say that the secret goal is being faster than c, which is nonsense, isn´t it? (I still didn´t get the joke though...). 'c' is also the speed of light. And since nothing can travel faster than light... nice theory, but wrong: The PyPy home page uses a capital letter C: http://codespeak.net/pypy/dist/pypy/doc/home.html Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
Re: Why not a Python compiler?
Hi Luis, Luis M. Gonzalez wrote: Well, lets suppose that being faster than C is the real goal... How about we call it a very long-term dream? Are you confident that it will be reached? We have ideas how to get there, but it is really rather long-term. There will be a lot of research needed to achieve that for sure. How far is it at this moment? Since a year already we have a JIT that makes quite fast code for purely integer-type operations. Quite fast meaning about the speed of gcc -O0. So far it is not faster than Psyco (it supports generators, though). We are confident that the JIT will get substantially better this year (probably better than Psyco). Where exactly this will leave us we don't know. I've been following the project but the scarcity of news is getting me anxious... Read the blog: morepypy.blogspot.com :-) Cheers, Carl Friedrich -- http://mail.python.org/mailman/listinfo/python-list
PyPy-Sprint 12th-19th January Leysin, Switzerland
= PyPy Leysin Winter Sprint (12-19th January 2008) = .. image:: http://www.ermina.ch/002.JPG The next PyPy sprint will be in Leysin, Switzerland, for the fifth time. This is a fully public sprint: newcomers and topics other than those proposed below are welcome. -- Goals and topics of the sprint -- * Like previous winters, the main side goal is to have fun in winter sports :-) We can take a couple of days off for ski; at this time of year, ski days end before 4pm, which still leaves plenty of time to recover (er, I mean hack). * the overall idea of the sprint is to continue working on making PyPy ready for general use. A few more specific tasks: - app-level ctypes: getting to a basically usable point would be really nice. - JIT: there is a long-standing timeshifter refactoring, towards making the JIT be more interpreter-like. Starting it during the sprint might be a good way to share some of the knowledge of how the JIT really works. Alternatively, we can work on supporting ootype in the timeshifter. - Testing: e.g. we run various nightly test runs but the results are not summarized in a single page yet. - LLVM: llvm 2 is now at version 2.1 and nicely stable again. Our llvm backend has improved in the last few months, but refactoring it together with the genc backend to share code more directly would be a nice task. * We are open to all sorts of other tasks during the sprint, just propose something. --- Location Accomodation --- Leysin, Switzerland, same place as before. Let me refresh your memory: both the sprint venue and the lodging will be in a very spacious pair of chalets built specifically for bed breakfast: http://www.ermina.ch/. The place has a good ADSL Internet connexion with wireless installed. You can of course arrange your own lodging anywhere (so long as you are in Leysin, you cannot be more than a 15 minute walk away from the sprint venue), but I definitely recommend lodging there too -- you won't find a better view anywhere else (though you probably won't get much worse ones easily, either :-) I made pre-reservations in the Chalet, so please *confirm* quickly that you are coming so that we can adjust the reservations as appropriate. The rate so far has been around 60 CHF a night all included in 2-person rooms, with breakfast. There are larger rooms too (less expensive) and maybe the possibility to get a single room if you really want to. Please register by svn: http://codespeak.net/svn/pypy/extradoc/sprintinfo/leysin-winter-2008/people.txt or on the pypy-sprint mailing list if you do not yet have check-in rights: http://codespeak.net/mailman/listinfo/pypy-sprint You need a Swiss-to-(insert country here) power adapter. There will be some Swiss-to-EU adapters around - bring a EU-format power strip if you have one. --- Exact times --- Officially, 12th-19th January 2008. Both dates are flexible, you can arrive or leave earlier or later. We will give introductions and tutorials depending on who needs them, either on the 13th or the 14th. -- http://mail.python.org/mailman/listinfo/python-list
[issue1642] segfault when deleting value member in ctypes types
New submission from Carl Friedrich Bolz: When trying to delete the .value member of ctypes simple types my python interpreter segfaults: $ python Python 2.5.1 (r251:54863, Oct 5 2007, 13:36:32) [GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2 Type help, copyright, credits or license for more information. from ctypes import c_long c = c_long(1) c.value 1 del c.value Segmentation fault (core dumped) Admittedly there is no good usecase for this, but it should raise an exception and not segfault. -- components: Extension Modules messages: 58705 nosy: cfbolz severity: normal status: open title: segfault when deleting value member in ctypes types type: crash versions: Python 2.5 __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1642 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: py.test generative tests behavior unexpected
Jim Vickroy wrote:
Hello all,
I'm a first time user of py.tests.
My setup is:
* py.test v0.9.0
* Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit
(Intel)] on win32
* Microsoft Windows XP Pro (service pack 2)
The attached file (py-test-generator-trial.py) demonstrates a behavior I
do not understand.
When test_filters() is implemented as a generator, the tests appear to
be run twice and all tests fail during the second iteration except for
the final case ('AL12_1','Gray') in the loop.
When the yield statement, in test_filters(), is disabled and the
assertion statement is enabled, the behavior is as expected -- namely
the tests are performed once and all pass.
Could someone help me to understand this behavior?
I think you will have more luck at the py-lib developer mailing list here:
[EMAIL PROTECTED]
The mailman list info is here:
http://codespeak.net/mailman/listinfo/py-dev
Cheers,
Carl Friedrich
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is a real C-Python possible?
sturlamolden wrote: On 11 Des, 20:25, John Nagle [EMAIL PROTECTED] wrote: Shed Skin effort. Its author writes Am I the only one seeing the potential of an implicitly statically typed Python-like-language that runs at practically the same speed as C++? Don't forget about Pyrex and PyPy's RPython. By the way, we don't need a hotspot JIT compiler. Lisp has a compiler invoked by the programmer. We could include optional static typing in Python, and have an optional static optimizing native compiler for selected portions of code. That would be easier to implement in the short run, with JIT-capabilities added later. Pyrex, ShedSkin or RPython are all good starting points. I just want to stress that adding type hints _won't_ make programs faster if you use a good specializing JIT compiler. Psyco in particular would not benefit from type hints at all (even if you changed Psyco take them into account) and would give you exactly the same speed as without them. Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
[issue1433] marshal roundtripping for unicode
New submission from Carl Friedrich Bolz: Marshal does not round-trip unicode surrogate pairs for wide unicode-builds: marshal.loads(marshal.dumps(u\ud800\udc00)) == u'\U0001' This is very annoying, because the size of unicode constants differs between when you run a module for the first time and subsequent runs (because the later runs use the pyc file). -- components: Unicode messages: 57444 nosy: cfbolz severity: normal status: open title: marshal roundtripping for unicode versions: Python 2.5 __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1433 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: finding out the call (and not only the caller)
Hi Neal, [EMAIL PROTECTED] wrote: The code doesn't handle all the cases (e.g., nested functions), but gives you an idea of what can be done and where to look for the info (ie, in the stack frame and how to parse the byte codes to some extent). For nested functions, look in co_consts. Bytecodes are specific to CPython. This code won't work on IronPython, Jython, or PyPy. This code really shouldn't be used for production. However, it demonstrates some of the info available. Actually that's not true for PyPy. PyPy uses CPython's bytecode, so the example works just fine. Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
Re: 128 or 96 bit integer types?
Paul Rubin wrote: [EMAIL PROTECTED] [EMAIL PROTECTED] writes: has 146 digits. And that's just the begining. The above actually represents a polynomial with 264 terms, the exponents of which range from 0 to 492. One of those polynomials can have over 5 decimal digits when solved. You should use gmpy rather than python longs if you're dealing with numbers of that size. Python multiplication uses a straightforward O(n**2) algorithm where n is the number of digits. That's untrue since quite a while. CPython now uses Karatsuba-multiplication if the number of digits is larger than a certain threshold. Karatsuba is O(n**(log(3) / log(2)). This is the best way for up to a few hundred or maybe a few thousand digits. After that, it's better to use more complicated FFT-based algorithms which are O(n log n) despite their increased constant overhead. Gmpy does this. Karatsuba is still slower than these algorithms, but only if you have quite a big number of digits. Of course the core of your argument remains valid: CPython is not up to providing extremely good big integer arithmetic, so if you have extreme needs, you shouldn't use the builtin longs. Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
Duncan Booth wrote: Robin Becker [EMAIL PROTECTED] wrote: I am hugely encouraged by this C:\Python\devel\pypy-1.0.0\python24\python \python\lib\test \pystone.py Pystone(1.1) time for 5 passes = 1.49586 This machine benchmarks at 33425.6 pystones/second C:\Python\devel\pypy-1.0.0.\pypy-c.exe \python\lib\test\pystone.py Pystone(1.1) time for 5 passes = 2.16123e-005 This machine benchmarks at 2.3135e+009 pystones/second :) not It looks like time.clock() is completely borked. time.clock was indeed borked under windows (of by a factor of 1000). fixed in svn now. Cheers, Carl Friedrich -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
Hi Luis! Luis M. González wrote: Carl Friedrich Bolz wrote: == PyPy 1.0: JIT compilers for free and more == [snip] Congratulations! Thanks :-) I just have a couple of questions: Attempting to execute pypy-c.exe (precompiled binary for Windows), raise an error message indicating that it cannot find gc_pypy.dll. Have you missed something or I'm doing something wrong? Brain error on our side: the gc_pypy.dll is the dll of the Boehm garbage collector, which you would need to compile yourself (which makes precompiled binaries a bit useless :-) ). We updated the zip file, would you mind checking whether it works better now? Regarding the jit integration, do you have any rough idea of when will it speed up arbitrary code, other than integer aritmethic? Relatively soon. The hard bits of that are done, the JIT compiler generator is in a pretty good shape, we need to apply it to more parts of the PyPy Python interpreter. There are some non-coding related reasons why we are not doing it _right_ now: we still need to write some EU-reports (about the JIT for example) then we will need a while to recover from the EU project. Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
Hi Christian! Christian Tismer wrote: On 28.03.2007, at 10:38, Carl Friedrich Bolz wrote: Brain error on our side: the gc_pypy.dll is the dll of the Boehm garbage collector, which you would need to compile yourself (which makes precompiled binaries a bit useless :-) ). We updated the zip file, would you mind checking whether it works better now? Why can't we provide a pre-compiled binary? Is this a license issue? We can. That's exactly what we did. Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
Hi! dmitrey wrote: Hi! Suppose I have a py-written module. Is it possible somehow run PyPy on the whole module? I didn't find it in documentation. And if yes (or if just run in every module func) what will be after computer restart? Should I restart PyPy on the module once again? And are there any chances/intends for PyPy to be included into Python core? PyPy contains a full Python interpreter (which can include a JIT compiler) and thus replaces the Python core. PyPy can never really be integrated into CPython. Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
Kay Schluehr wrote: Nice to read that things are going on. I've still a PyPy 0.7 version on my notebook. I guess I will upgrade :) A somewhat unrelated question. With Py3K Python gets optional type annotations. Are you already architecting an annotation handler that can process these annotations? This feature is somewhat competitive to all the complicated type inference and jitting you have been worked out so I don't know how it fits well into the current PyPy architecture? I don't see at all why type annotations are competitive to the type inference that PyPy's translation toolchain is doing. The type inference of PyPy is an artefact of the way we are implementing our interpreter (namely in RPython). You also wouldn't say that the static typing of C is competitive to type annotations because CPython is written in C, right? The JIT (which is completely independent from our type inference engine) will hopefully deal well with the eventual addition of type annotations (it's not clear to me how soon we will start supporting Py3k features, we are not even fully supporting Python 2.5 yet). Since the JIT is automatically generated out of the Python interpreter it should deal with any sort of language changes rather well. Whether that is true in practice remains to be seen, but this is one of the reason why PyPy was started in the first place. Also, I fail to see how type annotations can have a huge speed-advantage versus what our JIT and Psyco are doing. Imagine you have a function that is nicely algorithmic, takes only integers as arguments and only does some calculations with them. If you have a Psyco-like scheme, the JIT will note that you are using it mostly with ints, generate relatively efficient assembly for the whole function (at least after it has been called several times). If you call that functions, the arguments are checked whether they are integers and then the fast assembly code is used without any further type checks. How can you improve on this with type annotations? If the type annotations say that your arguments have to be ints, you _still_ have to check whether this is true. So it is not faster than what Psyco is doing and has the disadvantage that it only works with ints -- good bye, duck typing. Psyco on the other hand will be perfectly happy with you using a custom class that looks mostly like an int and generate assembly optimized for this situation. Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
PyPy 1.0: JIT compilers for free and more
== PyPy 1.0: JIT compilers for free and more == Welcome to the PyPy 1.0 release - a milestone integrating the results of four years of research, engineering, management and sprinting efforts, concluding the 28 months phase of EU co-funding! Although still not mature enough for general use, PyPy 1.0 materializes for the first time the full extent of our original vision: - A flexible Python interpreter, written in RPython: - Mostly unaware of threading, memory and lower-level target platform aspects. - Showcasing advanced interpreter features and prototypes. - Passing core CPython regression tests, translatable to C, LLVM and .NET. - An advanced framework to translate such interpreters and programs: - That performs whole type-inference on RPython programs. - Can weave in threading, memory and target platform aspects. - Has low level (C, LLVM) and high level (CLI, Java, JavaScript) backends. - A **Just-In-Time Compiler generator** able to **automatically** enhance the low level versions of our Python interpreter, leading to run-time machine code that runs algorithmic examples at speeds typical of JITs! Previous releases, particularly the 0.99.0 release from February, already highlighted features of our Python implementation and the abilities of our translation approach but the **new JIT generator** clearly marks a major research result and gives weight to our vision that one can generate efficient interpreter implementations, starting from a description in a high level language. We have prepared several entry points to help you get started: * The main entry point for JIT documentation and status: http://codespeak.net/pypy/dist/pypy/doc/jit.html * The main documentation and getting-started PyPy entry point: http://codespeak.net/pypy/dist/pypy/doc/index.html * Our online play1 demos showcasing various Python interpreters, features (and a new way to program AJAX applications): http://play1.codespeak.net/ * Our detailed and in-depth Reports about various aspects of the project: http://codespeak.net/pypy/dist/pypy/doc/index-report.html In the next few months we are going to discuss the goals and form of the next stage of development - now more than ever depending on your feedback and contributions - and we hope you appreciate PyPy 1.0 as an interesting basis for greater things to come, as much as we do ourselves! have fun, the PyPy release team, Samuele Pedroni, Armin Rigo, Holger Krekel, Michael Hudson, Carl Friedrich Bolz, Antonio Cuni, Anders Chrigstroem, Guido Wesdorp Maciej Fijalkowski, Alexandre Fayolle and many others: http://codespeak.net/pypy/dist/pypy/doc/contributor.html What is PyPy? Technically, PyPy is both a Python interpreter implementation and an advanced compiler, or more precisely a framework for implementing dynamic languages and generating virtual machines for them. The framework allows for alternative frontends and for alternative backends, currently C, LLVM and .NET. For our main target C, we can can mix in different garbage collectors and threading models, including micro-threads aka Stackless. The inherent complexity that arises from this ambitious approach is mostly kept away from the Python interpreter implementation, our main frontend. PyPy is now also a Just-In-Time compiler generator. The translation framework contains the now-integrated JIT generation technology. This depends only on a few hints added to the interpreter source and should be able to cope with the changes to the interpreter and be generally applicable to other interpreters written using the framework. Socially, PyPy is a collaborative effort of many individuals working together in a distributed and sprint-driven way since 2003. PyPy would not have gotten as far as it has without the coding, feedback and general support from numerous people. Formally, many of the current developers were involved in executing an EU contract with the goal of exploring and researching new approaches to language and compiler development and software engineering. This contract's duration is about to end this month (March 2007) and we are working and preparing the according final review which is scheduled for May 2007. For the future, we are in the process of setting up structures to help maintain conceptual integrity of the project and to discuss and deal with funding opportunities related to further PyPy sprinting and developments. See here for results of the discussion so far: http://codespeak.net/pipermail/pypy-dev/2007q1/003577.html 1.0.0 Feature highlights == Here is a summary list of key features included in PyPy 1.0: - The Just-In-Time compiler generator, now capable of generating the first JIT compiler versions of our Python interpreter: http
PyPy 0.99 released
== pypy-0.99.0: new object spaces, optimizations, configuration ... == Welcome to the PyPy 0.99.0 release - a major snapshot and milestone of the last 8 months of work and contributions since PyPy-0.9.0 came out in June 2006! Main entry point for getting-started/download and documentation: http://codespeak.net/pypy/dist/pypy/doc/index.html Further below you'll find some notes about PyPy, the 0.99.0 highlights and our aims for PyPy 1.0. have fun, the PyPy team, Samuele Pedroni, Carl Friedrich Bolz, Armin Rigo, Michael Hudson, Maciej Fijalkowski, Anders Chrigstroem, Holger Krekel, Guido Wesdorp and many others: http://codespeak.net/pypy/dist/pypy/doc/contributor.html What is PyPy? Technically, PyPy is both a Python Interpreter implementation and an advanced Compiler, actually a framework for implementing dynamic languages and generating virtual machines for them. The Framework allows for alternative frontends and for alternative backends, currently C, LLVM and .NET. For our main target C, we can can mix in different Garbage Collectors and threading models, including micro-threads aka Stackless. The inherent complexity that arises from this ambitious approach is mostly kept away from the Python interpreter implementation, our main frontend. Socially, PyPy is a collaborative effort of many individuals working together in a distributed and sprint-driven way since 2003. PyPy would not have gotten as far without the coding, feedback and general support from numerous people. Formally, many of the current developers are involved in executing an EU contract with the goal of exploring and researching new approaches to Language/Compiler development and software engineering. This contract's duration is about to end March 2007 and we are working and preparing the according final review which is scheduled for May 2007. Key 0.99.0 Features = * new object spaces: - Tainting: a 270-line proxy object space tracking and boxing sensitive information within an application. A tainted object is completely barred from crossing an I/O barrier, such as writing to files, databases or sockets. This allows to significantly reduce the effort of e.g. security reviews to the few places where objects are declassified in order to send information across I/O barriers. - Transparent proxies: allow to customize both application and builtin objects from application level code. Works as an addition to the Standard Object Space (and is translatable). For details see http://codespeak.net/pypy/dist/pypy/doc/proxy.html * optimizations: - Experimental new optimized implementations for various built in Python types (strings, dicts, lists) - Optimized builtin lookups to not require any dictionary lookups if the builtin is not shadowed by a name in the global dictionary. - Improved inlining (now also working for higher level backends) and malloc removal. - twice the speed of the 0.9 release, overall 2-3 slower than CPython * High level backends: - It is now possible to translate the PyPy interpreter to run on the .NET platform, which gives a very compliant (but somewhat slow) Python interpreter. - the JavaScript backend has evolved to a point where it can be used to write AJAX web applications with it. This is still an experimental technique, though. For demo applications see: http://play1.codespeak.net:8008/ * new configuration system: There is a new comprehensive configuration system that allows fine-grained configuration of the PyPy standard interpreter and the translation process. * new and improved modules: Since the last release, the signal, mmap, bz2 and fcntl standard library modules have been implemented for PyPy. The socket, _sre and os modules have been greatly improved. In addition we added a the pypymagic module that contains PyPy-specific functionality. * improved file implementation: Our file implementation was ported to RPython and is therefore faster (and not based on libc). * The stability of stackless features was greatly improved. For more details see: http://codespeak.net/pypy/dist/pypy/doc/stackless.html * RPython library: The release contains our emerging RPython library that tries to make programming in RPython more pleasant. It contains an experimental parser generator framework. For more details see: http://codespeak.net/pypy/dist/pypy/doc/rlib.html * improved documentation: - extended documentation about stackless features: http://codespeak.net/pypy/dist/pypy/doc/stackless.html - PyPy video documentation: eight hours of talks, interviews and features: http://codespeak.net/pypy/dist/pypy/doc/video
PyPy 0.99 released
== pypy-0.99.0: new object spaces, optimizations, configuration ... == Welcome to the PyPy 0.99.0 release - a major snapshot and milestone of the last 8 months of work and contributions since PyPy-0.9.0 came out in June 2006! Main entry point for getting-started/download and documentation: http://codespeak.net/pypy/dist/pypy/doc/index.html Further below you'll find some notes about PyPy, the 0.99.0 highlights and our aims for PyPy 1.0. have fun, the PyPy team, Samuele Pedroni, Carl Friedrich Bolz, Armin Rigo, Michael Hudson, Maciej Fijalkowski, Anders Chrigstroem, Holger Krekel, Guido Wesdorp and many others: http://codespeak.net/pypy/dist/pypy/doc/contributor.html What is PyPy? Technically, PyPy is both a Python Interpreter implementation and an advanced Compiler, actually a framework for implementing dynamic languages and generating virtual machines for them. The Framework allows for alternative frontends and for alternative backends, currently C, LLVM and .NET. For our main target C, we can can mix in different Garbage Collectors and threading models, including micro-threads aka Stackless. The inherent complexity that arises from this ambitious approach is mostly kept away from the Python interpreter implementation, our main frontend. Socially, PyPy is a collaborative effort of many individuals working together in a distributed and sprint-driven way since 2003. PyPy would not have gotten as far without the coding, feedback and general support from numerous people. Formally, many of the current developers are involved in executing an EU contract with the goal of exploring and researching new approaches to Language/Compiler development and software engineering. This contract's duration is about to end March 2007 and we are working and preparing the according final review which is scheduled for May 2007. Key 0.99.0 Features = * new object spaces: - Tainting: a 270-line proxy object space tracking and boxing sensitive information within an application. A tainted object is completely barred from crossing an I/O barrier, such as writing to files, databases or sockets. This allows to significantly reduce the effort of e.g. security reviews to the few places where objects are declassified in order to send information across I/O barriers. - Transparent proxies: allow to customize both application and builtin objects from application level code. Works as an addition to the Standard Object Space (and is translatable). For details see http://codespeak.net/pypy/dist/pypy/doc/proxy.html * optimizations: - Experimental new optimized implementations for various built in Python types (strings, dicts, lists) - Optimized builtin lookups to not require any dictionary lookups if the builtin is not shadowed by a name in the global dictionary. - Improved inlining (now also working for higher level backends) and malloc removal. - twice the speed of the 0.9 release, overall 2-3 slower than CPython * High level backends: - It is now possible to translate the PyPy interpreter to run on the .NET platform, which gives a very compliant (but somewhat slow) Python interpreter. - the JavaScript backend has evolved to a point where it can be used to write AJAX web applications with it. This is still an experimental technique, though. For demo applications see: http://play1.codespeak.net:8008/ * new configuration system: There is a new comprehensive configuration system that allows fine-grained configuration of the PyPy standard interpreter and the translation process. * new and improved modules: Since the last release, the signal, mmap, bz2 and fcntl standard library modules have been implemented for PyPy. The socket, _sre and os modules have been greatly improved. In addition we added a the pypymagic module that contains PyPy-specific functionality. * improved file implementation: Our file implementation was ported to RPython and is therefore faster (and not based on libc). * The stability of stackless features was greatly improved. For more details see: http://codespeak.net/pypy/dist/pypy/doc/stackless.html * RPython library: The release contains our emerging RPython library that tries to make programming in RPython more pleasant. It contains an experimental parser generator framework. For more details see: http://codespeak.net/pypy/dist/pypy/doc/rlib.html * improved documentation: - extended documentation about stackless features: http://codespeak.net/pypy/dist/pypy/doc/stackless.html - PyPy video documentation: eight hours of talks, interviews and features: http://codespeak.net/pypy/dist/pypy/doc/video
Re: Py 2.5 on Language Shootout
Ramon Diaz-Uriarte wrote: In England the corresponding expression is Counting Angels on a Pinhead http://dannyayers.com/2001/misc/angels.htm Thanks, that is neat. I find the discussion on the sex of the angels, well, sexier. But we are probably a few hundred years late to start a catholic-protestant religious war here :-). NOBODY expects the Spanish Inquisition! Our chief weapon is surprise...surprise and fear...fear and surprise. -- http://mail.python.org/mailman/listinfo/python-list
Re: Py 2.5 on Language Shootout
[EMAIL PROTECTED] wrote: Looking over the benchmarks, one gains the impression that Python is a slow language. What does that even mean - a slow language? The alioth benchmarks provide a set of numbers by which languages may be compared. Wrong. The benchmarks provide a set of numbers by which _implementations_ of languages can be compared. After all, it is possible that someone implements a magic-pixie-dust-interpreter that executes Python programs several orders of magnitude fastes than CPython. Or you could say that C is slow because if you use CINT, a C interpreter ( http://root.cern.ch/root/Cint.html ) to execute it, it is slow. Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
PyPy Leysin Winter Sports Sprint (8-14th January 2007)
= PyPy Leysin Winter Sports Sprint (8-14th January 2007) = .. image:: http://www.ermina.ch/002.JPG The next PyPy sprint will be in Leysin, Switzerland, for the fourth time. This sprint will be the final public sprint of our EU-funded period, and a kick-off for the final work on the upcoming PyPy 1.0 release (scheduled for mid-February). The sprint is the last chance for students looking for a summer job with PyPy this winter! If you have a proposal and would like to work with us in the mountains please send it in before 15th December to pypy-tb at codespeak.net and cross-read this page: http://codespeak.net/pypy/dist/pypy/doc/summer-of-pypy.html -- Goals and topics of the sprint -- * Like previous winter, the main side goal is to have fun in winter sports :-) We can take a couple of days off for ski; at this time of year, ski days end before 4pm, which still leaves plenty of time to recover (er, I mean hack). * Progress on the JIT compiler, which we are just starting to scale to the whole of PyPy. `[1]`_ * Polish the code and documentation of the py lib, and eventually release it. * Work on prototypes that use the new features that PyPy enables: distribution `[2]`_ (based on transparent proxying `[3]`_), security `[4]`_, persistence `[5]`_, etc. `[6]`_. .. _[1]: http://codespeak.net/pypy/dist/pypy/doc/jit.html .. _[2]: http://codespeak.net/svn/pypy/dist/pypy/lib/distributed/ .. _[3]: http://codespeak.net/pypy/dist/pypy/doc/proxy.html .. _[4]: http://codespeak.net/pypy/dist/pypy/doc/project-ideas.html#security .. _[5]: http://codespeak.net/pypy/dist/pypy/doc/project-ideas.html#persistence .. _[6]: http://codespeak.net/pypy/dist/pypy/doc/project-ideas.html --- Location Accomodation --- Leysin, Switzerland, same place as before. Let me refresh your memory: both the sprint venue and the lodging will be in a very spacious pair of chalets built specifically for bed breakfast: http://www.ermina.ch/. The place has a baseline ADSL Internet connexion (600Kb) with wireless installed. You can of course arrange your own lodging anywhere (so long as you are in Leysin, you cannot be more than a 15 minute walk away from the sprint venue), but I definitely recommend lodging there too -- you won't find a better view anywhere else (though you probably won't get much worse ones easily, either :-) I made pre-reservations in the Chalet, so please *confirm* quickly that you are coming so that we can adjust the reservations as appropriate. The rate so far has been 60 CHF a night all included in 2-person rooms, with breakfast. There are larger rooms too (less expensive, possibly more available too) and the possibility to get a single room if you really want to. Please register by svn: http://codespeak.net/svn/pypy/extradoc/sprintinfo/leysin-winter-2007/people.txt or on the pypy-sprint mailing list if you do not yet have check-in rights: http://codespeak.net/mailman/listinfo/pypy-sprint You need a Swiss-to-(insert country here) power adapter. There will be some Swiss-to-EU adapters around - bring a EU-format power strip if you have one. --- Exact times --- Officially, 8th-14th January 2007. Both dates are flexible, you can arrive or leave earlier or later, though it is recommended to arrive on the 7th (if many people arrive on the 6th we need to check for accomodation availability first). We will give introductions and tutorials on the 8th, in the morning if everybody is there, or in the afternoon otherwise. -- http://mail.python.org/mailman/listinfo/python-list
Summer of PyPy Call for Proposals
Last chance to join the Summer of PyPy! === Hopefully by now you have heard of the Summer of PyPy, our program for funding the expenses of attending a sprint for students. If not, you've just read the essence of the idea :-) However, the PyPy EU funding period is drawing to an end and there is now only one sprint left where we can sponsor the travel costs of interested students within our program. This sprint will probably take place in Leysin, Switzerland from 8th-14th of January 2007. So, as explained in more detail at: http://codespeak.net/pypy/dist/pypy/doc/summer-of-pypy.html we would encourage any interested students to submit a proposal in the next month or so. If you're stuck for ideas, you can find some at http://codespeak.net/pypy/dist/pypy/doc/project-ideas.html but please do not feel limited in any way by this list! Cheers, Carl Friedrich Bolz and the PyPy team -- http://mail.python.org/mailman/listinfo/python-list
Re: IDE that uses an external editor?
[EMAIL PROTECTED] wrote: One thing that's kept me from even looking at IDEs is that to the best of my knowledge none of them will integrate properly with external editors like Emacs or vi. I know lots of tools support Emacs-like keybindings, but believe me, I've never found one that does a decent job of that. There is just so much more to powerful editors like Emacs or vi than a handful of cursor movement commands. Once a person is proficient they generally won't accept substitutes. So, please prove me wrong. Are there any IDEs that will actually work with an external instance of Emacs (either by firing it up or by using a remote connection program like gnuclient)? I have never actually used it, but this project seems to integrate Vi and other external editors: http://pida.berlios.de/ Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
Re: references and buffer()
Fredrik Lundh wrote: [snip] is id similar to the address of a variable or a class ? in the CPython implementation, it's the address where the object is stored. but that's an implementation detail. Just as an obscure sidenote, in PyPy it is ~address some of the time. This is due to the fact that since PyPy can use the Boehm garbage collector. The Boehm collector is conservative and therefore has to assume that everything in RAM could be a pointer. Now if somebody stores the id of an object the GC could not distinguish this from a real pointer if id returned the address, which would keep the object alive. /sidenode Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
Re: Scientific computing and data visualization.
[EMAIL PROTECTED] wrote: A commonly used data analysis framework is root (http://root.cern.ch). It offers a object oriented C++ framework with all kind of things one needs for plotting and data visualization. It comes along with PyRoot, an interface making the root objects available to Python. Take a look at the root manual for examples, it also contains a section describing the use of PyRoot. I can definitively second that. ROOT is a bit hard to learn but very, very powerful and PyRoot is really a pleasure to work with. Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
Re: PATCH: Speed up direct string concatenation by 20+%!
Robin Becker wrote: Larry Hastings wrote: __ THE PATCH The core concept: adding two strings together no longer returns a pure string object. Instead, it returns a string concatenation object which holds references to the two strings but does not actually concatenate them... yet. The strings are concatenated only when someone requests the string's value, at which point it allocates all the space it needs and renders the concatenated string all at once. More to the point, if you add multiple strings together (a + b + c), it *doesn't* compute the intermediate strings (a + b). Upsides to this approach: wouldn't this approach apply to other additions eg list+list seq+seq etc etc. no, I think it depends on strings being immutable. If you do list1 + list2 that way and list1 is mutated then the resulting list would be changed too. I suppose the utility of such an approach depends on the frequency with which multiple strings/lists/sequences etc are added together in real code. I think there are quite a lot of string additions around, it's just that people get told to use join all the time, so they are not written using +. Cheers, Carl Friedrich -- http://mail.python.org/mailman/listinfo/python-list
Re: PATCH: Speed up direct string concatenation by 20+%!
Robin Becker wrote: Larry Hastings wrote: __ THE PATCH The core concept: adding two strings together no longer returns a pure string object. Instead, it returns a string concatenation object which holds references to the two strings but does not actually concatenate them... yet. The strings are concatenated only when someone requests the string's value, at which point it allocates all the space it needs and renders the concatenated string all at once. More to the point, if you add multiple strings together (a + b + c), it *doesn't* compute the intermediate strings (a + b). Upsides to this approach: wouldn't this approach apply to other additions eg list+list seq+seq etc etc. no, I think it depends on strings being immutable. If you do list1 + list2 that way and list1 is mutated then the resulting list would be changed too. I suppose the utility of such an approach depends on the frequency with which multiple strings/lists/sequences etc are added together in real code. I think there are quite a lot of string additions around, it's just that people get told to use join all the time, so they are not written using +. Cheers, Carl Friedrich -- http://mail.python.org/mailman/listinfo/python-list
Re: PATCH: Speed up direct string concatenation by 20+%!
Robin Becker wrote: Larry Hastings wrote: __ THE PATCH The core concept: adding two strings together no longer returns a pure string object. Instead, it returns a string concatenation object which holds references to the two strings but does not actually concatenate them... yet. The strings are concatenated only when someone requests the string's value, at which point it allocates all the space it needs and renders the concatenated string all at once. More to the point, if you add multiple strings together (a + b + c), it *doesn't* compute the intermediate strings (a + b). Upsides to this approach: wouldn't this approach apply to other additions eg list+list seq+seq etc etc. no, I think it depends on strings being immutable. If you do list1 + list2 that way and list1 is mutated then the resulting list would be changed too. I suppose the utility of such an approach depends on the frequency with which multiple strings/lists/sequences etc are added together in real code. I think there are quite a lot of string additions around, it's just that people get told to use join all the time, so they are not written using +. Cheers, Carl Friedrich -- http://mail.python.org/mailman/listinfo/python-list
Re: PATCH: Speed up direct string concatenation by 20+%!
Carl Friedrich Bolz wrote: Robin Becker wrote: Larry Hastings wrote: __ THE PATCH The core concept: adding two strings together no longer returns a pure string object. Instead, it returns a string concatenation object which holds references to the two strings but does not actually concatenate them... yet. The strings are concatenated only when someone requests the string's value, at which point it allocates all the space it needs and renders the concatenated string all at once. More to the point, if you add multiple strings together (a + b + c), it *doesn't* compute the intermediate strings (a + b). Upsides to this approach: wouldn't this approach apply to other additions eg list+list seq+seq etc etc. no, I think it depends on strings being immutable. If you do list1 + list2 that way and list1 is mutated then the resulting list would be changed too. I suppose the utility of such an approach depends on the frequency with which multiple strings/lists/sequences etc are added together in real code. I think there are quite a lot of string additions around, it's just that people get told to use join all the time, so they are not written using +. Sorry for the multiple post :-( -- http://mail.python.org/mailman/listinfo/python-list
Re: PATCH: Speed up direct string concatenation by 20+%!
Larry Hastings wrote: [snip] The core concept: adding two strings together no longer returns a pure string object. Instead, it returns a string concatenation object which holds references to the two strings but does not actually concatenate them... yet. The strings are concatenated only when someone requests the string's value, at which point it allocates all the space it needs and renders the concatenated string all at once. [snip] Sounds cool. If I understand it correctly you can write even meaner (and probably even more useless in practive) benchmarks where you safe some of the intermediate results of the concatenation, thus also explointing the better space behaviour: all = [] s = for i in range(1000): s = s + (str(i) + ) * 1000 all.append(s) This should take around 2GB of RAM (so maybe you shouldn't even run it :-) ) on an unpatched CPython but a lot less with your patch. sidenode This is exactly the sort of experiment that is extremely easy to do with PyPy. In fact, some other PyPyers and me wrote a very similar optimization, which can be compiled in if wanted (or not). The whole implementation took roughly 100 lines of code, of which 65 are in a separate file not touching anything else, the rest being minimally invasive changes. We also implemented a similar optimization for string slicing (only if the slice has a substantial length and a step of 1). /sidenode Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
