[pypy-dev] Re: How to test code that uses weakrefs
On 02.09.24 18:02, cpbu...@gmail.com wrote: This is just my 2 cents as a PyPy user. CPython eagerly destroys unreferenced objects while PyPy will destroy them eventually. I ran into a problem with the requests library where it would create connection pool object every request, and the unreferenced pool would not be cleaned up immediately which caused the maximum number of open file descriptors to be reached (if I recall correctly). The solution to my case was calling `gc.collect_step()` (a PyPy-only function) after each request. You can also try the standard `gc.collect()` function which may be more reliable for a unit test to ensure any unreferenced objects are garbage collected. This may not help with the general use of your library though. We have similar issues with garbage collection in PySide on PyPy. To reliably get rid of objects, we are calling gc.collect() twice. ciao -- Chris -- Christian Tismer-Sperling:^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Strandstr. 37: https://www.qt.io/qt-for-python 24217 Schönberg : GPG key -> 0xE7301150FB7BEE0E phone +49 176 624 8 ___ pypy-dev mailing list -- pypy-dev@python.org To unsubscribe send an email to pypy-dev-le...@python.org https://mail.python.org/mailman3/lists/pypy-dev.python.org/ Member address: arch...@mail-archive.com
Re: [pypy-dev] better line number information in PyPy3
Hi Carl, I happened to stumble into this and saw your mail. This makes sense to me, rather cheap correctness is a good thing. Can we not at the same time negotiate with CPython to make their lineno info equally correct, maybe in 3.8 9 or at least 3.9? Cheers -- Chris On 05.06.19 10:57, Carl Friedrich Bolz wrote: > Hi all, > > This is a discussion mail about improving the traceback line number > quality in PyPy3. Some context: > > CPython has started supporting negative line number offset from one > bytecode to the next in its lnotab encoding, see here: > > https://bugs.python.org/issue26107 > https://github.com/python/cpython/commit/f3914eb16d2#diff-cb296cc5109f5640ff3f6d7198a6abee > > (Please don't ask me why not a single test was added for this new feature.) > > However, the support is so far only theoretically possible, the Python > 3.7/3.8 bytecode compiler does not seem to make use of the feature. > Negative line number jumps occur quite regularly with continuation > lines, eg like this: > > def foobar(a, b): > return "" + (1, > a, > b, > 2 > ) > > If you call this function, the traceback will contain this line: > > File "x.py", line 5, in foobar > 2 > TypeError: can only concatenate str (not "tuple") to str > > Which is obviously misleading. > > > A slightly more realistic example are function decorators. There, the > execution order is regularly "backwards": first the function > definition is executed, then the individual decorators are run. There > is currently a hack to sometimes leave the line number on the line of > the first decorator, leading to confusing tracebacks. Example: > > def works(f): > return f > > def broken(f): > assert 0 > > @works > @broken > @works > def g(): > pass > > > Running this, you'll get the following traceback: > > Traceback (most recent call last): > File "y.py", line 9, in > @works > File "y.py", line 5, in broken > assert 0 > AssertionError > > Pointing you again to the wrong line. > > > We could fix this problem rather easily by removing the code that > prevents the bytecode compiler from emitting negative line number > offsets. Should I implement this? > > Advantages: > - A lot less confusing tracebacks in real-world cases > > Disadvantages: > - Different behavior from CPython (but arguably more correct) > - Tracing tools could potentially be confused > - Potentially larger lnotab strings due to more jumping around > > Cheers, > > Carl Friedrich > ___ > pypy-dev mailing list > pypy-dev@python.org > https://mail.python.org/mailman/listinfo/pypy-dev > -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 ___ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] sandbox small bug patch
On 05/11/15 09:14, Armin Rigo wrote: > Hi Dmitry, > > On Thu, Nov 5, 2015 at 12:40 AM, Dmitry Sagalovskiy > wrote: >> I've been wondering why it is that 'time' module is so broken in the >> sandbox? Is there any fundamental reason why it can't be supported? > > No, it's just that we improved the 'time' module at some point by > getting rid of very old code, and that broke sandboxed translations. > Someone needs to look at it. It's probably easy. Ok, makes sense. I did not expect that this is the final solution, but just shows the problem and an immediate fix against frustration. If my company (yes, I'm working again) permits, then I can work on fixing the true reason. cheers - Chris -- Christian Tismer :^) tis...@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam: GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023 signature.asc Description: OpenPGP digital signature ___ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
[pypy-dev] sandbox small bug patch
Hi guys,
I was playing with sandbox and found that the interactive mode
does not work because of 'import time'.
Wanted to create a pull request, but for some reason I can't push
to bitbucket right now.
A simple patch is attached.
Cheers - Chris
--
Christian Tismer :^) tis...@stackless.com
Software Consulting : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776 fax +49 (30) 700143-0023
# HG changeset patch
# User Christian Tismer
# Date 1446672957 -3600
# Branch fix-sandbox
# Node ID f4296318c6bf2077c7535cdeebb001a3653aef4a
# Parent f48af5699c242d420ade3a0389115c5697af3a03
fix the sandbox interactive shell
When the sandbox runs interactivelz, it complains and crashes because of the
import
on the time module, which is unsupported in sandbox.
The patch circumvents this case by intercepting MemoryError.
diff --git a/lib_pypy/_pypy_interact.py b/lib_pypy/_pypy_interact.py
--- a/lib_pypy/_pypy_interact.py
+++ b/lib_pypy/_pypy_interact.py
@@ -25,6 +25,12 @@
print(text)
except ImportError:
pass
+except MemoryError:
+if sys.pypy_translation_info['translation.sandbox']:
+print("sandbox: _pypy_irc_topic has a forbidden import")
+pass
+else:
+raise
#
try:
if not os.isatty(sys.stdin.fileno()):
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] progress with numpy and removal of numpy.py
On 17/10/13 10:59, Antonio Cuni wrote: Hi, On 15/10/13 12:45, matti picus wrote: and what about cloning the numpy repo into bitbucket/pypy to make it more of a "pypy owned" thing? I think it's a good idea. I propose the following: 1) we move your repo to bitbucket/pypy/numpypy 2) we package numpypy, so that people can just do "pip install numpypy" 3) once numpypy is installed, we no longer require the ugly "import numpypy"; a simple "import numpy" will just work. 4) for some time at least, we distribute a numpypy.py so that when imported it prints an error message which explain how to get the newer numpypy Just a note: A repo from github is definately a git repo, and it stays in git format when moved to bitbucket. May not be relevant, I just wanted to mention that. cheers - Chris p.s.: After having worked with both for quite a long time, I now would prefer if pypy and python would use git ;-) -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] A few notes trying to compile pypy
On 26/10/13 16:35, Armin Rigo wrote: Hi, On Sat, Oct 26, 2013 at 10:25 AM, Maciej Fijalkowski wrote: about libffi.h - Apple removed /usr/include and moved it somewhere else, no clue where. I also don't have access to Mavericks so I can't fix it. Ooooh. Traditionally, "libffi.h" was put in unexpectedly varying directories on every Linux distribution, just to make sure there was no reliable way to find it, but that was resolved some time ago with pkg-config. Apple's move is on a completely different level: killing the whole /usr/include is a perfect way to break compilation for every open source project under the sun. I'm impressed. That is much less of a problem because there is homebrew, and I use it for almost everything that I don't want to build/take care of. About mavericks: can I help you out? Works fine in a virtual machine using parallels, if you don't want to change the system. cheers - chris -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] x is y <=> id(x)==id(y)
On 06.05.13 08:54, Armin Rigo wrote: Hi Simon, On Mon, May 6, 2013 at 12:48 AM, Simon Cross wrote: I was thinking along similar signs -- we could ask for things like "x is ''" or "x is 3" to be added to PEP8 (I think any use of "is" with a constant on one or more sides is likely suspect). That may be a good idea. If the compiler emits SyntaxWarnings for these cases, then maybe it's all we need to cover most of the bad usages. I highly appreciate this idea, too! Educating people to avoid mis-use of "is" has probably more impact in the long term, because the pep8 module is pretty often used as a measure of code cleaning. cheers - chris -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] [Python-Dev] efficient string concatenation (yep, from 2004)
Hi Lennart,
Sent from my Ei4Steve
On Feb 13, 2013, at 8:42, Lennart Regebro wrote:
>> Something is needed - a patch for PyPy or for the documentation I guess.
>
> Not arguing that it wouldn't be good, but I disagree that it is needed.
>
> This is only an issue when you, as in your proof, have a loop that
> does concatenation. This is usually when looping over a list of
> strings that should be concatenated together. Doing so in a loop with
> concatenation may be the natural way for people new to Python, but the
> "natural" way to do it in Python is with a ''.join() call.
>
> This:
>
>s = ''.join(('X' for x in xrange(x)))
>
> Is more than twice as fast in Python 2.7 than your example. It is in
> fact also slower in PyPy 1.9 than Python 2.7, but only with a factor
> of two:
>
> Python 2.7:
> time for 1000 concats = 0.887
> Pypy 1.9:
> time for 1000 concats = 1.600
>
> (And of course s = 'X'* x takes only a bout a hundredth of the time,
> but that's cheating. ;-)
>
> //Lennart
This all does not really concern me, as long as it roughly has the same order
of magnitude, or better the same big Oh.
I'm not concerned by a constant factor.
I'm concerned by a freezing machine that suddenly gets 1 times slower
because the algorithms never explicitly state their algorithmic complexity.
( I think I said this too often, today?)
As a side note:
Something similar happened to me when somebody used "range" in Python3.3.
He ran the same code on Python 2.7.
with a crazy effect of having to re-boot:
Range() on 2.7 with arguments from some arbitrary input file. A newbie error
that was hard to understand, because
he was tought thinking 'xrange' when writing 'range'. Hard for me to understand
because I am no longer able to make these errors at all, or even expect them.
Cheers - Chris
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] [Python-Dev] efficient string concatenation (yep, from 2004)
On 13.02.13 08:42, Lennart Regebro wrote:
Something is needed - a patch for PyPy or for the documentation I guess.
Not arguing that it wouldn't be good, but I disagree that it is needed.
This is only an issue when you, as in your proof, have a loop that
does concatenation. This is usually when looping over a list of
strings that should be concatenated together. Doing so in a loop with
concatenation may be the natural way for people new to Python, but the
"natural" way to do it in Python is with a ''.join() call.
This:
s = ''.join(('X' for x in xrange(x)))
Is more than twice as fast in Python 2.7 than your example. It is in
fact also slower in PyPy 1.9 than Python 2.7, but only with a factor
of two:
Python 2.7:
time for 1000 concats = 0.887
Pypy 1.9:
time for 1000 concats = 1.600
(And of course s = 'X'* x takes only a bout a hundredth of the time,
but that's cheating. ;-)
This is not about how to write efficient concatenation and not
for me. It is also not about a constant factor, which I don't really
care about but in situations where speed matters.
This is about a possible algorithmic trap, where code written for
CPython may behave well with some roughly O(n) behavior,
and by switching to PyPy you get a surprise when the same
code now has O(n**2) behavior. Such runtime explosions can damage
the trust in PyPy, with code sitting in some module which you even
did not write but "pip install"-ed it.
So this is important to know, especially for newcomers, and for people
who are giving advice to them.
For algorithmic compatibility, there should no longer
be a feature with this drastic side effect, if that cannot be supported by
all other dialects.
To avoid such hidden traps in larger code bases, documentation is
needed that clearly gives a warning saying "don't do that", like CS
students learn for most other languages.
cheers - chris
--
Christian Tismer :^) <mailto:tis...@stackless.com>
Software Consulting : Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/
14482 Potsdam: PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776 fax +49 (30) 700143-0023
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev
[pypy-dev] efficient string concatenation (yep, from 2004)
Hi friends,
_efficient string concatenation_ has been a topic in 2004.
Armin Rigo proposed a patch with the name of the subject,
more precisely:
/[Patches] [ python-Patches-980695 ] efficient string concatenation//
//on sourceforge.net, on 2004-06-28.//
/
This patch was finally added to Python 2.4 on 2004-11-30.
Some people might remember the larger discussion if such a patch should be
accepted at all, because it changes the programming style for many of us
from "don't do that, stupid" to "well, you may do it in CPython", which
has quite
some impact on other implementations (is it fast on Jython, now?).
It changed for instance my programming and teaching style a lot, of course!
But I think nobody but people heavily involved in PyPy expected this:
Now, more than eight years after that patch appeared and made it into 2.4,
PyPy (!) still does _not_ have it!
Obviously I was mislead by other optimizations, and the fact that
this patch was from a/the major author of PyPy who invented the initial
patch for CPython. That this would be in PyPy as well sooner or later was
without question for me. Wrong... ;-)
Yes, I agree that for PyPy it is much harder to implement without the
refcounting trick, and probably even more difficult in case of the JIT.
But nevertheless, I tried to find any reference to this missing crucial
optimization,
with no success after an hour (*).
And I guess many other people are stepping in the same trap.
So I can imagine that PyPy looses some of its speed in many programs,
because
Armin's great hack did not make it into PyPy, and this is not loudly
declared
somewhere. I believe the efficiency of string concatenation is something
that people assume by default and add it to the vague CPython compatibility
claim, if not explicitly told otherwise.
Some silly proof, using python 2.7.3 vs PyPy 1.9:
$ cat strconc.py
#!env python
from timeit import default_timer as timer
tim = timer()
s = ''
for i in xrange(10):
s += 'X'
tim = timer() - tim
print 'time for {} concats = {:0.3f}'.format(len(s), tim)
$ python strconc.py
time for 10 concats = 0.028
$ pypy strconc.py
time for 10 concats = 0.804
Something is needed - a patch for PyPy or for the documentation I guess.
This is not just some unoptimized function in some module, but it is used
all over the place and became a very common pattern since introduced.
/How ironic that a foreseen problem occurs _now_, and _there_ :-)//
/
cheers -- chris
(*)
http://pypy.readthedocs.org/en/latest/cpython_differences.html
http://pypy.org/compat.html
http://pypy.org/performance.html
--
Christian Tismer :^) <mailto:tis...@stackless.com>
Software Consulting : Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/
14482 Potsdam: PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776 fax +49 (30) 700143-0023
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] __pycache__ folders
On 31.07.12 12:35, Stefano Rivera wrote: Hi Armin (2012.07.31_12:10:01_+0200) I think that PyPy shouldn't do that, because CPython doesn't. If Debian/Ubuntu also hacked CPython to do that, then fine; we are then free to point users to them (blame them?) for issues :-) We did seriously consider hacking cpython to do that, before introducing 2.7. But by that point, there was no real benefit, as 2.7 is the last of the 2.x series. https://lists.debian.org/debian-python/2010/04/msg00046.html I did it for pypy, as I thought it fairly likely that there would be a benefit (there's no guarantee that pypy's bytecode format won't change between pypy releases). Only very esoteric code would break as a result of this change. PyPy already has slightly different .pyc semantics to cpython, and has PEP3149, a vaguely similar change, for C extensions. It's worth stating that I did ask if there was any objection to the patches I was applying, more than once, before the first upload to Debian, and received none. The general opinion seemed to be that .pyc files aren't liked much in the pypy community, at all, and this was vaguely inline Well, good to know. Your backport makes sense, but the reasoning should be identical between cpython and pypy. If cpython does not do it, pypy does not do it. The bytecode compatibility is also identical between cpython and pypy, there is no issue here that is not there. pypy currently shows itself as python 2.7.2, and that is the truth. Different behavior introduced by a distribution is IMHO not a benefit, but goes into an incompatible direction. If a backport happens, then in CPython, and PyPy will adjusted, accordingly. One of the incompatibility issues _is_ the unexpected behavior of tests. To Marcus: Now that the issue is clear, it probably makes sense to add some extra check to setup.py that does not inquire the python version but does some probing. Better to be defensive ;-) cheers - chris -- Christian Tismer :^) <mailto:tis...@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Developer selection for Py3k
On 6/30/12 1:33 AM, Christian Tismer wrote: On 29.06.12 01:08, Antonio Cuni wrote: Hi all, as you probably know, the Py3k [1] proposal is getting funded thanks to our generous donors. During the first round, three of use were selected: me, Alex Gaynor and Benjamin Peterson. However, due to unforeseen unavailability of Alex and Benjamin, we are now looking for one more developer to help with the py3k work. If you are interested in getting paid to work on the Py3k proposal, please apply by replying to this email. To be applicable you need to be an experienced PyPy developer, preferably with a previous experience on the Python Interpreter part of PyPy. Hi Anto, Starting from August, I have some time that I could spend on this. Hi, has the selection process revealed a result, and where can I see it? cheers - chris -- Christian Tismer :^) <mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a. PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Developer selection for Py3k
On 29.06.12 01:08, Antonio Cuni wrote: Hi all, as you probably know, the Py3k [1] proposal is getting funded thanks to our generous donors. During the first round, three of use were selected: me, Alex Gaynor and Benjamin Peterson. However, due to unforeseen unavailability of Alex and Benjamin, we are now looking for one more developer to help with the py3k work. If you are interested in getting paid to work on the Py3k proposal, please apply by replying to this email. To be applicable you need to be an experienced PyPy developer, preferably with a previous experience on the Python Interpreter part of PyPy. Hi Anto, Starting from August, I have some time that I could spend on this. cheers -- Chris -- Christian Tismer :^) <mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a. PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] connecting multiple interpreters
On 6/4/12 10:09 PM, Amaury Forgeot d'Arc wrote: 2012/6/4 Christian Tismer <mailto:tis...@stackless.com>> When thinking of it, a slightly crazy concern popped up: I just followed the pyvenv discussions. PEP 405 will not be back-ported to Python2.7. If we now have two interpreter versions in one binary, which one of the Janus-heads will lead decisions like pyvenv startup? Even if there are two interpreters, only one will run at start-up, modules from the other interpreter need a special "import_py3k" function. Sure, I was just kidding. But if the py3k support is very marginal, just enough to start an imported py2.7 interpreter, then this would be a way to have pyvenv running with Python2.7. ok I'll shut up/down now - chris -- Christian Tismer :^)<mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a. PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] connecting multiple interpreters
On 6/4/12 4:26 PM, Michael Foord wrote: On 4 June 2012 14:33, Armin Rigo <mailto:ar...@tunes.org>> wrote: Hi Martijn, hi Holger, On 6/1/12, Martijn Faassen mailto:faas...@startifact.com>> wrote: > Concerning performance overhead of proxies, lifecycle issues would be > tricky If, hypothetically speaking, there is someone interested in writing a PyPy solution where both a Python2 and a Python3 interpreter are running in the same process, then you gain the advantage of having only one GC to run both. At least it transparently solves the issues of lifetime and reference cycles. (You also have for free only one JIT, which can do cross-language optimizations like inlining a Python2 function into a Python3 context or vice-versa). I see these two points as benefits that you don't have in any multi-process solution. It would require some work on the PyPy side, and I'm not aware of anybody ready to invest time in that, but it shouldn't be particularly hard (once PyPy's Python3 interpreter is ready, and once people agree about which API to use to do cross-language calls.) Having multiple interpreter instances within a single process allows for lots of interesting possibilities. IronPython permits this and it was used by Resolver One - just one of the reasons (along with the GIL) that it would have been much harder to write Resolver One in CPython than in IronPython (although .NET was chosen as a platform before Python was chosen as an implementation language). Allowing Python 2 and Python 3 to live within the same process would be very interesting. I too like this idea very much, because _that_ removes the need for a decision which I really can't base upon taste, but module availabilities, PyPy-readiness etc. Such a bridge would be very cool and quite 'peace-making' if may say so. When thinking of it, a slightly crazy concern popped up: I just followed the pyvenv discussions. PEP 405 will not be back-ported to Python2.7. If we now have two interpreter versions in one binary, which one of the Janus-heads will lead decisions like pyvenv startup? just-semi-seriously ;-) -- chris -- Christian Tismer :^)<mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a. PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] change of strategy for the py3k branch?
On 5/31/12 10:29 AM, Antonio Cuni wrote: On 05/30/2012 04:04 PM, Armin Rigo wrote: Then we can probably arrange things so that we use "translate.py" from default, and not from the "py3k branch", which would be stripped of the translation parts. More precisely, we could organize this "py3k branch" --- quotes, because likely living then in another repo --- with an only marginally different directory structure: e.g. call the top-level directory "py3k" instead of "pypy". Then you would use the default's "translate.py" to translate it, without getting conflicts between "pypy.interpreter" as used by translate.py and the new "py3k.interpreter" containing what you are translating. Of course the directories that would be in the py3k package would still have the same name as their original ones, so that we keep open the possibility to do merges without adding yet another layer of troubles. uhm, that's an interesting possibility, I didn't think of it. I wonder if mercurial handles merges well if we rename the top-level directory. To make things cleaner and easier to understand, we should probably also "hg rm" from py3k/ the directories which belongs to the toolchain, just to avoid confusion. I think that in this case at each merge mercurial would ask what to do with file X which has been deleted locally but changed remotely, but this is probably something that we can handle. As I said earlier, the drawback of such "decoupling" solutions is that as soon as you have two separate repos, you'll get troubles such as "you can translate revision XXX only if the pypy repo is at version YYY", which can be frustrating especially when you want to go back in the history. In theory mercurial subrepos are supposed to solve this problem, but in practice we should stay as far as we can from them :-(. Amaury: opinions on Armin's proposed solution? ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev Hi, I really have to second that last statement: Subrepos are a red herring for me, after having lots of trouble in our much smaller project.. The idea is qood, but not in the next half or one year at least. -- Christian Tismer :^)<mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a. PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] The Work Plan Re: STM proposal funding
On 4/1/12 11:23 AM, Armin Rigo wrote: Hi Andrew, hi all, On Wed, Mar 28, 2012 at 18:23, Andrew Francis wrote: Indeed, and it was around 2007, so I expect the authors to have been involved in completely different things for quite some time now... But I could try to contact them anyway. Communications is good :-) I'm also thinking about writing a short paper collecting things I said and think on various blog posts. A kind of "position paper". What do others think of this idea? My PyPy knowledge is still sketchy but I am changing that. I do understand the Twisted reactor model (thanks to my 2008 Pycon Talk) so I could follow discussions in that area. Is this discussed on IRC? This is not discussed a lot right now. But it is apparently relatively easy to adapt the epoll-based Twisted reactor to use the 'transaction' module. (Again, this module is present in the stm-gc branch; look for lib_pypy/transaction.py for the interface, and pypy/module/transaction/* for the Python implementation on top of STM as exposed by RPython.) This 'transaction' module is also meant to be used directly, for example in this kind of Python code: for n in range(...): do_something(n) If each call to do_something() has "reasonable chances" to be independent from other calls, and if the order doesn't matter, then it can be rewritten as: for n in range(...): transaction.add(do_something, n) transaction.run() In addition, each transaction can add more transactions that will be run after it. So if you want to play with lib_pypy/stackless.py to add calls to 'transaction', feel free :-) Maybe it will show that a slightly different API is required from the 'transaction' module; I don't really know so far. Hi A(rmin|ndrew), it is funny how familiar this code looks, re-writing it in terms of Stackless and tasklets: ''' for n in range(...): tasklet(do_something)(n) stackless.run() In addition, each tasklet can add more tasklets that will be run after it. So if you (...) ''' Well, sure, it is not much more than a simple match, and the tasklets are more like sequences of transactions, when they give up their computation by stackless.schedule(), that adds them back to the pool. Anyway, the underlying ideas have similarities that make me think a lot. Thinking of Stackless, I was looking for a way to isolate tasklets in a way to let them run in parallel, as long as they are independent. In STM, independence is enforced, currently at a relatively high price. If Stackless were able to provide some better isolation by design, maybe there could be a hybrid approach, where most operations would not need to rely on STM all the time? Just rolling ideas out -- Chris -- Christian Tismer :^)<mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a. PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] [pypy-commit] pypy default: relaxing int vs long
Hi Amaury,
This is debatable. In this particular case it makes in fact no sense, but in
general, we need to stop distinguishing int and long. Instead, a range check is
needed to determine an int.
Sent from my Ei4Steve
On Mar 13, 2012, at 5:25, "Amaury Forgeot d'Arc" wrote:
> Hi Christian,
>
> The changeset 8abd5883fa0c looks wrong to me:
> http://mail.python.org/pipermail/pypy-commit/2012-March/060785.html
>
> diff --git a/pypy/interpreter/test/test_compiler.py
> b/pypy/interpreter/test/test_compiler.py
> --- a/pypy/interpreter/test/test_compiler.py
> +++ b/pypy/interpreter/test/test_compiler.py
> @@ -794,7 +794,7 @@
> def test_tuple_constants(self):
> ns = {}
> exec "x = (1, 0); y = (1L, 0L)" in ns
> -assert isinstance(ns["x"][0], int)
> +assert isinstance(ns["x"][0], (int, long))
> assert isinstance(ns["y"][0], long)
>
> Why would x[0] be a long? isn't it an int constant?
>
> --
> Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Gothenburg Sprint Dates
On 9/13/11 1:35 PM, Carl Friedrich Bolz wrote: Some of us need to be in Stockholm Oct 24 and 28. Anto needs to be with his family Nov 1. Fscons starts Friday Nov 11 in Göteborg, and we're giving a talk. Thus I propose that we hold a Sprint at my house Wednesday Nov 2nd through Thursday Nov 10. fscons: http://fscons.org/ Nov 11-13 Not sure what day we will be speaking yet. What do the rest of you think of this idea? This should work for me as well. So if nothing suddenly happens, count me in. -- Christian Tismer :^)<mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a. PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] win64
On 7/18/11 10:15 AM, Armin Rigo wrote: Hi Christian, A quick poll about the 'win64 test' branch. Is it going fine? If ll2ctypes is continuing to give you troubles, I'd suggest to move on; keep in mind that ll2ctypes is only used for testing. I would recommend that your priority should be to pass the tests in pypy/rpython/test/ and pypy/translator/c/test/, and when it is done, move on to pypy/rlib/test/ and pypy/module/*/test/. Hi Armin, It is going fine, ll2ctypes works since quite a while. I'm working through rpython and some things in rlib at the moment. Last 10 days I was ill with flu, that slowed me down, and I'm in big trouble with my family, divorce, finding a new apartment for me, find a way for my son to finish his school, and a lot of fight with my wife. So I can't guarantee a steady work right now, but I'm not blocked and will bring that to an end. If I get stuck, I will certainly ask you. cheers - chris -- Christian Tismer :^)<mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A :*Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Bitbucket? Why not python.org
Hi Ronny, I heard you are very knowledgeable on hg and helped pypy quite a lot. May I ask you for some help for stackless as well? Are you on the stackless list? Well, I'll contact you directly. cheers - chris On 6/27/11 7:40 AM, Ronny Pfannschmidt wrote: Hi, having a synced repo on python.org might be a help however i wouldn't want push-permission handling there, since we are probably a bit ore lax about giving new talents access to the repo. Simply having a synced repo on pythong.org might have a better effect for visibility. -- Ronny On Sun, 2011-06-26 at 12:21 +, holger krekel wrote: On Sun, Jun 26, 2011 at 13:37 +0200, Christian Tismer wrote: On 6/26/11 12:27 PM, holger krekel wrote: Hi Christian, On Thu, Jun 23, 2011 at 17:51 +0200, Christian Tismer wrote: On 6/22/11 7:30 PM, Benjamin Peterson wrote: 2011/6/22 Christian Tismer: Hi friends, the subject line says it all... I'm in the progress of updating stackless to use mercurial on python.org and talked to Martin v. Loewis who pointed out the restrictions of Bitbucket. Besides the impression that Bitbucket is pretty slow, it is also not possible to add our own hooks to it. The impression that Bitbucket is slow? What does that mean? I don't find python.org any faster than Bitbucket. I'm pretty sure python.org would be happy to host PyPy. Is there any good reason why we don't ask and move to python.org? I didn't do the mercurial transition, but I'm pretty happy with bitbucket. This is no answer but an opinion ;-) I was asking why we don't use python.org instead of bitbucket. I don't remember any big comparison analysis. Some people pushed for bitbucket, a number was using it already, and the others didn't mind. The main effort and focus was on the conversion of the svn repository, anyway. Before, we had codespeak.net which was very convenient because I knew all relevant people in person. We have some personal contacts to bitbucket - they actually sponsor an unlimited plan for PyPy. Moreover, some pypy devs wanted a hosting solution where we do not depend on private connects or work but can rather rely on a company basing their business on such hosting. Well, I understand that all. Maybe I was implicitly assuming that everybody felt like me: It is an honor for Stackless Python to live on python.org, and probably also a positive sign, like some acceptance by core python. That made me wonder. If I had a chance to use python.org instead of anything else, I'd always prefer python.org, unless it has a significant drawback, or they told me "no, go somewhere else" ;-) Python used sourceforge before, but preferred to have the freedom to host their data themselves. By using python.org, PyPy would have similar convenience as before. Therefore my question: What makes bitbucket the better choice over python.org, despite free t-shirts? (which might be an important reason for some :-) ) PyPy has by now quite some integration code wrt to bitbucket. It seems all are quite happy with bitbucket services as it stands. So seen from now the question probably rather is why we should move anywhere else. Well, as said, I see a positive political effect in moving to python.org that I (personally) would not underestimate. I agree, it would have this positive political effect so that is clearly on the pro side. It did when i moved the mailing lists to mail.python.org. For what i know, we are welcome to move our repositories to python.org. But PyPy is maybe popular enough that my point doesn't really exist, or even vice versa - maybe the distinction is even welcome. ;-) Heh, dunno. At this stage it would cause some pain but not much technical gain to again move somewhere else. And i think a number of pypy devs are happy with not having to care or think about infrastructure issues or change of dev habits. cheers, holger holger cheers - chris -- Christian Tismer :^)<mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A :*Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev -- Christian Tismer :^)<mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A :*Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57
Re: [pypy-dev] Bitbucket? Why not python.org
On 6/26/11 3:15 PM, Sebastien Douche wrote: On Sun, Jun 26, 2011 at 13:37, Christian Tismer wrote: Well, I understand that all. Maybe I was implicitly assuming that everybody felt like me: It is an honor for Stackless Python to live on python.org, and probably also a positive sign, like some acceptance by core python. From a user point of view, I feel the same thing. That made me wonder. If I had a chance to use python.org instead of anything else, I'd always prefer python.org, unless it has a significant drawback, or they told me "no, go somewhere else" ;-) A solution could be to use bitbucket for the day to day job, and push on python.org regularly (each release?). The cost is very low. Yes, this sounds like a sensible solution. Thanks for all comments so far, I appreciate this! If it turns out to be the best solution anyway, well, then we could do this for the stackless repos too... I will discuss this with the pypy guys, too - we are sprinting right now in Genua, and also ask the core stackless people. cheers -- chris -- Christian Tismer :^)<mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A :*Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Bitbucket? Why not python.org
On 6/26/11 12:27 PM, holger krekel wrote: Hi Christian, On Thu, Jun 23, 2011 at 17:51 +0200, Christian Tismer wrote: On 6/22/11 7:30 PM, Benjamin Peterson wrote: 2011/6/22 Christian Tismer: Hi friends, the subject line says it all... I'm in the progress of updating stackless to use mercurial on python.org and talked to Martin v. Loewis who pointed out the restrictions of Bitbucket. Besides the impression that Bitbucket is pretty slow, it is also not possible to add our own hooks to it. The impression that Bitbucket is slow? What does that mean? I don't find python.org any faster than Bitbucket. I'm pretty sure python.org would be happy to host PyPy. Is there any good reason why we don't ask and move to python.org? I didn't do the mercurial transition, but I'm pretty happy with bitbucket. This is no answer but an opinion ;-) I was asking why we don't use python.org instead of bitbucket. I don't remember any big comparison analysis. Some people pushed for bitbucket, a number was using it already, and the others didn't mind. The main effort and focus was on the conversion of the svn repository, anyway. Before, we had codespeak.net which was very convenient because I knew all relevant people in person. We have some personal contacts to bitbucket - they actually sponsor an unlimited plan for PyPy. Moreover, some pypy devs wanted a hosting solution where we do not depend on private connects or work but can rather rely on a company basing their business on such hosting. Well, I understand that all. Maybe I was implicitly assuming that everybody felt like me: It is an honor for Stackless Python to live on python.org, and probably also a positive sign, like some acceptance by core python. That made me wonder. If I had a chance to use python.org instead of anything else, I'd always prefer python.org, unless it has a significant drawback, or they told me "no, go somewhere else" ;-) Python used sourceforge before, but preferred to have the freedom to host their data themselves. By using python.org, PyPy would have similar convenience as before. Therefore my question: What makes bitbucket the better choice over python.org, despite free t-shirts? (which might be an important reason for some :-) ) PyPy has by now quite some integration code wrt to bitbucket. It seems all are quite happy with bitbucket services as it stands. So seen from now the question probably rather is why we should move anywhere else. Well, as said, I see a positive political effect in moving to python.org that I (personally) would not underestimate. But PyPy is maybe popular enough that my point doesn't really exist, or even vice versa - maybe the distinction is even welcome. ;-) cheers - chris -- Christian Tismer :^)<mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A :*Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Bitbucket? Why not python.org
On 6/22/11 7:30 PM, Benjamin Peterson wrote: 2011/6/22 Christian Tismer: Hi friends, the subject line says it all... I'm in the progress of updating stackless to use mercurial on python.org and talked to Martin v. Loewis who pointed out the restrictions of Bitbucket. Besides the impression that Bitbucket is pretty slow, it is also not possible to add our own hooks to it. The impression that Bitbucket is slow? What does that mean? I don't find python.org any faster than Bitbucket. I'm pretty sure python.org would be happy to host PyPy. Is there any good reason why we don't ask and move to python.org? I didn't do the mercurial transition, but I'm pretty happy with bitbucket. This is no answer but an opinion ;-) I was asking why we don't use python.org instead of bitbucket. Before, we had codespeak.net which was very convenient because I knew all relevant people in person. Python used sourceforge before, but preferred to have the freedom to host their data themselves. By using python.org, PyPy would have similar convenience as before. Therefore my question: What makes bitbucket the better choice over python.org, despite free t-shirts? (which might be an important reason for some :-) ) still wondering -- chris -- Christian Tismer :^)<mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A :*Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
[pypy-dev] Bitbucket? Why not python.org
Hi friends, the subject line says it all... I'm in the progress of updating stackless to use mercurial on python.org and talked to Martin v. Loewis who pointed out the restrictions of Bitbucket. Besides the impression that Bitbucket is pretty slow, it is also not possible to add our own hooks to it. I'm pretty sure python.org would be happy to host PyPy. Is there any good reason why we don't ask and move to python.org? cheers - chris -- Christian Tismer :^)<mailto:tis...@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A :*Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
