Re: [Python-Dev] PEP 414 - some numbers from the Django port
Hi, On 3/3/12 2:28 AM, Vinay Sajip wrote: > So, looking at a large project in a relevant problem domain, unicode_literals > and native string markers would appear not to adversely impact readability or > performance. What are you trying to argue? That the overall Django testsuite does not do a lot of string processing, less processing with native strings? I'm surprised you see a difference at all over the whole Django testsuite and I wonder why you get a slowdown at all for the ported Django on 2.7. Regards, Armin` ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP-393/PEP-3118: unicode format specifiers
"Martin v. L?wis" wrote: > > I think it would be nice for Python3.3 to implement the PEP-3118 > > suggestion: > > > > 'c' -> UCS1 > > > > 'u' -> UCS2 > > > > 'w' -> UCS4 > > What is the use case for these format codes? Unfortunately I've only worked with UTF-8 so far and I'm not too familiar with UCS2 and UCS4. *If* the arrays that Victor mentioned give one character per array location, then memoryview(str) could be used for zero-copy slicing etc. The main reason why I raised the issue is this: If Python-3.3 is shipped with 'u' -> UCS4 in the array module and *then* someone figures out that the above format codes are a great idea, we'd be stuck with yet another format code incompatibility. Stefan Krah ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP-393/PEP-3118: unicode format specifiers
The main reason why I raised the issue is this: If Python-3.3 is shipped with 'u' -> UCS4 in the array module and *then* someone figures out that the above format codes are a great idea, we'd be stuck with yet another format code incompatibility. Ah. I think the array module should maintain compatibility with Python 3.2, i.e. "u" should continue to denote Py_UNICODE, i.e. 7fa098f6dc6a should be reverted. It may be that the 'u' code is not particularly useful, but AFAICT, it never was useful. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP-393/PEP-3118: unicode format specifiers
On Wed, Mar 7, 2012 at 8:50 PM, Stefan Krah wrote: > *If* the arrays that Victor mentioned give one character per array location, > then memoryview(str) could be used for zero-copy slicing etc. A slight tangent, but it's worth trying to stick to the "code point" term when talking about what Unicode strings contain. Even in UCS4, full characters may be expressed as multiple code points (to be honest, I still don't understand exactly how code points are composed into graphemes and characters and mapped to glyphs for display, I just know the mapping is a lot more complicated than the one-to-one implied by referring to code points as characters). Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP-393/PEP-3118: unicode format specifiers
On Wed, Mar 7, 2012 at 9:39 PM, wrote: > Ah. I think the array module should maintain compatibility with Python 3.2, > i.e. "u" should continue to denote Py_UNICODE, i.e. 7fa098f6dc6a should be > reverted. > > It may be that the 'u' code is not particularly useful, but AFAICT, it never > was useful. +1. It can go away at the same time that Py_UNICODE itself goes away. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Adding a builtins parameter to eval(), exec() and __import__().
I propose adding an optional (keyword-only?) 3rd parameter, "builtins" to exec(), eval(), __import__() and any other functions that take locals and globals as parameters. Currently, Python code is evaluated in a context of three name spaces; locals(), globals() and builtins. However, eval & exec only take 2 (optional) namespaces as parameters; locals and globals, so access to builtins is poorly defined. The reason I am proposing this here rather than on python-ideas is that treating the triple of [locals, globals, builtins] as a single "execution context" can be implemented in a really nice way. Internally, the execution context of [locals, globals, builtins] can be treated a single immutable object (custom object or tuple) Treating it as immutable means that it can be copied merely by taking a reference. A nice trick in the implementation is to make a NULL locals mean "fast" locals for function contexts. Frames, could then acquire their globals and builtins by a single reference copy from the function object, rather than searching globals for a '__builtins__' to find the builtins. A unified execution context will speed up all calls (to Python functions) as frame allocation and deallocation would be faster. I used this implementation in my original HotPy VM, and it worked well. It should also help with sandboxing, as it would make it easier to analyse and thus control access to builtins, since the execution context of all code would be easier to determine. Currently, it is impossible to allow one function access to sensitive functions like open(), while denying it to others, as any code can then get the builtins of another function via f.__globals__['builtins__']. Separating builtins from globals could solve this. Cheers, Mark. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding a builtins parameter to eval(), exec() and __import__().
2012/3/7 Mark Shannon : > Currently, it is impossible to allow one function access to sensitive > functions like open(), while denying it to others, as any code can then > get the builtins of another function via f.__globals__['builtins__']. > Separating builtins from globals could solve this. I like this idea. We could finally kill __builtins__, too, which has often been confusing for people. -- Regards, Benjamin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding a builtins parameter to eval(), exec() and __import__().
On Wed, Mar 7, 2012 at 10:56, Benjamin Peterson wrote: > 2012/3/7 Mark Shannon : > > Currently, it is impossible to allow one function access to sensitive > > functions like open(), while denying it to others, as any code can then > > get the builtins of another function via f.__globals__['builtins__']. > > Separating builtins from globals could solve this. > > I like this idea. We could finally kill __builtins__, too, which has > often been confusing for people. I like it as well. It's a mess right now to try to grab the __import__() implementation and this would actually help clarify import semantics by saying that __import__() for any chained imports comes from __import__()s locals, globals, or builtins arguments (in that order) or from the builtins module itself (i.e. tstate->builtins). ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] problem with recursive "yield from" delegation
Hi,
I found a problem in the current "yield from" implementation that I think
is worth discussing:
http://bugs.python.org/issue14220
Test code:
def g1():
yield "y1"
yield from g2()
yield "y4"
def g2():
yield "y2"
try:
yield from gi
except ValueError:
pass # catch "already running" error
yield "y3"
gi = g1()
for y in gi:
print("Yielded: %s" % (y,))
This is what it currently does:
1) g1() delegates to a new g2(), propagates its "y2" value and asks for the
next value
2) g2 delegates back to the g1 instance and asks for its next value
3) Python sees the active delegation in g1 and asks g2 for its next value
4) g2 sees that it's already running and throws an exception
Ok so far. Now:
5) the exception is propagated into g1 at call level 3) instead of the
original requestor g2 one level above
6) g1 undelegates and terminates by the exception
7) g2 catches the exception, yields "y3" and then terminates normally
8) g1 gets control back but has already terminated and does nothing
Effect: g1 does not yield "y4" anymore.
The problem is in steps 5) and 6), which are handled by g1 at the wrong
call level. They shouldn't lead to undelegation and termination in g1, just
to an exception being raised in g2.
I ran into this while trying to adapt the implementation for Cython, which
has a different generator type implementation but otherwise uses more or
less the same code now. But I'm not sure how to fix this one without major
changes to the implementation, especially not without special casing the
generator type on delegation (which won't work because CPython doesn't know
about Cython generators). Any ideas?
Stefan
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - some numbers from the Django port
Armin Ronacher active-4.com> writes: > What are you trying to argue? That the overall Django testsuite does > not do a lot of string processing, less processing with native strings? > > I'm surprised you see a difference at all over the whole Django > testsuite and I wonder why you get a slowdown at all for the ported > Django on 2.7. The point of the figures is to show there is *no* difference (statistically speaking) between the three sets of samples. Of course, any individual run or set of runs could be higher or lower due to other things happening on the machine (not that I was running any background tasks), so the idea of the simple statistical analysis is to determine whether these samples could all have come from the same populations. According to ministat, they could have (with a 95% confidence level). The Django test suite is pretty comprehensive, so it would presumably exercise every part of Django, including the parts that handle processing of requests and producing responses. I can't confirm this, not having done a coverage analysis of Django; but this seems like a more representative workload than any microbenchmark which just measures a single operation, like the overhead of a wrapper. And so my argument was that the microbenchmark numbers didn't give a meaningful indication of the actual performance in a real scenario, and they should be taken in that light. No doubt there are other, better (more useful) tests that could be performed (e.g. ab run against all three variants and requests/sec figures compared) but I had the Django test run figures to hand (since they're a byproduct of the porting work), and so presented them in my post. Anyway, it doesn't really matter now, since the latest version of the PEP no longer mentions those figures. Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] problem with recursive "yield from" delegation
2012/3/7 Stefan Behnel : > The problem is in steps 5) and 6), which are handled by g1 at the wrong > call level. They shouldn't lead to undelegation and termination in g1, just > to an exception being raised in g2. That looks wrong indeed. > > I ran into this while trying to adapt the implementation for Cython, which > has a different generator type implementation but otherwise uses more or > less the same code now. But I'm not sure how to fix this one without major > changes to the implementation Cython's or CPython's implementation? -- Regards, Benjamin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] importlib cleared for merging into default!
At the language summit today I got clearance to merge my importlib bootstrap branch (http://hg.python.org/sandbox/bcannon#bootstrap_importlib) thanks to performance being about 5% slower using the normal_startup (which, as Thomas Wouters said, is less than the difference of using the newest gcc in some benchmarking he recently did). Obviously thanks to everyone who has helped out with making this happen over the years! So, where does that leave us? First is getting full code review and sign off from somebody (http://bugs.python.org/issue2377 is the issue tracking this). Once I have that I will do the merge and then go through the tracker to update issues to see if they still apply or need to be re-targeted for importlib. Once importlib has been merged there is some stuff that needs to happen. I will first strip imp down to what has to be in import.c (e.g. importing a built-in module), rename it _imp, and then re-implement/extend whatever is needed in an imp.py module. This will allow for much of the C code left in import.c to go away (i.e. imp.find_module() is the reason the finder code in import.c has not been removed yet). It will also alleviate the other VMs from having to implement all of imp from scratch. Once importlib is in I will also do some proposing on how to undo some import design decisions that were caused because it was all C code (e.g. implicit stuff like a sys.meta_path entry that handles sys.path/sys.path_importer_cache/sys.path_hooks, the meaning of None in sys.path_importer_cache). Everyone at the language summit was supportive of this stuff and basically agreed to it but wanted it as a separate step from the merge to get everything moving faster. What can be done in parallel/while waiting is exposing more of importlib's innards. This ties into some of the proposals I will be making once the merge occurs. Everything else I have in mind is long term stdlib cleanup using importlib, but that is not important now. IOW, someone please review my bootstrap_importlib branch so I can merge it. =) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - some numbers from the Django port
On Thu, Mar 8, 2012 at 8:36 AM, Vinay Sajip wrote: > Anyway, it doesn't really > matter now, since the latest version of the PEP no longer mentions those > figures. Indeed, I deliberately removed the part about performance concerns, since I considered it a distraction from what I see as the heart of the problem PEP 414 is designed to address (i.e. that the purely mechanical changes previously required to Unicode text that is already clearly marked as such in the Python 2 version are irrelevant noise when it comes to identifying and reviewing the *actual* changes needed for a successful Python 3 port). Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] importlib cleared for merging into default!
On 7 March 2012 23:05, Brett Cannon wrote: > At the language summit today I got clearance to merge my importlib bootstrap > branch (http://hg.python.org/sandbox/bcannon#bootstrap_importlib) thanks to > performance being about 5% slower using the normal_startup (which, as Thomas > Wouters said, is less than the difference of using the newest gcc in some > benchmarking he recently did). Obviously thanks to everyone who has helped > out with making this happen over the years! Yay! Congratulations for getting this done. When I first co-authored PEP 302 I never realised what a long road it would be from there to here. Thanks for making it happen. Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 412 has been accepted
Since PEP 412 has code that doesn't break tests anymore (thanks to hash randomization), it was just accepted. Mark, can you make sure there is an up-to-date patch in the tracker so people can potentially look at the code at the sprints here at PyCon? And also please apply for core dev privileges (http://docs.python.org/devguide/coredev.html) so that we can make you fix bugs. =) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] problem with recursive "yield from" delegation
On Thu, Mar 8, 2012 at 6:40 AM, Stefan Behnel wrote: > I ran into this while trying to adapt the implementation for Cython, which > has a different generator type implementation but otherwise uses more or > less the same code now. But I'm not sure how to fix this one without major > changes to the implementation, especially not without special casing the > generator type on delegation (which won't work because CPython doesn't know > about Cython generators). Any ideas? After tinkering with it a bit, a couple of my original guesses as to the underlying problem were clearly wrong. I'm moving house next week, so it'll be a while before I get to look at it in detail, but I added Mark Shannon to the issue's nosy list. He's been working on a few patches lately to clean up generator related state handling in general, so he may have some insight into this. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] problem with recursive "yield from" delegation
2012/3/7 Benjamin Peterson : > 2012/3/7 Stefan Behnel : >> The problem is in steps 5) and 6), which are handled by g1 at the wrong >> call level. They shouldn't lead to undelegation and termination in g1, just >> to an exception being raised in g2. > > That looks wrong indeed. Fixed as of 3357eac1ba62 -- Regards, Benjamin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] problem with recursive "yield from" delegation
On Thu, Mar 8, 2012 at 10:00 AM, Benjamin Peterson wrote: > 2012/3/7 Benjamin Peterson : >> 2012/3/7 Stefan Behnel : >>> The problem is in steps 5) and 6), which are handled by g1 at the wrong >>> call level. They shouldn't lead to undelegation and termination in g1, just >>> to an exception being raised in g2. >> >> That looks wrong indeed. > > Fixed as of 3357eac1ba62 Thanks. And, since the fix was entirely internal to the generator implementation, Stefan should be right for the Cython generators, too. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] problem with recursive "yield from" delegation
http://mail.python.org/pipermail/python-dev/2012-March/117396.html Stefan Behnel posted: > I found a problem in the current "yield from" implementation ... [paraphrasing] g1 yields from g2 g2 yields from g1 XXX python follows the existing delegation without checking re-entrancy g2 (2nd call) checks re-entrancy, and raises an exception g1 (2nd call) gets to handle the exception, and doesn't g2 (1st call) gets to handle the exception, and does How is this a problem? Re-entering a generator is a bug. Python caught it and raised an appropriate exception. It would be nice if python caught the generator cycle as soon as it was created, just as it would be nice if reference cycles were collected as soon as they became garbage. But python doesn't promise to catch cycles immediately, and the checks required to do so would slow down all code, so in practice the checks are delayed. -jJ -- If there are still threading problems with my replies, please email me with details, so that I can try to resolve them. -jJ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] problem with recursive "yield from" delegation
On Thu, Mar 8, 2012 at 10:32 AM, Jim J. Jewett wrote: > How is this a problem? > > Re-entering a generator is a bug. Python caught it and raised an > appropriate exception. No, the problem was that the interpreter screwed up the state of the generators while attempting to deal with the erroneous reentry. The ValueError *should* just be caught and completely suppressed by the try/except block, but that wasn't quite happening properly - the failed attempt at reentry left the generators in a dodgy state (which is why the subsequent "3" was being produced, but then the expected final "4" vanished into the electronic ether). Benjamin figured out where the generator's reentrancy check was going wrong, so Stefan's example should do the right thing in the next alpha (i.e. the ValueError will still get raised and suppressed by the try/except block, the inner generator will complete, but the outer generator will also continue on to produce the final value). Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Adding a builtins parameter to eval(), exec() and __import__().
http://mail.python.org/pipermail/python-dev/2012-March/117395.html Brett Cannon posted: [in reply to Mark Shannon's suggestion of adding a builtins parameter to match locals and globals] > It's a mess right now to try to grab the __import__() > implementation and this would actually help clarify import semantics by > saying that __import__() for any chained imports comes from __import__()s > locals, globals, or builtins arguments (in that order) or from the builtins > module itself (i.e. tstate->builtins). How does that differ from today? If you're saying that the locals and (module-level) globals aren't always checked in order, then that is a semantic change. Probably a good change, but still a change -- and it can be made indepenently of Mark's suggestion. Also note that I would assume this was for sandboxing, and that missing names should *not* fall back to the "real" globals, although I would understand if bootstrapping required the import statement to get special treatment. (Note that I like Mark's proposed change; I just don't see how it cleans up import.) -jJ -- If there are still threading problems with my replies, please email me with details, so that I can try to resolve them. -jJ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Non-string keys in type dict
Hi, During the Language Summit 2011 (*), it was discussed that PyPy and Jython don't support non-string key in type dict. An issue was open to emit a warning on such dict, but the patch has not been commited yet. I'm trying to Lib/test/crashers/losing_mro_ref.py: I wrote a patch fixing the specific issue (keep a strong reference to the MRO during the lookup, see #14199), but I realized that the real problem is that we allow custom objects in the type dict. So my question is: what is the use case of such dict? Why do we still support it? Can't we simply raise an error if the dict contains non-string keys? (*) http://blog.python.org/2011/03/2011-language-summit-report.html Victor ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
2012/3/7 Victor Stinner : > So my question is: what is the use case of such dict? Why do we still > support it? Probably a side-effect of implementation. > Can't we simply raise an error if the dict contains > non-string keys? Sounds okay to me. -- Regards, Benjamin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
> During the Language Summit 2011 (*), it was discussed that PyPy and
> Jython don't support non-string key in type dict. An issue was open to
> emit a warning on such dict, but the patch has not been commited yet.
It's the issue #11455. As written in the issue, there are two ways to
create such type:
class A(object):
locals()[42] = "abc"
or
type("A", (object,), {42: "abc"})
Both look like an ugly hack.
Victor
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
On Wed, Mar 7, 2012 at 5:45 PM, Victor Stinner wrote:
> > During the Language Summit 2011 (*), it was discussed that PyPy and
> > Jython don't support non-string key in type dict. An issue was open to
> > emit a warning on such dict, but the patch has not been commited yet.
>
> It's the issue #11455. As written in the issue, there are two ways to
> create such type:
>
> class A(object):
>locals()[42] = "abc"
>
> or
>
> type("A", (object,), {42: "abc"})
>
> Both look like an ugly hack.
>
Here is a cleaner version, using metaclasses (Python 2.6):
class M(type):
def __new__(mcs, name, bases, dict):
dict[42] = 'abc'
return super(M, mcs).__new__(mcs, name, bases, dict)
class A(object):
__metaclass__ = M
>
> Victor
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ckaynor%40zindagigames.com
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
I see that I've misunderstood this entirely, nevermind me.
--Brett
On 08/03/12 14:48, Brett Wilkins wrote:
> I assume when you say "non-string keys" this includes numbers.
>
> But in Pypy, I can certainly use numbers:
{'1':1, 1:2}.keys()
> ['1', 1]
>
> I can even use a lambda (obviously not a string, a number, nor what I
> would consider a primitive):
{'1':1, (lambda x: x):2}.keys()
> ['1', at 0x7fdb0b837da8>]
>
> These are in Pypy 1.8.
>
> --Brett
>
> On Thu 08 Mar 2012 14:39:40 NZDT, Victor Stinner wrote:
>> Hi,
>>
>> During the Language Summit 2011 (*), it was discussed that PyPy and
>> Jython don't support non-string key in type dict. An issue was open to
>> emit a warning on such dict, but the patch has not been commited yet.
>>
>> I'm trying to Lib/test/crashers/losing_mro_ref.py: I wrote a patch
>> fixing the specific issue (keep a strong reference to the MRO during
>> the lookup, see #14199), but I realized that the real problem is that
>> we allow custom objects in the type dict.
>>
>> So my question is: what is the use case of such dict? Why do we still
>> support it? Can't we simply raise an error if the dict contains
>> non-string keys?
>>
>> (*) http://blog.python.org/2011/03/2011-language-summit-report.html
>>
>> Victor
>> ___
>> Python-Dev mailing list
>> [email protected]
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/brett%40brett.geek.nz
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
I assume when you say "non-string keys" this includes numbers.
But in Pypy, I can certainly use numbers:
>>> {'1':1, 1:2}.keys()
['1', 1]
I can even use a lambda (obviously not a string, a number, nor what I
would consider a primitive):
>>> {'1':1, (lambda x: x):2}.keys()
['1', at 0x7fdb0b837da8>]
These are in Pypy 1.8.
--Brett
On Thu 08 Mar 2012 14:39:40 NZDT, Victor Stinner wrote:
> Hi,
>
> During the Language Summit 2011 (*), it was discussed that PyPy and
> Jython don't support non-string key in type dict. An issue was open to
> emit a warning on such dict, but the patch has not been commited yet.
>
> I'm trying to Lib/test/crashers/losing_mro_ref.py: I wrote a patch
> fixing the specific issue (keep a strong reference to the MRO during
> the lookup, see #14199), but I realized that the real problem is that
> we allow custom objects in the type dict.
>
> So my question is: what is the use case of such dict? Why do we still
> support it? Can't we simply raise an error if the dict contains
> non-string keys?
>
> (*) http://blog.python.org/2011/03/2011-language-summit-report.html
>
> Victor
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40brett.geek.nz
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
On Thu, Mar 8, 2012 at 11:42 AM, Benjamin Peterson wrote: > 2012/3/7 Victor Stinner : >> Can't we simply raise an error if the dict contains >> non-string keys? > > Sounds okay to me. For 3.3, the most we can do is trigger a deprecation warning, since removing this feature *will* break currently running code. I don't have any objection to us starting down that path, though. Regards, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
On Thu, Mar 08, 2012 at 12:20:21PM +1000, Nick Coghlan wrote: > On Thu, Mar 8, 2012 at 11:42 AM, Benjamin Peterson > wrote: > > 2012/3/7 Victor Stinner : > >> Can't we simply raise an error if the dict contains > >> non-string keys? > > > > Sounds okay to me. > > For 3.3, the most we can do is trigger a deprecation warning, since > removing this feature *will* break currently running code. I don't > have any objection to us starting down that path, though. Could we make string-key-only dicts a public type instead of an implementation detail? I've used string-only keys in my code, and it seems silly to have to re-invent the wheel. I don't care if it is a built-in. I don't even care if I have to do something gnarly like StringDict = type(type.__dict__), so long as doing so is officially supported. (But "from collections import StringDict" would be better from the point of discoverability.) -- Steven ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
On Wed, Mar 7, 2012 at 8:39 PM, Victor Stinner wrote: > So my question is: what is the use case of such dict? Well, I use them for this: http://pypi.python.org/pypi/AddOns (And I have various other libraries that depend on that library.) Short version: AddOns are things you can use to dynamically extend instances -- a bit like the "decorator" in "decorator pattern" (not to be confused with Python decorators). Rather than synthesize a unique string as a dictionary key, I just used the AddOn classes themselves as keys. This works fine for object instances, but gets hairy once classes come into play. ( http://pypi.python.org/pypi/AddOns#class-add-ons - an orthogonal alternative to writing hairy metaclasses with registries for special methods, persisted attributes, and all other sorts of things one would ordinarily use metaclasses for.) In principle, I could refactor AddOns to use synthetic (i.e. made-up) strings as keys, but it honestly seemed unpythonic to me to make up a key when the One Obvious key to use is the AddOn type itself. (Or in some cases, a tuple comprised of an AddOn type plus additional values - which would mean string manipulation for every access.) Another possible solution would be to not store addons directly in a class' dictionary, but instead throw in an __addons__ key with a subdictionary; again this seemed like pointless indirection, wasted memory and access time when there's already a perfectly good dictionary lying about. IOW, it's one of those places where Python's simple orthogonality seems like a feature rather than a bug that needs fixing. I mean, next thing you know, people will be saying that *instance* dictionaries need to have only string keys or something. ;-) Of course, if my library has to change to be able to work on 3.3, then I guess it'll have to change. IIRC, this is *probably* the only place I'm using non-string keys in type or instance dictionaries, so in the big scheme of porting costs, it's not that much. But, since you asked, that's the main use case I know of for non-string keys in type dictionaries, and I wouldn't be terribly surprised if I'm the only person with public code that does this. ;-) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sandboxing Python
Hi Stefan, Stefan Behnel wrote: > could you please stop bashing CPython for no good reason, especially on > python-dev? Specifically, to call it broken beyond repair is a rather > offensive claim, especially when made in public. Sorry if you were offended. I am just trying to point out that CPython has a rather large number of *far-fetched* corner cases in which it is broken. (If this is news to anyone, sorry, but examples have been part of the CPython source tree for years and years.) This is of course very different from saying that CPython is generally broken --- I don't think anyone here considers that it is. My point is merely to repeat that CPython is not suited to be the (only) line of defence in any place that needs serious security. I personally think that the removal of 'rexec' back around Python 2.3(?) was a good idea, as such tools give people a false sense of security. A bientôt, Armin. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
On Wed, Mar 7, 2012 at 5:42 PM, Benjamin Peterson wrote: > 2012/3/7 Victor Stinner : >> So my question is: what is the use case of such dict? Why do we still >> support it? > > Probably a side-effect of implementation. > >> Can't we simply raise an error if the dict contains >> non-string keys? > > Sounds okay to me. > > > -- > Regards, > Benjamin > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fijall%40gmail.com I think the original reason given was that enforcing the type would make a performance hit, but I might be misremembering. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sandboxing Python
Maciej Fijalkowski, 06.03.2012 00:08: > For a comparison, PyPy sandbox is a compiled from higher-level > language program that by design does not have all sorts of problems > described. The amount of code you need to carefully review is very > minimal (as compared to the entire CPython interpreter). It does not > mean it has no bugs, but it does mean finding segfaults is a > significantly harder endeavour. Well, there's a bug tracker that lists some of them, which is not *that* hard to find. Does your claim about "a significantly harder endeavour" refer to finding a crash or to finding a fix for it? Stefan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
PJ Eby wrote: Short version: AddOns are things you can use to dynamically extend instances -- a bit like the "decorator" in "decorator pattern" (not to be confused with Python decorators). Rather than synthesize a unique string as a dictionary key, I just used the AddOn classes themselves as keys. This works fine for object instances, but gets hairy once classes come into play. Are you able to modify classes after class creation in Python 3? Without using a metaclass? ~Ethan~ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
