Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
A couple of people have said that 'native string' is spelt 'str', but I'm not sure that's the right answer. For example, 2.x's cString.StringIO expects native strings, not Unicode: Your counter-example is non-ASCII characters/bytes. I doubt that this is a valid use case; in a "native" string, these shouldn't occur (i.e. native strings should always be ASCII), since the semantics of non-ASCII changes drastically between 2.x and 3.x. So whoever defines some API to take "native" strings can't have defined a valid use of non-ASCII in that interface. I'm not saying this is the right thing to do for all cases - just that str() may not be, either. This should be elaborated in the PEP. Indeed it should. If there is a known application of non-ASCII native strings, I surely would like to know what that is. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Hi,
On 2/28/12 12:16 AM, mar...@v.loewis.de wrote:
> Armin, I propose that you correct the *factual* deficits of the PEP
> (i.e. remove all claims that cannot be supported by facts, or are otherwise
> incorrect or misleading). Many readers here would be more open to accepting
> the PEP if it was factual rather than polemic.
Please don't call this PEP polemic.
> The PEP author is supposed to collect all arguments, even the ones he
> doesn't agree with, and refute them.
I brought up all the arguments that were I knew about before I submitted
this mailinglist thread and I had since not updated it.
> In this specific issue, the PEP states "the unicode_literals import the
> native string type is no longer available and has to be incorrectly
> labeled as bytestring"
>
> This is incorrect: even though the native string type indeed is no longer
> available, it is *not* consequential that it has to be labeled as byte
> string. Instead, you can use the str() function.
Obviously it means not available by syntax.
> It may be that you don't like that solution for some reason. If so, please
> mention the approach in the PEP, along with your reason for not liking it.
If by str() you mean using "str('x')" as replacement for 'x' in both 2.x
and 3.x with __future__ imports as a replacement for native string
literals, please mention why this is better than u(), s(), n() etc. It
would be equally slow than a custom wrapper function and it would not
support non-ascii characters.
Regards,
Armin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Hi, On 2/27/12 11:54 PM, Steven D'Aprano wrote: > That would be one order of magnitude. I am aware of that :-) Regards, Armin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a frozendict builtin type
On Mon, Feb 27, 2012 at 19:53, Victor Stinner wrote: > A frozendict type is a common request from users and there are various > implementations. There are two main Python implementations: Perhaps this should also detail why namedtuple is not a viable alternative. Cheers, Dirkjan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
>> The PEP author is supposed to collect all arguments, even the ones he
>> doesn't agree with, and refute them.
> I brought up all the arguments that were I knew about before I submitted
> this mailinglist thread and I had since not updated it.
This is fine, of course. I still hope you will update it now, even
though it has been accepted.
>> This is incorrect: even though the native string type indeed is no longer
>> available, it is *not* consequential that it has to be labeled as byte
>> string. Instead, you can use the str() function.
> Obviously it means not available by syntax.
I agree that the native string type is no longer supported by syntax in
that approach.
>> It may be that you don't like that solution for some reason. If so, please
>> mention the approach in the PEP, along with your reason for not liking it.
> If by str() you mean using "str('x')" as replacement for 'x' in both 2.x
> and 3.x with __future__ imports as a replacement for native string
> literals, please mention why this is better than u(), s(), n() etc. It
> would be equally slow than a custom wrapper function and it would not
> support non-ascii characters.
That's not the point. The point is that the PEP ought to mention it as
an alternative, instead of making the false claim that "it has to be
labeled as byte string" (which I take as using a b"" prefix). Feel free
to write something like
"... it either has to be labelled as a byte string, or wrapped into
a function call, e.g. using the str() function. This would be slow and
would not support non-ascii characters"
My whole point here is that I want the PEP to mention it, not this
email thread.
In addition, if you are using this very phrasing that I propose,
I would then claim that
a) it is not slow (certainly not as slow as a custom wrapper (*)), and
b) it's not a problem that it is ASCII-only, since native strings
are *practically* restricted to ASCII, anyway (even though
not theoretically)
In turn, I would ask that this counter-argument of mine is also
reflected in the PEP.
The whole point of the PEP process is that it settles disputes.
Part of that settling is to avoid arguments which go in circles.
To that effect, the PEP author ideally should *quickly* update
the PEP, along with writing responses, so that anybody repeating
an argument could be pointed to the PEP in order to shut up.
HTH,
Martin
(*) This is also something that Guido requested at some point
from the PEP: that it fairly analyses efficient implementations
of potential wrapper functions, taking C implementations into
account as well.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Am 27.02.2012 22:35, schrieb Armin Ronacher: > Hi, > > On 2/27/12 4:44 PM, mar...@v.loewis.de wrote: >> Maybe I'm missing something, but there doesn't seem to be a benchmark >> that measures the 2to3 performance, supporting the claim that it >> runs "two orders of magnitude" slower (which I'd interpret as a >> factor of 100). > My Jinja2+Werkzeug's testsuite combined takes 2 seconds to run (Werkzeug > actually takes 3 because it pauses for two seconds in a cache expiration > test). 2to3 takes 45 seconds to run. And those are small code bases > (15K lines combined). I'm not quite able to reproduce that. I don't know how to run the Jinja2 and Werkzeug test suites combined (Werkzeug's setup.py install gives SyntaxError on Python3). So taking Jinja2 alone, this is what I get: - test suite run: 0.86s (python setup.py test) - 2to3 run: 6.7s (python3 setup.py build, using default:3328e388cb28) So this is less than a factor of ten, but more importantly, much shorter than 45s. I also claim that the example is atypical, in that the test suite completes so quickly. Taking distribute 0.6.24 as a counter-example: - test suite run: 9s - 2to3 run: 7s So the test suite runs longer than the build process. Therefore, even a claim "In many cases 2to3 runs 20 times slower than the testsuite for the library or application it's testing" cannot be substantiated, as cannot the claim "This for instance is the case for the Jinja2 library". On the contrary, I'd expect that the build time using 2to3 is significantly shorter than the test suite run times, *in particular* for large projects. For example, for Django, 2to3 takes less than 3 minutes (IIRC), and the test suite runs an hour or so (depending on how many tests get skipped). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On Tue, Feb 28, 2012 at 08:51, Vinay Sajip wrote: > Lennart Regebro gmail.com> writes: > >> I'm +1 on the PEP, for reasons already repeated here. >> We need three types of strings when supporting both Python 2 and >> Python 3. A binary string, a unicode string and a "native" string, ie >> one that is the old 8-bit str in python 2 but a Unicode str in Python >> 3. > > Well it's a done deal, and as I said elsewhere on the thread, I wasn't > opposing > the PEP, but wanting some improvements in it. ISTM that given the PEP as it > is, > working across 3.2 and 3.3 on a single codebase may not always be the easiest > process (IIUC you have to run a mini2to3 process, and it'll need to be > cleverer > than 2to3 about running over the entire codebase if it's to appear seamless), Distribute helps with this. I think we might have to add a support in distribute to easily exclude the fixer that removes u''-prefixes, I don't remember if there is an "exclude" feature. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a frozendict builtin type
Victor Stinner wrote: The blacklist implementation has a major issue: it is still possible to call write methods of the dict class (e.g. dict.set(my_frozendict, key, value)). It is also possible to use ctypes and violate even more invariants. For most purposes, this falls under "consenting adults". My primary usage of frozendict would be pysandbox, a security module. Attackers are not consenting adults :-) Read-only dict would also help optimization, in the CPython peephole or the PyPy JIT. Not w.r.t. PyPy. It wouldn't do any harm though. One use of frozendict that you haven't mentioned so far is communication between concurrent processes/tasks. These need to be able to copy objects without changing reference semantics, which demands immutability. Cheers, Mark. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a frozendict builtin type
> A frozendict type is a common request from users and there are various >> implementations. There are two main Python implementations: > > Perhaps this should also detail why namedtuple is not a viable alternative. It doesn't have the same API. Example: frozendict[key] vs namedtuple.attr (namedtuple.key). namedtuple has no .keys() or .items() method. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On 28.2.2012 01:16, mar...@v.loewis.de wrote: Armin, I propose that you correct the *factual* deficits of the PEP He cannot, because he would have to throw away whole PEP ... it is all based on non-sensical concept of "native string". There is no such animal (there are only strings and bytes, although they are incorrectly named Unicode strings and strings in Python 2), and whole PEP is just "I don't like Python 3 and I want it to be reverted back to Python 2". It doesn't matter anymore now, but I just needed to put it off my chest. Matěj ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On Tue, 28 Feb 2012 10:02:46 +0100 "Martin v. Löwis" wrote: > > On the contrary, I'd expect that the build time using 2to3 is > significantly shorter than the test suite run times, *in particular* > for large projects. For example, for Django, 2to3 takes less than > 3 minutes (IIRC), and the test suite runs an hour or so (depending > on how many tests get skipped). In the end, that's not particularly relevant, because you don't have to run the test suite entirely; when working on small changes, you usually re-run the impacted parts of the test suite until everything goes fine; on the other hand, 2to3 *has* to run on the entire code base. So, really, it's a couple of seconds to run a single bunch of tests vs. several minutes to run 2to3 on the code base. And it's not just the test suite: every concrete experiment with the library you're porting has a serial dependency on running 2to3. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On Tue, Feb 28, 2012 at 5:56 PM, Matej Cepl wrote: > He cannot, because he would have to throw away whole PEP ... it is all based > on non-sensical concept of "native string". There is no such animal (there > are only strings and bytes, although they are incorrectly named Unicode > strings and strings in Python 2), and whole PEP is just "I don't like Python > 3 and I want it to be reverted back to Python 2". > > It doesn't matter anymore now, but I just needed to put it off my chest. If you don't know what a native string is, then you need to study more to understand why Armin's PEP exists and why it is useful. I suggest starting with PEP (the WSGI update to v1.0.1 that first clearly defined the concept of a native string: http://www.python.org/dev/peps/pep-/#a-note-on-string-types). There are concrete, practical reasons why the lack of Unicode literals in Python 3 makes porting harder than it needs to be. Are they insurmountable? No, of course not - there are plenty of successful ports already that demonstate porting it quite feasible with existing tools. But the existing approaches require that, in order to be forward compatible with Python 3, a program must be made *worse* in Python 2 (i.e. harder to read and harder to write correctly for someone that hasn't learned Python 3 yet). Restoring unicode literal support in 3.3 is a pragmatic step that allows a lot of code to *just work* on Python 3. Most 2.6+ code that still doesn't work on Python 3 even after this change will be made *better* (or at least not made substantially worse) by the additional changes necessary for forward compatibility. Unicode literals are somewhat unique in their impact on porting efforts, as they show up *everywhere* in Unicode correct code in Python 2. The diffs that will be needed to correctly tag bytestrings in such code under Python 2 are tiny compared to those that would be needed to strip the u"" prefixes. Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a frozendict builtin type
> I think you need to elaborate on your use cases further, ... A frozendict can be used as a member of a set or as a key in a dictionary. For example, frozendict is indirectly needed when you want to use an object as a key of a dict, whereas one attribute of this object is a dict. Use a frozendict instead of a dict for this attribute answers to this problem. frozendict helps also in threading and multiprocessing. -- > ... and explain > what *additional* changes would be needed, such as allowing frozendict > instances as __dict__ attributes in order to create truly immutable > objects in pure Python code. > In current Python, you *can't* create a truly immutable object without > dropping > down to a C extension: Using frozendict in for type dictionary might be a use case, but please don't focus on this example. There is currently a discussion on python-ideas about this specific use case. I first proposed to use frozendict in type.__new__, but then I proposed something completly different: add a flag to a set to deny any modification of the type. The flag may be set using "__final__ = True" in the class body for example. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On Tue, 28 Feb 2012 21:42:54 +1000 Nick Coghlan wrote: > But the existing approaches require that, in order to be > forward compatible with Python 3, a program must be made *worse* in > Python 2 (i.e. harder to read and harder to write correctly for > someone that hasn't learned Python 3 yet). Wrong. The separate branches approach allows you to have a clean Python 3 codebase without crippling the Python 2 codebase. Of course that approach was downplayed from the start in favour of using 2to3 on a single codebase, and now we discover that this approach is cumbersome. Note that 2to3 is actually helpful when you choose the dual branches approach, and it isn't a serial dependency in that case. (see https://bitbucket.org/pitrou/t3k/) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a frozendict builtin type
On Tue, 28 Feb 2012 12:45:54 +0100 Victor Stinner wrote: > > I think you need to elaborate on your use cases further, ... > > A frozendict can be used as a member of a set or as a key in a dictionary. > > For example, frozendict is indirectly needed when you want to use an > object as a key of a dict, whereas one attribute of this object is a > dict. It isn't. You just have to define __hash__ correctly. > frozendict helps also in threading and multiprocessing. How so? Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a frozendict builtin type
Antoine Pitrou wrote: On Tue, 28 Feb 2012 12:45:54 +0100 Victor Stinner wrote: I think you need to elaborate on your use cases further, ... A frozendict can be used as a member of a set or as a key in a dictionary. For example, frozendict is indirectly needed when you want to use an object as a key of a dict, whereas one attribute of this object is a dict. It isn't. You just have to define __hash__ correctly. frozendict helps also in threading and multiprocessing. How so? Inter process/task communication requires copying. Inter/intra thread communication uses reference semantics. To ensure these are the same, the objects used in communication must be immutable. Cheers, Mark. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Lennart Regebro gmail.com> writes: > Distribute helps with this. I think we might have to add a support in > distribute to easily exclude the fixer that removes u''-prefixes, I > don't remember if there is an "exclude" feature. We might be at cross purposes here. I don't see how Distribute helps, because the use case I'm talking about is not about distributing or installing stuff, but iteratively changing and testing code which needs to work on 2.6+, 3.2 and 3.3+. If the 2.x code depends on having u'xxx' literals, then 3.2 testing will potentially involve running a fixer on all files in the project every time a change is made, writing to a separate directory, or else a fixer which is integrated into the editing environment so it knows what changed. This is painful, and what motivated PEP 314 in the first place - which seems ironic. The PEP 314 approach seems to assume that that if things work on 3.3, they will work on 3.2/3.1/3.0 without any changes other than replacing u'xxx' with 'xxx'. In other words, you aren't supposed to want to e.g. test 3.2 and 3.3 iteratively, using a workflow which intersperses edits with running tests using 3.2 and running tests with 3.3. In any case, a single code base seems not to be possible across 2.6+/3.0/3.1/3.2/3.3+ using the PEP 314 approach, though of course one will be possible for just 2.6+/3.3+. Early adopters of 3.x seem to be penalised by this approach: I for one will try to use the unicode_literals approach wherever I can. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a frozendict builtin type
On 28 February 2012 12:07, Mark Shannon wrote: >>> frozendict helps also in threading and multiprocessing. >> >> How so? > > Inter process/task communication requires copying. Inter/intra thread > communication uses reference semantics. To ensure these are the same, > the objects used in communication must be immutable. Does that imply that in a frozendict, the *values* as well as the *keys* must be immutable? Isn't that a pretty strong limitation (and hence, does it not make frozendicts a lot less useful than they might otherwise be)? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a frozendict builtin type
>> A frozendict can be used as a member of a set or as a key in a dictionary. >> >> For example, frozendict is indirectly needed when you want to use an >> object as a key of a dict, whereas one attribute of this object is a >> dict. > > It isn't. You just have to define __hash__ correctly. Define __hash__ on a mutable object can be surprising. Or do you mean that you deny somehow the modification of the dict attribute, and convert the dict to a immutable object before hashing it? >> frozendict helps also in threading and multiprocessing. > > How so? For example, you don't need a lock to read the frozendict content, because you cannot modify the content. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a frozendict builtin type
Updated patch and more justifications.
New patch:
- dict doesn't inherit from frozendict anymore
- frozendict is a subclass of collections.abc.Mutable
- more tests
> * frozendict.__hash__ computes hash(frozenset(self.items())) and
> caches the result is its private hash attribute
hash(frozenset(self.items())) is preferred over
hash(sorted(self.items())) because keys and values may be unorderable.
frozenset() is faster than sorted(): O(n) vs O(n*log(n)).
frozendict hash doesn't care of the item order creation:
>>> a=frozendict.fromkeys('ai')
>>> a
frozendict({'a': None, 'i': None})
>>> b=frozendict.fromkeys('ia')
>>> b
frozendict({'i': None, 'a': None})
>>> hash(a) == hash(b)
True
>>> a == b
True
>>> tuple(a.items()) == tuple(b.items())
False
frozendict supports unorderable keys and values:
>>> hash(frozendict({b'abc': 1, 'abc': 2}))
935669091
>>> hash(frozendict({1: b'abc', 2: 'abc'}))
1319859033
> * Add a frozendict abstract base class to collections?
I realized that Mapping already exists and so the following patch is enough:
+Mapping.register(frozendict)
> See also the PEP 351.
I read the PEP and the email explaining why it was rejected.
Just to be clear: the PEP 351 tries to freeze an object, try to
convert a mutable or immutable object to an immutable object. Whereas
my frozendict proposition doesn't convert anything: it just raises a
TypeError if you use a mutable key or value.
For example, frozendict({'list': ['a', 'b', 'c']}) doesn't create
frozendict({'list': ('a', 'b', 'c')}) but raises a TypeError.
Victor
diff --git a/Include/dictobject.h b/Include/dictobject.h
--- a/Include/dictobject.h
+++ b/Include/dictobject.h
@@ -84,10 +84,14 @@ struct _dictobject {
PyDictEntry *ma_table;
PyDictEntry *(*ma_lookup)(PyDictObject *mp, PyObject *key, Py_hash_t hash);
PyDictEntry ma_smalltable[PyDict_MINSIZE];
+
+/* only used by frozendict */
+Py_hash_t hash;
};
#endif /* Py_LIMITED_API */
PyAPI_DATA(PyTypeObject) PyDict_Type;
+PyAPI_DATA(PyTypeObject) PyFrozenDict_Type;
PyAPI_DATA(PyTypeObject) PyDictIterKey_Type;
PyAPI_DATA(PyTypeObject) PyDictIterValue_Type;
PyAPI_DATA(PyTypeObject) PyDictIterItem_Type;
@@ -97,7 +101,12 @@ PyAPI_DATA(PyTypeObject) PyDictValues_Ty
#define PyDict_Check(op) \
PyType_FastSubclass(Py_TYPE(op), Py_TPFLAGS_DICT_SUBCLASS)
+#define PyFrozenDict_Check(op) \
+(PyDict_Check(op) || \
+ Py_TYPE(op) == &PyFrozenDict_Type || \
+ PyType_IsSubtype(Py_TYPE(op), &PyFrozenDict_Type))
#define PyDict_CheckExact(op) (Py_TYPE(op) == &PyDict_Type)
+#define PyFrozenDict_CheckExact(op) (Py_TYPE(op) == &PyFrozenDict_Type)
#define PyDictKeys_Check(op) (Py_TYPE(op) == &PyDictKeys_Type)
#define PyDictItems_Check(op) (Py_TYPE(op) == &PyDictItems_Type)
#define PyDictValues_Check(op) (Py_TYPE(op) == &PyDictValues_Type)
diff --git a/Lib/collections/abc.py b/Lib/collections/abc.py
--- a/Lib/collections/abc.py
+++ b/Lib/collections/abc.py
@@ -401,6 +401,7 @@ class Mapping(Sized, Iterable, Container
def __ne__(self, other):
return not (self == other)
+Mapping.register(frozendict)
class MappingView(Sized):
diff --git a/Lib/test/test_dict.py b/Lib/test/test_dict.py
--- a/Lib/test/test_dict.py
+++ b/Lib/test/test_dict.py
@@ -788,11 +788,66 @@ class Dict(dict):
class SubclassMappingTests(mapping_tests.BasicTestMappingProtocol):
type2test = Dict
+
+class FrozenDictTests(unittest.TestCase):
+def test_constructor(self):
+# unorderable keys and values
+frozendict(x=b'abc', y='abc')
+frozendict({b'abc': 1, 'abc': 2})
+
+# hashable keys and values
+class Hashable:
+pass
+for hashable in (5, 1.0, "abc", (1, 2), Hashable()):
+frozendict(key=hashable)
+frozendict(hashable=0)
+
+# not hashable keys or values
+class NotHashable:
+def __hash__(self):
+raise TypeError("not hashable")
+for not_hashable in ([], {}, NotHashable()):
+self.assertRaises(TypeError, frozendict, key=not_hashable)
+self.assertRaises(TypeError, frozendict, [(not_hashable, 0)])
+self.assertRaises(TypeError, frozendict, [(0, not_hashable)])
+
+def test_copy(self):
+self.assertIsInstance(frozendict().copy(),
+ frozendict)
+self.assertEqual(frozendict(x=1, y=2).copy(),
+ frozendict(x=1, y=2))
+
+def test_dir(self):
+self.assertEqual(
+set(dir(frozendict)) - set(dir(object)),
+{'__contains__', '__getitem__', '__iter__', '__len__',
+ 'copy', 'fromkeys', 'get', 'items', 'keys', 'values'})
+
+def test_fromkeys(self):
+self.assertEqual(frozendict.fromkeys('ai'),
+ frozendict(a=None, i=None))
+
+def test_hash(self):
+self.assertEqual(hash(frozendict()),
+ hash(
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On Tue, Feb 28, 2012 at 9:52 PM, Antoine Pitrou wrote: > On Tue, 28 Feb 2012 21:42:54 +1000 > Nick Coghlan wrote: >> But the existing approaches require that, in order to be >> forward compatible with Python 3, a program must be made *worse* in >> Python 2 (i.e. harder to read and harder to write correctly for >> someone that hasn't learned Python 3 yet). > > Wrong. The separate branches approach allows you to have a clean > Python 3 codebase without crippling the Python 2 codebase. > Of course that approach was downplayed from the start in favour of > using 2to3 on a single codebase, and now we discover that this approach > is cumbersome. If you're using separate branches, then your Python 2 code isn't being made forward compatible with Python 3. Yes, it avoids making your Python 2 code uglier, but it means maintaining two branches in parallel until you drop Python 2 support. You've once again raised the barrier to entry: either people contribute two patches, or they accept that their patch may languish until someone else writes the patch for the other version. Again, as with 2to3, that approach obviously *works* (we've done it ourselves for years with the standard library), but it's hardly a low friction approach to porting. That's all PEP 414 is about - lowering the friction of porting to Python 3. Is it *necessary*? No, there are already enough successful ports to prove that, if sufficiently motivated, porting to Python 3 is feasible with the current toolset. However, that's the wrong question. The right question is "Does PEP 414 make porting substantially *easier*, by significantly reducing the volume of code that needs to change in order to attain Python 3 compatibility?". And the answer to *that* question is "Absolutely." Porting the web frameworks themselves to Python 3 is only the first step in migrating those ecosystems to Python 3, and because the web APIs exposed by those frameworks are so heavily Unicode based this is an issue that will hit pretty much every Python web app and library on the planet. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a frozendict builtin type
On Tue, 28 Feb 2012 12:07:32 + Mark Shannon wrote: > Antoine Pitrou wrote: > > On Tue, 28 Feb 2012 12:45:54 +0100 > > Victor Stinner wrote: > >>> I think you need to elaborate on your use cases further, ... > >> A frozendict can be used as a member of a set or as a key in a dictionary. > >> > >> For example, frozendict is indirectly needed when you want to use an > >> object as a key of a dict, whereas one attribute of this object is a > >> dict. > > > > It isn't. You just have to define __hash__ correctly. > > > >> frozendict helps also in threading and multiprocessing. > > > > How so? > > Inter process/task communication requires copying. Inter/intra thread > communication uses reference semantics. To ensure these are the same, > the objects used in communication must be immutable. You just need them to be practically constant. No need for an immutable type in the first place. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On Tue, Feb 28, 2012 at 10:10 PM, Vinay Sajip wrote: > If the 2.x code depends on having u'xxx' literals, then 3.2 testing will > potentially involve running a fixer on all files in the project every time a > change is made, writing to a separate directory, or else a fixer which is > integrated into the editing environment so it knows what changed. This is > painful, and what motivated PEP 314 in the first place - which seems ironic. No, the real idea behind PEP 414 is that most ports that rely on it simply won't support 3.2 - they will only target 3.3+. The u"" fixer will just be one more tool in the arsenal of those that *do* want to support 3.2 (either because they want to target Ubuntu's LTS 3.2 stack, or for their own reasons). All of the other alternatives (such as separate branches or the unicode_literals future import) will also remain available to them. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Le mardi 28 février 2012 à 22:14 +1000, Nick Coghlan a écrit : > If you're using separate branches, then your Python 2 code isn't being > made forward compatible with Python 3. Yes, it avoids making your > Python 2 code uglier, but it means maintaining two branches in > parallel until you drop Python 2 support. IMO, maintaining two branches shouldn't be much more work than maintaining hacks so that a single codebase works with two different programming languages. > You've once again raised the > barrier to entry: either people contribute two patches, or they accept > that their patch may languish until someone else writes the patch for > the other version. Again that's wrong. If you cleverly use 2to3 to port between branches, patches only have to be written against the 2.x version. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a frozendict builtin type
Hi, I don't know if an implementation of the frozendict actually exists, but if anyone is planning on writing one then can I suggest that they take a look at my new dict implementation: http://bugs.python.org/issue13903 https://bitbucket.org/markshannon/cpython_new_dict/ Making dicts immutable (at the C level) is quite easy with my new implementation. Cheers, Mark. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a frozendict builtin type
On Tue, 28 Feb 2012 13:14:15 +0100 Victor Stinner wrote: > > > See also the PEP 351. > > I read the PEP and the email explaining why it was rejected. I think you should write a separate PEP and explain the use cases clearly. cheers Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a frozendict builtin type
Victor Stinner wrote:
>> See also the PEP 351.
>
> I read the PEP and the email explaining why it was rejected.
>
> Just to be clear: the PEP 351 tries to freeze an object, try to
> convert a mutable or immutable object to an immutable object. Whereas
> my frozendict proposition doesn't convert anything: it just raises a
> TypeError if you use a mutable key or value.
>
> For example, frozendict({'list': ['a', 'b', 'c']}) doesn't create
> frozendict({'list': ('a', 'b', 'c')}) but raises a TypeError.
I fail to see the use case you're trying to address with this
kind of frozendict().
The purpose of frozenset() is to be able to use a set as dictionary
key (and to some extent allow for optimizations and safe
iteration). Your implementation can be used as dictionary key as well,
but why would you want to do that in the first place ?
If you're thinking about disallowing changes to the dictionary
structure, e.g. in order to safely iterate over its keys or items,
"freezing" the keys is enough.
Requiring the value objects not to change is too much of a restriction
to make the type useful in practice, IMHO.
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Feb 28 2012)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
2012-02-13: Released eGenix pyOpenSSL 0.13http://egenix.com/go26
2012-02-09: Released mxODBC.Zope.DA 2.0.2 http://egenix.com/go25
2012-02-06: Released eGenix mx Base 3.2.3 http://egenix.com/go24
::: Try our new mxODBC.Connect Python Database Interface for free !
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Il 28 febbraio 2012 13:19, Antoine Pitrou ha scritto: > > Le mardi 28 février 2012 à 22:14 +1000, Nick Coghlan a écrit : >> If you're using separate branches, then your Python 2 code isn't being >> made forward compatible with Python 3. Yes, it avoids making your >> Python 2 code uglier, but it means maintaining two branches in >> parallel until you drop Python 2 support. > > IMO, maintaining two branches shouldn't be much more work than > maintaining hacks so that a single codebase works with two different > programming languages. Would that mean distributing 2 separate tarballs? How would tools such as easy_install and pip work in respect of that? Is there a naming convention they can rely on? --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ http://code.google.com/p/pysendfile/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On Tue, Feb 28, 2012 at 10:19 PM, Antoine Pitrou wrote: > > Le mardi 28 février 2012 à 22:14 +1000, Nick Coghlan a écrit : >> If you're using separate branches, then your Python 2 code isn't being >> made forward compatible with Python 3. Yes, it avoids making your >> Python 2 code uglier, but it means maintaining two branches in >> parallel until you drop Python 2 support. > > IMO, maintaining two branches shouldn't be much more work than > maintaining hacks so that a single codebase works with two different > programming languages. Aside from the unicode literal problem, I find that the Python 2.6+/3.2+ subset is still a fairly nice language for an application level web program. Most of the rest of the bytes/text ugliness is hidden away below the framework layer where folks like Chris, Armin and Jacob have to deal with it, but it doesn't affect me as a framework user. >> You've once again raised the >> barrier to entry: either people contribute two patches, or they accept >> that their patch may languish until someone else writes the patch for >> the other version. > > Again that's wrong. If you cleverly use 2to3 to port between branches, > patches only have to be written against the 2.x version. Apparently *you* know how to do that, but I don't. If I, as a CPython core developer, don't know how to do that, how is it reasonable to expect J. Random Hacker to become a Python 3 porting export? PEP 414 is all about lowering the barrier to entry for successful Python 3 ports. OK, fine some very clever people have invested a lot of time in finding ways to deal with the status quo that make it less painful. That doesn't mean it isn't painful - it just means the early adopters have steeled themselves against the pain and learned to suck it up and cope. Now that we've discovered some of the key sources of pain, we can live with a few pragmatic concessions in the purity of Python 3's language definition to ease the transition for the vast number of Python 3 ports which have yet to begin. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Nick Coghlan gmail.com> writes: > > On Tue, Feb 28, 2012 at 10:10 PM, Vinay Sajip yahoo.co.uk> wrote: > > If the 2.x code depends on having u'xxx' literals, then 3.2 testing will > > potentially involve running a fixer on all files in the project every time a > > change is made, writing to a separate directory, or else a fixer which is > > integrated into the editing environment so it knows what changed. This is > > painful, and what motivated PEP 314 in the first place - which seems ironic. > > No, the real idea behind PEP 414 is that most ports that rely on it > simply won't support 3.2 - they will only target 3.3+. Well, yes in that the PEP will only be implemented in 3+, but the motivation was to make a single codebase easier to achieve. It does that if you take the narrow view of 2.6+/3.3+, but not if you factor 3.2 into the mix. Maybe 3.2 adoption is too low for us to worry about here, but I for one certainly wish it hadn't been relegated to being a 2nd-class citizen. > The u"" fixer will just be one more tool in the arsenal of those that > *do* want to support 3.2 (either because they want to target Ubuntu's > LTS 3.2 stack, or for their own reasons). All of the other > alternatives (such as separate branches or the unicode_literals future > import) will also remain available to them. Right, I get that - as I said, unicode_literals is my preferred path of the options available. It's a shame to see this sort of Balkanisation, though. For example, if Django retains u'xxx' literals (even though I've ported it using unicode_literals, they may choose a different path officially), users wanting to work with it using 2.6/2.7/3.2/3.3 (as I do now) are SOL as far as a single codebase is concerned. Of course, when you're working on your own project, you can call the shots. But problems can arise if you have to work with an external project, as many of us frequently do. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
v.loewis.de> writes: > > > A couple of people have said that 'native string' is spelt 'str', but I'm > > not > > sure that's the right answer. For example, 2.x's cString.StringIO > > expects native strings, not Unicode: > > Your counter-example is non-ASCII characters/bytes. I doubt that this > is a valid > use case; in a "native" string, these shouldn't occur (i.e. native > strings should > always be ASCII), since the semantics of non-ASCII changes drastically between > 2.x and 3.x. So whoever defines some API to take "native" strings > can't have defined > a valid use of non-ASCII in that interface. It might not be a valid usage, but the 2.x ecosystem has numerous occurrences of invalid usages, which tend to crop up when porting because of 3.x's increased strictness. In the example I gave, cStringIO.StringIO should be able to cope with text strings, but doesn't. Of course there are StringIO.StringIO and io.StringIO in 2.6, but when porting a project, you can't be sure which of these you might run into. > Indeed it should. If there is a known application of non-ASCII native strings, > I surely would like to know what that is. I can't think of a specific instance off-hand, but I seem to recall having problems with some of the cookie APIs insisting on native strings (rather than text, which is validated against ASCII where appropriate). Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
28.02.12 14:14, Nick Coghlan написав(ла): > However, that's the wrong question. > The right question is "Does PEP 414 make porting substantially > *easier*, by significantly reducing the volume of code that needs to > change in order to attain Python 3 compatibility?". Another pertinent question: "What are disadvantages of PEP 414 is adopted?" ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On Tue, 28 Feb 2012 22:21:11 +1000, Nick Coghlan wrote: > On Tue, Feb 28, 2012 at 10:10 PM, Vinay Sajip wrote: > > If the 2.x code depends on having u'xxx' literals, then 3.2 testing will > > potentially involve running a fixer on all files in the project every time a > > change is made, writing to a separate directory, or else a fixer which is > > integrated into the editing environment so it knows what changed. This is > > painful, and what motivated PEP 314 in the first place - which seems ironic. > > No, the real idea behind PEP 414 is that most ports that rely on it > simply won't support 3.2 - they will only target 3.3+. Hmm. It seems to me that this argument implies that PEP 414 is just as likely to *slow down* adoption of Python3 as it is to speed it up, since if this issue is as big a barrier as indicated, many potential porters may choose to wait until OS vendors are supporting 3.3 widely before starting their ports. We are clearly expecting that the reality is that the impact will be at worse neutral, and hopefully positive. --David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On 28/02/2012 14.19, Antoine Pitrou wrote: Le mardi 28 février 2012 à 22:14 +1000, Nick Coghlan a écrit : If you're using separate branches, then your Python 2 code isn't being made forward compatible with Python 3. Yes, it avoids making your Python 2 code uglier, but it means maintaining two branches in parallel until you drop Python 2 support. IMO, maintaining two branches shouldn't be much more work than maintaining hacks so that a single codebase works with two different programming languages. +10 For every CPython bug that I fix I first apply the patch on 2.7, then on 3.2 and then on 3.3. Most of the time I don't even need to change anything while applying the patch to 3.2, sometimes I have to do some trivial fixes. This is also true for another personal 12kloc project* where I'm using the two-branches approach. For me, the costs of having two branches are: 1) a one-time conversion when the Python3-compatible branch is created (can be done easily with 2to3); 2) merging the fix I apply to the Python2 branch (and with modern DVCS this is not really an issue). With the shared code base approach, the costs are: 1) a one-time conversion to "fix" the code base and make it run on both 2.x and 3.x; 2) keep using and having to deal with hacks in order to keep it running. With the first approach, you also have two clean and separate code bases, with no hacks; when you stop using Python 2, you end up with a clean Python 3 branch. The one-time conversion also seems easier in the first case. (Note: there are also other costs -- e.g. releasing -- that I haven't considered because they don't affect me personally, but I'm not sure they are big enough to make the two-branches approach worse.) You've once again raised the barrier to entry: either people contribute two patches, or they accept that their patch may languish until someone else writes the patch for the other version. Again that's wrong. If you cleverly use 2to3 to port between branches, patches only have to be written against the 2.x version. After the initial conversion of the code base, the fixes are mostly trivial, so people don't need to write two patches (most of the patches we get for CPython are either against 2.7 or 3.2, and sometimes they even apply clearly to both). Using 2to3 to generate the 3.x code automatically for every change applied to the 2.x branch (or to convert everything when a new package is installed) sounds wrong to me. I wouldn't trust generated code even if 2to3 was a better tool. That said, I successfully used the shared code base approach with print_function, unicode_literals, and a couple of try/except for the imports for a few one-file scripts (for 2.7/3.2) that I wrote recently. TL;DR the two-branches approach usually works better (at least IME) than the shared code base approach, doesn't necessarily require more work, and doesn't need ugly hacks to work. * in this case all the string literals I had were already text (rather than bytes) and even without using unicode_literals they worked out of the box when I moved the code to 3.x. There was however a place where it didn't work, and that turned out to be a bug even in Python 2 because I was mixing bytes and text. Best Regards, Ezio Melotti Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Spreading the Python 3 religion (was Re: PEP 414 - Unicode Literals for Python 3)
On Feb 28, 2012, at 08:41 AM, R. David Murray wrote: >Hmm. It seems to me that this argument implies that PEP 414 is just >as likely to *slow down* adoption of Python3 as it is to speed it up, >since if this issue is as big a barrier as indicated, many potential >porters may choose to wait until OS vendors are supporting 3.3 widely >before starting their ports. We are clearly expecting that the reality >is that the impact will be at worse neutral, and hopefully positive. If PEP 414 helps some projects migrate to Python 3, great. But I really hope we as a community don't perpetuate the myth that you cannot port to Python 3 without this, and I hope that we spend as much effort on educating other Python developers on how to port to Python 3 *right now* supporting Python 2.6, 2.7, and 3.2. That's the message we should be spreading and we should be helping developers understand exactly how to do this effectively, among the many great options that exist today. Only in the most extreme cases or the most inertially challenged projects should we say "wait for Python 3.3". Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a frozendict builtin type
M.-A. Lemburg wrote:
Victor Stinner wrote:
See also the PEP 351.
I read the PEP and the email explaining why it was rejected.
Just to be clear: the PEP 351 tries to freeze an object, try to
convert a mutable or immutable object to an immutable object. Whereas
my frozendict proposition doesn't convert anything: it just raises a
TypeError if you use a mutable key or value.
For example, frozendict({'list': ['a', 'b', 'c']}) doesn't create
frozendict({'list': ('a', 'b', 'c')}) but raises a TypeError.
I fail to see the use case you're trying to address with this
kind of frozendict().
The purpose of frozenset() is to be able to use a set as dictionary
key (and to some extent allow for optimizations and safe
iteration). Your implementation can be used as dictionary key as well,
but why would you want to do that in the first place ?
Because you have a mapping, and want to use a dict for speedy, convenient
lookups. Sometimes your mapping involves the key being a string, or an int, or
a tuple, or a set, and Python makes it easy to use that in a dict. Sometimes
the key is itself a mapping, and Python makes it very difficult.
Just google on "python frozendict" or "python immutabledict" and you will find
that this keeps coming up time and time again, e.g.:
http://www.cs.toronto.edu/~tijmen/programming/immutableDictionaries.html
http://code.activestate.com/recipes/498072-implementing-an-immutable-dictionary/
http://code.activestate.com/recipes/414283-frozen-dictionaries/
http://bob.pythonmac.org/archives/2005/03/04/frozendict/
http://python.6.n6.nabble.com/frozendict-td4377791.html
http://www.velocityreviews.com/forums/t648910-does-python3-offer-a-frozendict.html
http://stackoverflow.com/questions/2703599/what-would-be-a-frozen-dict
If you're thinking about disallowing changes to the dictionary
structure, e.g. in order to safely iterate over its keys or items,
"freezing" the keys is enough.
Requiring the value objects not to change is too much of a restriction
to make the type useful in practice, IMHO.
It's no more of a limitation than the limitation that strings can't change.
Frozendicts must freeze the value as well as the key. Consider the toy
example, mapping food combinations to calories:
d = { {appetizer => fried fish, main => double burger, drink => cola}: 5000,
{appetizer => None, main => green salad, drink => tea}: 200,
}
(syntax is only for illustration purposes)
Clearly the hash has to take the keys and values into account, which means
that both the keys and values have to be frozen.
(Values may be mutable objects, but then the frozendict can't be hashed --
just like tuples can't be hashed if any item in them is mutable.)
--
Steven
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Armin Ronacher active-4.com> writes:
> If by str() you mean using "str('x')" as replacement for 'x' in both 2.x
> and 3.x with __future__ imports as a replacement for native string
> literals, please mention why this is better than u(), s(), n() etc. It
> would be equally slow than a custom wrapper function and it would not
> support non-ascii characters.
Well, you can give it any name you like, but
if PY3:
def n(literal): return literal
else:
# used along with "from __future__ import unicode_literals" in client code
def n(literal): return literal.encode('utf-8')
will support non-ASCII characters. You have not provided anything other than a
microbenchmark regarding performance - as you are well aware, this does not
illustrate what the performance might be on a representative workload. While
there might be the odd percent in it, I didn't see any major degradation when
running the Django test suite - which I would think is a more balanced workload
than just benchmarking the wrapper. Of course, I don't claim to have studied the
performance characteristics closely - I haven't.
AFAICT, the incidence of native strings in an application is not that great (of
course there can be pathological cases), so the number of calls to n() or
whatever it's called is unlikely to have any significant impact. Even when I was
using u() calls with the 2.5 port - which are of course much more common - the
performance impact was unremarkable.
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On Feb 28, 2012, at 10:49 PM, Nick Coghlan wrote: >On Tue, Feb 28, 2012 at 10:19 PM, Antoine Pitrou wrote: >> Again that's wrong. If you cleverly use 2to3 to port between branches, >> patches only have to be written against the 2.x version. > >Apparently *you* know how to do that, but I don't. If I, as a CPython >core developer, don't know how to do that, how is it reasonable to >expect J. Random Hacker to become a Python 3 porting export? They don't need to, but *we* do, and it's incumbent on us to educate our users. I strongly believe that *now* is the time to be porting to Python 3. It's critical to the long-term health of Python. It's up to us to learn the strategies for accomplishing this, spread the message that it is not only possible, but usually easy (and yes even, from my own experience, fun!). Oh and here's how in three easy steps, 1, 2, 3. I've blogged about my own porting experiences extensively. My strategies may not work for everyone, but they will work for a great many projects. If they work for yours, spread the word. If they don't, figure out something better, write about it, and spread the word. We really need to stop saying that porting to Python 3 is hard, or should be delayed. It's not in the vast majority of cases. Yes, there are warts, and we should continue to improve Python 3 so it gets easier, but by no means is it impossible for most code to be working very nicely on Python 3 today. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Nick Coghlan gmail.com> writes: > tools. But the existing approaches require that, in order to be > forward compatible with Python 3, a program must be made *worse* in > Python 2 (i.e. harder to read and harder to write correctly for > someone that hasn't learned Python 3 yet). Restoring unicode literal How so? In the case of string literals, are you saying that it's worse in that you use 'xxx' instead of u'xxx' for text, and have to add a unicode_literals import? I don't feel that either of those make a 2.x program worse. > support in 3.3 is a pragmatic step that allows a lot of code to *just > work* on Python 3. Most 2.6+ code that still doesn't work on Python 3 > even after this change will be made *better* (or at least not made > substantially worse) by the additional changes necessary for forward > compatibility. Remember, the PEP advocates what it does in the name of a single codebase. If you want to (or have to) support 3.2 in addition to 3.3, 2.6, 2.7, the PEP does not work for you. It only works for you if you're interested in 2.6+ and 3.3+. > Unicode literals are somewhat unique in their impact on porting > efforts, as they show up *everywhere* in Unicode correct code in > Python 2. The diffs that will be needed to correctly tag bytestrings > in such code under Python 2 are tiny compared to those that would be > needed to strip the u"" prefixes. But that's a one-time operation using a lib2to3 fixer, and even for a big project like Django, we're not talking about a lot of time spent on this (at least, in my experience). Having a good test suite helps catch those byte-string cases more easily, of course. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Spreading the Python 3 religion (was Re: PEP 414 - Unicode Literals for Python 3)
On Tue, Feb 28, 2012 at 09:53, Barry Warsaw wrote: > On Feb 28, 2012, at 08:41 AM, R. David Murray wrote: > > >Hmm. It seems to me that this argument implies that PEP 414 is just > >as likely to *slow down* adoption of Python3 as it is to speed it up, > >since if this issue is as big a barrier as indicated, many potential > >porters may choose to wait until OS vendors are supporting 3.3 widely > >before starting their ports. We are clearly expecting that the reality > >is that the impact will be at worse neutral, and hopefully positive. > > If PEP 414 helps some projects migrate to Python 3, great. > > But I really hope we as a community don't perpetuate the myth that you > cannot > port to Python 3 without this, and I hope that we spend as much effort on > educating other Python developers on how to port to Python 3 *right now* > supporting Python 2.6, 2.7, and 3.2. That's the message we should be > spreading and we should be helping developers understand exactly how to do > this effectively, among the many great options that exist today. Only in > the > most extreme cases or the most inertially challenged projects should we say > "wait for Python 3.3". > Well, when the code is committed I will update the porting HOWTO and push the __future__ imports first since they cover more versions of Python (i.e. Python 3.2). But I will mention the options that skip the __future__ imports for those that choose not to use them (or have already done the work of using the u prefix in their code). Plus that doc probably will need an update of caveats that seem to bit everyone (e.g. the str(bytes) thing which I didn't know about) when trying to do source-compatible versions. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Antoine Pitrou pitrou.net> writes: > Wrong. The separate branches approach allows you to have a clean > Python 3 codebase without crippling the Python 2 codebase. There may be warts in a single codebase (you usually can't have something for nothing), but it's not necessarily *crippled* when running in 2.x. Of course two branches allow you to have a no-compromise approach for the code style, but you might pay for that in time spent doing merges etc. > Note that 2to3 is actually helpful when you choose the dual branches > approach, and it isn't a serial dependency in that case. > (see https://bitbucket.org/pitrou/t3k/) Yes, 2to3 is very useful when doing an initial porting exercise. I've used it just once in each port I've done. It also works well for a single codebase approach, only I just follow its advice rather than letting it do the conversion automatically. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Il 28 febbraio 2012 15:20, Ezio Melotti ha scritto: > On 28/02/2012 14.19, Antoine Pitrou wrote: >> >> Le mardi 28 février 2012 à 22:14 +1000, Nick Coghlan a écrit : >>> >>> If you're using separate branches, then your Python 2 code isn't being >>> made forward compatible with Python 3. Yes, it avoids making your >>> Python 2 code uglier, but it means maintaining two branches in >>> parallel until you drop Python 2 support. >> >> IMO, maintaining two branches shouldn't be much more work than >> maintaining hacks so that a single codebase works with two different >> programming languages. > > > +10 > > For every CPython bug that I fix I first apply the patch on 2.7, then on 3.2 > and then on 3.3. > Most of the time I don't even need to change anything while applying the > patch to 3.2, sometimes I have to do some trivial fixes. This is also true > for another personal 12kloc project* where I'm using the two-branches > approach. > > For me, the costs of having two branches are: > 1) a one-time conversion when the Python3-compatible branch is created (can > be done easily with 2to3); > 2) merging the fix I apply to the Python2 branch (and with modern DVCS this > is not really an issue). > > With the shared code base approach, the costs are: > 1) a one-time conversion to "fix" the code base and make it run on both 2.x > and 3.x; > 2) keep using and having to deal with hacks in order to keep it running. > > With the first approach, you also have two clean and separate code bases, > with no hacks; when you stop using Python 2, you end up with a clean Python > 3 branch. > The one-time conversion also seems easier in the first case. > > (Note: there are also other costs -- e.g. releasing -- that I haven't > considered because they don't affect me personally, but I'm not sure they > are big enough to make the two-branches approach worse.) They are. With that kind of approach you're basically forced to include the python version number as part of the tarball name (e.g. foo-0.3.1-py2.tar.gz and foo-0.3.1-py3.tar.gz). Just to name one, that means "foo" can't be installed via pip/easy_install. Regards, --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ http://code.google.com/p/pysendfile/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Porting Stories (was PEP 414 - Unicode Literals for Python 3)
On Tue, Feb 28, 2012 at 09:04, Barry Warsaw wrote: > We really need to stop saying that porting to Python 3 is hard, or should be > delayed. It's not in the vast majority of cases. Yes, there are warts, and > we should continue to improve Python 3 so it gets easier, but by no means is > it impossible for most code to be working very nicely on Python 3 today. I've been singing and dancing about the ease of porting for a while now, but it's mostly thanks to the fact that I never had to do any Unicode tomfoolery. Now with this PEP, the game gets easier for a lot more people. Does anyone have a good porting experience they'd like to share, which I could maybe use as a PR effort for us? Barry, I know you wrote some pretty solid coverage of your DBus port. Anyone else? Personal projects or work stuff (assuming its ok to share). blog.python.org has been asleep for a while and a good porting testimonial might be a way to jumpstart it, or I can get it on the python.org front page. If you have anything to share on that front, please contact me directly. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Spreading the Python 3 religion (was Re: PEP 414 - Unicode Literals for Python 3)
On Feb 28, 2012, at 10:23 AM, Brett Cannon wrote: >Well, when the code is committed I will update the porting HOWTO and push >the __future__ imports first since they cover more versions of Python (i.e. >Python 3.2). But I will mention the options that skip the __future__ >imports for those that choose not to use them (or have already done the >work of using the u prefix in their code). Plus that doc probably will need >an update of caveats that seem to bit everyone (e.g. the str(bytes) thing >which I didn't know about) when trying to do source-compatible versions. See, I think the emphasis should be on using the future imports and unadorning your unicode literals. Forget about this PEP except as a footnote. This strategy works today for most packages. You might think that this is ugly, but really, I think that doesn't matter (or maybe better: get over it! :). Definitely don't let that stop you from porting *now*. In the small minority of cases where this strategy cannot work for you (and I admit to not really understanding what those cases are), then the footnote about the reintroduction of the u-prefix should be enough. And yes, the str(bytes) thing is a pain, but it too can be worked around, and is such a minor wart that it should not delay your porting efforts. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Serhiy Storchaka gmail.com> writes: > Another pertinent question: "What are disadvantages of PEP 414 is adopted?" It's moot, but as I see it: the purpose of PEP 414 is to facilitate a single codebase across 2.x and 3.x. However, it only does this if your 3.x interest is 3.3+. If you also want to or need to support 3.0 - 3.2, it makes your workflow more painful, because you can't run tests on 2.x or 3.3 and then run them on 3.2 without an intermediate source conversion step - just like the 2to3 step that people find painful when it's part of maintenance workflow, and which in part prompted the PEP in the first place. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Vinay Sajip wrote: Serhiy Storchaka gmail.com> writes: Another pertinent question: "What are disadvantages of PEP 414 is adopted?" It's moot, but as I see it: the purpose of PEP 414 is to facilitate a single codebase across 2.x and 3.x. However, it only does this if your 3.x interest is 3.3+. If you also want to or need to support 3.0 - 3.2, it makes your workflow more painful, because you can't run tests on 2.x or 3.3 and then run them on 3.2 without an intermediate source conversion step - just like the 2to3 step that people find painful when it's part of maintenance workflow, and which in part prompted the PEP in the first place. I don't think it's fair to say it makes it *more* painful. Fair to say it doesn't make it less painful, but adding u'' to 3.3+ doesn't make it harder to port from 2.x to 3.1+. You're merely no better off with it than without it. Aside: in my opinion, people shouldn't actively support 3.0, or at least not advertise support for it, as it was end-of-lifed on the release of 3.1. As I see it, it is best to pretend that 3.0 never existed :) -- Steven ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Ezio Melotti gmail.com> writes: > For every CPython bug that I fix I first apply the patch on 2.7, then on > 3.2 and then on 3.3. > Most of the time I don't even need to change anything while applying the > patch to 3.2, sometimes I have to do some trivial fixes. This is also > true for another personal 12kloc project* where I'm using the > two-branches approach. I hear what you say about the personal project, but IMO CPython is atypical (as far as this discussion is concerned), not least because it's not a pure-Python project. > For me, the costs of having two branches are: > 1) a one-time conversion when the Python3-compatible branch is created > (can be done easily with 2to3); Yes, but the amount of ease is project-dependent. For example, 2to3 wraps values() method calls with list(), which is a reasonable thing to do for dicts; when presented Django's querysets, which have a values() method which should not be wrapped, then you have to go through and sort things out. I'm not knocking 2to3, which I think is great. Just that things go well sometimes, and less well at other times, > With the shared code base approach, the costs are: > 1) a one-time conversion to "fix" the code base and make it run on > both 2.x and 3.x; > 2) keep using and having to deal with hacks in order to keep it running. Which hacks do you mean, if you're only interested in 2.6+? > With the first approach, you also have two clean and separate code > bases, with no hacks; when you stop using Python 2, you end up with a > clean Python 3 branch. > The one-time conversion also seems easier in the first case. > > (Note: there are also other costs -- e.g. releasing -- that I haven't > considered because they don't affect me personally, but I'm not sure > they are big enough to make the two-branches approach worse.) I don't believe there's a one-size-fits-all. The two branches approach is appealing, and I have no quarrel with it: but I contend that big projects like Django would be reluctant to switch, or take much longer to switch to 3.x, if they had to maintain separate branches. Given the size of their user community, they have to follow strict release procedures, which (even with just running on 2.x) smaller projects can be more relaxed about. You forgot to mention the part which is most time-consuming day-to-day: making changes and testing. For the two-branch approach, its 1. Change on 2.x 2. Test on 2.x 3. Commit on 2.x 4. Merge to 3.x 5. Possibly change on 3.x 6. Test on 3.x 7. Commit on 3.x where each "test" step, if failures occur, might take you back to a previous "change" step. For the single codebase, that's 1. Change 2. Test on 2.x 3. Test on 3.x 4. Commit This, to me, is the single big advantage of the single codebase approach, and the productivity improvements outweigh code purity issues which are, in the grand scheme of things, not all that large. Another advantage is DRY: you don't have to worry about forgetting to merge some changes from 2.x to 3.x. Haven't we all been there one time or another? I know I have, though I try not to make a habit of it ;-) > After the initial conversion of the code base, the fixes are mostly > trivial, so people don't need to write two patches (most of the patches > we get for CPython are either against 2.7 or 3.2, and sometimes they > even apply clearly to both). Fixes may be trivial, but new features might not always be so. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Hi Armin, Could you please remove from the PEP the following statement: """As it stands, Python 3 is currently a bad choice for long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3.""" While it may be as such for you, I think it is incorrect for the rest. Moreover, it is harmful for the python 3 adoption, to put such documents on python.org. The python ecosystem is not just limited to WSGI apps, Django and Flask. Yes, we don't have all the packages on pypi support python 3, but many of those are portable within 10 minutes to couple of hours of work (and I did many of such ports for our internal systems.) And many of the essential packages do exist for python 3, like numpy, zeromq etc. I know several sturt-ups, including mine that develop huge commercial applications entirely on python 3. Thanks, -Yury On 2012-02-27, at 5:38 PM, Armin Ronacher wrote: > Hi, > > On 2/27/12 10:18 PM, Terry Reedy wrote: >> I would like to know if you think that this one change is enough to do >> agile development and testing, etc, or whether, as Chris McDonough >> hopes, this is just the first of a series of proposals you have planned. > Indeed I have three other PEPs in the work. The reintroduction of > "except (((ExceptionType),),)", the "<>" comparision operator and the > removal of "nonlocal", the latter to make Python 2.x developers feel > better about themselves. :-) > > > Regards, > Armin > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Steven D'Aprano pearwood.info> writes: > I don't think it's fair to say it makes it *more* painful. Fair to say it > doesn't make it less painful, but adding u'' to 3.3+ doesn't make it harder > to > port from 2.x to 3.1+. You're merely no better off with it than without it. No, it actually does make it *more* painful in some scenarios. Let's say Django decides to move to 3.x using a single codebase starting with 3.3, using PEP 414 to avoid changing u'xxx' in their source code. This is dandy for 3.3, and say I have to work with Django on 2.6, 2.7 and 3.3. Great - I make some changes, I run tests on 2.x, 3.3 - make changes as needed to fix failures, then commit. And on to the next set of changes. Now, suppose I also need to support 3.2, in addition to the other versions. I don't get the same easy workflow I had before: for 3.2, I have to run Armin's hook to remove the u'' prefixes between making changes and running tests, *every time*, but the output will be written to a separate directory, and I may have to maintain a separate test environment there in terms of test data files etc. It's exactly the complaint the PEP makes about having to have 2to3 in the workflow, and how that hurts your productivity! Though the experience may differ in degree because Armin's tool is faster, it's not going to make for a seamless workflow. Especially not if it has to run over all the files in the Django codebase. And if it's going to know only which files have changed and run only on those, how does it propose to do that, independently of my editing tools? Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 415: Implementing PEP 409 differently
Big +1. Indeed, this whole Ellipsis approach is just an awful hack. - Yury On 2012-02-26, at 8:30 PM, Benjamin Peterson wrote: > PEP: 415 > Title: Implementing PEP 409 differently > Version: $Revision$ > Last-Modified: $Date$ > Author: Benjamin Peterson > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 26-Feb-2012 > Post-History: 26-Feb-2012 > > > Abstract > > > PEP 409 allows PEP 3134 exception contexts and causes to be suppressed when > the > exception is printed. This is done using the ``raise exc from None`` > syntax. This PEP proposes to implement context and cause suppression > differently. > > Rationale > = > > PEP 409 changes ``__cause__`` to be ``Ellipsis`` by default. Then if > ``__cause__`` is set to ``None`` by ``raise exc from None``, no context or > cause > will be printed should the exception be uncaught. > > The main problem with this scheme is it complicates the role of > ``__cause__``. ``__cause__`` should indicate the cause of the exception not > whether ``__context__`` should be printed or not. This use of ``__cause__`` is > also not easily extended in the future. For example, we may someday want to > allow the programmer to select which of ``__context__`` and ``__cause__`` will > be printed. The PEP 409 implementation is not amendable to this. > > The use of ``Ellipsis`` is a hack. Before PEP 409, ``Ellipsis`` was used > exclusively in extended slicing. Extended slicing has nothing to do with > exceptions, so it's not clear to someone inspecting an exception object why > ``__cause__`` should be set to ``Ellipsis``. Using ``Ellipsis`` by default for > ``__cause__`` makes it asymmetrical with ``__context__``. > > Proposal > > > A new attribute on ``BaseException``, ``__suppress_context__``, will be > introduced. The ``raise exc from None`` syntax will cause > ``exc.__suppress_context__`` to be set to ``True``. Exception printing code > will > check for the attribute to determine whether context and cause will be > printed. ``__cause__`` will return to its original purpose and values. > > There is precedence for ``__suppress_context__`` with the > ``print_line_and_file`` exception attribute. > > Patches > === > > There is a patch on `Issue 14133`_. > > > References > == > > .. _issue 14133: > http://bugs.python.org/issue6210 > > Copyright > = > > This document has been placed in the public domain. > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
On Feb 28, 2012, at 11:29 AM, Yury Selivanov wrote: >Could you please remove from the PEP the following statement: > >"""As it stands, Python 3 is currently a bad choice for long-term >investments, since the ecosystem is not yet properly developed, and >libraries are still fighting with their API decisions for Python 3.""" > >While it may be as such for you, I think it is incorrect for the rest. >Moreover, it is harmful for the python 3 adoption, to put such documents >on python.org. +∞ -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
In the end, that's not particularly relevant, because you don't have to run the test suite entirely; when working on small changes, you usually re-run the impacted parts of the test suite until everything goes fine; on the other hand, 2to3 *has* to run on the entire code base. Not at all. If you are working on the code, 2to3 only needs to run on the parts of the code that you changed, since the unmodified parts will not need to be re-transformed using 2to3. So, really, it's a couple of seconds to run a single bunch of tests vs. several minutes to run 2to3 on the code base. Not in my experience. The incremental run-time of 2to3 after a single change is in the order of fractions of a second. And it's not just the test suite: every concrete experiment with the library you're porting has a serial dependency on running 2to3. Therefore, your build process should support incremental changes. Fortunately, distribute does support this approach. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Spreading the Python 3 religion (was Re: PEP 414 - Unicode Literals for Python 3)
If PEP 414 helps some projects migrate to Python 3, great. But I really hope we as a community don't perpetuate the myth that you cannot port to Python 3 without this, and I hope that we spend as much effort on educating other Python developers on how to port to Python 3 *right now* supporting Python 2.6, 2.7, and 3.2. One thing that the PEP will certainly achieve is to spread the myth that you cannot port to Python 3 if you also want to support Python 2.5. That's because people will accept the "single source" approach as the one right way, and will accept that this only works well with Python 2.6. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Spreading the Python 3 religion (was Re: PEP 414 - Unicode Literals for Python 3)
v.loewis.de> writes: > One thing that the PEP will certainly achieve is to spread the myth that > you cannot port to Python 3 if you also want to support Python 2.5. That's > because people will accept the "single source" approach as the one right > way, and will accept that this only works well with Python 2.6. Let's hope not. We can mitigate that by spelling out in the docs that there's no one right way, how to choose which approach is best for a given project, and so on. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Spreading the Python 3 religion (was Re: PEP 414 - Unicode Literals for Python 3)
On Tue, Feb 28, 2012 at 12:07, Vinay Sajip wrote: > v.loewis.de> writes: > > > One thing that the PEP will certainly achieve is to spread the myth that > > you cannot port to Python 3 if you also want to support Python 2.5. > That's > > because people will accept the "single source" approach as the one right > > way, and will accept that this only works well with Python 2.6. > > Let's hope not. We can mitigate that by spelling out in the docs that > there's > no one right way, how to choose which approach is best for a given > project, and > so on. > Changes to http://docs.python.org/howto/pyporting.html are welcome. I tried to make sure it exposed all possibilities with tips on how to support as far back as Python 2.5. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a frozendict builtin type
>> * frozendict values must be immutable, as dict keys
>
> Why? That may be useful, but an immutable dict whose values
> might mutate is also useful; by forcing that choice, it starts
> to feel too specialized for a builtin.
Hum, I realized that calling hash(my_frozendict) on a frozendict
instance is enough to check if a frozendict only contains immutable
objects. And it is also possible to check manually that values are
immutable *before* creating the frozendict.
I also prefer to not check for immutability because it does simplify
the code :-)
$ diffstat frozendict-3.patch
Include/dictobject.h |9 +
Lib/collections/abc.py |1
Lib/test/test_dict.py | 59 +++
Objects/dictobject.c | 256 ++---
Objects/object.c |3
Python/bltinmodule.c |1
6 files changed, 295 insertions(+), 34 deletions(-)
The patch is quite small to add a new builtin type. That's because
most of the code is shared with the builtin dict type. (But the patch
doesn't include the documentation, it didn't write it yet.)
Victor
diff --git a/Include/dictobject.h b/Include/dictobject.h
--- a/Include/dictobject.h
+++ b/Include/dictobject.h
@@ -84,10 +84,14 @@ struct _dictobject {
PyDictEntry *ma_table;
PyDictEntry *(*ma_lookup)(PyDictObject *mp, PyObject *key, Py_hash_t hash);
PyDictEntry ma_smalltable[PyDict_MINSIZE];
+
+/* only used by frozendict */
+Py_hash_t hash;
};
#endif /* Py_LIMITED_API */
PyAPI_DATA(PyTypeObject) PyDict_Type;
+PyAPI_DATA(PyTypeObject) PyFrozenDict_Type;
PyAPI_DATA(PyTypeObject) PyDictIterKey_Type;
PyAPI_DATA(PyTypeObject) PyDictIterValue_Type;
PyAPI_DATA(PyTypeObject) PyDictIterItem_Type;
@@ -97,7 +101,12 @@ PyAPI_DATA(PyTypeObject) PyDictValues_Ty
#define PyDict_Check(op) \
PyType_FastSubclass(Py_TYPE(op), Py_TPFLAGS_DICT_SUBCLASS)
+#define PyFrozenDict_Check(op) \
+(PyDict_Check(op) || \
+ Py_TYPE(op) == &PyFrozenDict_Type || \
+ PyType_IsSubtype(Py_TYPE(op), &PyFrozenDict_Type))
#define PyDict_CheckExact(op) (Py_TYPE(op) == &PyDict_Type)
+#define PyFrozenDict_CheckExact(op) (Py_TYPE(op) == &PyFrozenDict_Type)
#define PyDictKeys_Check(op) (Py_TYPE(op) == &PyDictKeys_Type)
#define PyDictItems_Check(op) (Py_TYPE(op) == &PyDictItems_Type)
#define PyDictValues_Check(op) (Py_TYPE(op) == &PyDictValues_Type)
diff --git a/Lib/collections/abc.py b/Lib/collections/abc.py
--- a/Lib/collections/abc.py
+++ b/Lib/collections/abc.py
@@ -401,6 +401,7 @@ class Mapping(Sized, Iterable, Container
def __ne__(self, other):
return not (self == other)
+Mapping.register(frozendict)
class MappingView(Sized):
diff --git a/Lib/test/test_dict.py b/Lib/test/test_dict.py
--- a/Lib/test/test_dict.py
+++ b/Lib/test/test_dict.py
@@ -788,11 +788,70 @@ class Dict(dict):
class SubclassMappingTests(mapping_tests.BasicTestMappingProtocol):
type2test = Dict
+
+class FrozenDictTests(unittest.TestCase):
+def test_constructor(self):
+# unorderable keys and values
+frozendict(x=b'abc', y='abc')
+frozendict({b'abc': 1, 'abc': 2})
+
+# hashable keys and values
+class Hashable:
+pass
+for hashable in (5, 1.0, "abc", (1, 2), Hashable()):
+frozendict(key=hashable)
+frozendict(hashable=0)
+
+# not hashable keys or values
+class NotHashable:
+def __hash__(self):
+raise TypeError("not hashable")
+for not_hashable in ([], {}, NotHashable()):
+frozendict(key=not_hashable)
+frozendict(not_hashable=0)
+
+def test_copy(self):
+self.assertIsInstance(frozendict().copy(),
+ frozendict)
+self.assertEqual(frozendict(x=1, y=2).copy(),
+ frozendict(x=1, y=2))
+
+def test_dir(self):
+self.assertEqual(
+set(dir(frozendict)) - set(dir(object)),
+{'__contains__', '__getitem__', '__iter__', '__len__',
+ 'copy', 'fromkeys', 'get', 'items', 'keys', 'values'})
+
+def test_fromkeys(self):
+self.assertEqual(frozendict.fromkeys('ai'),
+ frozendict(a=None, i=None))
+
+def test_hash(self):
+self.assertEqual(hash(frozendict()),
+ hash(frozenset()))
+self.assertEqual(hash(frozendict({1: 2})),
+ hash(frozenset({(1, 2)})))
+
+a = frozendict.fromkeys('ai')
+b = frozendict.fromkeys('ia')
+self.assertEqual(hash(a), hash(b))
+self.assertNotEqual(tuple(a.items()), tuple(b.items()))
+
+# not hashable
+f = frozendict(x=[])
+self.assertRaises(TypeError, hash, f)
+
+def test_repr(self):
+self.assertEqual(repr(frozendict()), "frozendict({})")
+self.assertEqual(repr(frozendict(x=1)), "frozendict({'x': 1})")
+
+
def test_main():
supp
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On 28/02/2012 18.08, Vinay Sajip wrote:
Ezio Melotti gmail.com> writes:
For every CPython bug that I fix I first apply the patch on 2.7, then on
3.2 and then on 3.3.
Most of the time I don't even need to change anything while applying the
patch to 3.2, sometimes I have to do some trivial fixes. This is also
true for another personal 12kloc project* where I'm using the
two-branches approach.
I hear what you say about the personal project, but IMO CPython is atypical (as
far as this discussion is concerned), not least because it's not a pure-Python
project.
Most of the things I fix are pure Python, I wasn't considering the C
patches and doc fixes here.
For me, the costs of having two branches are:
1) a one-time conversion when the Python3-compatible branch is created
(can be done easily with 2to3);
Yes, but the amount of ease is project-dependent. For example, 2to3 wraps
values() method calls with list(), which is a reasonable thing to do for dicts;
when presented Django's querysets, which have a values() method which should not
be wrapped, then you have to go through and sort things out. I'm not knocking
2to3, which I think is great. Just that things go well sometimes, and less well
at other times,
With the personal project this is what I did:
1) make a separate branch;
2) run 2to3 and let it overwrite the file;
3) review the changes as I would do with any other patch before
committing;
4) fix things that 2to3 missed and other minor glitches;
5) fix a few bugs that surfaced after the port (and were in the
original code too);
The fixes made by 2to3 were mostly:
* removing u'' from strings;
* renaming imports, methods (like the .iteritems);
* adding 'as' in the "except"s;
* adding () for a few "print"s;
These changes affected about 500 lines of code (out of 12kloc).
The changes I did manually after running 2to3 were (some where not
strictly necessary):
* removing 'object' from classes;
* removing ord() in a few places;
* removing the content of super(...);
* removing codecs.open() and use open() instead;
* removing a few .decode('utf-8');
* adding a couple of b'';
After a couple of days almost everything was working fine.
With the shared code base approach, the costs are:
1) a one-time conversion to "fix" the code base and make it run on
both 2.x and 3.x;
2) keep using and having to deal with hacks in order to keep it running.
Which hacks do you mean, if you're only interested in 2.6+?
Things like try/except for names that changed and wrappers for
bytes/strings.
Of course the situation is worse for projects that have to support
earlier versions.
With the first approach, you also have two clean and separate code
bases, with no hacks; when you stop using Python 2, you end up with a
clean Python 3 branch.
The one-time conversion also seems easier in the first case.
(Note: there are also other costs -- e.g. releasing -- that I haven't
considered because they don't affect me personally, but I'm not sure
they are big enough to make the two-branches approach worse.)
I don't believe there's a one-size-fits-all. The two branches approach is
appealing, and I have no quarrel with it: but I contend that big projects like
Django would be reluctant to switch, or take much longer to switch to 3.x, if
they had to maintain separate branches.
I would actually feel safer doing the port in a separate branch and keep
it there.
Changing all the code in the main branch just to make it work for 3.x
too doesn't strike like a really good idea to me.
Given the size of their user community,
they have to follow strict release procedures, which (even with just running on
2.x) smaller projects can be more relaxed about.
I don't have much experience regarding releases, but developing on a
separate branch shouldn't affect the release of the 2.x version. The
developers will have to merge the changes to the py3 branch too, and
eventually they will be able to ship an additional release for py3 too.
Sure, there's more work for the developers, but that's no news.
You forgot to mention the part which is most time-consuming day-to-day: making
changes and testing. For the two-branch approach, its
1. Change on 2.x
2. Test on 2.x
3. Commit on 2.x
4. Merge to 3.x
5. Possibly change on 3.x
6. Test on 3.x
7. Commit on 3.x
where each "test" step, if failures occur, might take you back to a previous
"change" step.
For the single codebase, that's
1. Change
2. Test on 2.x
3. Test on 3.x
4. Commit
And if something fails here, you will have to repeat both step 2 and 3,
until you get it right for both at the same time.
The step 1 of the single codebase is in the end more or less equivalent
to steps 1+4+5, just in a different way. The remaining extra commit
takes no time, and since the branches are independent, if you find a
problem with py3 you don't have to run the test suite for 2.x again.
In my experience with CPython, the most time-consuming part is making
Re: [Python-Dev] Spreading the Python 3 religion (was Re: PEP 414 - Unicode Literals for Python 3)
Brett Cannon python.org> writes: > Changes to http://docs.python.org/howto/pyporting.html are welcome. I tried to > make sure it exposed all possibilities with tips on how to support as far back > as Python 2.5. Right, will take a look. FYI a Google search for "python 3 porting guide" shows the Wiki PortingToPy3K page, then Brian Curtin's Python 3 Porting Guide, then Lennart Regebro's porting book website, and then the howto referred to above. Possibly the Wiki page and Brian's guide need to link to the howto, as I presume that's the canonical go-to guide - they don't seem to do so currently. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Spreading the Python 3 religion (was Re: PEP 414 - Unicode Literals for Python 3)
On Tue, Feb 28, 2012 at 11:51, Vinay Sajip wrote: > Brett Cannon python.org> writes: > >> Changes to http://docs.python.org/howto/pyporting.html are welcome. I tried >> to >> make sure it exposed all possibilities with tips on how to support as far >> back >> as Python 2.5. > > Right, will take a look. FYI a Google search for "python 3 porting guide" > shows > the Wiki PortingToPy3K page, then Brian Curtin's Python 3 Porting Guide, then > Lennart Regebro's porting book website, and then the howto referred to above. > Possibly the Wiki page and Brian's guide need to link to the howto, as I > presume > that's the canonical go-to guide - they don't seem to do so currently. Funny that you mention this: just a few minutes ago someone mentioned on twitter that they found and liked the guide I wrote, then I mentioned the howto/porting page since Brett's last message reminded me of it, and I mentioned that I should update and link to howto/porting. In the words of Guido, I will "make it so". ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a frozendict builtin type
Victor Stinner wrote: * frozendict values must be immutable, as dict keys Why? That may be useful, but an immutable dict whose values might mutate is also useful; by forcing that choice, it starts to feel too specialized for a builtin. Hum, I realized that calling hash(my_frozendict) on a frozendict instance is enough to check if a frozendict only contains immutable objects. And it is also possible to check manually that values are immutable *before* creating the frozendict. I also prefer to not check for immutability because it does simplify the code :-) $ diffstat frozendict-3.patch Include/dictobject.h |9 + Lib/collections/abc.py |1 Lib/test/test_dict.py | 59 +++ Objects/dictobject.c | 256 ++--- Objects/object.c |3 Python/bltinmodule.c |1 6 files changed, 295 insertions(+), 34 deletions(-) The patch is quite small to add a new builtin type. That's because most of the code is shared with the builtin dict type. (But the patch doesn't include the documentation, it didn't write it yet.) Could you create an issue for this on the tracker, maybe write a PEP. I don't think sending patches to this mailing list is the way to do this. Would you mind taking a look at how your code interacts with PEP 412. Cheers, Mark. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On 2/28/2012 7:10 AM, Vinay Sajip wrote: The PEP 314 approach seems to assume that that if things work on 3.3, they will work on 3.2/3.1/3.0 without any changes other than replacing u'xxx' with 'xxx'. (Delete 3.0. 3.1 is also less of a concern.) It actually assumes that if things work on 3.3 *and* 2.7 (or .6), then ... . At first glance, this seems reasonable. If the code works on 2.7, then it does not use any new 3.3 features. Nor does it depend on any 3.3-only bug fixes that were part of a feature patch. 2.6, of course, is essentially not getting any bugfixes. In other words, you aren't supposed to want to e.g. test 3.2 and 3.3 iteratively, using a workflow which intersperses edits with running tests using 3.2 and running tests with 3.3. Anyone who is also targeting 3.2 could run a test32 script whenever they need to take a break. Or it could be run in the background (perhaps on a different core) while editing continues. People will work this out on a project by project basis, or use one of the other solutions. In any case, a single code base seems not to be possible across 2.6+/3.0/3.1/3.2/3.3+ using the PEP 314 approach, though of course one will be possible for just 2.6+/3.3+. Early adopters of 3.x seem to be penalised by this approach: I for one will try to use the unicode_literals approach wherever I can. Early adoption of new tech typically has costs as well as benefits ;-). -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 414 - Unicode Literals for Python 3
In http://mail.python.org/pipermail/python-dev/2012-February/117070.html Vinay Sajip wrote: > It's moot, but as I see it: the purpose of PEP 414 is to facilitate a > single codebase across 2.x and 3.x. However, it only does this if your > 3.x interest is 3.3+ For many people -- particularly those who haven't ported yet -- 3.x will mean 3.3+. There are some who will support 3.2 because it is a LTS release on some distribution, just as there were some who supported Python 1.5 (but not 1.6) long into the 2.x cycle, but I expect them to be the minority. I certainly don't expect 3.2 to remain a primary development target, the way that 2.7 is. IIRC, the only ways to use 3.2 even today are: (a) Make an explicit choice to use something other than the default (b) Download directly and choose 3.x without OS support (c) Use Arch Linux These are the sort of people who can be expected to upgrade. Now also remember that we're talking specifically about projects that have *not* been ported to 3.x (==> no existing users to support), and that won't be ported until 3.2 is already in maintenance mode. > If you also want to or need to support 3.0 - 3.2, it makes your > workflow more painful, Compared to dropping 3.2, yes. Compared to supporting 3.2 today? I don't see how. > because you can't run tests on 2.x or 3.3 and then run them on 3.2 > without an intermediate source conversion step - just like the 2to3 > step that people find painful when it's part of maintenance workflow, > and which in part prompted the PEP in the first place. So the only differences compared to today are that: (a) Fewer branches are after the auto-conversion. (b) No "current" branches are after the auto-conversion. (c) The auto-conversion is much more limited in scope. -jJ -- If there are still threading problems with my replies, please email me with details, so that I can try to resolve them. -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On Feb 28, 2012, at 10:59 AM, Jim J. Jewett wrote: >For many people -- particularly those who haven't ported yet -- 3.x >will mean 3.3+. There are some who will support 3.2 because it is a >LTS release on some distribution, just as there were some who supported >Python 1.5 (but not 1.6) long into the 2.x cycle, but I expect them to >be the minority. > >I certainly don't expect 3.2 to remain a primary development target, >the way that 2.7 is. IIRC, the only ways to use 3.2 even today are: > > (a) Make an explicit choice to use something other than the default > (b) Download directly and choose 3.x without OS support > (c) Use Arch Linux On Debian and Ubuntu, installing Python 3.2 is easy, even if it isn't the default. However, once installed, 'python3' is Python 3.2. I personally think Python 3.2 makes for a fine platform for new code, and just as good for porting most existing libraries and applications to. You can get many Python 3.2 compatible packages from the Debian and Ubuntu archives by using the normal installation procedures, and generally, if there is a 'python-foo' package, the Python 3.2 compatible version will be called 'python3-foo'. I would expect other Linux distros to be in generally the same boat. There's a lot already available, and this will definitely increase over time. Although on Ubuntu we'll be planning future developments at UDS in May, I would expect Ubuntu 12.10 to have Python 3.3 (probably in addition to Python 3.2 since we can do that easily), and looking ahead at the expected Python release schedule, I'm expecting our next LTS in 2014 (Ubuntu 14.04) will probably ship with Python 3.4, either with or without the earlier Python 3 versions. So I think if you're starting a new project, write it in Python 3 and target Python 3.2. The only reason not to do that is if some critical part of your dependency stack hasn't yet been ported, and in that case, help them get there! IME, most are grateful for a patch or branch that added Python 3 support. >These are the sort of people who can be expected to upgrade. > >Now also remember that we're talking specifically about projects that >have *not* been ported to 3.x (==> no existing users to support), and >that won't be ported until 3.2 is already in maintenance mode. I really hope most people won't wait. Sure, the big frameworks by their nature are going to have more inertia, but if you are the author of a Python library, you can and should port *now* and target Python 3.2. Only this way will we as a community be able to build up the dependency stack so that when the large frameworks are ready, your library which they may depend on, will have a long and stable history on Python 3. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a frozendict builtin type
Steven D'Aprano wrote:
> M.-A. Lemburg wrote:
>> Victor Stinner wrote:
See also the PEP 351.
>>> I read the PEP and the email explaining why it was rejected.
>>>
>>> Just to be clear: the PEP 351 tries to freeze an object, try to
>>> convert a mutable or immutable object to an immutable object. Whereas
>>> my frozendict proposition doesn't convert anything: it just raises a
>>> TypeError if you use a mutable key or value.
>>>
>>> For example, frozendict({'list': ['a', 'b', 'c']}) doesn't create
>>> frozendict({'list': ('a', 'b', 'c')}) but raises a TypeError.
>>
>> I fail to see the use case you're trying to address with this
>> kind of frozendict().
>>
>> The purpose of frozenset() is to be able to use a set as dictionary
>> key (and to some extent allow for optimizations and safe
>> iteration). Your implementation can be used as dictionary key as well,
>> but why would you want to do that in the first place ?
>
> Because you have a mapping, and want to use a dict for speedy, convenient
> lookups. Sometimes your
> mapping involves the key being a string, or an int, or a tuple, or a set, and
> Python makes it easy
> to use that in a dict. Sometimes the key is itself a mapping, and Python
> makes it very difficult.
>
> Just google on "python frozendict" or "python immutabledict" and you will
> find that this keeps
> coming up time and time again, e.g.:
>
> http://www.cs.toronto.edu/~tijmen/programming/immutableDictionaries.html
> http://code.activestate.com/recipes/498072-implementing-an-immutable-dictionary/
> http://code.activestate.com/recipes/414283-frozen-dictionaries/
> http://bob.pythonmac.org/archives/2005/03/04/frozendict/
> http://python.6.n6.nabble.com/frozendict-td4377791.html
> http://www.velocityreviews.com/forums/t648910-does-python3-offer-a-frozendict.html
> http://stackoverflow.com/questions/2703599/what-would-be-a-frozen-dict
Only the first of those links appears to actually discuss reasons for
adding a frozendict, but it fails to provide real world use cases and
only gives theoretical reasons for why this would be nice to have.
>From a practical view, a frozendict would allow thread-safe iteration
over a dict and enable more optimizations (e.g. using an optimized
lookup function, optimized hash parameters, etc.) to make lookup
in static tables more efficient.
OTOH, using a frozendict as key in some other dictionary is, well,
not a very realistic use case - programmers should think twice before
using such a design :-)
>> If you're thinking about disallowing changes to the dictionary
>> structure, e.g. in order to safely iterate over its keys or items,
>> "freezing" the keys is enough.
>>
>> Requiring the value objects not to change is too much of a restriction
>> to make the type useful in practice, IMHO.
>
> It's no more of a limitation than the limitation that strings can't change.
>
> Frozendicts must freeze the value as well as the key. Consider the toy
> example, mapping food
> combinations to calories:
>
>
> d = { {appetizer => fried fish, main => double burger, drink => cola}: 5000,
> {appetizer => None, main => green salad, drink => tea}: 200,
> }
>
> (syntax is only for illustration purposes)
>
> Clearly the hash has to take the keys and values into account, which means
> that both the keys and
> values have to be frozen.
>
> (Values may be mutable objects, but then the frozendict can't be hashed --
> just like tuples can't be
> hashed if any item in them is mutable.)
Right, but that doesn't mean you have to require that values are hashable.
A frozendict could (and probably should) use the same logic as tuples:
if the values are hashable, the frozendict is hashable, otherwise not.
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Feb 28 2012)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
2012-02-13: Released eGenix pyOpenSSL 0.13http://egenix.com/go26
2012-02-09: Released mxODBC.Zope.DA 2.0.2 http://egenix.com/go25
2012-02-06: Released eGenix mx Base 3.2.3 http://egenix.com/go24
::: Try our new mxODBC.Connect Python Database Interface for free !
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Backporting PEP 414
Here's what I know: We don't add features to bug-fix releases. u'' is considered a feature. By not backporting to 3.1 and 3.2 we are not easing the migration pains from 2.x. Here's what I don't know: Why is readding u'' a feature and not a bug? (Just had a thought about this -- because the removal of u'' is documented.) To take a different example: callable() had been removed from 3.0, and was added back in 3.2. callable() is not a big deal as you can roll your own quite easily -- and that is the huge difference: a user *cannot* add u'' back to 3.0/3.1 (at least, not without modifying and rebuilding the Python interpreter source). If there is already a FAQ entry feel free to point me to it, but I would still be curious why, in this instance, practicality does not beat purity? My apologies if this type of question has been rehashed before. ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Backporting PEP 414
2012/2/28 Ethan Furman : > Here's what I know: > > We don't add features to bug-fix releases. > u'' is considered a feature. > By not backporting to 3.1 and 3.2 we are not easing the migration pains from > 2.x. > > > Here's what I don't know: > > Why is readding u'' a feature and not a bug? (Just had a thought about this > -- because the removal of u'' is documented.) Because it's a new "thing" which doesn't fix obviously broken behavior. > > > If there is already a FAQ entry feel free to point me to it, but I would > still be curious why, in this instance, practicality does not beat purity? Because it's practical not to break bugfix releases with new features. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Backporting PEP 414
On Tue, Feb 28, 2012 at 14:27, Ethan Furman wrote: > Here's what I know: > > We don't add features to bug-fix releases. > u'' is considered a feature. > By not backporting to 3.1 and 3.2 we are not easing the migration pains from > 2.x. Let's say it's 2013 and 3.3 has been out for a few months and you want to port your library to Python 3. Why would you worry about 3.1 or 3.2? You certainly see why we're not worried about 3.0. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Backporting PEP 414
On Tue, 2012-02-28 at 15:54 -0500, Benjamin Peterson wrote: > 2012/2/28 Ethan Furman : > > Here's what I know: > > > > We don't add features to bug-fix releases. > > u'' is considered a feature. > > By not backporting to 3.1 and 3.2 we are not easing the migration pains from > > 2.x. > > > > > > Here's what I don't know: > > > > Why is readding u'' a feature and not a bug? (Just had a thought about this > > -- because the removal of u'' is documented.) > > Because it's a new "thing" which doesn't fix obviously broken behavior. > > > > > > > If there is already a FAQ entry feel free to point me to it, but I would > > still be curious why, in this instance, practicality does not beat purity? > > Because it's practical not to break bugfix releases with new features. This change, by its nature, cannot break old programs. - C ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Backporting PEP 414
On Tue, 28 Feb 2012 16:23:40 -0500 Chris McDonough wrote: > On Tue, 2012-02-28 at 15:54 -0500, Benjamin Peterson wrote: > > 2012/2/28 Ethan Furman : > > > Here's what I know: > > > > > > We don't add features to bug-fix releases. > > > u'' is considered a feature. > > > By not backporting to 3.1 and 3.2 we are not easing the migration pains > > > from > > > 2.x. > > > > > > > > > Here's what I don't know: > > > > > > Why is readding u'' a feature and not a bug? (Just had a thought about > > > this > > > -- because the removal of u'' is documented.) > > > > Because it's a new "thing" which doesn't fix obviously broken behavior. > > > > > > > > > > > If there is already a FAQ entry feel free to point me to it, but I would > > > still be curious why, in this instance, practicality does not beat purity? > > > > Because it's practical not to break bugfix releases with new features. > > This change, by its nature, cannot break old programs. Unless the implementation is buggy, or has unintended side-effects. In theory, *most* changes done in feature releases cannot break old programs. Reality is often a bit more surprising :) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Backporting PEP 414
On Feb 28, 2012, at 03:54 PM, Benjamin Peterson wrote: >> If there is already a FAQ entry feel free to point me to it, but I would >> still be curious why, in this instance, practicality does not beat purity? > >Because it's practical not to break bugfix releases with new features. And because now your code is incompatible with three micro-release versions (3.2.0, 3.2.1, and 3.2.2), two of which are bug fix releases. Which means for example, you can't be sure which version of which distro your code will work on. Doesn't anybody else remember the True/False debacle in 2.2.1? Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Backporting PEP 414
On Tue, 2012-02-28 at 16:48 -0500, Barry Warsaw wrote: > On Feb 28, 2012, at 03:54 PM, Benjamin Peterson wrote: > > >> If there is already a FAQ entry feel free to point me to it, but I would > >> still be curious why, in this instance, practicality does not beat purity? > > > >Because it's practical not to break bugfix releases with new features. > > And because now your code is incompatible with three micro-release versions > (3.2.0, 3.2.1, and 3.2.2), two of which are bug fix releases. Which means for > example, you can't be sure which version of which distro your code will work > on. That I do sympathize with. > Doesn't anybody else remember the True/False debacle in 2.2.1? I do. It was slightly different than this because the feature was added twice, once in 2.2.1 with direct aliases to 0 and 1, which was found to be lacking, and then later again in 2.3 with explicit types, so it was sort of an extended-timeframe unpleasantness, and the feature's minor-dot-introduction was only a contributing factor, IIRC. But yeah. A year from now I wouldn't remember which version of 3.2 got a new feature, and neither would anybody else. The no-new-features guidelines are useful in the real world this way, because they represent a coherent policy, as tempting as it is to just jam it in. - C ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Backporting PEP 414
On Wed, Feb 29, 2012 at 8:17 AM, Chris McDonough wrote: > But yeah. A year from now I wouldn't remember which version of 3.2 got > a new feature, and neither would anybody else. The no-new-features > guidelines are useful in the real world this way, because they represent > a coherent policy, as tempting as it is to just jam it in. Also, I think there may be some confusion about Armin's plan to handle 3.2 - he aims to write an *import hook* that accepts the u/U prefixes during tokenisation, not a source-to-source transform like 2to3. It's clumsier than the plan for native syntactic support in 3.3 (since you'll need to make sure the import hook is installed, the presence of the hook will add overhead during application startup, and any attempts to compile affected modules that don't go through the import machinery will fail with a syntax error), but the presence of importlib in 3.2 makes it quite feasible. When loading from a cached .pyc, the hook won't even have to do anything special (since the tokenisation change only affects the compilation step). Assuming Armin can get the hook working as intended, then long running applications where startup overhead isn't a big deal will just need to ensure the hook is in place before they import any modules that use the old-style string literals. For cases where the startup overhead isn't acceptable (such as command line applications), then approaches that change the source in advance (i.e. separate branches or single source with the unicode_literals future import) will continue to be the preferred mechanism for providing 3.2 support. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Backporting PEP 414
Why is readding u'' a feature and not a bug? There is a really simple litmus test for whether something is a bug: does it deviate from the specification? In this case, the specification is the grammar, and the implementation certainly doesn't deviate from it. So it can't be a bug. Regards, Martin P.S. Before anybody over-interprets this criterion: there is certain "implicit behavior" assumed in Python that may not actually be documented, such as "the interpreter will not core dump", and "the source code will compile with any standard C compiler". Deviation from these implicit assumption is also a bug. However, they don't apply here. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Backporting PEP 414
Nick Coghlan gmail.com> writes: > Also, I think there may be some confusion about Armin's plan to handle > 3.2 - he aims to write an *import hook* that accepts the u/U prefixes > during tokenisation, not a source-to-source transform like 2to3. It's I must confess, I thought it was a source-to-source transform, because he called it an installation-time hook (which of course makes you think of 2to3) and not an import hook. That will have a much better chance of acceptable performance, since it'll only touch changed stuff automatically. I feel better about the prospects for 3.2 support :-) Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Le 28/02/2012 13:48, Giampaolo Rodolà a écrit : > Il 28 febbraio 2012 13:19, Antoine Pitrou ha scritto: >> IMO, maintaining two branches shouldn't be much more work than >> maintaining hacks so that a single codebase works with two different >> programming languages. > > Would that mean distributing 2 separate tarballs? > How would tools such as easy_install and pip work in respect of that? > Is there a naming convention they can rely on? Sadly, PyPI and the packaging tools don’t play nice with non-single-codebase projects, so you have to use a different name for your 3.x-compatible release, like “unittestpy3k”. Some bdists include the Python version in the file name, but sdists don’t. Regards ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On Tue, Feb 28, 2012 at 13:10, Vinay Sajip wrote: > We might be at cross purposes here. I don't see how Distribute helps, because > the use case I'm talking about is not about distributing or installing stuff, > but iteratively changing and testing code which needs to work on 2.6+, 3.2 and > 3.3+. Make sure you can run the tests with python setup.py test, and you're "in the butter", as we say in Sweden. :-) > If the 2.x code depends on having u'xxx' literals, then 3.2 testing will > potentially involve running a fixer on all files in the project every time a > change is made, writing to a separate directory, or else a fixer which is > integrated into the editing environment so it knows what changed. This is > painful Sure, and distribute does this for you. http://python3porting.com/2to3.html //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
All the various strategies for supporting Python 2 and Python 3 as well as their various drawbacks and ways around this is covered in my book, chapter 2. :-) http://python3porting.com/strategies.html I may be too late to point this out, but it feels like this discussion could have been shorter if everyone read this first. :-) //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On Tue, Feb 28, 2012 at 16:30, Giampaolo Rodolà wrote: > Il 28 febbraio 2012 15:20, Ezio Melotti ha scritto: >> (Note: there are also other costs -- e.g. releasing -- that I haven't >> considered because they don't affect me personally, but I'm not sure they >> are big enough to make the two-branches approach worse.) > > They are. > With that kind of approach you're basically forced to include the > python version number as part of the tarball name (e.g. > foo-0.3.1-py2.tar.gz and foo-0.3.1-py3.tar.gz). Not at all. You can include both code bases in one package. http://python3porting.com/2to3.html#distributing-packages //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On Tue, Feb 28, 2012 at 16:39, Vinay Sajip wrote: > Serhiy Storchaka gmail.com> writes: > >> Another pertinent question: "What are disadvantages of PEP 414 is adopted?" > > It's moot, but as I see it: the purpose of PEP 414 is to facilitate a single > codebase across 2.x and 3.x. The bytes/native/unicode issue is an issue even if you use 2to3. But of course that *is* a form of "single codebase" so maybe that's what you meant. :-) //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Backporting PEP 414
On Tue, Feb 28, 2012 at 21:27, Ethan Furman wrote: > Here's what I know: > > We don't add features to bug-fix releases. > u'' is considered a feature. > By not backporting to 3.1 and 3.2 we are not easing the migration pains from > 2.x. If this is added to 3.2.3, then some programs will work with 3.2.3, but not 3.2.2. I'm pretty sure this will confuse people no end. :-) //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Spreading the Python 3 religion (was Re: PEP 414 - Unicode Literals for Python 3)
mar...@v.loewis.de writes: > One thing that the PEP will certainly achieve is to spread the myth that > you cannot port to Python 3 if you also want to support Python 2.5. That's > because people will accept the "single source" approach as the one right way, > and will accept that this only works well with Python 2.6. Please, Martin, I dislike this idea as much as you do. (There was no -1 from me, though, because I don't work in the context of the claimed use cases at all, but lots of people obviously find them persuasive.) But in respect of myth-spreading, the problem with the PEP is the polemic tone. (Yeah, I've seen Armin's claim that it's not polemic. I disagree.) The unqualified claims that "2to3 is insufficient" and the PEP will "enable side-by-side support" of Python 2 and Python 3 by libraries are too extreme, and really unnecessary in light of Guido's logic for acceptance. As far as I can see, like 2to3, like u()/b(), this PEP introduces a device that will be the most *convenient* approach for *some* use cases. If it were presented that way, with recommendation for its use restricted to the particular intended use case, I don't think it would have a huge effect on people's perception of the difficulty of porting in general, including multiversion support including 2.5. If others want to use it, even though you and I think that's a bad idea, well, we can blog, and "consenting adults" covers those users. On the other hand, implementation of the PEP itself should have a positive effect on the community's perception of python-dev's responsiveness to its pain. Ie, a lot of us feel strongly that this is the wrong thing to do in principle -- but we're gonna do it anyway, because part of the community wants it. So, let's work on integrating this PEP into the more general framework of recommendations for porting Python 2 code to Python 3 and/or developing libraries targeting both. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Spreading the Python 3 religion (was Re: PEP 414 - Unicode Literals for Python 3)
On Wed, Feb 29, 2012 at 5:23 PM, Stephen J. Turnbull wrote: > mar...@v.loewis.de writes: > > > One thing that the PEP will certainly achieve is to spread the myth that > > you cannot port to Python 3 if you also want to support Python 2.5. That's > > because people will accept the "single source" approach as the one right > way, > > and will accept that this only works well with Python 2.6. > > Please, Martin, I dislike this idea as much as you do. (There was no > -1 from me, though, because I don't work in the context of the claimed > use cases at all, but lots of people obviously find them persuasive.) > > But in respect of myth-spreading, the problem with the PEP is the > polemic tone. (Yeah, I've seen Armin's claim that it's not polemic. > I disagree.) The unqualified claims that "2to3 is insufficient" and > the PEP will "enable side-by-side support" of Python 2 and Python 3 by > libraries are too extreme, and really unnecessary in light of Guido's > logic for acceptance. FWIW, I agree that much of the rhetoric in the current version of PEP 414 is excessive. Armin has given me permission to create an updated version of PEP 414 and toning down the hyperbole (or removing it entirely in cases where it's irrelevant to the final decision) is one of the things that I will be changing. I also plan to add a link to Lennart's guide to the various porting strategies that are currently available, more clearly articulate the cases where the new approach can most help (i.e. when there are project specific reasons to avoid the unicode_literals import), as well as name drop Pyramid (Chris McDonough), Flask (Armin), Django (Jacob Kaplan-Moss) and requests (Kenneth Reitz) as cases where key developers of web-related third party frameworks or libraries have indicated that PEP 414 will help greatly with bringing the sections of the Python ecosystem they're involved with into the Python 3 fold over the next few years. My aim is for the end result to better reflect the reasons why Guido *accepted* the PEP, moreso than Armin's own reasons for *wanting* it. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
