Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On 2008-03-24 09:22, Lennart Regebro wrote: I think 2to3 is a procedure that will work well for library type projects with a reasonably small set of developers that make regular releases. There you can release both a python 2 and a python 3 version of the module, for example. ... So, in short: Large projects with interconnected modules where the developers and users of module are the same people will have big difficulties with the 2to3 approach and would be the people who are most likely to not be able to in practice go forward to Python 3 unless they have some sort of smooth path forward. I don't think there's a lot to worry about: Companies using Python for applications typically have a completely different life-cycle of releases and applications compared to the Python release schedule, i.e. they often still run Python 2.3 or 2.4 and wait for major releases to settle before deciding to port to them. Every now and then, they make the decision to port to the next release (for the next version of their software) and this change is then managed accordingly - sometimes skipping a complete major release of Python. In such projects, 2to3 will get applied to the sources once and then all development continues on the Python 3.0 version of the code. In reality, I don't think that 2to3 will get used for continuous porting between a 2.x code base and a 3.0 one all that much. The transition from 2.x to 3.0 will happen during a longer period of time (probably a few years) and depend a lot on the release cycle of the applications using Python, whether or not the 3.0 version provides better features, more performance, etc. and whether the 2.x branches of Python and the used 3rd party modules are still supported or not. New applications will likely choose 3.0 right away - provided that the needed 3rd party modules are available and stable enough. In summary: 2to3 is a very useful tool to have. Whether or not it is used for continuous porting between the two worlds is really secondary. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 25 2008) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
Lennart Regebro wrote: On Mon, Mar 24, 2008 at 12:14 AM, Martin v. Löwis [EMAIL PROTECTED] wrote: You are still only seeing this as a case of libraries with a small number of people developing them and making regular well defined releases. That is not how the world I am talking about looks. Can you give me examples of such software? I'll repeat the link where I explained my point on this: http://regebro.wordpress.com/2008/03/22/why-python-26-and-30-compatibility-would-be-a-very-good-thing/ Yet, there you merely claim that such software exists (within larger organizations, and within large communities like Zope and Plone), without giving specific examples. This is not very convincing. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Mon, Mar 24, 2008 at 10:23 AM, Martin v. Löwis [EMAIL PROTECTED] wrote: Yet, there you merely claim that such software exists (within larger organizations, and within large communities like Zope and Plone), without giving specific examples. No I don't. Here is what I said: In many other cases, this is not how code is developed. Both within larger organisations and within large communities like Zope and Plone. I don't think chewing through this issue once more is meaningful, so I'll stop now. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Mon, Mar 24, 2008 at 12:14 AM, Martin v. Löwis [EMAIL PROTECTED] wrote: You are still only seeing this as a case of libraries with a small number of people developing them and making regular well defined releases. That is not how the world I am talking about looks. Can you give me examples of such software? I'll repeat the link where I explained my point on this: http://regebro.wordpress.com/2008/03/22/why-python-26-and-30-compatibility-would-be-a-very-good-thing/ -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Mon, Mar 24, 2008 at 2:30 AM, Jean-Paul Calderone [EMAIL PROTECTED] wrote: On Mon, 24 Mar 2008 00:14:13 +0100, \Martin v. Löwis\ [EMAIL PROTECTED] wrote: You are still only seeing this as a case of libraries with a small number of people developing them and making regular well defined releases. That is not how the world I am talking about looks. Can you give me examples of such software? Are you perhaps talking about closed source software? I'm not sure what software he was talking about. I can say that for the work I do on both Twisted and Divmod software, I'd be quite happy to see this feature. As either part of a migration path towards 3k _or_ as a feature entirely on its own merits, this would be very useful to me. I'm a bit curious about why Thomas said this sort of thing results in fragile code. Twisted has been using __future__ imports for years and they've never been a problem. Twisted currently supports Python 2.3 through Python 2.5, and the only thing that's really difficult about that is subtle changes in library behavior, not syntax. I wasn't talking about future imports. I was talking about writing Python 2.6 code and expecting it to work *the same way* in 3.0 without modification. It requires all programmers working on the project to have knowledge of how 3.0 and 2.6 differ. *I* can't even look at code and tell you how 2.6 and 3.0 will differ. Since Lennart was talking specifically about projects with a large number of developers, I do not believe this is a safe assumption to make. A simple preprocessor step involving 2to3 requires no such knowledge. Or, alternatively, a preprocessor step involving 3to2, which I think will result in better code. Unfortunately I don't have time to work on either anytime soon. -- Thomas Wouters [EMAIL PROTECTED] Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
Nuxeo CPS worked like this, but we can ignore them as that project is all but dead will never move to Python 3 in any case. Zope/CMF/Plone works like this. I don't understand. AFAICT, Zope *is* a library, i.e. you have to run setup.py for lots of packages. Do you not have to run setup.py, for, say, zope.interface, or zope.psycopgda? It's fine that all people develop on the same subversion repository. Doing so integrates nicely with 2to3. The Plone collective works like this, and it is *not* reasonably well managed, so there software quite often doens't get released, but people run against trunk. And that's fine. You still can integrate 2to3 with that transparently. I know Django people both strive to support multiple versions of Python and regularily run their production sites on trunk. Insane, perhaps, but that's how things are. And no need to change that, see http://wiki.python.org/moin/PortingDjangoTo3k Just to repeat myself: With that patch to Django, you can a) support all versions of Python simultaneously, from 2.x to 3.0 b) work from the subversion trunk all the time c) transparently invoke 2to3 on the trunk; you won't even notice that you do (except for all the diffs it currently prints to stdout also :-) FWIW, your approach (of adding __future__ imports to 2.6) wouldn't help Django at all, as they would need to give up 2.5 and earlier. So, in short: Large projects with interconnected modules where the developers and users of module are the same people will have big difficulties with the 2to3 approach and would be the people who are most likely to not be able to in practice go forward to Python 3 unless they have some sort of smooth path forward. I still don't see why that is. In the examples you gave, no such difficulties are apparent. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Mon, Mar 24, 2008 at 11:27 AM, Martin v. Löwis [EMAIL PROTECTED] wrote: Nuxeo CPS worked like this, but we can ignore them as that project is all but dead will never move to Python 3 in any case. Zope/CMF/Plone works like this. I don't understand. AFAICT, Zope *is* a library, i.e. you have to run setup.py for lots of packages. Do you not have to run setup.py, for, say, zope.interface, or zope.psycopgda? No, Zope is not a library, it's an application. No, you typically do not setup packages, although most (but not all) parts of Zope 3 is setup if you run Zope in a buildout configuration. Zope 2 does not. The Plone collective works like this, and it is *not* reasonably well managed, so there software quite often doens't get released, but people run against trunk. And that's fine. You still can integrate 2to3 with that transparently. I can not see how that would work. I still don't see why that is. In the examples you gave, no such difficulties are apparent. Maybe it's not apparent to people that hasn't developed in that kind of environment, and I'm sorry I'm not able to make this clearer. But that's just the way it is. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Mon, Mar 24, 2008 at 11:29 AM, Thomas Wouters [EMAIL PROTECTED] wrote: safe assumption to make. A simple preprocessor step involving 2to3 requires no such knowledge. As I understood it nobody has claimed 2to3 to be perfect either, but that using 2to3 will also require you to test the code in both environments, so I don't see how that is a difference. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
I don't understand. AFAICT, Zope *is* a library, i.e. you have to run setup.py for lots of packages. Do you not have to run setup.py, for, say, zope.interface, or zope.psycopgda? No, Zope is not a library, it's an application. No, you typically do not setup packages, although most (but not all) parts of Zope 3 is setup if you run Zope in a buildout configuration. Zope 2 does not. I was talking about http://svn.zope.org/zope.psycopgda/trunk/ Is that not the right source? In any case, using 2to3 for applications is even easier than using it for libraries, assuming there is an installation procedure for the application. The Plone collective works like this, and it is *not* reasonably well managed, so there software quite often doens't get released, but people run against trunk. And that's fine. You still can integrate 2to3 with that transparently. I can not see how that would work. To me run against trunk means that I check out some source, then run setup.py install (or buildout, or make). This will invoke 2to3 behind the scenes. I don't know exactly how plone works, but I could imagine that it's also possible to run 2to3 when plone loads a component (and I'm sure I'm using the wrong words here). Maybe it's not apparent to people that hasn't developed in that kind of environment, and I'm sorry I'm not able to make this clearer. But that's just the way it is. I guess I could better understand with a very specific example. You gave Django as a very specific example, and I looked at Django, and it works just fine with 2to3. The Plone collective is not a specific example, as it doesn't allow me to reproduce your problems. What is the specific thing I would want to do with it, and what specific source repository do I need to check out to do that? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Mon, Mar 24, 2008 at 12:26 PM, Martin v. Löwis [EMAIL PROTECTED] wrote: http://svn.zope.org/zope.psycopgda/trunk/ Is that not the right source? No, and yes. Many of the zope3 modules are installable as separate modules. Zope 3 in general has been eggified. This is not however how everybody installs Zope3. Zope2 is not installed this way. Also, most Zope2 products are not modules installed by setup.py. To me run against trunk means that I check out some source, then run setup.py install (or buildout, or make). This will invoke 2to3 behind the scenes. I repeat: There is no setup.py install to run. I guess I could better understand with a very specific example. You gave Django as a very specific example, and I looked at Django, and it works just fine with 2to3. The Plone collective is not a specific example It is a specific example. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On 24/03/2008, Lennart Regebro [EMAIL PROTECTED] wrote: On Mon, Mar 24, 2008 at 11:29 AM, Thomas Wouters [EMAIL PROTECTED] wrote: safe assumption to make. A simple preprocessor step involving 2to3 requires no such knowledge. As I understood it nobody has claimed 2to3 to be perfect either, but that using 2to3 will also require you to test the code in both environments, so I don't see how that is a difference. Do you not test the code you distribute under each version of Python that you (claim to) support, in any case? If not, what does supported under Python 2.4 mean to you? Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Mon, Mar 24, 2008 at 1:03 PM, Paul Moore [EMAIL PROTECTED] wrote: On 24/03/2008, Lennart Regebro [EMAIL PROTECTED] wrote: On Mon, Mar 24, 2008 at 11:29 AM, Thomas Wouters [EMAIL PROTECTED] wrote: safe assumption to make. A simple preprocessor step involving 2to3 requires no such knowledge. As I understood it nobody has claimed 2to3 to be perfect either, but that using 2to3 will also require you to test the code in both environments, so I don't see how that is a difference. Do you not test the code you distribute under each version of Python that you (claim to) support, in any case? Yes I do. I don't think my text above was unclear on the issue or could be misunderstood in this way. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On 24/03/2008, Lennart Regebro [EMAIL PROTECTED] wrote: On Mon, Mar 24, 2008 at 1:03 PM, Paul Moore [EMAIL PROTECTED] wrote: On 24/03/2008, Lennart Regebro [EMAIL PROTECTED] wrote: As I understood it nobody has claimed 2to3 to be perfect either, but that using 2to3 will also require you to test the code in both environments, so I don't see how that is a difference. Do you not test the code you distribute under each version of Python that you (claim to) support, in any case? Yes I do. I don't think my text above was unclear on the issue or could be misunderstood in this way. Your statement using 2to3 will also require you to test the code in both environments seemed to me to say that *not* having to use 2to3 would save you from doing this (as if this were either desirable, or your current practice). My apologies if I misread your comment. What *are* you trying to say, then? It seems that you're saying that using 2to3 doesn't make things any more complex, but that contradicts your previous argument. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Mon, Mar 24, 2008 at 1:26 PM, Paul Moore [EMAIL PROTECTED] wrote: Your statement using 2to3 will also require you to test the code in both environments seemed to me to say that *not* having to use 2to3 would save you from doing this (as if this were either desirable, or your current practice). I think maybe you missed the statement I responded to, claiming that 2to3 would require no knowledge about the differences between Python 2.6 and 3.0, implying that you could just run it, and it would always work, which I don't believe. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I've missed most of this thread, but let me just put my two cents in. I'd still like a future import to turn on unicode string literals (or more accurately, treat unadorned string literals as unicodes). As someone who is striving mightily to get various libraries and applications unicode clean, it's simply a matter of training my brain to correctly think, this is a bytes or this is a string, and training my fingers to type the right thing. I'd like to be able to start retraining the muscle memory so that by the time 3.0 comes around, it'll will be a much smoother transition, for me and other coders. Now, if it's not feasible to add such a future import, that's one thing. If it's feasible then I think we should do it. - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR+exl3EjvBPtnXfVAQLa0QQAl07BSwokgspNoIT0s2nn3kcWDn//PBmM ARgUCwd2fwZhHkiFsx5YgfzHJaBOuQjPNM4jOwUVy8vZpwUEVZNWmWE7rh+AHxQD FFLyier6/O1PkIe4US1RwuE3/53viP2qWo2Fr0z4zwbJbI6QOQvRVZeZ6OhU02jn GsNFuhuBz58= =1hYN -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
Lennart Regebro wrote: On Mon, Mar 24, 2008 at 12:26 PM, Martin v. Löwis [EMAIL PROTECTED] wrote: I guess I could better understand with a very specific example. You gave Django as a very specific example, and I looked at Django, and it works just fine with 2to3. The Plone collective is not a specific example It is a specific example. No it isn't. A specific example would be I have environment X setup on a machine. I go to website/SVN repository Y and retrieve source code Z and start using it. Without the step by step process, it is impossible to identify if there is any point in the procedure where an invocation of 2to3 could be inserted relatively painlessly. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Mon, Mar 24, 2008 at 4:34 PM, Nick Coghlan [EMAIL PROTECTED] wrote: No it isn't. A specific example would be I have environment X setup on a machine. I go to website/SVN repository Y and retrieve source code Z and start using it. I have environment Plone setup on a machine. I also have several products from the Plone collective which I use from the custom product that contains the custom site code. Thats it. It is a specific example. I can't get more specific than that without you learning Plone. Without the step by step process, it is impossible to identify if there is any point in the procedure where an invocation of 2to3 could be inserted relatively painlessly. It can't. That's the whole point. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
For example, if I'm using the (real source) py2.6 code, and I create a patch that works for me, it is ready for testing and submission. If I'm using the (generated) py3 code, then I first have to get a copy of the (source) 2.6, figure out how I *would* have written it there, then keep tweaking it so that the generator eventually puts out ... what I had originally written by hand. Yes, that's tedious. In that case, it is easier to edit the original source, and then rerun 2to3, rather than editing the compiler output. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Proposal: from __future__ import unicode_string_literals
Maybe it's not apparent to people that hasn't developed in that kind of environment, and I'm sorry I'm not able to make this clearer. I think I understand the issue. Some contributors will be running under 2.6, others will be running under 3.0. Either the code forks, or one of them is working with (and developing patches against) the result of a compilation step, instead of against the original source code. For example, if I'm using the (real source) py2.6 code, and I create a patch that works for me, it is ready for testing and submission. If I'm using the (generated) py3 code, then I first have to get a copy of the (source) 2.6, figure out how I *would* have written it there, then keep tweaking it so that the generator eventually puts out ... what I had originally written by hand. My (working in 3.0) task would be easier if there is also a 3to2 (so that I can treat my own code as the source), but then entire files will do flip-flops on a regular basis (depening on which version was generated), unless 2to3 and 3to2 somehow create a perfect round-trip. And that compile step -- it can be automated, but I suspect most python projects don't yet have a good place to put the hooks, simply because they haven't needed to in the past. The end result is that the barrier to contributions becomes much higher for people working in at least one of the versions. -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
Barry Warsaw schrieb: I've missed most of this thread, but let me just put my two cents in. I'd still like a future import to turn on unicode string literals (or more accurately, treat unadorned string literals as unicodes). As someone who is striving mightily to get various libraries and applications unicode clean, it's simply a matter of training my brain to correctly think, this is a bytes or this is a string, and training my fingers to type the right thing. I'd like to be able to start retraining the muscle memory so that by the time 3.0 comes around, it'll will be a much smoother transition, for me and other coders. Now, if it's not feasible to add such a future import, that's one thing. If it's feasible then I think we should do it. I'm working on a node based __future__ parser based on the Python 2.3 code as MvL suggested. I'm making good progress. If I get it working it should be trivial to implement a __future__ unicode_string_literals feature. Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
I have environment Plone setup on a machine. I also have several products from the Plone collective which I use from the custom product that contains the custom site code. Thats it. It is a specific example. I can't get more specific than that without you learning Plone. Ok, let me try to guess, then. You use https://svn.plone.org/svn/collective/LatexTool/ and you perform an svn checkout into /var/lib/zope2.10/instance/plone-site/Products You start zope, edit the source, or perform a svn update, and then restart or refresh the product. Correct? If so, there is a fairly simple way to integrate 2to3: In OFS.Application.import_product, run 2to3 first before importing the product, if a file run2to3.txt exists in the product's root directory. This will perform a copy of the product into /var/lib/zope2.10/instance/plone-site/Products.2to3 Voila, transparent invocation of 2to3. Now, if you want pdb to display the original source rather than the one being converted, subclass pdb and strip out .2to3 from the source filename. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Mon, Mar 24, 2008 at 8:30 PM, Martin v. Löwis [EMAIL PROTECTED] wrote: Ah, I see. Your point was that with enough magic there is some way to run 2to3 automatically. Sure, I never doubted that. I don't see how that changes anything I said really. I still think the forwards compatibility that exists in 2.6 is a good idea, and the more of it the better. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
Martin v. Löwis wrote: For example, if I'm using the (real source) py2.6 code, and I create a patch that works for me, it is ready for testing and submission. If I'm using the (generated) py3 code, then I first have to get a copy of the (source) 2.6, figure out how I *would* have written it there, then keep tweaking it so that the generator eventually puts out ... what I had originally written by hand. Yes, that's tedious. In that case, it is easier to edit the original source, and then rerun 2to3, rather than editing the compiler output. This technique has actually been the one recommended by Guido for migration for at least a year now. Clearly you can't have developers tweaking source on both sides of the great divide, as one may have to re-cast bits of one's 2.6 code in order to get a satisfactory translation into 3.0. Once you start editing 3.0 source you have to either leave the 2.X world behind or accept a dual-source development. regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Proposal: from __future__ import unicode_string_literals
Eric Smith wrote: It's not implementable because the work has to occur in ast.c (see Py_UnicodeFlag). It can't occur later, because you need to skip the encoding being done in parsestr(). But the __future__ import can only be interpreted after the AST is built, at which time the encoding has already been applied. There are some radical things you could do to work around this, but it would be a gigantic change. Oh, swear words! Pretty much. And even if it were possible, I don't see the point in doing it. The point is this: http://regebro.wordpress.com/2008/03/22/why-python-26-and-30-compatibility-would-be-a-very-good-thing/ Basically, the 2to3 strategy on large codebases supported by a wide set of developers isn't likely to be maintanable, and without a gradual path forward, 3.0 isn't likely to be adopted in some parts of the Python community. Worst case this splits the efforts of the community into two groups, which would be damaging to Python as a whole. Howver: If 2.6 doesn't support this it's not the end of the world. Because if it turn out to be necessary, a 2.7 that does could always be released. I don't know about other large codebases like Twisted, but the Zope/Plone world isn't likely to even try moving to 3.0 any time soon after it's final release, so since it's complicated you can probably wait with this support until it turns out to be needed. M.-A. Lemburg wrote: Could we point them to a special byte-code compiler such as Andrew Dalke's python4ply: ??? I'm not sure what this means... :) -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Sun, Mar 23, 2008 at 8:37 AM, Lennart Regebro [EMAIL PROTECTED] wrote: Eric Smith wrote: It's not implementable because the work has to occur in ast.c (see Py_UnicodeFlag). It can't occur later, because you need to skip the encoding being done in parsestr(). But the __future__ import can only be interpreted after the AST is built, at which time the encoding has already been applied. There are some radical things you could do to work around this, but it would be a gigantic change. Oh, swear words! I don't believe this to be true (we can simply store the source encoding information (which it might already be) and translate the *use* of string literals into unicode(literal, encoding).) But I still think this is a bad idea. Using the same codebase for 3.0 and 2.6 will leave you with inefficient and fragile code in one of the two codebases -- quite likely both. I know, I've tried. I don't see how it improves maintainability to leave your source full of forward- and backward-compatibility hacks. 3.0 was meant to be a clean break. Please treat it as such. Yes, that means you can't run the same source tree in two different Python versions -- too bad. It means adding a compilation-style step to one of the two Python versions -- too bad. It's quite easily automated, as proven by the vast majority of programming languages out there, which need such a step anyway. Howver: If 2.6 doesn't support this it's not the end of the world. Because if it turn out to be necessary, a 2.7 that does could always be released. I don't know about other large codebases like Twisted, but the Zope/Plone world isn't likely to even try moving to 3.0 any time soon after it's final release, so since it's complicated you can probably wait with this support until it turns out to be needed. That was always the assumption, and also the idea for 2.6 and 2.7. I would much rather 3.0 isn't widely accepted for a few years longer than that it be cluttered with backward compatibility crap, and even rather than that widely used code be riddled with such. But it's up to Guido in the end. -- Thomas Wouters [EMAIL PROTECTED] Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Sun, Mar 23, 2008 at 2:13 PM, Thomas Wouters [EMAIL PROTECTED] wrote: That was always the assumption, and also the idea for 2.6 and 2.7. I would much rather 3.0 isn't widely accepted for a few years longer than that it be cluttered with backward compatibility crap, and even rather than that widely used code be riddled with such. But it's up to Guido in the end. Sure but this is a bit misleading. The risk isn't that 3.0 is not widely accepted quickly. The risk is that the community splits in two. And the suggestion is not for backwards compatibility in 3.0, but forwards compatibility in 2.6. So the question is rather: Do you want to disk a community split, or add forwards compatibility? But as I noted, if it turns out to be necessary to add that forwards compatibility, it's always possible to do a 2.7 that has it. I have loads of experience in writing code for evolving APIs, this is how things have been done in the Zope community for years. It's not a problem. It does not lead to fragile code. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
So the question is rather: Do you want to disk a community split, or add forwards compatibility? I don't think the risk is big. As far as people start saying I will only support Python 3, or saying I will not support Python 3 - that's fine. In the former case, people can still continue to use the old versions of the software (assuming we are talking about open source here), and continue to use those with 2.x. They won't get all the new features, and perhaps that is a reason for them to move to 3.x. In the latter case, people relying on the library either have to stay with 2.x until all their dependencies get ported, or they will have to contribute 3.x ports themselves to the developers. In some cases, this may cause a fork of the project, but I guess these cases are rare (and occur only if the maintainer is not cooperative in at least incorporating patches even if its for stuff he doesn't care about). So in short: no, the risk that the community splits is very small. When people contribute code, it's not because of care about the community, but because of their own needs. That's how open-source software works. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Sun, Mar 23, 2008 at 10:45 PM, Martin v. Löwis [EMAIL PROTECTED] wrote: So the question is rather: Do you want to disk a community split, or add forwards compatibility? I don't think the risk is big. As far as people start saying I will only support Python 3, or saying I will not support Python 3 - that's fine. No, it isn't. That's the whole thing. It isn't fine. In the latter case, people relying on the library either have to stay with 2.x until all their dependencies get ported, or they will have to contribute 3.x ports themselves to the developers. You are still only seeing this as a case of libraries with a small number of people developing them and making regular well defined releases. That is not how the world I am talking about looks. I repeat: I have no doubt the 2to3 approach works well in that case, if you want to support both 2.6 and 3.0. So in short: no, the risk that the community splits is very small. No, it is a significant risk. Don't brush it away. We do NOT end up having a 2.x python world and a 3.x python world. The community doesn't have the resources to maintain momentum in a language if the energy is divided in half. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
You are still only seeing this as a case of libraries with a small number of people developing them and making regular well defined releases. That is not how the world I am talking about looks. Can you give me examples of such software? Are you perhaps talking about closed source software? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Mon, 24 Mar 2008 00:14:13 +0100, \Martin v. Löwis\ [EMAIL PROTECTED] wrote: You are still only seeing this as a case of libraries with a small number of people developing them and making regular well defined releases. That is not how the world I am talking about looks. Can you give me examples of such software? Are you perhaps talking about closed source software? I'm not sure what software he was talking about. I can say that for the work I do on both Twisted and Divmod software, I'd be quite happy to see this feature. As either part of a migration path towards 3k _or_ as a feature entirely on its own merits, this would be very useful to me. I'm a bit curious about why Thomas said this sort of thing results in fragile code. Twisted has been using __future__ imports for years and they've never been a problem. Twisted currently supports Python 2.3 through Python 2.5, and the only thing that's really difficult about that is subtle changes in library behavior, not syntax. I'm also curious about why Lennart thinks that this would only be relevant for large projects with lots of developers making regular releases. Sure, I'd use it in that scenario, but that's because it's a subset of all development. ;) Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
Eric Smith wrote: This proposal is to add from __future__ import unicode_string_literals, which would make all string literals in the importing module into unicode objects in 2.6. I'm going to withdraw this, for 2 reasons. 1) The more I think about it, the less sense it makes. 2) Without some extreme measures, it's not implementable. It's not implementable because the work has to occur in ast.c (see Py_UnicodeFlag). It can't occur later, because you need to skip the encoding being done in parsestr(). But the __future__ import can only be interpreted after the AST is built, at which time the encoding has already been applied. There are some radical things you could do to work around this, but it would be a gigantic change. As for it not making sense, this is really in the realm of 2to3. I'm beginning to really believe this statement in PEP 3000: There is no requirement that Python 2.6 code will run unmodified on Python 3.0. Not even a subset. (Of course there will be a tiny subset, but it will be missing major functionality.) For this particular issue, just use u'' in 2.6 and let 2to3 deal with it. If you have some 2.6 code that you want to run in 3.0 (by way of 2to3), I think all of your string literals should either be b'' or u''. Don't use plain ''. Eric. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
Eric Smith schrieb: It's not implementable because the work has to occur in ast.c (see Py_UnicodeFlag). It can't occur later, because you need to skip the encoding being done in parsestr(). But the __future__ import can only be interpreted after the AST is built, at which time the encoding has already been applied. There are some radical things you could do to work around this, but it would be a gigantic change. So this basically comes down to Either spend lots of time (and money) to rewrite the tokenizer and AST generator or keep the current behavior? :/ For this particular issue, just use u'' in 2.6 and let 2to3 deal with it. If you have some 2.6 code that you want to run in 3.0 (by way of 2to3), I think all of your string literals should either be b'' or u''. Don't use plain ''. For this particular issue one could probably and easily come up with a fast fixer. A simple regexp should be cover 99% of all occurrences of u'' and u. Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
Christian Heimes wrote: Eric Smith schrieb: It's not implementable because the work has to occur in ast.c (see Py_UnicodeFlag). It can't occur later, because you need to skip the encoding being done in parsestr(). But the __future__ import can only be interpreted after the AST is built, at which time the encoding has already been applied. There are some radical things you could do to work around this, but it would be a gigantic change. So this basically comes down to Either spend lots of time (and money) to rewrite the tokenizer and AST generator or keep the current behavior? :/ Pretty much. And even if it were possible, I don't see the point in doing it. For this particular issue, just use u'' in 2.6 and let 2to3 deal with it. If you have some 2.6 code that you want to run in 3.0 (by way of 2to3), I think all of your string literals should either be b'' or u''. Don't use plain ''. For this particular issue one could probably and easily come up with a fast fixer. A simple regexp should be cover 99% of all occurrences of u'' and u. 2to3 already does this. My current thinking is that only b'' and u'' strings should be in 2.6 code that you want to move to 3.0. Maybe -3 should warn about regular string literals? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Fri, Mar 21, 2008 at 11:06 AM, Eric Smith [EMAIL PROTECTED] wrote: Christian Heimes wrote: Eric Smith schrieb: It's not implementable because the work has to occur in ast.c (see Py_UnicodeFlag). It can't occur later, because you need to skip the encoding being done in parsestr(). But the __future__ import can only be interpreted after the AST is built, at which time the encoding has already been applied. There are some radical things you could do to work around this, but it would be a gigantic change. So this basically comes down to Either spend lots of time (and money) to rewrite the tokenizer and AST generator or keep the current behavior? :/ Pretty much. And even if it were possible, I don't see the point in doing it. For this particular issue, just use u'' in 2.6 and let 2to3 deal with it. If you have some 2.6 code that you want to run in 3.0 (by way of 2to3), I think all of your string literals should either be b'' or u''. Don't use plain ''. For this particular issue one could probably and easily come up with a fast fixer. A simple regexp should be cover 99% of all occurrences of u'' and u. 2to3 already does this. My current thinking is that only b'' and u'' strings should be in 2.6 code that you want to move to 3.0. Maybe -3 should warn about regular string literals? That's a possibility. It might also help to have a 3to2 fixer that goes through a module and adds the needed prefixes so one doesn't have to go through manually to tack them on. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
It's not implementable because the work has to occur in ast.c (see Py_UnicodeFlag). It can't occur later, because you need to skip the encoding being done in parsestr(). But the __future__ import can only be interpreted after the AST is built, at which time the encoding has already been applied. I think it would be possible to check for future statements on the basis of nodes already. Take a look at how Python 2.3 implemented future statements (why was that rewritten to use the AST, anyway?). As for it not making sense, this is really in the realm of 2to3. I'm beginning to really believe this statement in PEP 3000: There is still the original use case of people who don't want to run 2to3 (for whatever reasons - mostly probably subjective ones), and who would rather run a single code base unmodified. They don't care that documentation tells them this is impossible, when they feel they are so close to making it possible. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On 2008-03-21 22:32, Martin v. Löwis wrote: It's not implementable because the work has to occur in ast.c (see Py_UnicodeFlag). It can't occur later, because you need to skip the encoding being done in parsestr(). But the __future__ import can only be interpreted after the AST is built, at which time the encoding has already been applied. I think it would be possible to check for future statements on the basis of nodes already. Take a look at how Python 2.3 implemented future statements (why was that rewritten to use the AST, anyway?). As for it not making sense, this is really in the realm of 2to3. I'm beginning to really believe this statement in PEP 3000: There is still the original use case of people who don't want to run 2to3 (for whatever reasons - mostly probably subjective ones), and who would rather run a single code base unmodified. They don't care that documentation tells them this is impossible, when they feel they are so close to making it possible. Could we point them to a special byte-code compiler such as Andrew Dalke's python4ply: http://dalkescientific.com/Python/python4ply.html That approach appears to be a lot easier to implement than trying to tweak the C implementation of the Python parser. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 21 2008) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Proposal: from __future__ import unicode_string_literals
Following up on a python-3000 discussion about making porting from 2.6 to 3.0 easier. Martin suggested making this its own thread. This proposal is to add from __future__ import unicode_string_literals, which would make all string literals in the importing module into unicode objects in 2.6. This is similar to the -U flag, but would only affect a single module at a time. I think history has shown that -U isn't really usable when using any number of modules, including many in the standard library. There was another proposal from Christian Heimes to add from __future__ import py3k_literals, which would: 1) '' creates an unicode object instead of a str object 2) b'' creates a str object (aka bytes in Python 3.0) 3) 1 creates a long instead of an int 4) 1L and u'' are invalid 2) is already taken care of in 2.6, since: type(b'') == str. I don't think 3) is necessary. It's an implementation detail. 4) is really two issues. It's my understanding that there's a 2to3 fixer for both of these issues. But I'm open to debate on this. I'm willing to implement this if there's consensus on it. Eric. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
