Re: [Python-Dev] windows standard [was: PEP 365 (Adding thepkg_resources module)]
Jim Jewett [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] | Terry Reedy | The standard (and to me, preferable) way of dealing | with such things is to have an 'installation manager' | that can reinstall as well as delete and that has a | check box for various things to delete. This is what | Python needs. 'such things' referred to 'add-ons' as discussed in snipped text both mine and Paul's. | Paul Moore: | I'd dispute strongly that this is a standard. | It may be preferable, but I'm not sure where you | see evidence of it being a standard. | | When I install a large program (such as developer tools, or python | itself) on Windows, I expect a choice of default or custom. When | I choose custom, I expect a list of components, which can be chosen, | not chosen, or mixed (meaning that it has subcomponents, only some of | which are chosen). | | The whole thing only shows up once in Add/Remove programs. If I | select it, I do get options to Change or Repair. These let me change | my mind on which subcomponents are installed. This is what I am requesting for Python. | If I install python and then separately install Zope, it may or may | not make sense for Zope to be listed separately as a program to Add | or Remove. Neither Paul nor I defined 'add-on', but I would be willing to call Zope/Plone something more than that, preferably with its own multi-option entry. tjr ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Mon, Mar 24, 2008 at 12:14 AM, Martin v. Löwis [EMAIL PROTECTED] wrote: You are still only seeing this as a case of libraries with a small number of people developing them and making regular well defined releases. That is not how the world I am talking about looks. Can you give me examples of such software? I'll repeat the link where I explained my point on this: http://regebro.wordpress.com/2008/03/22/why-python-26-and-30-compatibility-would-be-a-very-good-thing/ -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Mon, Mar 24, 2008 at 2:30 AM, Jean-Paul Calderone [EMAIL PROTECTED] wrote: On Mon, 24 Mar 2008 00:14:13 +0100, \Martin v. Löwis\ [EMAIL PROTECTED] wrote: You are still only seeing this as a case of libraries with a small number of people developing them and making regular well defined releases. That is not how the world I am talking about looks. Can you give me examples of such software? Are you perhaps talking about closed source software? I'm not sure what software he was talking about. I can say that for the work I do on both Twisted and Divmod software, I'd be quite happy to see this feature. As either part of a migration path towards 3k _or_ as a feature entirely on its own merits, this would be very useful to me. I'm a bit curious about why Thomas said this sort of thing results in fragile code. Twisted has been using __future__ imports for years and they've never been a problem. Twisted currently supports Python 2.3 through Python 2.5, and the only thing that's really difficult about that is subtle changes in library behavior, not syntax. I wasn't talking about future imports. I was talking about writing Python 2.6 code and expecting it to work *the same way* in 3.0 without modification. It requires all programmers working on the project to have knowledge of how 3.0 and 2.6 differ. *I* can't even look at code and tell you how 2.6 and 3.0 will differ. Since Lennart was talking specifically about projects with a large number of developers, I do not believe this is a safe assumption to make. A simple preprocessor step involving 2to3 requires no such knowledge. Or, alternatively, a preprocessor step involving 3to2, which I think will result in better code. Unfortunately I don't have time to work on either anytime soon. -- Thomas Wouters [EMAIL PROTECTED] Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
Nuxeo CPS worked like this, but we can ignore them as that project is all but dead will never move to Python 3 in any case. Zope/CMF/Plone works like this. I don't understand. AFAICT, Zope *is* a library, i.e. you have to run setup.py for lots of packages. Do you not have to run setup.py, for, say, zope.interface, or zope.psycopgda? It's fine that all people develop on the same subversion repository. Doing so integrates nicely with 2to3. The Plone collective works like this, and it is *not* reasonably well managed, so there software quite often doens't get released, but people run against trunk. And that's fine. You still can integrate 2to3 with that transparently. I know Django people both strive to support multiple versions of Python and regularily run their production sites on trunk. Insane, perhaps, but that's how things are. And no need to change that, see http://wiki.python.org/moin/PortingDjangoTo3k Just to repeat myself: With that patch to Django, you can a) support all versions of Python simultaneously, from 2.x to 3.0 b) work from the subversion trunk all the time c) transparently invoke 2to3 on the trunk; you won't even notice that you do (except for all the diffs it currently prints to stdout also :-) FWIW, your approach (of adding __future__ imports to 2.6) wouldn't help Django at all, as they would need to give up 2.5 and earlier. So, in short: Large projects with interconnected modules where the developers and users of module are the same people will have big difficulties with the 2to3 approach and would be the people who are most likely to not be able to in practice go forward to Python 3 unless they have some sort of smooth path forward. I still don't see why that is. In the examples you gave, no such difficulties are apparent. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Mon, Mar 24, 2008 at 11:27 AM, Martin v. Löwis [EMAIL PROTECTED] wrote: Nuxeo CPS worked like this, but we can ignore them as that project is all but dead will never move to Python 3 in any case. Zope/CMF/Plone works like this. I don't understand. AFAICT, Zope *is* a library, i.e. you have to run setup.py for lots of packages. Do you not have to run setup.py, for, say, zope.interface, or zope.psycopgda? No, Zope is not a library, it's an application. No, you typically do not setup packages, although most (but not all) parts of Zope 3 is setup if you run Zope in a buildout configuration. Zope 2 does not. The Plone collective works like this, and it is *not* reasonably well managed, so there software quite often doens't get released, but people run against trunk. And that's fine. You still can integrate 2to3 with that transparently. I can not see how that would work. I still don't see why that is. In the examples you gave, no such difficulties are apparent. Maybe it's not apparent to people that hasn't developed in that kind of environment, and I'm sorry I'm not able to make this clearer. But that's just the way it is. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Mon, Mar 24, 2008 at 11:29 AM, Thomas Wouters [EMAIL PROTECTED] wrote: safe assumption to make. A simple preprocessor step involving 2to3 requires no such knowledge. As I understood it nobody has claimed 2to3 to be perfect either, but that using 2to3 will also require you to test the code in both environments, so I don't see how that is a difference. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3127 (Integer Literal Support and Syntax): %o and %b
Guido van Rossum wrote: On Tue, Mar 18, 2008 at 9:11 PM, Eric Smith [EMAIL PROTECTED] wrote: I've been double checking the PEP 3127 implementation in py3k and the backport I did to 2.6. The PEP says this about the % operator: The string (and unicode in 2.6) % operator will have 'b' format specifier added for binary, and the alternate syntax of the 'o' option will need to be updated to add '0o' in front, instead of '0'. The %b operator was not added to 3.0, so I'll look into doing that in both 2.6 and 3.0 (which I opened as issue 2416). What should be done for '%#o' formatting in 2.6? The above sentence from the PEP implies it should be modified to add '0o' instead of '0', even in 2.6. But that seems like a bad idea to me. Maybe it should stay as-is, but add a -3 warning? Unfortunately, there'd be no way to change your code to get rid of the warning, short of switching to str.format() or adding a __future__ import (shudder). In 3.0, '%#o' already adds the leading '0o'. I think this is such a tiny detail we shouldn't bother with a -3 warning. I agree that in 2.6, %#o should continue to do what it does in 2.5. Can you update the PEP? Done in r61845. Eric. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
I don't understand. AFAICT, Zope *is* a library, i.e. you have to run setup.py for lots of packages. Do you not have to run setup.py, for, say, zope.interface, or zope.psycopgda? No, Zope is not a library, it's an application. No, you typically do not setup packages, although most (but not all) parts of Zope 3 is setup if you run Zope in a buildout configuration. Zope 2 does not. I was talking about http://svn.zope.org/zope.psycopgda/trunk/ Is that not the right source? In any case, using 2to3 for applications is even easier than using it for libraries, assuming there is an installation procedure for the application. The Plone collective works like this, and it is *not* reasonably well managed, so there software quite often doens't get released, but people run against trunk. And that's fine. You still can integrate 2to3 with that transparently. I can not see how that would work. To me run against trunk means that I check out some source, then run setup.py install (or buildout, or make). This will invoke 2to3 behind the scenes. I don't know exactly how plone works, but I could imagine that it's also possible to run 2to3 when plone loads a component (and I'm sure I'm using the wrong words here). Maybe it's not apparent to people that hasn't developed in that kind of environment, and I'm sorry I'm not able to make this clearer. But that's just the way it is. I guess I could better understand with a very specific example. You gave Django as a very specific example, and I looked at Django, and it works just fine with 2to3. The Plone collective is not a specific example, as it doesn't allow me to reproduce your problems. What is the specific thing I would want to do with it, and what specific source repository do I need to check out to do that? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Mon, Mar 24, 2008 at 12:26 PM, Martin v. Löwis [EMAIL PROTECTED] wrote: http://svn.zope.org/zope.psycopgda/trunk/ Is that not the right source? No, and yes. Many of the zope3 modules are installable as separate modules. Zope 3 in general has been eggified. This is not however how everybody installs Zope3. Zope2 is not installed this way. Also, most Zope2 products are not modules installed by setup.py. To me run against trunk means that I check out some source, then run setup.py install (or buildout, or make). This will invoke 2to3 behind the scenes. I repeat: There is no setup.py install to run. I guess I could better understand with a very specific example. You gave Django as a very specific example, and I looked at Django, and it works just fine with 2to3. The Plone collective is not a specific example It is a specific example. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] windows standard [was: PEP 365 (Adding thepkg_resources module)]
On 24/03/2008, Terry Reedy [EMAIL PROTECTED] wrote: | If I install python and then separately install Zope, it may or may | not make sense for Zope to be listed separately as a program to Add | or Remove. Neither Paul nor I defined 'add-on', but I would be willing to call Zope/Plone something more than that, preferably with its own multi-option entry. Fair comment. I'd agree that Zope/Plone are probably more than an add-on. Actually, we agree - *if* you accept that my definition of add-on excludes anything you download separately, which has its own installer - which is what a bdist_wininst exe is. Of course, conversely, if all Python packages are add-ons (and so have their own internal means of installing - easy_install, for argument's sake - and listing/uninstalling - the mythical new package manager) then they shouldn't have add/remove program entries (any more than each MS Office option should). The purist in me says you can't have it both ways. But practicality says it's not a major issue (as long as there's *one* option that covers everything, no matter how it's installed). Personally, my only concern is having a single tool that can manage *all* of my non-core Python packages. (One ring to rule them all, and all that :-)) I have that at the moment, by refusing to use easy_install and eggs. But it's getting harder to do that, so this thread, for me, is about finding a better option. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On 24/03/2008, Lennart Regebro [EMAIL PROTECTED] wrote: On Mon, Mar 24, 2008 at 11:29 AM, Thomas Wouters [EMAIL PROTECTED] wrote: safe assumption to make. A simple preprocessor step involving 2to3 requires no such knowledge. As I understood it nobody has claimed 2to3 to be perfect either, but that using 2to3 will also require you to test the code in both environments, so I don't see how that is a difference. Do you not test the code you distribute under each version of Python that you (claim to) support, in any case? If not, what does supported under Python 2.4 mean to you? Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Mon, Mar 24, 2008 at 1:03 PM, Paul Moore [EMAIL PROTECTED] wrote: On 24/03/2008, Lennart Regebro [EMAIL PROTECTED] wrote: On Mon, Mar 24, 2008 at 11:29 AM, Thomas Wouters [EMAIL PROTECTED] wrote: safe assumption to make. A simple preprocessor step involving 2to3 requires no such knowledge. As I understood it nobody has claimed 2to3 to be perfect either, but that using 2to3 will also require you to test the code in both environments, so I don't see how that is a difference. Do you not test the code you distribute under each version of Python that you (claim to) support, in any case? Yes I do. I don't think my text above was unclear on the issue or could be misunderstood in this way. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On 24/03/2008, Lennart Regebro [EMAIL PROTECTED] wrote: On Mon, Mar 24, 2008 at 1:03 PM, Paul Moore [EMAIL PROTECTED] wrote: On 24/03/2008, Lennart Regebro [EMAIL PROTECTED] wrote: As I understood it nobody has claimed 2to3 to be perfect either, but that using 2to3 will also require you to test the code in both environments, so I don't see how that is a difference. Do you not test the code you distribute under each version of Python that you (claim to) support, in any case? Yes I do. I don't think my text above was unclear on the issue or could be misunderstood in this way. Your statement using 2to3 will also require you to test the code in both environments seemed to me to say that *not* having to use 2to3 would save you from doing this (as if this were either desirable, or your current practice). My apologies if I misread your comment. What *are* you trying to say, then? It seems that you're saying that using 2to3 doesn't make things any more complex, but that contradicts your previous argument. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Mon, Mar 24, 2008 at 1:26 PM, Paul Moore [EMAIL PROTECTED] wrote: Your statement using 2to3 will also require you to test the code in both environments seemed to me to say that *not* having to use 2to3 would save you from doing this (as if this were either desirable, or your current practice). I think maybe you missed the statement I responded to, claiming that 2to3 would require no knowledge about the differences between Python 2.6 and 3.0, implying that you could just run it, and it would always work, which I don't believe. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I've missed most of this thread, but let me just put my two cents in. I'd still like a future import to turn on unicode string literals (or more accurately, treat unadorned string literals as unicodes). As someone who is striving mightily to get various libraries and applications unicode clean, it's simply a matter of training my brain to correctly think, this is a bytes or this is a string, and training my fingers to type the right thing. I'd like to be able to start retraining the muscle memory so that by the time 3.0 comes around, it'll will be a much smoother transition, for me and other coders. Now, if it's not feasible to add such a future import, that's one thing. If it's feasible then I think we should do it. - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR+exl3EjvBPtnXfVAQLa0QQAl07BSwokgspNoIT0s2nn3kcWDn//PBmM ARgUCwd2fwZhHkiFsx5YgfzHJaBOuQjPNM4jOwUVy8vZpwUEVZNWmWE7rh+AHxQD FFLyier6/O1PkIe4US1RwuE3/53viP2qWo2Fr0z4zwbJbI6QOQvRVZeZ6OhU02jn GsNFuhuBz58= =1hYN -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
Lennart Regebro wrote: On Mon, Mar 24, 2008 at 12:26 PM, Martin v. Löwis [EMAIL PROTECTED] wrote: I guess I could better understand with a very specific example. You gave Django as a very specific example, and I looked at Django, and it works just fine with 2to3. The Plone collective is not a specific example It is a specific example. No it isn't. A specific example would be I have environment X setup on a machine. I go to website/SVN repository Y and retrieve source code Z and start using it. Without the step by step process, it is impossible to identify if there is any point in the procedure where an invocation of 2to3 could be inserted relatively painlessly. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Mon, Mar 24, 2008 at 4:34 PM, Nick Coghlan [EMAIL PROTECTED] wrote: No it isn't. A specific example would be I have environment X setup on a machine. I go to website/SVN repository Y and retrieve source code Z and start using it. I have environment Plone setup on a machine. I also have several products from the Plone collective which I use from the custom product that contains the custom site code. Thats it. It is a specific example. I can't get more specific than that without you learning Plone. Without the step by step process, it is impossible to identify if there is any point in the procedure where an invocation of 2to3 could be inserted relatively painlessly. It can't. That's the whole point. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
For example, if I'm using the (real source) py2.6 code, and I create a patch that works for me, it is ready for testing and submission. If I'm using the (generated) py3 code, then I first have to get a copy of the (source) 2.6, figure out how I *would* have written it there, then keep tweaking it so that the generator eventually puts out ... what I had originally written by hand. Yes, that's tedious. In that case, it is easier to edit the original source, and then rerun 2to3, rather than editing the compiler output. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Proposal: from __future__ import unicode_string_literals
Maybe it's not apparent to people that hasn't developed in that kind of environment, and I'm sorry I'm not able to make this clearer. I think I understand the issue. Some contributors will be running under 2.6, others will be running under 3.0. Either the code forks, or one of them is working with (and developing patches against) the result of a compilation step, instead of against the original source code. For example, if I'm using the (real source) py2.6 code, and I create a patch that works for me, it is ready for testing and submission. If I'm using the (generated) py3 code, then I first have to get a copy of the (source) 2.6, figure out how I *would* have written it there, then keep tweaking it so that the generator eventually puts out ... what I had originally written by hand. My (working in 3.0) task would be easier if there is also a 3to2 (so that I can treat my own code as the source), but then entire files will do flip-flops on a regular basis (depening on which version was generated), unless 2to3 and 3to2 somehow create a perfect round-trip. And that compile step -- it can be automated, but I suspect most python projects don't yet have a good place to put the hooks, simply because they haven't needed to in the past. The end result is that the barrier to contributions becomes much higher for people working in at least one of the versions. -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
Barry Warsaw schrieb: I've missed most of this thread, but let me just put my two cents in. I'd still like a future import to turn on unicode string literals (or more accurately, treat unadorned string literals as unicodes). As someone who is striving mightily to get various libraries and applications unicode clean, it's simply a matter of training my brain to correctly think, this is a bytes or this is a string, and training my fingers to type the right thing. I'd like to be able to start retraining the muscle memory so that by the time 3.0 comes around, it'll will be a much smoother transition, for me and other coders. Now, if it's not feasible to add such a future import, that's one thing. If it's feasible then I think we should do it. I'm working on a node based __future__ parser based on the Python 2.3 code as MvL suggested. I'm making good progress. If I get it working it should be trivial to implement a __future__ unicode_string_literals feature. Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
I have environment Plone setup on a machine. I also have several products from the Plone collective which I use from the custom product that contains the custom site code. Thats it. It is a specific example. I can't get more specific than that without you learning Plone. Ok, let me try to guess, then. You use https://svn.plone.org/svn/collective/LatexTool/ and you perform an svn checkout into /var/lib/zope2.10/instance/plone-site/Products You start zope, edit the source, or perform a svn update, and then restart or refresh the product. Correct? If so, there is a fairly simple way to integrate 2to3: In OFS.Application.import_product, run 2to3 first before importing the product, if a file run2to3.txt exists in the product's root directory. This will perform a copy of the product into /var/lib/zope2.10/instance/plone-site/Products.2to3 Voila, transparent invocation of 2to3. Now, if you want pdb to display the original source rather than the one being converted, subclass pdb and strip out .2to3 from the source filename. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py3k and asyncore/asynchat
I'm going to refresh this discussion since it seems no decisions are still taken. Any chance to see a commit finally done? --- Giampaolo http://code.google.com/p/pyftpdlib ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
On Mon, Mar 24, 2008 at 8:30 PM, Martin v. Löwis [EMAIL PROTECTED] wrote: Ah, I see. Your point was that with enough magic there is some way to run 2to3 automatically. Sure, I never doubted that. I don't see how that changes anything I said really. I still think the forwards compatibility that exists in 2.6 is a good idea, and the more of it the better. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__ import unicode_string_literals
Martin v. Löwis wrote: For example, if I'm using the (real source) py2.6 code, and I create a patch that works for me, it is ready for testing and submission. If I'm using the (generated) py3 code, then I first have to get a copy of the (source) 2.6, figure out how I *would* have written it there, then keep tweaking it so that the generator eventually puts out ... what I had originally written by hand. Yes, that's tedious. In that case, it is easier to edit the original source, and then rerun 2to3, rather than editing the compiler output. This technique has actually been the one recommended by Guido for migration for at least a year now. Clearly you can't have developers tweaking source on both sides of the great divide, as one may have to re-cast bits of one's 2.6 code in order to get a satisfactory translation into 3.0. Once you start editing 3.0 source you have to either leave the 2.X world behind or accept a dual-source development. regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py3k and asyncore/asynchat
On Fri, Feb 15, 2008 at 9:11 PM, Josiah Carlson [EMAIL PROTECTED] wrote: Twisted core has been proposed, but I believe the consensus was that it wasn't desirable, generally. I remember only a couple of dissenting voices, and only a small number of participants. Of the dissenting voices, I do not recall any actual arguments about undesireability, just misunderstandings of how Twisted actually works. Getting Twisted core (meaning Deferreds, a simple reactor and the Protocol class) into the core is still on my TODO list. I'm also pretty sure that people learn twisted because everyone learns twisted. It's one of those buzz-words ;). I think that's quite an unfair assessment, even in jest :) Twisted is well worth learning to actually use it, as it's a very versatile event loop and does it best to integrate nicely with other event systems. And including it in the standard library improves integration with other event loops by creating a single interface. It's not a matter of dropping it in, though; it requires some careful structuring to avoid embarrassing situations like we have with the xml package, but still people to provide their own reactor. In case you're wondering how the twisted reactor in the stdlib is useful to people not using Twisted, take a look at what you currently need to do to combine stdlib modules like urllib and ftplib with event systems like Tkinter and PyGTK. Not to mention that the Twisted implementations of various protocols are really quite, quite good -- in many cases quite a lot better than the stdlib ones. But including those takes yet more time. -- Thomas Wouters [EMAIL PROTECTED] Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py3k and asyncore/asynchat
On Mon, Mar 24, 2008 at 3:04 PM, Thomas Wouters [EMAIL PROTECTED] wrote: On Fri, Feb 15, 2008 at 9:11 PM, Josiah Carlson [EMAIL PROTECTED] wrote: Twisted core has been proposed, but I believe the consensus was that it wasn't desirable, generally. I remember only a couple of dissenting voices, and only a small number of participants. Of the dissenting voices, I do not recall any actual arguments about undesireability, just misunderstandings of how Twisted actually works. Getting Twisted core (meaning Deferreds, a simple reactor and the Protocol class) into the core is still on my TODO list. I'm also pretty sure that people learn twisted because everyone learns twisted. It's one of those buzz-words ;). I think that's quite an unfair assessment, even in jest :) Twisted is well worth learning to actually use it, as it's a very versatile event loop and does it best to integrate nicely with other event systems. And including it in the standard library improves integration with other event loops by creating a single interface. It's not a matter of dropping it in, though; it requires some careful structuring to avoid embarrassing situations like we have with the xml package, but still people to provide their own reactor. In case you're wondering how the twisted reactor in the stdlib is useful to people not using Twisted, take a look at what you currently need to do to combine stdlib modules like urllib and ftplib with event systems like Tkinter and PyGTK. Not to mention that the Twisted implementations of various protocols are really quite, quite good -- in many cases quite a lot better than the stdlib ones. But including those takes yet more time. In that sense it'd be competing with safethread for inclusion in Python. Whereas safethread requires little if any API changes, twisted requires an entirely new API that can be event-driven. Worse, we'd likely to be stuck maintaining both APIs for a long time, if not forever. Twisted may be one of the best (if not *the* best) ways of writing concurrent programs today, but it doesn't need to be in the stdlib for that. If safethread is going to solve many of the same problems, with less changes required by the users of the language, then this is the wrong time to add twisted. -- Adam Olsen, aka Rhamphoryncus ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py3k and asyncore/asynchat
On Mon, Mar 24, 2008 at 10:21 PM, Adam Olsen [EMAIL PROTECTED] wrote: On Mon, Mar 24, 2008 at 3:04 PM, Thomas Wouters [EMAIL PROTECTED] wrote: On Fri, Feb 15, 2008 at 9:11 PM, Josiah Carlson [EMAIL PROTECTED] wrote: Twisted core has been proposed, but I believe the consensus was that it wasn't desirable, generally. I remember only a couple of dissenting voices, and only a small number of participants. Of the dissenting voices, I do not recall any actual arguments about undesireability, just misunderstandings of how Twisted actually works. Getting Twisted core (meaning Deferreds, a simple reactor and the Protocol class) into the core is still on my TODO list. I'm also pretty sure that people learn twisted because everyone learns twisted. It's one of those buzz-words ;). I think that's quite an unfair assessment, even in jest :) Twisted is well worth learning to actually use it, as it's a very versatile event loop and does it best to integrate nicely with other event systems. And including it in the standard library improves integration with other event loops by creating a single interface. It's not a matter of dropping it in, though; it requires some careful structuring to avoid embarrassing situations like we have with the xml package, but still people to provide their own reactor. In case you're wondering how the twisted reactor in the stdlib is useful to people not using Twisted, take a look at what you currently need to do to combine stdlib modules like urllib and ftplib with event systems like Tkinter and PyGTK. Not to mention that the Twisted implementations of various protocols are really quite, quite good -- in many cases quite a lot better than the stdlib ones. But including those takes yet more time. In that sense it'd be competing with safethread for inclusion in Python. Whereas safethread requires little if any API changes, twisted requires an entirely new API that can be event-driven. Worse, we'd likely to be stuck maintaining both APIs for a long time, if not forever. I am not going to worry about this for the time being. Even if safethread goes in and becomes ubiquitous, Twisted is still a very valid approach to the problem. And I'm not just saying that because I don't subscribe to your enthusiasm of safethreads as a concept or as an implementation :) Twisted may be one of the best (if not *the* best) ways of writing concurrent programs today, but it doesn't need to be in the stdlib for that. If safethread is going to solve many of the same problems, with less changes required by the users of the language, then this is the wrong time to add twisted. You must have missed the part where we already have a large set of event loops, and not having a single interface to them is in fact hurting people. Twisted goes out of its way to interact nicely with event loops, but it can only do that with ones it knows about (and are versatile enough to hook into.) Having a single event system in the standard library is definitely advantageous, even if safethreads were available everywhere and the performance in the common case was satisfactory. It used to be the case that people thought asyncore was this standard event system, but it's long since ceased to be. -- Thomas Wouters [EMAIL PROTECTED] Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Martin v. Löwis wrote: Sure, but what is precisely the semantics of uninstallation, in terms of changes to the system state? I think any model where uninstallation is merely the removal of files is too limited to be practical. The distutils only support the *addition* of files, so I'm not sure how only removing files is a limit here. Could you explain? For files, yes, it only supports addition. But it supports arbitrary other actions, such as: - addition of registry keys - addition of user accounts - creation of database tables in a relational database - updating the shared library loader path - creation and start of a system service - integration of documentation into info - registration of DTDs with the system catalog - ... It's turing-complete, and it has full interface to the operating system, so installation of a distutils package can do *much* more than merely installing files. Which is exactly what is wrong with distutils: turing completeness in an installer is an *anti* goal, from the perspective of manageability. I'd be willing to bet that if you asked system packagers to list the dozen or so packages which they *hate* to maintain, the large majority of each list would be packages which acutally use the full power of distutils. (Note: I'm aware that people believe it to be necessary to munge the Windows registry when installing Python packages; I just don't agree with the practice, and don't think we should distort Python's process to coddle it). Uninstallation needs to revert anything installation did, so it is often more than mere removal of files. Practically speacking, nobody but the author of the TC-abusing setup.py is *ever* going to be able to do this, and most of them won't get the edge cases right. Maybe we can just punt on such packages, and make a tool which actually works for the huge majority of distributions which don't need to do more than install files. Tres. - -- === Tres Seaver +1 540-429-0999 [EMAIL PROTECTED] Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD4DBQFH6Cdr+gerLs4ltQ4RAlFzAJi3gs8fzb9o8/Dtct1G9P0EJxNSAKCc7V7m uT5MgTzltBDhdrgoNxt8nA== =zgqI -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py3k and asyncore/asynchat
On Mon, Mar 24, 2008 at 10:04:20PM +0100, Thomas Wouters wrote: I remember only a couple of dissenting voices, and only a small number of participants. Of the dissenting voices, I do not recall any actual arguments Weren't some of those dissenting voices the Twisted developers, though? --amk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Martin v. Löwis wrote: Oh, and application installation is (should be) completely different. On Windows, applications should probably be bundled with their own Python interpreter, a la py2exe. On Unix/Linux, I don't know what the standard is, so I'd have to defer to others. This I disagree with. I think it's an overall bad thing to have all kinds of applications ship their own copy of Python; see also Aza Raskin's PyCon keynote. On Linux, python typically comes with the system pre-installed; it is not even an option not to have it, except for minimalist installations. So if you write python scripts, you typically expect that #!/usr/bin/env python works; you might put python2.5 there if you don't trust that system one is good enough. For installing the application, you typically want the choice betaween a system installation (in /usr/bin, or perhaps /usr/local/bin), and a user installation. As distutils supports both cases, it works fairly well for applications as well. Sharing the system python is hugely problematic on a unix box which actually *uses* python for its own tools: the application is not safe from additions / updates / removeals of the packages in /usr/lib/python2.x/site-packages done to support those system tools. The problem gets worse as multiple non-system applications install files there: e.g., the 'twisted' package on Debian boxes depends on an ancient version of 'zope.interface', which can't be used with any currently supported version of Zope. virtualenv makes using the system python on such systems somewhat more tolerable, because each virtualenv can be isolated from the site-packages of the host environment (and therefore from the other applications). You still have to live with the choices the system packagers make (UCS4, for instance), but the pain is at least tolerable. For a long-running Python application (as opposed to desktop tools or scripts), installing a custom Python is the safest choice: it insulates the application not only from unexpected system-driven site-packages changes, but also allows greater control over other Python configuration choices (the UCS2 / UCS4 knob, etc.). Tres. - -- === Tres Seaver +1 540-429-0999 [EMAIL PROTECTED] Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFH6Cqi+gerLs4ltQ4RAlqZAKCxr2lraLSycVsksYAevtf+urALOgCeLzs9 fE2g7IAb+22B+UbSUFPqj4w= =re0h -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py3k and asyncore/asynchat
Let us not get side-tracked in this discussion. Whether or not to include any portion of Twisted into Python 2.6 is well past being a reasonable question; 2.6 alpha 1 has been released. It's a question as to whether someone with commit access can or will commit the patch as posted, run the tests to verify that they aren't broken, and perhaps actually look at the code to verify that we (Giampaolo and I) aren't insane. Mind you, I've been using very similar variants of this patch for months; it fixes outstanding bugs, improves performance, makes the handle* interfaces more consistent, and even offers a 'sample' implementation of a basic protocol in the source (for those who are willing to look). Do we want to fix asyncore/asynchat with work that has already been done and tested? If you want a reason as to why twisted shouldn't *replace* asyncore/asynchat, I'll give you a few: As stated previously by Guido and others (please see previous threads about this over the course of the last 4 years), asyncore/asynchat provide a more or less minimal interface for asynchronous sockets in Python. Any module/package that seeks to implement asynchronous sockets will need to, at least, implement that much. Asyncore/asynchat at present will play nicely with any event loop available, given certain caveats*. Further, if someone spends a half hour reading the source of a reasonably written asyncore server/client, they should understand the basics and be able to begin using it directly (see any one of the simple echo variants). As to whether twisted should be added to the standard library at some point in the future as a this is better than asyncore in every way, use this instead; that is a different discussion, not related to 2.6 (perhaps not even related to the 2.x series at all, depending on the future of 2.x). - Josiah * If your application strictly responds to socket IO, then implement your application as part of handle_* methods. If your application does more, then call asyncore.poll() often enough for data to be handled sufficiently fast. If neither offer sufficient performance or flexibility (maybe something like bittorrent + wxPython), use one thread for your GUI, one thread for your sockets, and use Queue.Queue() or some other threadsafe message delivery method. On Mon, Mar 24, 2008 at 3:18 PM, A.M. Kuchling [EMAIL PROTECTED] wrote: On Mon, Mar 24, 2008 at 10:04:20PM +0100, Thomas Wouters wrote: I remember only a couple of dissenting voices, and only a small number of participants. Of the dissenting voices, I do not recall any actual arguments Weren't some of those dissenting voices the Twisted developers, though? --amk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/josiah.carlson%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Commit access request
I've attached my SSH keys. On Mon, Mar 24, 2008 at 6:56 AM, Benjamin Peterson [EMAIL PROTECTED] wrote: Hi Python devs, I have been contributing to since December. (See me first issue on the tracker, #1828; it was a major learning experience.) :P In that time, I have contributed many patches and actively participated on this list. This will enable me to help triage bugs on the tracker, something I've been wanting to help with. I'm willing to field questions. -- Thanks for your consideration, Benjamin Peterson -- Cheers, Benjamin Peterson id_dsa.pub Description: Binary data id_rsa.pub Description: Binary data ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: from __future__import unicode_string_literals
| Just to repeat myself: With that patch to Django, you can | a) support all versions of Python simultaneously, from 2.x to 3.0 I find this surprising for two reasons. 1. I had the impression from discussions over the past year that fully automatic use of 2to3 would presume use of 2.6 and its backported 3.0 features and __future__ imports. If it really works with ealier 2.x code, great, but please pardon any initial surprise or scepticism. This is precisely why I started this porting experiment. If you are still skeptic, please substantiate your skepticism with facts: run my patch, and tell me why it doesn't work, or couldn't be completed. If you are now merely surprised, but not skeptic anymore: my pleasure. The believe that you must port to 2.6 first is wide-spread. It probably originates from the statement that the official porting strategy involves porting to 2.6 first. That strategy does so, however, to enable you to run the -3 option, so you can find porting problems more easily. If you can find the porting problems the hard way, i.e. by running the software and testing it, you don't need the -3 warnings. When I started a week ago, a few essential 2to3 fixers did not exist (in particular the one David Wolever wrote to make implicit relative imports explicit). That fixer really falls into the 2to2.5 category; it would have been possible to change the code to use relative imports everywhere, thereby breaking 2.3 compatibility. It is possible that other examples like this still exist (i.e. 2to3 doesn't fix it, but doesn't have to if you can assume 2.5), but I'm not aware of any (actually, that's not entirely true - the email module renaming is of the same kind. However, this can be dealt with by ImportError guards. Still, having a fixer for that might be useful) 2. You report has caveats such as * there are certainly large parts of the code base that I haven't touched, so more issues are likely to show up True. *This port attempts to maintain compatibility with older Python versions from a single code base; the goal is to support all versions that Django supports, plus 3.0. The current patch fails to do so in certain details, due to bugs in the 2to3 tool. *This approach mostly works, and has the following flaws: some of the fixers work incorrectly (bugs 2453, 2446, 2468) These bugs are really shallow, and some have been fixed already. *I have worked with sqlite3 only; all the other databases have not been tested. True. So your unqualified assertion seems more like an anticipated future (certain to you but maybe not to others) than present reality. Likewise, the statement that you *can't* possibly use the same code base from 2.1 to 3.0 is unfounded, and, unlike my claim, doesn't have any kind of proof behind it. 3. Also, you said you worked around some 2to3 failings with conditional blocks like, I presume, the following. if sys.version (3,0,0): old 2.x code else: revised 3.0 code Do I assume correctly that you, rather than 2to3 had to do such? Indeed. Will 2to3 remove the wrapper to leave just the 3.0 code? Currently, it leaves the code unchanged. It could fairly easily remove it, but doing so might shift line numbers, which in turn is undesirable. 2to3 has support for optional fixers, and that might be a use case. Or would someone have to go thru by hand to get clean 3.0 code? See above. Writing a fixer for it is fairly easy, and somebody will certainly do that, but I would only run it when using a burn the bridges run. I understand that this is a standard method for multiple release code, but it seems a bit like cheating since the point of 2to3 is to not to have to do this. Or is converting 'types.ClassType' to 'types' a future fixer item? No. This is the sort of change that 2to3 can't do right. If Django would require Python 2.5, the conditional could go away, as this appears in a context of creating exception classes on-the-fly; they can be new-style classes in 2.5. However, I consider conditional code blocks not as cheating at all. If you want to provide backwards compatibility, you *always* have to compromise readability for portability. This was even the case within 2.x, where you can't use True and False if you want to support Python versions that didn't have it, or where you can use generators, or iterators, or ..., when older Python versions need to be supported. The main point of 2to3 is not to have the 3.x code look nice, in fact, in many cases, it won't, since 2to3 must make conservative assumptions in some cases, so to not break the semantics of the program. Instead, the main point of 2to3 is to replace syntax that is going away with the 3.x equivalent syntax. In the cases where I conventionalize on the Python version, it's not syntax that has changed. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org
Re: [Python-Dev] [Python-checkins] r61709 - python/trunk/Doc/library/functions.rst python/trunk/Doc/library/future_builtins.rst python/trunk/Doc/library/python.rst
Jim Jewett wrote: In 2.5, the print statement ignores any overrides of the str builtin, but I'm not sure whether a _function_ should -- and I do think it should be specified. I expect there are plenty of other things that use str()-like behaviour without going through str(), so making print alone do this probably wouldn't be very useful. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python source code on Bazaar vcs
Barry All the gory details are documented here: Barry http://www.python.org/dev/bazaar Thanks. I checked out, made a branch named test3, changed Makefile.pre.in to have a test3 target, checked it in, then tried to push it: % pwd /Users/skip/src/python-bzr/test3 % bzr push bzr+ssh://[EMAIL PROTECTED]/python/users/skip/test3 bzr: ERROR: Parent directory of bzr+ssh://[EMAIL PROTECTED]/python/users/skip/test3 does not exist. You may supply --create-prefix to create all leading parent directories. Did I misread the directions or do I really need the --create-prefix arg? Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
On Mar 22, 5:13 pm, Martin v. Löwis [EMAIL PROTECTED] wrote: Can you give me a pointer to Aza Raskin's keynote? Is it online anywhere? I'd be interested in his point of view. Unfortunately no. I was looking for it, but couldn't find it. He mentioned a website with a call for action, but I couldn't find that, either :-( I guess the website could be http://www.toolness.com/wp/?p=23#more-23 - Python as a Platform. Via Ned Batchelder's notes at http://nedbatchelder.com/blog/200803/pycon_2008_notes.html From the post: Something that recently occurred to me is that the only operating system that doesn't come with Python pre-installed on it is Windows. While Linux and OS X both view Python as essentially a first-class development platform-i.e., as something that shrink-wrap applications can be built on-Windows does not. Instead, it's generally expected that a Python-based Windows application be frozen: bundled into a self-contained package that includes a copy of the Python interpreter and whatever libraries it uses, which are private to the particular application. While this ensures that the application will function as expected and not run into 'dependency hell', it also results in a relatively large download-distributing a simple 'Hello World' program is at least a megabyte in size, and makes extending the program's functionality more difficult. Regards, Daniel ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] 2to3 speedup patch
Under the instruction of Martin, I've made some small changes to 2to3 so keeps track of which fixers act on which level of node. The speedup isn't too shabby: running on the example file, processing time went from 9 to 7 seconds, and the test suite dropped from 400 to 350. I have attached the hacky, ugly, proof-of-concept patch to http:// bugs.python.org/issue2431 If there's no reason not to implement this sort of thing, I'll clean it up and commit it when I get home (or something). -- David Wolever - http://wolever.net/~wolever AIM: davidswolever MSN: [EMAIL PROTECTED] Phone: 416-906-0403 Without payment you have received; without payment you are to give. (Mat 10:8 ISV) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond
On Fri, Mar 21, 2008 at 10:04:45PM -0400, Phillip J. Eby wrote: At 02:31 AM 3/22/2008 +0100, Martin v. Löwis wrote: However, I'm extremely skeptical that this can ever succeed to the degree that whoever provides RPMs, .debs, or MSI files will actually use such data, as they will find that the data are incomplete, and they have to redo all of it, anyway. The data isn't for them to use to meet their use cases, it's for them to *provide* so that Python tools don't stomp on, uninstall, or otherwise interfere with files installed by the system. In other words, for system packagers, it's a communication from the system to Python, rather than the other way around. Even though the distutils will build the file in the bdist, the system packaging tools would be free to generate their own file listing and signatures and such. As long as systems (dpkg, rpm, ...) install the .egg-info files they do communicate which modules/distributions are installed. The installdb would just duplicate this information (according to the current PEP). And as I said, I'll be happy if all we do is get the distutils to abide by the spec for 2.6, even if it means we don't get an uninstall tool. That can always be installed later using Guido's bootstrap tool. :) I'm even more skeptical here. If the assumption is that the package database allows for uninstallation, I'm -1. IOW, RPM, deb, MSI should then *not* write to that package database, as they also write to a different database, out of the scope of the PEP, and this is what uninstallation should use. I probably should have brought this up, in fact, I think I mentioned it in a previous thread, but I would like to see PEP 262 add a way to say this is a system-installed package, *don't touch*. The idea again is not to do the job of the native packaging system, but rather to ensure that Python-specific tools (e.g. easy_install and friends) do not interfere or conflict with it. A big problem in the early development of easy_install, even using eggs, was that there was no way to tell whether it was safe to delete or overwrite an existing file or directory that was already installed on the system. A mechanism like this would allow tools like easy_install to say, oh, your system packager has a conflicting package here, you need to use that tool to sort this out if you really want to do something here. I'm not going to touch that. Without something like this, there is no way to tell the difference on many systems between a system package and something the user has put there with sudo python setup.py install. There is a way of telling if you have to keep you hands off a package (sorry to bring this up again): installation paths. /usr/lib is the system path, the local admin (and hence setuptools) should keep their hands off it at all times (unless requested with a --prefix or so for building the debs or rpms but an uninstall in those cases won't be required from setuptools). What an installdb could help with is tell setuptools to keep it's hands off a distribution manually installed (or by another tool) in the local admin directory (/usr/local) or user directory (~/). Your proposal of an installdb for each directory on sys.path would solve this, but the installdb in /usr/lib will be completely usused. But frankly, for that just an extra field in the .egg-info Installed-By: setuptools would do no? Possibly followed by a X-Setuptools-Installed-Files: section if you need that. -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond
On Sat, Mar 22, 2008 at 12:33:49PM +0100, Martin v. Löwis wrote: The data isn't for them to use to meet their use cases, it's for them to *provide* so that Python tools don't stomp on, uninstall, or otherwise interfere with files installed by the system. In other words, for system packagers, it's a communication from the system to Python, rather than the other way around. Even though the distutils will build the file in the bdist, the system packaging tools would be free to generate their own file listing and signatures and such. Ok, that's a reasonable requirement. It will be difficult to implement, though, as it will require Linux distributors (in particular) to include the database snippets in their packages. Essentially, one would have to contribute patches to all the distributions (we care about, at least), and then nag the respective maintainers to include these patches. Not true. You just need to make sure that setup.py install creates that database. With the proposed format of the database this is just a file in the correct location - exactly for this reason. Next time the distribution will build the package that database file will be in place. I still maintain that an installdb for the system packages doesn't bring anything to the party as it would be in a system-only directory. But I've argued that in my other emails... Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond
On Sat, Mar 22, 2008 at 03:14:05PM +0100, Martin v. Löwis wrote: Essentially, one would have to contribute patches to all the distributions (we care about, at least), and then nag the respective maintainers to include these patches. Not true. You just need to make sure that setup.py install creates that database. With the proposed format of the database this is just a file in the correct location - exactly for this reason. Next time the distribution will build the package that database file will be in place. How so? Are you /sure/ the packaging process even *runs* setup.py? And if they do, why do you think they will pick up the database file? I speak for Debian, so for Debian: yes. The setup.py would have to be pretty bad for a packager to not use it. There is no reason to re-write upstream's installation procedure as you would have to figure out which files need to be installed where and this would create many bugs. The canonical way is something like this: $ pythonX.Y setup.py --root=$(pwd)/debian/tmp $ # Fixup anything done wrong/badly (for debian) by setup.py $ # Make a tarball of $(pwd)/debian/tmp In reality it's slightly more complicated of course. At [1] there are many packages, paste and parallelpython are good examples if you're interested (look in the debian/rules file). Regards Floris [1] http://svn.debian.org/wsvn/python-modules/packages/?rev=0sc=0 -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond
On Sat, Mar 22, 2008 at 04:42:36PM +0100, Martin v. Löwis wrote: I speak for Debian, so for Debian: yes. The setup.py would have to be pretty bad for a packager to not use it. There is no reason to re-write upstream's installation procedure as you would have to figure out which files need to be installed where and this would create many bugs. The canonical way is something like this: $ pythonX.Y setup.py --root=$(pwd)/debian/tmp $ # Fixup anything done wrong/badly (for debian) by setup.py $ # Make a tarball of $(pwd)/debian/tmp In reality it's slightly more complicated of course. More than slightly so, with pycentral, pysupport, and all that. I still doubt that the database would show up in a directory on sys.path. If not it would be a bug in pycentral/pysupport. Only two bugs to file, not that bad I think! At [1] there are many packages, paste and parallelpython are good examples if you're interested (look in the debian/rules file). I started with nevow. It uses cdbs, so its debian/rules is nearly empty, and does not include a call to setup.py at all. Instead, the distutils support is in /usr/share/cdbs/1/class/python-distutils.mk [...] Again, that would be a bug in CDBS. For the specific snippet you showed, yes that does essentially pythonX.Y setup.py --root=$some_alternate_root as I said above. As an aside I also happen to be in the camp that dislikes CDBS... The specifics and complications don't matter for this discussion I think. If setup.py installs the correct file into the installdb then it will work in almost all cases, Neal Becker confirmed this is true for Fedora as well. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Distutils] PEP 365 (Adding the pkg_resources module)
On Mar 20, 2008, at 11:31 AM, Martin v. Löwis wrote: I'll note that I use easy_install *only* to work in *non-system* locations: if I want to install Python packages to /usr/lib/ python2.x/, I use the standard system installer, e.g. 'apt-get install python-frobnatz'. This is probably not the Windows way of doing things (at least not how I use Windows). Windows doesn't really have the notion of system location (or multiple levels of them, where \Windows and \Windows\system32 is more system than \Program Files, say). Windows users typically view the entire system as theirs, and have no concerns at all about putting things into Program Files, system32, or, for that matter, \python25. In fact, they don't care or even know where stuff ends up - they expect that the system will find it, and they expect that they can remove anything they installed even without known where it is - because there is a standard place to look for uninstalling things. While these observations are accurate for most home users, it is worth noting that many IT departments deploy locked-down versions of windows that either have fine-grained group policies to forbid modifications to the system disk (and require the user to write things to a mounted network home directory), or that give write access to the system disk but then re-image it upon reboot. IT departments that deploy this sort of setup usually have the hostile user mentality, and that is strongly correlated, in turn, with users that are reluctant to engage IT to allow them install an app. We have run into this a few times, and it would be good to keep this scenario in mind. -Peter ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond
On Fri, Mar 21, 2008 at 9:31 PM, Martin v. Löwis [EMAIL PROTECTED] wrote: The objections to the PEP remain the same as they were then, though: In the requirements, it says we need, without saying why we need. It then goes on saying we want (rephrased) to duplicate APT and RPM, without saying why we want that, or why that brings us closer to what we need. IOW, the PEP is lacking a rationale. It seems to me that this discussion is being undermined by not acknowledging the many use cases up front. There is no rationale because there are too many tacit rationales. Nevertheless, the many use cases are related at some level and would benefit from some form of lowest-common-denominator support in the standard library. As an example, IF I wanted to use a library to support multi-version packages or one to ensure I had all the dependencies, I would need to know what versions of things were currently installed. I can't create a library to provided these functionalities and use it downstream of the package creator without some form of standardization to report package versions. (Or at least without running into the assimilation problem that setuptools has). My personal use case is for multi-version packages. I deploy many small scripts (not applications) that use an evolving set of base libraries. I don't want the older scripts to hold me back from pushing forward with the base libraries, so I peg the scripts to their respective major versions as I release them. Regards, Alex ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-3000] Python source code on Bazaar vcs
[EMAIL PROTECTED] wrote: Barry All the gory details are documented here: Barry http://www.python.org/dev/bazaar Thanks. I checked out, made a branch named test3, changed Makefile.pre.in to have a test3 target, checked it in, then tried to push it: % pwd /Users/skip/src/python-bzr/test3 % bzr push bzr+ssh://[EMAIL PROTECTED]/python/users/skip/test3 bzr: ERROR: Parent directory of bzr+ssh://[EMAIL PROTECTED]/python/users/skip/test3 does not exist. You may supply --create-prefix to create all leading parent directories. Did I misread the directions or do I really need the --create-prefix arg? Skip I don't know if there's a special setup here, but http://code.python.org/python/users/ doesn't list a skip directory yet. You'll need to use --create-prefix to get bzr to create it. -- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
In article [EMAIL PROTECTED], ajaksu [EMAIL PROTECTED] wrote: [...] While Linux and OS X both view Python as essentially a first-class development platform-i.e., as something that shrink-wrap applications can be built on-Windows does not. Instead, it's generally expected that a Python-based Windows application be frozen: bundled into a self-contained package that includes a copy of the Python interpreter and whatever libraries it uses, which are private to the particular application. While this ensures that the application will function as expected and not run into 'dependency hell', it also results in a relatively large download-distributing a simple 'Hello World' program is at least a megabyte in size, and makes extending the program's functionality more difficult. While it is true that OS X does provide a built-in Python that can be used as a shared component for shrink-wrap applications, it should be noted that py2app, the de facto standard tool for packaging Python apps on OS X, by default includes a private copy of the Python interpreter and library within the application bundle for similar reasons, i.e. avoiding dependency hell. py2app does, however, go to some trouble to analyze dependencies and include only a minimal set of required packages. http://svn.pythonmac.org/py2app/py2app/trunk/doc/index.html#id35 -- Ned Deily, [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
