Re: [Python-Dev] [python-committers] Proposed Python 3.0 schedule
> at such a script, which I promptly stopped looking at as soon as it > worked Which is quite obvious really given that: > # nuke ourselves from argv > del sys.argv[1] is removing the wrong value! Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [python-committers] Proposed Python 3.0 schedule
> > * Better support for 2to3 in distutils (specifically, the support in
> > build_py is stale, plus 'build_scripts' and 'install_data' should
> > convert
> > .py files to py3k syntax.)
>
> Please do create a bug report for that. It sounds like it's easy to
> fix.
Yeah, build_py is fairly easy to fix, but I also needed to extend the
support to build_scripts and install_data. In addition, some already
reported bugs in 2to3 mean that some files fail to convert, and this breaks
the entire process - so as a result I ended up duplicating lib2to3's
'refactor_items()' but with exceptions being logged and ingored rather than
aborting the process. Oh - and I deleted the .bak files (a copy of the
sources are converted, not the sources themselves)
Please see bugs 4072 and 4073 - but as mentioned below, the lack of a test
case means I didn't supply a tested patch.
> > An 'example' project that uses py2k syntax and
> > "just works" on py3k using this strategy might be useful here.
>
> Perhaps pywin32 :-?
>
> I don't think a demo project would do much good, as it doesn't exercise
> all the issues that may occur.
My idea was that the demo project would simply demonstrate the 2to3 concepts
that such a project could use. pywin32 isn't a good example as it has a
very non-trivial setup.py and a large set of C extensions (the demo I had in
mind could avoid C extensions completely - C developers will already assume
#ifdef will be their friend, but .py code is the unknown...)
It would basically be a 'distutils demo', could have a single .py module and
a single .py script. setup.py would support both 2.x and 3.x and would
demonstrate how the source is converted to py3k syntax before it is
installed into the py3k distribution.
It would also provide a useful test case - eg, for the distutils bug above,
I'm not sure how I can (a) demonstrate it is currently broken and (b)
demonstrate a patch corrects the problem.
> > * A standard 'helper script' that allows people to use py3k to
> > execute a py2x syntax script by auto-converting the code. I've
> > a 10ish-line script that uses lib2to3 plus exec() to achieve that
> > result, but a helper in 2to3
> > for this would be nice. For a concrete use-case, we want to keep our
> > distutils script in py2x syntax, but execute it via py3k. Its very
> > possible this already exists and I've just missed it...
>
> For the case of setup.py, I was hoping that it could be written in
> compatible syntax even without needing conversion. That worked fine for
> my Django port. Is that not the case for pywin32?
setup.py catches and examines some exceptions. Consider the more general
case though - pywin32 has a number of tests all of which will also be
maintained in py2x syntax. It is extremely convenient to be able to
execute:
% py3k run2.py my_test.py etc
And have 'my_test.py' (which is 2.x syntax) be executed directly by py3k
without doing a full 'setup.py install' or manually invoking 2to3 via a temp
file, etc. As mentioned, 'run2.py' is quite short and just uses
lib2to3+exec, but I'm not sure everyone will work out how to roll their
own...
Specifically, I believe that a script with similar capabilities could be
installed with py3k in the "scripts" directory and it advertised as a
reasonable way to directly execute your *scripts* which, although py3x
compatible, are being maintained in py2x syntax. Below is my quick attempt
at such a script, which I promptly stopped looking at as soon as it worked
(ie, I'm not sure if all those options are needed, etc), but it does let me
execute my tests using py3k directly from the source tree.
Cheers,
Mark
---
# This is a Python 3.x script to execute a python 2.x script by 2to3'ing it.
import sys
from lib2to3.refactor import RefactoringTool, get_fixers_from_package
fixers = get_fixers_from_package('lib2to3.fixes')
options = dict(doctests_only=False, fix=[], list_fixes=[],
print_function=False, verbose=False,
write=True)
r = RefactoringTool(fixers, options)
script = sys.argv[1]
data = open(script).read()
print("Converting...")
got = r.refactor_string(data, script)
print("Executing...")
# nuke ourselves from argv
del sys.argv[1]
exec(str(got))
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] 2.5.3: assessing commits
I've begun the task of assessing the 2.6 commits, but the job is unexpectedly large. What I did: * Took the output of 'svn log -r60999:66717'. (2.5.2 was branched from rev. 60999, so I'm ignoring commits to the trunk before 2.5.2 was branch, which may miss some things.) * Wrote a little script to explode the log into a separate file for each commit. This resulted in about 2200 files. * Did a long series of 'grep' and 'rm' commands to remove irrelevant files. For example, if the commit touches abc.py, bytesobject.c, bytearrayobject.c, etc. it was removed. * Wrote a little script to find commits that only touch the docs, and moved them aside. At this point I still have 1191 files left. Many of these commits are still irrelevant, but this is a lot for me to look through. A tarball with the remaining commits is at http://www.amk.ca/files/python/2.6-changes.tgz Can we parallelize the job? One person could take commits starting with '01', another with '02', etc. Or each person could assess the commits they made. --amk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Fwd: In Python 2.6, bytes is str]
Christian Heimes wrote: > Guido van Rossum wrote: [...] >> I have no intention of rolling this back. It isn't compatible with >> 3.0, but then, 2.6 and 3.0 aren't in general meant to be compatible -- >> 2.6 is a stepping stone, but that's not the same thing. It is >> backwards compatible with prior versions because it is new in 2.6, so >> it shouldn't break old code. I have only limited sympathy for people >> who don't read documentation in this case. > > Maybe the documentation isn't clear enough what purpose the bytes alias > serves? I spent 5 minutes with the docs but I wasn't able to find a good > explanation of the bytes alias > Yes, I think all that's really needed is a clarification in the documentation. Just so people expect slightly kooky behavior of the kind originally noted. regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Fwd: In Python 2.6, bytes is str]
Guido van Rossum wrote: It was well understood that the bytes "type" in 2.6 and the bytes type in 3.0 would behave quite different. Attempts to come up with a separate bytes type that behaved more like its 3.0 counterpart were doomed, because there just are too many places where the usage was ambiguous. We should probably have written a PEP about this just to prevent the discussion from being repeated all over again in this thread. The only two anticipated *reasonable* uses in 2.6 were the bytes literal (b'abc') and tests for isinstance(x, bytes), which are flags for 2to3 to keep these usages as bytes, not str. Full ack! I replies this to the op on the general Python list: --- I guess you got the intention of the bytes alias wrong. It's partly my fault because I didn't document the bytes alias. We are well aware that the bytes alias in 2.6 isn't fully compatible with the bytes type in 3.0. The bytes alias isn't MEANT to be compatible, too. :) At first I wanted to backport the bytes type from 3.0 to 2.6. But it was too many work and the implications of yet another types were too complex. So I just added the alias in order to help people with writing forward compatible code like e.g. isinstance(egg, bytes). Summa summarum the bytes alias was added for documentary purpose and to aid the 2to3 transition of code where 'str' is ambiguous. -- I have no intention of rolling this back. It isn't compatible with 3.0, but then, 2.6 and 3.0 aren't in general meant to be compatible -- 2.6 is a stepping stone, but that's not the same thing. It is backwards compatible with prior versions because it is new in 2.6, so it shouldn't break old code. I have only limited sympathy for people who don't read documentation in this case. Maybe the documentation isn't clear enough what purpose the bytes alias serves? I spent 5 minutes with the docs but I wasn't able to find a good explanation of the bytes alias Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] porting pycxx and pysvn to python 3.0 hit a problem
On Oct 7, 2008, at 23:31, Martin v. Löwis wrote: In V3 is it your intention that to be a key you must implement tp_hash and tp_richcompare? If not I'll raise a bug against 3.0 on this issue. I believe that cmp/tp_compare are being phased out, although I think there was a heavy debate about this. Given the problems I am seeing with tp_compare I'd advise that you get rid of it for 3.0. The half hearted support is worst then no support. At least if you remove tp_compare it forces a porter to implement tp_richcompare. In any case, I think you really need to implement tp_richcompare. I've added support already in PyCXX. I'll now implement tp_richcompare for pysvn and document this in my PyCXX porting guide. Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] porting pycxx and pysvn to python 3.0 hit a problem
> In V3 is it your intention that to be a key you must implement > tp_hash and tp_richcompare? If not I'll raise a bug against > 3.0 on this issue. I believe that cmp/tp_compare are being phased out, although I think there was a heavy debate about this. In any case, I think you really need to implement tp_richcompare. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-3000] Proposed Python 3.0 schedule
On Oct 7, 2008, at 4:45 PM, Adam Olsen wrote: So what does Qt do when given a file name already using those PUA? Looks like they get passed through untouched when decoded, but will get translated into invalid names upon encoding. Well, I'd say that looks like a bug. It should probably decode those PUA characters as if they were undecodeable sequences so that they too roundtrip properly. So you still have file names you can't open In practical terms, I suspect nobody has ever run into a file which has this problem. You certainly can't say that is the case for Python-3's current behavior; my suspicion is that anyone who uses any non-ascii filenames at all will run into issues with Python3's behavior at least once. , and you're incompatible with what other libraries do. I'm sure there's a situation where that matters, but, at least I can run kpdf /any/arbitrary/file.pdf and have it work. And use the KDE file chooser, and have it able to browse my files, and choose any file, no matter what random characters it has in it. If there is an issue with interfacing to another library, the string can be converted to whatever the other library expects at the interface point... People keep claiming that odd filenames are only going to be an issue for "backup tools", but I don't think that's true. I think it'll be an issue for most any program that reads user-specified files. Whether it be by running Python in an ASCII (e.g. "C") locale when there are files created with UTF-8 names, or by having copied/downloaded a file with an incorrectly encoded name, it's going to come up, and be an irritant when it does. That Qt felt the need to make this change rather strengthens that point IMO... The only thing going for Qt is that they seem specifically interested in latin-1, rather than arbitrary bad names. The latin-1 strings that would correspond to the UTF-8 PUA used would include at least one control character, as well as other unusual bits, so it's pretty unlikely to encounter a real latin-1 file name like that. I'd say they're most concerned about files that their users are likely to run into, yes. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-3000] Proposed Python 3.0 schedule
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Oct 7, 2008, at 6:01 PM, Barry Warsaw wrote: I won't be able to cut another release between the 15th and 5th, so at least that one should be 2 weeks. If we don't need the additional rc, then we can release early, which would put us just before the US Thanksgiving holiday. Er, /3/ weeks between rc2 and rc3. - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSOvf2HEjvBPtnXfVAQJDsQP8DRL2gQDMf1eEvgmmijPtVdbfAypZ1XMY huNzPu91v6dpvrogIP5MJbmJnSnka5yk78JIlkbTU4ZHS0ADsQX+IApU5y/SlO9Y FDtIqb+NFoVRFj5xQaN/EEqO8kNpq3WPmaEQJ4HHeDUIzcrbsPxfCm+vbePgnGzI AwhQqCzmX1I= =aQnH -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-3000] Proposed Python 3.0 schedule
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Oct 7, 2008, at 4:28 PM, Guido van Rossum wrote: 15-Oct-2008 3.0 rc 2 05-Nov-2008 3.0 rc 3 19-Nov-2008 3.0 rc 4 03-Dec-2008 3.0 final I've updated PEP 361 and the Google calendar with this schedule, except that the PEP says that rc3 and rc4 are planned "if needed". - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSOvfg3EjvBPtnXfVAQKDfwP/Sz9Ioe1tIrKtvD7JPG2cg2F+wfDJrc+9 vqfh6/eMWiUIOeSKJu6+gye7oXRcHwQXAPivNza3993HesOu0TjudnwXfkAlfsdE m09Rh70AXQQiY7JX46etugRC4BwkuNeBo253cvmfo6hPK0ZhOHZSy3H1LkhvvLA6 Cq56CVqDUgs= =i/Km -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-3000] Proposed Python 3.0 schedule
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Oct 7, 2008, at 5:47 PM, Nick Coghlan wrote: Barry Warsaw wrote: On Oct 6, 2008, at 9:48 PM, Raymond Hettinger wrote: [Barry Warsaw] So, we need to come up with a new release schedule for Python 3.0. My suggestion: 15-Oct-2008 3.0 beta 4 05-Nov-2008 3.0 rc 2 19-Nov-2008 3.0 rc 3 03-Dec-2008 3.0 final Given what still needs to be done, is this a reasonable schedule? Do we need two more betas? Yes to both questions. I think that's contradictory :). If we need two betas, then 05-Nov becomes beta 5, 19-Nov is rc 2. If we don't need another rc then we can still do a final release on 03-Dec, otherwise we probably go 2 weeks later. I don't want to go much later than that though because then we get into the holiday season. Do we need the full two weeks between rc's? Or is it too much of a pain to cut releases 3 weeks in a row? E.g. something like: 15-Oct-2008 3.0 beta 4 05-Nov-2008 3.0 beta 5 19-Nov-2008 3.0 rc 2 26-Nov-2008 3.0 rc 3 (if needed) 03-Dec-2008 3.0 final I won't be able to cut another release between the 15th and 5th, so at least that one should be 2 weeks. If we don't need the additional rc, then we can release early, which would put us just before the US Thanksgiving holiday. - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSOvcQ3EjvBPtnXfVAQK5mwP9GQfw3zNvGhJWiSkZ2gQ1LNr0rnmfVmpF WcDePkz3e5nsOjtkwiN0rlYHIQE9ySPfvtqqrInBW8y97y79mTjiM4S32XHLyAsd WEWRb0ClcLuZs+JveAb8KF5pO0RlDgX9Dd6puuPr8kGa5aN/rosfsnXra1GrYpj3 JQghQ89JNkE= =+Ymq -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-3000] Proposed Python 3.0 schedule
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Oct 7, 2008, at 4:28 PM, Guido van Rossum wrote: On Mon, Oct 6, 2008 at 5:47 PM, Barry Warsaw <[EMAIL PROTECTED]> wrote: So, we need to come up with a new release schedule for Python 3.0. My suggestion: 15-Oct-2008 3.0 beta 4 05-Nov-2008 3.0 rc 2 19-Nov-2008 3.0 rc 3 03-Dec-2008 3.0 final Given what still needs to be done, is this a reasonable schedule? Do we need two more betas? I know I'm contradicting what I said earlier, but perhaps we should just forget going back to beta and stick to ever-more-perfect release candidates? In other worlds release candidates often contain tons of imperfections (I believe I've seen this both for Java and Windows) and the label "release candidate" more clearly encourages people to download and play with it, which is what we need at this point! Then the schedule would be something like 15-Oct-2008 3.0 rc 2 05-Nov-2008 3.0 rc 3 19-Nov-2008 3.0 rc 4 03-Dec-2008 3.0 final I'm okay with that too. It does seem odd to go back to beta then release another rc. What's in a name, anyway? . And it is good that more people are downloading it now that it's rc. - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSOvb93EjvBPtnXfVAQJTQAP/cmNdzd/SRymxXvW85EnW2NTHUkh1Auw9 bGlbSC0BF2p9ArgbDLPh/X4uatB3UaqoNeq5LTWHL2f9iCnsI7lFMPuexGr+3t4l Xmld8qN77j4GpU6bXL8uUo3/vlhU4MiG5ETl0kMH30f47srOAAGEGZAqW9jAM92I YSkQPSgBdYo= =+s9t -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] porting pycxx and pysvn to python 3.0 hit a problem
On Oct 5, 2008, at 19:47, Martin v. Löwis wrote: Why does "key in wc_status_kind_wc" work when I use an object returned by keys() by not when I use pysvn.wc_status_kind.unversioned? This is too little detail to come up with an explanation. Do your objects support __eq__. Regards, Martin I wrote a smaller version of the code that fails and had a session with gdb. My object implements tp_compare but lookdict() uses richcompare. If richcompare is not implemented do_richcompare() falls back to comparing PyObject * pointers - which is cause of the KeyError as the objects cmp() eq but have different PyObect * values. This is a change from V2 python where to be a key implementing tp_hash and tp_compare is sufficient. In V3 is it your intention that to be a key you must implement tp_hash and tp_richcompare? If not I'll raise a bug against 3.0 on this issue. Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [python-committers] [Python-3000] Proposed Python 3.0 schedule
> Do we need the full two weeks between rc's? If they are just other names for betas, yes. If they are true release candidates (in the sense of "we really want to release this as-is unless somebody tells us why this is a really bad idea"), then no. > Or is it too much of a pain > to cut releases 3 weeks in a row? It's a lot of effort, yes. Also for users, who will have barely installed one release candidate when the next one comes out. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-3000] Proposed Python 3.0 schedule
Barry Warsaw wrote: > On Oct 6, 2008, at 9:48 PM, Raymond Hettinger wrote: > >> [Barry Warsaw] >>> So, we need to come up with a new release schedule for Python 3.0. >>> My suggestion: >>> 15-Oct-2008 3.0 beta 4 >>> 05-Nov-2008 3.0 rc 2 >>> 19-Nov-2008 3.0 rc 3 >>> 03-Dec-2008 3.0 final >>> Given what still needs to be done, is this a reasonable schedule? >>> Do we need two more betas? > >> Yes to both questions. > > I think that's contradictory :). If we need two betas, then 05-Nov > becomes beta 5, 19-Nov is rc 2. If we don't need another rc then we can > still do a final release on 03-Dec, otherwise we probably go 2 weeks > later. I don't want to go much later than that though because then we > get into the holiday season. Do we need the full two weeks between rc's? Or is it too much of a pain to cut releases 3 weeks in a row? E.g. something like: 15-Oct-2008 3.0 beta 4 05-Nov-2008 3.0 beta 5 19-Nov-2008 3.0 rc 2 26-Nov-2008 3.0 rc 3 (if needed) 03-Dec-2008 3.0 final Cheers, Nick. ___ Python-3000 mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/ncoghlan%40gmail.com -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.5.3: call for patches
Allow me to suggest that these get some attention: http://bugs.python.org/issue3677 http://bugs.python.org/issue3367 Kristján ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python3UnicodeDecodeError
On 2008-10-07 22:18, Fred Drake wrote: > On Oct 7, 2008, at 4:06 PM, Martin v. Löwis wrote: >> b) I would propose that the notion of a default encoding is entirely >> eliminated from Python, along with sys.(get|set)defaultencoding > > +1 As already mentioned in my reply to Viktor: +1. It's not adjustable anymore, so we might as well get rid off the sys module APIs. The term "default encoding" itself still has some value in that it is associated with the C API char* encoding used for PyUnicode objects in Python 3.0. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 07 2008) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-3000] Proposed Python 3.0 schedule
On Tue, Oct 7, 2008 at 9:51 AM, James Y Knight <[EMAIL PROTECTED]> wrote: > On Oct 7, 2008, at 3:47 AM, Martin v. Löwis wrote: >>> >>> - Having os.getcwdb isn't much use when you can't even run python in >>> the first place when the current directory has "bad" bytes in it. >> >> That's not true: it *is* of much use. Python will live in /usr/bin, >> which has a nicely-decodable path. >> >>> Currently Python outputs: >>> Could not find platform independent libraries >>> Could not find platform dependent libraries >>> Consider setting $PYTHONHOME to [:] >>> Fatal Python error: Py_Initialize: can't initialize sys standard streams >>> ImportError: No module named encodings.utf_8 >>> Aborted >> >> I can't reproduce that. This happens (for me) when Python lives in >> a directory that has an undecodable path - not when the current >> directory is undecodable. > > Sorry about that: this test was indeed in error: I ran "../python" from an > undecodeable current directory, rather than "/full/path/to/python", or > putting python on the PATH and running it as "python". The first does not > work, but the other more common ways to start it do. > >>> >>> I'm sure there's even more APIs dealing with pathnames, command line >>> arguments, or environment variables that ought to be able to handle both >>> bytes and strings, that currently don't. >> >> Please, no. > > I completely and totally agree with your distate, it's rather gross to allow > bytes-or-str for every API that touches anything like > filenames/argv/environ. That's why I was pushing for the reversible > conversion to str...But if bytes-or-str is the solution that's been chosen > for this issue, it ought to either be fully committed to and implemented, or > at least fully recognized and documented as a half-baked solution. > > Of course, if an reversible encoding into string solution is used instead, > none of these things would need special treatment: they would all work > already. > > FWIW: Qt works fine with undecodeable filenames, and it too uses unicode > strings everywhere in its API. I looked into what it does, and found that it > uses your (Martin)'s original idea for solving this: it stores undecodeable > bytes as characters from 0x10fe00 to 0x10feff (which is valid private-use > codespace). While that might not be ideally correct, since you lose those > 256 PUA characters, even that is IMO better than pushing out bytes to every > API, or worse, giving up and just having python unable to access files, as > it is now. > > See lines 3074: QString::toUtf8() and 3408: QString::fromUtf8()) of > > http://www.google.com/codesearch?q=+show:o7fNK6SzOYs:NO-Bv-AR2rI:toIOngLf1V8&cs_p=http://ie.archive.ubuntu.com/trolltech/pub/qt/snapshots/qt-x11-opensource-src-4.4.0-snapshot-20070402.tar.bz2&cs_f=qt-x11-opensource-src-4.4.0-snapshot-20070402/src/corelib/tools/qstring.cpp So what does Qt do when given a file name already using those PUA? Looks like they get passed through untouched when decoded, but will get translated into invalid names upon encoding. So you still have file names you can't open, and you're incompatible with what other libraries do. The only thing going for Qt is that they seem specifically interested in latin-1, rather than arbitrary bad names. The latin-1 strings that would correspond to the UTF-8 PUA used would include at least one control character, as well as other unusual bits, so it's pretty unlikely to encounter a real latin-1 file name like that. -- Adam Olsen, aka Rhamphoryncus ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-3000] Python3UnicodeDecodeError
On Tue, Oct 7, 2008 at 1:18 PM, Fred Drake <[EMAIL PROTECTED]> wrote: > On Oct 7, 2008, at 4:06 PM, Martin v. Löwis wrote: >> >> b) I would propose that the notion of a default encoding is entirely >>eliminated from Python, along with sys.(get|set)defaultencoding > > +1 I expect that the only effect of this change would be that the filesystem encoding would become the de-facto default encoding for other contexts as well. Not that that is necessarily a bad thing... -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-3000] Proposed Python 3.0 schedule
On Mon, Oct 6, 2008 at 5:47 PM, Barry Warsaw <[EMAIL PROTECTED]> wrote: > So, we need to come up with a new release schedule for Python 3.0. My > suggestion: > > 15-Oct-2008 3.0 beta 4 > 05-Nov-2008 3.0 rc 2 > 19-Nov-2008 3.0 rc 3 > 03-Dec-2008 3.0 final > > Given what still needs to be done, is this a reasonable schedule? Do we > need two more betas? I know I'm contradicting what I said earlier, but perhaps we should just forget going back to beta and stick to ever-more-perfect release candidates? In other worlds release candidates often contain tons of imperfections (I believe I've seen this both for Java and Windows) and the label "release candidate" more clearly encourages people to download and play with it, which is what we need at this point! Then the schedule would be something like 15-Oct-2008 3.0 rc 2 05-Nov-2008 3.0 rc 3 19-Nov-2008 3.0 rc 4 03-Dec-2008 3.0 final -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.5.3: call for patches
On Tue, Oct 07, 2008 at 09:27:24AM +0200, "Martin v. Löwis" wrote: > Within a few weeks, we will release Python 2.5.3. This will be the last > bug fix release of Python 2.5, afterwards, future releases of 2.5 will > only include security fixes, and no binaries (for Windows or OSX) will > be provided anymore (from python.org). I'm going to the library this evening, and can make my task looking through the 2.5->2.6 log for candidates. I won't do anything in Roundup just yet, but I'll put the list in the wiki or post it here, and then we can cut the list down further before creating any new issues (or re-opening old ones) in Roundup. --amk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python3UnicodeDecodeError
On Oct 7, 2008, at 4:06 PM, Martin v. Löwis wrote: b) I would propose that the notion of a default encoding is entirely eliminated from Python, along with sys.(get|set)defaultencoding +1 -Fred -- Fred Drake ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Fwd: In Python 2.6, bytes is str]
On Sun, Oct 5, 2008 at 11:43 PM, Steve Holden <[EMAIL PROTECTED]> wrote: > This does make it look rather as though bytes == str was a decision > whose consequences weren't fully appreciated before implementation. > > Was this horror anticipated? It was well understood that the bytes "type" in 2.6 and the bytes type in 3.0 would behave quite different. Attempts to come up with a separate bytes type that behaved more like its 3.0 counterpart were doomed, because there just are too many places where the usage was ambiguous. We should probably have written a PEP about this just to prevent the discussion from being repeated all over again in this thread. The only two anticipated *reasonable* uses in 2.6 were the bytes literal (b'abc') and tests for isinstance(x, bytes), which are flags for 2to3 to keep these usages as bytes, not str. I have no intention of rolling this back. It isn't compatible with 3.0, but then, 2.6 and 3.0 aren't in general meant to be compatible -- 2.6 is a stepping stone, but that's not the same thing. It is backwards compatible with prior versions because it is new in 2.6, so it shouldn't break old code. I have only limited sympathy for people who don't read documentation in this case. --Guido > Original Message > Subject: In Python 2.6, bytes is str > Date: Sun, 05 Oct 2008 22:30:17 -0700 > From: Bryan Olson <[EMAIL PROTECTED]> > Organization: at&t http://my.att.net/ > To: [EMAIL PROTECTED] > Newsgroups: gmane.comp.python.general > > > Python 3 has the 'bytes' type, which the string type I've long wanted in > various languages. Among other advantages, it is immutable, and > therefore bytes objects can be dict keys. There's a mutable version too, > called "bytearray". > > In Python 2.6, the name 'bytes' is defined, and bound to str. The 2.6 > assignment presents some pitfalls. Be aware. > > Consider constructing a bytes object as: > >b = bytes([68, 255, 0]) > > In Python 3.x, len(b) will be 3, which feels right. > > In Python 2.6, len(b) will be 12, because b is the str, '[68, 255, 0]'. > > > I'm thinking I should just avoid using 'bytes' in Python 2.6. If there's > another Python release between 2.6 and 3.gold, I'd advocate removing > the pre-defined 'bytes', or maybe defining it as something else. > > > -- > --Bryan > -- > http://mail.python.org/mailman/listinfo/python-list > > > -- > Steve Holden+1 571 484 6266 +1 800 494 3119 > Holden Web LLC http://www.holdenweb.com/ > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] porting pycxx and pysvn to python 3.0 hit a problem
>> Given that p.we.x seems to always return the same object (since the >> hashes, which which appear to be ids, are the same), an __eq__ method >> (which gets called in preference to __cmp__), possibly inherited, that >> always return False is the only thing I can think of. (Hence Martin's >> question, I presume). I have no idea, however, how porting could make >> that happen. >> > > I see no reply from Martin. What was his question? Whether __eq__ was implemented for the specific type (and whether that implementation was consistent with __hash__). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-3000] Proposed Python 3.0 schedule
James Y Knight wrote: > or at least fully recognized and documented as a half-baked > solution. I would prefer that, leaving a full resolution to 3.1 (or perhaps 3.2). If we wait long enough, the issue will disappear (a strategy that Sun is apparently taking for Java :-) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python3UnicodeDecodeError (Was: Proposed Python 3.0 schedule)
> First of all, please read my document: > http://wiki.python.org/moin/Python3UnicodeDecodeError I have problems understanding that document. Is it supposed to be a PEP (i.e. a proposal to enhance Python), or is it a description of the status quo? If it is a PEP, it should clearly separate status quo, specification, and rationale (in any order that you find reasonable). It should also have an "open issues" section, explicitly listing the questions that haven't been resolved, and it should record objections to the proposal. I think I would object to the specification (perhaps to the degree of proposing a counter-PEP), but to do so, I first need a specification to object to. In terms of time-line, I think any such PEP is *clearly* out of scope for Python 3.0. All the remaining issues should deferred to 3.1. That the approach "we can use bytes in the file system API" was so rushed into the code base is already unfortunate, but I can understand the motivation - people want to write backup programs in Python. If I take the text as if it was a specification, here are some of my objections: - Default encoding: a) seems irrelevant for the PEP. The default encoding doesn't nearly have the role anymore that it had in 2.x, and shouldn't have any effect on how file names are treated. b) I would propose that the notion of a default encoding is entirely eliminated from Python, along with sys.(get|set)defaultencoding - argv and environ: are you suggesting that the behavior described in the PEP is desirable? I don't think it is (but I don't think it should change for 3.0, either, only for 3.1) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] porting pycxx and pysvn to python 3.0 hit a problem
On Oct 5, 2008, at 22:49, Terry Reedy wrote: Given that p.we.x seems to always return the same object (since the hashes, which which appear to be ids, are the same), an __eq__ method (which gets called in preference to __cmp__), possibly inherited, that always return False is the only thing I can think of. (Hence Martin's question, I presume). I have no idea, however, how porting could make that happen. I see no reply from Martin. What was his question? Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.5.3: call for patches
> Just to emphasize this, "changes" means "bugfixes". (I'm mentioning this > mainly because of the people who joined for 2.6/3.0.) For more info, > see PEP6 about bugfix releases: > http://www.python.org/dev/peps/pep-0006/ Thanks for clarifying this. For the last 2.5.x release in particular, we will strictly enforce the "no new features" policy; users interested in new features should switch to 2.6. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] porting pycxx and pysvn to python 3.0 hit a problem
On Oct 5, 2008, at 22:49, Terry Reedy wrote: Barry Scott wrote: for key in [pysvn.wc_status_kind.added, pysvn.wc_status_kind.replaced, pysvn.wc_status_kind.unversioned]: print( '2 key', key, key in wc_status_kind_map, cmp( key, pysvn.wc_status_kind.unversioned ), hash( key ) ) try: print( '2 lookup', wc_status_kind_map[ key ] ) except: print( '2 failed' ) 2 key added False 1 -586300914 2 failed 2 key replaced False 1 -586300911 2 failed 2 key unversioned False 0 -586300916 2 failed Given that p.we.x seems to always return the same object (since the hashes, which which appear to be ids, are the same), an __eq__ method (which gets called in preference to __cmp__), possibly inherited, that always return False is the only thing I can think of. (Hence Martin's question, I presume). I have no idea, however, how porting could make that happen. The type is not derived so the __eq__ cannot be happening. I guess I need to use gdb and figure out what is going on. Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [python-committers] Proposed Python 3.0 schedule
> More specifically, I think 2to3 is shaping up well. pywin32 is taking the > approach of "port where possible, but keep in py2x syntax and convert at > 'setup.py' time" and this is working out fairly well I can't say how glad I am that you say that. It supports lib2to3 being a proper library, despite the problems that this may cause in itself. > * Better support for 2to3 in distutils (specifically, the support in > build_py is stale, plus 'build_scripts' and 'install_data' should convert > .py files to py3k syntax.) Please do create a bug report for that. It sounds like it's easy to fix. > An 'example' project that uses py2k syntax and > "just works" on py3k using this strategy might be useful here. Perhaps pywin32 :-? I don't think a demo project would do much good, as it doesn't exercise all the issues that may occur. > * A standard 'helper script' that allows people to use py3k to execute a > py2x syntax script by auto-converting the code. I've a 10ish-line script > that uses lib2to3 plus exec() to achieve that result, but a helper in 2to3 > for this would be nice. For a concrete use-case, we want to keep our > distutils script in py2x syntax, but execute it via py3k. Its very possible > this already exists and I've just missed it... For the case of setup.py, I was hoping that it could be written in compatible syntax even without needing conversion. That worked fine for my Django port. Is that not the case for pywin32? This specific issue might be out of scope for 3.x, IMO. > Either way, I'm fairly confident a pywin32 build for py3k will be available > in the next month or 2 (but as a result, I'm not really in a position to > help with the above for that period...) But please do file bug reports, preferably along with any patches to distutils that you already have. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-3000] Proposed Python 3.0 schedule
On Oct 7, 2008, at 3:47 AM, Martin v. Löwis wrote: - Having os.getcwdb isn't much use when you can't even run python in the first place when the current directory has "bad" bytes in it. That's not true: it *is* of much use. Python will live in /usr/bin, which has a nicely-decodable path. Currently Python outputs: Could not find platform independent libraries Could not find platform dependent libraries Consider setting $PYTHONHOME to [:] Fatal Python error: Py_Initialize: can't initialize sys standard streams ImportError: No module named encodings.utf_8 Aborted I can't reproduce that. This happens (for me) when Python lives in a directory that has an undecodable path - not when the current directory is undecodable. Sorry about that: this test was indeed in error: I ran "../python" from an undecodeable current directory, rather than "/full/path/to/ python", or putting python on the PATH and running it as "python". The first does not work, but the other more common ways to start it do. I'm sure there's even more APIs dealing with pathnames, command line arguments, or environment variables that ought to be able to handle both bytes and strings, that currently don't. Please, no. I completely and totally agree with your distate, it's rather gross to allow bytes-or-str for every API that touches anything like filenames/ argv/environ. That's why I was pushing for the reversible conversion to str...But if bytes-or-str is the solution that's been chosen for this issue, it ought to either be fully committed to and implemented, or at least fully recognized and documented as a half-baked solution. Of course, if an reversible encoding into string solution is used instead, none of these things would need special treatment: they would all work already. FWIW: Qt works fine with undecodeable filenames, and it too uses unicode strings everywhere in its API. I looked into what it does, and found that it uses your (Martin)'s original idea for solving this: it stores undecodeable bytes as characters from 0x10fe00 to 0x10feff (which is valid private-use codespace). While that might not be ideally correct, since you lose those 256 PUA characters, even that is IMO better than pushing out bytes to every API, or worse, giving up and just having python unable to access files, as it is now. See lines 3074: QString::toUtf8() and 3408: QString::fromUtf8()) of http://www.google.com/codesearch?q=+show:o7fNK6SzOYs:NO-Bv-AR2rI:toIOngLf1V8&cs_p=http://ie.archive.ubuntu.com/trolltech/pub/qt/snapshots/qt-x11-opensource-src-4.4.0-snapshot-20070402.tar.bz2&cs_f=qt-x11-opensource-src-4.4.0-snapshot-20070402/src/corelib/tools/qstring.cpp James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.5.3: call for patches
On Tue, Oct 07, 2008, "Martin v. L?wis" wrote: > > In principle, the release will include all changes that are already on > the release25-maint branch in subversion [1]. If you think that specific > changes should be considered, please create an issue in the bug tracker > [2], and label it with the 2.5.3 version. Backports of changes that > are already released in Python 2.6 but may apply to 2.5 are of > particular interest. Just to emphasize this, "changes" means "bugfixes". (I'm mentioning this mainly because of the people who joined for 2.6/3.0.) For more info, see PEP6 about bugfix releases: http://www.python.org/dev/peps/pep-0006/ -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ "...if I were on life-support, I'd rather have it run by a Gameboy than a Windows box." --Cliff Wells, comp.lang.python, 3/13/2002 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [python-committers] Proposed Python 3.0 schedule
[when 2 mailing lists are not enough... :-] > I'm seeing that people are just starting to download and play with 3.0. > I expect that we'll start getting more feedback on conversion issues +1 from this direction too. pywin32 has recently started looking seriously at py3k, and while things are in fairly good shape for us who are already "on the bandwagon", cleaning up a few rough edges would help people's first impressions - and as they say, you only get one chance at a good first impression... More specifically, I think 2to3 is shaping up well. pywin32 is taking the approach of "port where possible, but keep in py2x syntax and convert at 'setup.py' time" and this is working out fairly well (in fact, with just a couple of helpers in pywintypes, I think we can support python 2.3 upwards). I believe that many projects may well take a similar approach as it allows them to defer a full commitment to py3k, so doing all we can to support this might help with that first impression. My experience is that this could best be achieved by addressing the following issues before release: * Almost all open 2to3 issues that aren't truly edge cases should be resolved - if 2to3 doesn't work for people, they may be forced to (even temporarily) "fork" their project, which will cause concern. I'll note that good recent progress is being made here, but its still worth mentioning... * Better support for 2to3 in distutils (specifically, the support in build_py is stale, plus 'build_scripts' and 'install_data' should convert .py files to py3k syntax.) An 'example' project that uses py2k syntax and "just works" on py3k using this strategy might be useful here. * A standard 'helper script' that allows people to use py3k to execute a py2x syntax script by auto-converting the code. I've a 10ish-line script that uses lib2to3 plus exec() to achieve that result, but a helper in 2to3 for this would be nice. For a concrete use-case, we want to keep our distutils script in py2x syntax, but execute it via py3k. Its very possible this already exists and I've just missed it... Either way, I'm fairly confident a pywin32 build for py3k will be available in the next month or 2 (but as a result, I'm not really in a position to help with the above for that period...) Hopefully-helpfully, Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-3000] Proposed Python 3.0 schedule
2008/10/6 Raymond Hettinger <[EMAIL PROTECTED]>: >> 15-Oct-2008 3.0 beta 4 >> 05-Nov-2008 3.0 rc 2 >> 19-Nov-2008 3.0 rc 3 >> 03-Dec-2008 3.0 final >> >> Given what still needs to be done, is this a reasonable schedule? Do we >> need two more betas? > > Yes to both questions. I agree with you here. > I'm seeing that people are just starting to download and play with 3.0. > I expect that we'll start getting more feedback on conversion issues, > the C API, screwy interactions with operating systems, bytes/text issues, > unanticipated interactions with other tools, etc. Each user will stress > it in new ways and perhaps reveal a bunch of little integration issues > and documentation issues. Those little fixups way go a long way toward > establishing a good first impression and reputation for 3.0 from the outset. And maybe also here, but bounded. I don't want to keep deferring 3.0 months and months, I prefer to have a redesigned schedule now, and stick to it as much as possible, even if the 3.0 version is not as robust as we would want. Regards, -- .Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] status of 2.5
Raymond Hettinger rcn.com> writes: > > The 2.6/3.0 development process was so disruptive that I doubt > that 2.5 received adequate attention for bug fixes. Why not wait > two or three months for the dust to settle? I know that I, and a couple of others, have tried to backport "important" bug fixes (by that I mean security fixes, crashers, memory leaks, as well as glaring behaviour problems) to 2.5 when there was no risk to reduce stability or compatibility. It's also true that 2.6/3.0 were so disruptive that the reason why few things were backported is simply that few things could be backported at all. Most checkins were related to new functionality or modified behaviour, or new bugs introduced by either of those. Everyone can look at http://code.python.org/hg/branches/release2.5-maint/shortlog to get an idea of where 2.5 is. 2.5.2 was tagged ~7 months ago. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed Python 3.0 schedule
On Tue, 2008-10-07 at 11:30 +0200, Victor Stinner wrote:
> > - I'd think "find . -type f -print0 | xargs -0 python -c 'pass'"
> > ought to work (with files with "bad" bytes being returned by find),
>
> First, fix your home directory :-) There are good tools (convmv?) to fix
> invalid filenames.
Fixing the home directory doesn't help in the long run because files
with non-UTF-8 file names on a nominally UTF-8 system are not that
exceptional, they crop up all over the place in non-ASCII countries.
One can obtain them simply by copying stuff from a DVD someone else
burned, by downloading a Japanese-released torrent, or by copying files
from a shared hard drive.
> > which means that Python shouldn't blow up and refuse to start when
> > there's a non-properly-encoding argv ("Could not convert argument 1 to
> > string" and exiting isn't appropriate behavior)
>
> Why not? It's a good idea to break compatibility to refuse invalid bytes
> sequences. You can still uses the command line, an input file or a GUI to
> read raw bytes sequences.
Maybe I am misunderstanding you, but if python blows up at startup when
unable to encode argv to Unicode, then how can you still use the command
line to access the actual file name?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed Python 3.0 schedule
Hi,
First of all, please read my document:
http://wiki.python.org/moin/Python3UnicodeDecodeError
I moved the document to a public wiki to allow anyone to edit it!
Le Tuesday 07 October 2008 05:22:09 James Y Knight, vous avez écrit :
> On Oct 6, 2008, at 8:52 PM, Benjamin Peterson wrote:
> > I'm not sure we do. Correct me if I'm wrong, but the "big ticket",
> > issue bytes/unicode filepaths, has been resolved.
Python3 now accepts bytes for os.listdir(), open() (io.open()), os.unlink(),
os.path.*(), etc. But it's not enough to say that Python3 can use bytes
everywhere. It would take months or *years* to fix all issues related to
bytes and unicode. Remember, this task started in 2000 with Python *2.0*
(creation of the unicode type).
> Well, if you mean that the resolution decided upon is to "simply"
> allow access to all system APIs using either byte or unicode strings,
> then it seems to me that there's a rather large amount of work left to
> do...
If you know a problem, open a ticket and propose a solution. It's not possible
to list all new problems since we don't know them yet :-)
> - Having os.getcwdb isn't much use when you can't even run python in
> the first place when the current directory has "bad" bytes in it.
My python3.0 works correctly in a directory with an invalid name. What is your
OS / locale / Python version? Please create a ticket if needed.
> - I'd think "find . -type f -print0 | xargs -0 python -c 'pass'"
> ought to work (with files with "bad" bytes being returned by find),
First, fix your home directory :-) There are good tools (convmv?) to fix
invalid filenames.
> which means that Python shouldn't blow up and refuse to start when
> there's a non-properly-encoding argv ("Could not convert argument 1 to
> string" and exiting isn't appropriate behavior)
Why not? It's a good idea to break compatibility to refuse invalid bytes
sequences. You can still uses the command line, an input file or a GUI to
read raw bytes sequences.
> - Of course, just being able to start the interpreter isn't quite
> enough: you'll want to be able to access that argument list too,
> somehow (add sys.argvb?).
If we create sys.argvb, what shoul be done if sys.argv creation failed?
sys.argv would be empty or unset? Or some values would be removed (and so
argv[2] is argv[1])? I think that many (a lot of) programs suppose that
sys.argv exists and "is valid". If you introduce a special case (sometimes,
sys.argv doesn't exist or is truncated !?), it will introduce new issues.
> - There's no os.environb for bytewise access to the environment.
> Seems important.
It would be strange if you can put a variable in bytes to os.environb whereas
os.environ would not get the key. I know two major usages of the environment:
(1) read a variable in Python
(2) put a variable for a child process
(1) can be done with os.getenv() and returns None if the variable (key or
value) is an invalid bytes sequence.
(2) can be done with subprocess.Popen(). subprocess doesn't support bytes yet
but I wrote patches: #4035 and #4036.
> - Isn't it a potential security issue that " 'WHATEVER' in
> os.environ" can return False if WHATEVER had some "bad" bytes in it,
> but spawning a subprocess actually will include WHATEVER in the
> subprocess's environment?
Yes. Python should remove the key while creating os.environ.
> - Shouldn't this work? subprocess.call(b'/bin/echo')
Yes. Most programs (at least on Linux and Mac) supports bytes and so you
should be able use bytes arguments in their command lines, see issues #4035
and #4036.
> - I suppose sys.path should handle bytestrings on the path, and
> should be populated using the bytes-version of os.environ so that
> PYTHONPATH gets read in properly. Which of course implies that all the
> importers need to handle byte filenames.
If your file system is broken, rename your directory but don't introduce a
special case for sys.path.
> - zipfile.ZipFile(b'whatever.zip') doesn't work.
Since zipfile uses bytes in its file structure, zipfile should accept bytes.
But the right question is: should this issue block Python3 or can we wait for
Python 3.1 (maybe 3.0.1)?
--
People wants to try the new Python version! Python3 introduces new amazing
features like "keyword only arguments". The bytes/unicode problem is old and
only affects broken systems
Windows (90% of the computers in the world?) only uses characters for the
filenames, environment and command line. Mac and Linux use UTF-8 most of the
time, and slowly everything speaks UTF-8! Python3 should not be delayed
because of this problem.
About the initial barry's question: why Python3 is delayed until december?
There are too much open issues?
--
Victor Stinner aka haypo
http://www.haypocalc.com/blog/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/option
Re: [Python-Dev] [Python-3000] Proposed Python 3.0 schedule
> Here's some I found from a few minutes of futzing around with r66821 of
> py3k on Linux.
>
> - Having os.getcwdb isn't much use when you can't even run python in
> the first place when the current directory has "bad" bytes in it.
That's not true: it *is* of much use. Python will live in /usr/bin,
which has a nicely-decodable path.
> Currently Python outputs:
> Could not find platform independent libraries
> Could not find platform dependent libraries
> Consider setting $PYTHONHOME to [:]
> Fatal Python error: Py_Initialize: can't initialize sys standard streams
> ImportError: No module named encodings.utf_8
> Aborted
I can't reproduce that. This happens (for me) when Python lives in
a directory that has an undecodable path - not when the current
directory is undecodable.
> - I'd think "find . -type f -print0 | xargs -0 python -c 'pass'" ought
> to work (with files with "bad" bytes being returned by find), which
> means that Python shouldn't blow up and refuse to start when there's a
> non-properly-encoding argv ("Could not convert argument 1 to string" and
> exiting isn't appropriate behavior).
Contributions are welcome. *Of course* can you access these files with
POSIX API. However, Python's path handling can't.
See above why I don't consider this as a serious bug, on Unix.
> - Of course, just being able to start the interpreter isn't quite
> enough: you'll want to be able to access that argument list too, somehow
> (add sys.argvb?).
Perhaps. However, I don't see the need to be able to do so in Python
3.0.
> - And then, getopt and optparse modules should work on bytestring
> vectors, so that you can use sys.argvb without writing your own argument
> parser. They don't currently.
And I hope they never will. Using bytes to represent this stuff will
just bring back the 2.x status, so some other solution must be found -
for 3.1 (or 3.2).
> - There's no os.environb for bytewise access to the environment. Seems
> important.
Not to me. I don't have environment variables with non-ASCII characters
in them, and I think few other people do.
> I'm sure there's even more APIs dealing with pathnames, command line
> arguments, or environment variables that ought to be able to handle both
> bytes and strings, that currently don't.
Please, no.
Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python 2.5.3: call for patches
Within a few weeks, we will release Python 2.5.3. This will be the last bug fix release of Python 2.5, afterwards, future releases of 2.5 will only include security fixes, and no binaries (for Windows or OSX) will be provided anymore (from python.org). In principle, the release will include all changes that are already on the release25-maint branch in subversion [1]. If you think that specific changes should be considered, please create an issue in the bug tracker [2], and label it with the 2.5.3 version. Backports of changes that are already released in Python 2.6 but may apply to 2.5 are of particular interest. Regards, Martin [1] http://svn.python.org/projects/python/branches/release25-maint/ [2] http://bugs.python.org/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] status of 2.5
> ISTM, that 3.0 and 2.6 backports have consumed substantial developer > attention over the last few months. Sure - but who is going to sit down and study the commit logs, to do backporting? Unless someone specifically volunteers to do that (with a specific timeline when he will start, and when he will complete), I see little chance that that any additional backporting will happen. I will send an announcement asking people to propose missing backports. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
