Re: [Python-Dev] Proposed: add support for UNC paths to all functions in ntpath
I've taken the liberty of explicitly CCing Martin just incase he missed the thread with all the noise regarding PEP383. If there are no objections from Martin It's fine with me - I just won't have time to look into the details of that change. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 and GUI libraries
Zooko O'Whielacronx wrote:
[snip...]
Would it be possible for Python unicode objects to have a flag
indicating whether the 'python-escape' error handler was present? That
would serve the same purpose as my failed_decode flag above, and would
basically allow me to use the Python APIs directory and make all this
work-around code disappear.
Failing that, I can't see any way to use the os.listdir() in its
unicode-oriented mode to satisfy Tahoe's requirements.
If you take the above code and then add the fact that you want to use
the failed_decode flag when *encoding* the d argument to os.listdir(),
then you get this code: [2].
Oh, I just realized that I *could* use the PEP 383 os.listdir(), like
this:
def listdir(d):
fse = sys.getfilesystemencoding()
if fse == 'utf-8b':
fse = 'utf-8'
ns = []
for fn in os.listdir(d):
bytes = fn.encode(fse, 'python-escape')
try:
ns.append(FName(bytes.decode(fse, 'strict')))
except UnicodeDecodeError:
ns.append(FName(fn.decode('utf-8', 'python-escape'),
failed_decode=True))
return ns
(And I guess I could define listdir() like this only on the
non-unicode-safe platforms, as above.)
However, that strikes me as even more horrible than the previous
listdir() work-around, in part because it means decoding, re-encoding,
and re-decoding every name, so I think I would stick with the previous
version.
The current unicode mode would skip the filenames you are interested
(those that fail to decode correctly) - so you would have been forced to
use the bytes mode. If you need access to the original bytes then you
should continue to do this. PEP-383 is entirely neutral for your use
case as far as I can see.
Michael
Oh, one more note: for Tahoe's purposes you can, in all of the code
above, replace .decode('utf-8', 'python-replace') with
.decode('windows-1252') and it works just as well. While UTF-8b seems
like a really cool hack, and it would produce more legible results if
utf-8-encoded strings were partially corrupted, I guess I should just
use 'windows-1252' which is already implemented in Python 2 (as well as
in all other software in the world).
I guess this means that PEP 383, which I have approved of and liked so
far in this discussion, would actually not help Tahoe at all and would
in fact harm Tahoe -- I would have to remember to detect and work-around
the automatic 'utf-8b' filesystem encoding when porting Tahoe to Python
3.
If anyone else has a concrete, real use case which would be helped by
PEP 383, I would like to hear about it. Perhaps Tahoe can learn
something from it.
Oh, if this PEP could be extended to add a flag to each unicode object
indicating whether it was created with the python-escape handler or not,
then it would be useful to me.
Regards,
Zooko
[1] http://mail.python.org/pipermail/python-dev/2009-April/089020.html
[2] http://allmydata.org/trac/tahoe/attachment/ticket/534/fsencode.3.py
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 and GUI libraries
On Thu, 30 Apr 2009 at 23:44, Zooko O'Whielacronx wrote: Would it be possible for Python unicode objects to have a flag indicating whether the 'python-escape' error handler was present? That Unless I'm misunderstanding something, couldn't you implement what you need by looking in a given string for the half surrogates? If you find one, you have a string python-escape modified, if you don't, it didn't. What does Tahoe do on Windows when it gets a filename that is not valid Unicode? You might not even have to conditionalize the above code on platform (ie: instead you have a generalized is_valid_unicode test function that you always use). --David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383: Non-decodable Bytes in System Character Interfaces
James Y Knight writes: in python. It seems like the most common reason why people want to use SJIS is to make old pre-unicode apps work right in WINE -- in which case it doesn't actually affect unix python at all. Mounting external drives, especially USB memory sticks which tend to be FAT-initialized by the manufacturers, is another common case. But I don't understand why PEP 383 needs to care at all. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 and GUI libraries
Following-up to my own post to correct a major error:
On Thu, Apr 30, 2009 at 11:44 PM, Zooko O'Whielacronx zoo...@gmail.com wrote:
Folks:
My use case (Tahoe-LAFS [1]) requires that I am *able* to read arbitrary
binary names from the filesystem and store them so that I can regenerate
the same byte string later, but it also requires that I *know* whether
what I got was a valid string in the expected encoding (which might be
utf-8) or whether it was not and I need to fall back to storing the
bytes.
Okay, I am wrong about this. Having a flag to remember whether I had to
fall back to the utf-8b trick is one method to implement my requirement,
but my actual requirement is this:
Requirement: either the unicode string or the bytes are faithfully
transmitted from one system to another.
That is: if you read a filename from the filesystem, and transmit that
filename to another system and use it, then there are two cases:
Requirement 1: the byte string was valid in the encoding of source
system, in which case the unicode name is faithfully transmitted
(i.e. the bytes that finally land on the target system are the result of
sourcebytes.decode(source_sys_encoding).encode(target_sys_encoding).
Requirement 2: the byte string was not valid in the encoding of source
system, in which case the bytes are faithfully transmitted (i.e. the
bytes that finally land on the target system are the same as the bytes
that originated in the source system).
Now I finally understand how fiendishly clever MvL's PEP 383
generalization of Markus Kuhn's utf-8b trick is! The only thing
necessary to achieve both of those requirements above is that the
'python-escape' error handler is used on the target system .encode() as
well as on the source system .decode()!
Well, I'm going to have to let this sink in and maybe write some code to
see if I really understand it.
But if this is right, then I can do away with some of the mechanism that
I've built up, and instead:
Backport PEP 383 to Python 2.
And, document the PEP 383 trick in some generic, widely respected format
such as an Internet Draft so that I can explain to other users of the
Tahoe data (many of whom use other languages than Python) what they have
to do if they find invalid utf-8 in the data. Oh good, I just realized
that Tahoe emits only utf-8, so all I have to do is point them to the
utf-8b documents (such as they are) and explain that to read filenames
produced by Tahoe they have to implement utf-8b. That's really good
that they don't have to implement MvL's generalization of that trick to
other encodings, since utf-8b is already understood by some folks.
Okay, I find it surprisingly easy to make subtle errors in this encoding
stuff, so please let me know if you spot one. Is it true that
srcbytes.encode(srcencoding, 'python-escape').decode('utf-8',
'python-escape') will always produce srcbytes ? That is my Requirement
2.
Regards,
Zooko
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Oddity PEP 0 key
I've just noticed an oddity in the key in PEP 0. Most letters are used more than once. Wouldn't it be clearer if different letters were used for Accepted and Active instead of them both being 'A', for example? - A - Accepted proposal - R - Rejected proposal W - Withdrawn proposal - D - Deferred proposal F - Final proposal - A - Active proposal - D - Draft proposal - R - Replaced proposal ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 and GUI libraries
Zooko O'Whielacronx wrote:
Following-up to my own post to correct a major error:
On Thu, Apr 30, 2009 at 11:44 PM, Zooko O'Whielacronx zoo...@gmail.com wrote:
Folks:
My use case (Tahoe-LAFS [1]) requires that I am *able* to read arbitrary
binary names from the filesystem and store them so that I can regenerate
the same byte string later, but it also requires that I *know* whether
what I got was a valid string in the expected encoding (which might be
utf-8) or whether it was not and I need to fall back to storing the
bytes.
Okay, I am wrong about this. Having a flag to remember whether I had to
fall back to the utf-8b trick is one method to implement my requirement,
but my actual requirement is this:
Requirement: either the unicode string or the bytes are faithfully
transmitted from one system to another.
That is: if you read a filename from the filesystem, and transmit that
filename to another system and use it, then there are two cases:
Requirement 1: the byte string was valid in the encoding of source
system, in which case the unicode name is faithfully transmitted
(i.e. the bytes that finally land on the target system are the result of
sourcebytes.decode(source_sys_encoding).encode(target_sys_encoding).
Requirement 2: the byte string was not valid in the encoding of source
system, in which case the bytes are faithfully transmitted (i.e. the
bytes that finally land on the target system are the same as the bytes
that originated in the source system).
Now I finally understand how fiendishly clever MvL's PEP 383
generalization of Markus Kuhn's utf-8b trick is! The only thing
necessary to achieve both of those requirements above is that the
'python-escape' error handler is used on the target system .encode() as
well as on the source system .decode()!
Well, I'm going to have to let this sink in and maybe write some code to
see if I really understand it.
But if this is right, then I can do away with some of the mechanism that
I've built up, and instead:
Backport PEP 383 to Python 2.
And, document the PEP 383 trick in some generic, widely respected format
such as an Internet Draft so that I can explain to other users of the
Tahoe data (many of whom use other languages than Python) what they have
to do if they find invalid utf-8 in the data. Oh good, I just realized
that Tahoe emits only utf-8, so all I have to do is point them to the
utf-8b documents (such as they are) and explain that to read filenames
produced by Tahoe they have to implement utf-8b. That's really good
that they don't have to implement MvL's generalization of that trick to
other encodings, since utf-8b is already understood by some folks.
Okay, I find it surprisingly easy to make subtle errors in this encoding
stuff, so please let me know if you spot one. Is it true that
srcbytes.encode(srcencoding, 'python-escape').decode('utf-8',
'python-escape') will always produce srcbytes ? That is my Requirement
2.
No, but srcbytes.encode('utf-8', 'python-escape').decode('utf-8',
'python-escape') == srcbytes. The encodings on both ends need to be the
same.
For example:
b'\x80'.decode('windows-1252')
u'\u20ac'
u'\u20ac'.encode('utf-8')
'\xe2\x82\xac'
Currently:
b'\x80'.decode('utf-8')
Traceback (most recent call last):
File pyshell#7, line 1, in module
b'\x80'.decode('utf-8')
File C:\Python26\lib\encodings\utf_8.py, line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x80 in position 0:
unexpected code byte
But under this PEP:
b'x80'.decode('utf-8', 'python-escape')
u'\xdc80'
u'\xdc80'.encode('utf-8', 'python-escape')
'\x80'
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (04/24/09 - 05/01/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2190 open (+34) / 15527 closed (+29) / 17717 total (+63) Open issues with patches: 861 Average duration of open issues: 645 days. Median duration of open issues: 394 days. Open Issues Breakdown open 2156 (+33) pending33 ( +1) Issues Created Or Reopened (63) ___ os.path.walk fails to descend into a directory whose name ends w 04/24/09 CLOSED http://bugs.python.org/issue5832created linuxelf readline update 04/24/09 http://bugs.python.org/issue5833created jrevans1 patch The word error used instead of failure 04/25/09 CLOSED http://bugs.python.org/issue5834created kurtmckee Deprecate PyOS_ascii_formatd 04/25/09 CLOSED http://bugs.python.org/issue5835created eric.smith Clean up float parsing code for nans and infs04/25/09 CLOSED http://bugs.python.org/issue5836created marketdickinson support.EnvironmentVarGuard broken 04/25/09 CLOSED http://bugs.python.org/issue5837created doerwalter easy Test issue 04/25/09 CLOSED http://bugs.python.org/issue5838created ajaksu2 RegOpenKeyEx key failed on Vista 64Bit with return 2 04/25/09 http://bugs.python.org/issue5839created makursi Thread State and the Global Interpreter Lock section of the do 04/25/09 http://bugs.python.org/issue5840created exarkun patch add py3k warnings to commands04/25/09 CLOSED http://bugs.python.org/issue5841created dsm001 patch Move test outside of urlparse module 04/25/09 http://bugs.python.org/issue5842created Merwok Possible normalization error in urlparse.urlunparse 04/25/09 http://bugs.python.org/issue5843created Merwok internal error on write while reading04/25/09 http://bugs.python.org/issue5844created dsm001 patch rlcompleter should be enabled automatically 04/25/09 http://bugs.python.org/issue5845created cben Deprecate obsolete functions in unittest 04/25/09 http://bugs.python.org/issue5846created michael.foord IDLE/Win Installer: drop -n switch for 2.7/3.1; install 3.1 as i 04/26/09 http://bugs.python.org/issue5847created kbk Minor unittest doc patch 04/26/09 CLOSED http://bugs.python.org/issue5848created michael.foord patch, patch, easy, needs review Idle 3.01 - invalid syntec error 04/26/09 CLOSED http://bugs.python.org/issue5849created r2d2floyd Full example for emulating a container type 04/27/09 CLOSED http://bugs.python.org/issue5850created yaneurabeya
Re: [Python-Dev] .pth files are evil
M.-A. Lemburg wrote: If the package really requires adding one or more directories on sys.path (e.g. because it has not yet been structured to support dotted-name import), a path configuration file named package.pth can be placed in either the site-python or site-packages directory. ... A typical installation should have no or very few .pth files or something is wrong, and if you need to play with the search order, something is very wrong. I'll say! I think .pth files are absolute evil and I wish they could just be banned. +1 on anything that makes them closer to going away or reduces the possibility of yet another similar feature from hurting the comprehensibility of a python setup. Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 382: little help for stupid people?
M.-A. Lemburg wrote: The much more common use case is that of wanting to have a base package installation which optional add-ons that live in the same logical package namespace. The PEP provides a way to solve this use case by giving both developers and users a standard at hand which they can follow without having to rely on some non-standard helpers and across Python implementations. My proposal tries to solve this without adding yet another .pth file like mechanism - hopefully in the spirit of the original Python package idea. Okay, I need to issue a plea for a little help. I think I kinda get what this PEP is about now, and as someone who wants to ship a base package with several add-ons that live in the same logical package namespace, I'm very interested. However, despite trying to follow this thread *and* having tried to read the PEP a couple of times, I still don't know how I'd go about doing this. I did give some examples from what I'd be looking to do much earlier. I'll ask again in the vague hope of you or someone else explaining things to me like I'm a 5 year old - something I'm mentally equipped to be well ;-) In either of the proposals on the table, what code would I write and where to have a base package with a set of add-on packages? Simple examples would be greatly appreciated, and might bring things into focus for some of the less mentally able bystanders - like myself! cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 382: Namespace Packages
P.J. Eby wrote: At 06:15 PM 4/15/2009 +0200, M.-A. Lemburg wrote: The much more common use case is that of wanting to have a base package installation which optional add-ons that live in the same logical package namespace. Please see the large number of Zope and PEAK distributions on PyPI as minimal examples that disprove this being the common use case. If you mean the common use case as opposed to having code in the __init__.py of the namespace package, I think you'll find that's because people (especially me!) don't know how to do this, not because we don't want to! Chris - who would actually like to know how to do this, with or without the PEP, and how to indicate interdependencies in situations like this to setuptools... -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 382: Namespace Packages
P.J. Eby wrote: It's unclear, however, who is using base packages besides mx.* and ll.*, although I'd guess from the PyPI listings that perhaps Django is. (It seems that base packages are more likely to use a 'base-extension' naming pattern, vs. the 'namespace.project' pattern used by pure packages.) I'll stress it again in case you missed it the first time: I think the main reason people use pure namespace versus base namespace packages is because hardly anyone know how to do the latter, not because there is no desire to do so! I, for one, have been trying to figure out how to do base namespace packages for years... Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 382: little help for stupid people?
In either of the proposals on the table, what code would I write and where to have a base package with a set of add-on packages? I don't quite understand the question. Why would you want to write code (except for the code that actually is in the packages)? PEP 382 is completely declarative - no need to write code. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 382: little help for stupid people?
Martin v. Löwis wrote: In either of the proposals on the table, what code would I write and where to have a base package with a set of add-on packages? I don't quite understand the question. Why would you want to write code (except for the code that actually is in the packages)? PEP 382 is completely declarative - no need to write code. code is anything I need to write to make this work... So, what do I need to do? Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] headers api for email package
Where you just want a damned valid email and stop making my life
hard!:
Message['Subject']='Some text'
Yes. In which case I propose we guess the encoding as 1) ascii, 2)
utf-8, 3) wtf?
Well, we're talking about Python 3 here right? In which case the above
involves only unicode, so why do we need to guess anything? Just use
utf-8 and be done with it...
However, it's not supposed to be used by mail composers, who are
expected to know the encoding. It's for mail gateways that are
transforming something and don't know the encoding. I'm not
sure what this means for the email module, which certainly
will be used in a mail gatewaysmaybe it's the responsibility
of the application code to explicitly say 'unknown encoding'?
Indeed, surely this happens when you have bytes and need to do something
with it? That's not what my example above is about...
Where you care about what encoding is used:
Message['Subject']=Header('Some text',encoding='utf-8')
Yes.
...it's covered by this.
If you have bytes, for whatever reason:
Message['Subject']=b'some bytes'.decode('utf-8')
...because only you know what encoding those bytes use!
So you're saying that __setitem__() should not accept raw bytes?
Indeed :-)
Chris
--
Simplistix - Content Management, Zope Python Consulting
- http://www.simplistix.co.uk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Email-SIG] headers api for email package
Stephen J. Turnbull wrote: str(message['Subject']) Yes for unstructured headers like Subject. For structured headers... hmm. Well, suppose we get really radical here. *People* see email as (rich-)text. So ... message['Subject'] returns an object, partly to be consistent with more complex headers' APIs, but partly to remind us that nothing in email is as simple as it seems. Now, str(message['Subject']) is really for presentation to the user, right? OK, so let's make it a presentation function! Decode the MIME-words, optionally unfold folded lines, optionally compress spaces, etc. This by default returns the subject field as a single, possibly quite long, line. Then a higher-level API can rewrap it, add fonts etc, for fancy presentation. This also suggests that we don't the field tag (ie, Subject) to be part of this value. Of course a *really* smart higher-level API would access structured headers based on their structure, not on the one-size-fits-all str() conversion. All sounds good to me. Then MTAs see email as a string of octets. So guess what: bytes(message['Subject']) gives wire format. Yow! I think I'm just joking. Right? Why? That also sounds fine to me and feels right... Where you just want a damned valid email and stop making my life hard!: -1 I mean, yeah, Brother, I feel your pain but it just isn't that easy. If that were feasible, it would be *criminal* to have a .set_header() method at all! In fact, Don't agree... Message['Subject']='Some text' is going to (a) need to take *only* unicodes, or (b) raise Exceptions at the slightest provocation when handed bytes. It should only take unicodes and bitch profusely about anything else. And things only get worse if you try to provide this interface for say From (let alone Content-Type). Is it really worth doing the mapping interface if it's only usable with free-form headers (ie, only Subject among the commonly used headers)? Sure, for other headers it might *not* accept unicodes... How do you distinguish raw bytes from encoded bytes? __setitem__() shouldn't accept bytes at all. Right on :-) Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 382: little help for stupid people?
In either of the proposals on the table, what code would I write and where to have a base package with a set of add-on packages? I don't quite understand the question. Why would you want to write code (except for the code that actually is in the packages)? PEP 382 is completely declarative - no need to write code. code is anything I need to write to make this work... So, what do I need to do? Ok, so create three tar files: 1. base.tar, containing simplistix/ simplistix/__init__.py 2. addon1.tar, containing simplistix/addon1.pth (containing a single *) simplistix/feature1.py 3. addon2.tar, containing simplistix/addon2.pth simplistix/feature2.py Unpack each of them anywhere on sys.path, in any order. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 382: Namespace Packages
It's unclear, however, who is using base packages besides mx.* and ll.*, although I'd guess from the PyPI listings that perhaps Django is. (It seems that base packages are more likely to use a 'base-extension' naming pattern, vs. the 'namespace.project' pattern used by pure packages.) I'll stress it again in case you missed it the first time: I think the main reason people use pure namespace versus base namespace packages is because hardly anyone know how to do the latter, not because there is no desire to do so! I, for one, have been trying to figure out how to do base namespace packages for years... You mean, without PEP 382? That won't be possible, unless you can coordinate all addon packages. Base packages are a feature solely of PEP 382. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 382: Namespace Packages
At 05:35 PM 5/1/2009 +0100, Chris Withers wrote: P.J. Eby wrote: It's unclear, however, who is using base packages besides mx.* and ll.*, although I'd guess from the PyPI listings that perhaps Django is. (It seems that base packages are more likely to use a 'base-extension' naming pattern, vs. the 'namespace.project' pattern used by pure packages.) I'll stress it again in case you missed it the first time: I think the main reason people use pure namespace versus base namespace packages is because hardly anyone know how to do the latter, not because there is no desire to do so! I didn't say there's *no* desire, however IIRC the only person who *ever* asked on distutils-sig how to do a base package with setuptools was the author of the ll.* packages. And in the case of at least the zope.* peak.* and osaf.* namespace packages it was specifically *not* the intention to have a base __init__. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 382: Namespace Packages
At 07:41 PM 5/1/2009 +0200, Martin v. Löwis wrote: It's unclear, however, who is using base packages besides mx.* and ll.*, although I'd guess from the PyPI listings that perhaps Django is. (It seems that base packages are more likely to use a 'base-extension' naming pattern, vs. the 'namespace.project' pattern used by pure packages.) I'll stress it again in case you missed it the first time: I think the main reason people use pure namespace versus base namespace packages is because hardly anyone know how to do the latter, not because there is no desire to do so! I, for one, have been trying to figure out how to do base namespace packages for years... You mean, without PEP 382? That won't be possible, unless you can coordinate all addon packages. Base packages are a feature solely of PEP 382. Actually, if you are using only the distutils, you can do this by listing only modules in the addon projects; this is how the ll.* tools are doing it. That only works if the packages are all being installed in the same directory, though, not as eggs. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 382: Namespace Packages
Actually, if you are using only the distutils, you can do this by listing only modules in the addon projects; this is how the ll.* tools are doing it. That only works if the packages are all being installed in the same directory, though, not as eggs. Right: if all portions install into the same directory, you can have base packages already. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Oddity PEP 0 key
2009/5/1 MRAB goo...@mrabarnett.plus.com: I've just noticed an oddity in the key in PEP 0. Most letters are used more than once. Wouldn't it be clearer if different letters were used for Accepted and Active instead of them both being 'A', for example? - A - Accepted proposal - R - Rejected proposal W - Withdrawn proposal - D - Deferred proposal F - Final proposal - A - Active proposal - D - Draft proposal - R - Replaced proposal Yes, that makes more sense. Would you like to submit a patch against the PEP 0 generator? (It's in peps/pep0) -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 and GUI libraries
Zooko O'Whielacronx wrote:
Following-up to my own post to correct a major error:
Is it true that
srcbytes.encode(srcencoding, 'python-escape').decode('utf-8',
'python-escape') will always produce srcbytes ? That is my Requirement
If you start with bytes, decode with utf-8b to unicode (possibly
'invalid'), and encode the result back to bytes with utf-8b, you should
get the original bytes, regardless of what they were. That is the point
of PEP 383 -- to reliably roundtrip file 'names' that start as bytes and
must end as the same bytes but which may not otherwise have a unicode
decoding.
If you start with invalid unicode text, encode to bytes with utf-8b, and
decode back to unicode, you might instead get a different and valid
unicode text. An example was given in the discussion. I believe this
would be hard to avoid. An any case, it does not matter for the use
case of starting with bytes that one wants to temporarily but surely
work with as text.
Terry Jan Reedy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Oddity PEP 0 key
Benjamin Peterson wrote: 2009/5/1 MRAB goo...@mrabarnett.plus.com: I've just noticed an oddity in the key in PEP 0. Most letters are used more than once. Wouldn't it be clearer if different letters were used for Accepted and Active instead of them both being 'A', for example? - A - Accepted proposal - R - Rejected proposal W - Withdrawn proposal - D - Deferred proposal F - Final proposal - A - Active proposal - D - Draft proposal - R - Replaced proposal Yes, that makes more sense. Would you like to submit a patch against the PEP 0 generator? (It's in peps/pep0) I'm still trying to think which letters to use! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Oddity PEP 0 key
MRAB wrote: Benjamin Peterson wrote: 2009/5/1 MRAB goo...@mrabarnett.plus.com: I've just noticed an oddity in the key in PEP 0. Most letters are used more than once. Wouldn't it be clearer if different letters were used for Accepted and Active instead of them both being 'A', for example? - A - Accepted proposal - R - Rejected proposal W - Withdrawn proposal - D - Deferred proposal F - Final proposal - A - Active proposal - D - Draft proposal - R - Replaced proposal Yes, that makes more sense. Would you like to submit a patch against the PEP 0 generator? (It's in peps/pep0) I'm still trying to think which letters to use! P for Proposal (to replace Active Proposal)? Every active PEP is a proposal... Michael ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Oddity PEP 0 key
On May 1, 2009, at 5:55 PM, Michael Foord wrote: P for Proposal (to replace Active Proposal)? Every active PEP is a proposal... +1 Maybe even s/Active/Proposed/g ? -Barry PGP.sig Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 and GUI libraries
On 01May2009 18:38, Martin v. L?wis mar...@v.loewis.de wrote:
| Okay, I am wrong about this. Having a flag to remember whether I had to
| fall back to the utf-8b trick is one method to implement my requirement,
| but my actual requirement is this:
|
| Requirement: either the unicode string or the bytes are faithfully
| transmitted from one system to another.
|
| I don't understand this requirement very well, in particular not
| the faithfully part.
|
| That is: if you read a filename from the filesystem, and transmit that
| filename to another system and use it, then there are two cases:
|
| What do you mean by use it? Things like opening files? How does
| that work? In general, a file name valid on one system is invalid
| on a different system - or, at least, refers to a different file
| over there. This is independent of encodings.
I think he's doing a file transfer of some kind and needs to preserve
the names. Or I would guess the two systems are not both UNIX or there
is some subtlety not yet mentioned, or he'd just use tar or some other
byte-level UNIX tool.
| Requirement 1: the byte string was valid in the encoding of source
| system, in which case the unicode name is faithfully transmitted
| (i.e. the bytes that finally land on the target system are the result of
| sourcebytes.decode(source_sys_encoding).encode(target_sys_encoding).
|
| In all your descriptions, I'm puzzled as to where exactly you get
| the source bytes from. If you use the PEP 383 interfaces, you will
| start with character strings, not byte strings, always.
But if both system do present POSIX layers, it's bytes underneath and
the system tools will natively use bytes. He wants to ensure that he can
read using python, using listdir, and elsewhere when he writing using
python, preserve the bytes layer. I think.
In fact it sounds like he may be translating valid unicode and carefully not
altering byte names that don't decode. That in turn implies that the codec
may be different on the two systems.
| Okay, I find it surprisingly easy to make subtle errors in this encoding
| stuff, so please let me know if you spot one. Is it true that
| srcbytes.encode(srcencoding, 'python-escape').decode('utf-8',
| 'python-escape') will always produce srcbytes ?
|
| I think you mixed up bytes and unicode here: if srcbytes is indeed
| a bytes object, then you can't apply .encode to it.
I think he has encode/decode swapped (I did too back in the uber-thread;
if your mapping is one-to-one the distinction is almost arbitrary).
However, his assertion/hope is true only if srcencoding == 'utf-8'.
The PEP itself says that it works if the decode and encode use the same
mapping.
--
Cameron Simpson c...@zip.com.au DoD#743
http://www.cskk.ezoshosting.com/cs/
How do you know I'm Mad? asked Alice.
You must be, said the Cat, or you wouldn't have come here.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Oddity PEP 0 key
Michael Foord wrote: MRAB wrote: Benjamin Peterson wrote: 2009/5/1 MRAB goo...@mrabarnett.plus.com: I've just noticed an oddity in the key in PEP 0. Most letters are used more than once. Wouldn't it be clearer if different letters were used for Accepted and Active instead of them both being 'A', for example? - A - Accepted proposal - R - Rejected proposal W - Withdrawn proposal - D - Deferred proposal F - Final proposal - A - Active proposal - D - Draft proposal - R - Replaced proposal Yes, that makes more sense. Would you like to submit a patch against the PEP 0 generator? (It's in peps/pep0) I'm still trying to think which letters to use! P for Proposal (to replace Active Proposal)? Every active PEP is a proposal... The full list is: S - Standards Track PEP I - Informational PEP P - Process PEP A - Accepted proposal R - Rejected proposal W - Withdrawn proposal D - Deferred proposal F - Final proposal A - Active proposal D - Draft proposal R - Replaced proposal using one letter from each set. From looking more closely at the code: Only 'Informational' or 'Process' PEPs can be 'Active'. 'Draft' and 'Active' are shown as a single space instead of 'D' or 'A'. Therefore: S - Standards Track PEP I - Informational PEP P - Process PEP A - Accepted proposal R - Rejected proposal W - Withdrawn proposal D - Deferred proposal F - Final proposal [A - Active proposal # blank, so can be omitted from key] [D - Draft proposal # blank, so can be omitted from key] R - Replaced proposal leaving just 'Rejected' and 'Replaced' to be disambiguated. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] svn down?
When checking in, I get: Transmitting file data .svn: Commit failed (details follow): svn: Can't create directory '/data/repos/projects/db/transactions/72186-1.txn': Read-only file system With 'svn up', I get: svn: Can't find a temporary directory: Internal error ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] svn down?
2009/5/1 Eric Smith e...@trueblade.com: When checking in, I get: Transmitting file data .svn: Commit failed (details follow): svn: Can't create directory '/data/repos/projects/db/transactions/72186-1.txn': Read-only file system With 'svn up', I get: svn: Can't find a temporary directory: Internal error I get that, too. In addition, I can't ssh to dinsdale. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] yield from?
What's the status of yield from? There's still a small window open for a patch to be checked into 3.1's branch. I haven't been following the python-ideas threads, so I'm not sure if it's ready yet. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 and GUI libraries
Folks:
Being new to the use of gmail, I accidentally sent the following only
to MvL and not to the list. He promptly replied with a helpful
counterexample showing that my design can suffer collisions. :-)
Regards,
Zooko
On Fri, May 1, 2009 at 10:38 AM, Martin v. Löwis mar...@v.loewis.de wrote:
Requirement: either the unicode string or the bytes are faithfully
transmitted from one system to another.
I don't understand this requirement very well, in particular not
the faithfully part.
That is: if you read a filename from the filesystem, and transmit that
filename to another system and use it, then there are two cases:
What do you mean by use it? Things like opening files? How does
that work? In general, a file name valid on one system is invalid
on a different system - or, at least, refers to a different file
over there. This is independent of encodings.
Tahoe is a backup and filesharing program, so you might for example,
execute tahoe cp -r Motörhead tahoe: to copy all the contents of
your Motörhead directory to your Tahoe filesystem. Later you or a
friend, might execute tahoe cp -r tahoe:Motörhead . to copy
everything from that directory within your Tahoe filesystem to your
local filesystem. So in this case the flow of information is
local_system_1 - Tahoe - local_system_2.
The Requirement 1 is that for each filename encountered which is a
valid encoding in local_system_1, then the resulting (unicode) name is
transmitted through the Tahoe filesystem and then written out into
local_system_2 in the expected way (i.e. just by using the Python
unicode APIs and passing the unicode object to them).
Requirement 2 is that for each filename encountered which is not a
valid encoding in local_system_1, then the original bytes are
transmitted through the Tahoe filesystem and then, if the target
system is a byte-oriented system such as Linux, the original bytes are
written into the target filesystem. (If the target is not Linux then
mojibake! but we don't have to go into that now.)
Does that make sense?
In all your descriptions, I'm puzzled as to where exactly you get
the source bytes from. If you use the PEP 383 interfaces, you will
start with character strings, not byte strings, always.
On Mac and Windows, we use the Python unicode APIs e.g.
os.listdir(uMotörhead). On Linux and Solaris, we use the Python
bytestring APIs e.g.
os.listdir(Motörhead.encode(sys.getfilesystemencoding())).
Okay, I find it surprisingly easy to make subtle errors in this encoding
stuff, so please let me know if you spot one. Is it true that
srcbytes.encode(srcencoding, 'python-escape').decode('utf-8',
'python-escape') will always produce srcbytes ?
I think you mixed up bytes and unicode here: if srcbytes is indeed
a bytes object, then you can't apply .encode to it.
Yep, I reversed the order of encode() and decode(). However, my whole
statement was utterly wrong and shows that I still didn't fully get it
yet. I have flip-flopped again and currently think that PEP 383 is
useless for this use case and that my original plan [1] is still the
way to go. Please let me know if you spot a flaw in my plan or a
ridiculousity in my requirements, or if you see a way that PEP 383 can
help me.
Thank you very much.
Regards,
Zooko
[1] http://allmydata.org/trac/tahoe/ticket/534#comment:47
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] yield from?
Alas, I haven't been following it either recently. Too bad, really, because before I left (now three weeks ago) it was already pretty close. We could perhaps even check in Greg's patch (which I tried and looked like a solid implementation of his proposal at the time) and finagle it for b2. One problem though is that Greg's code is based on 2.6... On Fri, May 1, 2009 at 6:27 PM, Benjamin Peterson benja...@python.org wrote: What's the status of yield from? There's still a small window open for a patch to be checked into 3.1's branch. I haven't been following the python-ideas threads, so I'm not sure if it's ready yet. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 and GUI libraries
On May 1, 2009, at 9:42 PM, Zooko O'Whielacronx wrote: Yep, I reversed the order of encode() and decode(). However, my whole statement was utterly wrong and shows that I still didn't fully get it yet. I have flip-flopped again and currently think that PEP 383 is useless for this use case and that my original plan [1] is still the way to go. Please let me know if you spot a flaw in my plan or a ridiculousity in my requirements, or if you see a way that PEP 383 can help me. If I were designing a new system such as this, I'd probably just go for utf8b *always*. That is, set the filesystem encoding to utf-8b. The end. All files always keep the same bytes transferring between unix systems. Thus, for the 99% of the world that uses either windows or a utf-8 locale, they get useful filenames inside tahoe. The other 1% of the world that uses something like latin-1, EUC_JP, etc. on their local system sees mojibake filenames in tahoe, but will see the same filename that they put in when they take it back out. Gnome already uses only utf-8 for filename displays for a few years now, for example, so this isn't exactly an unheard-of position to take... But if you don't do that, then, I still don't see what purpose your requirements serve. If I have two systems: one with a UTF-8 locale, and one with a Latin-1 locale, why should transmitting filenames from system 1 to system 2 through tahoe preserve the raw bytes, but doing the reverse *not* preserve the raw bytes? (all byte-sequences are valid in latin-1, remember, so they'll all decode into unicode without error, and then be reencoded in utf-8...). This seems rather a useless behavior to me. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Oddity PEP 0 key
.. leaving just 'Rejected' and 'Replaced' to be disambiguated. 'X' or 'Z' for Rejected? Looks like a perfect start for a bikeshed discussion. :-) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Oddity PEP 0 key
Barry Warsaw writes: On May 1, 2009, at 5:55 PM, Michael Foord wrote: P for Proposal (to replace Active Proposal)? Every active PEP is a proposal... +1 Maybe even s/Active/Proposed/g ? Shouldn't that be s/Active/Proposed/g duck / ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Oddity PEP 0 key
Alexander Belopolsky writes: .. leaving just 'Rejected' and 'Replaced' to be disambiguated. 'X' or 'Z' for Rejected? Looks like a perfect start for a bikeshed discussion. :-) The Japanese contingent suggests O (UPPERCASE LATIN LETTER O) for accepted and X for rejected. (Actually these should be U+25EF and U+00D7, respectively.) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
