On 2008-12-08 22:32, Adam Olsen wrote: > On Mon, Dec 8, 2008 at 2:01 PM, M.-A. Lemburg <[EMAIL PROTECTED]> wrote: >> On 2008-12-08 21:45, Antoine Pitrou wrote: >>> M.-A. Lemburg <mal <at> egenix.com> writes: >>>> Such application specific error handlers could then also apply >>>> whatever fancy round-trip safe encoding of non-decodable bytes >>>> to Unicode escapes, private code points, etc. as seen fit by the >>>> application. >>> I'd argue that such fancy round-trip safe error handler should be provided >>> by >>> Python. It's not reasonable to expect application coders to come up with >>> their >>> own codec variation based on subtle details of the unicode spec. >> Fair enough. We could add some e.g. >> >> * a round-trip safe escape error handler that uses a Unicode private >> code point area which we officially reserve for the Python >> interpreter > > This would of course alter the behaviour of those private code points, > preventing them from round-tripping properly. > > I don't think round-tripping can be done from an error handler. You > need a full codec to do it. A simple option is 8859-1. Or, ya know, > bytes. This has long since gotten repetitive..
The error handler would just map the problem bytes to the private area. The application would then have to decide what to do with them, ie. the error handler only provides one half of the round- tripping. And that's on purpose: I don't believe we can come up with some magic solution for the encodings problem. This is essentially something that applications will have to solve on a case-by-case basis. >> * a human readable escape error handler that encodes the problem >> bytes to say hex escapes, e.g. gives Andr\xe9 for a Latin-1 >> encoded directory name instead of failing > > Similar to 'รถ'.encode('ascii', 'backslashreplace')? I'm +1 on making that > work. Yes. >> * a warning error handler that replaces the problem cases with >> a question mark and issues a warning through the warning >> framework > > I dub thee errors='warnreplace'. Yep, something along those lines. Perhaps there are more and better alternatives. These suggestions are just to show how the idea could be put to some real-life use. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Dec 08 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-12-02: Released mxODBC.Connect 1.0.0 http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com