Karen Tracey <[EMAIL PROTECTED]> added the comment:
I believe the problem is in your test file, not doctest. The enclosing
doctest string is not specified as a unicode literal, so the file
encoding specification ultimately has no effect on it. At least that is
how I read the documentation regarding the effect of the ecoding
declaration ("The encoding is used for all lexical analysis, in
particular to find the end of a string, and to interpret the contents of
Unicode literals. String literals are converted to Unicode for
syntactical analysis, then converted back to their original encoding
before interpretation starts.")
If you change the test file so that the string enclosing the test is a
unicode literal then the test passes:
[EMAIL PROTECTED]:~/tmp$ cat test_iso-8859-15.py
# -*- coding: utf-8 -*-
import doctest
def normalize(s):
u"""
>>> normalize(u'á')
u'b'
"""
return s.translate({ord(u'á'): u'b'})
doctest.testmod()
print 'without doctest ===>>>', normalize(u'á')
[EMAIL PROTECTED]:~/tmp$ python test_iso-8859-15.py
without doctest ===>>> b
-----
There is a problem with this, though: doctest now will be unable to
correctly report errors when there are output mismatches involving
unicode strings with non-ASCII chars. For example if you add an 'x' to
the front of your unicode literal to be normalized you'll get this when
you try to run it:
[EMAIL PROTECTED]:~/tmp$ python test_iso-8859-15.py
Traceback (most recent call last):
File "test_iso-8859-15.py", line 12, in <module>
doctest.testmod()
File "/usr/lib/python2.5/doctest.py", line 1799, in testmod
runner.run(test)
File "/usr/lib/python2.5/doctest.py", line 1345, in run
return self.__run(test, compileflags, out)
File "/usr/lib/python2.5/doctest.py", line 1261, in __run
self.report_failure(out, test, example, got)
File "/usr/lib/python2.5/doctest.py", line 1125, in report_failure
self._checker.output_difference(example, got, self.optionflags))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in
position 149: ordinal not in range(128)
[EMAIL PROTECTED]:~/tmp$
This issue is reported in #1293741, but there is no fix or guidance
offered there on how to work around the problem.
I'd appreciate feedback on whether what I've said here is correct. I'm
currently trying to diagnose/fix problems with use of unicode literals
in some tests and this is as far as I've got. That is, I think I need to
be specifying the enclosing strings as unicode literals, but then I run
into #1293741. If the conclusion I've reached is correct, then trying
to figure out a fix for that problem should be where I focus my efforts.
If, however, I shouldn't be specifying the enclosing string as a
unicode literal, then attempting to fix the problem as described here
would perhaps be more useful. Though I do not know how the doctest code
can know the file's encoding specification?
----------
nosy: +kmtracey
_______________________________________
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue2811>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
