[issue7262] codecs.open() + eol (windows)
New submission from shamilbi : different eol when writing to fp = codecs.open(.., 'w', 'cp866') (windows, python-2.6.4) def write(fp): fp.write("""\ a """) # eol=0d0a (windows, python-2.6.4) with open('0d0a.tmp', 'w') as fp: write(fp) # eol=0d0a (windows, python-2.6.4) with codecs.open('0d0a-codecs.tmp', 'w') as fp: write(fp) # --- BUG --- # eol=0a (windows, python-2.6.4) with codecs.open('0a-codecs.tmp', 'w', 'cp866') as fp: write(fp) -- components: Library (Lib) files: eol-bug.py messages: 94888 nosy: shamilbi severity: normal status: open title: codecs.open() + eol (windows) type: behavior versions: Python 2.6 Added file: http://bugs.python.org/file15256/eol-bug.py ___ Python tracker <http://bugs.python.org/issue7262> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5170] logging to file + encoding
New submission from shamilbi : if i configure logging into a file with encoding = 'cp1251' and do logger.debug(u'...') then i get crash with UnicodeError i suggest reimplementing method FileHandler.emit(): ... if isinstance(msg, unicode): stream.write(f % msg)# it works! ... -- components: Library (Lib) messages: 81274 nosy: shamilbi severity: normal status: open title: logging to file + encoding type: crash versions: Python 2.6 ___ Python tracker <http://bugs.python.org/issue5170> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5170] logging to file + encoding
shamilbi added the comment: test_log.py: --- #! -*- coding: windows-1251 -*- import logging logger = logging.getLogger('test_log') logger.addHandler(logging.FileHandler('test.log', encoding='cp1251')) logger.setLevel(logging.DEBUG) logger.debug(u'Привет')# russian Hello exception: - Traceback (most recent call last): File "e:\bin\python\0\lib\logging\__init__.py", line 765, in emit self.stream.write(fs % msg.encode("UTF-8")) File "e:\bin\python\0\lib\codecs.py", line 686, in write return self.writer.write(data) File "e:\bin\python\0\lib\codecs.py", line 351, in write data, consumed = self.encode(object, self.errors) File "e:\bin\python\0\lib\encodings\cp1251.py", line 12, in encode return codecs.charmap_encode(input,errors,encoding_table) UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 0: ordinal not in range(128) ___ Python tracker <http://bugs.python.org/issue5170> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5170] logging to file + encoding
shamilbi added the comment: (python 2.6.2, WinXP) logging to console stopped working. Here is a workaround: logging/__init__.py: class StreamHandler(Handler): ... def emit(self, record): ... if (isinstance(msg, unicode) or getattr(stream, 'encoding', None) is None): - stream.write(fs % msg) + if stream == sys.stdout: + print(msg) + else: + stream.write(fs % msg) -- status: closed -> open type: crash -> behavior ___ Python tracker <http://bugs.python.org/issue5170> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5170] logging to file + encoding
shamilbi added the comment: > > Can you retry with setting the "encoding" attribute of the file to > "cp1251"? That should work and that should be the appropriate method to > avoid the problem. > > test_logging.py in the Python distribution has a test which exercises > Unicode functionality using cp1251, does that test work in your > environment? logging to file is OK (since 2.6.2), logging to console __was__ OK (2.6.1), not now (2.6.2) shamil -- Added file: http://bugs.python.org/file13726/unnamed ___ Python tracker <http://bugs.python.org/issue5170> ___Can you retry with setting the "encoding" attribute of the file to "cp1251"? That should work and that should be the appropriate method to avoid the problem. test_logging.py in the Python distribution has a test which exercises Unicode functionality using cp1251, does that test work in your environment? logging to file is OK (since 2.6.2), logging to console __was__ OK (2.6.1), not now (2.6.2)shamil ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5170] logging to file + encoding
shamilbi added the comment: > > Trunk and release26-maint were recently changed (r71657, r71658) to use > the following logic, which differs from the code snippet you posted. > >if (isinstance(msg, unicode) and >getattr(stream, 'encoding', None)): >stream.write(fs.decode(stream.encoding) % msg) > else: >stream.write(fs % msg) > > If the stream is stderr and you are passing a unicode msg, the else > branch will not be taken; as long as the stream has an encoding > attribute, it should output correctly. > > The change was made when another, similar issue was posted by another > user (issue #5768). > > Can you confirm what happens with the current code as it is in > release26-maint? ÑпаÑибо! it still doesn't work for console (but OK for files). the following works in both cases: if (isinstance(msg, unicode) and getattr(stream, 'encoding', None) and (stream == sys.stdout or stream == sys.stderr)): stream.write(fs % msg.encode(stream.encoding)) else: stream.write(fs % msg) i think it's all about the difference betwin print(msg) and sys.stdout.write('%s\n' % msg) shamil -- Added file: http://bugs.python.org/file13734/unnamed ___ Python tracker <http://bugs.python.org/issue5170> ___Trunk and release26-maint were recently changed (r71657, r71658) to use the following logic, which differs from the code snippet you posted.           if (isinstance(msg, unicode) and             getattr(stream, 'encoding', None)):             stream.write(fs.decode(stream.encoding) % msg)           else:             stream.write(fs % msg) If the stream is stderr and you are passing a unicode msg, the else branch will not be taken; as long as the stream has an encoding attribute, it should output correctly. The change was made when another, similar issue was posted by another user (issue #5768). Can you confirm what happens with the current code as it is in release26-maint? ÑпаÑибо! it still doesn't work for console (but OK for files).the following works in both cases:if (isinstance(msg, unicode) and    getattr(stream, 'encoding', None) and   (stream == sys.stdout or stream == sys.stderr)):   stream.write(fs % msg.encode(stream.encoding))else:    stream.write(fs % msg)i think it's all about the difference betwin print(msg) and sys.stdout.write('%s\n' % msg) shamil ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com