[issue7262] codecs.open() + eol (windows)

[shamilbi]

New submission from shamilbi :

different eol when writing to fp = codecs.open(.., 'w', 'cp866')
(windows, python-2.6.4)

def write(fp):
fp.write("""\
a
""")

# eol=0d0a (windows, python-2.6.4)
with open('0d0a.tmp', 'w') as fp:
write(fp)

# eol=0d0a (windows, python-2.6.4)
with codecs.open('0d0a-codecs.tmp', 'w') as fp:
write(fp)

# --- BUG ---
# eol=0a (windows, python-2.6.4)
with codecs.open('0a-codecs.tmp', 'w', 'cp866') as fp:
write(fp)

--
components: Library (Lib)
files: eol-bug.py
messages: 94888
nosy: shamilbi
severity: normal
status: open
title: codecs.open() + eol (windows)
type: behavior
versions: Python 2.6
Added file: http://bugs.python.org/file15256/eol-bug.py

___
Python tracker 
<http://bugs.python.org/issue7262>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5170] logging to file + encoding

[shamilbi]

New submission from shamilbi :

if i configure logging into a file with encoding = 'cp1251' and do
logger.debug(u'...') then i get crash with UnicodeError

i suggest reimplementing method FileHandler.emit():
...
if isinstance(msg, unicode):
stream.write(f % msg)# it works!
...

--
components: Library (Lib)
messages: 81274
nosy: shamilbi
severity: normal
status: open
title: logging to file + encoding
type: crash
versions: Python 2.6

___
Python tracker 
<http://bugs.python.org/issue5170>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5170] logging to file + encoding

[shamilbi]

shamilbi  added the comment:

test_log.py:
---
#! -*- coding: windows-1251 -*-

import logging

logger = logging.getLogger('test_log')
logger.addHandler(logging.FileHandler('test.log', encoding='cp1251'))
logger.setLevel(logging.DEBUG)

logger.debug(u'Привет')# russian Hello

exception:
-
Traceback (most recent call last):
  File "e:\bin\python\0\lib\logging\__init__.py", line 765, in emit
self.stream.write(fs % msg.encode("UTF-8"))
  File "e:\bin\python\0\lib\codecs.py", line 686, in write
return self.writer.write(data)
  File "e:\bin\python\0\lib\codecs.py", line 351, in write
data, consumed = self.encode(object, self.errors)
  File "e:\bin\python\0\lib\encodings\cp1251.py", line 12, in encode
return codecs.charmap_encode(input,errors,encoding_table)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 0:
ordinal not in range(128)

___
Python tracker 
<http://bugs.python.org/issue5170>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5170] logging to file + encoding

[shamilbi]

shamilbi  added the comment:

(python 2.6.2, WinXP)
logging to console stopped working. Here is a workaround:

logging/__init__.py:
class StreamHandler(Handler):
...
def emit(self, record):
...
if (isinstance(msg, unicode) or
getattr(stream, 'encoding', None) is None):
- stream.write(fs % msg)
+ if stream == sys.stdout:
+ print(msg)
+ else:
+ stream.write(fs % msg)

--
status: closed -> open
type: crash -> behavior

___
Python tracker 
<http://bugs.python.org/issue5170>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5170] logging to file + encoding

[shamilbi]

shamilbi  added the comment:

>
> Can you retry with setting the "encoding" attribute of the file to
> "cp1251"? That should work and that should be the appropriate method to
> avoid the problem.
>
> test_logging.py in the Python distribution has a test which exercises
> Unicode functionality using cp1251, does that test work in your
> environment?

logging to file is OK (since 2.6.2), logging to console __was__ OK (2.6.1),
not now (2.6.2)

shamil

--
Added file: http://bugs.python.org/file13726/unnamed

___
Python tracker 
<http://bugs.python.org/issue5170>
___Can you retry with setting the "encoding" attribute of the file 
to

"cp1251"? That should work and that should be the appropriate method 
to
avoid the problem.

test_logging.py in the Python distribution has a test which exercises
Unicode functionality using cp1251, does that test work in your 
environment? logging to file is OK (since 2.6.2), 
logging to console __was__ OK (2.6.1), not now (2.6.2)shamil

___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5170] logging to file + encoding

[shamilbi]

shamilbi  added the comment:

>
> Trunk and release26-maint were recently changed (r71657, r71658) to use
> the following logic, which differs from the code snippet you posted.
>
>if (isinstance(msg, unicode) and
>getattr(stream, 'encoding', None)):
>stream.write(fs.decode(stream.encoding) % msg)
> else:
>stream.write(fs % msg)
>
> If the stream is stderr and you are passing a unicode msg, the else
> branch will not be taken; as long as the stream has an encoding
> attribute, it should output correctly.
>
> The change was made when another, similar issue was posted by another
> user (issue #5768).
>
> Can you confirm what happens with the current code as it is in
> release26-maint? спасибо!

it still doesn't work for console (but OK for files).

the following works in both cases:
if (isinstance(msg, unicode) and
getattr(stream, 'encoding', None) and
(stream == sys.stdout or stream == sys.stderr)):
stream.write(fs % msg.encode(stream.encoding))
else:
 stream.write(fs % msg)

i think it's all about the difference betwin print(msg) and
sys.stdout.write('%s\n' % msg)

shamil

--
Added file: http://bugs.python.org/file13734/unnamed

___
Python tracker 
<http://bugs.python.org/issue5170>
___Trunk and release26-maint were recently changed (r71657, r71658) to 
use

the following logic, which differs from the code snippet you posted.

                    if (isinstance(msg, unicode) and
                        getattr(stream, 'encoding', 
None)):
                        stream.write(fs.decode(stream.encoding) % 
msg)
                    else:
                        stream.write(fs % msg)

If the stream is stderr and you are passing a unicode msg, the else
branch will not be taken; as long as the stream has an encoding
attribute, it should output correctly.

The change was made when another, similar issue was posted by another
user (issue #5768).

Can you confirm what happens with the current code as it is in
release26-maint? спасибо! it still 
doesn't work for console (but OK for files).the following works in 
both cases:if (isinstance(msg, unicode) and
     getattr(stream, 'encoding', None) and    (stream == 
sys.stdout or stream == sys.stderr)):    stream.write(fs % 
msg.encode(stream.encoding))else:     stream.write(fs % 
msg)i think it's all about the difference betwin print(msg) and 
sys.stdout.write('%s\n' % msg)
shamil
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com