[issue3727] poplib module broken by str to unicode conversion
Christian Heimes [EMAIL PROTECTED] added the comment: Patch applied in r67109 -- nosy: +christian.heimes resolution: accepted - fixed status: open - closed ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
Changes by STINNER Victor [EMAIL PROTECTED]: Removed file: http://bugs.python.org/file11813/poplib-bytes-2.patch ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
STINNER Victor [EMAIL PROTECTED] added the comment:
Le Tuesday 04 November 2008 01:03:42 Barry A. Warsaw, vous avez écrit :
Benjamin's reviewed this and the only thing that jumps out at me is some
funky indentation at about line 331
It's not related to my patch (I did'nt change POP3_SSL comment). But well, as
you want: a new patch re-indent the See the methods of the parent (...)
line (331).
Added file: http://bugs.python.org/file11940/poplib-bytes-3.patch
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3727
___Index: Lib/test/test_poplib.py
===
--- Lib/test/test_poplib.py (révision 67094)
+++ Lib/test/test_poplib.py (copie de travail)
@@ -1,43 +1,284 @@
+Test script for poplib module.
+
+# Modified by Giampaolo Rodola' to give poplib.POP3 and poplib.POP3_SSL
+# a real test suite
+
+import poplib
+import threading
+import asyncore
+import asynchat
import socket
-import threading
-import poplib
+import os
import time
from unittest import TestCase
-from test import support
+from test import support as test_support
-HOST = support.HOST
+HOST = test_support.HOST
+PORT = 0
-def server(evt, serv):
-serv.listen(5)
-try:
-conn, addr = serv.accept()
-except socket.timeout:
-pass
-else:
-conn.send(b+ Hola mundo\n)
-conn.close()
-finally:
-serv.close()
-evt.set()
+# the dummy data returned by server when LIST and RETR commands are issued
+LIST_RESP = b'1 1\r\n2 2\r\n3 3\r\n4 4\r\n5 5\r\n.\r\n'
+RETR_RESP = bFrom: [EMAIL PROTECTED]
+\r\nContent-Type: text/plain\r\n\
+MIME-Version: 1.0\r\n\
+Subject: Dummy\r\n\
+\r\n\
+line1\r\n\
+line2\r\n\
+line3\r\n\
+.\r\n
-class GeneralTests(TestCase):
+class DummyPOP3Handler(asynchat.async_chat):
+
+def __init__(self, conn):
+asynchat.async_chat.__init__(self, conn)
+self.set_terminator(b\r\n)
+self.in_buffer = []
+self.push('+OK dummy pop3 server ready.')
+
+def collect_incoming_data(self, data):
+self.in_buffer.append(data)
+
+def found_terminator(self):
+line = b''.join(self.in_buffer)
+line = str(line, 'ISO-8859-1')
+self.in_buffer = []
+cmd = line.split(' ')[0].lower()
+space = line.find(' ')
+if space != -1:
+arg = line[space + 1:]
+else:
+arg =
+if hasattr(self, 'cmd_' + cmd):
+method = getattr(self, 'cmd_' + cmd)
+method(arg)
+else:
+self.push('-ERR unrecognized POP3 command %s.' %cmd)
+
+def handle_error(self):
+raise
+
+def push(self, data):
+asynchat.async_chat.push(self, data.encode(ISO-8859-1) + b'\r\n')
+
+def cmd_echo(self, arg):
+# sends back the received string (used by the test suite)
+self.push(arg)
+
+def cmd_user(self, arg):
+if arg != guido:
+self.push(-ERR no such user)
+self.push('+OK password required')
+
+def cmd_pass(self, arg):
+if arg != python:
+self.push(-ERR wrong password)
+self.push('+OK 10 messages')
+
+def cmd_stat(self, arg):
+self.push('+OK 10 100')
+
+def cmd_list(self, arg):
+if arg:
+self.push('+OK %s %s' %(arg, arg))
+else:
+self.push('+OK')
+asynchat.async_chat.push(self, LIST_RESP)
+
+cmd_uidl = cmd_list
+
+def cmd_retr(self, arg):
+self.push('+OK %s bytes' %len(RETR_RESP))
+asynchat.async_chat.push(self, RETR_RESP)
+
+cmd_top = cmd_retr
+
+def cmd_dele(self, arg):
+self.push('+OK message marked for deletion.')
+
+def cmd_noop(self, arg):
+self.push('+OK done nothing.')
+
+def cmd_rpop(self, arg):
+self.push('+OK done nothing.')
+
+
+class DummyPOP3Server(asyncore.dispatcher, threading.Thread):
+
+handler = DummyPOP3Handler
+
+def __init__(self, address, af=socket.AF_INET):
+threading.Thread.__init__(self)
+asyncore.dispatcher.__init__(self)
+self.create_socket(af, socket.SOCK_STREAM)
+self.bind(address)
+self.listen(5)
+self.active = False
+self.active_lock = threading.Lock()
+self.host, self.port = self.socket.getsockname()[:2]
+
+def start(self):
+assert not self.active
+self.__flag = threading.Event()
+threading.Thread.start(self)
+self.__flag.wait()
+
+def run(self):
+self.active = True
+self.__flag.set()
+while self.active and asyncore.socket_map:
+self.active_lock.acquire()
+asyncore.loop(timeout=0.1, count=1)
+self.active_lock.release()
+asyncore.close_all(ignore_all=True)
+
+def stop(self):
+assert self.active
+self.active = False
+self.join()
+
+def
[issue3727] poplib module broken by str to unicode conversion
Barry A. Warsaw [EMAIL PROTECTED] added the comment: Benjamin's reviewed this and the only thing that jumps out at me is some funky indentation at about line 331. If you fix that, you can land this patch. -- keywords: -needs review nosy: +barry resolution: - accepted ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
Changes by Benjamin Peterson [EMAIL PROTECTED]: -- assignee: - benjamin.peterson ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
Benjamin Peterson [EMAIL PROTECTED] added the comment: I'm happy with Victor's patch. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
STINNER Victor [EMAIL PROTECTED] added the comment: Can anyone review my last patch (poplib-bytes-2.patch)? -- keywords: +needs review ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
Benjamin Peterson [EMAIL PROTECTED] added the comment: I like this patch. -- nosy: +benjamin.peterson ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
Changes by STINNER Victor [EMAIL PROTECTED]: ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
STINNER Victor [EMAIL PROTECTED] added the comment: Oooops, I removed the message74562 from giampaolo.rodola, sorry: As for issue #3911 this is another module for which an actual test suite would be very necessary. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
Changes by STINNER Victor [EMAIL PROTECTED]: Removed file: http://bugs.python.org/file11782/poplib_unicode-5.patch ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
STINNER Victor [EMAIL PROTECTED] added the comment: After testing real world message, my patch using pure unicode doesn't work. The problem comes with message encoding with 8-bit encoding. If the email charset is different than POP3.encoding, the message in not decoded correctly. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
STINNER Victor [EMAIL PROTECTED] added the comment: New patch: resp() returns bytes - self.file is now a binary file - encode commands using POP3.encoding charset, default is UTF-8 - use md5.hexdigest() - factorize POP3_SSL code: code specific for SSL is just the creation of the socket The default charset is UTF-8, but most servers only accept pure ASCII login/password (eg. gmail.com) or a smaller subset of ASCII (eg. only A-Z, a-z, 0-9 and some ponctuation signs :-/). If you user non-ASCII login/password and your server doesn't use UTF-8, change POP3.encoding or your pop object.encoding (encoding is not used in the constructor). welcome attribute (and getwelcome() results) is a bytes string. You have to parse the message headers to get the right charset to decode bytes to unicode characters. A multipart message may contains two or more charsets and different encoding. But poplib is not responsible to decode messages, just to download a message as bytes. Arguments (username (login), password, a message identifier) are unicode strings. For a message identifier, you can also use an integer (nothing new, it was already possible). I hope that apop() works. No Python error is raised but no server does support his authentication method. I tested 3 servers (pop3.haypocalc.com, pop.laposte.net and pop.gmail.com) and none supports APOP. I tested POP3 and POP3_SSL (gmail requires SSL). Added file: http://bugs.python.org/file11812/poplib-bytes.patch ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
STINNER Victor [EMAIL PROTECTED] added the comment: About apop(): the second argument is the user password, not a shared password which is the local variable timestamp read from welcome attribute. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
Changes by STINNER Victor [EMAIL PROTECTED]: Removed file: http://bugs.python.org/file11812/poplib-bytes.patch ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
STINNER Victor [EMAIL PROTECTED] added the comment: I forgot the new unit tests. New patch: - port python trunk unit tests Added file: http://bugs.python.org/file11813/poplib-bytes-2.patch ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
STINNER Victor [EMAIL PROTECTED] added the comment: Here is a patch proposition: - a socket uses bytes - makefile() creates an unicode file using 'r' mode - default encoding ISO-8859-1 because I guess that most servers use this encoding, but you can change the encoding using encoding constructor optioan argument - read unicode and write unicode: convert convert from/to bytes at the last moment (just after/before reading/writing the socket) - cosmetic: use .startswith() instead of for example b[:2] == '..' Test updates: - replace localhost by HOST - write a test for a logging (user + password) Missing: no SSL unit test. I tested SSL on my personal POP3 account, but only the login. -- keywords: +patch nosy: +haypo Added file: http://bugs.python.org/file11775/poplib_unicode.patch ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
Changes by STINNER Victor [EMAIL PROTECTED]: Removed file: http://bugs.python.org/file11775/poplib_unicode.patch ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
STINNER Victor [EMAIL PROTECTED] added the comment: Ooops, my previous patch was wrong (startswith = not startswith). I tested python trunk test for poplib: with minor changes, all tests are ok except tests using SSL. I get a select.error: (9, 'Bad file descriptor') from asyncore. So I tried to synchronize python3 ssl with python2 trunk, but it depends on python2 trunk version of the socket module and this module is very complex and hard to port to python3. About EBADF error from select(), it may comes from missing makefile() method of the ssl socket wrapper. Added file: http://bugs.python.org/file11778/poplib_unicode-2.patch ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
STINNER Victor [EMAIL PROTECTED] added the comment: New version: - fix SSL: self.file contains the SSL socket and not the raw socket! - upgrade test_poplib.py from Python trunk poplib should be refactored to reuse the new IO library. But I don't know how to build a TextIO wrapper for a socket. Using TextIO, it would be possible to remove newline (CR/LF) and unicode encoding/decoding code. Added file: http://bugs.python.org/file11780/poplib_unicode-3.patch ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
Giampaolo Rodola' [EMAIL PROTECTED] added the comment: I haven't tried the patch but I think that encoding should be a class attribute as it is in ftplib and similar py3k network related modules. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
STINNER Victor [EMAIL PROTECTED] added the comment:
New version:
- remove duplicate methods of POP3_SSL()
- use makefile('r', encoding=self.encoding) to get a nice text
wrapper with universal newline
- remove newline conversion (done by TextIOWrapper)
Finally my patch removes more code in poplib.py than it adds :-D I
like such patch.
Python3 new I/O library rocks!
Added file: http://bugs.python.org/file11781/poplib_unicode-4.patch
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3727
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
Changes by STINNER Victor [EMAIL PROTECTED]: Removed file: http://bugs.python.org/file11778/poplib_unicode-2.patch ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
Changes by STINNER Victor [EMAIL PROTECTED]: Removed file: http://bugs.python.org/file11780/poplib_unicode-3.patch ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
STINNER Victor [EMAIL PROTECTED] added the comment: @giampaolo.rodola: Right, I also prefer encoding as a class attribute. So I wrote a new patch: - encoding is now a class attribute - continue SSL code factorization: SSL code is now around 10 lines instead of 70 lines! Added file: http://bugs.python.org/file11782/poplib_unicode-5.patch ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
Changes by STINNER Victor [EMAIL PROTECTED]: Removed file: http://bugs.python.org/file11781/poplib_unicode-4.patch ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
Giampaolo Rodola' [EMAIL PROTECTED] added the comment: As for issue #3911 this is another module for which an actual test suite would be very necessary. -- nosy: +giampaolo.rodola ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
Changes by Benjamin Peterson [EMAIL PROTECTED]: -- title: poplib module broken by str to unicode conversionhttp://bugs.python.org/issue3727 - poplib module broken by str to unicode conversion ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3727 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3727] poplib module broken by str to unicode conversion
New submission from Dmitry Vasiliev [EMAIL PROTECTED]:
Example:
from poplib import POP3
p = POP3(localhost)
p.user(user)
Traceback (most recent call last):
File stdin, line 1, in module
File /py3k/Lib/poplib.py, line 179, in user
return self._shortcmd('USER %s' % user)
File /py3k/Lib/poplib.py, line 151, in _shortcmd
self._putcmd(line)
File /py3k/Lib/poplib.py, line 98, in _putcmd
self._putline(line)
File /py3k/Lib/poplib.py, line 91, in _putline
self.sock.sendall('%s%s' % (line, CRLF))
TypeError: sendall() argument 1 must be string or buffer, not str
p.user(buser)
Traceback (most recent call last):
File stdin, line 1, in module
File /py3k/Lib/poplib.py, line 179, in user
return self._shortcmd('USER %s' % user)
File /py3k/Lib/poplib.py, line 151, in _shortcmd
self._putcmd(line)
File /py3k/Lib/poplib.py, line 98, in _putcmd
self._putline(line)
File /py3k/Lib/poplib.py, line 91, in _putline
self.sock.sendall('%s%s' % (line, CRLF))
TypeError: sendall() argument 1 must be string or buffer, not str
--
components: Library (Lib)
messages: 72136
nosy: hdima
severity: normal
status: open
title: poplib module broken by str to unicode conversion
type: crash
versions: Python 3.0
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3727
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
