[Python-Dev] Should bytearray(b'a')[0] should 1-char or number?
# I tried to post this to bug tracker, but couldn't, so posted here... r62095 says >Fix and enable a skipped test: >with python 2.6, enumerating bytes yields 1-char strings, not numbers. > >Don't merge this into the py3k branch. This is true for bytes, but not for bytearray. >>> bytearray(b'a')[0] 97 [28493 refs] >>> b'a'[0] 'a' And this causes error on my environment like this. == FAIL: testDecoder (__main__.StatefulIncrementalDecoderTest) -- Traceback (most recent call last): File "test_io.py", line 629, in testDecoder self.assertEquals(d.decode(input, eof), output) AssertionError: u'o.i.a.b.c.d.' != u'abcd.' But strange, I cannot see this error on python.org buildbot. ??? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Is Py_WIN_WIDE_FILENAMES still alive?
> I believe the macro should be removed, as Python currently assumes Unicode
> APIs are available in a number of places.
My +1 for removal. Even 2.5 cannot be compiled without this macro, probably
no one is using this.
> This consistent with the versions
> of Windows Python currently supports. It is possible that patches would
be
> accepted which enable Python to be built without this functionality - in
> which case the next-best behavior would probably be to convert Unicode to
> MBCS encoded strings and call the *A versions of the API. Looking at the
> history for posixmodule.c etc will offer some clues as to how this could
> best be done.
One problem with removal, PyArg_ParseTuple doesn't have option to convert
to unicode (like "et" exists for unicode -> 8bit char buffer), so it's
harder to
report argument error.
>>> os.rename(4, 2)
Traceback (most recent call last):
File "", line 1, in
TypeError: rename() argument 1 must be string, not int
/* After removal of *A win32 APIs */
if (PyArg_ParseTuple("OO:rename", &o1, &o2)) {
convert_to_unicode(&o1); /* if these methods failed, should we report
same error above
convert_to_unicode(&o2); * by ourselves? */
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Is Py_WIN_WIDE_FILENAMES still alive?
Hello. I noticed when I removes following line in trunk/PC/pyconfig.h #define Py_WIN_WIDE_FILENAMES _fileio.c and posixmodule.c (and maybe more) cannot be compiled on Windows. When Py_WIN_WIDE_FILENAMES is not defined, how should python behave? - call posix functions like open(2) - call ANSI Win32 APIs like MoveFileA Or maybe this macro is not used anymore? Thank you. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.0 buildbots all red
> Yeah, sounds like a memory issue. Did you try running with valgrind > or purify? I haven't done so for a long time, perhaps never on 3k > branch. It would be a good thing to run a tool soon. Maybe is this related? [Potential overflows due to incorrect usage of PyUnicode_AsString] http://bugs.python.org/issue1950 Thank you. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5.2 release coming up
> But on windows, ":" is special letter after drive letter. (ex:
> "C:/Winnt/foo/bar")
> So I imagine open() or something converts ":" to "!" (valid character as
> file name).
Sorry, I lied. :-(
open() didn't change ":", test_mailbox.py (TestMaildir.setUp) changed
self._box.colon to "!"
Otherwise, newly created mail box in test_consistent_factory 's "colon" is
":".
This mismatch causes error.
Following patch solves error, but I don't know if this is good solution.
Index: Lib/test/test_mailbox.py
===
--- Lib/test/test_mailbox.py (revision 60233)
+++ Lib/test/test_mailbox.py (working copy)
@@ -458,12 +458,11 @@
class TestMaildir(TestMailbox):
-_factory = lambda self, path, factory=None: mailbox.Maildir(path,
factory)
-
-def setUp(self):
-TestMailbox.setUp(self)
+def _factory(self, path, factory=None):
+box = mailbox.Maildir(path, factory)
if os.name in ('nt', 'os2') or sys.platform == 'cygwin':
-self._box.colon = '!'
+box.colon = '!'
+return box
def test_add_MM(self):
# Add a MaildirMessage instance
@@ -518,7 +517,7 @@
# Create new mailbox with
class FakeMessage(mailbox.MaildirMessage):
pass
-box = mailbox.Maildir(self._path, factory=FakeMessage)
+box = self._factory(self._path, factory=FakeMessage)
msg2 = box.get_message(key)
self.assert_(isinstance(msg2, FakeMessage))
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5.2 release coming up
> From:
>
http://www.python.org/dev/buildbot/all/x86%20XP-4%202.5/builds/107/step-test/0
>
> The errors are:
>
> File
"E:\cygwin\home\db3l\buildarea\2.5.bolen-windows\build\lib\test\test_mailbox
.py",
> line 522, in test_consistent_factory
> msg2 = box.get_message(key)
> File
"E:\cygwin\home\db3l\buildarea\2.5.bolen-windows\build\lib\mailbox.py",
> line 315, in get_message
> subpath = self._lookup(key)
> File
"E:\cygwin\home\db3l\buildarea\2.5.bolen-windows\build\lib\mailbox.py",
> line 484, in _lookup
> raise KeyError('No message with key: %s' % key)
> KeyError: 'No message with key: 1201127998.M194999P232Q203.buildbot'
I did quick investigation on this error.
After self._refresh() (line 480 in _loopkup - Lib/mailbox.py) runs,
self._toc contains key like this.
1201171711.M848000P1176Q16.whiterab-c2znlh!2,FR
Please look at exclamation mark. Probably this is not intended on most
platforms like Unix.
It should be ":" colon.
But on windows, ":" is special letter after drive letter. (ex:
"C:/Winnt/foo/bar")
So I imagine open() or something converts ":" to "!" (valid character as
file name).
After I applied following experimental patch, test_mailbox.py run
successfully on windows.
Index: Lib/mailbox.py
===
--- Lib/mailbox.py (revision 60233)
+++ Lib/mailbox.py (working copy)
@@ -223,7 +223,8 @@
class Maildir(Mailbox):
"""A qmail-style Maildir mailbox."""
-colon = ':'
+# colon = ':'
+colon = "!"
def __init__(self, dirname, factory=rfc822.Message, create=True):
"""Initialize a Maildir instance."""
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5.2 release coming up
Is threre any chance to fix this bug before releasing 2.5.2? http://bugs.python.org/issue1736 It contains potential buffer overrun, I think this is somewhat important. If multibyte support (CharNext) is not needed, I 'll rewrite the patch gracefully. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Investigated ref leak report related to thread (regrtest.py -R ::)
> Please post a bug report to SF and report the bug number here. When you > post bugs only to the list they get lost. > -- > Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ > > "as long as we like the same operating system, things are cool." --piranha Thank you for pointing it out. Done. http://www.python.org/sf/1739118 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Investigated ref leak report related to thread (regrtest.py -R ::)
Hello. I investigated ref leak report related to thread.
Please run python regrtest.py -R :: test_leak.py (attached file)
Sometimes ref leak is reported.
# I saw this as regression failure on python-checkins.
# total ref count 92578 -> 92669
_Condition 2
Thread 6
_Event 1
bool 10
instancemethod 1
code 2
dict 9
file 1
frame 3
function 2
int 1
list 2
builtin_function_or_method 5
NoneType 2
str 27
thread.lock 7
tuple 5
type 5
Probably this happens because threading.Thread is implemented as Python
code,
(expecially threading.Thread#join), the code of regrtest.py
if i >= nwarmup:
deltas.append(sys.gettotalrefcount() - rc - 2)
can run before thread really quits. (before Moudles/threadmodule.c
t_bootstrap()'s
Py_DECREF(boot->func);
Py_DECREF(boot->args);
Py_XDECREF(boot->keyw);
runs)
So I experimentally inserted the code to wait for thread termination.
(attached file experimental.patch) And I confirmed error was gone.
# Sorry for hackish patch which only runs on windows. It should run
# on other platforms if you replace Sleep() in Python/sysmodule.c
# sys_debug_ref_leak_leave() with appropriate function.
import threading
import time
import unittest
from test import test_support
class Thread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self._stop = False
self.ready = threading.Event()
def stop(self):
self._stop = True
self.join()
def run(self):
self.ready.set()
while not self._stop:
time.sleep(0.5)
class LeakTests(unittest.TestCase):
def test_leak(self):
def foo():
raise RuntimeError("foo")
self.assertRaises(RuntimeError, foo)
def setUp(self):
self._thread = Thread()
self._thread.start()
self._thread.ready.wait()
def tearDown(self):
self._thread.stop()
def test_main():
test_support.run_unittest(LeakTests)
if __name__ == "__main__":
test_main()
experimental.patch
Description: Binary data
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 slower than 2.4 for some things?
> > Probably I found one reason comparation for classic style class is
slower on
> > Python2.5.
> > Comparation function instance_compare() calls
PyErr_GivenExceptionMatches(),
> > and it was just flag operation on 2.4. But on 2.5, probably related to
> > introduction of BaseException,
> > it checks inherited type tuple. (ie: PyExceptionInstance_Check)
>
> I'm curious about the speed of 2.6 (trunk). I think this should have
> become faster due to the introduction of fast subtype checks (he says
> without looking at the code).
>
> n
>
Yes, I confirmed trunk is faster than 2.5.
///
// Code
import timeit
t = timeit.Timer("""
f1 < f2
""", """
class Foo:
pass
f1 = Foo()
f2 = Foo()
""")
print t.timeit(1)
///
// Result
release-maint24 0.337sec
release-maint25 0.625sec
trunk 0.494sec
//
// Result of plex_test2.py
release-maint24 2.944sec
release-maint25 4.026sec
trunk 3.625sec
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 slower than 2.4 for some things?
> Meanwhile I tried to replace the parsing I did with Plex by re.Scanner.
And
> again there is a remarkable speed difference. Again python2.5 is slower:
>
> try:
> from re import Scanner
> except:
> from sre import Scanner
>
> pars = {}
> order = []
> count = 0
>
> def par(scanner,name):
> global count, order, pars
>
> if name in ['caller','e','pi']:
> return name
> if name not in pars.keys():
> pars[name] = ('ns', count)
> order.append(name)
> ret = 'a[%d]'%count
> count += 1
> else:
> ret = 'a[%d]'%(order.index(name))
> return ret
>
> scanner = Scanner([
> (r"x", lambda y,x: x),
> (r"[a-zA-Z]+\.", lambda y,x: x),
> (r"[a-z]+\(", lambda y,x: x),
> (r"[a-zA-Z_]\w*", par),
> (r"\d+\.\d*", lambda y,x: x),
> (r"\d+", lambda y,x: x),
> (r"\+|-|\*|/", lambda y,x: x),
> (r"\s+", None),
> (r"\)+", lambda y,x: x),
> (r"\(+", lambda y,x: x),
> (r",", lambda y,x: x),
> ])
>
> import profile
> import pstats
>
> def run():
> arg = '+amp*exp(-(x-pos)/fwhm)'
> for i in range(100):
> scanner.scan(arg)
>
> profile.run('run()','profscanner')
> p = pstats.Stats('profscanner')
> p.strip_dirs()
> p.sort_stats('cumulative')
> p.print_stats()
Well, I tried this script, there was no big difference.
Python2.4 0.772sec
Python2.5 0.816sec
Probably I found one reason comparation for classic style class is slower on
Python2.5.
Comparation function instance_compare() calls PyErr_GivenExceptionMatches(),
and it was just flag operation on 2.4. But on 2.5, probably related to
introduction of BaseException,
it checks inherited type tuple. (ie: PyExceptionInstance_Check)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 slower than 2.4 for some things?
> I've had a report from a user that Plex runs about half
> as fast in 2.5 as it did in 2.4. In particular, the
> NFA-to-DFA conversion phase, which does a lot of
> messing about with dicts representing mappings between
> sets of states.
>
> Does anyone in the Ministry for Making Python Blazingly
> fast happen to know of some change that might have
> pessimised things in this area?
Hello, I investigated. On my environment, consumed time is
E:\Plex-1.1.5>py24 plex_test2.py
0.71065668
E:\Plex-1.1.5>py25 plex_test2.py
0.92131335
And after I applied this patch to Plex/Machines, (make `Node' new style
class)
62c62
< class Node:
---
> class Node(object):
E:\Plex-1.1.5>py24 plex_test2.py
0.40122888
E:\Plex-1.1.5>py25 plex_test2.py
0.350999832153
So, probably hash, comparation mechanizm of old/new style class has changed.
# improved for new style class, worse for old style class. Maybe optimized
for new style class?
Try this for minimum test.
import timeit
init = """
class Class:
pass
c1 = Class()
c2 = Class()
"""
t1 = timeit.Timer("""
c1 < c2
""", init)
t2 = timeit.Timer("""
hash(c1)
hash(c2)
""", init)
print t1.timeit(1000)
print t2.timeit(1000)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] python/trunk/Lib/test/test_urllib.py (for ftpwrapper)
# Sorry, I posted to inapropreate mailing list. (Python-3000)
http://mail.python.org/pipermail/python-checkins/2007-May/060507.html
Hello. I'm using Windows2000, I tried some investigation for
test_ftpwrapper.
After I did this change, most errors were gone.
Index: Lib/urllib.py
===
--- Lib/urllib.py (revision 55584)
+++ Lib/urllib.py (working copy)
@@ -833,7 +833,7 @@
self.busy = 0
self.ftp = ftplib.FTP()
self.ftp.connect(self.host, self.port, self.timeout)
-self.ftp.login(self.user, self.passwd)
+#self.ftp.login(self.user, self.passwd)
for dir in self.dirs:
self.ftp.cwd(dir)
I don't know, but probably 'login' on Win2000 is problamatic.
Remaining error is:
File "e:\python-dev\trunk\lib\threading.py", line 460, in __bootstrap
self.run()
File "e:\python-dev\trunk\lib\threading.py", line 440, in run
self.__target(*self.__args, **self.__kwargs)
File "test_urllib.py", line 565, in server
conn.recv(13)
error: (10035, 'The socket operation could not complete without blocking')
And after commented out conn.recv block in test_urllib.py, test passed fine.
def server(evt):
serv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serv.settimeout(3)
serv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
serv.bind(("", 9093))
serv.listen(5)
try:
conn, addr = serv.accept()
conn.send("1 Hola mundo\n")
"""
cantdata = 0
while cantdata < 13:
data = conn.recv(13-cantdata)
cantdata += len(data)
time.sleep(.3)
"""
conn.send("2 No more lines\n")
conn.close()
except socket.timeout:
pass
finally:
serv.close()
evt.set()
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] VC6 support on release25-maint
Hello. I noticed VisualC++6 support came back. I'm glad with that, but still it seems incomplete. (for example, _sqlite3 support) Maybe does this patch help process? On my machine, testcases other than distutils runs fine. http://sourceforge.net/tracker/?func=detail&aid=1457736&group_id=5470&atid=305470 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
