[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-22 Thread STINNER Victor 

STINNER Victor added the comment:

> Shouldn't this issue be closed now?

Correct, I forgot to close it.

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-22 Thread Antoine Pitrou 

Antoine Pitrou added the comment:

Shouldn't this issue be closed now?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-07 Thread Roundup Robot 

Roundup Robot added the comment:

New changeset 3b316ea5aa82 by Victor Stinner in branch '3.3':
Issue #17137: When an Unicode string is resized, the internal wide character
http://hg.python.org/cpython/rev/3b316ea5aa82

New changeset c10a3ddba483 by Victor Stinner in branch 'default':
(Merge 3.3) Issue #17137: When an Unicode string is resized, the internal wide
http://hg.python.org/cpython/rev/c10a3ddba483

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-07 Thread Antoine Pitrou 

Antoine Pitrou added the comment:

The "import random" isn't needed in your patch.

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-07 Thread STINNER Victor 

Changes by STINNER Victor :


--
nosy: +larry
priority: high -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-07 Thread STINNER Victor 

STINNER Victor added the comment:

@Jan Lachnitt: Thanks for your patience and having executed all my commands :-) 
Thanks for the short script reproducing the issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-07 Thread STINNER Victor 

STINNER Victor added the comment:

Ok, it's a bug in the function resize a compact Unicode string,
resize_compact(): wstr field is not updated to the new size.

Attached patch should fix it. The bug was introduced by me in Python 3.3.

I don't think that it's possible to resize wstr buffer instead of
freeing it: it will not be refilled by PyUnicode_AsUnicodeAndSize() if
wstr is not NULL. An alternative is to create a new string (instead of
using realloc) if wstr is not NULL.

2013/2/7 Florent Xicluna :
>
> Changes by Florent Xicluna :
>
>
> --
> components:  -Windows
>
> ___
> Python tracker 
> 
> ___

--
keywords: +patch
Added file: http://bugs.python.org/file28987/pep393.patch

___
Python tracker 

___diff -r b8a6bc70fc08 Lib/test/test_unicode.py
--- a/Lib/test/test_unicode.py  Thu Feb 07 17:05:32 2013 +0200
+++ b/Lib/test/test_unicode.py  Thu Feb 07 17:15:48 2013 +0100
@@ -7,6 +7,7 @@ Written by Marc-Andre Lemburg (mal@lembu
 """#"
 import _string
 import codecs
+import random
 import struct
 import sys
 import unittest
@@ -2191,6 +2192,20 @@ class UnicodeTest(string_tests.CommonTes
 self.assertEqual(args[0], text)
 self.assertEqual(len(args), 1)
 
+def test_resize(self):
+for length in range(1, 100, 7):
+# generate a fresh string (refcount=1)
+text = 'a' * length + 'b'
+
+# fill wstr internal field
+abc = text.encode('unicode_internal')
+self.assertEqual(abc.decode('unicode_internal'), text)
+
+text += 'c'
+abcdef = text.encode('unicode_internal')
+self.assertNotEqual(abc, abcdef)
+self.assertEqual(abcdef.decode('unicode_internal'), text)
+
 
 class StringModuleTest(unittest.TestCase):
 def test_formatter_parser(self):
diff -r b8a6bc70fc08 Objects/unicodeobject.c
--- a/Objects/unicodeobject.c   Thu Feb 07 17:05:32 2013 +0200
+++ b/Objects/unicodeobject.c   Thu Feb 07 17:15:48 2013 +0100
@@ -717,6 +717,10 @@ resize_compact(PyObject *unicode, Py_ssi
 if (!PyUnicode_IS_ASCII(unicode))
 _PyUnicode_WSTR_LENGTH(unicode) = length;
 }
+else if (_PyUnicode_HAS_WSTR_MEMORY(unicode)) {
+PyObject_DEL(_PyUnicode_WSTR(unicode));
+_PyUnicode_WSTR(unicode) = NULL;
+}
 #ifdef Py_DEBUG
 unicode_fill_invalid(unicode, old_length);
 #endif
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-07 Thread Florent Xicluna 

Changes by Florent Xicluna :


--
components:  -Windows

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-07 Thread Jan Lachnitt 

Jan Lachnitt added the comment:

...
print(ctypes.sizeof(ctypes.c_wchar))
_PyObject_Dump=ctypes.pythonapi._PyObject_Dump
_PyObject_Dump.argtypes=(ctypes.py_object,)
print(_PyObject_Dump(dirname))
print(list(dirname))

in Python 3.3.0 64-bit on Windows 8 produces:

2
object  : 'D:\\Bug reports\\Python\\test\\sub-fcc\\bulk'
type: str
refcount: 3
address : 028AC298
54
['D', ':', '\\', 'B', 'u', 'g', ' ', 'r', 'e', 'p', 'o', 'r', 't', 's', '\\', 
'P', 'y', 't', 'h', 'o', 'n', '\\', 't', 'e', 's', 't', '\\', 's', 'u', 'b', 
'-', 'f', 'c', 'c', '\\', 'b', 'u', 'l', 'k']

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-07 Thread Florent Xicluna 

Florent Xicluna added the comment:

Confirmed on OSX 64bits with Mark's sample.

$ python3.3
Python 3.3.0 (default, Jan 24 2013, 08:28:09) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> dir1 = "D:\\Bug reports\\Python\\test\\sub-fcc"
>>> dir1 += "\\bulk"
>>> s1 = ascii(dir1.encode('unicode_internal'))
>>> dir1 += "\\bulk"
>>> s1 = ascii(dir1.encode('unicode_internal'))
>>> len(s1), s1[499:]
(586, 
"00k\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\xf0\\xbda\\x00\\x01\\x00\\x00\\x00X\\x1da\\x00\\x01\\x00\\x00\\x00'")
>>> dir1 = "D:\\Bug reports\\Python\\test\\sub-fcc"
>>> dir1 += "\\bulk"
>>> s1 = ascii(dir1.encode('unicode_internal'))
>>> dir1 += "\\bulk"
>>> s1 = ascii(dir1.encode('unicode_internal'))
>>> len(s1), s1[499:]
(586, 
"00k\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x10\\xbca\\x00\\x01\\x00\\x00\\x00X\\x16a\\x00\\x01\\x00\\x00\\x00'")
>>> dir1 = "D:\\Bug reports\\Python\\test\\sub-fcc"
>>> dir1 += "\\bulk"
>>> s1 = ascii(dir1.encode('unicode_internal'))
>>> dir1 += "\\bulk"
>>> s1 = ascii(dir1.encode('unicode_internal'))
>>> len(s1), s1[499:]
(595, 
"00k\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00'")
>>> dir1 = "D:\\Bug reports\\Python\\test\\sub-fcc"
>>> dir1 += "\\bulk"
>>> s1 = ascii(dir1.encode('unicode_internal'))
>>> dir1 += "\\bulk"
>>> s1 = ascii(dir1.encode('unicode_internal'))
>>> len(s1), s1[499:]
(586, 
"00k\\x00\\x00\\x00\\x00\\x00\\x00\\x00p\\xbba\\x00\\x01\\x00\\x00\\x00\\x88\\x14a\\x00\\x01\\x00\\x00\\x00'")
>>> 


Darwin Kernel Version 10.8.0: Tue Jun  7 16:32:41 PDT 2011; 
root:xnu-1504.15.3~1/RELEASE_X86_64 x86_64

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-07 Thread Jan Lachnitt 

Jan Lachnitt added the comment:

On Windows XP 32-bit: 3.2.3 works, 3.3.0 fails.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-07 Thread Mark Dickinson 

Mark Dickinson added the comment:

I don't think this is just windows;  I see similarly odd results on OS X.  The 
first encode call gives expected results;  the second ends in garbage.


Python 3.4.0a0 (default:eb0370d4686c+, Feb  7 2013, 14:59:41) 
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> dir1 = "D:\\Bug reports\\Python\\test\\sub-fcc"
[66291 refs, 23475 blocks]
>>> dir1 += "\\bulk"
[66291 refs, 23474 blocks]
>>> ascii(dir1.encode('unicode_internal'))
"b'D\\x00\\x00\\x00:\\x00\\x00\\x00\\x00\\x00\\x00B\\x00\\x00\\x00u\\x00\\x00\\x00g\\x00\\x00\\x00
 
\\x00\\x00\\x00r\\x00\\x00\\x00e\\x00\\x00\\x00p\\x00\\x00\\x00o\\x00\\x00\\x00r\\x00\\x00\\x00t\\x00\\x00\\x00s\\x00\\x00\\x00\\x00\\x00\\x00P\\x00\\x00\\x00y\\x00\\x00\\x00t\\x00\\x00\\x00h\\x00\\x00\\x00o\\x00\\x00\\x00n\\x00\\x00\\x00\\x00\\x00\\x00t\\x00\\x00\\x00e\\x00\\x00\\x00s\\x00\\x00\\x00t\\x00\\x00\\x00\\x00\\x00\\x00s\\x00\\x00\\x00u\\x00\\x00\\x00b\\x00\\x00\\x00-\\x00\\x00\\x00f\\x00\\x00\\x00c\\x00\\x00\\x00c\\x00\\x00\\x00\\x00\\x00\\x00b\\x00\\x00\\x00u\\x00\\x00\\x00l\\x00\\x00\\x00k\\x00\\x00\\x00'"
[69015 refs, 24925 blocks]
>>> dir1 += "\\bulk"
[69015 refs, 24925 blocks]
>>> ascii(dir1.encode('unicode_internal'))
"b'D\\x00\\x00\\x00:\\x00\\x00\\x00\\x00\\x00\\x00B\\x00\\x00\\x00u\\x00\\x00\\x00g\\x00\\x00\\x00
 
\\x00\\x00\\x00r\\x00\\x00\\x00e\\x00\\x00\\x00p\\x00\\x00\\x00o\\x00\\x00\\x00r\\x00\\x00\\x00t\\x00\\x00\\x00s\\x00\\x00\\x00\\x00\\x00\\x00P\\x00\\x00\\x00y\\x00\\x00\\x00t\\x00\\x00\\x00h\\x00\\x00\\x00o\\x00\\x00\\x00n\\x00\\x00\\x00\\x00\\x00\\x00t\\x00\\x00\\x00e\\x00\\x00\\x00s\\x00\\x00\\x00t\\x00\\x00\\x00\\x00\\x00\\x00s\\x00\\x00\\x00u\\x00\\x00\\x00b\\x00\\x00\\x00-\\x00\\x00\\x00f\\x00\\x00\\x00c\\x00\\x00\\x00c\\x00\\x00\\x00\\x00\\x00\\x00b\\x00\\x00\\x00u\\x00\\x00\\x00l\\x00\\x00\\x00k\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\xfb\\xfb\\xfb\\xfb\\xfb\\xfb\\xfb\\xfb\\x00\\x00\\x00\\x00\\x00\\x015\\x1a'"
[69015 refs, 24925 blocks]

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-07 Thread STINNER Victor 

STINNER Victor added the comment:

Can you try to following command to get the size in bytes of the wchar_t type?

>>> import types
>>> ctypes.sizeof(ctypes.c_wchar)
4

You can also use _PyObject_Dump() to dump your string:

>>> import ctypes
>>> x="abc"
>>> _PyObject_Dump=ctypes.pythonapi._PyObject_Dump
>>> _PyObject_Dump.argtypes=(ctypes.py_object,)
>>> _PyObject_Dump(x)
object  : 'abc'
type: str
refcount: 5
address : 0xb70bf980
48

Then you can use _PyObject_Dump() on your string.

You may also try: print(list(dirname)).

It's really strange that something very common like string concatenation 
returns an invalid string.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-07 Thread STINNER Victor 

STINNER Victor added the comment:

> It works correctly in Python 3.2.3 64-bit on Windows 8.

Can you reproduce the issue on other Windows versions?

2013/2/7 Jan Lachnitt :
>
> Jan Lachnitt added the comment:
>
> Knowing that the problem is related to the internal representation of the 
> strings, I have written a short script which reproduces the problem. It is 
> this simple:
>
> import os
> name = 'sub-fcc'
> wrkdir = 'D:\\Bug reports\\Python\\test'
> dirname = wrkdir+os.sep+name
> print(dirname)
> print(ascii(dirname.encode("unicode_internal")))
> dirname += os.sep+'bulk'
> print(dirname)
> print(ascii(dirname.encode("unicode_internal")))
>
> Output:
>
> D:\Bug reports\Python\test\sub-fcc
> b'D\x00:\x00\\\x00B\x00u\x00g\x00 
> \x00r\x00e\x00p\x00o\x00r\x00t\x00s\x00\\\x00P\x00y\x00t\x00h\x00o\x00n\x00\\\x00t\x00e\x00s\x00t\x00\\\x00s\x00u\x00b\x00-\x00f\x00c\x00c\x00'
> D:\Bug reports\Python\test\sub-fcc\bulk
> b'D\x00:\x00\\\x00B\x00u\x00g\x00 
> \x00r\x00e\x00p\x00o\x00r\x00t\x00s\x00\\\x00P\x00y\x00t\x00h\x00o\x00n\x00\\\x00t\x00e\x00s\x00t\x00\\\x00s\x00u\x00b\x00-\x00f\x00c\x00c\x00\x00\x00\x00\x00\xd8\xa3\x90\x02\x00\x00'
>
> The end of the output varies from run to run.
>
> It works correctly in Python 3.2.3 64-bit on Windows 8.
>
> --
>
> ___
> Python tracker 
> 
> ___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-07 Thread Jan Lachnitt 

Jan Lachnitt added the comment:

Knowing that the problem is related to the internal representation of the 
strings, I have written a short script which reproduces the problem. It is this 
simple:

import os
name = 'sub-fcc'
wrkdir = 'D:\\Bug reports\\Python\\test'
dirname = wrkdir+os.sep+name
print(dirname)
print(ascii(dirname.encode("unicode_internal")))
dirname += os.sep+'bulk'
print(dirname)
print(ascii(dirname.encode("unicode_internal")))

Output:

D:\Bug reports\Python\test\sub-fcc
b'D\x00:\x00\\\x00B\x00u\x00g\x00 
\x00r\x00e\x00p\x00o\x00r\x00t\x00s\x00\\\x00P\x00y\x00t\x00h\x00o\x00n\x00\\\x00t\x00e\x00s\x00t\x00\\\x00s\x00u\x00b\x00-\x00f\x00c\x00c\x00'
D:\Bug reports\Python\test\sub-fcc\bulk
b'D\x00:\x00\\\x00B\x00u\x00g\x00 
\x00r\x00e\x00p\x00o\x00r\x00t\x00s\x00\\\x00P\x00y\x00t\x00h\x00o\x00n\x00\\\x00t\x00e\x00s\x00t\x00\\\x00s\x00u\x00b\x00-\x00f\x00c\x00c\x00\x00\x00\x00\x00\xd8\xa3\x90\x02\x00\x00'

The end of the output varies from run to run.

It works correctly in Python 3.2.3 64-bit on Windows 8.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-06 Thread STINNER Victor 

STINNER Victor added the comment:

It would really help if you can write a short script reproducing the problem. 
Can you reproduce the problem with Python 3.2 on Windows 8, or with Python 3.3 
on Windows XP or 7?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-06 Thread Jan Lachnitt 

Jan Lachnitt added the comment:

print(ascii(struct.name))
print(ascii(struct.name.encode("unicode_internal")))
print(ascii(struct.name.encode("utf-8")))

produces:

'sub-fcc'
b's\x00u\x00b\x00-\x00f\x00c\x00c\x00'
b'sub-fcc'

and that looks correct.

struct.name originally comes from an ini-file:

cp = configparser.ConfigParser(interpolation=None)
try:
cp.read(filename)
...

The ini-file is encoded in pure ASCII (while my Python sources are in UTF-8 
with the identification bytes at the beginning of the file). struct.name is the 
name of a section in this file, as provided by cp.sections() . The name gets 
through several objects. I am not pasting all the relevant code pieces here 
because there are too many relevant pieces but they do nothing special (just 
passing and copying the name). I do not use ctypes.

wrkdir is generated from inp_file_name, which is 'default.ini', by this 
statement:

wrkdir = os.path.splitext(os.path.abspath(inp_file_name))[0]

BTW, ascii(dirname.encode("unicode_internal")) result is different in this run:

b'D\x00:\x00\\\x00B\x00u\x00g\x00 
\x00r\x00e\x00p\x00o\x00r\x00t\x00s\x00\\\x00P\x00y\x00t\x00h\x00o\x00n\x00\\\x00A\x00u\x00t\x00o\x00L\x00E\x00E\x00D\x00\\\x00d\x00e\x00f\x00a\x00u\x00l\x00t\x00\\\x00s\x00u\x00b\x00-\x00f\x00c\x00c\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-06 Thread STINNER Victor 

STINNER Victor added the comment:

I'm interested by your struct.name string: can you also dump it? Where
does it come from? Does it use ctypes?

 * print(ascii(struct.name))
 * print(ascii(struct.name.encode("unicode_internal")))
 * print(ascii(struct.name.encode("utf-8")))

I'm interested by all variables used to build the final path.

nt._isdir() doesn't check if the path contains a NUL character. It
should: see aksi #13617

> b'D\x00:\x00\\\x00B\x00u\x00g\x00 
> \x00r\x00e\x00p\x00o\x00r\x00t\x00s\x00\\\x00P\x00y\x00t\x00h\x00o\x00n\x00\\\x00A\x00u\x00t\x00o\x00L\x00E\x00E\x00D\x00\\\x00d\x00e\x00f\x00a\x00u\x00l\x00t\x00\\\x00s\x00u\x00b\x00-\x00f\x00c\x00c\x00\x00\x002\x00\x03\x00\x00\x00\x00\x00'

Decoded from UTF-16-LE, it gives:
'D:\\Bug reports\\Python\\AutoLEED\\default\\sub-fcc\x002\x03\x00\x00'

> b'D:\\Bug reports\\Python\\AutoLEED\\default\\sub-fcc\\bulk'

Decode from UTF-8, it gives:
'D:\\Bug reports\\Python\\AutoLEED\\default\\sub-fcc\\bulk'

It looks like the wstr representation of the string is corrupted.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-06 Thread Jan Lachnitt 

Jan Lachnitt added the comment:

Here is a part of my code (with some comments added):

for struct in ini_structures:
dirname = wrkdir+os.sep+struct.name
if not os.path.isdir(dirname):  # This works fine. If the directory 
doesn't exist,...
try:
os.mkdir(dirname)   # ... it is created here.
except OSError:
raise AutoLEEDError('Cannot create directory "'+dirname+'".')
dirname += os.sep+'bulk'# This defines a subdirectory.
if not os.path.isdir(dirname): ## Though it doesn't exist, 
os.path.isdir returns True,...
try:
os.mkdir(dirname)   # ... so it is not created here.
except OSError:
raise AutoLEEDError('Cannot create directory "'+dirname+'".')
fn = dirname+os.sep+'cluster.i' # This defines a filename.
print('Writing file "'+fn+'"...')
straos = struct.write_bulk_cluster(fn,at_rad) # Here it fails (cannot 
write to file).

According to Victor's post, I have inserted these lines before the line marked 
## (and added necessary imports):

print('dirname =', dirname)
print('os.path.isdir(dirname) =', os.path.isdir(dirname))
print('nt._isdir(dirname) =', nt._isdir(dirname))
print('stat.S_ISDIR(os.stat(dirname).st_mode) =', 
stat.S_ISDIR(os.stat(dirname).st_mode))
print(ascii(dirname.encode("unicode_internal")))
print(ascii(dirname.encode("utf-8")))

Here is the output of these lines (that directory really does not exist but its 
parent directory does):

dirname = D:\Bug reports\Python\AutoLEED\default\sub-fcc\bulk
os.path.isdir(dirname) = True
nt._isdir(dirname) = True
stat.S_ISDIR(os.stat(dirname).st_mode) = True
b'D\x00:\x00\\\x00B\x00u\x00g\x00 
\x00r\x00e\x00p\x00o\x00r\x00t\x00s\x00\\\x00P\x00y\x00t\x00h\x00o\x00n\x00\\\x00A\x00u\x00t\x00o\x00L\x00E\x00E\x00D\x00\\\x00d\x00e\x00f\x00a\x00u\x00l\x00t\x00\\\x00s\x00u\x00b\x00-\x00f\x00c\x00c\x00\x00\x002\x00\x03\x00\x00\x00\x00\x00'
b'D:\\Bug reports\\Python\\AutoLEED\\default\\sub-fcc\\bulk'

Yeah, the result of ascii(dirname.encode("unicode_internal")) seems to be wrong 
(at the end).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-05 Thread STINNER Victor 

STINNER Victor added the comment:

On Windows, os.path.isdir calls nt._isdir(). Core of this C function:

wchar_t *wpath = PyUnicode_AsUnicode(po);
if (wpath == NULL)
return NULL;

attributes = GetFileAttributesW(wpath);
if (attributes == INVALID_FILE_ATTRIBUTES)
Py_RETURN_FALSE;
...

Can you please try to call directly nt._isdir()? Can also also compare with 
stat.S_ISDIR(os.stat(fn).st_mode)?

If the problem is something with the implementation of Unicode, it would be 
interesting to try to get the content of the string using:

 * print(ascii(path.encode("unicode_internal"))) # should be result of 
PyUnicode_AsUnicode() which is cached
 * print(ascii(path.encode("utf-8")))

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-05 Thread STINNER Victor 

Changes by STINNER Victor :


--
components: +Unicode, Windows
nosy: +ezio.melotti
versions: +Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-05 Thread Georg Brandl 

Changes by Georg Brandl :


--
priority: normal -> high

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-05 Thread Florent Xicluna 

Changes by Florent Xicluna :


--
keywords: +3.3regression
nosy: +flox, haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-05 Thread Georg Brandl 

Changes by Georg Brandl :


--
nosy: +loewis

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-05 Thread Georg Brandl 

Georg Brandl added the comment:

The SO post is scary.  Maybe a non-normalized (smallest representation) PEP393 
string is escaping into the wild?

--
nosy: +georg.brandl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-05 Thread Mark Dickinson 

Mark Dickinson added the comment:

Reminds me of this question on StackOverflow:

http://stackoverflow.com/questions/14135846/string-concatenation-with-python-33-isdir-always-returns-true-3-hours-head

--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17137] Malfunctioning compiled code in Python 3.3 x64

2013-02-05 Thread Jan Lachnitt 

New submission from Jan Lachnitt:

Python 3.3 64-bit seems to compile one of my files incorrectly. Specifically, 
os.path.isdir returns True for a nonexistent folder. The important point is 
that the code works correctly when it is performed step-by-step in pdb.

Python version:
Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:57:17) [MSC v.1600 64 bit 
(AMD64)] on win32

OS:
Windows 8 64-bit

The code works fine in Python 3.2.3 32-bit on Windows XP.

My project is quite complex and it interacts with other software packages. I 
tried to make a reduced test-case but I could not reproduce the problem this 
way. What files do you need for processing this bug report? Will e.g. the 
source file in question and the corresponding compiled file (*.pyc) be enough? 
Or should I upload the whole project here, along with the instructions on how 
to run it?

--
messages: 181465
nosy: pepalogik
priority: normal
severity: normal
status: open
title: Malfunctioning compiled code in Python 3.3 x64
type: behavior
versions: Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com