[issue18843] Py_FatalError (msg=0x7f0e3b373232 "bad leading pad byte") at Python-2.7.5/Python/pythonrun.c:1689

[Charles-François Natali]

Charles-François Natali added the comment:

Just two things:
- running under valgrind with the suppression file would help pinpoint this
- usually, stack/heap overflows or invalid pointer dereference affect a 
contiguous chunk of memory: here, there's a *single bit* flipped, not even a 
byte
0xfb == 0b1011
0xfa == 0b1010

This looks like an hardware issue to me.

--
nosy: +neologix

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14937] IDLE's deficiency in the completion of file names (Python 32, Windows XP)

[Westley Martínez]

Westley Martínez added the comment:

On Thu, Oct 11, 2012 at 3:06 PM, Terry J. Reedy  wrote:
>
> Terry J. Reedy added the comment:
>
> This patch (I suspect it is this one) disabled the use of '/' in filenames on 
> windows when using filename completion.
>
> 'c:\in 3.2.3 and 3.3.0>
> (If there is no 'r' prefix, it really should require '\\' to be safe.)

+1 for requiring \\.

I'll test this tomorrow and report back the behaviour.  On Linux it
seems that the window only pops up when you press tab.  I think
behaviour should be as identical on all platforms as possible.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13951] Document that Seg Fault in .so called by ctypes causes the interpreter to Seg Fault

[Westley Martínez]

Westley Martínez added the comment:

Can we have this committed?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1500504] Alternate RFC 3986 compliant URI parsing module

[Martin Panter]

Changes by Martin Panter :


--
nosy: +vadmium

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18828] urljoin behaves differently with custom and standard schemas

[Martin Panter]

Martin Panter added the comment:

Similarly, I expected this to return "rtmp://host/app?auth=token":

urljoin("rtmp://host/app", "?auth=token")

I'm not sure adding everybody's custom scheme to a hard-coded whitelist is the 
best way to do solve this.

Below I have identified some other schemes not in the "uses_relative" list. Is 
there any reason why one would use urljoin() with them, but want the base URL 
to be ignored (as is the current behaviour)? I looked at test_urlparse.py and 
there doesn't seem to be any test cases for these schemes.

>>> all = set().union(uses_relative, uses_netloc, uses_params, 
>>> non_hierarchical, uses_query, uses_fragment)
>>> sorted(all.difference(uses_relative))
['git', 'git+ssh', 'hdl', 'mailto', 'news', 'nfs', 'rsync', 'sip', 'sips', 
'snews', 'tel', 'telnet']

Even if the behaviour can't be changed, could the documentation for urljoin() 
say something like this:

Only the following [uses_relative] schemes are allowed in the base URL; any 
other schemes result in the relative URL being returned without being joined to 
the base.

--
nosy: +vadmium

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18843] Py_FatalError (msg=0x7f0e3b373232 "bad leading pad byte") at Python-2.7.5/Python/pythonrun.c:1689

[Tim Peters]

Tim Peters added the comment:

Martin, can you please supply exact commands Stephen can use to try to 
reproduce the problem you saw running `emerge`?  And try them yourself first, 
to make sure you can reproduce the problem too.

Any detail may be important.  For example, it's possible this is due to a bug 
in the C compiler used to compile Python, or due to some flag passed to the 
compiler.  That's why "exact commands" are important.

The good news:  if Stephen can reproduce it, he'll probably figure out all the 
rest, and you'll be free to resume enjoying your life ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18843] Py_FatalError (msg=0x7f0e3b373232 "bad leading pad byte") at Python-2.7.5/Python/pythonrun.c:1689

[Tim Peters]

Tim Peters added the comment:

Thanks for chiming in, Stephen!  I can't answer your questions, but agree the 
original poster needs to tell you exactly what he did -- else we'd just be 
thrashing at random.  There's been enough of that already ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18828] urljoin behaves differently with custom and standard schemas

[Madison May]

Changes by Madison May :


--
components: +Library (Lib)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18850] xml.etree.ElementTree accepts control chars.

[Stefan Behnel]

Stefan Behnel added the comment:

> As an advice I hope you do not take as insult, saying
> "in section {section} the spec says {argument}" 
> is much more constructive than 
> "read the spec on that", "{extremely_obvious_link}",
> at least to people not familiar with the spec and asking for the source > of 
> your arguments (msg196360). Can shorten threads, too.

No harm done. The reason why I just posted the spec URL is that it's actually 
the entire spec that backs the argument. XML is (essentially) specified as a 
mapping from a sequence of bytes to a hierarchical structure (and back again). 
That's why there is an XML declaration header that names the encoding, for 
example. It wouldn't be needed if XML was defined as Unicode data.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18843] Py_FatalError (msg=0x7f0e3b373232 "bad leading pad byte") at Python-2.7.5/Python/pythonrun.c:1689

[Stephen J. Turnbull]

Stephen J. Turnbull added the comment:

I have a gentoo host, but I'd like to know how did the OP get a debug Python in 
the first place?  The ebuild for python 2.7.5-r1 doesn't say anything about 
debug options.  "How" would preferably include information about the C compiler 
used, etc.  If there's no information, I can probably just hack --with-pydebug 
into a local ebuild, but I have something like 7 C compilers installed, I'd 
really like a good guess at the right one.  Also, did he add any optimization 
flags etc when building the debug Python?

(ebuild = the emerge configuration file that describes the build and install 
process for a package.  In "2.7.5-r1" the "r1" indicates the first revised 
ebuild for the same upstream version.)

--
nosy: +sjt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18550] internal_setblocking() doesn't check return value of fcntl()

[Madison May]

Madison May added the comment:

The attached patch?  :)

--
nosy: +madison.may

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18550] internal_setblocking() doesn't check return value of fcntl()

[Andrew Svetlov]

Changes by Andrew Svetlov :


--
nosy: +asvetlov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable

[Meador Inge]

Meador Inge added the comment:

'tempfile.NamedTemporaryFile' returns an instance of '_TemporaryFileWrapper' 
that wraps the created file object.  The wrapper object implements 
'__getattr__' and we end up with a situation like the following simplified 
example where the wrapper gets destroyed before the looked up method is called:

$ cat test.py 
class S(object):
   def __init__(self):
  print("S.__init__")   def __del__(self):  print("S.__del__")
   def foo(self):
  print("S.foo")

class T(object):
   def __init__(self, s):
  self.s = s
  print("T.__init__")
   def __getattr__(self, name):
s = self.__dict__['s']
a = getattr(s, name)
print("__getattr__('%s')" % name)
if not isinstance(a, int):
setattr(self, name, a)
return a
   def __del__(self):
  print("T.__del__")
T(S()).foo()
$ ./python test.py 
S.__init__
T.__init__
__getattr__('foo')
T.__del__
S.foo
S.__del__

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18489] IDLE Unit test for SearchEngine.py

[Terry J. Reedy]

Terry J. Reedy added the comment:

Adding
engine.patvar.set(None)
Equal(search(text), None)

to test_search completed the coverage.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18489] IDLE Unit test for SearchEngine.py

[Terry J. Reedy]

Terry J. Reedy added the comment:

The attached patch works on 3.3 and just need a small addition for 100$ 
coverage. After that, I intend to commit after porting to 2.7 (only two lines 
to change, I think) and testing it there.

I split the search test case into three, one for each method. The tests for 
search_text repeated the tests for search_forward/backward. Simple mocks for 
the latter two allow to test for what search_text actually does: pick the right 
function to call and right starting point.

The effect of the ok parameter is complicated because it has different effects 
in search_text and the other two (and less effect in search_backward than in 
search_forward because they are not symmetric).

'\w+' never matches an empty string; so there was no test of 'ok=true' in 
search_forward. '\w*' can have empty matches and there now is a test. I 
discovered what seems to me a bug. If an initial null match is ignored, the 
search skips the rest of the line and skips to the next. This seems wrong but I 
am not sure what the proper behavior should be. I want to investigate the use 
of ok in the dialogs, especially the replace dialog before deciding

The test of search_forward uses the mock Text. The remaining gui uses here (and 
many in other tests) could be eliminated with just a partial implementation of 
marks and selections, one that kept them fixed rather than moving them with 
insert and delete. So far we have been inserting text just once and then 
setting and getting things on a fixed text.

--
stage: patch review -> commit review
Added file: http://bugs.python.org/file31516/18489-test_searchengine6.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable

[R. David Murray]

Changes by R. David Murray :


--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16296] Patch to fix building on Win32/64 under VS 2010

[Trent Nelson]

Changes by Trent Nelson :


--
versions:  -Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16296] Patch to fix building on Win32/64 under VS 2010

[Trent Nelson]

Changes by Trent Nelson :


--
nosy: +trent

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable

[Meador Inge]

Meador Inge added the comment:

Confirmed in trunk:

>>> tempfile.NamedTemporaryFile(dir=".",delete=False).write(b"hello")
Traceback (most recent call last):
  File "", line 1, in 
ValueError: write to closed file

--
nosy: +meador.inge

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18843] Py_FatalError (msg=0x7f0e3b373232 "bad leading pad byte") at Python-2.7.5/Python/pythonrun.c:1689

[Tim Peters]

Tim Peters added the comment:

I sent a msg to Python-Dev, asking for a Gentoo user to try to reproduce this.  
Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18881] Can someone try to duplicate corruption on Gentoo?

[Tim Peters]

Changes by Tim Peters :


--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18881] Can someone try to duplicate corruption on Gentoo?

[Tim Peters]

Changes by Tim Peters :


--
stage:  -> committed/rejected

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18881] Can someone try to duplicate corruption on Gentoo?

[Tim Peters]

Changes by Tim Peters :


--
resolution:  -> invalid

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18881] Can someone try to duplicate corruption on Gentoo?

[Tim Peters]

New submission from Tim Peters:

In

http://bugs.python.org/issue18843

a user reported a debug PyMalloc "bad leading pad byte" memory
corruption death while running their code.  After some thrashing, they
decided to rebuild Python, and got the same kind of error while
rebuilding Python.  See

http://bugs.python.org/msg196481

in that bug report:

"""
# emerge dev-lang/python:2.7

 * IMPORTANT: 11 news items need reading for repository 'gentoo'.
 * Use eselect news to read news items.

Calculating dependencies... done!
Debug memory block at address p=0xa7f5900: API 'o'
80 bytes originally requested
The 7 pad bytes at p-7 are not all FORBIDDENBYTE (0xfb):
at p-7: 0xfb
at p-6: 0xfb
at p-5: 0xfa *** OUCH
at p-4: 0xfb
at p-3: 0xfb
at p-2: 0xfb
at p-1: 0xfb
Because memory is corrupted at the start, the count of bytes requested
   may be bogus, and checking the trailing pad bytes may segfault.
The 8 pad bytes at tail=0xa7f5950 are FORBIDDENBYTE, as expected.
The block was made by call #21242094 to debug malloc/realloc.
Data at p: 73 00 00 00 79 00 00 00 ... 67 00 00 00 00 00 00 00
Fatal Python error: bad leading pad byte
Aborted (core dumped)
#
"""

I don't have access to Gentoo, and don't know squat about its
`emerge`, but if someone else can do this it might help ;-)  The
Python used to run `emerge` here was a --with-pydebug Python the bug
reporter built earlier.

--
messages: 196502
nosy: tim.peters
priority: normal
severity: normal
status: open
title: Can someone try to duplicate corruption on Gentoo?

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable

[Nick Coghlan]

Nick Coghlan added the comment:

The method call should keep the file object alive until it completes (due to 
the self reference). What behaviour prompted the conclusion that it is being 
closed early?

If the symptom is that the data isn't being written to disk, we may have a 
missing flush() call somewhere causing a race condition.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18843] Py_FatalError (msg=0x7f0e3b373232 "bad leading pad byte") at Python-2.7.5/Python/pythonrun.c:1689

[Martin Mokrejs]

Martin Mokrejs added the comment:

http://www.gentoo.org/proj/en/portage/index.xml
http://dev.gentoo.org/~zmedico/portage/archives

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18843] Py_FatalError (msg=0x7f0e3b373232 "bad leading pad byte") at Python-2.7.5/Python/pythonrun.c:1689

[Martin Mokrejs]

Martin Mokrejs added the comment:

That is why I asked if other process can interfere. So, they are isolated on 
Linux, good. ;-) The crash in #msg196481 is just the emerge written in python, 
at the start it is resolving some graph of package dependencies ... once it 
resolves order of packages and prints a lot of output. so, it did not get to 
that before it crashed. In sum with what you said, my application running as 
another process should not interfere with the emerge process, so it turns out 
that was irrelevant comment from my side. ;)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18843] Py_FatalError (msg=0x7f0e3b373232 "bad leading pad byte") at Python-2.7.5/Python/pythonrun.c:1689

[Tim Peters]

Tim Peters added the comment:

It would be a severely lame OS that allowed a process to overwrite another 
process's memory ;-)  "Bad C or C++ code", in the process you're running, is 
still the best guess.

A memory module that sometimes dropped the last bit _could_ be at fault, but I 
doubt it (0xfb changes to 0xfa if the last bit is cleared).

That the address is different across the two failure examples doesn't rule out 
bad memory, though:  the memory address an OS shows you is a "logical address", 
an _offset_ from the start of the physical memory the OS happened to assign to 
the process.  Across different runs, the same physical memory location may be 
shown as any number of different logical addresses.

We'll understand why it's off by 1 eventually ;-)  For example, it could be C 
code as simple as:

char* p = (char *)allocate_some_memory(80);
...
p -= 5;
...
if (something)
--*p;

It's extremely easy for C code to do insane stores; indeed, that's why there 
are so many sophisticated tools for trying to find the source of memory 
corruption in C/C++ code.

Question:  did the last failure come from your code, or from merely running 
`emerge`?  You said your code "was parsing an XML file" at the time, but 
there's no trace of expat (or any other xml parsing code) in the traceback, and 
the traceback "bottoms out" at `emerge`.  Maybe this has nothing at all to do 
with your code

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18843] Py_FatalError (msg=0x7f0e3b373232 "bad leading pad byte") at Python-2.7.5/Python/pythonrun.c:1689

[Martin Mokrejs]

Martin Mokrejs added the comment:

Thanks, will recompile without pymalloc.

I don't understand why always is only 1 bit different. Could that be 
overwritten by another use process or do you believe it must be the python or 
some of the modules imported into it on runtime?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable

[jort bloem]

jort bloem added the comment:

Sorry, my python-foo is not quite up to providing a solution. A partial 
solution would be for the file not to be closed at NamedTemporaryFile 
destruction, if delete==False. If delete==True, then there is no point writing 
to a file (though it shouldn't raise an exception, either).

Alternatively, on a unix based system, the file could be deleted, but left 
open. Read & write could continue as per normal, and the file would be closed 
automatically later.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18876] Problems with files opened in append mode with io module

[Erik Bray]

Erik Bray added the comment:

Here's an initial stab at a simple patch that just addresses the issue of 
'append' not being in the mode string.  Amazingly this did not break a single 
existing test, though I added a new one to test how FileIO objects display 
their mode string after being initialized (normalizes modes like 'r+b' to 
'rb+', etc.)  Most of what it tests is the existing behavior, but it also tests 
for the reintroduction of the 'a' mode.

This doesn't address the BufferedWriter issue, nor does it address 
cross-platform issues.

--
keywords: +patch
Added file: http://bugs.python.org/file31515/issue_18876_patch_1.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18880] ssl.SSLSocket shutdown doesn't behave like socket.shutdown

[Michał Zieliński]

New submission from Michał Zieliński:

SSLSocket documentation mentions shutdown as analogue to socket.shutdown. 
However, instead of forbidding communication, it removes SSL wrapper from 
socket. For example, the following script doesn't work and returns garbage:

import socket
import ssl

s = socket.socket()
s.connect(('google.com', 443))
client = ssl.wrap_socket(s)
client.sendall(b'GET / HTTP/1.0\nConnection: close\n\n')
client.shutdown(socket.SHUT_WR)

print(repr(client.recv(40)))

Attached patch makes shutdown raise exception if how != SHUT_RDWR, as closing 
one side of socket over SSL doesn't make sense (unless I'm missing something).

--
components: Library (Lib)
files: ssl-shutdown-fail.patch
keywords: patch
messages: 196494
nosy: zielmicha
priority: normal
severity: normal
status: open
title: ssl.SSLSocket shutdown doesn't behave like socket.shutdown
type: behavior
versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 
3.4, Python 3.5
Added file: http://bugs.python.org/file31514/ssl-shutdown-fail.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18870] eval() uses latin-1 to decode str

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

See also issue15809.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18843] Py_FatalError (msg=0x7f0e3b373232 "bad leading pad byte") at Python-2.7.5/Python/pythonrun.c:1689

[Tim Peters]

Tim Peters added the comment:

Yet Another Tool ;-)  Python's "small object" allocator grabs memory in chunks 
of 256KB from the system, and carves up the space itself.  Other memory tools 
(like Valgrind ...) only see that Python has grabbed 256KB chunks, so can't 
detect anything wrong _within_ those large blocks.  For that reason, compiling 
Python --without-pymalloc usually increases the usefulness of other tools.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
components: +IO, Library (Lib)
nosy: +georg.brandl, ncoghlan, serhiy.storchaka
versions: +Python 3.3, Python 3.4 -Python 2.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18843] Py_FatalError (msg=0x7f0e3b373232 "bad leading pad byte") at Python-2.7.5/Python/pythonrun.c:1689

[Martin Mokrejs]

Martin Mokrejs added the comment:

Sorry, I should explain a bit more. Gentoo Linux uses a tool named emerge to 
handle packages. It is written in python. So, by that command I initiated 
re-compilation of python itself but it crashed quickly. At that moment emerge 
called the python available on the system at that time which was configured 
using --with-pydebug.


Meanwhile I went to install DUMA http://duma.sourceforge.net/ and started to 
re-run emerge.

# emerge duma
# export LD_PRELOAD=/usr/lib64/libduma.so.0.0.0
# sysctl -w vm.max_map_count=100
# emerge dev-lang/python:2.7 


Thanks for your analysis, will try to stitch something.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18876] Problems with files opened in append mode with io module

[STINNER Victor]

Changes by STINNER Victor :


--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18876] Problems with files opened in append mode with io module

[Erik Bray]

Erik Bray added the comment:

>> Whereas the behavior of O_APPEND causes an automatic seek to the end
>> before any write().

> True, but IIRC some systems seek on open() and some systems seek just before 
> write().

I figured that workaround that seeks to the end on open was an attempt to 
normalize behavior between different systems, and seems to me like a reasonable 
compromise to support systems that somehow don't support append.

But a better compromise I think would be to seek to the beginning of the file 
and modify fileio_write and friends to always seek to the end before a write if 
the file was opened in append mode. This would be more in line with the POSIX 
behavior.

I have no idea on what systems O_APPEND isn't supported, but I figured they 
must exist because someone put an #ifdef in there once :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15507] test_subprocess assumes SIGINT is not being ignored.

[Gregory P. Smith]

Changes by Gregory P. Smith :


--
assignee:  -> gregory.p.smith
nosy: +gregory.p.smith
resolution:  -> fixed
stage: patch review -> committed/rejected
status: open -> closed
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15507] test_subprocess assumes SIGINT is not being ignored.

[Roundup Robot]

Roundup Robot added the comment:

New changeset 9810844126e1 by Gregory P. Smith in branch '3.3':
Fixes issue #15507: test_subprocess's test_send_signal could fail if the test
http://hg.python.org/cpython/rev/9810844126e1

New changeset 6aa7ca40b100 by Gregory P. Smith in branch 'default':
Fixes Issue #15507: test_subprocess's test_send_signal could fail if the test
http://hg.python.org/cpython/rev/6aa7ca40b100

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18850] xml.etree.ElementTree accepts control chars.

[Michele Orrù]

Michele Orrù added the comment:

> Is that you actual use case? That you *want* to store binary data in XML, 
> instead of getting it properly rejected as non well-formed content?
No, Stefan.

What I was saying in my last message was  just "you're right, the user shall 
always use repr() when printing an xml tree" (msg196313) because "xml does 
*not* guarantee to have only printable chars by itself" (msg196368, msg196379).

As an advice I hope you do not take as insult, saying
"in section {section} the spec says {argument}" 
is much more constructive than 
"read the spec on that", "{extremely_obvious_link}",
at least to people not familiar with the spec and asking for the source of your 
arguments (msg196360). Can shorten threads, too.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable

[STINNER Victor]

Changes by STINNER Victor :


--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18843] Py_FatalError (msg=0x7f0e3b373232 "bad leading pad byte") at Python-2.7.5/Python/pythonrun.c:1689

[Tim Peters]

Tim Peters added the comment:

Hmm.  I don't quite know what you're doing:  you said you're getting away from 
--with-pydebug, but these "bad leading pad byte" messages can't be generated 
unless Python is compiled with (at least) PYMALLOC_DEBUG defined.

That said, my advice remains the same:  whittle down the program (instead of 
fiddling with tools) to get a failing case as small & fast as possible.  In 
your "few dozens of seconds at most", many _millions_ of things happen, and 
there's scant chance of out-thinking them.

In the newest traceback (core.emerge.8425.gdb.txt), Python detected the 
corruption while running its cyclic-garbage collector.  Forcing gc to run more 
often may catch the corruption sooner (use Python's gc.set_threshold() to force 
more frequent collections).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18870] eval() uses latin-1 to decode str

[Merlijn van Deen]

Merlijn van Deen added the comment:

On the lowest level, this affects exec, eval(), compile() and input() (!). On a 
higher level, more modules are affected:

modules ast, codeop, compiler, cProfile, dis, distutils (not sure), doctest, 
idlelib, ihooks, pdb, pkgutil, plat-mac, py_compile, rexec, runpy and timeit 
all call compile()

modules dbd, compiler, gettext, idlelib, lib2to3, lib-tk.turtle, logging, 
mhlib, pdb, plat-irix5, plat-mac, rexec, rlcompleter and warnings all call 
eval()

and modules Bastion, bdb, code, collections, cProfile, distutils, doctest, 
idlelib, ihooks, imputil, pdb, plat-irix5, plat-irix6, plat-mac, profile, 
rexec, site, timeit and trace all call exec.

Not all of them necessarily take user-supplied code - I haven't checked that.


After checking tests/test_pep263.py, it seems the behavior is a bit more 
complicated than I initially thought: a str parameter is considered latin-1 
unless either
 a) an utf-8 bom is present, in which case it is considered utf-8
 b) an # encoding: XXX  line is present, in which case it is considered
to be in that encoding

In any case, I have attached a doc patch for exec, eval(), compile(), and 
ast.literal_eval(), because I think these are the most widely used. I think 
input() does not need a doc change because it explicitly refers to eval().

I ignored the subtleties noted above for the doc patch, simplifying to 'pass 
either a Unicode or a latin-1 encoded string'.

--
keywords: +patch
Added file: http://bugs.python.org/file31513/doc_18870.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable

[jort bloem]

New submission from jort bloem:

Issue occurs in the following code: 
tempfile.NamedTemporaryFile(dir=".",delete=False).write("hello")

The NamedTemporaryFile is deleted before the write is complete; deleting the 
object closes the file, so the write fails.

--
messages: 196485
nosy: jort.bloem
priority: normal
severity: normal
status: open
title: tempfile.NamedTemporaryFile can close the file too early, if not 
assigning to a variable
type: behavior
versions: Python 2.6, Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8934] aifc should use str instead of bytes (wave, sunau compatibility)

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +serhiy.storchaka
versions: +Python 3.4 -Python 3.1, Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8934] aifc should use str instead of bytes (wave, sunau compatibility)

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18874] Add a new tracemalloc module to trace memory allocations

[STINNER Victor]

STINNER Victor added the comment:

> Is it really impossible to use a standard Python dict instead of cfuhash?

I don't know if it is possible or not, I didn't try. Using PyDict has many 
requirements:

* the GIL must be held: I plan to hook also PyMem_Raw (which is called without 
the GIL being held)
* PyDict uses PyMem_Malloc() which would be a reentrant call
* the API uses PyObject* whereas tracemalloc uses C types and C structures
* PyDict can raises Python exceptions, which is not something expected from a 
memory allocator (especially PyMem_Malloc)
* etc.

The implementation is not done yet. I would like to be able to compute the size 
of memory allocated by the _tracemalloc module: size of hash tables.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18878] Add support of the 'with' statement to sunau.open.

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
keywords: +patch
stage:  -> patch review
Added file: http://bugs.python.org/file31512/sunau_context_manager.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18843] Py_FatalError (msg=0x7f0e3b373232 "bad leading pad byte") at Python-2.7.5/Python/pythonrun.c:1689

[Martin Mokrejs]

Martin Mokrejs added the comment:

One more note. At that time I had running my application which at that time was 
parsing an XML file using xml.parsers.expat! That is being run in my pipeline 
before I render figures (the initially reported crash case). So, 
matplotlib/numpy is ruled out.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18878] Add support of the 'with' statement to sunau.open.

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

The proposed patch adds support of the 'with' statement to sunau.open (as it 
already was added for aifc.open and wave.open). It based on patches from 
issue17616.

--
components: Library (Lib)
messages: 196482
nosy: r.david.murray, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Add support of the 'with' statement to sunau.open.
type: enhancement
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18843] Py_FatalError (msg=0x7f0e3b373232 "bad leading pad byte") at Python-2.7.5/Python/pythonrun.c:1689

[Martin Mokrejs]

Martin Mokrejs added the comment:

I wanted to move away from the --with-pydebug to a normal python and I failed 
with:

# emerge dev-lang/python:2.7

 * IMPORTANT: 11 news items need reading for repository 'gentoo'.
 * Use eselect news to read news items.

Calculating dependencies... done!
Debug memory block at address p=0xa7f5900: API 'o'
80 bytes originally requested
The 7 pad bytes at p-7 are not all FORBIDDENBYTE (0xfb):
at p-7: 0xfb
at p-6: 0xfb
at p-5: 0xfa *** OUCH
at p-4: 0xfb
at p-3: 0xfb
at p-2: 0xfb
at p-1: 0xfb
Because memory is corrupted at the start, the count of bytes requested
   may be bogus, and checking the trailing pad bytes may segfault.
The 8 pad bytes at tail=0xa7f5950 are FORBIDDENBYTE, as expected.
The block was made by call #21242094 to debug malloc/realloc.
Data at p: 73 00 00 00 79 00 00 00 ... 67 00 00 00 00 00 00 00
Fatal Python error: bad leading pad byte
Aborted (core dumped)
#

Great. So, what can we gather now from this? It got overwritten within few 
dozens of seconds at most since I started emerge. It is in a different memory 
region (so no memory module HW bug). I am attaching the resolved stacktrace 
just in case ...

--
Added file: http://bugs.python.org/file31511/core.emerge.8425.gdb.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17201] Use allowZip64=True by default

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The patch LGTM. If anyone has objections, say them now.

--
assignee:  -> serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18877] tkinter askopenfilenames does not work in Windows library folder

[tegavu]

New submission from tegavu:

Using Python 3.3 on W7x64 I wanted to create a file dialog that allows me to 
select multiple files.
The code below is an example of how to re-create the bug.

# -*- coding: iso-8859-15 -*-
# Python 3

import tkinter 
import tkinter.filedialog  

def callback():
name = tkinter.filedialog.askopenfilenames() 
print(name)
 
tkinter.Button(text='File Open', command=callback).pack()
tkinter.mainloop()


-
THE PROBLEM:
When selecting any file in the windows library\* folders (those: 
http://beingpc.com/wp-content/uploads/2011/05/How-to-Change-the-Windows-7-Library-Icon.jpg
 ) the return value that will be printed is always {}.
tkinter.filedialog.askopenfilename() (without the 's') works fine also in those 
directories.

Can anyone reproduce this behavior?

--
components: Tkinter, Windows
messages: 196479
nosy: tegavu
priority: normal
severity: normal
status: open
title: tkinter askopenfilenames does not work in Windows library folder
type: behavior
versions: Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18876] Problems with files opened in append mode with io module

[Charles-François Natali]

Charles-François Natali added the comment:

> On which systems is O_APPEND not supported? It's part of the POSIX standard, 
> and even Windows seems to have it.

That would be surprising.
An easy way to find this out would be to remove the ifdef:
352 #ifdef O_APPEND
353 if (append)
354 flags |= O_APPEND;
355 #endif

>> Whereas the behavior of O_APPEND causes an automatic seek to the end
>> before any write().
>
> True, but IIRC some systems seek on open() and some systems seek just before 
> write().

POSIX makes it clear that only the later behavior is legal:
O_APPENDIf set, the file offset shall be set to the end of the file
prior to each write.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17003] Unification of read() and readline() argument names

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Are there any objections to the patch?

--
assignee: docs@python -> serhiy.storchaka
versions: +Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1617161] Instance methods compare equal when their self's are equal

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Could you please update the patch for tip? I looks too desynchronized.

--
stage: patch review -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18643] implement socketpair() on Windows

[Charles-François Natali]

Charles-François Natali added the comment:

Alright, I chose a 16MB size for SOCK_MAX_SIZE because nowadays, one
can encounter large buffers for Gigabit/10Gb Ethernet.

Regarding the original issue: is it OK to add it as
socket.socketpair(), or should it only be added to test.support?
Since I don't use Windows, I don't know if this deserves being expose
in socket module, but having socketpair() available in the test suite
would be really useful (for example, most select tests aren't run on
Windows because they use pipes).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18874] Add a new tracemalloc module to trace memory allocations

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc added the comment:

Is it really impossible to use a standard Python dict instead of cfuhash?

--
nosy: +amaury.forgeotdarc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18810] Stop doing stat calls in importlib.machinery.FileFinder to see if something is a file or folder

[Brett Cannon]

Brett Cannon added the comment:

I also just tried using os.listdir() for a package before searching for 
__init__ and it didn't speed anything up either (actually timeit suggests that 
os.listdir() is worse than an isdir + 3 * isfile checks). Nor did caching 
directory stuff at the class level so that they were shared when an 
os.listdir() was done for an __init__ so that didn't have to be done a second 
time if the cache wasn't stale already.

I have attached the patch with all of the changes to cut the number of stat 
calls down significantly, but benchmarking on my work machine shows no 
definitive benefit.

--
keywords: +patch
Added file: http://bugs.python.org/file31510/less_stats.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18851] subprocess's Popen closes stdout/stderr filedescriptors used in another thread when Popen errors

[Antoine Pitrou]

Antoine Pitrou added the comment:

Sorry, wrong patch. Uploading again.

--
Added file: http://bugs.python.org/file31509/subprocess_issue18851_3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18851] subprocess's Popen closes stdout/stderr filedescriptors used in another thread when Popen errors

[Antoine Pitrou]

Changes by Antoine Pitrou :


Removed file: http://bugs.python.org/file31508/subprocess_issue18851_3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18851] subprocess's Popen closes stdout/stderr filedescriptors used in another thread when Popen errors

[Antoine Pitrou]

Antoine Pitrou added the comment:

Ha, sorry. Here is a non-git patch.

--
Added file: http://bugs.python.org/file31508/subprocess_issue18851_3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8713] multiprocessing needs option to eschew fork() under Linux

[Richard Oudkerk]

Richard Oudkerk added the comment:

> I've seen test_multiprocessing_forkserver giving warnings too, while 
> running the whole test suite, but can't reproduce them while running it 
> alone.  The warnings seems quite similar though, so a single fix might 
> resolve the problem with all the tests.

> The "Using start method '...'" should also be displayed only when the 
> tests are run in verbose mode.

Seems to be fixed now.

--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18876] Problems with files opened in append mode with io module

[Antoine Pitrou]

Antoine Pitrou added the comment:

On which systems is O_APPEND not supported? It's part of the POSIX standard, 
and even Windows seems to have it.

> Whereas the behavior of O_APPEND causes an automatic seek to the end
> before any write().

True, but IIRC some systems seek on open() and some systems seek just before 
write().

Agreed about the "mode" attribute problem, though. And the BufferedWriter bug 
is truly embarassing, sorry :-(

--
nosy: +neologix

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18876] Problems with files opened in append mode with io module

[Erik Bray]

Erik Bray added the comment:

Ah right, sorry about that. I just came over from the Trac site for one of my 
projects where the version field is used for affected versions :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18643] implement socketpair() on Windows

[Roundup Robot]

Roundup Robot added the comment:

New changeset 2de7fc69d2d4 by Charles-François Natali in branch '2.7':
Issue #18643: Fix some test_socket failures due to large default socket buffer
http://hg.python.org/cpython/rev/2de7fc69d2d4

New changeset 498957c97c2b by Charles-François Natali in branch '3.3':
Issue #18643: Fix some test_socket failures due to large default socket buffer
http://hg.python.org/cpython/rev/498957c97c2b

New changeset 9b27cf72c79b by Charles-François Natali in branch 'default':
Issue #18643: Fix some test_socket failures due to large default socket buffer
http://hg.python.org/cpython/rev/9b27cf72c79b

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18876] Problems with files opened in append mode with io module

[R. David Murray]

R. David Murray added the comment:

Hi, Erik.  I changed the versions because we use versions in this tracker to 
indicate which versions we intend to *fix* the problem in.  I left 2.7 and 3.3 
marked for the moment, but I have a feeling that this will need to be a 
feature-release-only change.  Not only does it require a new attribute, it also 
changes the semantics (albeit in a sensible way), and as such could break 
working code if applied to a maintenance release.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18876] Problems with files opened in append mode with io module

[R. David Murray]

Changes by R. David Murray :


--
nosy: +pitrou
versions: +Python 3.4 -Python 3.1, Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16201] socket.gethostbyname incorrectly parses ip

[Michele Orrù]

Michele Orrù added the comment:

Cool, thanks for pointing the correct library function. 

Tested on my machine - Debian 3.9 x86_64 GNU/Linux.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18876] Problems with files opened in append mode with io module

[Erik Bray]

New submission from Erik Bray:

I've come across a few difficulties of late with the io module's handling of 
files opened in append mode (any variation on 'a', 'ab', 'a+', 'ab+', etc.

The biggest problem is that the io module does not in any way keep track of 
whether a file was opened in append mode, and it's essentially impossible to 
determine the original mode string that was provided by the user.  For example:

>>> f = open('test', mode='ab+', buffering=0)
>>> f
<_io.FileIO name='test' mode='rb+'>

The 'a' is gone.  That doesn't mean the file *isn't* in append mode.  If 
supported, in fileio_init this still causes the O_APPEND flag to be added to 
the open() call.  But the *only* way to find out after the fact that the file  
was actually opened in append mode is with fcntl:

>>> fcntl.fcntl(f.fileno(), fnctl.F_GETFL) & os.O_APPEND
1024

but this is hardly easily accessible or portable.  So it's possible to have two 
files open in 'rb+' mode but that have wildly differing behaviors.

The only other thing fileio_init does differently with append mode is that it 
seeks to the end of the file by default.  But that does not make the append 
behavior "portable".  If, on a system where O_APPEND was not supported, I seek 
to a different part of the file and the call write() it will *not* append to 
the end of the file.  Whereas the behavior of O_APPEND causes an automatic seek 
to the end before any write().

The fact that no record of the request for 'append' mode is kept leads to 
further bugs, particularly in BufferedWriter.  It doesn't know the raw file was 
opened with O_APPEND so the writes it shows in the buffer differ from what will 
actually end up in the file.  For example:

>>> f = open('test', 'wb')
>>> f.write(b'testest')
7
>>> f.close()
>>> f = open('test', 'ab+')
>>> f.tell()
7
>>> f.write(b'A')
1
>>> f.seek(0)
0
>>> f.read()
b'testestA'
>>> f.seek(0)
0
>>> f.read(1)
b't'
>>> f.write(b'B')
1
>>> f.seek(0)
0
>>> f.read()
b'tBstestA'
>>> f.flush()
>>> f.seek(0)
0
>>> f.read()
b'testestAB'

In this example, I read 1 byte from the beginning of the file, then write one 
byte.  Because of O_APPEND, the effect of the write() call on the raw file is 
to append, regardless of where BufferedWriter seeks it to first.  But before 
the f.flush() call f.read() just shows what's in the buffer which is not what 
will actually be written to the file.  (Naturally, unbuffered io does not have 
this particular problem.)

So, I'm thinking maybe the fileio struct needs to grow an 'append' member.  
This could be used to provide a more accurate mode string, and could for 
example in fileio_write to provide append-like support where it isn't natively 
supported (though perhaps without any guarantees as to atomicity).

--
components: IO
messages: 196464
nosy: erik.bray
priority: normal
severity: normal
status: open
title: Problems with files opened in append mode with io module
versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18851] subprocess's Popen closes stdout/stderr filedescriptors used in another thread when Popen errors

[Charles-François Natali]

Charles-François Natali added the comment:

Rietveld's choking on git diffs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16201] socket.gethostbyname incorrectly parses ip

[Charles-François Natali]

Changes by Charles-François Natali :


Removed file: http://bugs.python.org/file31506/parse_inet.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16201] socket.gethostbyname incorrectly parses ip

[Charles-François Natali]

Charles-François Natali added the comment:

> For Windows >= Vista, inet_pton() is available.
> I'm not sure whether XP has one of them: if not, then we might keep
> the hand-parsing as a fallback.

Apparently, XP doesn't have inet_aton()...

So I changed the code to use inet_pton() if available, otherwise use 
inet_addr(), which is available everywhere (the only gotcha is that it doesn't 
handle "255.255.255.255", so we special-case it like "").

We thus get rid of the hand-parsing completely, and get a more efficient  
handling of numeric IPv4 addresses, and a much more efficient handling for 
numeric IPv6 addresses (see above benchmarks).

--
keywords: +needs review
stage:  -> patch review
Added file: http://bugs.python.org/file31507/parse_inet.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14597] Cannot unload dll in ctypes until script exits

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc added the comment:

Did you consider creating a copy with a random suffix?

--
nosy: +amaury.forgeotdarc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16201] socket.gethostbyname incorrectly parses ip

[Charles-François Natali]

Charles-François Natali added the comment:

Here's a patch using inet_pton() if available, otherwise inet_aton()
if available, otherwise fallback to getaddrinfo().
This should work on every platform, but if a platform has neither
inet_pton() nor inet_aton(), calling getaddrinfo() will incur an
overhead.
All Unices have either inet_aton() or inet_pton().
For Windows >= Vista, inet_pton() is available.
I'm not sure whether XP has one of them: if not, then we might keep
the hand-parsing as a fallback.

This adds support for numeric IPv6 addresses (if inet_pton() is
available), which notably speeds up sendto() on IPv6 sockets, and
gives a tiny speedup for IPv4 sockets:
before:
$ ./python -m timeit -s "import socket; s =
socket.socket(socket.AF_INET, socket.SOCK_DGRAM); DATA = b'x'"
"s.sendto(DATA, ('127.0.0.1', 4242))"
10 loops, best of 3: 13.2 usec per loop
$ ./python -m timeit -s "import socket; s =
socket.socket(socket.AF_INET6, socket.SOCK_DGRAM); DATA = b'x'"
"s.sendto(DATA, ('::1', 4242))"
1 loops, best of 3: 32.1 usec per loop
after:
$ ./python -m timeit -s "import socket; s =
socket.socket(socket.AF_INET, socket.SOCK_DGRAM); DATA = b'x'"
"s.sendto(DATA, ('127.0.0.1', 4242))"
10 loops, best of 3: 11.5 usec per loop
$ ./python -m timeit -s "import socket; s =
socket.socket(socket.AF_INET6, socket.SOCK_DGRAM); DATA = b'x'"
"s.sendto(DATA, ('::1', 4242))"
10 loops, best of 3: 12.8 usec per loop

Note that if the IPv6 address contains a scope ID ('%' suffix), then
we fallback to getaddrinfo() which handles it properly (the scope ID
can be an interface index but also an interface name).

--
Added file: http://bugs.python.org/file31506/parse_inet.diff

___
Python tracker 

___diff -r 4b3238923b01 Lib/test/test_socket.py
--- a/Lib/test/test_socket.py   Fri May 10 19:57:44 2013 -0700
+++ b/Lib/test/test_socket.py   Thu Aug 29 16:10:40 2013 +0200
@@ -761,6 +761,31 @@
 if not fqhn in all_host_names:
 self.fail("Error testing host resolution mechanisms. (fqdn: %s, 
all: %s)" % (fqhn, repr(all_host_names)))
 
+def test_hosts_resolution(self):
+valid_hosts = [support.HOST]
+valid_hosts_ipv6 = [support.HOSTv6]
+
+invalid_hosts = ['0.1.1.1.', 'a1.1.1.1', '10.1a.1.1',
+ '4294967306.4294967296.4294967296.1']
+invalid_hosts_ipv6 = ['::1q', '::1::2', '1:1:1:1:1:1:1:1:1']
+
+for host in valid_hosts:
+self.assertEqual(socket.gethostbyname(host), host)
+
+for host in invalid_hosts:
+self.assertRaises(OSError, socket.gethostbyname, host)
+
+# gethostbyaddr() support IPv6, contrarily to gethosbyname()
+if support.IPV6_ENABLED:
+valid_hosts.extend(valid_hosts_ipv6)
+invalid_hosts.extend(invalid_hosts_ipv6)
+
+for host in valid_hosts:
+self.assertIn(host, socket.gethostbyaddr(host)[2])
+
+for host in invalid_hosts:
+self.assertRaises(OSError, socket.gethostbyaddr, host)
+
 @unittest.skipUnless(hasattr(socket, 'sethostname'), "test needs 
socket.sethostname()")
 @unittest.skipUnless(hasattr(socket, 'gethostname'), "test needs 
socket.gethostname()")
 def test_sethostname(self):
diff -r 4b3238923b01 Modules/socketmodule.c
--- a/Modules/socketmodule.cFri May 10 19:57:44 2013 -0700
+++ b/Modules/socketmodule.cThu Aug 29 16:10:40 2013 +0200
@@ -799,8 +799,6 @@
 {
 struct addrinfo hints, *res;
 int error;
-int d1, d2, d3, d4;
-char ch;
 
 memset((void *) addr_ret, '\0', sizeof(*addr_ret));
 if (name[0] == '\0') {
@@ -865,20 +863,49 @@
 sin->sin_addr.s_addr = INADDR_BROADCAST;
 return sizeof(sin->sin_addr);
 }
-if (sscanf(name, "%d.%d.%d.%d%c", &d1, &d2, &d3, &d4, &ch) == 4 &&
-0 <= d1 && d1 <= 255 && 0 <= d2 && d2 <= 255 &&
-0 <= d3 && d3 <= 255 && 0 <= d4 && d4 <= 255) {
-struct sockaddr_in *sin;
-sin = (struct sockaddr_in *)addr_ret;
-sin->sin_addr.s_addr = htonl(
-((long) d1 << 24) | ((long) d2 << 16) |
-((long) d3 << 8) | ((long) d4 << 0));
-sin->sin_family = AF_INET;
+/* avoid a name resolution in case of numeric address */
+#ifdef HAVE_INET_PTON
+/* check for an IPv4 address */
+if (af == AF_UNSPEC || af == AF_INET) {
+struct sockaddr_in *sin = (struct sockaddr_in *)addr_ret;
+if (inet_pton(AF_INET, name, &sin->sin_addr) > 0) {
+sin->sin_family = AF_INET;
 #ifdef HAVE_SOCKADDR_SA_LEN
-sin->sin_len = sizeof(*sin);
-#endif
-return 4;
-}
+sin->sin_len = sizeof(*sin);
+#endif
+return 4;
+}
+}
+#ifdef ENABLE_IPV6
+/* check for an IPv6 address - if the address contains a scope ID, we
+ * fallback to getaddrinfo(), which can handle translation from interface
+ * name

[issue16510] Using appropriate checks in tests

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Patch updated to sync with tip. It is 5% less than first version because some 
changes already applied in other commits.

--
Added file: http://bugs.python.org/file31505/tests_asserts_2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8713] multiprocessing needs option to eschew fork() under Linux

[Roundup Robot]

Roundup Robot added the comment:

New changeset 6d998a43102b by Richard Oudkerk in branch 'default':
Issue #8713: Print dangling processes/threads, if any.
http://hg.python.org/cpython/rev/6d998a43102b

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18808] Thread.join returns before PyThreadState is destroyed

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8713] multiprocessing needs option to eschew fork() under Linux

[Roundup Robot]

Roundup Robot added the comment:

New changeset f6c7ad7d029a by Richard Oudkerk in branch 'default':
Issue #8713: Test should not print message about start method.
http://hg.python.org/cpython/rev/f6c7ad7d029a

New changeset e99832a60e63 by Richard Oudkerk in branch 'default':
Issue #8713: Cleanup before saving process._dangling.
http://hg.python.org/cpython/rev/e99832a60e63

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18875] Automatic insertion of the closing parentheses, brackets, and braces

[irdb]

New submission from irdb:

Many other IDE's have this feature that allows automatic insertion of closing 
parentheses as user is typing something. I think it saves time and increases 
programming speed. Is it possible to have this on IDLE, too?

--
components: IDLE
messages: 196456
nosy: irdb
priority: normal
severity: normal
status: open
title: Automatic insertion of the closing parentheses, brackets, and braces
type: enhancement
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13107] Text width in optparse.py can become negative

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Yes, argparse has same problem.

>>> import os, argparse
>>> p = argparse.ArgumentParser(prog='PROG')
>>> os.environ['COLUMNS'] = '16'
>>> print(p.format_help())
Traceback (most recent call last):
  File "", line 1, in 
  File "/home/serhiy/py/cpython/Lib/argparse.py", line 2329, in format_help
return formatter.format_help()
  File "/home/serhiy/py/cpython/Lib/argparse.py", line 276, in format_help
help = self._root_section.format_help()
  File "/home/serhiy/py/cpython/Lib/argparse.py", line 206, in format_help
func(*args)
  File "/home/serhiy/py/cpython/Lib/argparse.py", line 206, in format_help
func(*args)
  File "/home/serhiy/py/cpython/Lib/argparse.py", line 514, in _format_action
help_lines = self._split_lines(help_text, help_width)
  File "/home/serhiy/py/cpython/Lib/argparse.py", line 614, in _split_lines
return _textwrap.wrap(text, width)
  File "/home/serhiy/py/cpython/Lib/textwrap.py", line 355, in wrap
return w.wrap(text)
  File "/home/serhiy/py/cpython/Lib/textwrap.py", line 300, in wrap
return self._wrap_chunks(chunks)
  File "/home/serhiy/py/cpython/Lib/textwrap.py", line 227, in _wrap_chunks
raise ValueError("invalid width %r (must be > 0)" % self.width)
ValueError: invalid width 0 (must be > 0)

--
assignee: serhiy.storchaka -> 
versions: +Python 2.7, Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18684] Pointers point out of array bound in _sre.c

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
assignee:  -> serhiy.storchaka
keywords: +needs review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16201] socket.gethostbyname incorrectly parses ip

[Charles-François Natali]

Charles-François Natali added the comment:

Note that for IPv6 addresses, we should just extract manually the
scope ID ('%' prefix) if present.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18672] Fix format specifiers for debug output in _sre.c

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
assignee:  -> serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18685] Restore re performance to pre-PEP393 level

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
dependencies: +Fix format specifiers for debug output in _sre.c, Pointers point 
out of array bound in _sre.c

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18672] Fix format specifiers for debug output in _sre.c

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
keywords: +needs review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18830] Remove duplicates from a result of getclasstree()

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Here is a patch with additional test.

--
Added file: http://bugs.python.org/file31504/getclasstree_no_dups_2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16201] socket.gethostbyname incorrectly parses ip

[Charles-François Natali]

Charles-François Natali added the comment:

Stupid question: why use scanf()/strtol() to parse an IP address, and not 
simply inet_pton()/inet_aton(), which are made precisely for that purpose?
inet_pton() is POSIX (it was introduced at the same time as getaddrinfo(), 
which is used unconditionally a couple lines below). If it's not available, we 
could use inet_aton() as fallback (which is amusingly not POSIX, but I doubt 
we'll find a platform with neither inet_pton() nor inet_aton()).

Using inet_pton() would have the added advantage of working with IPv6 
addresses: right now, a numeric IPv6 address is always resolved by 
getaddrinfo(), which results in a performance overhead:

$ python -m timeit -s "from socket import *; s = socket(AF_INET, SOCK_DGRAM); 
DATA = 'hello'" "s.sendto(DATA, ('127.0.0.1', 42))"
10 loops, best of 3: 6.86 usec per loop
$ python -m timeit -s "from socket import *; s = socket(AF_INET6, SOCK_DGRAM); 
DATA = 'hello'" "s.sendto(DATA, ('::1', 42))"
1 loops, best of 3: 24.2 usec per loop

--
nosy: +neologix

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17974] Migrate unittest to argparse

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17974] Migrate unittest to argparse

[Roundup Robot]

Roundup Robot added the comment:

New changeset 340f45374cf5 by Serhiy Storchaka in branch 'default':
Issue #17974: Switch unittest from using getopt to using argparse.
http://hg.python.org/cpython/rev/340f45374cf5

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16799] start using argparse.Namespace in regrtest

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
resolution:  -> fixed
stage: test needed -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18806] socketmodule: fix/improve setipaddr() numeric addresses handling

[Charles-François Natali]

Changes by Charles-François Natali :


--
resolution:  -> duplicate
stage:  -> committed/rejected
status: open -> closed
superseder:  -> socket.gethostbyname incorrectly parses ip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16799] start using argparse.Namespace in regrtest

[Roundup Robot]

Roundup Robot added the comment:

New changeset 997de0edc5bd by Serhiy Storchaka in branch 'default':
Issue #16799: Switched from getopt to argparse style in regrtest's argument
http://hg.python.org/cpython/rev/997de0edc5bd

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18851] subprocess's Popen closes stdout/stderr filedescriptors used in another thread when Popen errors

[Antoine Pitrou]

Antoine Pitrou added the comment:

(in any case, the test is probably worth adding)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18851] subprocess's Popen closes stdout/stderr filedescriptors used in another thread when Popen errors

[Antoine Pitrou]

Antoine Pitrou added the comment:

Gregory, it seems fixed indeed under Unix but I'm not sure about Windows. Read 
what I wrote above: apparently the cleanup code calls os.close() on a Windows 
HANDLE; then conveniently it swallows the exception :-)

Also, the Windows version of _execute_child doesn't set _closed_child_pipe_fds.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17974] Migrate unittest to argparse

[Antoine Pitrou]

Antoine Pitrou added the comment:

I think this should be committed. This is the first step in making command-line 
args easily customizable (e.g. by exposing the parsers as a public API in 
TestProgram).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18758] Fix internal references in the documentation

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
assignee: docs@python -> serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18743] References to non-existant "StringIO" module

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18743] References to non-existant "StringIO" module

[Roundup Robot]

Roundup Robot added the comment:

New changeset 6c0f5af2f5a5 by Serhiy Storchaka in branch '3.3':
Issue #18743: Fix references to non-existant "StringIO" module
http://hg.python.org/cpython/rev/6c0f5af2f5a5

New changeset 676bbd5b0254 by Serhiy Storchaka in branch 'default':
Issue #18743: Fix references to non-existant "StringIO" module
http://hg.python.org/cpython/rev/676bbd5b0254

New changeset 8c768bbacd92 by Serhiy Storchaka in branch '2.7':
Issue #18743: Improved cross-references to the StringIO class.
http://hg.python.org/cpython/rev/8c768bbacd92

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com