[issue22273] abort when passing certain structs by value using ctypes
Weeble added the comment: I had a closer look at the cif object in gdb. The ffi_type of the argument in question has size 16, alignment 1, type FFI_TYPE_STRUCT and elements contains a single nested ffi_type, of size 8, alignment 8, type FFI_TYPE_POINTER. I think this pointer type is wrong. The struct should indeed be size 16, but its contents (in this case) should be 16 bytes of uint8s, rather than a single pointer. I'm not certain how to correctly describe an array to libffi. While you might be able to hack it with a nested struct filled with 16 individual integers, I have no idea if that would work consistently across platforms. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22273 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22273] abort when passing certain structs by value using ctypes
New submission from Weeble:
I'm not 100% certain this is a bug yet, but I'm beginning to think it's likely.
On 64-bit Linux, I can't pass a struct like this:
struct S { uint8_t data[16]; };
...to a function declared like this:
void f(struct S);
From experimentation with various integer types and array sizes, it seems this
causes an abort somewhere in libffi any time the array is between 9 and 16
bytes in size. If the array is smaller or larger than that, the calls work as
expected.
I've asked about this here:
http://stackoverflow.com/questions/25487928/is-this-the-correct-way-to-pass-a-struct-by-value-in-ctypes
Here's some test code:
## sum.cpp
#include cstdint
using std::size_t;
struct ArrayStruct {
// We'll define ARRAY_TYPE and ARRAY_SIZE on the
// command-line when we compile.
std::ARRAY_TYPE data[ARRAY_SIZE];
};
extern C int64_t sum(struct ArrayStruct array)
{
int64_t acc=0;
for (size_t i=0; i!=ARRAY_SIZE; ++i)
{
acc+=array.data[i];
}
return acc;
}
## sum.py
import ctypes
import sys
def main():
array_size = int(sys.argv[1])
array_type = sys.argv[2]
libsum = ctypes.cdll.LoadLibrary('./libsum.so')
ArrType = getattr(ctypes, 'c_' + array_type) * array_size
class MyStruct(ctypes.Structure):
_fields_ = [(data, ArrType)]
m=MyStruct()
for i in range(array_size):
m.data[i]=i
print(libsum.sum(m))
if __name__ == '__main__':
main()
## Build/run
$ g++ -g -shared -Wall -fPIC sum.cpp -o libsum.so -std=c++11 -D
ARRAY_SIZE=16 -D ARRAY_TYPE=uint8_t python3 sum.py 16 uint8
Aborted (core dumped)
I poked around a little bit in gdb. It's aborting in libffi's ffi_call
function: https://github.com/atgreen/libffi/blob/v3.0.13/src/x86/ffi64.c#L516
(gdb) bt
#0 0x7782cf79 in __GI_raise (sig=sig@entry=6) at
../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x77830388 in __GI_abort () at abort.c:89
#2 0x767134f5 in ffi_call (cif=0x7fffd7b0, fn=0x7650c625
sum(ArrayStruct), rvalue=0x7fffd6f0, avalue=0x7fffd6d0) at
../src/x86/ffi64.c:516
#3 0x7691fee3 in _ctypes_callproc () from
/usr/lib/python3.4/lib-dynload/_ctypes.cpython-34m-x86_64-linux-gnu.so
#4 0x76920578 in ?? () from
/usr/lib/python3.4/lib-dynload/_ctypes.cpython-34m-x86_64-linux-gnu.so
#5 0x0043810a in PyObject_Call ()
#6 0x00579f45 in PyEval_EvalFrameEx ()
#7 0x0057d3d3 in PyEval_EvalCodeEx ()
#8 0x0057bfaa in PyEval_EvalFrameEx ()
#9 0x0057d3d3 in PyEval_EvalCodeEx ()
#10 0x0060ba83 in PyRun_FileExFlags ()
#11 0x0060bc85 in PyRun_SimpleFileExFlags ()
#12 0x0060d3ac in Py_Main ()
#13 0x0041ec0d in main ()
(gdb) frame 2
#2 0x767134f5 in ffi_call (cif=0x7fffd7b0, fn=0x7650c625
sum(ArrayStruct), rvalue=0x7fffd6f0, avalue=0x7fffd6d0) at
../src/x86/ffi64.c:516
516 abort();
(gdb) info args
cif = 0x7fffd7b0
fn = 0x7650c625 sum(ArrayStruct)
rvalue = 0x7fffd6f0
avalue = 0x7fffd6d0
(gdb) info locals
a = optimized out
j = optimized out
size = 8
n = optimized out
classes = {X86_64_INTEGER_CLASS, X86_64_NO_CLASS, 4294956784, 32767}
stack = 0x7fffd4f0
argp = 0x7fffd5a0 \001
arg_types = 0x7fffd6b0
gprcount = 1
ssecount = optimized out
ngpr = 1
nsse = 0
i = optimized out
avn = optimized out
ret_in_memory = optimized out
reg_args = 0x7fffd4f0
(gdb) print *cif
$2 = {abi = FFI_UNIX64, nargs = 1, arg_types = 0x7fffd6b0, rtype =
0x76b5e228, bytes = 0, flags = 10}
It looks like we're trying to pass the struct in two registers, which I think
is what's supposed to happen, but something is going wrong with the second
register. It aborted because it has class X86_64_NO_CLASS and that's not
handled by the switch.
I don't know if this is a bug in libffi, or if ctypes is feeding it bad
information, or if I'm feeding ctypes bad information. I hope this information
is useful for anyone investigating.
I get the same abort in both Python 2.7.6 and 3.4.0.
I originally stumbled across this issue trying to use PySDL2:
http://pysdl2.readthedocs.org/en/rel_0_9_3/
I was trying to call SDL_JoystickGetGUIDString, which uses a similar
struct-by-value call:
http://hg.libsdl.org/SDL/file/92ca74200ea5/include/SDL_joystick.h
--
components: ctypes
messages: 225881
nosy: weeble
priority: normal
severity: normal
status: open
title: abort when passing certain structs by value using ctypes
type: crash
versions: Python 3.4
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22273
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs
[issue21194] json.dumps with ensure_ascii=False doesn't escape control characters
Weeble added the comment: Ah, sorry for the confusion. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21194 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21194] json.dumps with ensure_ascii=False doesn't escape control characters
New submission from Weeble: The JSON spec (http://www.json.org/) does not allow unescaped control characters. (See the railroad diagram for strings and the grammar on the right.) If json.dumps is called with ensure_ascii=False, it fails to escape control codes in the range U+007F to U+009F. Here's an example: import json import unicodedata for i in range(256): ... jsonstring = json.dumps(chr(i), ensure_ascii=False) ... if any(unicodedata.category(ch) == 'Cc' for ch in jsonstring): ... print(Fail:,repr(chr(i))) Fail: '\x7f' Fail: '\x80' Fail: '\x81' Fail: '\x82' Fail: '\x83' Fail: '\x84' Fail: '\x85' Fail: '\x86' Fail: '\x87' Fail: '\x88' Fail: '\x89' Fail: '\x8a' Fail: '\x8b' Fail: '\x8c' Fail: '\x8d' Fail: '\x8e' Fail: '\x8f' Fail: '\x90' Fail: '\x91' Fail: '\x92' Fail: '\x93' Fail: '\x94' Fail: '\x95' Fail: '\x96' Fail: '\x97' Fail: '\x98' Fail: '\x99' Fail: '\x9a' Fail: '\x9b' Fail: '\x9c' Fail: '\x9d' Fail: '\x9e' Fail: '\x9f' -- components: Library (Lib) messages: 215868 nosy: weeble priority: normal severity: normal status: open title: json.dumps with ensure_ascii=False doesn't escape control characters type: behavior versions: Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21194 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue444582] Finding programs in PATH, adding shutil.which
Weeble clockworksa...@gmail.com added the comment: I'm not sure what rules are used by Windows to process the PATH string, but I think they are similar to the rules used to parse the command-line into argv in a C/C++ program: http://msdn.microsoft.com/en-us/library/17w5ykft.aspx I have tested various arrangements of double-quotes in path elements. It appears the quotes can appear anywhere, not just at the start and end of entries. As far as I can tell, *all* they do is toggle the interpretation of the semicolon character between a separator and a character in the directory path. Note in particular: quotes may surround an entire path, segments of a path, fragments of segments of a path, or even may be completely empty. Any number of quotes can appear in a single path entry. There doesn't even need to be an even number of quotes - it seems that an odd number of quotes are treated the same as if a final quote was appended to the very end of the PATH string. Running my attached test batch file, I see these results: c:\Users\weeblepathtest PATH= FAIL PATH=c:\Users\weeble\foo;bar FAIL PATH=c:\Users\weeble\foo;bar SUCCESS PATH=c:\Users\weeble\foo;bar SUCCESS PATH=c:\Users\weeble\foo;bar SUCCESS PATH=c:\Users\weeble\foo;bar SUCCESS PATH=c:\Users\weeble\foo;bar SUCCESS PATH= FAIL PATH=c:\Users\weeble\foo;bar SUCCESS -- nosy: +weeble Added file: http://bugs.python.org/file23797/pathtest.bat ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue444582 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11699] Documentation for get_option_group is wrong
New submission from Weeble clockworksa...@gmail.com: The docs for optparse say this: OptionParser.get_option_group(opt_str) Return, if defined, the OptionGroup that has the title or the long description equals to opt_str After failing to get this to work at all, I looked at the implementation, and the documentation is completely wrong. The method takes an option string, like '-o' or '--option' and returns the option group that it belongs to. Personally, I would much prefer the function to do what the documentation describes, i.e. find the option group with the given name, but I guess it's far too late for that. The scenario I'm in is that I'm writing a plugin for an existing tool which provides access to its OptionParser to add new options, but doesn't provide references to any of the groups that are already created. It looks like this documentation was added recently: http://bugs.python.org/issue1665333 -- messages: 132385 nosy: weeble priority: normal severity: normal status: open title: Documentation for get_option_group is wrong ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11699 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1665333] Documentation missing for OptionGroup class in optparse
Weeble clockworksa...@gmail.com added the comment: I think the documentation for get_option_group is not right. I've created a new bug: http://bugs.python.org/issue11699 -- nosy: +weeble ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1665333 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2304] subprocess under windows fails to quote properly when shell=True
Weeble clockworksa...@gmail.com added the comment:
Is there any way to write code that's safe with or without this fix? If I have
code that currently does this:
subprocess.check_call('c:\some path with spaces arg1 arg2 a quoted arg',
shell=True)
...will it break when running on a version of Python with this fix? Is there
any good way to detect whether the running Python has this fix or not?
--
nosy: +weeble
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2304
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5124] IDLE - pasting text doesn't delete selection
New submission from Weeble clockworksa...@gmail.com: Steps to reproduce: 1. Start IDLE. 2. Enter some text: spam eggs 3. Select the first line (spam) and press control+c to copy. 4. Select the second line (eggs) and press control+v to paste. Expected result: spam spam (With the caret at the end of the second line and no selection.) Actual result (IDLE 2.6, Tk/Tcl 8.4, Linux): spam eggsspam (With eggs selected and the caret at the end of the second line.) I don't think this happens to me on Windows. It might be deliberate; I know that Tk has slightly different behaviour on Windows and Linux. But it doesn't seem to be consistent with other behaviour: for example, if instead of pasting I start typing, the selected text is erased and the typed text replaces it. I can look into this myself, but I would appreciate if someone else can confirm that this is not the expected behaviour, since perhaps it is only my Windows background that leads me to expect this. -- components: IDLE messages: 80914 nosy: weeble severity: normal status: open title: IDLE - pasting text doesn't delete selection ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5124 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4676] python3 closes + home keys
Weeble clockworksa...@gmail.com added the comment: You're right: we should find a solution that's safe and supported. I *think* the primary reason for overriding the home key behaviour was not for the interactive shell, but to make it easier to edit code. Most programmer's editors that do automatic indentation also let you use home to move to the beginning of the text on a line, rather than the absolute beginning of the line. I couldn't say for sure though: I'm new here. I've proposed some changes to the ctrl-left, ctrl-right behaviour on idle-dev that would also require custom handling of selection behaviour. If it were to be decided that reimplementing the selection-handling was worthwhile I would be willing to work on such a patch to redo the selection-handling using only supported Tk features, but I get the impression I'd need lots of help to get it tested with every possible version of Tcl/Tk. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4676 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4676] python3 closes + home keys
Weeble clockworksa...@gmail.com added the comment: I can't see any useful reason to go to the absolute start of the line in the interactive shell. However, it does make sense in the source editor, and it is consistent with, for example, Visual Studio. The first use- case off the top of my head is when you want to copy a block of code and not lose the relative indentation of the first line. You press home- twice to go to the absolute start of line and select from there. It's also the only quick way in IDLE to be sure that the text widget is scrolled all the way to the left, since there's no horizontal scrollbar. (Any idea why that is? I assumed it's to discourage long lines, but I don't know.) The proposal I refer to in IDLE-dev is this: http://mail.python.org/pipermail/idle-dev/2009-January/002742.html I only mention it because it demonstrates another reason why we might want to override the default selection/navigation mechanics. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4676 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4676] python3 closes + home keys
Weeble clockworksa...@gmail.com added the comment: Just got a chance to test this on a Windows desktop with a proper keyboard. (My laptop does weird things with num-lock and scroll-lock.) I got it to crash once, but I have no idea what was special about that time. Otherwise I can reproduce the exception traceback printed to the console, with the following steps: 1) Start IDLE from the console, e.g.: python c:\python26\Lib\idlelib\idle.py 2) Make sure num-lock is turned OFF. 3) Enter a few characters, e.g: rhubarb 4) Press shift+left to create a selection of at least one character. 5) Press shift+home. The stack trace appears in the console: Exception in Tkinter callback Traceback (most recent call last): File C:\Python26\lib\lib-tk\Tkinter.py, line 1410, in __call__ return self.func(*args) File C:\Python26\lib\idlelib\MultiCall.py, line 150, in handler r = l[i](event) File C:\Python26\lib\idlelib\EditorWindow.py, line 333, in home_callback if self.text.compare(first,,last): File C:\Python26\lib\lib-tk\Tkinter.py, line 2858, in compare self._w, 'compare', index1, op, index2)) TclError: expected boolean value but got This is consistent with my understanding that Tk 8.5 does not use a mark named anchor to indicate the selection anchor. I have written a patch that directly calls tk::TextKeySelect instead of trying to duplicate its behaviour using the anchor mark. I'm not terribly confident with using diff and patch, so please let me know if I've done it wrong. I did use -u. Added file: http://bugs.python.org/file12866/IDLE_fix_shift_home.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4676 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1529142] Allowing multiple instances of IDLE with sub-processes
Weeble clockworksa...@gmail.com added the comment: A thought occurs to me: would this patch make it harder to cope with awkward firewalls that block the connection? Are they more or less likely to intervene when passing a port of 0 and letting it pick a port automatically? And if they do intervene, would users be able to create an exception if there is no small list of ports that might be used? Apart from that, what do I need to do to move this forward? Go hunt for more testers willing to try out the patch? Find an expert to review it? ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1529142 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3068] IDLE - Add an extension configuration dialog
Changes by Weeble clockworksa...@gmail.com: -- nosy: +weeble ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3068 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1529142] Allowing multiple instances of IDLE with sub-processes
Weeble clockworksa...@gmail.com added the comment: I installed Ubuntu on a laptop at the beginning of January and have applied the patch I submitted above to IDLE. I've been using it regularly for the last few weeks and have had no problems with it. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1529142 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4676] python3 closes + home keys
Weeble clockworksa...@gmail.com added the comment: I have experienced similar problems in Python 2.6.1 on Windows, and found them to be due to Tk 8.5. I posted my findings here: http://mail.python.org/pipermail/idle-dev/2009-January/002738.html In summary, Tk 8.5 changed the name of the anchor mark to be unique to each Text widget. The code to make the home key toggle between column 0 and the start of the text tries to make use of the anchor mark and gets confused. In the email I proposed a simple fix which I'm testing out right now. Should I attach it as a patch? -- nosy: +weeble ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4676 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4676] python3 closes + home keys
Weeble clockworksa...@gmail.com added the comment: Another complication. On Windows, this line doesn't do what it claims: if (event.state 12) != 0 and event.keysym == Home: # state1==shift, state4==control, state8==alt return # Modifier-Home; fall back to class binding The comment says state8==alt, but this is wrong. state8==mod1, and on Windows Tk defines mod1 to be num-lock. So if you have num-lock on, home_callback will always fall back to the standard binding. See the Tk source: xlib/X11/X.h defines Mod1Mask win/tkWinX.c maps VK_NUMLOCK to Mod1Mask ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4676 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1529142] Allowing multiple instances of IDLE with sub-processes
Changes by Weeble [EMAIL PROTECTED]: Added file: http://bugs.python.org/file12180/IDLE_automatic_ports.diff ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue1529142 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1529142] Allowing multiple instances of IDLE with sub-processes
Weeble [EMAIL PROTECTED] added the comment: Okay, I've uploaded a patch. As suggested, it passes 0 for the port number and the port is automatically assigned. It then extracts the port from the socket and passes it to the subprocess. Note that this means the subprocess inherits the listening socket, which I gather is less than ideal. However, as far as I can tell, this happens anyway if the subprocess is restarted, since the socket remains bound and listening the whole time. The patch also removes the use of SO_REUSEADDR because it is no longer needed. This works without a problem for me on Windows XP, but I can't test it on anything more Unixy. I removed the three retries when it fails to bind a socket - it's not clear when this might fail and yet still be a good idea to retry, but I can put that back if that's something that shouldn't be messed with. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue1529142 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1529142] Allowing multiple instances of IDLE with sub-processes
Weeble [EMAIL PROTECTED] added the comment: Is this ever likely to make it into IDLE? Running without a subprocess in Windows appears to interact badly with the multiprocessing library (attempting to spawn a process just creates a new IDLE window). I couldn't figure out how to apply the patch on Windows - I couldn't tell if the errors were because I was using the patch program wrong or because the file is different now. Nevertheless, I made the changes manually and they seemed to work. What needs to happen for this issue to be fixed, whether by this patch or another solution? ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue1529142 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1529142] Allowing multiple instances of IDLE with sub-processes
Changes by Weeble [EMAIL PROTECTED]: -- nosy: +weeble ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue1529142 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
