The stack space allocated by ffi_call for the non-register arguments is not big enough (and not properly aligned) when calling the target function, and depending on what the called function does with the stack, it can end up overwriting ffi_call_unix64's stack. A case where this occurred is with a target function with a lot of arguments, the last one being a boolean. The target function itself, depending on gcc version and optimization level would rewrite all boolean arguments with padding to pass them to another function, and the allocated stack space for that was insufficient, such that rewriting this last boolean argument would actually overflow, overwriting the flags in ffi_call_unix64 and making the returned value ignored.
See https://bugzilla.mozilla.org/show_bug.cgi?id=594611 -- Summary: Bad stack allocation for ffi function calls on x86-64 Product: gcc Version: unknown Status: UNCONFIRMED Severity: normal Priority: P3 Component: libffi AssignedTo: unassigned at gcc dot gnu dot org ReportedBy: mh+gcc at glandium dot org GCC target triplet: x86_64-*-linux http://gcc.gnu.org/bugzilla/show_bug.cgi?id=45677