Dear GMP maintainers,
the Windows x64 ABI requires callers to allocate a 32 byte "parameter
area" before calling into a function, which the callee is allowed to use
as it pleases[1]. fat_init does not do this before calling
__gmpn_cpuvec_init, thus violating the ABI.
This is not usually an issue, since neither gcc nor clang seem to use
the space. MSVC, however, does, which results in rax/rcx/r8/r9
incorrectly being restored to clobbered values, and, in turn, random
crashes as fat_init jumps into the real routines with incorrect arguments.
I'm not sure whether MSVC is a supported compiler (I assume it isn't,
because --enable-fat requires support for variable-length arrays, which
MSVC will not compile in C mode), but it's possible that other compilers
may also use the parameter area as a scratch space, or even that
gcc/clang start using it in future releases.
Attached patch solves this problem by just allocating 32 bytes of stack
space before calling __gmpn_cpuvec_init, and discarding them afterwards.
Thanks,
Nicolas
[1] "Even if the called function has fewer than 4 parameters, these 4
stack locations ... may be used by the called function for other
purposes ..." https://msdn.microsoft.com/en-us/library/ew5tede7.aspx
--- a/mpn/x86_64/fat/fat_entry.asm
+++ b/mpn/x86_64/fat/fat_entry.asm
@@ -167,7 +167,13 @@
push %r8
push %r9
push %rax
+ifdef(`HOST_DOS64',`dnl
+ subq $32, %rsp
+')
CALL( __gmpn_cpuvec_init)
+ifdef(`HOST_DOS64',`dnl
+ addq $32, %rsp
+')
pop %rax
pop %r9
pop %r8
_______________________________________________
gmp-bugs mailing list
gmp-bugs@gmplib.org
https://gmplib.org/mailman/listinfo/gmp-bugs
