Nicolas Hake <n...@nosebud.de> writes: the Windows x64 ABI requires callers to allocate a 32 byte "parameter area" before calling into a function, which the callee is allowed to use as it pleases[1]. fat_init does not do this before calling __gmpn_cpuvec_init, thus violating the ABI. Right; apparently we're aware of this as such allocations are performed in most cases in GMP. There seem to be 5 forgotten places, the one you found and 4 more.
I'm not sure whether MSVC is a supported compiler (I assume it isn't, because --enable-fat requires support for variable-length arrays, which MSVC will not compile in C mode), but it's possible that other compilers may also use the parameter area as a scratch space, or even that gcc/clang start using it in future releases. We should follow the ABI, of course. Where do we (unconditionally) rely on variable-length arrays? While such arrays are in the C standard since 18 years, it is missing from C++. Attached patch solves this problem by just allocating 32 bytes of stack space before calling __gmpn_cpuvec_init, and discarding them afterwards. Thanks. Here is a slightly different patch: diff -Nrc2 gmp-main.253deadf9fc8/mpn/x86_64/divrem_2.asm gmp-main/mpn/x86_64/divrem_2.asm *** gmp-main.253deadf9fc8/mpn/x86_64/divrem_2.asm Tue Apr 25 20:52:18 2017 --- gmp-main/mpn/x86_64/divrem_2.asm Tue Apr 25 20:52:18 2017 *************** *** 101,106 **** --- 101,108 ---- IFSTD(` mov %r11, %rdi ') IFDOS(` mov %r11, %rcx ') + IFDOS(` sub $32, %rsp ') ASSERT(nz, `test $15, %rsp') CALL( mpn_invert_limb) + IFDOS(` add $32, %rsp ') pop %r11 pop %r10 diff -Nrc2 gmp-main.253deadf9fc8/mpn/x86_64/fat/fat_entry.asm gmp-main/mpn/x86_64/fat/fat_entry.asm *** gmp-main.253deadf9fc8/mpn/x86_64/fat/fat_entry.asm Tue Apr 25 20:52:18 2017 --- gmp-main/mpn/x86_64/fat/fat_entry.asm Tue Apr 25 20:52:18 2017 *************** *** 168,172 **** --- 168,174 ---- push %r9 push %rax + IFDOS(` sub $32, %rsp ') CALL( __gmpn_cpuvec_init) + IFDOS(` add $32, %rsp ') pop %rax pop %r9 diff -Nrc2 gmp-main.253deadf9fc8/mpn/x86_64/mod_1_1.asm gmp-main/mpn/x86_64/mod_1_1.asm *** gmp-main.253deadf9fc8/mpn/x86_64/mod_1_1.asm Tue Apr 25 20:52:18 2017 --- gmp-main/mpn/x86_64/mod_1_1.asm Tue Apr 25 20:52:18 2017 *************** *** 199,204 **** --- 199,206 ---- IFSTD(` mov %r12, %rdi ') C pass parameter IFDOS(` mov %r12, %rcx ') C pass parameter + IFDOS(` sub $32, %rsp ') ASSERT(nz, `test $15, %rsp') CALL( mpn_invert_limb) + IFDOS(` add $32, %rsp ') neg %r12 mov %r12, %r8 diff -Nrc2 gmp-main.253deadf9fc8/mpn/x86_64/mod_1_2.asm gmp-main/mpn/x86_64/mod_1_2.asm *** gmp-main.253deadf9fc8/mpn/x86_64/mod_1_2.asm Tue Apr 25 20:52:18 2017 --- gmp-main/mpn/x86_64/mod_1_2.asm Tue Apr 25 20:52:18 2017 *************** *** 184,189 **** --- 184,191 ---- IFSTD(` mov %r12, %rdi ') C pass parameter IFDOS(` mov %r12, %rcx ') C pass parameter + IFDOS(` sub $32, %rsp ') ASSERT(nz, `test $15, %rsp') CALL( mpn_invert_limb) + IFDOS(` add $32, %rsp ') mov %r12, %r8 mov %rax, %r11 diff -Nrc2 gmp-main.253deadf9fc8/mpn/x86_64/mod_1_4.asm gmp-main/mpn/x86_64/mod_1_4.asm *** gmp-main.253deadf9fc8/mpn/x86_64/mod_1_4.asm Tue Apr 25 20:52:18 2017 --- gmp-main/mpn/x86_64/mod_1_4.asm Tue Apr 25 20:52:18 2017 *************** *** 191,196 **** --- 191,198 ---- IFSTD(` mov %r12, %rdi ') C pass parameter IFDOS(` mov %r12, %rcx ') C pass parameter + IFDOS(` sub $32, %rsp ') ASSERT(nz, `test $15, %rsp') CALL( mpn_invert_limb) + IFDOS(` add $32, %rsp ') mov %r12, %r8 mov %rax, %r11 -- Torbjörn Please encrypt, key id 0xC8601622 _______________________________________________ gmp-bugs mailing list gmp-bugs@gmplib.org https://gmplib.org/mailman/listinfo/gmp-bugs