Re: Unsigned integer overflow in `toom_eval_pm2.c`

2023-09-04 Thread Andrew Teylu
On Sun, Sep 3, 2023 at 11:40 PM wrote: > I attach a possible patch. > As an outsider, does it make sense to also change instances of "neg" to be "sign" (if that's the interpretation) for consistency? For someone who isn't literate in the GMP code, neg vs. sign seems to be used slightly inconsist

Re: Unsigned integer overflow in `toom_eval_pm2.c`

2023-09-04 Thread marco . bodrato
Ciao, 4 set 2023, 08:02 da ni...@lysator.liu.se: > As Vincent suggested, it would be good to document somewhere what the > convention is (values 0 or ~0). > Would a comment "It returns 0 or ~0, depending on the sign of the result" added to all the mpn_toom_eval_ functions suffice? > And maybe

Re: Unsigned integer overflow in `toom_eval_pm2.c`

2023-09-04 Thread Niels Möller
marco.bodr...@tutanota.com writes: > Would a comment "It returns 0 or ~0, depending on the sign of the result" > added to all the mpn_toom_eval_ functions suffice? Yes. It would be nice if there were some reasonable place to document that only once, but don't know where that could be. > From the

Re: Unsigned integer overflow in `toom_eval_pm2.c`

2023-09-04 Thread Vincent Lefevre
On 2023-09-04 09:52:23 +0200, marco.bodr...@tutanota.com wrote: > Should the value ~0 be written as ~0U to be correctly assigned to an > unsigned? IMHO, this would be better as this would make the signedness of the types more consistent (the goal being to have an unsigned int for all these masks).