[issue31619] Strange error when convert hexadecimal with underscores to int

Change by Serhiy Storchaka : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker

[issue31619] Strange error when convert hexadecimal with underscores to int

Serhiy Storchaka added the comment: New changeset 30a6bc842945e3e9c9c7db887ab495c428ec7074 by Serhiy Storchaka (Miss Islington (bot)) in branch '3.6': bpo-31619: Fixed integer overflow in converting huge strings to int. (GH-3884) (#4690)

[issue31619] Strange error when convert hexadecimal with underscores to int

Change by Roundup Robot : -- pull_requests: +4603 ___ Python tracker ___

[issue31619] Strange error when convert hexadecimal with underscores to int

Serhiy Storchaka added the comment: New changeset 29ba688034fc4eef0693b86002cf7bee55d692af by Serhiy Storchaka in branch 'master': bpo-31619: Fixed integer overflow in converting huge strings to int. (#3884)

[issue31619] Strange error when convert hexadecimal with underscores to int

Serhiy Storchaka added the comment: I missed the issue mentioned by Mark msg303331. I looked at the type of n, and okay, it is Py_ssize_t. But I forgot to check the type of digits and bits_per_char, and it is int. My fix doesn't guard from integer overflow. The

[issue31619] Strange error when convert hexadecimal with underscores to int

Change by Serhiy Storchaka : -- pull_requests: +3857 ___ Python tracker ___ ___

[issue31619] Strange error when convert hexadecimal with underscores to int

Serhiy Storchaka added the comment: New changeset b5a630f3dd30ed628e088efe7523e650087adba2 by Serhiy Storchaka (Miss Islington (bot)) in branch '3.6': [3.6] bpo-31619: Fixed a ValueError when convert a string with large number of underscores (GH-3827) (#3863)

[issue31619] Strange error when convert hexadecimal with underscores to int

Change by Roundup Robot : -- pull_requests: +3843 ___ Python tracker ___

[issue31619] Strange error when convert hexadecimal with underscores to int

Serhiy Storchaka added the comment: New changeset 85c0b8941f0c8ef3ed787c9d504712c6ad3eb5d3 by Serhiy Storchaka in branch 'master': bpo-31619: Fixed a ValueError when convert a string with large number of underscores (#3827)

[issue31619] Strange error when convert hexadecimal with underscores to int

Stefan Krah added the comment: > We see if digits * bits_per_char + PyLong_SHIFT -1 overflows an int? Yes, but the check is too late: UB can already occur in this calculation and then all bets are off. It looks like 'n' was Py_ssize_t in 2.7. :) -- nosy: +skrah

[issue31619] Strange error when convert hexadecimal with underscores to int

Serhiy Storchaka added the comment: The error is raised when the number of underscores is at least (PyLong_SHIFT - 1) // bits_per_char. -- ___ Python tracker

[issue31619] Strange error when convert hexadecimal with underscores to int

Change by Serhiy Storchaka : -- pull_requests: +3811 ___ Python tracker ___ ___

[issue31619] Strange error when convert hexadecimal with underscores to int

Nitish added the comment: >> PR 3816 fixes the symptom, but not the core issue -- an overflow check >> depending on undefined behaviour. > I don't understand this check completely actually. When exactly is an int too > large to convert? We see if digits *

[issue31619] Strange error when convert hexadecimal with underscores to int

Nitish added the comment: > PR 3816 fixes the symptom, but not the core issue -- an overflow check > depending on undefined behaviour. I don't understand this check completely actually. When exactly is an int too large to convert? --

[issue31619] Strange error when convert hexadecimal with underscores to int

Mark Dickinson added the comment: There's also the (lesser) issue that we're using C `int`s for things that should really be `ssize_t`s. But that can be fixed / changed separately for the fix for this issue. > I'll see if I can find time for a fix this evening (UTC+1).

[issue31619] Strange error when convert hexadecimal with underscores to int

Serhiy Storchaka added the comment: PR 3816 fixes the symptom, but not the core issue -- an overflow check depending on undefined behaviour. -- ___ Python tracker

[issue31619] Strange error when convert hexadecimal with underscores to int

Nitish added the comment: The problem is with the following check: n = digits * bits_per_char + PyLong_SHIFT - 1; if (n / bits_per_char < p - start) { PyErr_SetString(PyExc_ValueError, "int string too large to convert");

[issue31619] Strange error when convert hexadecimal with underscores to int

Change by Nitish : -- keywords: +patch pull_requests: +3800 stage: -> patch review ___ Python tracker ___

[issue31619] Strange error when convert hexadecimal with underscores to int

Mark Dickinson added the comment: Looks like an overflow check that was questionable in the first place (depending on undefined behaviour), and is now both questionable and wrong. I'll see if I can find time for a fix this evening (UTC+1). --

[issue31619] Strange error when convert hexadecimal with underscores to int

New submission from Serhiy Storchaka : >>> int('1_2_3_4_5_6_7_89', 16) 4886718345 >>> int('1_2_3_4_5_6_7_8_9', 16) Traceback (most recent call last): File "", line 1, in ValueError: int string too large to convert -- components: Interpreter Core messages: