New submission from pdox <p...@alum.mit.edu>:

USE_STACKCHECK is a Windows-only feature which provides additional safety 
against C stack overflow by periodically calling PyOS_CheckStack to determine 
whether the current thread is too close to the end of the stack.

The way USE_STACKCHECK ensures that PyOS_CheckStack is called frequently is 
surprising. It does this by artificially decrementing _Py_CheckRecursionLimit 
with every call to Py_EnterRecursiveCall:

    #ifdef USE_STACKCHECK
    /* With USE_STACKCHECK, we artificially decrement the recursion limit in 
order
       to trigger regular stack checks in _Py_CheckRecursiveCall(), except if
       the "overflowed" flag is set, in which case we need the true value
       of _Py_CheckRecursionLimit for _Py_MakeEndRecCheck() to function 
properly.
    */
    #  define _Py_MakeRecCheck(x)  \
        (++(x) > (_Py_CheckRecursionLimit += PyThreadState_GET()->overflowed - 
1))
    #else
    #  define _Py_MakeRecCheck(x)  (++(x) > _Py_CheckRecursionLimit)
    #endif

_Py_CheckRecursionLimit defaults to 1000, and is constant when USE_STACKCHECK 
is off. (except when changed by Py_SetRecursionLimit)

With a single thread, each call to Py_EnterRecursiveCall causes 
_Py_CheckRecursionLimit to decrement by 1 until it equals the current recursion 
depth, at which point _Py_CheckRecursiveCall is triggered, resetting 
_Py_CheckRecursionLimit back to the default. This could be anywhere from 500 to 
1000 calls depending on the recursion depth. With multiple threads, the 
behavior may be more chaotic, as each thread will be decrementing 
_Py_CheckRecursionLimit independently.

I propose that instead, the call to PyOS_CheckStack is triggered in a 
predictable fashion, using a separate counter for each thread, so that it 
occurs every N'th call to Py_EnterRecursiveCall. (N = 64 seems reasonable, as 
PyOS_CheckStack guarantees 2048 * sizeof(void*) bytes remaining on the C stack).

----------
components: Windows
messages: 304854
nosy: paul.moore, pdox, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Make the behavior of USE_STACKCHECK deterministic
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue31857>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Reply via email to