[issue15274] Patch for issue 5765: stack overflow evaluating eval("()" * 30000)

[Andrea Griffini]

New submission from Andrea Griffini :

This is a fix for issue #5765: stack overflow evaluating eval("()" * 3)

The solution was to add two fields (recursion_depth and
recursion_limit) to the symbol table object and just increment and
check the depth in symtable_visit_expr raising a RuntimeError in case
the limit is exceeded.

The test case added also covers other similar cases (a.b.b.b.b.b...
and a[0][0][0][0])

There is no depth check in when visiting statement because I cannot
think to a way to nesting statements too much without getting other
errors before. If there is a way to do it, it would be trivial to also
cover that part.

The patch uses the current depth and current recursion limit but
multiplies them for a factor because I suppose that the amount of C
stack used by the compiler per recursion is smaller than the amount
used by the interpreter; the constant in the patch is 4. Using a
constant of 1 (i.e. just using the normal recursion limit) doesn't
allow a previously existing test about big expressions to pass.

--
files: compiler_recursion_limit_check.patch
keywords: patch
messages: 164839
nosy: ag6502
priority: normal
severity: normal
status: open
title: Patch for issue 5765: stack overflow evaluating eval("()" * 3)
Added file: 
http://bugs.python.org/file26292/compiler_recursion_limit_check.patch

___
Python tracker 

___diff -r d9c98730e2e8 Include/symtable.h
--- a/Include/symtable.hSat Jul 07 13:34:50 2012 +1000
+++ b/Include/symtable.hSat Jul 07 14:39:38 2012 +0200
@@ -30,6 +30,8 @@
 PyObject *st_private;   /* name of current class or NULL */
 PyFutureFeatures *st_future;/* module's future features that affect
the symbol table */
+int recursion_depth;/* current recursion depth */
+int recursion_limit;/* recursion limit */
 };
 
 typedef struct _symtable_entry {
diff -r d9c98730e2e8 Lib/test/test_compile.py
--- a/Lib/test/test_compile.py  Sat Jul 07 13:34:50 2012 +1000
+++ b/Lib/test/test_compile.py  Sat Jul 07 14:39:38 2012 +0200
@@ -474,6 +474,14 @@
 self.assertInvalidSingle('f()\nxy # blah\nblah()')
 self.assertInvalidSingle('x = 5 # comment\nx = 6\n')
 
+def test_compiler_recursion_limit(self):
+self.assertRaises(RuntimeError, self.compile_single, "()" * 10)
+self.assertRaises(RuntimeError, self.compile_single, "a" + ".b" * 
10)
+self.assertRaises(RuntimeError, self.compile_single, "a" + "[0]" * 
10)
+self.compile_single("()" * 2000)
+self.compile_single("a" + ".b" * 2000)
+self.compile_single("a" + "[0]" * 2000)
+
 def test_main():
 support.run_unittest(TestSpecifics)
 
diff -r d9c98730e2e8 Python/symtable.c
--- a/Python/symtable.c Sat Jul 07 13:34:50 2012 +1000
+++ b/Python/symtable.c Sat Jul 07 14:39:38 2012 +0200
@@ -218,17 +218,37 @@
 return NULL;
 }
 
+/* When compiling the use of C stack is probably going to be a lot
+   lighter than when executing Python code but still can overflow
+   and causing a Python crash if not checked (e.g. eval("()"*30)).
+   Using the current recursion limit for the compiler too seems
+   overconstraining so a factor is used to allow deeper recursion
+   when compiling an expression.
+*/
+#define COMPILER_STACK_FRAME_SCALE 4
+
 struct symtable *
 PySymtable_Build(mod_ty mod, const char *filename, PyFutureFeatures *future)
 {
 struct symtable *st = symtable_new();
 asdl_seq *seq;
 int i;
+PyThreadState *tstate;
 
 if (st == NULL)
 return st;
 st->st_filename = filename;
 st->st_future = future;
+
+/* Setup recursion depth check counters */
+tstate = PyThreadState_GET();
+if (!tstate) {
+PySymtable_Free(st);
+return NULL;
+}
+st->recursion_depth = tstate->recursion_depth * COMPILER_STACK_FRAME_SCALE;
+st->recursion_limit = Py_GetRecursionLimit() * COMPILER_STACK_FRAME_SCALE;
+
 /* Make the initial symbol information gathering pass */
 if (!GET_IDENTIFIER(top) ||
 !symtable_enter_block(st, top, ModuleBlock, (void *)mod, 0, 0)) {
@@ -1268,6 +1288,12 @@
 static int
 symtable_visit_expr(struct symtable *st, expr_ty e)
 {
+if (++st->recursion_depth > st->recursion_limit) {
+PyErr_SetString(PyExc_RuntimeError,
+"maximum recursion depth exceeded while compiling an 
expression");
+--st->recursion_depth;
+return 0;
+}
 switch (e->kind) {
 case BoolOp_kind:
 VISIT_SEQ(st, expr, e->v.BoolOp.values);
@@ -1280,8 +1306,10 @@
 VISIT(st, expr, e->v.UnaryOp.operand);
 break;
 case Lambda_kind: {
-if (!GET_IDENTIFIER(lambda))
+if (!GET_IDENTIFIER(lambda)) {
+--st->recursion_depth;
 return 0;
+}
 if (e->v.Lambda.a

[issue15274] Patch for issue 5765: stack overflow evaluating eval("()" * 30000)

[Andrea Griffini]

Andrea Griffini  added the comment:

I sent an email because I was not able to log in. The patch has been submitted 
to the correct issue (6765).

--
resolution:  -> duplicate
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com