New submission from Ned Deily:
If you run test_idle using the standard Python regression test runner,
regrtest, and use regrtest's -w option (to re-run failure test verbosely)
*without* using regrtest's -j option (to run tests in separate processes), any
real test failure triggering a rerun will cause a number of additional false
IDLE test case failures because some test cases modify global state in such a
way as to be not re-runable.
An example: at the moment there is a real IDLE test case failure (see
Issue27830). If I run test_idle with both -w and -j options:
$ /usr/local/bin/python3.6 -m test -w -j2 -uall test_idle
Run tests in parallel using 2 child processes
0:00:01 [1/1/1] test_idle failed
can't invoke "event" command: application has been destroyed
while executing
"event generate $w <<ThemeChanged>>"
(procedure "ttk::ThemeChanged" line 6)
invoked from within
"ttk::ThemeChanged"
test test_idle failed -- Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py",
line 389, in test_click_help_source
Equal(dialog.result, ('__test__', __file__))
AssertionError: Tuples differ: ('__test__',
'file:///Library/Frameworks/Python.framewo[58 chars].py') != ('__test__',
'/Library/Frameworks/Python.framework/Vers[51 chars].py')
First differing element 1:
'file:///Library/Frameworks/Python.framewo[57 chars]y.py'
'/Library/Frameworks/Python.framework/Vers[50 chars]y.py'
('__test__',
-
'file:///Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py')
? -------
+
'/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py')
1 test failed:
test_idle
Re-running failed tests in verbose mode
Re-running test 'test_idle' in verbose mode
test_autocomplete_event (idlelib.idle_test.test_autocomplete.AutoCompleteTest)
... ok
test_delayed_open_completions
(idlelib.idle_test.test_autocomplete.AutoCompleteTest) ... ok
[...]
test_shell_show (idlelib.idle_test.test_warning.ShellWarnTest) ... ok
test_showwarnings (idlelib.idle_test.test_warning.ShellWarnTest) ... ok
======================================================================
FAIL: test_click_help_source (idlelib.idle_test.test_query.HelpsourceGuiTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py",
line 389, in test_click_help_source
Equal(dialog.result, ('__test__', __file__))
AssertionError: Tuples differ: ('__test__',
'file:///Library/Frameworks/Python.framewo[58 chars].py') != ('__test__',
'/Library/Frameworks/Python.framework/Vers[51 chars].py')
First differing element 1:
'file:///Library/Frameworks/Python.framewo[57 chars]y.py'
'/Library/Frameworks/Python.framework/Vers[50 chars]y.py'
('__test__',
-
'file:///Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py')
? -------
+
'/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py')
----------------------------------------------------------------------
Ran 219 tests in 0.977s
FAILED (failures=1)
only the 1 valid test case failure shows up.
But if I run test_idle with just -w (no -j) options:
$ /usr/local/bin/python3.6 -m test -w -uall test_idle
Run tests sequentially
0:00:00 [1/1] test_idle
can't invoke "event" command: application has been destroyed
while executing
"event generate $w <<ThemeChanged>>"
(procedure "ttk::ThemeChanged" line 6)
invoked from within
"ttk::ThemeChanged"
test test_idle failed -- Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py",
line 389, in test_click_help_source
Equal(dialog.result, ('__test__', __file__))
AssertionError: Tuples differ: ('__test__',
'file:///Library/Frameworks/Python.framewo[58 chars].py') != ('__test__',
'/Library/Frameworks/Python.framework/Vers[51 chars].py')
First differing element 1:
'file:///Library/Frameworks/Python.framewo[57 chars]y.py'
'/Library/Frameworks/Python.framework/Vers[50 chars]y.py'
('__test__',
-
'file:///Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py')
? -------
+
'/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py')
test_idle failed
1 test failed:
test_idle
Re-running failed tests in verbose mode
Re-running test 'test_idle' in verbose mode
test_autocomplete_event (idlelib.idle_test.test_autocomplete.AutoCompleteTest)
... ok
test_delayed_open_completions
(idlelib.idle_test.test_autocomplete.AutoCompleteTest) ... ok
[...]
test_no_delete (idlelib.idle_test.test_text.TkTextTest) ... ok
test_init_modal (idlelib.idle_test.test_textview.TextViewTest) ... ERROR
test_init_nonmodal (idlelib.idle_test.test_textview.TextViewTest) ... ERROR
test_ok (idlelib.idle_test.test_textview.TextViewTest) ... ERROR
test_view_file (idlelib.idle_test.test_textview.ViewFunctionTest) ... ok
test_view_text (idlelib.idle_test.test_textview.ViewFunctionTest) ... ok
ERROR
test_init (idlelib.idle_test.test_tree.TreeTest) ... ok
[...]
test_showwarnings (idlelib.idle_test.test_warning.ShellWarnTest) ... ok
======================================================================
ERROR: test_init_modal (idlelib.idle_test.test_textview.TextViewTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_textview.py",
line 40, in setUp
TV.transient.__init__()
NameError: name 'TV' is not defined
======================================================================
ERROR: test_init_nonmodal (idlelib.idle_test.test_textview.TextViewTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_textview.py",
line 40, in setUp
TV.transient.__init__()
NameError: name 'TV' is not defined
======================================================================
ERROR: test_ok (idlelib.idle_test.test_textview.TextViewTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_textview.py",
line 40, in setUp
TV.transient.__init__()
NameError: name 'TV' is not defined
======================================================================
ERROR: tearDownModule (idlelib.idle_test.test_textview)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_textview.py",
line 26, in tearDownModule
del TV
NameError: name 'TV' is not defined
======================================================================
FAIL: test_click_help_source (idlelib.idle_test.test_query.HelpsourceGuiTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py",
line 389, in test_click_help_source
Equal(dialog.result, ('__test__', __file__))
AssertionError: Tuples differ: ('__test__',
'file:///Library/Frameworks/Python.framewo[58 chars].py') != ('__test__',
'/Library/Frameworks/Python.framework/Vers[51 chars].py')
First differing element 1:
'file:///Library/Frameworks/Python.framewo[57 chars]y.py'
'/Library/Frameworks/Python.framework/Vers[50 chars]y.py'
('__test__',
-
'file:///Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py')
? -------
+
'/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py')
----------------------------------------------------------------------
Ran 219 tests in 0.998s
FAILED (failures=1, errors=4)
now there are 4 or 5 errors in addition to the 1 failure.
The fifth false error is somewhat timing dependent, e.g. it shows up some of
the time while the other 4 always show up:
======================================================================
ERROR: setUpClass (idlelib.idle_test.test_autocomplete.AutoCompleteTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_autocomplete.py",
line 34, in setUpClass
macosx.setupApp(cls.root, None)
File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/macosx.py",
line 245, in setupApp
overrideRootMenu(root, flist)
File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/macosx.py",
line 215, in overrideRootMenu
del mainmenu.menudefs[-1][1][0]
IndexError: list assignment index out of range
It's not a critical problem but it can be very confusing if you have not run
into the problem before and can cause lost time trying to track down the false
errors.
----------
assignee: terry.reedy
components: IDLE
messages: 272200
nosy: ned.deily, terry.reedy
priority: low
severity: normal
status: open
title: some test_idle tests are not re-runnable, producing false failures with
regrtest -w option
versions: Python 3.5, Python 3.6
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue27714>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
