Public bug reported:
We've got a lot of tests that depend on how long the test takes to run. Tests can take a long time just because you have a slow or overloaded system, or maybe you're trying to step through it with the debugger. The tests that fail generally don't care about the time and aren't attempting to verify performance, but still require that the test run quickly enough. Tests shouldn't depend on the wall clock time, just like they shouldn't depend on any external factors. Here's an example of a failing test: keystone.tests.unit.test_auth.AuthWithRemoteUser.test_scoped_remote_authn ------------------------------------------------------------------------- Captured traceback: ~~~~~~~~~~~~~~~~~~~ Traceback (most recent call last): File "keystone/tests/unit/test_auth.py", line 741, in test_scoped_remote_authn enforce_audit_ids=False) File "keystone/tests/unit/test_auth.py", line 104, in assertEqualTokens timeutils.parse_isotime(b['access']['token']['expires'])) File "keystone/tests/unit/core.py", line 521, in assertCloseEnoughForGovernmentWork self.assertTrue(abs(a - b).seconds <= delta, msg) File "/home/jenkins/workspace/keystone/.tox/py27/local/lib/python2.7/site-packages/unittest2/case.py", line 678, in assertTrue raise self.failureException(msg) AssertionError: False is not true : 2015-06-11 13:34:46+00:00 != 2015-06-11 13:34:50+00:00 within 3 delta It took 4 seconds rather than 3. ** Affects: keystone Importance: Undecided Assignee: Brant Knudson (blk-u) Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to Keystone. https://bugs.launchpad.net/bugs/1464366 Title: unit tests fail based on wall clock time Status in OpenStack Identity (Keystone): New Bug description: We've got a lot of tests that depend on how long the test takes to run. Tests can take a long time just because you have a slow or overloaded system, or maybe you're trying to step through it with the debugger. The tests that fail generally don't care about the time and aren't attempting to verify performance, but still require that the test run quickly enough. Tests shouldn't depend on the wall clock time, just like they shouldn't depend on any external factors. Here's an example of a failing test: keystone.tests.unit.test_auth.AuthWithRemoteUser.test_scoped_remote_authn ------------------------------------------------------------------------- Captured traceback: ~~~~~~~~~~~~~~~~~~~ Traceback (most recent call last): File "keystone/tests/unit/test_auth.py", line 741, in test_scoped_remote_authn enforce_audit_ids=False) File "keystone/tests/unit/test_auth.py", line 104, in assertEqualTokens timeutils.parse_isotime(b['access']['token']['expires'])) File "keystone/tests/unit/core.py", line 521, in assertCloseEnoughForGovernmentWork self.assertTrue(abs(a - b).seconds <= delta, msg) File "/home/jenkins/workspace/keystone/.tox/py27/local/lib/python2.7/site-packages/unittest2/case.py", line 678, in assertTrue raise self.failureException(msg) AssertionError: False is not true : 2015-06-11 13:34:46+00:00 != 2015-06-11 13:34:50+00:00 within 3 delta It took 4 seconds rather than 3. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1464366/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp