Hi.

I am working on SUnit improvements. I open issue 19015
<https://pharo.fogbugz.com/f/cases/19015/Tests-should-never-hang-and-leave-forked-processes-live>.
Slice is inbox which waits your review and feedback.
I was trying to address three problems:

*1) Tests should never hang. They should be always executed within time
limit.*

I give them 10 minutes for now to not change existing behaviour of tests.
At next step it should be  really reduced to ~100 milliseconds (?).
Any TestCase could redefine time limit by method #defaultTimeLimit.
Or it could be specified directly in test code by
self timeLimit: 10 seconds
(could be changed at any time of test execution).

To implement this logic special watch dog process is running for given test
suite to control execution time of tests. It is single process for full
test suite.

*2) When test completes all forked processes should be terminated.*

If your tested code produced zombie processes SUnit will take care about
destroying all such garbage.
(it not means that you don't need to clean table in #tearDown but any code
could be broken and running tests should not produce dirty system).

*3) Any failures inside forked processes should not spawn debugger while
running tests.*
Only when we debug tests we need debugger on forked failed processes.
During normal run SUnit should prevent such "background debuggers" and mark
such tests as failed.

To implement this behaviour SUnit will handle errors from forked processes
by suspending them and collecting them in special collection.
I introduce TestFailedByForkedProcess error to signal these kind of
problems at the end of tests. This error is resumable and resume will opens
debuggers of suspended failures (in fact it will resume suspended
processes).
So to debug background failures you will need extra Proceed action on
debugger when TestFailedByForkedProcess is signalled.
But in normal run such tests will be just failed by
TestFailedByForkedProcess error.

*Now details on how it is done:*

I introduce special process specific variable CurrentExecutionEnvironment.
It is not installed by default and as default value it returns
DefaultExecutionEnvironment instance.
This variable is inheritable. If your install concrete environment into
process it will be installed to any child process.

So value of variable is instance of ExecutionEnvironment subclasses and you
can install it by:

anYourExecutionEnvironment beActiveDuring: aBlock


When block completes previous environment is restored.
For default environment there is class side method:

DefaultExecutionEnvironment beActiveDuring: aBlock


And to reset current environment to default:

DefaultExecutionEnvironment beActive.


SUnit introduces TestExecutionEnvironment which implements all described
behaviour for time limits and forked processes.
To activate environment there is new method #runCaseManaged. And submitted
slice uses it instead of simple runCase.

TestCase>>runCaseManaged
CurrentExecutionEnvironment runTestCase: self


DefaultExecutionEnvironment will install new TestExecutionEnvironment and
delegate processing to it. And if TestExecutionEnvironment is already
active it will just process new test case.

Now monkey has problem in checking slice (annoying timeout for loading).
So I can't see real system impact. But it should not stop you from review
:))
I think it is very important features for all our tests. And environment
idea will lead to very interesting future.

Best regards,
Denis

Reply via email to