I think the current behavior of check-expect is unfortunate w.r.t. errors that 
occur during the evaluation of the "expected" values.

Consider this program:

(check-expect (+ 3 4) 7)
(check-expect (* 4 5) 20)
(check-expect (+ 4 9) (/ 1 0))
(check-expect (+ 2 3) 5)

It produces this output in intermediate:

/: division by zero
Both tests passed!

In the example given, it's pretty clear that something major went wrong.  If 
you have thirty or forty test cases, though, this can look a lot like success.

I propose one of the following two solutions; I think the first is less work 
than the second.

1) keep a flag around during evaluation that records whether an exception has 
occurred during the evaluation of an 'expected' value, and update the report 
accordingly, e.g.:

2 tests passed. Some tests failed to run!

(could be better, I know.)

2) Record this as a test failure, and continue running the tests. This would 
mean adding a new kind of test failure message. Actually, this might not be too 
much work, either.

Thoughts?

John



Attachment: smime.p7s
Description: S/MIME cryptographic signature

_________________________
  Racket Developers list:
  http://lists.racket-lang.org/dev

Reply via email to