What is the correct way to run the tests which test rackunit? I am running the following after building with a clean tree:
$ racket/bin/raco test pkgs/rackunit-pkgs/rackunit-test/tests/rackunit This seems to show several "failures", though since it is testing rackunit I'm pretty sure most if not all are intentional (to verify that things which should fail do fail.) The full output is below [*]. What is not clear is how do people run this and determine whether they have any unintentional failures? Is there a separate command I should be using? Perhaps people just diff the output from a clean tree vs the output with their changes? BTW, I have read: http://blog.racket-lang.org/2012/11/tutorial-contributing-to-racket.html which has been very helpful since Joe's example was a change to rackunit, but I didn't see anything addressing this problem. For additional context: my motivation for running these tests is to improve check-exn so it no longer passes when given a non-thunk. I think it should raise an error instead. I already have a patch for this but running the tests has me thinking I'm not using the correct command to test rackunit. David [*] Here is the output from the above 'raco test ...' command, with stdout and stderr redirected to a file. Note the redirection changes the ordering of some lines compared to the terminal output. raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/all-rackunit-tests.rkt" raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/base-test.rkt" raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/check-info-test.rkt" raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/check-test.rkt" raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/counter-test.rkt" raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/format-test.rkt" raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/hash-monad-test.rkt" raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/location-test.rkt" raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/log.rkt" -------------------- FAILURE name: check-true location: (#<path:/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/log.rkt> 38 2 1185 15) expression: (check-true #f) params: (#f) Check failure -------------------- raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/monad-test.rkt" raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/pr10950.rkt" 1 test-------------------- ERROR Outta here! context...: /home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/private/check.rkt:133:29 /home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/private/check.rkt:58:0: top-level-check-around /home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/private/check.rkt:119:21: check-pred492750 /home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone-check-test.rkt: [running body] f88 loop loop loop f88 /home/dtp/src/dtp-racket/pkgs/compiler-pkgs/compiler-lib/compiler/commands/test.rkt: [running body] /home/dtp/src/dtp-racket/racket/collects/raco/raco.rkt: [running body] /home/dtp/src/dtp-racket/racket/collects/raco/main.rkt: [running body] -------------------- -------------------- FAILURE name: check location: (#<path:/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone-check-test.rkt> 44 0 1344 17) expression: (check = 1 2) params: (#<procedure:=> 1 2) message: 0.0 Check failure -------------------- -------------------- ERROR First Outta here! context...: /home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone-test-case-test.rkt: [running body] f88 loop loop loop f88 /home/dtp/src/dtp-racket/pkgs/compiler-pkgs/compiler-lib/compiler/commands/test.rkt: [running body] /home/dtp/src/dtp-racket/racket/collects/raco/raco.rkt: [running body] /home/dtp/src/dtp-racket/racket/collects/raco/main.rkt: [running body] -------------------- -------------------- error ERROR Second Outta here! context...: /home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone-test-case-test.rkt: [running body] f88 loop loop loop f88 /home/dtp/src/dtp-racket/pkgs/compiler-pkgs/compiler-lib/compiler/commands/test.rkt: [running body] /home/dtp/src/dtp-racket/racket/collects/raco/raco.rkt: [running body] /home/dtp/src/dtp-racket/racket/collects/raco/main.rkt: [running body] -------------------- -------------------- FAILURE actual: 1 expected: 2 name: check-eq? location: (#<path:/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone-test-case-test.rkt> 19 12 520 15) expression: (check-eq? 1 2) Check failure -------------------- -------------------- failure FAILURE actual: 1 expected: 2 name: check-eq? location: (#<path:/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone-test-case-test.rkt> 20 21 558 15) expression: (check-eq? 1 2) Check failure -------------------- -------------------- FAILURE actual: #"" expected: #"Oh HAI!\nI didn't run\n" name: check-equal? location: (#<path:/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone.rkt> 29 4 956 23) expression: (check-equal? cout #"Oh HAI!\nI didn't run\n") Check failure -------------------- -------------------- FAILURE actual: #"" expected: #"--------------------\nERROR\nOutta here!\n\n--------------------\n--------------------\nFAILURE\nname: check\nlocation: (#<path:PLTHOME/collects/tests/rackunit/standalone-check-test.rkt> 44 0 1344 17)\nexpression: (check = 1 2)\nparams: (#<procedure:=> 1 2)\nmessage: 0.0\n\nCheck failure\n--------------------\n" name: check-equal? location: (#<path:/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone.rkt> 30 4 984 23) expression: (check-equal? cerr #"--------------------\nERROR\nOutta here!\n\n--------------------\n--------------------\nFAILURE\nname: check\nlocation: (#<path:PLTHOME/collects/tests/rackunit/standalone-check-test.rkt> 44 0 1344 17)\nexpression: (check = 1 2)\nparams: (#<procedure:=> 1 2)\nmessage: 0.0\n\nCheck failure\n--------------------\n") Check failure -------------------- -------------------- FAILURE actual: #"" expected: #"--------------------\nERROR\nFirst Outta here!\n\n--------------------\n--------------------\nerror\nERROR\nSecond Outta here!\n\n--------------------\n--------------------\nFAILURE\nactual: 1\nexpected: 2\nname: check-eq?\nlocation: (#<path:PLTHOME/collects/tests/rackunit/standalone-test-case-test.rkt> 19 12 520 15)\nexpression: (check-eq? 1 2)\n\nCheck failure\n--------------------\n--------------------\nfailure\nFAILURE\nactual: 1\nexpected: 2\nname: check-eq?\nlocation: (#<path:PLTHOME/collects/tests/rackunit/standalone-test-case-test.rkt> 20 21 558 15)\nexpression: (check-eq? 1 2)\n\nCheck failure\n--------------------\n" name: check-equal? location: (#<path:/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone.rkt> 30 4 984 23) expression: (check-equal? cerr #"--------------------\nERROR\nFirst Outta here!\n\n--------------------\n--------------------\nerror\nERROR\nSecond Outta here!\n\n--------------------\n--------------------\nFAILURE\nactual: 1\nexpected: 2\nname: check-eq?\nlocation: (#<path:PLTHOME/collects/tests/rackunit/standalone-test-case-test.rkt> 19 12 520 15)\nexpression: (check-eq? 1 2)\n\nCheck failure\n--------------------\n--------------------\nfailure\nFAILURE\nactual: 1\nexpected: 2\nname: check-eq?\nlocation: (#<path:PLTHOME/collects/tests/rackunit/standalone-test-case-test.rkt> 20 21 558 15)\nexpression: (check-eq? 1 2)\n\nCheck failure\n--------------------\n") Check failure -------------------- passed raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/result-test.rkt" raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/run-tests.rkt" 127 success(es) 0 failure(s) 0 error(s) 127 test(s) run 0 -------------------- Failures > Intended to fail Intended to fail FAILURE name: fail location: pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/all-rackunit-tests.rkt:47:33 params: Check failure -------------------- -------------------- Failures > Also intended to fail Also intended to fail FAILURE name: check-eq? location: pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/all-rackunit-tests.rkt:48:38 actual: 'apples expected: 'orange Check failure -------------------- -------------------- Failures > Yet again intended to fail Yet again intended to fail FAILURE name: check-equal? location: pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/all-rackunit-tests.rkt:49:3 actual: "apples" expected: "oranges" Check failure -------------------- -------------------- Failures > Intended to throw error Intended to throw error ERROR testing: <<This is an error message>> context...: /home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/private/result.rkt:99:3 /home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/private/test-suite.rkt:28:2 the-tests /home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/private/test-suite.rkt:60:0: apply-test-suite /home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/text-ui.rkt:238:0: run-tests14 /home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/run-tests.rkt: [running body] f88 loop loop loop f88 /home/dtp/src/dtp-racket/pkgs/compiler-pkgs/compiler-lib/compiler/commands/test.rkt: [running body] /home/dtp/src/dtp-racket/racket/collects/raco/raco.rkt: [running body] /home/dtp/src/dtp-racket/racket/collects/raco/main.rkt: [running body] -------------------- -------------------- Failures > Error within a check Error within a check ERROR name: check location: pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/all-rackunit-tests.rkt:51:37 params: #<procedure:error> 'foo 'bar error: contract violation expected: string? given: 'bar argument position: 2nd other arguments...: 'foo context...: /home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/private/check.rkt:133:29 /home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/private/check.rkt:119:21: check491437 /home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/private/result.rkt:99:3 /home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/private/test-suite.rkt:28:2 the-tests /home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/private/test-suite.rkt:60:0: apply-test-suite /home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/text-ui.rkt:238:0: run-tests14 /home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/run-tests.rkt: [running body] f88 loop loop loop f88 /home/dtp/src/dtp-racket/pkgs/compiler-pkgs/compiler-lib/compiler/commands/test.rkt: [running body] /home/dtp/src/dtp-racket/racket/collects/raco/raco.rkt: [running body] /home/dtp/src/dtp-racket/racket/collects/raco/main.rkt: [running body] -------------------- 0 success(es) 3 failure(s) 2 error(s) 5 test(s) run 5 raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone-check-test.rkt" Oh HAI! I didn't run raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone-test-case-test.rkt" raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone.rkt" raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/test-case-test.rkt" raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/test-suite-test.rkt" raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/test-test.rkt" raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/text-ui-test.rkt" raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/text-ui-util-test.rkt" 55/340 test failures raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/tl.rkt" raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/util-test.rkt" _________________________ Racket Developers list: http://lists.racket-lang.org/dev
