This thread got started due to the DIV15 problem on the Sphere Online Judge.
This is almost a "black box" test environment. Submit a program. If it generates the expected output you get "accepted". The other results are " compilation error", "time limit exceeded", " runtime error (followed by (SIGSEGV), (SIGABRT), (other), (NZEC))", or "wrong answer". If the program did not exceed the time limit you also get the execution time. (Some problems give problem-specific results but that does not apply to the DIV15 problem) In looking over the status list of submitted programs I noted cases where the results were "100", "102". I wonder what that means and how it was generated So there is a (limited) way to get information out of a program, and that is to use time.sleep() to control the execution time. But that costs one program submission for each thing one might want to test. That's how I determined there was a bad character in the input. So now I imagine creating an automated program submitter, so I could create a large number of test cases and an automated way to read the execution times off the results page. Eventually one might garner enough information. Of course that might get the attention of the site operators and lead to a change in how things are run to discourage automated testing. -- Bob Gailer 919-636-4239 Chapel Hill, NC _______________________________________________ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
