Many thanks to everyone who replied to my request for advice. I've now got a number of references to have a look at.
Here's another for the list... BOSS <http://boss.org.uk/> is the Online Submission System developed and used at the Department of Computer Science at the University of Warwick, a project originally dating from 1993 managed by Dr Mike Joy. It is written in Java and can perform automated tests on submissions. It used to set limits on CPU time to deal with student submissions containing unbounded loops -- I think this is still the case.
The testing was originally done by comparing program output (from both stdout and stderr) with the expected output for a given set of inputs. This can of course reveal mistakes in students' spelling and capitalisation as well as more "genuine" errors. The module manager can enable the students to run a subset of the tests themselves before submission to help with this.
I believe the system now also implements testing of submissions written in Java by calling methods directly, giving input and comparing the returned value with that expected. This should eliminate some of the problems introduced by comparing textual IO, although I should think moving away further away from something directly observed by the tester would also introduce problems with comprehension of test results.
BOSS also now includes facilities for plagiarism detection and calculation of code metrics. It has a (retroactively fitted) web interface. It has a GPL license and is a project at SourceForge.net.
Mike Joy <http://www.warwick.ac.uk/M.S.Joy/> is really the expert on all this -- I've just been lurking on the PPIG list for several years and thought this was an appropriate moment to say hello!
Dr Ashley Ward (a recent PhD graduate from Warwick: supervisor Dr Meurig Beynon)
---------------------------------------------------------------------- PPIG Discuss List ([EMAIL PROTECTED]) Discuss admin: http://limitlessmail.net/mailman/listinfo/discuss Announce admin: http://limitlessmail.net/mailman/listinfo/announce PPIG Discuss archive: http://www.mail-archive.com/discuss%40ppig.org/
