> I think that it would make a lot of sense to run the hidden test > for the last submission that passed visible tests. And in the > example above it would be "Attempt 5". > > I mean, if the last submission happens to be broken and even > fails visible tests, then re-submitting some variant that at least > passed visible tests is a purely mechanical action for a contestant. > The GCJ website could still judge it automatically as a convenience > feature rather than not running *any* hidden tests at all. > > Anyone who passed visible tests should be eligible for exactly one > hidden test run. > > This is an interesting idea, but notice that the "equivalence" is not perfect. First, if Attempt 5 in your example was counted, you would have submitted Attempt 6 without any risk of increasing your penalty to get Hidden points, which is not how the current system works. If we made a "virtual" attempt that is a copy of Attempt 5 automatically instead of making you resubmit Attempt 5 manually, we could count both the real Attempt 5 and Attempt 6 for penalty like it happens now. We would need to assign a timestamp to that "virtual" attempt for penalty reasons. The end of the contest is a reasonable choice, since you can always make it earlier by really submitting if you want. However, the problem persist:
Suppose you do realize Attempt 6 fails and decide that resubmitting Attempt 5 is the right way to go. In the current system, you submit Attempt 5. In the proposed change, you don't, because it will be done automatically. Later, you discover a solution that is better than Attempt 5 and decide to submit it. In the current system, that will get you 7 penalty attempts (the 6 attempts in your example plus the resubmission of Attempt 5). If you didn't have to explicitly submit Attempt 5, that would get you 6 penalty attempts (the 6 in the example, since you never really submitted Attempt 5). Notice that at this point we have a scoring system that is much more complicated, with a virtual attempt and a fake timestamp, and still it's not equivalent (minus mechanical resubmission) than what we have right now. It would be equivalent to "at the very end of the contest when there's no time for anything else, go back and resubmit the last attempt that passed all visible tests for each problem". Either way, the improvement is pretty minor and it's not worth complicating a scoring system that it's already almost as hard to define as it is to implement. As you may have seen from other threads, we always choose the simplest (in terms of definition, not implementation) alternative unless the improvement of the complication is impactful. For example, choosing always the last Attempt would be simpler, but it would make you think really hard about every resubmission for fear of losing your hard-earned visible points that you know you have, so we complicate the rule a bit to avoid that uninteresting analysis being part of the equation. There are a lot of tweaks to the rule that would make it a little better in some corner cases or for some subset of cases, however, to improve those cases we have to make everyone read and understand the more complicated rule, which is a significant cost on its own for the user base in general (even if it may be negligible for some users in particular that like or don't mind reading and parsing the rules). Best, Pablo -- You received this message because you are subscribed to the Google Groups "Google Code Jam" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/google-code/CANa5jcBS1EYMtndJGwbBD%3Dss2MMErfZAXsyo6kVyrP1_9DrmRg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
