On Wed, Oct 18, 2017 at 10:28 AM, David Teller <dtel...@mozilla.com> wrote:
> Hi everyone, > > Yesterday, Nightly was broken on Linux and MacOS because of a typo in > JS code [1]. If I understand correctly, this triggered the usual > "undefined is not a function", which was > > 1/ uncaught during testing, as these things often are; > 2/ basically impossible to diagnose in the wild, because there was no > error message of any kind. > > I remember that we had bugs of this kind lurking for years in our > codebase, in code that was triggered daily and that everybody believed > to be tested. > > I'd like to think that there is a better way to handle these bugs, > without waiting for them to explode in our user's face. Opening this > thread to see if we can find a way to somehow "solve" these bugs, either > by making them impossible, or by making them much easier to solve. > > I have one proposal. Could we change the behavior of the JS VM as follows? > > - The changes affect only Nightly. > - The changes affect only mozilla chrome code (including system add-ons > but not user add-ons or test code). > - Any programmer error (e.g. SyntaxError) causes a crash a crash that > displays (and attaches to the CrashReporter) both the JS stack in and > the native stack. > - Any SyntaxError is a programmer error. > - Any TypeError is a programmer error. > > I expect that this will find a number of lurking errorsy, so we may want > to migrate code progressively, using a directive, say "use strict > moz-platform" and static analysis to help encourage using this directive. > > What do you think? > I agree that errors like this should have better visibility in order to help catch bugs. I'm not sure changing behavior of the JS VM is the proper layer to accomplish this. I think reporting messages from the JS console is a better place to start. We could change the test harnesses to fail tests if certain errors (like SyntaxError or TypeError) are raised. If there is a "hook" in the JS VM to catch said errors at error time, we could have the test harnesses run Firefox in a mode that makes said errors more fatal (even potentially crashing as you suggest) and/or included additional metadata, such as stacks. Another idea would be to require all non-log output in the JS console to be accounted for. Kinda like reftest's expected assertion count. Assuming most JS errors/warnings are unwanted, this would allow us to fail all tests reporting JS errors/warnings while allowing wanted/known failures to not fail the test. A problem though is background services "randomly" injecting their output during test execution depending on non-deterministic timing. It could be difficult to roll this out in practice. But it feels like we should be able to filter out messages or stacks accordingly. But why stop there? Shouldn't Firefox installs in the wild report JS errors and warnings in Firefox code back to Mozilla (like we do crashes)? I know this has been discussed. I'm not sure what we're doing/planning about it though. _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
