TLDR: Site scout is a tool by the fuzzing team that reports assertion failures and crashes detected when visiting popular websites. Bugs reported via site scout are real-world stability problems which likely affect many users. We encourage you to debug and fix site-scouts bugs as soon as possible as live site changes can quickly invalidate the trace. While we can always provide backtraces for a bug, it’s not entirely possible to isolate those into a test case. Please tell us if and how we can improve things at https://github.com/MozillaSecurity/site-scout/issues.
Site scout is a testing tool that visits popular web sites with the primary goal to detect issues triggered by content consumed by end users. Web sites are provided via a list of URLs that is curated and maintained independently of the tool itself. Unique lists can be created to align with the focus of the deployment and multiple deployments (top crasher, top sites, etc) can be run in parallel. Many components are made up of existing fuzzing tools: To help detect issues web sites are loaded using a range of debugging tools and build types on multiple platforms. This includes ASan + UBSan builds, TSan builds <https://firefox-source-docs.mozilla.org/tools/sanitizer/index.html> and debug builds on x86_64 and x86. It is currently running on Windows and Linux. All issues detected are reported to and triaged by the fuzzing team, such that reported issues are easily mapped to issues reported by fuzzing. The two approaches complement each other well: Fuzzing can find test cases that are then bisected. Live testing detects issues encountered by end users. Live site testing can also help expose issues that are missed by existing tests and fuzzing. Once we have identified a truly new bug, it will be filed in Bugzilla with as many details as possible (stack trace, URL, pernosco session), just like any other bug filed by the fuzzing team. So far, site scout has found 30 bugs <https://bugzilla.mozilla.org/show_bug.cgi?id=site-scout>, many of which were assertion failures exposing behavior that we think was unlikely or impossible to occur in the real world. Most of those bugs are real crashes and therefore clear stability problems that our users face when visiting these websites. We also found some real high-severity security bugs. And all that, just by visiting popular sites. Fixing bugs identified by site-scout will help maintain the good security and stability of our browser. While we know that some of the newly found issues are hard to test and reproduce due to changing web content, we want to emphasize that these are real-world stability problems that users are facing right now and deserve the appropriate attention. Site scout is still young and in active development. Please tell us if things should work differently by filing an issue at https://github.com/MozillaSecurity/site-scout/issues Thanks! [1] meta bug for site scout issues https://bugzilla.mozilla.org/show_bug.cgi?id=site-scout [2] The project code is available here: https://github.com/MozillaSecurity/site-scout -- You received this message because you are subscribed to the Google Groups "[email protected]" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/mozilla.org/d/msgid/dev-platform/CAOGW_n4ZaQ1RbQ3OQgmVHN801RZxKavq5TE-YwO5hHNUBKJrzg%40mail.gmail.com.
