Hi Dom and Mike, I have made multiple attempts to debug and resolve the flakiness of these tests throughout the development of this feature, and that effort is still in progress.
As pointed out, we do have tests, but landing them as enabled has been challenging due to long-standing flakiness issues. I believe Adam Rice has also attempted to solve this flakiness, as seen in the CRBug: Timeouts in ReportingBrowserTest.CrashReport* browser tests [355141780] - Chromium <https://issues.chromium.org/issues/355141780> but later had to disable <https://chromium-review.googlesource.com/c/chromium/src/+/5961957> the test. Enabling these tests for good is indeed a priority. However, I don't believe this should block the I2S. We will continue to work on stabilizing the tests and aim to have them enabled as soon as possible. On Thursday, March 27, 2025 at 1:33:35 PM UTC-7 mike...@chromium.org wrote: > Thanks Dom - that's not a great scenario that I didn't understand when > approving. > > Issack, what is the plan for tests? Are they in progress, or should we > unship/not ship the feature until they're ready? > On 3/27/25 4:26 PM, Dominic Farolino wrote: > > Non API OWNER here, but when looking through this feature I noticed that > there are no tests for it. This line > <https://source.chromium.org/chromium/chromium/src/+/main:chrome/browser/net/reporting_browsertest.cc;l=604-616;drc=2fc66f9ead0532c2b2eb05bd99d16259a6976e09> > in > reporting_browsertest.cc disables all tests that were once just marked as > flaky. > > Adam Rice pointed this out in the original review here > <https://chromium-review.googlesource.com/c/chromium/src/+/4959773/comment/56b13d2a_3b9d390f/>, > > and it looks like we tried a follow-up with CL 5422052 > <https://chromium-review.googlesource.com/c/chromium/src/+/5422052> to > enable the tests, but this was reverted in CL 5433594 > <https://chromium-review.googlesource.com/c/chromium/src/+/5433594> with > no obvious attempts to re-enable them. I only raise this because this means > there are no web platform tests, *and* no Chromium-specific browser > tests, so the feature is not tested at all. Maybe that shouldn't block the > I2S, but enabling the tests for good should be a priority, ideally before > hitting stable. Please let me know if I'm missing something! > > On Wed, Mar 19, 2025 at 2:10 PM 'Issack John' via blink-dev < > blin...@chromium.org> wrote: > >> Thank you all for the reviews! :) >> >> On Tuesday, March 18, 2025 at 7:32:56 PM UTC-7 vmp...@chromium.org wrote: >> >>> LGTM3 >>> >>> On Tuesday, March 18, 2025 at 8:47:30 AM UTC-4 Mike Taylor wrote: >>> >>>> LGTM2 - I see that Mozilla has proposed a positive position via >>>> comment. >>>> On 3/17/25 2:25 PM, 'Dan Clark' via blink-dev wrote: >>>> >>>> LGTM1 >>>> >>>> On Monday, March 17, 2025 at 11:24:06 AM UTC-7 sligh...@chromium.org >>>> wrote: >>>> >>>>> I'm recused on this one, but would like to see it move forward >>>>> quickly. Great work. >>>>> >>>>> On Friday, March 14, 2025 at 1:32:38 PM UTC-7 Chromestatus wrote: >>>>> >>>>>> Contact emails issac...@microsoft.com, seth.b...@microsoft.com, >>>>>> icle...@google.com >>>>>> >>>>>> Explainer >>>>>> https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/main/CrashReporting/AddStackToCrashReports.md >>>>>> >>>>>> https://github.com/WICG/crash-reporting/issues/12 >>>>>> >>>>>> Specification https://wicg.github.io/crash-reporting >>>>>> >>>>>> Design docs >>>>>> >>>>>> https://docs.google.com/document/d/19DpvHIiYbmB9wgIP0BdI4vOnfVLcAZFmfIAml7SqRQA/edit?usp=sharing >>>>>> >>>>>> >>>>>> Summary >>>>>> >>>>>> This feature captures the JS call stack when a web page becomes >>>>>> unresponsive due to JavaScript code running an infinite loop or other >>>>>> very >>>>>> long computation. This helps developers to identify the cause of the >>>>>> unresponsiveness and fix it more easily. The JS call stack is included >>>>>> in >>>>>> the crash reporting API when the reason is unresponsive. >>>>>> >>>>>> >>>>>> Blink component Chromium > Internals > Network > ReportingAndNEL >>>>>> <https://issues.chromium.org/issues?q=customfield1222907:%22Chromium%20%3E%20Internals%20%3E%20Network%20%3E%20ReportingAndNEL%22> >>>>>> >>>>>> >>>>>> TAG review https://github.com/w3ctag/design-reviews/issues/981 >>>>>> >>>>>> TAG review status Issues addressed >>>>>> >>>>>> Origin Trial Name Call stacks in crash reports >>>>>> >>>>>> Chromium Trial Name DocumentPolicyIncludeJSCallStacksInCrashReports >>>>>> >>>>>> Origin Trial documentation link >>>>>> https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/main/CrashReporting/AddStackToCrashReports.md >>>>>> >>>>>> >>>>>> WebFeature UseCounter name >>>>>> kDocumentPolicyIncludeJSCallStacksInCrashReports >>>>>> >>>>>> Risks >>>>>> >>>>>> >>>>>> Interoperability and Compatibility >>>>>> >>>>>> "The stack trace format itself is not compatible across browsers." >>>>>> However, "It is already exposed throughout the web platform (via the >>>>>> `error.stack` getter), and there is already a lot of software, both >>>>>> client- >>>>>> and server-side, which deals with parsing the different browsers' >>>>>> formats." >>>>>> >>>>>> >>>>>> *Gecko*: No signal ( >>>>>> https://github.com/mozilla/standards-positions/issues/1057) >>>>>> >>>>>> *WebKit*: No signal ( >>>>>> https://github.com/WebKit/standards-positions/issues/380) >>>>>> >>>>>> *Web developers*: Positive >>>>>> https://github.com/denoland/deno/issues/26919#issuecomment-2584648202 >>>>>> https://github.com/WICG/crash-reporting/issues/21 >>>>>> https://github.com/electron/electron/issues/45356#issuecomment-2635086589 >>>>>> >>>>>> https://github.com/electron/electron/pull/44204#discussion_r1845994904 >>>>>> https://github.com/TurboWarp/desktop/issues/1113 >>>>>> https://github.com/getsentry/sentry-javascript/pull/14044#issuecomment-2468480434 >>>>>> >>>>>> https://www.electronjs.org/blog/electron-34-0#unresponsive-renderer-javascript-call-stacks >>>>>> >>>>>> >>>>>> *Other signals*: Origin trial participants have found the feature >>>>>> useful, and their feedback has been positive. >>>>>> >>>>>> Security >>>>>> >>>>>> Stack frames from cross-domain scripts that were not loaded with CORS >>>>>> are omitted. >>>>>> >>>>>> >>>>>> WebView application risks >>>>>> >>>>>> Does this intent deprecate or change behavior of existing APIs, such >>>>>> that it has potentially high risk for Android WebView-based applications? >>>>>> >>>>>> No, the feature does not deprecate or change the behavior of existing >>>>>> APIs such that it has potentially high risk for Android WebView-based >>>>>> applications. >>>>>> >>>>>> >>>>>> Debuggability >>>>>> >>>>>> Developers can launch DevTools, go to the "Application" Tab, then in >>>>>> the "Background services" section click on "Reporting API" where they >>>>>> can >>>>>> inspect reports that are queued to be sent. Application --> Reporting API >>>>>> >>>>>> >>>>>> Will this feature be supported on all six Blink platforms (Windows, >>>>>> Mac, Linux, ChromeOS, Android, and Android WebView)? Yes >>>>>> >>>>>> Is this feature fully tested by web-platform-tests >>>>>> <https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md> >>>>>> ? No >>>>>> >>>>>> This feature is not currently testable on WPT, since triggering it >>>>>> requires crashing the browser. >>>>>> >>>>>> >>>>>> DevTrial instructions >>>>>> https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/main/CrashReporting/HOWTO.md >>>>>> >>>>>> >>>>>> Flag name on about://flags >>>>>> >>>>>> Finch feature name DocumentPolicyIncludeJSCallStacksInCrashReports >>>>>> >>>>>> Requires code in //chrome? False >>>>>> >>>>>> Tracking bug >>>>>> https://bugs.chromium.org/p/chromium/issues/detail?id=1445539 >>>>>> >>>>>> Availability expectation Initially the feature will only be >>>>>> available in Chromium browsers. >>>>>> >>>>>> Non-OSS dependencies >>>>>> >>>>>> Does the feature depend on any code or APIs outside the Chromium open >>>>>> source repository and its open-source dependencies to function? >>>>>> No. >>>>>> >>>>>> Estimated milestones >>>>>> Shipping on desktop 136 >>>>>> Origin trial desktop first 127 >>>>>> Origin trial desktop last 132 >>>>>> Origin trial extension 1 end milestone 135 >>>>>> DevTrial on desktop 125 >>>>>> Shipping on Android 136 >>>>>> Origin trial Android first 127 >>>>>> Origin trial Android last 132 >>>>>> DevTrial on Android 125 >>>>>> Shipping on WebView 136 >>>>>> Origin trial WebView first 127 >>>>>> Origin trial WebView last 132 >>>>>> >>>>>> Anticipated spec changes >>>>>> >>>>>> Open questions about a feature may be a source of future web compat >>>>>> or interop issues. Please list open issues (e.g. links to known github >>>>>> issues in the project for the feature specification) whose resolution >>>>>> may >>>>>> introduce web compat/interop risk (e.g., changing to naming or structure >>>>>> of >>>>>> the API in a non-backward-compatible way). >>>>>> None >>>>>> >>>>>> Link to entry on the Chrome Platform Status >>>>>> https://chromestatus.com/feature/4731248572628992?gate=5150803040141312 >>>>>> >>>>>> Links to previous Intent discussions Intent to Prototype: >>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/MW2PPF6784DDB763E2DA7BFC75AE51613ABC27B2%40MW2PPF6784DDB76.namprd00.prod.outlook.com >>>>>> >>>>>> Ready for Trial: >>>>>> https://groups.google.com/a/chromium.org/g/blink-dev/c/aVpeVRO1Sy4 >>>>>> Intent to Experiment: >>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/f43b4e9f-5af8-4c4b-bcd9-342332dd57edn%40chromium.org >>>>>> >>>>>> Intent to Extend Experiment 1: >>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/67819b5d.2b0a0220.275672.09ce.GAE%40google.com >>>>>> >>>>>> >>>>>> >>>>>> This intent message was generated by Chrome Platform Status >>>>>> <https://chromestatus.com>. >>>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "blink-dev" group. >>>> >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to blink-dev+...@chromium.org. >>>> >>>> >>>> To view this discussion visit >>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/1f369b7c-91f2-4074-a902-1cefdeaa7dfbn%40chromium.org >>>> >>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/1f369b7c-91f2-4074-a902-1cefdeaa7dfbn%40chromium.org?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> -- >> You received this message because you are subscribed to the Google Groups >> "blink-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to blink-dev+...@chromium.org. >> To view this discussion visit >> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/ca68fda3-fe34-4485-836b-80615059c0ebn%40chromium.org >> >> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/ca68fda3-fe34-4485-836b-80615059c0ebn%40chromium.org?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/e9ccccb6-f655-42b3-8e57-e3789a2fe6den%40chromium.org.
