On Tuesday, November 14, 2023 at 8:12:41 AM UTC+1 Yoav Weiss wrote:
On Mon, Nov 13, 2023 at 4:04 PM Henrik Boström <h...@chromium.org> wrote: Contact emails h...@chromium.org, o...@chromium.org, h...@chromium.org Specification https://w3c.github.io/mediacapture-extensions/#the- mediastreamtrackaudiostats-interface Summary The `track.stats` API allows an application to measure quality related to the capturing of a MediaStreamTrack (getUserMedia). This API has already shipped for video tracks (Chrome Status <https://chromestatus.com/feature/5087376775053312>, Intent to Ship <https://groups.google.com/a/chromium.org/g/blink-dev/c/ttzYv-30gY4>). This intent relates to the audio version of the same API, which has similar metrics plus input latency metrics. The spec is actively under development. It currently contains frame counters <https://w3c.github.io/mediacapture-extensions/#the-mediastreamtrackaudiostats-interface> (like the video counterpart) which in the audio case allows calculating ratio of dropped audio which is a measure of capture glitches. There is also a PR in review <https://github.com/w3c/mediacapture-extensions/pull/124> which will add current input latency and a follow-up issue to add min/max/avg latency <https://github.com/w3c/mediacapture-extensions/issues/128>. Blink component Blink>GetUserMedia <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EGetUserMedia> Motivation The motivation is similar to that of video stats <https://github.com/mozilla/standards-positions/issues/895>, but this time it is audio related. Quality measurements are important to understand user reports that app gets (e.g. in Google Meet, users may file bugs containing quality dumps) and A/B experiments to understand how features impact quality (e.g. adding heavy video processing in an app may impact audio quality). Latency may be useful for audio processing. Together with WebRTC metrics, capture metrics help provide context as to which parts of the pipeline are contributing to quality in what way. TAG review status N/A small addition to existing spec and the `track.stats` API shape has already shipped for video. Risks Interoperability and Compatibility Risk is relatively small since this is a stats API. The MediaStreamTrack functions whether or not you can measure quality related properties of the track. *Gecko*: No signal *WebKit*: No signal Have we reached out? The spec is still being worked on and I have not officially reached out, but representatives from both Mozilla and Apple are part of the WG discussions. On the video stats side (same API shape but not the same metrics), Mozilla was positive <https://github.com/mozilla/standards-positions/issues/895> and Apple was unresponsive <https://github.com/WebKit/standards-positions/issues/260>. At the moment, there is a disagreement about whether or not frame dropping is worth measuring <https://github.com/w3c/mediacapture-extensions/issues/129> but we are in agreement - at least Google and Mozilla - that capture input latency is something we want to have, but the PR needs more work to get the definition right before it can be merged. This sends both positive and negative signals, but I do not have an official statement yet. *Web developers*: Positive Any links? I don't have any links but I've been told that web developers have been "asking for input latency metrics for years". On behalf of Google, Google Meet web developers are asking for both dropped frames ratio metrics and input latency metrics, so I am developing these PRs together with the WebRTC audio experts who requested them. *Other signals*: WebView application risks None Debuggability None Is this feature fully tested by web-platform-tests <https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md> ? Test coverage will be added as part of implementation, including which metrics are supported by the browser. In addition to WPTs, correctness of quality metrics may require browser tests e.g. for fake devices. Link to entry on the Chrome Platform Status https://chromestatus.com/feature/5141112910249984 -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/ chromium.org/d/msgid/blink-dev/bb6c1af3-9eb3-4c6f-a136- dee709b7f906n%40chromium.org <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/bb6c1af3-9eb3-4c6f-a136-dee709b7f906n%40chromium.org?utm_medium=email&utm_source=footer> . -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/d404e904-3a76-43e3-b9ba-c5e27d26d490n%40chromium.org.
