Hello all, We landed the metric collection CL (https://crrev.com/c/3413794) and have gathered some data:
# Notable latency range - Android Chrome: Mean 40.96, Peak 3~5ms - Chrome OS: Mean 46.37, Peak at 15ms - Mac OS X: Mean 935.36, Peak at 17ms - Linux (dev): Mean 45.32, Peak at 33ms - Windows: Mean 31.77, Peak at 31ms More details can be found here: go/webaudio-outputlatency-metric-collection (sorry, googler-only) # Thoughts - Notable peaks roughly match the platform. - Mac OS X has a diverse distribution and there is a small peak at 200ms on Mac OSX: it may indicate that the client is using the “AirPods”. (~1%) We believe the value exposed by this property does not pose a significant issue on the user privacy, but any feedback would be appreciated! Best, Hongchan On Tuesday, December 21, 2021 at 2:11:25 PM UTC-8 Ajay Rahatekar wrote: > Will do. Ty Joe. > > On Tue, Dec 21, 2021 at 1:32 PM Joe Medley <[email protected]> wrote: > >> Please update the Chrome Status entry as soon as you know. (That will >> inform me as well.) >> Joe Medley | Technical Writer, Chrome DevRel | [email protected] | >> 816-678-7195 <(816)%20678-7195> >> *If an API's not documented it doesn't exist.* >> >> >> On Tue, Dec 21, 2021 at 12:32 PM Ajay Rahatekar <[email protected]> >> wrote: >> >>> The metrics are landing in Jan. So likely shipping in 99. >>> >>> -Ajay >>> >>> On Tue, Dec 21, 2021 at 11:54 AM Joe Medley <[email protected]> wrote: >>> >>>> Is this shipping in 98? >>>> >>>> On Wednesday, December 15, 2021 at 9:02:09 AM UTC-8 hongchan wrote: >>>> >>>>> Thank you all! >>>>> >>>>> 1. I'll land the metrics CL and follow up here. >>>>> 2. The discussion in the WebKit mailing list is ongoing, so I'll >>>>> report back when it's done. >>>>> >>>>> Cheers, >>>>> Hongchan >>>>> >>>>> >>>>> On Wed, Dec 15, 2021 at 8:58 AM Chris Harrelson <[email protected]> >>>>> wrote: >>>>> >>>>>> LGTM3. >>>>>> >>>>>> On Wed, Dec 15, 2021 at 8:47 AM Mike Taylor <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> LGTM2 w/ same condition. >>>>>>> >>>>>>> On 12/15/21 11:47 AM, Yoav Weiss wrote: >>>>>>> >>>>>>> LGTM1 conditional on those metrics landing (but no need to wait for >>>>>>> data from the metrics to come in). >>>>>>> >>>>>>> On Monday, December 6, 2021 at 3:22:26 PM UTC+1 Mike Taylor wrote: >>>>>>> >>>>>>>> To circle back - last week we (myself + some other >>>>>>>> privacy/anti-covert tracking reviewers) met with some folks working on >>>>>>>> this >>>>>>>> feature. We ended up with an AI for Hongchan to land metrics on the >>>>>>>> values >>>>>>>> that getOutputTimestamp would produce (if called, when a new >>>>>>>> AudioContext >>>>>>>> is created, IIRC) as a way to get some data on the utility for using >>>>>>>> either >>>>>>>> of these APIs for device fingerprinting. >>>>>>>> >>>>>>>> On 12/1/21 11:27 AM, Hongchan Choi wrote: >>>>>>>> >>>>>>>> So far we have not discussed the deprecation plan, but I believe >>>>>>>> that's reasonable as well. >>>>>>>> >>>>>>>> I registered myself to the webkit-dev list to post a question, but >>>>>>>> somehow I am getting 403 forbidden. The email is sent to the list, I >>>>>>>> am not >>>>>>>> seeing the message is getting posted on the list. Not sure what to do >>>>>>>> there. >>>>>>>> >>>>>>>> On Wed, Dec 1, 2021 at 3:18 AM Yoav Weiss <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> So are y'all planning to deprecate and remove `getOutputTimestamp` >>>>>>>>> once this ships? If so, that sounds reasonable. >>>>>>>>> >>>>>>>>> I note Chris asked y'all to file for a webkit signal upthread. Did >>>>>>>>> that happen? >>>>>>>>> >>>>>>>>> On Tuesday, November 30, 2021 at 12:54:17 AM UTC+1 hongchan wrote: >>>>>>>>> >>>>>>>>>> I agree that outputLatency is better in terms of both usability >>>>>>>>>> and interoperability. >>>>>>>>>> >>>>>>>>>> FYI, the counter shows getOutputTimestamp is about 0.001% and >>>>>>>>>> there are no partners who are using this API: >>>>>>>>>> >>>>>>>>>> https://chromestatus.com/metrics/feature/popularity#AudioContextGetOutputTimestamp >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, Nov 22, 2021 at 3:00 PM Chris Cunningham < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Re: privacy discussion, Hongchan and I scheduled to meet with >>>>>>>>>>> Paul Jensen next week. >>>>>>>>>>> >>>>>>>>>>> Also, I recently determined that this new WebAudio API >>>>>>>>>>> (outputLatency) is largely redundant with an existing API >>>>>>>>>>> (getOutputTimestamp()). >>>>>>>>>>> https://github.com/WebAudio/web-audio-api/issues/2461 >>>>>>>>>>> >>>>>>>>>>> Chrome has already shipped the latter API, so I believe this new >>>>>>>>>>> API doesn't add potential for additional fingerprinting. Shipping >>>>>>>>>>> both APIs >>>>>>>>>>> is still a good idea for the sake of inter-op. >>>>>>>>>>> >>>>>>>>>>> On Friday, November 19, 2021 at 8:45:28 AM UTC-8 hongchan wrote: >>>>>>>>>>> >>>>>>>>>>>> Thanks for the clarification, Yoav! >>>>>>>>>>>> >>>>>>>>>>>> I hope the mitigation approach in the previous comment makes >>>>>>>>>>>> sense from the privacy perspective, but it will hurt the >>>>>>>>>>>> usability/value of >>>>>>>>>>>> this API significantly. >>>>>>>>>>>> For the privacy-focused discussion, should I use other venues >>>>>>>>>>>> rather than this I2S thread? >>>>>>>>>>>> >>>>>>>>>>>> Cheers, >>>>>>>>>>>> Hongchan >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Thu, Nov 18, 2021 at 10:35 PM Yoav Weiss < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Apologies, I misread the discussion. A spec issue is indeed >>>>>>>>>>>>> not needed, as this is already covered. >>>>>>>>>>>>> >>>>>>>>>>>>> From my perspective, since this exposes active entropy, the >>>>>>>>>>>>> fact that it exposes more data than the (passively exposed) >>>>>>>>>>>>> platform is not >>>>>>>>>>>>> prohibitive on its own. >>>>>>>>>>>>> At the same time, it'd be good to work with privacy-focused >>>>>>>>>>>>> folks to make sure we expose as little information as needed to >>>>>>>>>>>>> maintain >>>>>>>>>>>>> the API's functionality, but not more. >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, Nov 19, 2021 at 4:57 AM Chris Cunningham < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> > If the reported value is incorrect, the A/V synchronization >>>>>>>>>>>>>> won't be possible. (Perhaps chcunningham@ can say more about >>>>>>>>>>>>>> this use >>>>>>>>>>>>>> case.) >>>>>>>>>>>>>> >>>>>>>>>>>>>> The typical video player design drives the clock with audio. >>>>>>>>>>>>>> In short, you send buffers of audio to the OS, use the latency >>>>>>>>>>>>>> value to >>>>>>>>>>>>>> know when those buffers actually reach the user's ears, and >>>>>>>>>>>>>> choose a video >>>>>>>>>>>>>> frame accordingly. Apps using WebCodecs will design apps this >>>>>>>>>>>>>> way (example >>>>>>>>>>>>>> here >>>>>>>>>>>>>> <https://github.com/chcunningham/wc-talk#simple_video_playerhtml> >>>>>>>>>>>>>> ). >>>>>>>>>>>>>> >>>>>>>>>>>>>> As Mike notes, the latency value can vary by quite a lot >>>>>>>>>>>>>> depending on the device (huge differences for bluetooth vs >>>>>>>>>>>>>> wired), so it's >>>>>>>>>>>>>> critical to have the platform tell you when this changes. In >>>>>>>>>>>>>> terms of >>>>>>>>>>>>>> precision, we can _probably_ do some rounding like to the >>>>>>>>>>>>>> nearest >>>>>>>>>>>>>> millisecond. Higher values may lead to issues; user studies >>>>>>>>>>>>>> <https://en.wikipedia.org/wiki/Audio-to-video_synchronization#Recommendations> >>>>>>>>>>>>>> >>>>>>>>>>>>>> have shown sensitivity to fairly low values of drift. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Aside: we may eventually desire a display latency value for >>>>>>>>>>>>>> similar reasons. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Thu, Nov 18, 2021 at 12:45 PM Hongchan Choi < >>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Yoav, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Could you clarify what needs to be discussed from the spec >>>>>>>>>>>>>>> side? The feature (and its behavior) is properly specified, and >>>>>>>>>>>>>>> its privacy >>>>>>>>>>>>>>> aspect is already reviewed by W3C: >>>>>>>>>>>>>>> https://github.com/WebAudio/web-audio-api/issues/1498 >>>>>>>>>>>>>>> (outputLatency) >>>>>>>>>>>>>>> https://github.com/WebAudio/web-audio-api/issues/1495 >>>>>>>>>>>>>>> (general security/privacy consideration) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I thought the request from this thread is to devise a >>>>>>>>>>>>>>> Chrome-specific mitigation, not reopening the spec discussion. >>>>>>>>>>>>>>> Please >>>>>>>>>>>>>>> correct me if I'm mistaken. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>> Hongchan >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Thu, Nov 18, 2021 at 12:37 PM Yoav Weiss < >>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hey folks! >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> It seems like there are some things to figure out here >>>>>>>>>>>>>>>> before shipping. Could y'all file a spec issue to discuss this >>>>>>>>>>>>>>>> with the >>>>>>>>>>>>>>>> broader community? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Tuesday, November 16, 2021 at 6:16:14 PM UTC+1 hongchan >>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks for the feedback, Mike. This is really helpful! >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Mitigation strategies include adding jitter (dithering) >>>>>>>>>>>>>>>>>> and quantization so that the exact skew is incorrectly >>>>>>>>>>>>>>>>>> reported >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> This bit refers to the randomization mechanism deployed by >>>>>>>>>>>>>>>>> other browsers, but this approach defeats the purpose of this >>>>>>>>>>>>>>>>> specific API >>>>>>>>>>>>>>>>> and it is also argued in the spec: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Note however that most audio systems aim for low latency, >>>>>>>>>>>>>>>>>> to synchronise the audio generated by WebAudio to other >>>>>>>>>>>>>>>>>> audio or video >>>>>>>>>>>>>>>>>> sources or to visual cues (for example in a game, or an >>>>>>>>>>>>>>>>>> audio recording or >>>>>>>>>>>>>>>>>> music making environment). Excessive latency decreases >>>>>>>>>>>>>>>>>> usability and may be >>>>>>>>>>>>>>>>>> an accessibility issue. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> If the reported value is incorrect, the A/V >>>>>>>>>>>>>>>>> synchronization won't be possible. (Perhaps chcunningham@ can >>>>>>>>>>>>>>>>> say more >>>>>>>>>>>>>>>>> about this use case.) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> One mitigation idea: we can do some research to collect >>>>>>>>>>>>>>>>> values reported by major platforms/devices and devise a >>>>>>>>>>>>>>>>> reasonable "bucket >>>>>>>>>>>>>>>>> system" for them. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Tue, Nov 16, 2021 at 8:14 AM Mike Taylor < >>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I suppose it would be nice to do some testing to verify >>>>>>>>>>>>>>>>>> the idea that there's a small number of platform + audio >>>>>>>>>>>>>>>>>> device pairs. I >>>>>>>>>>>>>>>>>> was hoping Mozilla's hardware survey >>>>>>>>>>>>>>>>>> <https://data.firefox.com/dashboard/hardware> might have >>>>>>>>>>>>>>>>>> something on that, but it doesn't track speakers. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Just from some simple testing with the following script >>>>>>>>>>>>>>>>>> (thanks dmcardle@ for the snippet) in Firefox on macOS by 3 >>>>>>>>>>>>>>>>>> people, we have >>>>>>>>>>>>>>>>>> very different values that change dramatically when you >>>>>>>>>>>>>>>>>> unplug a headset: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>> (function getNextSample() { >>>>>>>>>>>>>>>>>> console.log((new AudioContext).outputLatency); >>>>>>>>>>>>>>>>>> setTimeout(getNextSample, 10); >>>>>>>>>>>>>>>>>> })(); >>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>> Here's the values on my macbook, using Firefox Nightly: >>>>>>>>>>>>>>>>>> bose headset plugged in: 0.005895833 >>>>>>>>>>>>>>>>>> take headset out: 0.013083333 >>>>>>>>>>>>>>>>>> plug bose headset back in: 0.005875 (different, but >>>>>>>>>>>>>>>>>> stable) >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> https://www.w3.org/TR/webaudio/#priv-sec mentions some >>>>>>>>>>>>>>>>>> mitigation strategies - do we have a plan to apply any of >>>>>>>>>>>>>>>>>> them? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 11/15/21 11:53 AM, Ajay Rahatekar wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> The small number of platform + audio output device >>>>>>>>>>>>>>>>>> combination i.e. a large number of users with the same >>>>>>>>>>>>>>>>>> platform and output >>>>>>>>>>>>>>>>>> device (users on macOS using airpods) would likely be a >>>>>>>>>>>>>>>>>> natural >>>>>>>>>>>>>>>>>> obfuscator. Not sure if we have data about the distribution. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Mon, Nov 15, 2021 at 8:05 AM Mike Taylor < >>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 11/15/21 10:56 AM, Hongchan Choi wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Mon, Nov 15, 2021 at 6:49 AM Mike Taylor < >>>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 11/15/21 1:47 AM, Yoav Weiss wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Fri, Nov 12, 2021 at 5:33 PM 'Ajay Rahatekar' via >>>>>>>>>>>>>>>>>>>> blink-dev <[email protected]> wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> TAG review status >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Completed: Web Audio API specification is W3C >>>>>>>>>>>>>>>>>>>>> Recommendation. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Risks >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> There is a risk of the feature being used for >>>>>>>>>>>>>>>>>>>>> fingerprinting. However outputLatency is the buffer >>>>>>>>>>>>>>>>>>>>> size of the platform-provided audio callback, so the >>>>>>>>>>>>>>>>>>>>> value is inherently >>>>>>>>>>>>>>>>>>>>> platform-specific. That said, the majority of the >>>>>>>>>>>>>>>>>>>>> platform audio buffer >>>>>>>>>>>>>>>>>>>>> size is widely known. (MacOS = 128 frames, Windows = >>>>>>>>>>>>>>>>>>>>> 10ms, Android = 96 >>>>>>>>>>>>>>>>>>>>> frames, etc) >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> This feature does not expose more than what you can >>>>>>>>>>>>>>>>>>>>> query/infer from the UA string. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Would that exposure map cleanly to UA-Platform >>>>>>>>>>>>>>>>>>>> <https://wicg.github.io/ua-client-hints/#sec-ch-ua-platform>, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> which is considered low-entropy and exposed by default? Or >>>>>>>>>>>>>>>>>>>> would it add >>>>>>>>>>>>>>>>>>>> more than that? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> /cc +Mike Taylor >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Looking at >>>>>>>>>>>>>>>>>>>> https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> it states that it depends on the platform _and_ the >>>>>>>>>>>>>>>>>>>> hardware output device. >>>>>>>>>>>>>>>>>>>> If I use an app using outputLatency with speaker A, then >>>>>>>>>>>>>>>>>>>> switch to speaker >>>>>>>>>>>>>>>>>>>> B, will the outputLatency remain the same? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The specification says: If the audio output device is >>>>>>>>>>>>>>>>>>> changed the outputLatency attribute value will be updated >>>>>>>>>>>>>>>>>>> accordingly. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> So the answer is no. The value will change accordingly. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> If that's the case, then outputLatency can reveal more >>>>>>>>>>>>>>>>>>> entropy than just platform alone, right? It would be useful >>>>>>>>>>>>>>>>>>> to know what >>>>>>>>>>>>>>>>>>> this looks like in practice, or what mitigations we might >>>>>>>>>>>>>>>>>>> be able to apply >>>>>>>>>>>>>>>>>>> depending on the size of these latency differences. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "blink-dev" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/936f3e2f-2001-0449-13fd-d3a0548b673c%40chromium.org >>>>>>> >>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/936f3e2f-2001-0449-13fd-d3a0548b673c%40chromium.org?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>> -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/fa61f0d0-3668-4bed-a149-2910c10a6944n%40chromium.org.
