Thank you all! 1. I'll land the metrics CL and follow up here. 2. The discussion in the WebKit mailing list is ongoing, so I'll report back when it's done.
Cheers, Hongchan On Wed, Dec 15, 2021 at 8:58 AM Chris Harrelson <[email protected]> wrote: > LGTM3. > > On Wed, Dec 15, 2021 at 8:47 AM Mike Taylor <[email protected]> > wrote: > >> LGTM2 w/ same condition. >> >> On 12/15/21 11:47 AM, Yoav Weiss wrote: >> >> LGTM1 conditional on those metrics landing (but no need to wait for data >> from the metrics to come in). >> >> On Monday, December 6, 2021 at 3:22:26 PM UTC+1 Mike Taylor wrote: >> >>> To circle back - last week we (myself + some other privacy/anti-covert >>> tracking reviewers) met with some folks working on this feature. We ended >>> up with an AI for Hongchan to land metrics on the values that >>> getOutputTimestamp would produce (if called, when a new AudioContext is >>> created, IIRC) as a way to get some data on the utility for using either of >>> these APIs for device fingerprinting. >>> >>> On 12/1/21 11:27 AM, Hongchan Choi wrote: >>> >>> So far we have not discussed the deprecation plan, but I believe that's >>> reasonable as well. >>> >>> I registered myself to the webkit-dev list to post a question, but >>> somehow I am getting 403 forbidden. The email is sent to the list, I am not >>> seeing the message is getting posted on the list. Not sure what to do there. >>> >>> On Wed, Dec 1, 2021 at 3:18 AM Yoav Weiss <[email protected]> >>> wrote: >>> >>>> So are y'all planning to deprecate and remove `getOutputTimestamp` once >>>> this ships? If so, that sounds reasonable. >>>> >>>> I note Chris asked y'all to file for a webkit signal upthread. Did that >>>> happen? >>>> >>>> On Tuesday, November 30, 2021 at 12:54:17 AM UTC+1 hongchan wrote: >>>> >>>>> I agree that outputLatency is better in terms of both usability and >>>>> interoperability. >>>>> >>>>> FYI, the counter shows getOutputTimestamp is about 0.001% and there >>>>> are no partners who are using this API: >>>>> >>>>> https://chromestatus.com/metrics/feature/popularity#AudioContextGetOutputTimestamp >>>>> >>>>> >>>>> On Mon, Nov 22, 2021 at 3:00 PM Chris Cunningham < >>>>> [email protected]> wrote: >>>>> >>>>>> Re: privacy discussion, Hongchan and I scheduled to meet with Paul >>>>>> Jensen next week. >>>>>> >>>>>> Also, I recently determined that this new WebAudio API >>>>>> (outputLatency) is largely redundant with an existing API >>>>>> (getOutputTimestamp()). >>>>>> https://github.com/WebAudio/web-audio-api/issues/2461 >>>>>> >>>>>> Chrome has already shipped the latter API, so I believe this new API >>>>>> doesn't add potential for additional fingerprinting. Shipping both APIs >>>>>> is >>>>>> still a good idea for the sake of inter-op. >>>>>> >>>>>> On Friday, November 19, 2021 at 8:45:28 AM UTC-8 hongchan wrote: >>>>>> >>>>>>> Thanks for the clarification, Yoav! >>>>>>> >>>>>>> I hope the mitigation approach in the previous comment makes sense >>>>>>> from the privacy perspective, but it will hurt the usability/value of >>>>>>> this >>>>>>> API significantly. >>>>>>> For the privacy-focused discussion, should I use other venues rather >>>>>>> than this I2S thread? >>>>>>> >>>>>>> Cheers, >>>>>>> Hongchan >>>>>>> >>>>>>> >>>>>>> On Thu, Nov 18, 2021 at 10:35 PM Yoav Weiss <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Apologies, I misread the discussion. A spec issue is indeed not >>>>>>>> needed, as this is already covered. >>>>>>>> >>>>>>>> From my perspective, since this exposes active entropy, the fact >>>>>>>> that it exposes more data than the (passively exposed) platform is not >>>>>>>> prohibitive on its own. >>>>>>>> At the same time, it'd be good to work with privacy-focused folks >>>>>>>> to make sure we expose as little information as needed to maintain the >>>>>>>> API's functionality, but not more. >>>>>>>> >>>>>>>> On Fri, Nov 19, 2021 at 4:57 AM Chris Cunningham < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> > If the reported value is incorrect, the A/V synchronization >>>>>>>>> won't be possible. (Perhaps chcunningham@ can say more about this >>>>>>>>> use case.) >>>>>>>>> >>>>>>>>> The typical video player design drives the clock with audio. In >>>>>>>>> short, you send buffers of audio to the OS, use the latency value to >>>>>>>>> know >>>>>>>>> when those buffers actually reach the user's ears, and choose a video >>>>>>>>> frame >>>>>>>>> accordingly. Apps using WebCodecs will design apps this way (example >>>>>>>>> here >>>>>>>>> <https://github.com/chcunningham/wc-talk#simple_video_playerhtml> >>>>>>>>> ). >>>>>>>>> >>>>>>>>> As Mike notes, the latency value can vary by quite a lot depending >>>>>>>>> on the device (huge differences for bluetooth vs wired), so it's >>>>>>>>> critical >>>>>>>>> to have the platform tell you when this changes. In terms of >>>>>>>>> precision, we >>>>>>>>> can _probably_ do some rounding like to the nearest millisecond. >>>>>>>>> Higher >>>>>>>>> values may lead to issues; user studies >>>>>>>>> <https://en.wikipedia.org/wiki/Audio-to-video_synchronization#Recommendations> >>>>>>>>> have shown sensitivity to fairly low values of drift. >>>>>>>>> >>>>>>>>> Aside: we may eventually desire a display latency value for >>>>>>>>> similar reasons. >>>>>>>>> >>>>>>>>> On Thu, Nov 18, 2021 at 12:45 PM Hongchan Choi < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi Yoav, >>>>>>>>>> >>>>>>>>>> Could you clarify what needs to be discussed from the spec side? >>>>>>>>>> The feature (and its behavior) is properly specified, and its privacy >>>>>>>>>> aspect is already reviewed by W3C: >>>>>>>>>> https://github.com/WebAudio/web-audio-api/issues/1498 >>>>>>>>>> (outputLatency) >>>>>>>>>> https://github.com/WebAudio/web-audio-api/issues/1495 (general >>>>>>>>>> security/privacy consideration) >>>>>>>>>> >>>>>>>>>> I thought the request from this thread is to devise a >>>>>>>>>> Chrome-specific mitigation, not reopening the spec discussion. Please >>>>>>>>>> correct me if I'm mistaken. >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> Hongchan >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thu, Nov 18, 2021 at 12:37 PM Yoav Weiss <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hey folks! >>>>>>>>>>> >>>>>>>>>>> It seems like there are some things to figure out here before >>>>>>>>>>> shipping. Could y'all file a spec issue to discuss this with the >>>>>>>>>>> broader >>>>>>>>>>> community? >>>>>>>>>>> >>>>>>>>>>> On Tuesday, November 16, 2021 at 6:16:14 PM UTC+1 hongchan wrote: >>>>>>>>>>> >>>>>>>>>>>> Thanks for the feedback, Mike. This is really helpful! >>>>>>>>>>>> >>>>>>>>>>>> Mitigation strategies include adding jitter (dithering) and >>>>>>>>>>>>> quantization so that the exact skew is incorrectly reported >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> This bit refers to the randomization mechanism deployed by >>>>>>>>>>>> other browsers, but this approach defeats the purpose of this >>>>>>>>>>>> specific API >>>>>>>>>>>> and it is also argued in the spec: >>>>>>>>>>>> >>>>>>>>>>>> Note however that most audio systems aim for low latency, to >>>>>>>>>>>>> synchronise the audio generated by WebAudio to other audio or >>>>>>>>>>>>> video sources >>>>>>>>>>>>> or to visual cues (for example in a game, or an audio recording >>>>>>>>>>>>> or music >>>>>>>>>>>>> making environment). Excessive latency decreases usability and >>>>>>>>>>>>> may be an >>>>>>>>>>>>> accessibility issue. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> If the reported value is incorrect, the A/V synchronization >>>>>>>>>>>> won't be possible. (Perhaps chcunningham@ can say more about >>>>>>>>>>>> this use case.) >>>>>>>>>>>> >>>>>>>>>>>> One mitigation idea: we can do some research to collect values >>>>>>>>>>>> reported by major platforms/devices and devise a reasonable >>>>>>>>>>>> "bucket system" >>>>>>>>>>>> for them. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Nov 16, 2021 at 8:14 AM Mike Taylor < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> I suppose it would be nice to do some testing to verify the >>>>>>>>>>>>> idea that there's a small number of platform + audio device >>>>>>>>>>>>> pairs. I was >>>>>>>>>>>>> hoping Mozilla's hardware survey >>>>>>>>>>>>> <https://data.firefox.com/dashboard/hardware> might have >>>>>>>>>>>>> something on that, but it doesn't track speakers. >>>>>>>>>>>>> >>>>>>>>>>>>> Just from some simple testing with the following script >>>>>>>>>>>>> (thanks dmcardle@ for the snippet) in Firefox on macOS by 3 >>>>>>>>>>>>> people, we have very different values that change dramatically >>>>>>>>>>>>> when you >>>>>>>>>>>>> unplug a headset: >>>>>>>>>>>>> >>>>>>>>>>>>> ``` >>>>>>>>>>>>> (function getNextSample() { >>>>>>>>>>>>> console.log((new AudioContext).outputLatency); >>>>>>>>>>>>> setTimeout(getNextSample, 10); >>>>>>>>>>>>> })(); >>>>>>>>>>>>> ``` >>>>>>>>>>>>> Here's the values on my macbook, using Firefox Nightly: >>>>>>>>>>>>> bose headset plugged in: 0.005895833 >>>>>>>>>>>>> take headset out: 0.013083333 >>>>>>>>>>>>> plug bose headset back in: 0.005875 (different, but stable) >>>>>>>>>>>>> >>>>>>>>>>>>> https://www.w3.org/TR/webaudio/#priv-sec mentions some >>>>>>>>>>>>> mitigation strategies - do we have a plan to apply any of them? >>>>>>>>>>>>> >>>>>>>>>>>>> On 11/15/21 11:53 AM, Ajay Rahatekar wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> The small number of platform + audio output device combination >>>>>>>>>>>>> i.e. a large number of users with the same platform and output >>>>>>>>>>>>> device (users on macOS using airpods) would likely be a natural >>>>>>>>>>>>> obfuscator. Not sure if we have data about the distribution. >>>>>>>>>>>>> >>>>>>>>>>>>> On Mon, Nov 15, 2021 at 8:05 AM Mike Taylor < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> On 11/15/21 10:56 AM, Hongchan Choi wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Mon, Nov 15, 2021 at 6:49 AM Mike Taylor < >>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 11/15/21 1:47 AM, Yoav Weiss wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Fri, Nov 12, 2021 at 5:33 PM 'Ajay Rahatekar' via >>>>>>>>>>>>>>> blink-dev <[email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> TAG review status >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Completed: Web Audio API specification is W3C >>>>>>>>>>>>>>>> Recommendation. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Risks >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> There is a risk of the feature being used for >>>>>>>>>>>>>>>> fingerprinting. However outputLatency is the buffer size >>>>>>>>>>>>>>>> of the platform-provided audio callback, so the value is >>>>>>>>>>>>>>>> inherently >>>>>>>>>>>>>>>> platform-specific. That said, the majority of the platform >>>>>>>>>>>>>>>> audio buffer >>>>>>>>>>>>>>>> size is widely known. (MacOS = 128 frames, Windows = 10ms, >>>>>>>>>>>>>>>> Android = 96 >>>>>>>>>>>>>>>> frames, etc) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> This feature does not expose more than what you can >>>>>>>>>>>>>>>> query/infer from the UA string. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Would that exposure map cleanly to UA-Platform >>>>>>>>>>>>>>> <https://wicg.github.io/ua-client-hints/#sec-ch-ua-platform>, >>>>>>>>>>>>>>> which is considered low-entropy and exposed by default? Or >>>>>>>>>>>>>>> would it add >>>>>>>>>>>>>>> more than that? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> /cc +Mike Taylor >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Looking at >>>>>>>>>>>>>>> https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency, >>>>>>>>>>>>>>> it states that it depends on the platform _and_ the hardware >>>>>>>>>>>>>>> output device. >>>>>>>>>>>>>>> If I use an app using outputLatency with speaker A, then switch >>>>>>>>>>>>>>> to speaker >>>>>>>>>>>>>>> B, will the outputLatency remain the same? >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> The specification says: If the audio output device is changed >>>>>>>>>>>>>> the outputLatency attribute value will be updated accordingly. >>>>>>>>>>>>>> >>>>>>>>>>>>>> So the answer is no. The value will change accordingly. >>>>>>>>>>>>>> >>>>>>>>>>>>>> If that's the case, then outputLatency can reveal more >>>>>>>>>>>>>> entropy than just platform alone, right? It would be useful to >>>>>>>>>>>>>> know what >>>>>>>>>>>>>> this looks like in practice, or what mitigations we might be >>>>>>>>>>>>>> able to apply >>>>>>>>>>>>>> depending on the size of these latency differences. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "blink-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/936f3e2f-2001-0449-13fd-d3a0548b673c%40chromium.org >> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/936f3e2f-2001-0449-13fd-d3a0548b673c%40chromium.org?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGJqXNvvTiTfQ-hQPpDP2FRMoOvWgbRhKEPoqZfi4bQVT2DsnQ%40mail.gmail.com.
