Re: privacy discussion, Hongchan and I scheduled to meet with Paul Jensen next week.
Also, I recently determined that this new WebAudio API (outputLatency) is largely redundant with an existing API (getOutputTimestamp()). https://github.com/WebAudio/web-audio-api/issues/2461 Chrome has already shipped the latter API, so I believe this new API doesn't add potential for additional fingerprinting. Shipping both APIs is still a good idea for the sake of inter-op. On Friday, November 19, 2021 at 8:45:28 AM UTC-8 hongchan wrote: > Thanks for the clarification, Yoav! > > I hope the mitigation approach in the previous comment makes sense from > the privacy perspective, but it will hurt the usability/value of this API > significantly. > For the privacy-focused discussion, should I use other venues rather than > this I2S thread? > > Cheers, > Hongchan > > > On Thu, Nov 18, 2021 at 10:35 PM Yoav Weiss <[email protected]> wrote: > >> Apologies, I misread the discussion. A spec issue is indeed not needed, >> as this is already covered. >> >> From my perspective, since this exposes active entropy, the fact that it >> exposes more data than the (passively exposed) platform is not prohibitive >> on its own. >> At the same time, it'd be good to work with privacy-focused folks to make >> sure we expose as little information as needed to maintain the API's >> functionality, but not more. >> >> On Fri, Nov 19, 2021 at 4:57 AM Chris Cunningham <[email protected]> >> wrote: >> >>> > If the reported value is incorrect, the A/V synchronization won't be >>> possible. (Perhaps chcunningham@ can say more about this use case.) >>> >>> The typical video player design drives the clock with audio. In short, >>> you send buffers of audio to the OS, use the latency value to know when >>> those buffers actually reach the user's ears, and choose a video frame >>> accordingly. Apps using WebCodecs will design apps this way (example >>> here <https://github.com/chcunningham/wc-talk#simple_video_playerhtml>). >>> >>> As Mike notes, the latency value can vary by quite a lot depending on >>> the device (huge differences for bluetooth vs wired), so it's critical to >>> have the platform tell you when this changes. In terms of precision, we can >>> _probably_ do some rounding like to the nearest millisecond. Higher values >>> may lead to issues; user studies >>> <https://en.wikipedia.org/wiki/Audio-to-video_synchronization#Recommendations> >>> >>> have shown sensitivity to fairly low values of drift. >>> >>> Aside: we may eventually desire a display latency value for similar >>> reasons. >>> >>> On Thu, Nov 18, 2021 at 12:45 PM Hongchan Choi <[email protected]> >>> wrote: >>> >>>> Hi Yoav, >>>> >>>> Could you clarify what needs to be discussed from the spec side? The >>>> feature (and its behavior) is properly specified, and its privacy aspect >>>> is >>>> already reviewed by W3C: >>>> https://github.com/WebAudio/web-audio-api/issues/1498 (outputLatency) >>>> https://github.com/WebAudio/web-audio-api/issues/1495 (general >>>> security/privacy consideration) >>>> >>>> I thought the request from this thread is to devise a Chrome-specific >>>> mitigation, not reopening the spec discussion. Please correct me if I'm >>>> mistaken. >>>> >>>> Cheers, >>>> Hongchan >>>> >>>> >>>> On Thu, Nov 18, 2021 at 12:37 PM Yoav Weiss <[email protected]> >>>> wrote: >>>> >>>>> Hey folks! >>>>> >>>>> It seems like there are some things to figure out here before >>>>> shipping. Could y'all file a spec issue to discuss this with the broader >>>>> community? >>>>> >>>>> On Tuesday, November 16, 2021 at 6:16:14 PM UTC+1 hongchan wrote: >>>>> >>>>>> Thanks for the feedback, Mike. This is really helpful! >>>>>> >>>>>> Mitigation strategies include adding jitter (dithering) and >>>>>>> quantization so that the exact skew is incorrectly reported >>>>>>> >>>>>> >>>>>> This bit refers to the randomization mechanism deployed by other >>>>>> browsers, but this approach defeats the purpose of this specific API and >>>>>> it >>>>>> is also argued in the spec: >>>>>> >>>>>> Note however that most audio systems aim for low latency, to >>>>>>> synchronise the audio generated by WebAudio to other audio or video >>>>>>> sources >>>>>>> or to visual cues (for example in a game, or an audio recording or >>>>>>> music >>>>>>> making environment). Excessive latency decreases usability and may be >>>>>>> an >>>>>>> accessibility issue. >>>>>> >>>>>> >>>>>> If the reported value is incorrect, the A/V synchronization won't be >>>>>> possible. (Perhaps chcunningham@ can say more about this use case.) >>>>>> >>>>>> One mitigation idea: we can do some research to collect values >>>>>> reported by major platforms/devices and devise a reasonable "bucket >>>>>> system" >>>>>> for them. >>>>>> >>>>>> >>>>>> On Tue, Nov 16, 2021 at 8:14 AM Mike Taylor <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> I suppose it would be nice to do some testing to verify the idea >>>>>>> that there's a small number of platform + audio device pairs. I was >>>>>>> hoping Mozilla's >>>>>>> hardware survey <https://data.firefox.com/dashboard/hardware> might >>>>>>> have something on that, but it doesn't track speakers. >>>>>>> >>>>>>> Just from some simple testing with the following script (thanks >>>>>>> dmcardle@ for the snippet) in Firefox on macOS by 3 people, we have >>>>>>> very >>>>>>> different values that change dramatically when you unplug a headset: >>>>>>> >>>>>>> ``` >>>>>>> (function getNextSample() { >>>>>>> console.log((new AudioContext).outputLatency); >>>>>>> setTimeout(getNextSample, 10); >>>>>>> })(); >>>>>>> ``` >>>>>>> Here's the values on my macbook, using Firefox Nightly: >>>>>>> bose headset plugged in: 0.005895833 >>>>>>> take headset out: 0.013083333 >>>>>>> plug bose headset back in: 0.005875 (different, but stable) >>>>>>> >>>>>>> https://www.w3.org/TR/webaudio/#priv-sec mentions some mitigation >>>>>>> strategies - do we have a plan to apply any of them? >>>>>>> >>>>>>> On 11/15/21 11:53 AM, Ajay Rahatekar wrote: >>>>>>> >>>>>>> The small number of platform + audio output device combination i.e. >>>>>>> a large number of users with the same platform and output device >>>>>>> (users on >>>>>>> macOS using airpods) would likely be a natural obfuscator. Not sure if >>>>>>> we >>>>>>> have data about the distribution. >>>>>>> >>>>>>> On Mon, Nov 15, 2021 at 8:05 AM Mike Taylor <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> On 11/15/21 10:56 AM, Hongchan Choi wrote: >>>>>>>> >>>>>>>> On Mon, Nov 15, 2021 at 6:49 AM Mike Taylor <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> On 11/15/21 1:47 AM, Yoav Weiss wrote: >>>>>>>>> >>>>>>>>> On Fri, Nov 12, 2021 at 5:33 PM 'Ajay Rahatekar' via blink-dev < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> TAG review status >>>>>>>>>> >>>>>>>>>> Completed: Web Audio API specification is W3C Recommendation. >>>>>>>>>> >>>>>>>>>> Risks >>>>>>>>>> >>>>>>>>>> There is a risk of the feature being used for fingerprinting. >>>>>>>>>> However outputLatency is the buffer size of the >>>>>>>>>> platform-provided audio callback, so the value is inherently >>>>>>>>>> platform-specific. That said, the majority of the platform audio >>>>>>>>>> buffer >>>>>>>>>> size is widely known. (MacOS = 128 frames, Windows = 10ms, Android = >>>>>>>>>> 96 >>>>>>>>>> frames, etc) >>>>>>>>>> >>>>>>>>>> This feature does not expose more than what you can query/infer >>>>>>>>>> from the UA string. >>>>>>>>>> >>>>>>>>> >>>>>>>>> Would that exposure map cleanly to UA-Platform >>>>>>>>> <https://wicg.github.io/ua-client-hints/#sec-ch-ua-platform>, >>>>>>>>> which is considered low-entropy and exposed by default? Or would it >>>>>>>>> add >>>>>>>>> more than that? >>>>>>>>> >>>>>>>>> /cc +Mike Taylor >>>>>>>>> >>>>>>>>>> Looking at >>>>>>>>> https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency, >>>>>>>>> it states that it depends on the platform _and_ the hardware output >>>>>>>>> device. >>>>>>>>> If I use an app using outputLatency with speaker A, then switch to >>>>>>>>> speaker >>>>>>>>> B, will the outputLatency remain the same? >>>>>>>>> >>>>>>>> >>>>>>>> The specification says: If the audio output device is changed the >>>>>>>> outputLatency attribute value will be updated accordingly. >>>>>>>> >>>>>>>> So the answer is no. The value will change accordingly. >>>>>>>> >>>>>>>> If that's the case, then outputLatency can reveal more entropy than >>>>>>>> just platform alone, right? It would be useful to know what this looks >>>>>>>> like >>>>>>>> in practice, or what mitigations we might be able to apply depending >>>>>>>> on the >>>>>>>> size of these latency differences. >>>>>>>> >>>>>>> >>>>>>> -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/0a289307-74ab-4099-9899-2f0231fc05d2n%40chromium.org.
