LGTM1 conditional on those metrics landing (but no need to wait for data from the metrics to come in).
On Monday, December 6, 2021 at 3:22:26 PM UTC+1 Mike Taylor wrote: > To circle back - last week we (myself + some other privacy/anti-covert > tracking reviewers) met with some folks working on this feature. We ended > up with an AI for Hongchan to land metrics on the values that > getOutputTimestamp would produce (if called, when a new AudioContext is > created, IIRC) as a way to get some data on the utility for using either of > these APIs for device fingerprinting. > > On 12/1/21 11:27 AM, Hongchan Choi wrote: > > So far we have not discussed the deprecation plan, but I believe that's > reasonable as well. > > I registered myself to the webkit-dev list to post a question, but somehow > I am getting 403 forbidden. The email is sent to the list, I am not seeing > the message is getting posted on the list. Not sure what to do there. > > On Wed, Dec 1, 2021 at 3:18 AM Yoav Weiss <yoavwe...@chromium.org> wrote: > >> So are y'all planning to deprecate and remove `getOutputTimestamp` once >> this ships? If so, that sounds reasonable. >> >> I note Chris asked y'all to file for a webkit signal upthread. Did that >> happen? >> >> On Tuesday, November 30, 2021 at 12:54:17 AM UTC+1 hongchan wrote: >> >>> I agree that outputLatency is better in terms of both usability and >>> interoperability. >>> >>> FYI, the counter shows getOutputTimestamp is about 0.001% and there are >>> no partners who are using this API: >>> >>> https://chromestatus.com/metrics/feature/popularity#AudioContextGetOutputTimestamp >>> >>> >>> On Mon, Nov 22, 2021 at 3:00 PM Chris Cunningham < >>> chcunning...@google.com> wrote: >>> >>>> Re: privacy discussion, Hongchan and I scheduled to meet with Paul >>>> Jensen next week. >>>> >>>> Also, I recently determined that this new WebAudio API (outputLatency) >>>> is largely redundant with an existing API (getOutputTimestamp()). >>>> https://github.com/WebAudio/web-audio-api/issues/2461 >>>> >>>> Chrome has already shipped the latter API, so I believe this new API >>>> doesn't add potential for additional fingerprinting. Shipping both APIs is >>>> still a good idea for the sake of inter-op. >>>> >>>> On Friday, November 19, 2021 at 8:45:28 AM UTC-8 hongchan wrote: >>>> >>>>> Thanks for the clarification, Yoav! >>>>> >>>>> I hope the mitigation approach in the previous comment makes sense >>>>> from the privacy perspective, but it will hurt the usability/value of >>>>> this >>>>> API significantly. >>>>> For the privacy-focused discussion, should I use other venues rather >>>>> than this I2S thread? >>>>> >>>>> Cheers, >>>>> Hongchan >>>>> >>>>> >>>>> On Thu, Nov 18, 2021 at 10:35 PM Yoav Weiss <yoav...@chromium.org> >>>>> wrote: >>>>> >>>>>> Apologies, I misread the discussion. A spec issue is indeed not >>>>>> needed, as this is already covered. >>>>>> >>>>>> From my perspective, since this exposes active entropy, the fact that >>>>>> it exposes more data than the (passively exposed) platform is not >>>>>> prohibitive on its own. >>>>>> At the same time, it'd be good to work with privacy-focused folks to >>>>>> make sure we expose as little information as needed to maintain the >>>>>> API's >>>>>> functionality, but not more. >>>>>> >>>>>> On Fri, Nov 19, 2021 at 4:57 AM Chris Cunningham < >>>>>> chcunn...@chromium.org> wrote: >>>>>> >>>>>>> > If the reported value is incorrect, the A/V synchronization won't >>>>>>> be possible. (Perhaps chcunningham@ can say more about this use case.) >>>>>>> >>>>>>> The typical video player design drives the clock with audio. In >>>>>>> short, you send buffers of audio to the OS, use the latency value to >>>>>>> know >>>>>>> when those buffers actually reach the user's ears, and choose a video >>>>>>> frame >>>>>>> accordingly. Apps using WebCodecs will design apps this way (example >>>>>>> here >>>>>>> <https://github.com/chcunningham/wc-talk#simple_video_playerhtml>). >>>>>>> >>>>>>> As Mike notes, the latency value can vary by quite a lot depending >>>>>>> on the device (huge differences for bluetooth vs wired), so it's >>>>>>> critical >>>>>>> to have the platform tell you when this changes. In terms of precision, >>>>>>> we >>>>>>> can _probably_ do some rounding like to the nearest millisecond. Higher >>>>>>> values may lead to issues; user studies >>>>>>> <https://en.wikipedia.org/wiki/Audio-to-video_synchronization#Recommendations> >>>>>>> >>>>>>> have shown sensitivity to fairly low values of drift. >>>>>>> >>>>>>> Aside: we may eventually desire a display latency value for similar >>>>>>> reasons. >>>>>>> >>>>>>> On Thu, Nov 18, 2021 at 12:45 PM Hongchan Choi <hong...@chromium.org> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Yoav, >>>>>>>> >>>>>>>> Could you clarify what needs to be discussed from the spec side? >>>>>>>> The feature (and its behavior) is properly specified, and its privacy >>>>>>>> aspect is already reviewed by W3C: >>>>>>>> https://github.com/WebAudio/web-audio-api/issues/1498 >>>>>>>> (outputLatency) >>>>>>>> https://github.com/WebAudio/web-audio-api/issues/1495 (general >>>>>>>> security/privacy consideration) >>>>>>>> >>>>>>>> I thought the request from this thread is to devise a >>>>>>>> Chrome-specific mitigation, not reopening the spec discussion. Please >>>>>>>> correct me if I'm mistaken. >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Hongchan >>>>>>>> >>>>>>>> >>>>>>>> On Thu, Nov 18, 2021 at 12:37 PM Yoav Weiss <yoav...@chromium.org> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hey folks! >>>>>>>>> >>>>>>>>> It seems like there are some things to figure out here before >>>>>>>>> shipping. Could y'all file a spec issue to discuss this with the >>>>>>>>> broader >>>>>>>>> community? >>>>>>>>> >>>>>>>>> On Tuesday, November 16, 2021 at 6:16:14 PM UTC+1 hongchan wrote: >>>>>>>>> >>>>>>>>>> Thanks for the feedback, Mike. This is really helpful! >>>>>>>>>> >>>>>>>>>> Mitigation strategies include adding jitter (dithering) and >>>>>>>>>>> quantization so that the exact skew is incorrectly reported >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> This bit refers to the randomization mechanism deployed by other >>>>>>>>>> browsers, but this approach defeats the purpose of this specific API >>>>>>>>>> and it >>>>>>>>>> is also argued in the spec: >>>>>>>>>> >>>>>>>>>> Note however that most audio systems aim for low latency, to >>>>>>>>>>> synchronise the audio generated by WebAudio to other audio or video >>>>>>>>>>> sources >>>>>>>>>>> or to visual cues (for example in a game, or an audio recording or >>>>>>>>>>> music >>>>>>>>>>> making environment). Excessive latency decreases usability and may >>>>>>>>>>> be an >>>>>>>>>>> accessibility issue. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> If the reported value is incorrect, the A/V synchronization won't >>>>>>>>>> be possible. (Perhaps chcunningham@ can say more about this use >>>>>>>>>> case.) >>>>>>>>>> >>>>>>>>>> One mitigation idea: we can do some research to collect values >>>>>>>>>> reported by major platforms/devices and devise a reasonable "bucket >>>>>>>>>> system" >>>>>>>>>> for them. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, Nov 16, 2021 at 8:14 AM Mike Taylor <mike...@chromium.org> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> I suppose it would be nice to do some testing to verify the idea >>>>>>>>>>> that there's a small number of platform + audio device pairs. I was >>>>>>>>>>> hoping Mozilla's >>>>>>>>>>> hardware survey <https://data.firefox.com/dashboard/hardware> >>>>>>>>>>> might have something on that, but it doesn't track speakers. >>>>>>>>>>> >>>>>>>>>>> Just from some simple testing with the following script (thanks >>>>>>>>>>> dmcardle@ for the snippet) in Firefox on macOS by 3 people, we have >>>>>>>>>>> very >>>>>>>>>>> different values that change dramatically when you unplug a headset: >>>>>>>>>>> >>>>>>>>>>> ``` >>>>>>>>>>> (function getNextSample() { >>>>>>>>>>> console.log((new AudioContext).outputLatency); >>>>>>>>>>> setTimeout(getNextSample, 10); >>>>>>>>>>> })(); >>>>>>>>>>> ``` >>>>>>>>>>> Here's the values on my macbook, using Firefox Nightly: >>>>>>>>>>> bose headset plugged in: 0.005895833 >>>>>>>>>>> take headset out: 0.013083333 >>>>>>>>>>> plug bose headset back in: 0.005875 (different, but stable) >>>>>>>>>>> >>>>>>>>>>> https://www.w3.org/TR/webaudio/#priv-sec mentions some >>>>>>>>>>> mitigation strategies - do we have a plan to apply any of them? >>>>>>>>>>> >>>>>>>>>>> On 11/15/21 11:53 AM, Ajay Rahatekar wrote: >>>>>>>>>>> >>>>>>>>>>> The small number of platform + audio output device combination >>>>>>>>>>> i.e. a large number of users with the same platform and output >>>>>>>>>>> device (users on macOS using airpods) would likely be a natural >>>>>>>>>>> obfuscator. Not sure if we have data about the distribution. >>>>>>>>>>> >>>>>>>>>>> On Mon, Nov 15, 2021 at 8:05 AM Mike Taylor < >>>>>>>>>>> mike...@chromium.org> wrote: >>>>>>>>>>> >>>>>>>>>>>> On 11/15/21 10:56 AM, Hongchan Choi wrote: >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Nov 15, 2021 at 6:49 AM Mike Taylor < >>>>>>>>>>>> mike...@chromium.org> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> On 11/15/21 1:47 AM, Yoav Weiss wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, Nov 12, 2021 at 5:33 PM 'Ajay Rahatekar' via blink-dev >>>>>>>>>>>>> <blin...@chromium.org> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> TAG review status >>>>>>>>>>>>>> >>>>>>>>>>>>>> Completed: Web Audio API specification is W3C Recommendation. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Risks >>>>>>>>>>>>>> >>>>>>>>>>>>>> There is a risk of the feature being used for fingerprinting. >>>>>>>>>>>>>> However outputLatency is the buffer size of the >>>>>>>>>>>>>> platform-provided audio callback, so the value is inherently >>>>>>>>>>>>>> platform-specific. That said, the majority of the platform audio >>>>>>>>>>>>>> buffer >>>>>>>>>>>>>> size is widely known. (MacOS = 128 frames, Windows = 10ms, >>>>>>>>>>>>>> Android = 96 >>>>>>>>>>>>>> frames, etc) >>>>>>>>>>>>>> >>>>>>>>>>>>>> This feature does not expose more than what you can >>>>>>>>>>>>>> query/infer from the UA string. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Would that exposure map cleanly to UA-Platform >>>>>>>>>>>>> <https://wicg.github.io/ua-client-hints/#sec-ch-ua-platform>, >>>>>>>>>>>>> which is considered low-entropy and exposed by default? Or would >>>>>>>>>>>>> it add >>>>>>>>>>>>> more than that? >>>>>>>>>>>>> >>>>>>>>>>>>> /cc +Mike Taylor >>>>>>>>>>>>> >>>>>>>>>>>>>> Looking at >>>>>>>>>>>>> https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency, >>>>>>>>>>>>> it states that it depends on the platform _and_ the hardware >>>>>>>>>>>>> output device. >>>>>>>>>>>>> If I use an app using outputLatency with speaker A, then switch >>>>>>>>>>>>> to speaker >>>>>>>>>>>>> B, will the outputLatency remain the same? >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> The specification says: If the audio output device is changed >>>>>>>>>>>> the outputLatency attribute value will be updated accordingly. >>>>>>>>>>>> >>>>>>>>>>>> So the answer is no. The value will change accordingly. >>>>>>>>>>>> >>>>>>>>>>>> If that's the case, then outputLatency can reveal more entropy >>>>>>>>>>>> than just platform alone, right? It would be useful to know what >>>>>>>>>>>> this looks >>>>>>>>>>>> like in practice, or what mitigations we might be able to apply >>>>>>>>>>>> depending >>>>>>>>>>>> on the size of these latency differences. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> > -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/18800ec4-580e-4cea-bbaf-9e39345b9d53n%40chromium.org.
