LGTM3. On Wed, Dec 15, 2021 at 8:47 AM Mike Taylor <miketa...@chromium.org> wrote:
> LGTM2 w/ same condition. > > On 12/15/21 11:47 AM, Yoav Weiss wrote: > > LGTM1 conditional on those metrics landing (but no need to wait for data > from the metrics to come in). > > On Monday, December 6, 2021 at 3:22:26 PM UTC+1 Mike Taylor wrote: > >> To circle back - last week we (myself + some other privacy/anti-covert >> tracking reviewers) met with some folks working on this feature. We ended >> up with an AI for Hongchan to land metrics on the values that >> getOutputTimestamp would produce (if called, when a new AudioContext is >> created, IIRC) as a way to get some data on the utility for using either of >> these APIs for device fingerprinting. >> >> On 12/1/21 11:27 AM, Hongchan Choi wrote: >> >> So far we have not discussed the deprecation plan, but I believe that's >> reasonable as well. >> >> I registered myself to the webkit-dev list to post a question, but >> somehow I am getting 403 forbidden. The email is sent to the list, I am not >> seeing the message is getting posted on the list. Not sure what to do there. >> >> On Wed, Dec 1, 2021 at 3:18 AM Yoav Weiss <yoavwe...@chromium.org> wrote: >> >>> So are y'all planning to deprecate and remove `getOutputTimestamp` once >>> this ships? If so, that sounds reasonable. >>> >>> I note Chris asked y'all to file for a webkit signal upthread. Did that >>> happen? >>> >>> On Tuesday, November 30, 2021 at 12:54:17 AM UTC+1 hongchan wrote: >>> >>>> I agree that outputLatency is better in terms of both usability and >>>> interoperability. >>>> >>>> FYI, the counter shows getOutputTimestamp is about 0.001% and there are >>>> no partners who are using this API: >>>> >>>> https://chromestatus.com/metrics/feature/popularity#AudioContextGetOutputTimestamp >>>> >>>> >>>> On Mon, Nov 22, 2021 at 3:00 PM Chris Cunningham < >>>> chcunning...@google.com> wrote: >>>> >>>>> Re: privacy discussion, Hongchan and I scheduled to meet with Paul >>>>> Jensen next week. >>>>> >>>>> Also, I recently determined that this new WebAudio API (outputLatency) >>>>> is largely redundant with an existing API (getOutputTimestamp()). >>>>> https://github.com/WebAudio/web-audio-api/issues/2461 >>>>> >>>>> Chrome has already shipped the latter API, so I believe this new API >>>>> doesn't add potential for additional fingerprinting. Shipping both APIs is >>>>> still a good idea for the sake of inter-op. >>>>> >>>>> On Friday, November 19, 2021 at 8:45:28 AM UTC-8 hongchan wrote: >>>>> >>>>>> Thanks for the clarification, Yoav! >>>>>> >>>>>> I hope the mitigation approach in the previous comment makes sense >>>>>> from the privacy perspective, but it will hurt the usability/value of >>>>>> this >>>>>> API significantly. >>>>>> For the privacy-focused discussion, should I use other venues rather >>>>>> than this I2S thread? >>>>>> >>>>>> Cheers, >>>>>> Hongchan >>>>>> >>>>>> >>>>>> On Thu, Nov 18, 2021 at 10:35 PM Yoav Weiss <yoav...@chromium.org> >>>>>> wrote: >>>>>> >>>>>>> Apologies, I misread the discussion. A spec issue is indeed not >>>>>>> needed, as this is already covered. >>>>>>> >>>>>>> From my perspective, since this exposes active entropy, the fact >>>>>>> that it exposes more data than the (passively exposed) platform is not >>>>>>> prohibitive on its own. >>>>>>> At the same time, it'd be good to work with privacy-focused folks to >>>>>>> make sure we expose as little information as needed to maintain the >>>>>>> API's >>>>>>> functionality, but not more. >>>>>>> >>>>>>> On Fri, Nov 19, 2021 at 4:57 AM Chris Cunningham < >>>>>>> chcunn...@chromium.org> wrote: >>>>>>> >>>>>>>> > If the reported value is incorrect, the A/V synchronization won't >>>>>>>> be possible. (Perhaps chcunningham@ can say more about this use >>>>>>>> case.) >>>>>>>> >>>>>>>> The typical video player design drives the clock with audio. In >>>>>>>> short, you send buffers of audio to the OS, use the latency value to >>>>>>>> know >>>>>>>> when those buffers actually reach the user's ears, and choose a video >>>>>>>> frame >>>>>>>> accordingly. Apps using WebCodecs will design apps this way (example >>>>>>>> here >>>>>>>> <https://github.com/chcunningham/wc-talk#simple_video_playerhtml>). >>>>>>>> >>>>>>>> As Mike notes, the latency value can vary by quite a lot depending >>>>>>>> on the device (huge differences for bluetooth vs wired), so it's >>>>>>>> critical >>>>>>>> to have the platform tell you when this changes. In terms of >>>>>>>> precision, we >>>>>>>> can _probably_ do some rounding like to the nearest millisecond. Higher >>>>>>>> values may lead to issues; user studies >>>>>>>> <https://en.wikipedia.org/wiki/Audio-to-video_synchronization#Recommendations> >>>>>>>> have shown sensitivity to fairly low values of drift. >>>>>>>> >>>>>>>> Aside: we may eventually desire a display latency value for similar >>>>>>>> reasons. >>>>>>>> >>>>>>>> On Thu, Nov 18, 2021 at 12:45 PM Hongchan Choi < >>>>>>>> hong...@chromium.org> wrote: >>>>>>>> >>>>>>>>> Hi Yoav, >>>>>>>>> >>>>>>>>> Could you clarify what needs to be discussed from the spec side? >>>>>>>>> The feature (and its behavior) is properly specified, and its privacy >>>>>>>>> aspect is already reviewed by W3C: >>>>>>>>> https://github.com/WebAudio/web-audio-api/issues/1498 >>>>>>>>> (outputLatency) >>>>>>>>> https://github.com/WebAudio/web-audio-api/issues/1495 (general >>>>>>>>> security/privacy consideration) >>>>>>>>> >>>>>>>>> I thought the request from this thread is to devise a >>>>>>>>> Chrome-specific mitigation, not reopening the spec discussion. Please >>>>>>>>> correct me if I'm mistaken. >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> Hongchan >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, Nov 18, 2021 at 12:37 PM Yoav Weiss <yoav...@chromium.org> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hey folks! >>>>>>>>>> >>>>>>>>>> It seems like there are some things to figure out here before >>>>>>>>>> shipping. Could y'all file a spec issue to discuss this with the >>>>>>>>>> broader >>>>>>>>>> community? >>>>>>>>>> >>>>>>>>>> On Tuesday, November 16, 2021 at 6:16:14 PM UTC+1 hongchan wrote: >>>>>>>>>> >>>>>>>>>>> Thanks for the feedback, Mike. This is really helpful! >>>>>>>>>>> >>>>>>>>>>> Mitigation strategies include adding jitter (dithering) and >>>>>>>>>>>> quantization so that the exact skew is incorrectly reported >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> This bit refers to the randomization mechanism deployed by other >>>>>>>>>>> browsers, but this approach defeats the purpose of this specific >>>>>>>>>>> API and it >>>>>>>>>>> is also argued in the spec: >>>>>>>>>>> >>>>>>>>>>> Note however that most audio systems aim for low latency, to >>>>>>>>>>>> synchronise the audio generated by WebAudio to other audio or >>>>>>>>>>>> video sources >>>>>>>>>>>> or to visual cues (for example in a game, or an audio recording or >>>>>>>>>>>> music >>>>>>>>>>>> making environment). Excessive latency decreases usability and may >>>>>>>>>>>> be an >>>>>>>>>>>> accessibility issue. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> If the reported value is incorrect, the A/V synchronization >>>>>>>>>>> won't be possible. (Perhaps chcunningham@ can say more about >>>>>>>>>>> this use case.) >>>>>>>>>>> >>>>>>>>>>> One mitigation idea: we can do some research to collect values >>>>>>>>>>> reported by major platforms/devices and devise a reasonable "bucket >>>>>>>>>>> system" >>>>>>>>>>> for them. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tue, Nov 16, 2021 at 8:14 AM Mike Taylor < >>>>>>>>>>> mike...@chromium.org> wrote: >>>>>>>>>>> >>>>>>>>>>>> I suppose it would be nice to do some testing to verify the >>>>>>>>>>>> idea that there's a small number of platform + audio device pairs. >>>>>>>>>>>> I was >>>>>>>>>>>> hoping Mozilla's hardware survey >>>>>>>>>>>> <https://data.firefox.com/dashboard/hardware> might have >>>>>>>>>>>> something on that, but it doesn't track speakers. >>>>>>>>>>>> >>>>>>>>>>>> Just from some simple testing with the following script (thanks >>>>>>>>>>>> dmcardle@ for the snippet) in Firefox on macOS by 3 people, we >>>>>>>>>>>> have very different values that change dramatically when you >>>>>>>>>>>> unplug a >>>>>>>>>>>> headset: >>>>>>>>>>>> >>>>>>>>>>>> ``` >>>>>>>>>>>> (function getNextSample() { >>>>>>>>>>>> console.log((new AudioContext).outputLatency); >>>>>>>>>>>> setTimeout(getNextSample, 10); >>>>>>>>>>>> })(); >>>>>>>>>>>> ``` >>>>>>>>>>>> Here's the values on my macbook, using Firefox Nightly: >>>>>>>>>>>> bose headset plugged in: 0.005895833 >>>>>>>>>>>> take headset out: 0.013083333 >>>>>>>>>>>> plug bose headset back in: 0.005875 (different, but stable) >>>>>>>>>>>> >>>>>>>>>>>> https://www.w3.org/TR/webaudio/#priv-sec mentions some >>>>>>>>>>>> mitigation strategies - do we have a plan to apply any of them? >>>>>>>>>>>> >>>>>>>>>>>> On 11/15/21 11:53 AM, Ajay Rahatekar wrote: >>>>>>>>>>>> >>>>>>>>>>>> The small number of platform + audio output device combination >>>>>>>>>>>> i.e. a large number of users with the same platform and output >>>>>>>>>>>> device (users on macOS using airpods) would likely be a natural >>>>>>>>>>>> obfuscator. Not sure if we have data about the distribution. >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Nov 15, 2021 at 8:05 AM Mike Taylor < >>>>>>>>>>>> mike...@chromium.org> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> On 11/15/21 10:56 AM, Hongchan Choi wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> On Mon, Nov 15, 2021 at 6:49 AM Mike Taylor < >>>>>>>>>>>>> mike...@chromium.org> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> On 11/15/21 1:47 AM, Yoav Weiss wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Fri, Nov 12, 2021 at 5:33 PM 'Ajay Rahatekar' via >>>>>>>>>>>>>> blink-dev <blin...@chromium.org> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> TAG review status >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Completed: Web Audio API specification is W3C Recommendation. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Risks >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> There is a risk of the feature being used for >>>>>>>>>>>>>>> fingerprinting. However outputLatency is the buffer size of >>>>>>>>>>>>>>> the platform-provided audio callback, so the value is inherently >>>>>>>>>>>>>>> platform-specific. That said, the majority of the platform >>>>>>>>>>>>>>> audio buffer >>>>>>>>>>>>>>> size is widely known. (MacOS = 128 frames, Windows = 10ms, >>>>>>>>>>>>>>> Android = 96 >>>>>>>>>>>>>>> frames, etc) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> This feature does not expose more than what you can >>>>>>>>>>>>>>> query/infer from the UA string. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Would that exposure map cleanly to UA-Platform >>>>>>>>>>>>>> <https://wicg.github.io/ua-client-hints/#sec-ch-ua-platform>, >>>>>>>>>>>>>> which is considered low-entropy and exposed by default? Or would >>>>>>>>>>>>>> it add >>>>>>>>>>>>>> more than that? >>>>>>>>>>>>>> >>>>>>>>>>>>>> /cc +Mike Taylor >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Looking at >>>>>>>>>>>>>> https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency, >>>>>>>>>>>>>> it states that it depends on the platform _and_ the hardware >>>>>>>>>>>>>> output device. >>>>>>>>>>>>>> If I use an app using outputLatency with speaker A, then switch >>>>>>>>>>>>>> to speaker >>>>>>>>>>>>>> B, will the outputLatency remain the same? >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> The specification says: If the audio output device is changed >>>>>>>>>>>>> the outputLatency attribute value will be updated accordingly. >>>>>>>>>>>>> >>>>>>>>>>>>> So the answer is no. The value will change accordingly. >>>>>>>>>>>>> >>>>>>>>>>>>> If that's the case, then outputLatency can reveal more entropy >>>>>>>>>>>>> than just platform alone, right? It would be useful to know what >>>>>>>>>>>>> this looks >>>>>>>>>>>>> like in practice, or what mitigations we might be able to apply >>>>>>>>>>>>> depending >>>>>>>>>>>>> on the size of these latency differences. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >> > -- > You received this message because you are subscribed to the Google Groups > "blink-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to blink-dev+unsubscr...@chromium.org. > To view this discussion on the web visit > https://groups.google.com/a/chromium.org/d/msgid/blink-dev/936f3e2f-2001-0449-13fd-d3a0548b673c%40chromium.org > <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/936f3e2f-2001-0449-13fd-d3a0548b673c%40chromium.org?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOMQ%2Bw-SSSnqs2%2BN%3DUVdH9V5pO67CVMg_tzRYObUPOLd5NhKVw%40mail.gmail.com.
