So far we have not discussed the deprecation plan, but I believe
that's reasonable as well.
I registered myself to the webkit-dev list to post a question,
but somehow I am getting 403 forbidden. The email is sent to the
list, I am not seeing the message is getting posted on the list.
Not sure what to do there.
On Wed, Dec 1, 2021 at 3:18 AM Yoav Weiss
<[email protected]> wrote:
So are y'all planning to deprecate and remove
`getOutputTimestamp` once this ships? If so, that sounds
reasonable.
I note Chris asked y'all to file for a webkit signal
upthread. Did that happen?
On Tuesday, November 30, 2021 at 12:54:17 AM UTC+1 hongchan
wrote:
I agree that outputLatency is better in terms of both
usability and interoperability.
FYI, the counter shows getOutputTimestamp is about 0.001%
and there are no partners who are using this API:
https://chromestatus.com/metrics/feature/popularity#AudioContextGetOutputTimestamp
<https://chromestatus.com/metrics/feature/popularity#AudioContextGetOutputTimestamp>
On Mon, Nov 22, 2021 at 3:00 PM Chris Cunningham
<[email protected]> wrote:
Re: privacy discussion, Hongchan and I scheduled to
meet with Paul Jensen next week.
Also, I recently determined that this new WebAudio
API (outputLatency) is largely redundant with an
existing API (getOutputTimestamp()).
https://github.com/WebAudio/web-audio-api/issues/2461
<https://github.com/WebAudio/web-audio-api/issues/2461>
Chrome has already shipped the latter API, so I
believe this new API doesn't add potential for
additional fingerprinting. Shipping both APIs is
still a good idea for the sake of inter-op.
On Friday, November 19, 2021 at 8:45:28 AM UTC-8
hongchan wrote:
Thanks for the clarification, Yoav!
I hope the mitigation approach in the previous
comment makes sense from the privacy perspective,
but it will hurt the usability/value of this API
significantly.
For the privacy-focused discussion, should I use
other venues rather than this I2S thread?
Cheers,
Hongchan
On Thu, Nov 18, 2021 at 10:35 PM Yoav Weiss
<[email protected]> wrote:
Apologies, I misread the discussion. A spec
issue is indeed not needed, as this is
already covered.
From my perspective, since this exposes
active entropy, the fact that it exposes more
data than the (passively exposed) platform is
not prohibitive on its own.
At the same time, it'd be good to work with
privacy-focused folks to make sure we expose
as little information as needed to maintain
the API's functionality, but not more.
On Fri, Nov 19, 2021 at 4:57 AM Chris
Cunningham <[email protected]> wrote:
> If the reported value is incorrect, the
A/V synchronization won't be possible.
(Perhaps chcunningham@ can say more about
this use case.)
The typical video player design drives
the clock with audio. In short, you send
buffers of audio to the OS, use the
latency value to know when those buffers
actually reach the user's ears, and
choose a video frame accordingly. Apps
using WebCodecs will design apps this way
(example here
<https://github.com/chcunningham/wc-talk#simple_video_playerhtml>).
As Mike notes, the latency value can vary
by quite a lot depending on the device
(huge differences for bluetooth vs
wired), so it's critical to have the
platform tell you when this changes. In
terms of precision, we can _probably_ do
some rounding like to the nearest
millisecond. Higher values may lead to
issues; user studies
<https://en.wikipedia.org/wiki/Audio-to-video_synchronization#Recommendations>
have shown sensitivity to fairly low
values of drift.
Aside: we may eventually desire a display
latency value for similar reasons.
On Thu, Nov 18, 2021 at 12:45 PM Hongchan
Choi <[email protected]> wrote:
Hi Yoav,
Could you clarify what needs to be
discussed from the spec side? The
feature (and its behavior) is
properly specified, and its privacy
aspect is already reviewed by W3C:
https://github.com/WebAudio/web-audio-api/issues/1498
<https://github.com/WebAudio/web-audio-api/issues/1498>
(outputLatency)
https://github.com/WebAudio/web-audio-api/issues/1495
<https://github.com/WebAudio/web-audio-api/issues/1495>
(general security/privacy consideration)
I thought the request from this
thread is to devise a Chrome-specific
mitigation, not reopening the spec
discussion. Please correct me if I'm
mistaken.
Cheers,
Hongchan
On Thu, Nov 18, 2021 at 12:37 PM Yoav
Weiss <[email protected]> wrote:
Hey folks!
It seems like there are some
things to figure out here before
shipping. Could y'all file a spec
issue to discuss this with the
broader community?
On Tuesday, November 16, 2021 at
6:16:14 PM UTC+1 hongchan wrote:
Thanks for the feedback,
Mike. This is really helpful!
Mitigation strategies
include adding jitter
(dithering) and
quantization so that the
exact skew is incorrectly
reported
This bit refers to the
randomization mechanism
deployed by other browsers,
but this approach defeats the
purpose of this specific API
and it is also argued in the
spec:
Note however that most
audio systems aim for low
latency, to synchronise
the audio generated by
WebAudio to other audio
or video sources or to
visual cues (for example
in a game, or an audio
recording or music making
environment). Excessive
latency decreases
usability and may be an
accessibility issue.
If the reported value is
incorrect, the A/V
synchronization won't be
possible. (Perhaps
chcunningham@ can say more
about this use case.)
One mitigation idea: we can
do some research to collect
values reported by major
platforms/devices and devise
a reasonable "bucket system"
for them.
On Tue, Nov 16, 2021 at 8:14
AM Mike Taylor
<[email protected]> wrote:
I suppose it would be
nice to do some testing
to verify the idea that
there's a small number of
platform + audio device
pairs. I was hoping
Mozilla's hardware survey
<https://data.firefox.com/dashboard/hardware>
might have something on
that, but it doesn't
track speakers.
Just from some simple
testing with the
following script (thanks
dmcardle@ for the
snippet) in Firefox on
macOS by 3 people, we
have very different
values that change
dramatically when you
unplug a headset:
```
(function getNextSample() {
console.log((new
AudioContext).outputLatency);
setTimeout(getNextSample,
10);
})();
```
Here's the values on my
macbook, using Firefox
Nightly:
bose headset plugged
in:0.005895833
take headset out: 0.013083333
plug bose headset back
in: 0.005875 (different,
but stable)
https://www.w3.org/TR/webaudio/#priv-sec
<https://www.w3.org/TR/webaudio/#priv-sec>
mentions some mitigation
strategies - do we have a
plan to apply any of them?
On 11/15/21 11:53 AM,
Ajay Rahatekar wrote:
The small number of
platform + audio output
device combination i.e.
a large number of users
with the same platform
and output
device (users on macOS
using airpods) would
likely be a natural
obfuscator. Not sure if
we have data about the
distribution.
On Mon, Nov 15, 2021 at
8:05 AM Mike Taylor
<[email protected]>
wrote:
On 11/15/21 10:56
AM, Hongchan Choi wrote:
On Mon, Nov 15,
2021 at 6:49 AM
Mike Taylor
<[email protected]>
wrote:
On 11/15/21
1:47 AM, Yoav
Weiss wrote:
On Fri, Nov
12, 2021 at
5:33 PM 'Ajay
Rahatekar' via
blink-dev
<[email protected]>
wrote:
TAG
review
status
Completed:
Web Audio
API
specification
is W3C
Recommendation.
Risks
There is a
risk of
the
feature
being used
for
fingerprinting.
However
outputLatency
is the
buffer
size of
the
platform-provided
audio
callback,
so the
value is
inherently
platform-specific.
That said,
the
majority
of the
platform
audio
buffer
size is
widely
known.
(MacOS =
128
frames,
Windows =
10ms,
Android =
96 frames,
etc)
This
feature
does not
expose
more than
what you
can
query/infer
from the
UA string.
Would that
exposure map
cleanly to
UA-Platform
<https://wicg.github.io/ua-client-hints/#sec-ch-ua-platform>,
which is
considered
low-entropy
and exposed by
default? Or
would it add
more than that?
/cc +Mike Taylor
Looking at
https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency
<https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency>,
it states that
it depends on
the platform
_and_ the
hardware output
device. If I
use an app
using
outputLatency
with speaker A,
then switch to
speaker B, will
the
outputLatency
remain the same?
The specification
says: If the audio
output device is
changed the
outputLatency
attribute value
will be updated
accordingly.
So the answer is
no. The value will
change accordingly.
If that's the case,
then outputLatency
can reveal more
entropy than just
platform alone,
right? It would be
useful to know what
this looks like in
practice, or what
mitigations we might
be able to apply
depending on the
size of these
latency differences.
