- Revision
- 283825
- Author
- [email protected]
- Date
- 2021-10-08 12:29:52 -0700 (Fri, 08 Oct 2021)
Log Message
Merge optimizations to AudioParamTimeline::hasValues() from Blink
https://bugs.webkit.org/show_bug.cgi?id=231428
Reviewed by Eric Carlson.
LayoutTests/imported/w3c:
Merge https://github.com/web-platform-tests/wpt/pull/31165. Thanks to the new optimizations, we are now
using K-Rate processing in more cases. For some of the StereoPannerNode tests, we went from A-Rate processing
to K-Rate. Because our K-Rate processing implementation is vectorized (and doing all operations as float), it
does not have the same precision as the A-Rate one which does operations of double variables and only casts
to float at the end.
* web-platform-tests/webaudio/resources/stereopanner-testing.js:
(Test):
* web-platform-tests/webaudio/the-audio-api/the-stereopanner-interface/no-dezippering-expected.txt:
* web-platform-tests/webaudio/the-audio-api/the-stereopanner-interface/no-dezippering.html:
* web-platform-tests/webaudio/the-audio-api/the-stereopanner-interface/stereopannernode-panning-expected.txt:
Source/WebCore:
Merge optimizations to AudioParamTimeline::hasValues() from Blink:
- https://github.com/chromium/chromium/blob/03cb001d63071529abb42492eacee8d8bba7a4db/third_party/blink/renderer/modules/webaudio/audio_param_timeline.cc#L620
It is important for AudioParamTimeline::hasValues() to return false whenever
possible because it allows our implementation to use K-Rate processing (instead
of A-Rate), which is much faster.
No new tests, no expected Web-facing behavior change.
* Modules/webaudio/AudioParamTimeline.cpp:
(WebCore::AudioParamTimeline::hasValues const):
Merge optimizations to AudioParamTimeline::hasValues() from Blink:
- https://github.com/chromium/chromium/blob/03cb001d63071529abb42492eacee8d8bba7a4db/third_party/blink/renderer/modules/webaudio/audio_param_timeline.cc#L620
When there is a single event on the timeline (which is fairly common), do our best to
find out if the event is relevant for the current rendering quantum, based on the event
type and attributes.
* Modules/webaudio/PannerNode.cpp:
(WebCore::PannerNode::process):
I recently introduced a cache of PannerNode's azimuth/elevavation/coneGain. The cache
works fine as long as PannerNode processing stays in K-Rate or in A-Rate the whole
time. However, if switching from A-Rate to K-rate, it could make bad calculations
because we failed to invalidate the cached attributes while in A-Rate. For this
reason, we have to invalidate the cached attributes if dirty no matter what rate we
are using (not just while in K-Rate).
* platform/audio/EqualPowerPanner.cpp:
(WebCore::EqualPowerPanner::EqualPowerPanner):
(WebCore::EqualPowerPanner::pan):
* platform/audio/EqualPowerPanner.h:
* platform/audio/Panner.cpp:
(WebCore::Panner::create):
Now that more Panner tests are using K-Rate processing, it found a bug in our
implementation where we were still doing dezippering. Dezippering is no longer
part of the specification and Blink no longer does this either. We've already
dropped dezippering in other places but we missed it here.
I have also noticed that the implementation is not vectorized but easily could
be so I have added a FIXME comment and will follow-up to fix it.
Modified Paths
Diff
Modified: trunk/LayoutTests/imported/w3c/ChangeLog (283824 => 283825)
--- trunk/LayoutTests/imported/w3c/ChangeLog 2021-10-08 19:19:16 UTC (rev 283824)
+++ trunk/LayoutTests/imported/w3c/ChangeLog 2021-10-08 19:29:52 UTC (rev 283825)
@@ -1,3 +1,22 @@
+2021-10-08 Chris Dumez <[email protected]>
+
+ Merge optimizations to AudioParamTimeline::hasValues() from Blink
+ https://bugs.webkit.org/show_bug.cgi?id=231428
+
+ Reviewed by Eric Carlson.
+
+ Merge https://github.com/web-platform-tests/wpt/pull/31165. Thanks to the new optimizations, we are now
+ using K-Rate processing in more cases. For some of the StereoPannerNode tests, we went from A-Rate processing
+ to K-Rate. Because our K-Rate processing implementation is vectorized (and doing all operations as float), it
+ does not have the same precision as the A-Rate one which does operations of double variables and only casts
+ to float at the end.
+
+ * web-platform-tests/webaudio/resources/stereopanner-testing.js:
+ (Test):
+ * web-platform-tests/webaudio/the-audio-api/the-stereopanner-interface/no-dezippering-expected.txt:
+ * web-platform-tests/webaudio/the-audio-api/the-stereopanner-interface/no-dezippering.html:
+ * web-platform-tests/webaudio/the-audio-api/the-stereopanner-interface/stereopannernode-panning-expected.txt:
+
2021-10-08 Tim Nguyen <[email protected]>
Re-sync the-dialog-element and inert WPT from upstream
Modified: trunk/LayoutTests/imported/w3c/web-platform-tests/webaudio/resources/stereopanner-testing.js (283824 => 283825)
--- trunk/LayoutTests/imported/w3c/web-platform-tests/webaudio/resources/stereopanner-testing.js 2021-10-08 19:19:16 UTC (rev 283824)
+++ trunk/LayoutTests/imported/w3c/web-platform-tests/webaudio/resources/stereopanner-testing.js 2021-10-08 19:29:52 UTC (rev 283825)
@@ -79,7 +79,7 @@
// The max error we allow between the rendered impulse and the
// expected value. This value is experimentally determined. Set
// to 0 to make the test fail to see what the actual error is.
- this.maxAllowedError = 9.8015e-8;
+ this.maxAllowedError = 1.284318e-7;
// Max (absolute) error and the index of the maxima for the left
// and right channels.
Modified: trunk/LayoutTests/imported/w3c/web-platform-tests/webaudio/the-audio-api/the-stereopanner-interface/no-dezippering-expected.txt (283824 => 283825)
--- trunk/LayoutTests/imported/w3c/web-platform-tests/webaudio/the-audio-api/the-stereopanner-interface/no-dezippering-expected.txt 2021-10-08 19:19:16 UTC (rev 283824)
+++ trunk/LayoutTests/imported/w3c/web-platform-tests/webaudio/the-audio-api/the-stereopanner-interface/no-dezippering-expected.txt 2021-10-08 19:29:52 UTC (rev 283825)
@@ -19,21 +19,21 @@
PASS Stereo: Right channel, pan = 1: contains only the constant 3.
PASS < [test stereo input] All assertions passed. (total 4 assertions)
PASS > [test mono input setValue] Test StereoPanner with mono input value setter vs setValueAtTime
-PASS Mono Left .value setter output is identical to the array [expected array].
+PASS Mono Left .value setter output equals [expected array] with an element-wise tolerance of {"absoluteThreshold":1.192094e-7,"relativeThreshold":0}.
PASS Mono Left .value setter output matches setValueAtTime output is true.
-PASS Mono Right .value setter output is identical to the array [expected array].
+PASS Mono Right .value setter output equals [expected array] with an element-wise tolerance of {"absoluteThreshold":1.192094e-7,"relativeThreshold":0}.
PASS Mono Right .value setter output matches setValueAtTime output is true.
PASS < [test mono input setValue] All assertions passed. (total 4 assertions)
PASS > [test stereo input setValue] Test StereoPanner with mono input value setter vs setValueAtTime
-PASS Stereo Left .value setter output is identical to the array [expected array].
+PASS Stereo Left .value setter output equals [expected array] with an element-wise tolerance of {"absoluteThreshold":1.192094e-7,"relativeThreshold":0}.
PASS Stereo Left .value setter output matches setValueAtTime output is true.
-PASS Stereo Right .value setter output is identical to the array [expected array].
+PASS Stereo Right .value setter output equals [expected array] with an element-wise tolerance of {"absoluteThreshold":1.192094e-7,"relativeThreshold":0}.
PASS Stereo Right .value setter output matches setValueAtTime output is true.
PASS < [test stereo input setValue] All assertions passed. (total 4 assertions)
PASS > [test mono input automation] Test StereoPanner with mono input and automation
-PASS Modulated Stereo Left .value setter output is identical to the array [expected array].
+PASS Modulated Stereo Left .value setter output equals [expected array] with an element-wise tolerance of {"absoluteThreshold":1.192094e-7,"relativeThreshold":0}.
PASS Modulated Stereo Left .value setter output matches setValueAtTime output is true.
-PASS Modulated Stereo Right .value setter output is identical to the array [expected array].
+PASS Modulated Stereo Right .value setter output equals [expected array] with an element-wise tolerance of {"absoluteThreshold":1.192094e-7,"relativeThreshold":0}.
PASS Modulated Stereo Right .value setter output matches setValueAtTime output is true.
PASS < [test mono input automation] All assertions passed. (total 4 assertions)
PASS # AUDIT TASK RUNNER FINISHED: 5 tasks ran successfully.
Modified: trunk/LayoutTests/imported/w3c/web-platform-tests/webaudio/the-audio-api/the-stereopanner-interface/no-dezippering.html (283824 => 283825)
--- trunk/LayoutTests/imported/w3c/web-platform-tests/webaudio/the-audio-api/the-stereopanner-interface/no-dezippering.html 2021-10-08 19:19:16 UTC (rev 283824)
+++ trunk/LayoutTests/imported/w3c/web-platform-tests/webaudio/the-audio-api/the-stereopanner-interface/no-dezippering.html 2021-10-08 19:29:52 UTC (rev 283825)
@@ -244,7 +244,7 @@
should(
actual[k],
options.prefix + ' ' + label[k] + ' .value setter output')
- .beEqualToArray(expected[k]);
+ .beCloseToArray(expected[k], {absoluteThreshold: 1.192094e-7});
should(
match,
options.prefix + ' ' + label[k] +
Modified: trunk/LayoutTests/imported/w3c/web-platform-tests/webaudio/the-audio-api/the-stereopanner-interface/stereopannernode-panning-expected.txt (283824 => 283825)
--- trunk/LayoutTests/imported/w3c/web-platform-tests/webaudio/the-audio-api/the-stereopanner-interface/stereopannernode-panning-expected.txt 2021-10-08 19:19:16 UTC (rev 283824)
+++ trunk/LayoutTests/imported/w3c/web-platform-tests/webaudio/the-audio-api/the-stereopanner-interface/stereopannernode-panning-expected.txt 2021-10-08 19:29:52 UTC (rev 283825)
@@ -6,14 +6,14 @@
PASS > [mono-test]
PASS Mono: Number of impulses found is equal to 100.
PASS Mono: Number of impulse at the wrong offset is equal to 0.
-PASS Mono: Left channel error magnitude is less than or equal to 9.8015e-8.
-PASS Mono: Right channel error magnitude is less than or equal to 9.8015e-8.
+PASS Mono: Left channel error magnitude is less than or equal to 1.284318e-7.
+PASS Mono: Right channel error magnitude is less than or equal to 1.284318e-7.
PASS < [mono-test] All assertions passed. (total 4 assertions)
PASS > [stereo-test]
PASS Stereo: Number of impulses found is equal to 100.
PASS Stereo: Number of impulse at the wrong offset is equal to 0.
-PASS Stereo: Left channel error magnitude is less than or equal to 9.8015e-8.
-PASS Stereo: Right channel error magnitude is less than or equal to 9.8015e-8.
+PASS Stereo: Left channel error magnitude is less than or equal to 1.284318e-7.
+PASS Stereo: Right channel error magnitude is less than or equal to 1.284318e-7.
PASS < [stereo-test] All assertions passed. (total 4 assertions)
PASS # AUDIT TASK RUNNER FINISHED: 2 tasks ran successfully.
Modified: trunk/Source/WebCore/ChangeLog (283824 => 283825)
--- trunk/Source/WebCore/ChangeLog 2021-10-08 19:19:16 UTC (rev 283824)
+++ trunk/Source/WebCore/ChangeLog 2021-10-08 19:29:52 UTC (rev 283825)
@@ -1,3 +1,49 @@
+2021-10-08 Chris Dumez <[email protected]>
+
+ Merge optimizations to AudioParamTimeline::hasValues() from Blink
+ https://bugs.webkit.org/show_bug.cgi?id=231428
+
+ Reviewed by Eric Carlson.
+
+ Merge optimizations to AudioParamTimeline::hasValues() from Blink:
+ - https://github.com/chromium/chromium/blob/03cb001d63071529abb42492eacee8d8bba7a4db/third_party/blink/renderer/modules/webaudio/audio_param_timeline.cc#L620
+
+ It is important for AudioParamTimeline::hasValues() to return false whenever
+ possible because it allows our implementation to use K-Rate processing (instead
+ of A-Rate), which is much faster.
+
+ No new tests, no expected Web-facing behavior change.
+
+ * Modules/webaudio/AudioParamTimeline.cpp:
+ (WebCore::AudioParamTimeline::hasValues const):
+ Merge optimizations to AudioParamTimeline::hasValues() from Blink:
+ - https://github.com/chromium/chromium/blob/03cb001d63071529abb42492eacee8d8bba7a4db/third_party/blink/renderer/modules/webaudio/audio_param_timeline.cc#L620
+ When there is a single event on the timeline (which is fairly common), do our best to
+ find out if the event is relevant for the current rendering quantum, based on the event
+ type and attributes.
+
+ * Modules/webaudio/PannerNode.cpp:
+ (WebCore::PannerNode::process):
+ I recently introduced a cache of PannerNode's azimuth/elevavation/coneGain. The cache
+ works fine as long as PannerNode processing stays in K-Rate or in A-Rate the whole
+ time. However, if switching from A-Rate to K-rate, it could make bad calculations
+ because we failed to invalidate the cached attributes while in A-Rate. For this
+ reason, we have to invalidate the cached attributes if dirty no matter what rate we
+ are using (not just while in K-Rate).
+
+ * platform/audio/EqualPowerPanner.cpp:
+ (WebCore::EqualPowerPanner::EqualPowerPanner):
+ (WebCore::EqualPowerPanner::pan):
+ * platform/audio/EqualPowerPanner.h:
+ * platform/audio/Panner.cpp:
+ (WebCore::Panner::create):
+ Now that more Panner tests are using K-Rate processing, it found a bug in our
+ implementation where we were still doing dezippering. Dezippering is no longer
+ part of the specification and Blink no longer does this either. We've already
+ dropped dezippering in other places but we missed it here.
+ I have also noticed that the implementation is not vectorized but easily could
+ be so I have added a FIXME comment and will follow-up to fix it.
+
2021-10-08 Andres Gonzalez <[email protected]>
AX: AXMathRootRadicand should return an array of elements, not just the first
Modified: trunk/Source/WebCore/Modules/webaudio/AudioParamTimeline.cpp (283824 => 283825)
--- trunk/Source/WebCore/Modules/webaudio/AudioParamTimeline.cpp 2021-10-08 19:19:16 UTC (rev 283824)
+++ trunk/Source/WebCore/Modules/webaudio/AudioParamTimeline.cpp 2021-10-08 19:29:52 UTC (rev 283825)
@@ -1005,11 +1005,44 @@
{
if (!m_eventsLock.tryLock())
return true;
+
Locker locker { AdoptLock, m_eventsLock };
- // Return false if there are no events in the time range.
- auto endFrame = startFrame + AudioUtilities::renderQuantumSize;
- return !m_events.isEmpty() && endFrame / sampleRate > m_events[0].time().value();
+ if (m_events.isEmpty())
+ return false;
+
+ if (m_events[0].time().value() > (startFrame + AudioUtilities::renderQuantumSize) / sampleRate) {
+ // The first event starts after the end of this rendering quantum so no automation is needed.
+ auto eventType = m_events[0].type();
+ if (eventType == ParamEvent::SetTarget || eventType == ParamEvent::SetValue || eventType == ParamEvent::SetValueCurve)
+ return false;
+ }
+
+ // Don't try and optimize when there is more than one event in the timeline as it gets complicated.
+ if (m_events.size() > 1)
+ return true;
+
+ switch (m_events[0].type()) {
+ case ParamEvent::SetTarget:
+ // Need automation if the event starts somewhere before the end of the current render quantum.
+ return m_events[0].time().value() <= (startFrame + AudioUtilities::renderQuantumSize) / sampleRate;
+ case ParamEvent::SetValue:
+ case ParamEvent::LinearRampToValue:
+ case ParamEvent::ExponentialRampToValue:
+ case ParamEvent::CancelValues:
+ // If these events are in the past, we don't need any automation; the value is a constant.
+ return m_events[0].time().value() >= startFrame / sampleRate;
+ case ParamEvent::SetValueCurve: {
+ auto curveEndTime = m_events[0].time() + m_events[0].duration();
+ double startTime = startFrame / sampleRate;
+ return m_events[0].time().value() <= startTime && startTime < curveEndTime.value();
+ }
+ case ParamEvent::LastType:
+ ASSERT_NOT_REACHED();
+ break;
+ }
+
+ return true;
}
} // namespace WebCore
Modified: trunk/Source/WebCore/Modules/webaudio/PannerNode.cpp (283824 => 283825)
--- trunk/Source/WebCore/Modules/webaudio/PannerNode.cpp 2021-10-08 19:19:16 UTC (rev 283824)
+++ trunk/Source/WebCore/Modules/webaudio/PannerNode.cpp 2021-10-08 19:29:52 UTC (rev 283825)
@@ -143,13 +143,13 @@
}
}
+ invalidateCachedPropertiesIfNecessary();
+
if ((hasSampleAccurateValues() || listener().hasSampleAccurateValues()) && (shouldUseARate() || listener().shouldUseARate())) {
processSampleAccurateValues(destination, source, framesToProcess);
return;
}
- invalidateCachedPropertiesIfNecessary();
-
// Apply the panning effect.
auto [azimuth, elevation] = azimuthElevation();
m_panner->pan(azimuth, elevation, source, destination, framesToProcess);
Modified: trunk/Source/WebCore/platform/audio/EqualPowerPanner.cpp (283824 => 283825)
--- trunk/Source/WebCore/platform/audio/EqualPowerPanner.cpp 2021-10-08 19:19:16 UTC (rev 283824)
+++ trunk/Source/WebCore/platform/audio/EqualPowerPanner.cpp 2021-10-08 19:29:52 UTC (rev 283825)
@@ -33,15 +33,11 @@
#include <algorithm>
#include <wtf/MathExtras.h>
-// Use a 50ms smoothing / de-zippering time-constant.
-constexpr float SmoothingTimeConstant = 0.050f;
-
namespace WebCore {
-EqualPowerPanner::EqualPowerPanner(float sampleRate)
+EqualPowerPanner::EqualPowerPanner()
: Panner(PanningModelType::Equalpower)
{
- m_smoothingConstant = AudioUtilities::discreteTimeConstantForSampleRate(SmoothingTimeConstant, sampleRate);
}
void EqualPowerPanner::calculateDesiredGain(double& desiredGainL, double& desiredGainR, double azimuth, unsigned numberOfChannels)
@@ -114,8 +110,6 @@
azimuth = 180 - azimuth;
double desiredPanPosition;
- double desiredGainL;
- double desiredGainR;
if (numberOfInputChannels == 1) { // For mono source case.
// Pan smoothly from left to right with azimuth going from -90 -> +90 degrees.
@@ -132,32 +126,17 @@
}
}
- desiredGainL = cos(0.5 * piDouble * desiredPanPosition);
- desiredGainR = sin(0.5 * piDouble * desiredPanPosition);
-
- // Don't de-zipper on first render call.
- if (m_isFirstRender) {
- m_isFirstRender = false;
- m_gainL = desiredGainL;
- m_gainR = desiredGainR;
- }
+ double desiredGainL = cos(piOverTwoDouble * desiredPanPosition);
+ double desiredGainR = sin(piOverTwoDouble * desiredPanPosition);
- // Cache in local variables.
- double gainL = m_gainL;
- double gainR = m_gainR;
-
- // Get local copy of smoothing constant.
- const double SmoothingConstant = m_smoothingConstant;
-
int n = framesToProcess;
+ // FIXME: We should vectorize this.
if (numberOfInputChannels == 1) { // For mono source case.
while (n--) {
float inputL = *sourceL++;
- gainL += (desiredGainL - gainL) * SmoothingConstant;
- gainR += (desiredGainR - gainR) * SmoothingConstant;
- *destinationL++ = static_cast<float>(inputL * gainL);
- *destinationR++ = static_cast<float>(inputL * gainR);
+ *destinationL++ = static_cast<float>(inputL * desiredGainL);
+ *destinationR++ = static_cast<float>(inputL * desiredGainR);
}
} else { // For stereo source case.
if (azimuth <= 0) { // from -90 -> 0
@@ -164,25 +143,18 @@
while (n--) {
float inputL = *sourceL++;
float inputR = *sourceR++;
- gainL += (desiredGainL - gainL) * SmoothingConstant;
- gainR += (desiredGainR - gainR) * SmoothingConstant;
- *destinationL++ = static_cast<float>(inputL + inputR * gainL);
- *destinationR++ = static_cast<float>(inputR * gainR);
+ *destinationL++ = static_cast<float>(inputL + inputR * desiredGainL);
+ *destinationR++ = static_cast<float>(inputR * desiredGainR);
}
} else { // from 0 -> +90
while (n--) {
float inputL = *sourceL++;
float inputR = *sourceR++;
- gainL += (desiredGainL - gainL) * SmoothingConstant;
- gainR += (desiredGainR - gainR) * SmoothingConstant;
- *destinationL++ = static_cast<float>(inputL * gainL);
- *destinationR++ = static_cast<float>(inputR + inputL * gainR);
+ *destinationL++ = static_cast<float>(inputL * desiredGainL);
+ *destinationR++ = static_cast<float>(inputR + inputL * desiredGainR);
}
}
}
-
- m_gainL = gainL;
- m_gainR = gainR;
}
void EqualPowerPanner::panWithSampleAccurateValues(double* azimuth, double*, const AudioBus* inputBus, AudioBus* outputBus, size_t framesToProcess)
Modified: trunk/Source/WebCore/platform/audio/EqualPowerPanner.h (283824 => 283825)
--- trunk/Source/WebCore/platform/audio/EqualPowerPanner.h 2021-10-08 19:19:16 UTC (rev 283824)
+++ trunk/Source/WebCore/platform/audio/EqualPowerPanner.h 2021-10-08 19:29:52 UTC (rev 283825)
@@ -33,12 +33,12 @@
class EqualPowerPanner final : public Panner {
public:
- explicit EqualPowerPanner(float sampleRate);
+ EqualPowerPanner();
void pan(double azimuth, double elevation, const AudioBus* inputBus, AudioBus* outputBuf, size_t framesToProcess) final;
void panWithSampleAccurateValues(double* azimuth, double* elevation, const AudioBus* inputBus, AudioBus* outputBus, size_t framesToProcess) final;
- void reset() override { m_isFirstRender = true; }
+ void reset() override { }
double tailTime() const override { return 0; }
double latencyTime() const override { return 0; }
@@ -46,13 +46,6 @@
private:
void calculateDesiredGain(double& desiredGainL, double& desiredGainR, double azimuth, unsigned numberOfChannels);
-
- // For smoothing / de-zippering
- bool m_isFirstRender { true };
- double m_smoothingConstant;
-
- double m_gainL { 0 };
- double m_gainR { 0 };
};
} // namespace WebCore
Modified: trunk/Source/WebCore/platform/audio/Panner.cpp (283824 => 283825)
--- trunk/Source/WebCore/platform/audio/Panner.cpp 2021-10-08 19:19:16 UTC (rev 283824)
+++ trunk/Source/WebCore/platform/audio/Panner.cpp 2021-10-08 19:29:52 UTC (rev 283825)
@@ -43,7 +43,7 @@
switch (model) {
case PanningModelType::Equalpower:
- panner = makeUnique<EqualPowerPanner>(sampleRate);
+ panner = makeUnique<EqualPowerPanner>();
break;
case PanningModelType::HRTF:
