Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: 1774dbef65c9c409eb4a1c61b476033c6255e601
https://github.com/WebKit/WebKit/commit/1774dbef65c9c409eb4a1c61b476033c6255e601
Author: Jer Noble <[email protected]>
Date: 2026-04-03 (Fri, 03 Apr 2026)
Changed paths:
M LayoutTests/TestExpectations
M
LayoutTests/imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/preserves-pitch-expected.txt
M LayoutTests/platform/glib/TestExpectations
M Source/WebCore/Modules/webaudio/MediaElementAudioSourceNode.cpp
M Source/WebCore/Modules/webaudio/MediaElementAudioSourceNode.h
M Source/WebCore/PAL/pal/cf/AudioToolboxSoftLink.cpp
M Source/WebCore/PAL/pal/cf/AudioToolboxSoftLink.h
M Source/WebCore/SourcesCocoa.txt
M Source/WebCore/WebCore.xcodeproj/project.pbxproj
M Source/WebCore/html/HTMLMediaElement.cpp
M Source/WebCore/platform/audio/cocoa/AudioSampleDataSource.mm
M Source/WebCore/platform/audio/cocoa/CARingBuffer.cpp
M Source/WebCore/platform/audio/cocoa/CARingBuffer.h
A Source/WebCore/platform/audio/cocoa/PitchShiftAudioUnit.h
A Source/WebCore/platform/audio/cocoa/PitchShiftAudioUnit.mm
M Source/WebCore/platform/cocoa/NetworkExtensionContentFilter.mm
M Source/WebCore/platform/cocoa/PowerSourceNotifier.h
M Source/WebCore/platform/graphics/avfoundation/AudioSourceProviderAVFObjC.h
M
Source/WebCore/platform/graphics/avfoundation/AudioSourceProviderAVFObjC.mm
M
Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h
M
Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm
M Source/WebCore/platform/mediastream/WebAudioSourceProvider.h
M
Source/WebCore/platform/mediastream/cocoa/MediaStreamTrackAudioSourceProviderCocoa.cpp
M
Source/WebCore/platform/mediastream/cocoa/MediaStreamTrackAudioSourceProviderCocoa.h
M Source/WebCore/platform/mediastream/cocoa/WebAudioSourceProviderCocoa.h
M Source/WebCore/platform/mediastream/cocoa/WebAudioSourceProviderCocoa.mm
M Source/WebCore/platform/video-codecs/cocoa/WebRTCVideoDecoderVTB.mm
M Source/WebKit/GPUProcess/media/RemoteAudioSourceProviderProxy.cpp
M Source/WebKit/GPUProcess/media/RemoteAudioSourceProviderProxy.h
M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp
M Source/WebKit/WebKit.xcodeproj/project.pbxproj
M Source/WebKit/WebProcess/GPU/GPUProcessConnection.cpp
M Source/WebKit/WebProcess/GPU/media/RemoteAudioSourceProvider.cpp
M Source/WebKit/WebProcess/GPU/media/RemoteAudioSourceProvider.h
M Source/WebKit/WebProcess/GPU/media/RemoteAudioSourceProviderManager.cpp
M Source/WebKit/WebProcess/GPU/media/RemoteAudioSourceProviderManager.h
M
Source/WebKit/WebProcess/GPU/media/RemoteAudioSourceProviderManager.messages.in
M Tools/TestWebKitAPI/SourcesCocoa.txt
M Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj
A Tools/TestWebKitAPI/Tests/WebKitCocoa/PitchShiftAudioUnitTests.cpp
Log Message:
-----------
HTMLMediaElement preservesPitch=true does not work when using AudioContext
rdar://173727365
https://bugs.webkit.org/show_bug.cgi?id=311000
Reviewed by Eric Carlson.
Add support for playing HTMLMediaElements at non-1x rates with
`preservesPitch=true` through
MediaElementSourceNode. Due to the nature of AVAudioMix, this requires pitch
correction to be
applied after the tap emits audio samples.
When an AVPlayer is configured to use "varispeed" and plays at a rate of (e.g.)
2x, the tap will
emit 88,200 samples per second for content encoded at 44.1khz. This is
essentially a sample rate
conversion, and the output can be converted by MultiChannelSampleRateConverter
to 44.1khz, with the
side effect of pitch shifting upwards by (e.g., again) one octave. To
counteract this pitch change
and simultaneously resample the content, a new class PitchShiftAudioNode is
added, which wraps an
AudioUnit provided by CoreAudio. This new class operates very similarly to the
MultiChannelSampleRateConverter, insofar as it uses a callback to request more
data from the
caller, then writes the output data to a AudioBus provided by the caller.
Two concrete classes were modified to incorporate this new PitchShiftAudioNode:
AudioSourceProviderAVFObjC
and WebAudioSourceProviderCocoa. These two classes are now very similar, and
are targets for merging
together in the future. The MultiChannelSampleRateConverter was removed from
MediaElementSourceNode
and added to those same two classes, to handle the case where the connected
audio element had
`preservesPitch=false`.
Both playbackRate and preservesPitch were piped through from the GPU process to
the WebContent
process so that both ends of the audio provider could be configured correctly.
With this change, two subtests in
imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/preserves-pitch.html
now pass.
Test: Tools/TestWebKitAPI/Tests/WebKitCocoa/PitchShiftAudioUnitTests.cpp
* LayoutTests/TestExpectations:
*
LayoutTests/imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/preserves-pitch-expected.txt:
* Source/WebCore/Modules/webaudio/MediaElementAudioSourceNode.cpp:
(WebCore::MediaElementAudioSourceNode::MediaElementAudioSourceNode):
(WebCore::MediaElementAudioSourceNode::updateResamplerIfNeeded):
(WebCore::MediaElementAudioSourceNode::setPlaybackRate): Deleted.
* Source/WebCore/Modules/webaudio/MediaElementAudioSourceNode.h:
* Source/WebCore/PAL/pal/cf/AudioToolboxSoftLink.cpp:
* Source/WebCore/PAL/pal/cf/AudioToolboxSoftLink.h:
* Source/WebCore/SourcesCocoa.txt:
* Source/WebCore/WebCore.xcodeproj/project.pbxproj:
* Source/WebCore/html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::mediaPlayerRateChanged):
* Source/WebCore/platform/audio/cocoa/AudioSampleDataSource.mm:
(WebCore::AudioSampleDataSource::pullSamples):
(WebCore::AudioSampleDataSource::pullAvailableSamplesAsChunks):
* Source/WebCore/platform/audio/cocoa/CARingBuffer.cpp:
(WebCore::CARingBuffer::store):
* Source/WebCore/platform/audio/cocoa/CARingBuffer.h:
* Source/WebCore/platform/audio/cocoa/PitchShiftAudioUnit.h: Added.
(WebCore::PitchShiftAudioUnit::rate const):
(WebCore::PitchShiftAudioUnit::pitch const):
* Source/WebCore/platform/audio/cocoa/PitchShiftAudioUnit.mm: Added.
(WebCore::PitchShiftAudioUnit::PitchShiftAudioUnit):
(WebCore::PitchShiftAudioUnit::~PitchShiftAudioUnit):
(WebCore::PitchShiftAudioUnit::setRate):
(WebCore::PitchShiftAudioUnit::setPitch):
(WebCore::PitchShiftAudioUnit::setInputCallback):
(WebCore::PitchShiftAudioUnit::render):
(WebCore::PitchShiftAudioUnit::renderCallback):
* Source/WebCore/platform/cocoa/NetworkExtensionContentFilter.mm:
* Source/WebCore/platform/cocoa/PowerSourceNotifier.h:
* Source/WebCore/platform/graphics/avfoundation/AudioSourceProviderAVFObjC.h:
* Source/WebCore/platform/graphics/avfoundation/AudioSourceProviderAVFObjC.mm:
(WebCore::AudioSourceProviderAVFObjC::provideInput):
(WebCore::AudioSourceProviderAVFObjC::provideInputInternal):
(WebCore::AudioSourceProviderAVFObjC::setPlaybackRate):
(WebCore::AudioSourceProviderAVFObjC::setPreservesPitch):
(WebCore::AudioSourceProviderAVFObjC::prepare):
(WebCore::AudioSourceProviderAVFObjC::unprepare):
(WebCore::AudioSourceProviderAVFObjC::process):
*
Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h:
*
Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
(WebCore::MediaPlayerPrivateAVFoundationObjC::createAVPlayerItem):
(WebCore::MediaPlayerPrivateAVFoundationObjC::setPreservesPitch):
*
Source/WebCore/platform/mediastream/cocoa/MediaStreamTrackAudioSourceProviderCocoa.cpp:
(WebCore::MediaStreamTrackAudioSourceProviderCocoa::audioSamplesAvailable):
* Source/WebCore/platform/mediastream/cocoa/WebAudioSourceProviderCocoa.h:
* Source/WebCore/platform/mediastream/cocoa/WebAudioSourceProviderCocoa.mm:
(WebCore::WebAudioSourceProviderCocoa::setClient):
(WebCore::WebAudioSourceProviderCocoa::setPlaybackRate):
(WebCore::WebAudioSourceProviderCocoa::setPreservesPitch):
(WebCore::WebAudioSourceProviderCocoa::audioStorageChanged):
(WebCore::WebAudioSourceProviderCocoa::provideInput):
(WebCore::WebAudioSourceProviderCocoa::provideInputInternal):
(WebCore::WebAudioSourceProviderCocoa::prepare):
(WebCore::WebAudioSourceProviderCocoa::receivedNewAudioSamples):
(WebCore::WebAudioSourceProviderCocoa::setNeedsFlush):
* Source/WebCore/platform/video-codecs/cocoa/WebRTCVideoDecoderVTB.mm:
* Source/WebKit/GPUProcess/media/RemoteAudioSourceProviderProxy.cpp:
(WebKit::RemoteAudioSourceProviderProxy::newAudioSamples):
(WebKit::RemoteAudioSourceProviderProxy::setPlaybackRate):
(WebKit::RemoteAudioSourceProviderProxy::setPreservesPitch):
* Source/WebKit/GPUProcess/media/RemoteAudioSourceProviderProxy.h:
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp:
(WebKit::RemoteMediaPlayerProxy::setPreservesPitch):
(WebKit::RemoteMediaPlayerProxy::mediaPlayerRateChanged):
(WebKit::RemoteMediaPlayerProxy::createAudioSourceProvider):
* Source/WebKit/WebKit.xcodeproj/project.pbxproj:
* Source/WebKit/WebProcess/GPU/GPUProcessConnection.cpp:
(WebKit::GPUProcessConnection::audioSourceProviderManager):
(WebKit::GPUProcessConnection::dispatchMessage):
* Source/WebKit/WebProcess/GPU/media/RemoteAudioSourceProvider.cpp:
(WebKit::RemoteAudioSourceProvider::audioSamplesAvailable):
* Source/WebKit/WebProcess/GPU/media/RemoteAudioSourceProvider.h:
* Source/WebKit/WebProcess/GPU/media/RemoteAudioSourceProviderManager.cpp:
(WebKit::RemoteAudioSourceProviderManager::RemoteAudioSourceProviderManager):
(WebKit::m_queue):
(WebKit::RemoteAudioSourceProviderManager::~RemoteAudioSourceProviderManager):
(WebKit::RemoteAudioSourceProviderManager::stopListeningForIPC):
(WebKit::RemoteAudioSourceProviderManager::addProvider):
(WebKit::RemoteAudioSourceProviderManager::removeProvider):
(WebKit::RemoteAudioSourceProviderManager::providerFor):
(WebKit::RemoteAudioSourceProviderManager::audioStorageChanged):
(WebKit::RemoteAudioSourceProviderManager::setNeedsFlush):
(WebKit::RemoteAudioSourceProviderManager::setPlaybackRate):
(WebKit::RemoteAudioSourceProviderManager::setPreservesPitch):
(WebKit::RemoteAudioSourceProviderManager::setConnection): Deleted.
(WebKit::RemoteAudioSourceProviderManager::audioSamplesAvailable): Deleted.
(WebKit::RemoteAudioSourceProviderManager::RemoteAudio::RemoteAudio): Deleted.
(WebKit::RemoteAudioSourceProviderManager::RemoteAudio::setStorage): Deleted.
(WebKit::RemoteAudioSourceProviderManager::RemoteAudio::audioSamplesAvailable):
Deleted.
* Source/WebKit/WebProcess/GPU/media/RemoteAudioSourceProviderManager.h:
(WebKit::RemoteAudioSourceProviderManager::create):
*
Source/WebKit/WebProcess/GPU/media/RemoteAudioSourceProviderManager.messages.in:
* Tools/TestWebKitAPI/Tests/WebKitCocoa/PitchShiftAudioUnitTests.cpp: Added.
(generateSineWave):
(countZeroCrossings):
(audioFormat):
(TEST(PitchShiftAudioUnit, BasicProcessing)):
Canonical link: https://commits.webkit.org/310521@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
