Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: 7096369deae13feb0406190c08315fb99d5ae28d
https://github.com/WebKit/WebKit/commit/7096369deae13feb0406190c08315fb99d5ae28d
Author: Jean-Yves Avenard <j...@apple.com>
Date: 2024-01-08 (Mon, 08 Jan 2024)
Changed paths:
M Source/WebCore/Modules/webcodecs/WebCodecsVideoDecoder.cpp
M Source/WebCore/SourcesCocoa.txt
M Source/WebCore/WebCore.xcodeproj/project.pbxproj
M Source/WebCore/platform/PlatformMediaError.h
M Source/WebCore/platform/VideoDecoder.cpp
M Source/WebCore/platform/VideoDecoder.h
M
Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h
M
Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm
M Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.h
M Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.mm
A Source/WebCore/platform/graphics/cocoa/VideoMediaSampleRenderer.h
A Source/WebCore/platform/graphics/cocoa/VideoMediaSampleRenderer.mm
M Source/WebCore/platform/graphics/cocoa/WebCoreDecompressionSession.h
M Source/WebCore/platform/graphics/cocoa/WebCoreDecompressionSession.mm
M Source/WebCore/platform/graphics/gstreamer/VideoDecoderGStreamer.cpp
M Source/WebCore/platform/graphics/gstreamer/VideoDecoderGStreamer.h
M Source/WebCore/platform/libwebrtc/LibWebRTCVPXVideoDecoder.cpp
M Source/WebCore/platform/libwebrtc/LibWebRTCVPXVideoDecoder.h
M Source/WebKit/WebProcess/GPU/media/RemoteVideoCodecFactory.cpp
Log Message:
-----------
[iOS] WebM video does not display
https://bugs.webkit.org/show_bug.cgi?id=266748
rdar://119493652
Reviewed by Youenn Fablet.
We registered the VP8 decoder in both the content and GPU process but on iOS
decoding was done
in mediaserverd.
So instead we decode in the GPU process before passing the decoded frames to
the AVSBDL.
This is achieved with a new VideoMediaSampleRenderer AVSBDL wrapper which will
first decode
as necessary.
For VP8 and VP9 (where hardware decoding isn't available), on iOS we use the
WebCodec's VideoDecoder
as the decoders aren't available in mediaserverd process.
To not have to rely on WebCodec being conditionally enabled, we make
VideoDecoder available
even when ENABLE_WEB_CODECS isn't set.
Add VideoDecoder config option to use a CVPixelBufferPool as CoreMedia requires
the use of
IOSurface backed CVPixelBuffer. We don't make this option the default to
prevent unexpected
regressions that would impact WebCodecs.
Manually tested on iOS as currently all webm tests are disabled on iOS.
* Source/WebCore/Modules/webcodecs/WebCodecsVideoDecoder.cpp:
(WebCore::createVideoDecoderConfig):
* Source/WebCore/SourcesCocoa.txt:
* Source/WebCore/WebCore.xcodeproj/project.pbxproj:
* Source/WebCore/platform/PlatformMediaError.h:
* Source/WebCore/platform/VideoDecoder.cpp:
(WebCore::VideoDecoder::fourCCToCodecString):
(WebCore::VideoDecoder::createLocalDecoder):
* Source/WebCore/platform/VideoDecoder.h:
*
Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h:
*
Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
(WebCore::SourceBufferPrivateAVFObjC::destroyRenderers):
(WebCore::SourceBufferPrivateAVFObjC::setCDMInstance):
(WebCore::SourceBufferPrivateAVFObjC::flushIfNeeded):
(WebCore::SourceBufferPrivateAVFObjC::outputObscuredDueToInsufficientExternalProtectionChanged):
(WebCore::SourceBufferPrivateAVFObjC::layerRequiresFlushToResumeDecodingChanged):
(WebCore::SourceBufferPrivateAVFObjC::layerReadyForDisplayChanged):
(WebCore::SourceBufferPrivateAVFObjC::flushVideo):
(WebCore::SourceBufferPrivateAVFObjC::canEnqueueSample):
(WebCore::SourceBufferPrivateAVFObjC::enqueueSample):
(WebCore::SourceBufferPrivateAVFObjC::enqueueSampleBuffer):
(WebCore::SourceBufferPrivateAVFObjC::isReadyForMoreSamples):
(WebCore::SourceBufferPrivateAVFObjC::didBecomeReadyForMoreSamples):
(WebCore::SourceBufferPrivateAVFObjC::notifyClientWhenReadyForMoreSamples):
(WebCore::SourceBufferPrivateAVFObjC::canSetMinimumUpcomingPresentationTime
const):
(WebCore::SourceBufferPrivateAVFObjC::setMinimumUpcomingPresentationTime):
(WebCore::SourceBufferPrivateAVFObjC::clearMinimumUpcomingPresentationTime):
(WebCore::SourceBufferPrivateAVFObjC::setVideoLayer):
* Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.h:
* Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.mm:
(WebCore::MediaPlayerPrivateWebM::platformLayer const):
(WebCore::MediaPlayerPrivateWebM::updateLastPixelBuffer):
(WebCore::MediaPlayerPrivateWebM::shouldEnsureLayer const):
(WebCore::MediaPlayerPrivateWebM::setPresentationSize):
(WebCore::MediaPlayerPrivateWebM::enqueueSample):
(WebCore::MediaPlayerPrivateWebM::notifyClientWhenReadyForMoreSamples):
(WebCore::MediaPlayerPrivateWebM::setMinimumUpcomingPresentationTime):
(WebCore::MediaPlayerPrivateWebM::clearMinimumUpcomingPresentationTime):
(WebCore::MediaPlayerPrivateWebM::isReadyForMoreSamples):
(WebCore::MediaPlayerPrivateWebM::didBecomeReadyForMoreSamples):
(WebCore::MediaPlayerPrivateWebM::provideMediaData):
(WebCore::MediaPlayerPrivateWebM::flushIfNeeded):
(WebCore::MediaPlayerPrivateWebM::flushTrack):
(WebCore::MediaPlayerPrivateWebM::flushVideo):
(WebCore::MediaPlayerPrivateWebM::ensureLayer):
(WebCore::MediaPlayerPrivateWebM::destroyLayer):
(WebCore::MediaPlayerPrivateWebM::canSetMinimumUpcomingPresentationTime const):
Deleted.
* Source/WebCore/platform/graphics/cocoa/VideoMediaSampleRenderer.h: Added.
(WebCore::VideoMediaSampleRenderer::create):
(WebCore::VideoMediaSampleRenderer::displayLayer const):
* Source/WebCore/platform/graphics/cocoa/VideoMediaSampleRenderer.mm: Added.
(WebCore::VideoMediaSampleRenderer::VideoMediaSampleRenderer):
(WebCore::VideoMediaSampleRenderer::~VideoMediaSampleRenderer):
(WebCore::VideoMediaSampleRenderer::isReadyForMoreMediaData const):
(WebCore::VideoMediaSampleRenderer::stopRequestingMediaData):
(WebCore::VideoMediaSampleRenderer::enqueueSample):
(WebCore::VideoMediaSampleRenderer::initializeDecompressionSession):
(WebCore::VideoMediaSampleRenderer::flush):
(WebCore::VideoMediaSampleRenderer::requestMediaDataWhenReady):
(WebCore::VideoMediaSampleRenderer::resetReadyForMoreSample):
(WebCore::VideoMediaSampleRenderer::expectMinimumUpcomingSampleBufferPresentationTime):
(WebCore::VideoMediaSampleRenderer::resetUpcomingSampleBufferPresentationTimeExpectations):
(WebCore::VideoMediaSampleRenderer::copyDisplayedPixelBuffer const):
(WebCore::VideoMediaSampleRenderer::bounds const):
* Source/WebCore/platform/graphics/cocoa/WebCoreDecompressionSession.h:
* Source/WebCore/platform/graphics/cocoa/WebCoreDecompressionSession.mm:
(WebCore::WebCoreDecompressionSession::invalidate):
(WebCore::WebCoreDecompressionSession::setTimebase):
(WebCore::WebCoreDecompressionSession::setTimebaseWithLockHeld):
(WebCore::WebCoreDecompressionSession::timebase const):
(WebCore::WebCoreDecompressionSession::maybeBecomeReadyForMoreMediaData):
(WebCore::WebCoreDecompressionSession::enqueueSample):
(WebCore::WebCoreDecompressionSession::shouldDecodeSample):
(WebCore::WebCoreDecompressionSession::ensureDecompressionSessionForSample):
(WebCore::WebCoreDecompressionSession::enqueueCompressedSample):
(WebCore::WebCoreDecompressionSession::maybeDecodeNextSample):
(WebCore::WebCoreDecompressionSession::decodeSample):
(WebCore::WebCoreDecompressionSession::setErrorListener):
(WebCore::WebCoreDecompressionSession::removeErrorListener):
(WebCore::WebCoreDecompressionSession::finishCurrentDecodingAndReportError):
(WebCore::WebCoreDecompressionSession::finishCurrentDecodingAndDecodeNextSample):
(WebCore::WebCoreDecompressionSession::decodeSampleSync):
(WebCore::WebCoreDecompressionSession::handleDecompressionOutput):
(WebCore::WebCoreDecompressionSession::automaticDequeue):
(WebCore::WebCoreDecompressionSession::enqueueDecodedSample):
(WebCore::WebCoreDecompressionSession::isReadyForMoreMediaData const):
(WebCore::WebCoreDecompressionSession::requestMediaDataWhenReady):
(WebCore::WebCoreDecompressionSession::decodedFrameWhenAvailable):
(WebCore::WebCoreDecompressionSession::stopRequestingMediaData):
(WebCore::WebCoreDecompressionSession::notifyWhenHasAvailableVideoFrame):
(WebCore::WebCoreDecompressionSession::imageForTime):
(WebCore::WebCoreDecompressionSession::flush):
(WebCore::WebCoreDecompressionSession::resetQosTier):
(WebCore::WebCoreDecompressionSession::increaseQosTier):
(WebCore::WebCoreDecompressionSession::decreaseQosTier):
(WebCore::WebCoreDecompressionSession::updateQosWithDecodeTimeStatistics):
(WebCore::WebCoreDecompressionSession::initializeVideoDecoder):
* Source/WebCore/platform/graphics/gstreamer/VideoDecoderGStreamer.cpp:
* Source/WebCore/platform/graphics/gstreamer/VideoDecoderGStreamer.h:
* Source/WebCore/platform/libwebrtc/LibWebRTCVPXVideoDecoder.cpp:
(WebCore::LibWebRTCVPXInternalVideoDecoder::create):
(WebCore::LibWebRTCVPXVideoDecoder::create):
(WebCore::LibWebRTCVPXVideoDecoder::LibWebRTCVPXVideoDecoder):
(WebCore::LibWebRTCVPXInternalVideoDecoder::LibWebRTCVPXInternalVideoDecoder):
(WebCore::LibWebRTCVPXInternalVideoDecoder::pixelBufferPool):
(WebCore::LibWebRTCVPXInternalVideoDecoder::Decoded):
* Source/WebCore/platform/libwebrtc/LibWebRTCVPXVideoDecoder.h:
Canonical link: https://commits.webkit.org/272760@main
_______________________________________________
webkit-changes mailing list
webkit-changes@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-changes
