Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: 93f6303ff83e5c20dbd79f5e6ae7c8a801ee519d
https://github.com/WebKit/WebKit/commit/93f6303ff83e5c20dbd79f5e6ae7c8a801ee519d
Author: Jean-Yves Avenard <j...@apple.com>
Date: 2024-03-18 (Mon, 18 Mar 2024)
Changed paths:
M LayoutTests/http/tests/media/video-throttled-load-metadata-expected.txt
M LayoutTests/http/tests/media/video-throttled-load-metadata.html
A LayoutTests/media/video-src-mp4-blob-expected.txt
A LayoutTests/media/video-src-mp4-blob.html
A LayoutTests/media/video-src-webm-blob-expected.txt
A LayoutTests/media/video-src-webm-blob.html
A LayoutTests/media/video-srcobject-mp4-blob-expected.txt
A LayoutTests/media/video-srcobject-mp4-blob.html
M LayoutTests/platform/mac-wk1/TestExpectations
M LayoutTests/platform/mac/TestExpectations
M Source/WTF/wtf/NativePromise.h
M Source/WebCore/Headers.cmake
M Source/WebCore/Sources.txt
M Source/WebCore/WebCore.xcodeproj/project.pbxproj
M Source/WebCore/html/HTMLMediaElement.cpp
M Source/WebCore/html/HTMLMediaElement.h
M Source/WebCore/platform/CommonAtomStrings.h
M Source/WebCore/platform/ContentType.cpp
M Source/WebCore/platform/ContentType.h
M Source/WebCore/platform/PlatformMediaError.cpp
M Source/WebCore/platform/PlatformMediaError.h
A Source/WebCore/platform/graphics/MIMESniffer.cpp
A Source/WebCore/platform/graphics/MIMESniffer.h
M Source/WebCore/platform/graphics/MediaPlayer.cpp
M Source/WebCore/platform/graphics/MediaPlayer.h
A Source/WebCore/platform/graphics/MediaResourceSniffer.cpp
A Source/WebCore/platform/graphics/MediaResourceSniffer.h
M
Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.cpp
M
Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.h
M
Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm
M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp
M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h
M Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in
M Tools/TestWebKitAPI/CMakeLists.txt
M Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj
A Tools/TestWebKitAPI/Tests/WebCore/MIMESniffer.cpp
Log Message:
-----------
Add ability to sniff media content should we failed to play a file or if the
provided content type was invalid.
https://bugs.webkit.org/show_bug.cgi?id=270975
rdar://problem/124614908
Reviewed by Jer Noble.
We have always relied on the server to provide a valid content-type and if that
failed
we always used the URL's file name extension instead.
While the HTML5 specs clearly states that determining the type of a resource
should be done through sniffing, this is however a too significant change
to enable right away. There are also performance advantages in using the
provided
content-type: it's typically immediately available.
Should the provided content-type be invalid or non-existent (such as with some
blobs)
playback would have failed.
We add a MIMEtype sniffer for media content as per HTML5 specs.
Should we fail to play a media using the older content type detection rather
than immediately fail, we will also sniff the type as a last attempt before
retrying.
In order to not unnecessarily retry following a sniffing, we needed to
distinguish a format error from a network error which the
MediaPlayerPrivateAVFoundationObjC
didn't do. As a fly-by fix, we add a way to distinguish the two errors by
recording in the WebCoreNSURLSession any network failures.
For now will limit sniffing to media element where the src attribute is set
(and so exclude element where alternative sources are defined)
Added API tests for the mimetype sniffer and tests for webm and mp4 in a blob.
* LayoutTests/media/video-src-mp4-blob-expected.txt: Added.
* LayoutTests/media/video-src-mp4-blob.html: Added.
* LayoutTests/media/video-src-webm-blob-expected.txt: Added.
* LayoutTests/media/video-src-webm-blob.html: Added.
* LayoutTests/media/video-srcobject-mp4-blob-expected.txt: Added.
* LayoutTests/media/video-srcobject-mp4-blob.html: Added.
* LayoutTests/http/tests/media/video-throttled-load-metadata-expected.txt:
* LayoutTests/http/tests/media/video-throttled-load-metadata.html: The test was
racy and could
caused the loadedmetadata message to have been received before the worker's
message.
* LayoutTests/platform/mac-wk1/TestExpectations:
* LayoutTests/platform/mac/TestExpectations: Remove failure expectations. Tests
were failing as an incorrect
content-type was provided. Now that we sniff if the provided content-type was
incorrect
files can properly play again.
* Source/WTF/wtf/NativePromise.h:
* Source/WebCore/Headers.cmake:
* Source/WebCore/Sources.txt:
* Source/WebCore/WebCore.xcodeproj/project.pbxproj:
* Source/WebCore/html/HTMLMediaElement.cpp: Move all the logic from MediaPlayer
related
to guessing the ContentType if missing into the HTMLMediaElement.
If all have failed, we will sniff the content.
(WebCore::HTMLMediaElement::~HTMLMediaElement):
(WebCore::HTMLMediaElement::loadResource): If the content is blocked, signal it
as a network error rather than content error.
Otherwise we will attempt to load the content twice (once for sniffing and once
for playing) causing test failures.
(WebCore::HTMLMediaElement::needsContentTypeToPlay const): Add methods to
determine if sniffing is ultimately required.
(WebCore::HTMLMediaElement::sniffForContentType):
(WebCore::HTMLMediaElement::mediaLoadingFailed): Add one ultimate attempt if we
failed to decode and
didn't encounter a network error and didn't attempt to previously sniff content.
(WebCore::HTMLMediaElement::cancelPendingTasks):
(WebCore::HTMLMediaElement::cancelSniffer):
(WebCore::HTMLMediaElement::mediaPlayerEngineFailedToLoad): Save if a network
error was encountered.
If so, we won't attempt to sniff later on. This is required as multiple tests
are
relying on the network being accessed only once, doing one extra access for
sniffing
made the tests fail.
(WebCore::HTMLMediaElement::mediaPlayerEngineFailedToLoad const): Deleted.
* Source/WebCore/html/HTMLMediaElement.h:
* Source/WebCore/platform/CommonAtomStrings.h:
* Source/WebCore/platform/ContentType.cpp:
(WebCore::ContentType::ContentType):
(WebCore::ContentType::fromURL):
* Source/WebCore/platform/ContentType.h:
(WebCore::ContentType::typeWasInferredFromExtension const):
* Source/WebCore/platform/PlatformMediaError.cpp:
(WebCore::convertEnumerationToString):
* Source/WebCore/platform/PlatformMediaError.h:
* Source/WebCore/platform/graphics/MIMESniffer.cpp: Added.
(WebCore::MIMESniffer::span8):
(WebCore::MIMESniffer::hasSignatureForMP4):
(WebCore::MIMESniffer::parseWebMVint):
(WebCore::MIMESniffer::hasSignatureForWebM):
(WebCore::MIMESniffer::matchMP3Header):
(WebCore::MIMESniffer::mp3FrameSize):
(WebCore::MIMESniffer::parseMP3Frame):
(WebCore::MIMESniffer::hasSignatureForMP3WithoutID3):
(WebCore::MIMESniffer::mimeTypeFromSnifferEntries):
(WebCore::MIMESniffer::getMIMETypeFromContent):
* Source/WebCore/platform/graphics/MIMESniffer.h: Added.
* Source/WebCore/platform/graphics/MediaPlayer.cpp:
(WebCore::applicationOctetStream):
(WebCore::MediaPlayer::load):
(WebCore::MediaPlayer::loadWithNextMediaEngine):
* Source/WebCore/platform/graphics/MediaPlayer.h:
(WebCore::MediaPlayerClient::mediaPlayerEngineFailedToLoad):
(WebCore::MediaPlayerClient::mediaPlayerEngineFailedToLoad const): Deleted.
* Source/WebCore/platform/graphics/MediaResourceSniffer.cpp: Added.
(WebCore::MediaResourceSniffer::create):
(WebCore::MediaResourceSniffer::MediaResourceSniffer):
(WebCore::MediaResourceSniffer::~MediaResourceSniffer):
(WebCore::MediaResourceSniffer::cancel):
(WebCore::MediaResourceSniffer::promise const):
(WebCore::MediaResourceSniffer::dataReceived):
(WebCore::MediaResourceSniffer::loadFailed):
(WebCore::MediaResourceSniffer::loadFinished):
* Source/WebCore/platform/graphics/MediaResourceSniffer.h: Added.
*
Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.cpp:
(WebCore::MediaPlayerPrivateAVFoundation::updateStates): We needed to be able
to distinguish FormatError vs NetworkError so that we don't unnecessarily retry
playback following sniffing.
*
Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.h:
*
Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
(WebCore::MediaPlayerPrivateAVFoundationObjC::assetStatus const): Distinguish
load failure due to decoding error vs networking error.
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp:
(WebKit::RemoteMediaPlayerProxy::mediaPlayerEngineFailedToLoad):
(WebKit::RemoteMediaPlayerProxy::mediaPlayerEngineFailedToLoad const): Deleted.
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h:
* Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in:
* Tools/TestWebKitAPI/CMakeLists.txt:
* Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* Tools/TestWebKitAPI/Tests/WebCore/MIMESniffer.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/276258@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
_______________________________________________
webkit-changes mailing list
webkit-changes@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-changes
