Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: 5832f3aed9c14469cc8fae109479ad4132ce664d
https://github.com/WebKit/WebKit/commit/5832f3aed9c14469cc8fae109479ad4132ce664d
Author: Jean-Yves Avenard <[email protected]>
Date: 2026-01-13 (Tue, 13 Jan 2026)
Changed paths:
M Source/WebCore/html/HTMLVideoElement.cpp
M Source/WebCore/html/HTMLVideoElement.h
M Source/WebCore/rendering/RenderVideo.h
M Source/WebKit/GPUProcess/GPUProcess.cpp
M Source/WebKit/GPUProcess/GPUProcess.h
M Source/WebKit/GPUProcess/GPUProcess.messages.in
M Source/WebKit/GPUProcess/media/RemoteMediaPlayerManagerProxy.cpp
M Source/WebKit/GPUProcess/media/RemoteMediaPlayerManagerProxy.h
M Source/WebKit/UIProcess/Cocoa/PlaybackSessionManagerProxy.mm
M Source/WebKit/UIProcess/Cocoa/VideoPresentationManagerProxy.h
M Source/WebKit/UIProcess/Cocoa/VideoPresentationManagerProxy.mm
M Source/WebKit/UIProcess/Cocoa/WebPageProxyCocoa.mm
M Source/WebKit/UIProcess/GPU/GPUProcessProxy.cpp
M Source/WebKit/UIProcess/GPU/GPUProcessProxy.h
M Source/WebKit/UIProcess/WebPageProxy.cpp
M Source/WebKit/UIProcess/WebPageProxy.h
M Source/WebKit/UIProcess/WebPageProxy.messages.in
M Source/WebKit/UIProcess/WebPageProxyInternals.h
M Source/WebKit/WebProcess/WebCoreSupport/ShareableBitmapUtilities.cpp
M Source/WebKit/WebProcess/WebPage/WebPage.cpp
Log Message:
-----------
en.wikipedia.org/wiki/videos cannot select text after video paused when
player runs in content process
https://bugs.webkit.org/show_bug.cgi?id=305318
rdar://167727538
Reviewed by Wenson Hsieh.
The text recognition in videos could only handle the case where the MediaPlayer
was running in the GPU process due to the UI process directly fetching the
player's bitmap
from the GPUP.
This would fail when the webm or MSE player would run in the web content
process as there was no associated MediaPlayerIdentifier living in the GPUP.
Instead with now have the WebPage in the content process retrieves the video's
bitmap image directly from the HTMLMediaVideoElement and send it to the
UIP to start the text recognition process.
Following this approach we can remove quite a bit of the existing code.
There were two possible ways to start text recognition in the video:
1- Inline video element.
2- Fullscreen video element.
For 1- when the user hover the mouse or finger over a video element
EventHandler::textRecognitionHoverTimerFired() will start the image capture.
similarly, for 2- WebFullScreenManager::scheduleTextRecognitionForMainVideo
would schedule a 250ms
timer to start the image capture.
the bitmap image capture was refreshed whenever:
1- we enter fullscreen screen
2- we paused the video
3- the rate changed
4- the current time changed and we were paused.
All of those were redundant from the first two. Additionally, we had some
dead code that were doing the same work as above. We delete those.
The retrieval of the ShareableBitmap from the UIP to the GPUP was asynchronous.
Following this change, the call will be synchronous. In a follow-up change
we will make this operation asynchronous.
Manually tested on iOS and macOS, with the webm/MSE player running in the
content process and GPUP.
* Source/WebCore/html/HTMLVideoElement.cpp:
(WebCore::HTMLVideoElement::nativeImageForCurrentTime const): make method const.
(WebCore::HTMLVideoElement::bitmapImageForCurrentTime const):
(WebCore::HTMLVideoElement::nativeImageForCurrentTime): Deleted.
* Source/WebCore/html/HTMLVideoElement.h:
* Source/WebCore/rendering/RenderVideo.h: export protectedVideoElement symbol.
* Source/WebKit/GPUProcess/GPUProcess.cpp:
(WebKit::GPUProcess::requestBitmapImageForCurrentTime): Deleted.
* Source/WebKit/GPUProcess/GPUProcess.h:
* Source/WebKit/GPUProcess/GPUProcess.messages.in: Remove
bitmapImageForCurrentTime API.
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerManagerProxy.cpp:
(WebKit::RemoteMediaPlayerManagerProxy::bitmapImageForCurrentTime): Deleted.
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerManagerProxy.h:
* Source/WebKit/UIProcess/Cocoa/PlaybackSessionManagerProxy.mm:
(WebKit::PlaybackSessionManagerProxy::currentTimeChanged): No bitmap capture is
now happening in the UIP, no need to refresh the bitmap
(WebKit::PlaybackSessionManagerProxy::rateChanged): Ditto.
* Source/WebKit/UIProcess/Cocoa/VideoPresentationManagerProxy.h: Remove unused
code
* Source/WebKit/UIProcess/Cocoa/VideoPresentationManagerProxy.mm: Remove unused
code
(WebKit::VideoPresentationManagerProxy::requestBitmapImageForCurrentTime):
Deleted.
* Source/WebKit/UIProcess/Cocoa/WebPageProxyCocoa.mm: Remove now unused code.
(WebKit::WebPageProxy::didChangePlaybackRate): Deleted.
(WebKit::WebPageProxy::didChangeCurrentTime): Deleted.
(WebKit::WebPageProxy::updateFullscreenVideoTextRecognition): Deleted.
(WebKit::WebPageProxy::fullscreenVideoTextRecognitionTimerFired): Deleted.
* Source/WebKit/UIProcess/GPU/GPUProcessProxy.cpp:
(WebKit::GPUProcessProxy::requestBitmapImageForCurrentTime): Deleted.
* Source/WebKit/UIProcess/GPU/GPUProcessProxy.h:
* Source/WebKit/UIProcess/WebPageProxy.cpp:
(WebKit::WebPageProxy::didEnterFullscreen):
(WebKit::WebPageProxy::didExitFullscreen):
(WebKit::WebPageProxy::resetStateAfterProcessExited):
(WebKit::WebPageProxy::beginTextRecognitionForVideoInElementFullScreen):
(WebKit::WebPageProxy::cancelTextRecognitionForVideoInElementFullScreen):
* Source/WebKit/UIProcess/WebPageProxy.h:
* Source/WebKit/UIProcess/WebPageProxy.messages.in: Pass the
ShareableBitmapHandle with the call to
WebPageProxy::beginTextRecognitionForVideoInElementFullScreen
* Source/WebKit/UIProcess/WebPageProxyInternals.h:
* Source/WebKit/WebProcess/WebCoreSupport/ShareableBitmapUtilities.cpp:
(WebKit::createShareableBitmap): Simplify code and reduce duplication by
re-using the HTMLMediaVideoElement implementation directly.
* Source/WebKit/WebProcess/WebPage/WebPage.cpp:
(WebKit::WebPage::beginTextRecognitionForVideoInElementFullScreen):
Canonical link: https://commits.webkit.org/305540@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
