Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: f2195aa52bec637d3a35273da68217f83e845350
https://github.com/WebKit/WebKit/commit/f2195aa52bec637d3a35273da68217f83e845350
Author: Myles C. Maxfield <mmaxfi...@apple.com>
Date: 2023-04-18 (Tue, 18 Apr 2023)
Changed paths:
M Source/WTF/wtf/PlatformHave.h
M Source/WebCore/Headers.cmake
M Source/WebCore/Modules/ShapeDetection/BarcodeDetector.cpp
M Source/WebCore/Modules/ShapeDetection/BarcodeDetector.h
M Source/WebCore/Modules/ShapeDetection/BarcodeDetector.idl
M Source/WebCore/Modules/ShapeDetection/FaceDetector.cpp
M Source/WebCore/Modules/ShapeDetection/FaceDetector.h
M Source/WebCore/Modules/ShapeDetection/FaceDetector.idl
M
Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/BarcodeDetectorImplementation.h
M
Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/FaceDetectorImplementation.h
M
Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/TextDetectorImplementation.h
M
Source/WebCore/Modules/ShapeDetection/Interfaces/BarcodeDetectorOptionsInterface.h
M Source/WebCore/Modules/ShapeDetection/Interfaces/BarcodeFormatInterface.h
M
Source/WebCore/Modules/ShapeDetection/Interfaces/DetectedBarcodeInterface.h
M Source/WebCore/Modules/ShapeDetection/Interfaces/DetectedFaceInterface.h
M Source/WebCore/Modules/ShapeDetection/Interfaces/DetectedTextInterface.h
M Source/WebCore/Modules/ShapeDetection/Interfaces/LandmarkInterface.h
M Source/WebCore/Modules/ShapeDetection/Interfaces/LandmarkTypeInterface.h
M Source/WebCore/Modules/ShapeDetection/TextDetector.cpp
M Source/WebCore/Modules/ShapeDetection/TextDetector.h
M Source/WebCore/Modules/ShapeDetection/TextDetector.idl
M Source/WebCore/WebCore.xcodeproj/project.pbxproj
M Source/WebCore/page/Chrome.cpp
M Source/WebCore/page/Chrome.h
M Source/WebCore/page/ChromeClient.h
M Source/WebKit/CMakeLists.txt
M Source/WebKit/DerivedSources-input.xcfilelist
M Source/WebKit/DerivedSources-output.xcfilelist
M Source/WebKit/DerivedSources.make
A Source/WebKit/GPUProcess/ShapeDetection/RemoteBarcodeDetector.cpp
A Source/WebKit/GPUProcess/ShapeDetection/RemoteBarcodeDetector.h
A Source/WebKit/GPUProcess/ShapeDetection/RemoteBarcodeDetector.messages.in
A Source/WebKit/GPUProcess/ShapeDetection/RemoteFaceDetector.cpp
A Source/WebKit/GPUProcess/ShapeDetection/RemoteFaceDetector.h
A Source/WebKit/GPUProcess/ShapeDetection/RemoteFaceDetector.messages.in
A Source/WebKit/GPUProcess/ShapeDetection/RemoteTextDetector.cpp
A Source/WebKit/GPUProcess/ShapeDetection/RemoteTextDetector.h
A Source/WebKit/GPUProcess/ShapeDetection/RemoteTextDetector.messages.in
A Source/WebKit/GPUProcess/ShapeDetection/ShapeDetectionObjectHeap.cpp
A Source/WebKit/GPUProcess/ShapeDetection/ShapeDetectionObjectHeap.h
M Source/WebKit/GPUProcess/graphics/RemoteDisplayListRecorder.cpp
M Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.cpp
M Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.h
M Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.messages.in
M
Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererInternalUnitManager.cpp
M Source/WebKit/Scripts/webkit/messages.py
M Source/WebKit/Scripts/webkit/tests/MessageArgumentDescriptions.cpp
M Source/WebKit/Scripts/webkit/tests/MessageNames.cpp
M Source/WebKit/Scripts/webkit/tests/MessageNames.h
M
Source/WebKit/Scripts/webkit/tests/TestWithCVPixelBufferMessageReceiver.cpp
M Source/WebKit/Scripts/webkit/tests/TestWithCVPixelBufferMessages.h
M Source/WebKit/Scripts/webkit/tests/TestWithIfMessageMessageReceiver.cpp
M Source/WebKit/Scripts/webkit/tests/TestWithIfMessageMessages.h
M Source/WebKit/Scripts/webkit/tests/TestWithImageDataMessageReceiver.cpp
M Source/WebKit/Scripts/webkit/tests/TestWithImageDataMessages.h
M
Source/WebKit/Scripts/webkit/tests/TestWithLegacyReceiverMessageReceiver.cpp
M Source/WebKit/Scripts/webkit/tests/TestWithLegacyReceiverMessages.h
M Source/WebKit/Scripts/webkit/tests/TestWithSemaphoreMessageReceiver.cpp
M Source/WebKit/Scripts/webkit/tests/TestWithSemaphoreMessages.h
M
Source/WebKit/Scripts/webkit/tests/TestWithStreamBatchedMessageReceiver.cpp
M Source/WebKit/Scripts/webkit/tests/TestWithStreamBatchedMessages.h
M Source/WebKit/Scripts/webkit/tests/TestWithStreamBufferMessageReceiver.cpp
M Source/WebKit/Scripts/webkit/tests/TestWithStreamBufferMessages.h
M Source/WebKit/Scripts/webkit/tests/TestWithStreamMessageReceiver.cpp
M Source/WebKit/Scripts/webkit/tests/TestWithStreamMessages.h
M Source/WebKit/Scripts/webkit/tests/TestWithSuperclassMessageReceiver.cpp
M Source/WebKit/Scripts/webkit/tests/TestWithSuperclassMessages.h
M
Source/WebKit/Scripts/webkit/tests/TestWithoutAttributesMessageReceiver.cpp
M Source/WebKit/Scripts/webkit/tests/TestWithoutAttributesMessages.h
A Source/WebKit/Shared/ShapeDetectionIdentifier.h
M Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in
M Source/WebKit/Sources.txt
M Source/WebKit/WebKit.xcodeproj/project.pbxproj
A Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteBarcodeDetectorProxy.cpp
A Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteBarcodeDetectorProxy.h
A Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteFaceDetectorProxy.cpp
A Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteFaceDetectorProxy.h
A Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteTextDetectorProxy.cpp
A Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteTextDetectorProxy.h
M Source/WebKit/WebProcess/GPU/graphics/RemoteGraphicsContextGLProxy.cpp
M Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.h
M Source/WebKit/WebProcess/Inspector/WebInspectorUIExtensionController.cpp
M Source/WebKit/WebProcess/WebCoreSupport/WebChromeClient.cpp
M Source/WebKit/WebProcess/WebCoreSupport/WebChromeClient.h
M Source/WebKit/WebProcess/WebPage/RemoteLayerTree/GraphicsLayerCARemote.cpp
M Source/WebKitLegacy/mac/WebCoreSupport/WebChromeClient.h
M Source/WebKitLegacy/mac/WebCoreSupport/WebChromeClient.mm
Log Message:
-----------
Plumb shape detection API to the GPU Process
https://bugs.webkit.org/show_bug.cgi?id=255473
rdar://108070184
Reviewed by Mike Wyrzykowski.
This patch is really just plumbing, though there are a few pieces to it.
1. The shape detection API is already behind a runtime flag, which guards the
JavaScript entry points.
So, the code is getting compiled everywhere. However, eventually
something somewhere has to
actually implement the API itself (for Cocoa ports, that's the Vision
framework), and not all
ports have this support. So, there also needs to be a build-time flag,
HAVE(SHAPE_DETECTION_API_IMPLEMENTATION), which _just_ guards the calls
to the final
implementation. This is just to make the code compile on all ports.
2. WebCore doesn't know anything about IPC, so the object creation methods need
to be plumbed up
through ChromeClient, so WebKitLegacy or WebKit2 can choose whether to
back the objects by
WebKit::ShapeDetection::RemoteFooProxy objects, or by
WebCore::ShapeDetection::FooImpl objects.
Because these object creation methods are free functions in JavaScript,
they need to have the
[CallWith=CurrentScriptExecutionContext] annotation, to get the context
they need to talk to
the Page and ChromeClient.
3. WebCore::ShapeDetection::FooImpl objects need to be exported from WebCore,
so WebKit and
WebKitLegacy can create them
4. Create RemoteFooProxy and RemoteFoo classes. RemoteFooProxy classes live in
the web process, and
forward all their method calls to IPC send(). RemoteFoo classes live in
the GPU process, and own
an instance of the corresponding WebCore::ShapeDetection::FooImpl
object. There are messages
defined in RemoteFoo.messages.in files.
5. Handle object lifetime in the GPU process. When an object is created in the
web process, that
corresponds to a RemoteRenderingBackend::CreateFoo() message. The
RemoteRenderingBackend owns a
WebKit::ShapeDetection::ObjectHeap, which retains all the objects in the
GPU process that get
created. The objects that get created start listening on the preexisting
StreamConnection of the
RemoteRenderingBackend - I didn't want to give each shape detection
object its own totally new
StreamConnection (like how WebGPU does) because these shape detection
objects are so small and
only have 1 or 2 methods on them. Object destructors in the web process
send a
RemoteRenderingBackend::ReleaseFoo() message.
6. Add serialization information for the relevant structs to
WebCoreArgumentCoders.serialization.in
The last thing that's worth mentioning is I haven't hooked up the
ImageBitmapSource parameters to the
detect() functions, because those sources can be in many different forms, and
the logic to handle all
the forms is quite large, and outside the scope of this simple plumbing patch.
My next patch will
implement these parameters.
No tests because there is no behavior change - I haven't actually hooked up the
Vision framework to
actually implement any of the calls yet.
* Source/WTF/wtf/PlatformHave.h:
* Source/WebCore/Modules/ShapeDetection/BarcodeDetector.cpp:
(WebCore::BarcodeDetector::create):
(WebCore::BarcodeDetector::getSupportedFormats):
* Source/WebCore/Modules/ShapeDetection/BarcodeDetector.h:
* Source/WebCore/Modules/ShapeDetection/BarcodeDetector.idl:
* Source/WebCore/Modules/ShapeDetection/FaceDetector.cpp:
(WebCore::FaceDetector::create):
* Source/WebCore/Modules/ShapeDetection/FaceDetector.h:
* Source/WebCore/Modules/ShapeDetection/FaceDetector.idl:
*
Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/BarcodeDetectorImplementation.h:
*
Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/FaceDetectorImplementation.h:
*
Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/TextDetectorImplementation.h:
*
Source/WebCore/Modules/ShapeDetection/Interfaces/BarcodeDetectorOptionsInterface.h:
* Source/WebCore/Modules/ShapeDetection/Interfaces/BarcodeFormatInterface.h:
* Source/WebCore/Modules/ShapeDetection/Interfaces/DetectedBarcodeInterface.h:
* Source/WebCore/Modules/ShapeDetection/Interfaces/DetectedFaceInterface.h:
* Source/WebCore/Modules/ShapeDetection/Interfaces/DetectedTextInterface.h:
* Source/WebCore/Modules/ShapeDetection/Interfaces/LandmarkInterface.h:
* Source/WebCore/Modules/ShapeDetection/Interfaces/LandmarkTypeInterface.h:
* Source/WebCore/Modules/ShapeDetection/TextDetector.cpp:
(WebCore::TextDetector::create):
* Source/WebCore/Modules/ShapeDetection/TextDetector.h:
* Source/WebCore/Modules/ShapeDetection/TextDetector.idl:
* Source/WebCore/WebCore.xcodeproj/project.pbxproj:
* Source/WebCore/page/Chrome.cpp:
(WebCore::Chrome::createBarcodeDetector const):
(WebCore::Chrome::getBarcodeDetectorSupportedFormats const):
(WebCore::Chrome::createFaceDetector const):
(WebCore::Chrome::createTextDetector const):
* Source/WebCore/page/Chrome.h:
* Source/WebCore/page/ChromeClient.h:
(WebCore::ChromeClient::createBarcodeDetector const):
(WebCore::ChromeClient::getBarcodeDetectorSupportedFormats const):
(WebCore::ChromeClient::createFaceDetector const):
(WebCore::ChromeClient::createTextDetector const):
* Source/WebKit/CMakeLists.txt:
* Source/WebKit/DerivedSources-input.xcfilelist:
* Source/WebKit/DerivedSources-output.xcfilelist:
* Source/WebKit/DerivedSources.make:
* Source/WebKit/GPUProcess/ShapeDetection/RemoteBarcodeDetector.cpp: Copied
from Source/WebCore/Modules/ShapeDetection/TextDetector.h.
(WebKit::RemoteBarcodeDetector::RemoteBarcodeDetector):
(WebKit::RemoteBarcodeDetector::detect):
* Source/WebKit/GPUProcess/ShapeDetection/RemoteBarcodeDetector.h: Added.
(WebKit::RemoteBarcodeDetector::create):
(WebKit::RemoteBarcodeDetector::backing):
* Source/WebKit/GPUProcess/ShapeDetection/RemoteBarcodeDetector.messages.in:
Added.
* Source/WebKit/GPUProcess/ShapeDetection/RemoteFaceDetector.cpp: Copied from
Source/WebCore/Modules/ShapeDetection/TextDetector.h.
(WebKit::RemoteFaceDetector::RemoteFaceDetector):
(WebKit::RemoteFaceDetector::detect):
* Source/WebKit/GPUProcess/ShapeDetection/RemoteFaceDetector.h: Added.
(WebKit::RemoteFaceDetector::create):
(WebKit::RemoteFaceDetector::backing):
* Source/WebKit/GPUProcess/ShapeDetection/RemoteFaceDetector.messages.in: Added.
* Source/WebKit/GPUProcess/ShapeDetection/RemoteTextDetector.cpp: Copied from
Source/WebCore/Modules/ShapeDetection/TextDetector.h.
(WebKit::RemoteTextDetector::RemoteTextDetector):
(WebKit::RemoteTextDetector::detect):
* Source/WebKit/GPUProcess/ShapeDetection/RemoteTextDetector.h: Added.
(WebKit::RemoteTextDetector::create):
(WebKit::RemoteTextDetector::backing):
* Source/WebKit/GPUProcess/ShapeDetection/RemoteTextDetector.messages.in: Added.
* Source/WebKit/GPUProcess/ShapeDetection/ShapeDetectionObjectHeap.cpp: Copied
from Source/WebCore/Modules/ShapeDetection/TextDetector.cpp.
(WebKit::ShapeDetection::ObjectHeap::addObject):
(WebKit::ShapeDetection::ObjectHeap::removeObject):
(WebKit::ShapeDetection::ObjectHeap::clear):
* Source/WebKit/GPUProcess/ShapeDetection/ShapeDetectionObjectHeap.h: Copied
from
Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/BarcodeDetectorImplementation.h.
* Source/WebKit/GPUProcess/graphics/RemoteDisplayListRecorder.cpp:
* Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.cpp:
(WebKit::RemoteRenderingBackend::RemoteRenderingBackend):
(WebKit::RemoteRenderingBackend::createRemoteBarcodeDetector):
(WebKit::RemoteRenderingBackend::releaseRemoteBarcodeDetector):
(WebKit::RemoteRenderingBackend::getRemoteBarcodeDetectorSupportedFormats):
(WebKit::RemoteRenderingBackend::createRemoteFaceDetector):
(WebKit::RemoteRenderingBackend::releaseRemoteFaceDetector):
(WebKit::RemoteRenderingBackend::createRemoteTextDetector):
(WebKit::RemoteRenderingBackend::releaseRemoteTextDetector):
* Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.h:
* Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.messages.in:
*
Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererInternalUnitManager.cpp:
(WebKit::RemoteAudioMediaStreamTrackRendererInternalUnitManager::Unit::captureUnitHasStopped):
* Source/WebKit/Scripts/webkit/messages.py:
(serialized_identifiers):
(headers_for_type):
* Source/WebKit/Scripts/webkit/tests/MessageArgumentDescriptions.cpp:
(IPC::serializedIdentifiers):
* Source/WebKit/Shared/ShapeDetectionIdentifier.h: Copied from
Source/WebCore/Modules/ShapeDetection/Interfaces/BarcodeDetectorOptionsInterface.h.
* Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in:
* Source/WebKit/Sources.txt:
* Source/WebKit/WebKit.xcodeproj/project.pbxproj:
* Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteBarcodeDetectorProxy.cpp:
Added.
(WebKit::ShapeDetection::RemoteBarcodeDetectorProxy::create):
(WebKit::ShapeDetection::RemoteBarcodeDetectorProxy::RemoteBarcodeDetectorProxy):
(WebKit::ShapeDetection::RemoteBarcodeDetectorProxy::~RemoteBarcodeDetectorProxy):
(WebKit::ShapeDetection::RemoteBarcodeDetectorProxy::getSupportedFormats):
(WebKit::ShapeDetection::RemoteBarcodeDetectorProxy::detect):
* Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteBarcodeDetectorProxy.h:
Added.
(WebKit::ShapeDetection::RemoteBarcodeDetectorProxy::backing const):
* Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteFaceDetectorProxy.cpp:
Added.
(WebKit::ShapeDetection::RemoteFaceDetectorProxy::create):
(WebKit::ShapeDetection::RemoteFaceDetectorProxy::RemoteFaceDetectorProxy):
(WebKit::ShapeDetection::RemoteFaceDetectorProxy::~RemoteFaceDetectorProxy):
(WebKit::ShapeDetection::RemoteFaceDetectorProxy::detect):
* Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteFaceDetectorProxy.h: Added.
(WebKit::ShapeDetection::RemoteFaceDetectorProxy::backing const):
* Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteTextDetectorProxy.cpp:
Added.
(WebKit::ShapeDetection::RemoteTextDetectorProxy::create):
(WebKit::ShapeDetection::RemoteTextDetectorProxy::RemoteTextDetectorProxy):
(WebKit::ShapeDetection::RemoteTextDetectorProxy::~RemoteTextDetectorProxy):
(WebKit::ShapeDetection::RemoteTextDetectorProxy::detect):
* Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteTextDetectorProxy.h: Added.
(WebKit::ShapeDetection::RemoteTextDetectorProxy::backing const):
* Source/WebKit/WebProcess/GPU/graphics/RemoteGraphicsContextGLProxy.cpp:
* Source/WebKit/WebProcess/GPU/graphics/WebGPU/RemoteGPUProxy.h:
* Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.h:
* Source/WebKit/WebProcess/Inspector/WebInspectorUIExtensionController.cpp:
* Source/WebKit/WebProcess/WebCoreSupport/WebChromeClient.cpp:
(WebKit::WebChromeClient::createBarcodeDetector const):
(WebKit::WebChromeClient::getBarcodeDetectorSupportedFormats const):
(WebKit::WebChromeClient::createFaceDetector const):
(WebKit::WebChromeClient::createTextDetector const):
* Source/WebKit/WebProcess/WebCoreSupport/WebChromeClient.h:
* Source/WebKit/WebProcess/WebPage/RemoteLayerTree/GraphicsLayerCARemote.cpp:
* Source/WebKitLegacy/mac/WebCoreSupport/WebChromeClient.h:
* Source/WebKitLegacy/mac/WebCoreSupport/WebChromeClient.mm:
(WebChromeClient::createBarcodeDetector const):
(WebChromeClient::getBarcodeDetectorSupportedFormats const):
(WebChromeClient::createFaceDetector const):
(WebChromeClient::createTextDetector const):
Canonical link: https://commits.webkit.org/263094@main
_______________________________________________
webkit-changes mailing list
webkit-changes@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-changes
