Branch: refs/heads/main Home: https://github.com/WebKit/WebKit Commit: f2195aa52bec637d3a35273da68217f83e845350 https://github.com/WebKit/WebKit/commit/f2195aa52bec637d3a35273da68217f83e845350 Author: Myles C. Maxfield <mmaxfi...@apple.com> Date: 2023-04-18 (Tue, 18 Apr 2023)
Changed paths: M Source/WTF/wtf/PlatformHave.h M Source/WebCore/Headers.cmake M Source/WebCore/Modules/ShapeDetection/BarcodeDetector.cpp M Source/WebCore/Modules/ShapeDetection/BarcodeDetector.h M Source/WebCore/Modules/ShapeDetection/BarcodeDetector.idl M Source/WebCore/Modules/ShapeDetection/FaceDetector.cpp M Source/WebCore/Modules/ShapeDetection/FaceDetector.h M Source/WebCore/Modules/ShapeDetection/FaceDetector.idl M Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/BarcodeDetectorImplementation.h M Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/FaceDetectorImplementation.h M Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/TextDetectorImplementation.h M Source/WebCore/Modules/ShapeDetection/Interfaces/BarcodeDetectorOptionsInterface.h M Source/WebCore/Modules/ShapeDetection/Interfaces/BarcodeFormatInterface.h M Source/WebCore/Modules/ShapeDetection/Interfaces/DetectedBarcodeInterface.h M Source/WebCore/Modules/ShapeDetection/Interfaces/DetectedFaceInterface.h M Source/WebCore/Modules/ShapeDetection/Interfaces/DetectedTextInterface.h M Source/WebCore/Modules/ShapeDetection/Interfaces/LandmarkInterface.h M Source/WebCore/Modules/ShapeDetection/Interfaces/LandmarkTypeInterface.h M Source/WebCore/Modules/ShapeDetection/TextDetector.cpp M Source/WebCore/Modules/ShapeDetection/TextDetector.h M Source/WebCore/Modules/ShapeDetection/TextDetector.idl M Source/WebCore/WebCore.xcodeproj/project.pbxproj M Source/WebCore/page/Chrome.cpp M Source/WebCore/page/Chrome.h M Source/WebCore/page/ChromeClient.h M Source/WebKit/CMakeLists.txt M Source/WebKit/DerivedSources-input.xcfilelist M Source/WebKit/DerivedSources-output.xcfilelist M Source/WebKit/DerivedSources.make A Source/WebKit/GPUProcess/ShapeDetection/RemoteBarcodeDetector.cpp A Source/WebKit/GPUProcess/ShapeDetection/RemoteBarcodeDetector.h A Source/WebKit/GPUProcess/ShapeDetection/RemoteBarcodeDetector.messages.in A Source/WebKit/GPUProcess/ShapeDetection/RemoteFaceDetector.cpp A Source/WebKit/GPUProcess/ShapeDetection/RemoteFaceDetector.h A Source/WebKit/GPUProcess/ShapeDetection/RemoteFaceDetector.messages.in A Source/WebKit/GPUProcess/ShapeDetection/RemoteTextDetector.cpp A Source/WebKit/GPUProcess/ShapeDetection/RemoteTextDetector.h A Source/WebKit/GPUProcess/ShapeDetection/RemoteTextDetector.messages.in A Source/WebKit/GPUProcess/ShapeDetection/ShapeDetectionObjectHeap.cpp A Source/WebKit/GPUProcess/ShapeDetection/ShapeDetectionObjectHeap.h M Source/WebKit/GPUProcess/graphics/RemoteDisplayListRecorder.cpp M Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.cpp M Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.h M Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.messages.in M Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererInternalUnitManager.cpp M Source/WebKit/Scripts/webkit/messages.py M Source/WebKit/Scripts/webkit/tests/MessageArgumentDescriptions.cpp M Source/WebKit/Scripts/webkit/tests/MessageNames.cpp M Source/WebKit/Scripts/webkit/tests/MessageNames.h M Source/WebKit/Scripts/webkit/tests/TestWithCVPixelBufferMessageReceiver.cpp M Source/WebKit/Scripts/webkit/tests/TestWithCVPixelBufferMessages.h M Source/WebKit/Scripts/webkit/tests/TestWithIfMessageMessageReceiver.cpp M Source/WebKit/Scripts/webkit/tests/TestWithIfMessageMessages.h M Source/WebKit/Scripts/webkit/tests/TestWithImageDataMessageReceiver.cpp M Source/WebKit/Scripts/webkit/tests/TestWithImageDataMessages.h M Source/WebKit/Scripts/webkit/tests/TestWithLegacyReceiverMessageReceiver.cpp M Source/WebKit/Scripts/webkit/tests/TestWithLegacyReceiverMessages.h M Source/WebKit/Scripts/webkit/tests/TestWithSemaphoreMessageReceiver.cpp M Source/WebKit/Scripts/webkit/tests/TestWithSemaphoreMessages.h M Source/WebKit/Scripts/webkit/tests/TestWithStreamBatchedMessageReceiver.cpp M Source/WebKit/Scripts/webkit/tests/TestWithStreamBatchedMessages.h M Source/WebKit/Scripts/webkit/tests/TestWithStreamBufferMessageReceiver.cpp M Source/WebKit/Scripts/webkit/tests/TestWithStreamBufferMessages.h M Source/WebKit/Scripts/webkit/tests/TestWithStreamMessageReceiver.cpp M Source/WebKit/Scripts/webkit/tests/TestWithStreamMessages.h M Source/WebKit/Scripts/webkit/tests/TestWithSuperclassMessageReceiver.cpp M Source/WebKit/Scripts/webkit/tests/TestWithSuperclassMessages.h M Source/WebKit/Scripts/webkit/tests/TestWithoutAttributesMessageReceiver.cpp M Source/WebKit/Scripts/webkit/tests/TestWithoutAttributesMessages.h A Source/WebKit/Shared/ShapeDetectionIdentifier.h M Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in M Source/WebKit/Sources.txt M Source/WebKit/WebKit.xcodeproj/project.pbxproj A Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteBarcodeDetectorProxy.cpp A Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteBarcodeDetectorProxy.h A Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteFaceDetectorProxy.cpp A Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteFaceDetectorProxy.h A Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteTextDetectorProxy.cpp A Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteTextDetectorProxy.h M Source/WebKit/WebProcess/GPU/graphics/RemoteGraphicsContextGLProxy.cpp M Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.h M Source/WebKit/WebProcess/Inspector/WebInspectorUIExtensionController.cpp M Source/WebKit/WebProcess/WebCoreSupport/WebChromeClient.cpp M Source/WebKit/WebProcess/WebCoreSupport/WebChromeClient.h M Source/WebKit/WebProcess/WebPage/RemoteLayerTree/GraphicsLayerCARemote.cpp M Source/WebKitLegacy/mac/WebCoreSupport/WebChromeClient.h M Source/WebKitLegacy/mac/WebCoreSupport/WebChromeClient.mm Log Message: ----------- Plumb shape detection API to the GPU Process https://bugs.webkit.org/show_bug.cgi?id=255473 rdar://108070184 Reviewed by Mike Wyrzykowski. This patch is really just plumbing, though there are a few pieces to it. 1. The shape detection API is already behind a runtime flag, which guards the JavaScript entry points. So, the code is getting compiled everywhere. However, eventually something somewhere has to actually implement the API itself (for Cocoa ports, that's the Vision framework), and not all ports have this support. So, there also needs to be a build-time flag, HAVE(SHAPE_DETECTION_API_IMPLEMENTATION), which _just_ guards the calls to the final implementation. This is just to make the code compile on all ports. 2. WebCore doesn't know anything about IPC, so the object creation methods need to be plumbed up through ChromeClient, so WebKitLegacy or WebKit2 can choose whether to back the objects by WebKit::ShapeDetection::RemoteFooProxy objects, or by WebCore::ShapeDetection::FooImpl objects. Because these object creation methods are free functions in JavaScript, they need to have the [CallWith=CurrentScriptExecutionContext] annotation, to get the context they need to talk to the Page and ChromeClient. 3. WebCore::ShapeDetection::FooImpl objects need to be exported from WebCore, so WebKit and WebKitLegacy can create them 4. Create RemoteFooProxy and RemoteFoo classes. RemoteFooProxy classes live in the web process, and forward all their method calls to IPC send(). RemoteFoo classes live in the GPU process, and own an instance of the corresponding WebCore::ShapeDetection::FooImpl object. There are messages defined in RemoteFoo.messages.in files. 5. Handle object lifetime in the GPU process. When an object is created in the web process, that corresponds to a RemoteRenderingBackend::CreateFoo() message. The RemoteRenderingBackend owns a WebKit::ShapeDetection::ObjectHeap, which retains all the objects in the GPU process that get created. The objects that get created start listening on the preexisting StreamConnection of the RemoteRenderingBackend - I didn't want to give each shape detection object its own totally new StreamConnection (like how WebGPU does) because these shape detection objects are so small and only have 1 or 2 methods on them. Object destructors in the web process send a RemoteRenderingBackend::ReleaseFoo() message. 6. Add serialization information for the relevant structs to WebCoreArgumentCoders.serialization.in The last thing that's worth mentioning is I haven't hooked up the ImageBitmapSource parameters to the detect() functions, because those sources can be in many different forms, and the logic to handle all the forms is quite large, and outside the scope of this simple plumbing patch. My next patch will implement these parameters. No tests because there is no behavior change - I haven't actually hooked up the Vision framework to actually implement any of the calls yet. * Source/WTF/wtf/PlatformHave.h: * Source/WebCore/Modules/ShapeDetection/BarcodeDetector.cpp: (WebCore::BarcodeDetector::create): (WebCore::BarcodeDetector::getSupportedFormats): * Source/WebCore/Modules/ShapeDetection/BarcodeDetector.h: * Source/WebCore/Modules/ShapeDetection/BarcodeDetector.idl: * Source/WebCore/Modules/ShapeDetection/FaceDetector.cpp: (WebCore::FaceDetector::create): * Source/WebCore/Modules/ShapeDetection/FaceDetector.h: * Source/WebCore/Modules/ShapeDetection/FaceDetector.idl: * Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/BarcodeDetectorImplementation.h: * Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/FaceDetectorImplementation.h: * Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/TextDetectorImplementation.h: * Source/WebCore/Modules/ShapeDetection/Interfaces/BarcodeDetectorOptionsInterface.h: * Source/WebCore/Modules/ShapeDetection/Interfaces/BarcodeFormatInterface.h: * Source/WebCore/Modules/ShapeDetection/Interfaces/DetectedBarcodeInterface.h: * Source/WebCore/Modules/ShapeDetection/Interfaces/DetectedFaceInterface.h: * Source/WebCore/Modules/ShapeDetection/Interfaces/DetectedTextInterface.h: * Source/WebCore/Modules/ShapeDetection/Interfaces/LandmarkInterface.h: * Source/WebCore/Modules/ShapeDetection/Interfaces/LandmarkTypeInterface.h: * Source/WebCore/Modules/ShapeDetection/TextDetector.cpp: (WebCore::TextDetector::create): * Source/WebCore/Modules/ShapeDetection/TextDetector.h: * Source/WebCore/Modules/ShapeDetection/TextDetector.idl: * Source/WebCore/WebCore.xcodeproj/project.pbxproj: * Source/WebCore/page/Chrome.cpp: (WebCore::Chrome::createBarcodeDetector const): (WebCore::Chrome::getBarcodeDetectorSupportedFormats const): (WebCore::Chrome::createFaceDetector const): (WebCore::Chrome::createTextDetector const): * Source/WebCore/page/Chrome.h: * Source/WebCore/page/ChromeClient.h: (WebCore::ChromeClient::createBarcodeDetector const): (WebCore::ChromeClient::getBarcodeDetectorSupportedFormats const): (WebCore::ChromeClient::createFaceDetector const): (WebCore::ChromeClient::createTextDetector const): * Source/WebKit/CMakeLists.txt: * Source/WebKit/DerivedSources-input.xcfilelist: * Source/WebKit/DerivedSources-output.xcfilelist: * Source/WebKit/DerivedSources.make: * Source/WebKit/GPUProcess/ShapeDetection/RemoteBarcodeDetector.cpp: Copied from Source/WebCore/Modules/ShapeDetection/TextDetector.h. (WebKit::RemoteBarcodeDetector::RemoteBarcodeDetector): (WebKit::RemoteBarcodeDetector::detect): * Source/WebKit/GPUProcess/ShapeDetection/RemoteBarcodeDetector.h: Added. (WebKit::RemoteBarcodeDetector::create): (WebKit::RemoteBarcodeDetector::backing): * Source/WebKit/GPUProcess/ShapeDetection/RemoteBarcodeDetector.messages.in: Added. * Source/WebKit/GPUProcess/ShapeDetection/RemoteFaceDetector.cpp: Copied from Source/WebCore/Modules/ShapeDetection/TextDetector.h. (WebKit::RemoteFaceDetector::RemoteFaceDetector): (WebKit::RemoteFaceDetector::detect): * Source/WebKit/GPUProcess/ShapeDetection/RemoteFaceDetector.h: Added. (WebKit::RemoteFaceDetector::create): (WebKit::RemoteFaceDetector::backing): * Source/WebKit/GPUProcess/ShapeDetection/RemoteFaceDetector.messages.in: Added. * Source/WebKit/GPUProcess/ShapeDetection/RemoteTextDetector.cpp: Copied from Source/WebCore/Modules/ShapeDetection/TextDetector.h. (WebKit::RemoteTextDetector::RemoteTextDetector): (WebKit::RemoteTextDetector::detect): * Source/WebKit/GPUProcess/ShapeDetection/RemoteTextDetector.h: Added. (WebKit::RemoteTextDetector::create): (WebKit::RemoteTextDetector::backing): * Source/WebKit/GPUProcess/ShapeDetection/RemoteTextDetector.messages.in: Added. * Source/WebKit/GPUProcess/ShapeDetection/ShapeDetectionObjectHeap.cpp: Copied from Source/WebCore/Modules/ShapeDetection/TextDetector.cpp. (WebKit::ShapeDetection::ObjectHeap::addObject): (WebKit::ShapeDetection::ObjectHeap::removeObject): (WebKit::ShapeDetection::ObjectHeap::clear): * Source/WebKit/GPUProcess/ShapeDetection/ShapeDetectionObjectHeap.h: Copied from Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/BarcodeDetectorImplementation.h. * Source/WebKit/GPUProcess/graphics/RemoteDisplayListRecorder.cpp: * Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.cpp: (WebKit::RemoteRenderingBackend::RemoteRenderingBackend): (WebKit::RemoteRenderingBackend::createRemoteBarcodeDetector): (WebKit::RemoteRenderingBackend::releaseRemoteBarcodeDetector): (WebKit::RemoteRenderingBackend::getRemoteBarcodeDetectorSupportedFormats): (WebKit::RemoteRenderingBackend::createRemoteFaceDetector): (WebKit::RemoteRenderingBackend::releaseRemoteFaceDetector): (WebKit::RemoteRenderingBackend::createRemoteTextDetector): (WebKit::RemoteRenderingBackend::releaseRemoteTextDetector): * Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.h: * Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.messages.in: * Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererInternalUnitManager.cpp: (WebKit::RemoteAudioMediaStreamTrackRendererInternalUnitManager::Unit::captureUnitHasStopped): * Source/WebKit/Scripts/webkit/messages.py: (serialized_identifiers): (headers_for_type): * Source/WebKit/Scripts/webkit/tests/MessageArgumentDescriptions.cpp: (IPC::serializedIdentifiers): * Source/WebKit/Shared/ShapeDetectionIdentifier.h: Copied from Source/WebCore/Modules/ShapeDetection/Interfaces/BarcodeDetectorOptionsInterface.h. * Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in: * Source/WebKit/Sources.txt: * Source/WebKit/WebKit.xcodeproj/project.pbxproj: * Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteBarcodeDetectorProxy.cpp: Added. (WebKit::ShapeDetection::RemoteBarcodeDetectorProxy::create): (WebKit::ShapeDetection::RemoteBarcodeDetectorProxy::RemoteBarcodeDetectorProxy): (WebKit::ShapeDetection::RemoteBarcodeDetectorProxy::~RemoteBarcodeDetectorProxy): (WebKit::ShapeDetection::RemoteBarcodeDetectorProxy::getSupportedFormats): (WebKit::ShapeDetection::RemoteBarcodeDetectorProxy::detect): * Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteBarcodeDetectorProxy.h: Added. (WebKit::ShapeDetection::RemoteBarcodeDetectorProxy::backing const): * Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteFaceDetectorProxy.cpp: Added. (WebKit::ShapeDetection::RemoteFaceDetectorProxy::create): (WebKit::ShapeDetection::RemoteFaceDetectorProxy::RemoteFaceDetectorProxy): (WebKit::ShapeDetection::RemoteFaceDetectorProxy::~RemoteFaceDetectorProxy): (WebKit::ShapeDetection::RemoteFaceDetectorProxy::detect): * Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteFaceDetectorProxy.h: Added. (WebKit::ShapeDetection::RemoteFaceDetectorProxy::backing const): * Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteTextDetectorProxy.cpp: Added. (WebKit::ShapeDetection::RemoteTextDetectorProxy::create): (WebKit::ShapeDetection::RemoteTextDetectorProxy::RemoteTextDetectorProxy): (WebKit::ShapeDetection::RemoteTextDetectorProxy::~RemoteTextDetectorProxy): (WebKit::ShapeDetection::RemoteTextDetectorProxy::detect): * Source/WebKit/WebProcess/GPU/ShapeDetection/RemoteTextDetectorProxy.h: Added. (WebKit::ShapeDetection::RemoteTextDetectorProxy::backing const): * Source/WebKit/WebProcess/GPU/graphics/RemoteGraphicsContextGLProxy.cpp: * Source/WebKit/WebProcess/GPU/graphics/WebGPU/RemoteGPUProxy.h: * Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.h: * Source/WebKit/WebProcess/Inspector/WebInspectorUIExtensionController.cpp: * Source/WebKit/WebProcess/WebCoreSupport/WebChromeClient.cpp: (WebKit::WebChromeClient::createBarcodeDetector const): (WebKit::WebChromeClient::getBarcodeDetectorSupportedFormats const): (WebKit::WebChromeClient::createFaceDetector const): (WebKit::WebChromeClient::createTextDetector const): * Source/WebKit/WebProcess/WebCoreSupport/WebChromeClient.h: * Source/WebKit/WebProcess/WebPage/RemoteLayerTree/GraphicsLayerCARemote.cpp: * Source/WebKitLegacy/mac/WebCoreSupport/WebChromeClient.h: * Source/WebKitLegacy/mac/WebCoreSupport/WebChromeClient.mm: (WebChromeClient::createBarcodeDetector const): (WebChromeClient::getBarcodeDetectorSupportedFormats const): (WebChromeClient::createFaceDetector const): (WebChromeClient::createTextDetector const): Canonical link: https://commits.webkit.org/263094@main _______________________________________________ webkit-changes mailing list webkit-changes@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-changes