Branch: refs/heads/main Home: https://github.com/WebKit/WebKit Commit: ce99b4d5dd33bd28ffb58899e356841cb5ed7285 https://github.com/WebKit/WebKit/commit/ce99b4d5dd33bd28ffb58899e356841cb5ed7285 Author: Myles C. Maxfield <mmaxfi...@apple.com> Date: 2023-04-25 (Tue, 25 Apr 2023)
Changed paths: M Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml M Source/WTF/wtf/PlatformHave.h M Source/WTF/wtf/PlatformUse.h M Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/BarcodeDetectorImplementation.h M Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/BarcodeDetectorImplementation.mm M Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/FaceDetectorImplementation.h M Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/FaceDetectorImplementation.mm M Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/TextDetectorImplementation.h M Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/TextDetectorImplementation.mm A Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/VisionUtilities.h A Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/VisionUtilities.mm M Source/WebCore/Modules/async-clipboard/ClipboardImageReader.h M Source/WebCore/PAL/pal/spi/cocoa/SpeechSPI.h M Source/WebCore/SourcesCocoa.txt M Source/WebCore/WebCore.xcodeproj/project.pbxproj Log Message: ----------- [Shape Detection] [Cocoa] Implement first draft of the shape detection API using the Vision framework https://bugs.webkit.org/show_bug.cgi?id=255803 rdar://108380805 Reviewed by Mike Wyrzykowski. This is the first draft implementation of the shape detection API. This patch makes WebCore.framework link with Vision.framework. The shape detection API is very very similar to Vision's API, so the implementation is almost trivial: each call to detect() just turns into the creation of a VNRequest and a VNRequestHandler, and calls performRequests() on the request handler. VNRequest has a callback-based codepath, where you can say -[VNRequest initWithCompletionHandler:], but this isn't really suitable for our purposes, at least right out-of-the-box, for 2 reasons: 1. Even if you set up the VNRequest this way, the call to -[VNImageRequestHandler performRequest:error:] is still synchronous. It blocks until all the requests are complete, and AFAICT there's no way to make it not block. So, using the callback doesn't actually free up the main thread to do other work. (The purpose of the completion handler is when you pass many requests to a single performRequests: call, you can be notified as the requests complete, one by one - *not* so that the call to performRequests: can be asynchronous. If you want the call to performRequests: to be asynchronous, you're supposed to call it from a background thread yourself.) 2. The callbacks are called on another thread internal to Vision, so if we wanted to use them, we'd have to immediately bounce back to the RemoteRenderingBackend's worker thread in WebKit2, or the main thread in WebKitLegacy. This means the Shape Detection objects would have to have more plumbing to take a generic handler which can schedule work on the right thread. At some point, it probably would make sense to do this plumbing work + calling Vision on a background thread, but it isn't really necessary for this first-draft implementation. I haven't profiled these calls yet to know if they're even expensive enough to be worth calling on a background thread or not. This patch also sets the usesCPUOnly flag, because otherwise the API wouldn't work in WebKit 2, because the GPU Process's sandbox is blocking the connection to aned on Apple Silicon devices (I haven't tried on Intel devices, maybe it works using the GPU there?). I'm working with Per Arne to see what we can do about this. If you manually disable the sandbox, this flag isn't necessary. The tests for this patch are at https://github.com/web-platform-tests/wpt/commit/b5a2b1e1b0bd51817dca49b49eb3e8825833b216. I will import them in a follow-up patch. This patch marks the shape detection as "testable." * Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/BarcodeDetectorImplementation.h: * Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/BarcodeDetectorImplementation.mm: (WebCore::ShapeDetection::convertSymbology): (WebCore::ShapeDetection::convertBarcodeFormat): (WebCore::ShapeDetection::convertRequestedBarcodeFormatSet): (WebCore::ShapeDetection::BarcodeDetectorImpl::BarcodeDetectorImpl): (WebCore::ShapeDetection::request): (WebCore::ShapeDetection::BarcodeDetectorImpl::getSupportedFormats): (WebCore::ShapeDetection::BarcodeDetectorImpl::detect): * Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/FaceDetectorImplementation.h: * Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/FaceDetectorImplementation.mm: (WebCore::ShapeDetection::FaceDetectorImpl::FaceDetectorImpl): (WebCore::ShapeDetection::convertLandmark): (WebCore::ShapeDetection::convertLandmarks): (WebCore::ShapeDetection::FaceDetectorImpl::detect): * Source/WebCore/Modules/ShapeDetection/Implementation/Cocoa/TextDetectorImplementation.mm: (WebCore::ShapeDetection::TextDetectorImpl::detect): * Source/WebCore/WebCore.xcodeproj/project.pbxproj: Canonical link: https://commits.webkit.org/263397@main _______________________________________________ webkit-changes mailing list webkit-changes@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-changes