Thank you Alex. WebGPU can use VideoFrames in any worker so as long as they can be produced off-main thread <https://www.w3.org/TR/webcodecs/#best-practices-developers> then it shouldn't block. VideoFrames are also possible to postMessage between workers <https://www.w3.org/TR/webcodecs/#videoframe-transfer-serialization> if needed.
On Fri, Jun 16, 2023 at 10:21 PM Alex Russell <slightly...@chromium.org> wrote: > LGTM1, but does this need to bounce through main thread video decode for > every frame sampled? Is it not possible to target a WebGPU texture with the > decoded buffer contents without needing to lean on the main thread's > `requestVideoFrameCallback()`? > > On Friday, June 16, 2023 at 1:20:31 AM UTC-7 fbea...@google.com wrote: > >> Contact emails >> >> cwal...@google.com, kain...@google.com, bajo...@google.com >> >> Explainer >> >> >> https://developer.chrome.com/blog/new-in-webgpu-113/#use-webcodecs-videoframe-source-in-importexternaltexture >> >> https://github.com/gpuweb/gpuweb/issues/1380 >> >> https://github.com/gpuweb/gpuweb/issues/4165 >> >> https://gpuweb.github.io/gpuweb/explainer/#image-input >> >> Specification >> >> https://gpuweb.github.io/gpuweb/#gpuexternaltexture >> >> Design docs >> >> https://github.com/gpuweb/gpuweb/issues/1380 >> >> Summary >> >> WebGPU exposes an API to create opaque "external texture" objects from >> HTMLVideoElement. These object can be used to sample the video frames >> efficiently, potentially in a zero-copy way directly from the source YUV >> data. However the WebGPU specification for the first version of WebGPU does >> not allow creating GPUExternalTextures from WebCodecs VideoFrame objects. >> This capability is important for advanced video processing applications >> that are already using WebCodecs and would like to integrate WebGPU in the >> video processing pipeline. This feature adds support for using a VideoFrame >> as the source for a GPUExternalTexture and a copyExternalImageToTexture >> call. >> >> Blink component >> >> Blink>WebGPU >> <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EWebGPU> >> >> TAG review >> >> A new TAG review is not needed in this case as WebGPU had one recently >> already: https://github.com/w3ctag/design-reviews/issues/626 >> >> This small but important addition is about adding VideoFrame support on >> top of HTMLVideoElement support to GPUExternalTextures and >> copyExternalImageToTexture call. >> >> TAG review status >> >> Not applicable >> >> Risks >> Interoperability and Compatibility >> >> Gecko: Positive ( >> https://github.com/gpuweb/gpuweb/wiki/Minutes-2023-04-19#investigation-import-videoframe-from-webcodec-to-webgpu-1380:~:text=KG%3A%20the%20proposal%20above%20makes%20sense) >> WebCodecs is listed as "worth prototyping" which likely means this >> intergration is the same. >> >> WebKit: In development (https://github.com/WebKit/WebKit/pull/14055) >> >> Web developers: Positive >> >> Other signals: >> >> Ergonomics >> >> No ergonomic risk. This API would be used at the intersection of WebGPU >> and WebCodec. It is designed to keep performance as high as possible by >> allowing zero-copy sampling of YUV frame data. >> >> >> Security >> >> The lifetime management of VideoFrame was taken into account of this >> feature. No other security considerations. >> >> >> WebView application risks >> >> N/A >> >> >> Debuggability >> >> No support. >> >> Will this feature be supported on all six Blink platforms (Windows, Mac, >> Linux, Chrome OS, Android, and Android WebView)? >> >> Yes >> >> It will be supported where WebGPU is supported. For now, it is only >> ChromeOS, macOS, and Windows. >> >> Is this feature fully tested by web-platform-tests >> <https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md> >> ? >> >> Yes >> >> DevTrial instructions >> >> https://github.com/gpuweb/gpuweb/issues/1380 >> >> Requires code in //chrome? >> >> False >> >> Tracking bug >> >> https://bugs.chromium.org/p/chromium/issues/detail?id=1455083 >> >> Availability expectation >> >> This feature will be part of the WebGPU API in all browsers when they >> ship it. >> >> Sample links >> >> https://webgpu.github.io/webgpu-samples/samples/videoUploadingWebCodecs >> >> Link to entry on the Chrome Platform Status >> >> https://chromestatus.com/feature/5078348864159744 >> >> Links to previous Intent discussions >> >> Intent to Experiment: >> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGdfWNPH6Jk-g%2B3Nkx0F_rmn05kWcWU%3Dd3cOJNpjfZ3nAWjXcQ%40mail.gmail.com >> >> >> This intent message was generated by Chrome Platform Status >> <https://chromestatus.com/>. >> >> -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPpwU5Kb%3DNMqiZURzoUK%2BJ45Au2P_rY7yAzK8-OeCZ-g3%2Bo98g%40mail.gmail.com.
