LGTM1, but does this need to bounce through main thread video decode for every frame sampled? Is it not possible to target a WebGPU texture with the decoded buffer contents without needing to lean on the main thread's `requestVideoFrameCallback()`?
On Friday, June 16, 2023 at 1:20:31 AM UTC-7 fbea...@google.com wrote: > Contact emails > > cwal...@google.com, kain...@google.com, bajo...@google.com > > Explainer > > > https://developer.chrome.com/blog/new-in-webgpu-113/#use-webcodecs-videoframe-source-in-importexternaltexture > > https://github.com/gpuweb/gpuweb/issues/1380 > > https://github.com/gpuweb/gpuweb/issues/4165 > > https://gpuweb.github.io/gpuweb/explainer/#image-input > > Specification > > https://gpuweb.github.io/gpuweb/#gpuexternaltexture > > Design docs > > https://github.com/gpuweb/gpuweb/issues/1380 > > Summary > > WebGPU exposes an API to create opaque "external texture" objects from > HTMLVideoElement. These object can be used to sample the video frames > efficiently, potentially in a zero-copy way directly from the source YUV > data. However the WebGPU specification for the first version of WebGPU does > not allow creating GPUExternalTextures from WebCodecs VideoFrame objects. > This capability is important for advanced video processing applications > that are already using WebCodecs and would like to integrate WebGPU in the > video processing pipeline. This feature adds support for using a VideoFrame > as the source for a GPUExternalTexture and a copyExternalImageToTexture > call. > > Blink component > > Blink>WebGPU > <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EWebGPU> > > TAG review > > A new TAG review is not needed in this case as WebGPU had one recently > already: https://github.com/w3ctag/design-reviews/issues/626 > > This small but important addition is about adding VideoFrame support on > top of HTMLVideoElement support to GPUExternalTextures and > copyExternalImageToTexture call. > > TAG review status > > Not applicable > > Risks > Interoperability and Compatibility > > Gecko: Positive ( > https://github.com/gpuweb/gpuweb/wiki/Minutes-2023-04-19#investigation-import-videoframe-from-webcodec-to-webgpu-1380:~:text=KG%3A%20the%20proposal%20above%20makes%20sense) > > WebCodecs is listed as "worth prototyping" which likely means this > intergration is the same. > > WebKit: In development (https://github.com/WebKit/WebKit/pull/14055) > > Web developers: Positive > > Other signals: > > Ergonomics > > No ergonomic risk. This API would be used at the intersection of WebGPU > and WebCodec. It is designed to keep performance as high as possible by > allowing zero-copy sampling of YUV frame data. > > > Security > > The lifetime management of VideoFrame was taken into account of this > feature. No other security considerations. > > > WebView application risks > > N/A > > > Debuggability > > No support. > > Will this feature be supported on all six Blink platforms (Windows, Mac, > Linux, Chrome OS, Android, and Android WebView)? > > Yes > > It will be supported where WebGPU is supported. For now, it is only > ChromeOS, macOS, and Windows. > > Is this feature fully tested by web-platform-tests > <https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md> > ? > > Yes > > DevTrial instructions > > https://github.com/gpuweb/gpuweb/issues/1380 > > Requires code in //chrome? > > False > > Tracking bug > > https://bugs.chromium.org/p/chromium/issues/detail?id=1455083 > > Availability expectation > > This feature will be part of the WebGPU API in all browsers when they ship > it. > > Sample links > > https://webgpu.github.io/webgpu-samples/samples/videoUploadingWebCodecs > > Link to entry on the Chrome Platform Status > > https://chromestatus.com/feature/5078348864159744 > > Links to previous Intent discussions > > Intent to Experiment: > https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGdfWNPH6Jk-g%2B3Nkx0F_rmn05kWcWU%3Dd3cOJNpjfZ3nAWjXcQ%40mail.gmail.com > > > This intent message was generated by Chrome Platform Status > <https://chromestatus.com/>. > > -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/df47c5f9-27ee-4197-8a3a-1d7a69a919dcn%40chromium.org.
