I didn't have much background on this topic to be honest and I wanted to understand the real usage scenario. Thanks a lot for clarifying this.
Nicola On Tuesday, May 9, 2023 at 9:05:36 AM UTC Tony Herre wrote: > Thanks for the reply - hard to judge how much context to include for small > launches like this. > > The motivation is in helping WebRTC web applications which are using both > Mediacapture > Transforms <https://www.w3.org/TR/mediacapture-transform/> to modify raw > audio/video (adding video effects, background replace etc) and WebRTC > Encoded Transforms <https://www.w3.org/TR/webrtc-encoded-transform/> for > modifying the WebRTC encoded data which gets sent on over the network (to > do client-side encryption, appending additional application metadata within > the media, observing per-frame WebRTC metadata like ContributingSources > etc). > Adding this `timestamp` field to RTCEncodedVideoFrameMetadata means that, > when getting an encoded video frame in the encoded transform, the app can > now tell which raw video frame in the mediacapture transform was encoded to > produce this (or when receiving video, which raw frame is later > produced after decoding this encoded frame), by matching > RTCEncodedVideoFrameMetadata.timestamp with VideoFrame.timestamp > <https://www.w3.org/TR/webcodecs/#videoframe-interface>. > > With this, the application can coordinate the two transforms a lot better > - eg changing how raw frames are processed or rendered on the exact frame > in which webrtc metadata changed, or change details of the encoded data on > the exact frame when something in the raw video transform was changed. > > As for developer interest, I'm working closely with at least one quick > adopter, and the quick agreement in the WebRTC Working Group shows this is > generally useful and a part of the roadmap. > > Thanks! > > On Mon, 8 May 2023 at 16:21, Nicola Tommasi <tomma...@chromium.org> wrote: > >> Hi Tony, >> >> Thanks for this intent, could you please clarify why this new field is >> needed and important?Without an explainer is hard to follow this change. >> >> Cheers, >> Nicola >> >> On Tuesday, May 2, 2023 at 2:29:22 PM UTC Tony Herre wrote: >> >>> Contact emails >>> >>> he...@google.com, gui...@chromium.com >>> >>> Spec >>> https://w3c.github.io/webrtc-encoded-transform/#RTCEncodedVideoFrameMetadata >>> >>> particularly >>> PR#173 <https://github.com/w3c/webrtc-encoded-transform/pull/173>. >>> >>> >>> Summary >>> >>> Add a 'timestamp' field to RTCEncodedVideoFrameMetadata containing the >>> presentation timestamp of the associated encoded video frame. >>> >>> Is this feature supported on all six Blink platforms (Windows, Mac, >>> Linux, Chrome OS, Android, and Android WebView)? >>> >>> Yes >>> >>> Risks >>> >>> Interoperability and Compatibility >>> >>> Positive response from all members at W3C WebRTC WG April 2023 Interim, >>> PR landed with no open issues. >>> >>> Ergonomics >>> >>> Added specifically to align with the timestamp field on the WebCodecs >>> <https://www.w3.org/TR/webcodecs/#encodedvideochunk-interface> >>> EncodedVideoChunk >>> <https://www.w3.org/TR/webcodecs/#encodedvideochunk-interface>, and >>> allow matching video frames with the timestamp in the WebCodecs >>> VideoFrame <https://www.w3.org/TR/webcodecs/#videoframe-interface> >>> object. >>> >>> Will also provide the same timestamp already exposed in >>> requestVideoFrameCallback's 'mediaTime'. >>> >>> Is this feature fully tested by web-platform-tests >>> <https://chromium.googlesource.com/chromium/src/+/master/docs/testing/web_platform_tests.md> >>> ? >>> >>> Tested internally by RTCEncodedVideoFrame-capture-timestamp-id.html >>> <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/fast/peerconnection/RTCEncodedVideoFrame-capture-timestamp-id.html>, >>> >>> upstreaming to WPT tracked in crbug.com/1441888. >>> >>> >>> >>> Entry on the feature dashboard >>> >>> None, small delta to launched API. >>> >> -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/9f5fa678-699e-4525-bfbf-e941db8ea1ben%40chromium.org.
