Thanks for the reply - hard to judge how much context to include for small launches like this.
The motivation is in helping WebRTC web applications which are using both Mediacapture Transforms <https://www.w3.org/TR/mediacapture-transform/> to modify raw audio/video (adding video effects, background replace etc) and WebRTC Encoded Transforms <https://www.w3.org/TR/webrtc-encoded-transform/> for modifying the WebRTC encoded data which gets sent on over the network (to do client-side encryption, appending additional application metadata within the media, observing per-frame WebRTC metadata like ContributingSources etc). Adding this `timestamp` field to RTCEncodedVideoFrameMetadata means that, when getting an encoded video frame in the encoded transform, the app can now tell which raw video frame in the mediacapture transform was encoded to produce this (or when receiving video, which raw frame is later produced after decoding this encoded frame), by matching RTCEncodedVideoFrameMetadata.timestamp with VideoFrame.timestamp <https://www.w3.org/TR/webcodecs/#videoframe-interface>. With this, the application can coordinate the two transforms a lot better - eg changing how raw frames are processed or rendered on the exact frame in which webrtc metadata changed, or change details of the encoded data on the exact frame when something in the raw video transform was changed. As for developer interest, I'm working closely with at least one quick adopter, and the quick agreement in the WebRTC Working Group shows this is generally useful and a part of the roadmap. Thanks! On Mon, 8 May 2023 at 16:21, Nicola Tommasi <tomma...@chromium.org> wrote: > Hi Tony, > > Thanks for this intent, could you please clarify why this new field is > needed and important?Without an explainer is hard to follow this change. > > Cheers, > Nicola > > On Tuesday, May 2, 2023 at 2:29:22 PM UTC Tony Herre wrote: > >> Contact emails >> >> he...@google.com, gui...@chromium.com >> >> Spec >> https://w3c.github.io/webrtc-encoded-transform/#RTCEncodedVideoFrameMetadata >> particularly >> PR#173 <https://github.com/w3c/webrtc-encoded-transform/pull/173>. >> >> >> Summary >> >> Add a 'timestamp' field to RTCEncodedVideoFrameMetadata containing the >> presentation timestamp of the associated encoded video frame. >> >> Is this feature supported on all six Blink platforms (Windows, Mac, >> Linux, Chrome OS, Android, and Android WebView)? >> >> Yes >> >> Risks >> >> Interoperability and Compatibility >> >> Positive response from all members at W3C WebRTC WG April 2023 Interim, >> PR landed with no open issues. >> >> Ergonomics >> >> Added specifically to align with the timestamp field on the WebCodecs >> <https://www.w3.org/TR/webcodecs/#encodedvideochunk-interface> >> EncodedVideoChunk >> <https://www.w3.org/TR/webcodecs/#encodedvideochunk-interface>, and >> allow matching video frames with the timestamp in the WebCodecs >> VideoFrame <https://www.w3.org/TR/webcodecs/#videoframe-interface> >> object. >> >> Will also provide the same timestamp already exposed in >> requestVideoFrameCallback's 'mediaTime'. >> >> Is this feature fully tested by web-platform-tests >> <https://chromium.googlesource.com/chromium/src/+/master/docs/testing/web_platform_tests.md> >> ? >> >> Tested internally by RTCEncodedVideoFrame-capture-timestamp-id.html >> <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/fast/peerconnection/RTCEncodedVideoFrame-capture-timestamp-id.html>, >> upstreaming to WPT tracked in crbug.com/1441888. >> >> >> >> Entry on the feature dashboard >> >> None, small delta to launched API. >> > -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAArnMxFuMAb_rsoynj2XDDZtSQ8-zZmVtxg3Crd-HimnK-5Exw%40mail.gmail.com.
