[blink-dev] Re: Intent to Ship: timestamp field on RTCEncodedVideoFrameMetadata

2023-05-10 Thread Nicola Tommasi 
I didn't have much background on this topic to be honest and I wanted to 
understand the real usage scenario. Thanks a lot for clarifying this. 

Nicola

On Tuesday, May 9, 2023 at 9:05:36 AM UTC Tony Herre wrote:

> Thanks for the reply - hard to judge how much context to include for small 
> launches like this.
>
> The motivation is in helping WebRTC web applications which are using both 
> Mediacapture 
> Transforms  to modify raw 
> audio/video (adding video effects, background replace etc) and WebRTC 
> Encoded Transforms  for 
> modifying the WebRTC encoded data which gets sent on over the network (to 
> do client-side encryption, appending additional application metadata within 
> the media, observing per-frame WebRTC metadata like ContributingSources 
> etc).
> Adding this `timestamp` field to RTCEncodedVideoFrameMetadata means that, 
> when getting an encoded video frame in the encoded transform, the app can 
> now tell which raw video frame in the mediacapture transform was encoded to 
> produce this (or when receiving video, which raw frame is later 
> produced after decoding this encoded frame), by matching 
> RTCEncodedVideoFrameMetadata.timestamp with VideoFrame.timestamp 
> .
>
> With this, the application can coordinate the two transforms a lot better 
> - eg changing how raw frames are processed or rendered on the exact frame 
> in which webrtc metadata changed, or change details of the encoded data on 
> the exact frame when something in the raw video transform was changed.
>
> As for developer interest, I'm working closely with at least one quick 
> adopter, and the quick agreement in the WebRTC Working Group shows this is 
> generally useful and a part of the roadmap.
>
> Thanks!
>
> On Mon, 8 May 2023 at 16:21, Nicola Tommasi  wrote:
>
>> Hi Tony,
>>
>> Thanks for this intent, could you please clarify why this new field is 
>> needed and important?Without an explainer is hard to follow this change.
>>
>> Cheers,
>> Nicola
>>
>> On Tuesday, May 2, 2023 at 2:29:22 PM UTC Tony Herre wrote:
>>
>>> Contact emails
>>>
>>> he...@google.com, gui...@chromium.com
>>>
>>> Spec
>>> https://w3c.github.io/webrtc-encoded-transform/#RTCEncodedVideoFrameMetadata
>>>  
>>> particularly
>>> PR#173 .
>>>
>>>
>>> Summary
>>>
>>> Add a 'timestamp' field to RTCEncodedVideoFrameMetadata containing the 
>>> presentation timestamp of the associated encoded video frame.
>>>
>>> Is this feature supported on all six Blink platforms (Windows, Mac, 
>>> Linux, Chrome OS, Android, and Android WebView)?
>>>
>>> Yes
>>>
>>> Risks
>>>
>>> Interoperability and Compatibility
>>>
>>> Positive response from all members at W3C WebRTC WG April 2023 Interim, 
>>> PR landed with no open issues.
>>>
>>> Ergonomics
>>>
>>> Added specifically to align with the timestamp field on the WebCodecs 
>>> 
>>> EncodedVideoChunk 
>>> , and 
>>> allow matching video frames with the timestamp in the WebCodecs 
>>> VideoFrame  
>>> object.
>>>
>>> Will also provide the same timestamp already exposed in 
>>> requestVideoFrameCallback's 'mediaTime'.
>>>
>>> Is this feature fully tested by web-platform-tests 
>>> 
>>> ?
>>>
>>> Tested internally by RTCEncodedVideoFrame-capture-timestamp-id.html 
>>> ,
>>>  
>>> upstreaming to WPT tracked in crbug.com/1441888.
>>>
>>>
>>>
>>> Entry on the feature dashboard
>>>
>>> None, small delta to launched API. 
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/9f5fa678-699e-4525-bfbf-e941db8ea1ben%40chromium.org.

[blink-dev] Re: Intent to Ship: timestamp field on RTCEncodedVideoFrameMetadata

2023-05-09 Thread 'Tony Herre' via blink-dev 
Thanks for the reply - hard to judge how much context to include for small
launches like this.

The motivation is in helping WebRTC web applications which are using
both Mediacapture
Transforms  to modify raw
audio/video (adding video effects, background replace etc) and WebRTC
Encoded Transforms  for
modifying the WebRTC encoded data which gets sent on over the network (to
do client-side encryption, appending additional application metadata within
the media, observing per-frame WebRTC metadata like ContributingSources
etc).
Adding this `timestamp` field to RTCEncodedVideoFrameMetadata means that,
when getting an encoded video frame in the encoded transform, the app can
now tell which raw video frame in the mediacapture transform was encoded to
produce this (or when receiving video, which raw frame is later
produced after decoding this encoded frame), by matching
RTCEncodedVideoFrameMetadata.timestamp with VideoFrame.timestamp
.

With this, the application can coordinate the two transforms a lot better -
eg changing how raw frames are processed or rendered on the exact frame in
which webrtc metadata changed, or change details of the encoded data on the
exact frame when something in the raw video transform was changed.

As for developer interest, I'm working closely with at least one quick
adopter, and the quick agreement in the WebRTC Working Group shows this is
generally useful and a part of the roadmap.

Thanks!

On Mon, 8 May 2023 at 16:21, Nicola Tommasi  wrote:

> Hi Tony,
>
> Thanks for this intent, could you please clarify why this new field is
> needed and important?Without an explainer is hard to follow this change.
>
> Cheers,
> Nicola
>
> On Tuesday, May 2, 2023 at 2:29:22 PM UTC Tony Herre wrote:
>
>> Contact emails
>>
>> he...@google.com, gui...@chromium.com
>>
>> Spec
>> https://w3c.github.io/webrtc-encoded-transform/#RTCEncodedVideoFrameMetadata
>> particularly
>> PR#173 .
>>
>>
>> Summary
>>
>> Add a 'timestamp' field to RTCEncodedVideoFrameMetadata containing the
>> presentation timestamp of the associated encoded video frame.
>>
>> Is this feature supported on all six Blink platforms (Windows, Mac,
>> Linux, Chrome OS, Android, and Android WebView)?
>>
>> Yes
>>
>> Risks
>>
>> Interoperability and Compatibility
>>
>> Positive response from all members at W3C WebRTC WG April 2023 Interim,
>> PR landed with no open issues.
>>
>> Ergonomics
>>
>> Added specifically to align with the timestamp field on the WebCodecs
>> 
>> EncodedVideoChunk
>> , and
>> allow matching video frames with the timestamp in the WebCodecs
>> VideoFrame 
>> object.
>>
>> Will also provide the same timestamp already exposed in
>> requestVideoFrameCallback's 'mediaTime'.
>>
>> Is this feature fully tested by web-platform-tests
>> 
>> ?
>>
>> Tested internally by RTCEncodedVideoFrame-capture-timestamp-id.html
>> ,
>> upstreaming to WPT tracked in crbug.com/1441888.
>>
>>
>>
>> Entry on the feature dashboard
>>
>> None, small delta to launched API.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAArnMxFuMAb_rsoynj2XDDZtSQ8-zZmVtxg3Crd-HimnK-5Exw%40mail.gmail.com.

[blink-dev] Re: Intent to Ship: timestamp field on RTCEncodedVideoFrameMetadata

2023-05-08 Thread Nicola Tommasi 
Hi Tony,

Thanks for this intent, could you please clarify why this new field is 
needed and important?Without an explainer is hard to follow this change.

Cheers,
Nicola

On Tuesday, May 2, 2023 at 2:29:22 PM UTC Tony Herre wrote:

> Contact emails
>
> he...@google.com, gui...@chromium.com
>
> Spec
> https://w3c.github.io/webrtc-encoded-transform/#RTCEncodedVideoFrameMetadata 
> particularly
> PR#173 .
>
>
> Summary
>
> Add a 'timestamp' field to RTCEncodedVideoFrameMetadata containing the 
> presentation timestamp of the associated encoded video frame.
>
> Is this feature supported on all six Blink platforms (Windows, Mac, Linux, 
> Chrome OS, Android, and Android WebView)?
>
> Yes
>
> Risks
>
> Interoperability and Compatibility
>
> Positive response from all members at W3C WebRTC WG April 2023 Interim, PR 
> landed with no open issues.
>
> Ergonomics
>
> Added specifically to align with the timestamp field on the WebCodecs 
> 
> EncodedVideoChunk 
> , and allow 
> matching video frames with the timestamp in the WebCodecs VideoFrame 
>  object.
>
> Will also provide the same timestamp already exposed in 
> requestVideoFrameCallback's 'mediaTime'.
>
> Is this feature fully tested by web-platform-tests 
> 
> ?
>
> Tested internally by RTCEncodedVideoFrame-capture-timestamp-id.html 
> ,
>  
> upstreaming to WPT tracked in crbug.com/1441888.
>
>
>
> Entry on the feature dashboard
>
> None, small delta to launched API. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/cfa25949-7d3d-4dfd-920d-e89eab6933d5n%40chromium.org.