Re: [blink-dev] Intent to Extend Experiment: WebRTC encoded transform - Modify Metadata functions (setMetadata method)

2024-05-03 Thread Yoav Weiss (@Shopify) 
LGTM to extend experimentation until M129 inclusive

On Fri, May 3, 2024 at 5:05 PM Guido Urdaneta  wrote:

> Yes, but since the main motivation is to let developers migrate to the
> final version of the API (which changed in the last milestone of the
> original OT), it would be acceptable to have a shorter extension. This is,
> of course, assuming the I2S for the final version of the API is approved.
>

That's fine! :)


>
> On Fri, May 3, 2024 at 2:39 PM Yoav Weiss (@Shopify) <
> yoavwe...@chromium.org> wrote:
>
>> Do I understand correctly that you want to extend the OT for 3 more
>> milestones, up to 129 (inclusive)?
>>
>>
>>
>> On Thu, May 2, 2024 at 2:45 PM Guido Urdaneta 
>> wrote:
>>
>>> Contact emails...@chromium.org, gui...@chromium.org,
>>> agpa...@chromium.org
>>>
>>> Explainer
>>> https://github.com/guidou/webrtc-extensions/blob/main/constructor-explainer.md
>>>
>>> Specification
>>> https://w3c.github.io/webrtc-encoded-transform/#dom-rtcencodedvideoframe-constructor
>>>
>>> Summary
>>>
>>> Allow WebRTC Encoded Transform API to manipulate audio and video frame
>>> metadata. Some WebRTC Encoded Transform use cases involve manipulation of
>>> not only the payload of encoded video / audio frames but also its metadata.
>>> For example, if a peer connection negotiates a custom codec and an encoded
>>> transform is used to implement part or all of the the custom codec and
>>> needs to set the output codec type as part of the metadata of the output
>>> frame. See https://www.w3.org/2024/04/23-webrtc-minutes.html#t01
>>>
>>>
>>>
>>> Blink componentBlink>WebRTC
>>> 
>>>
>>> TAG reviewThe original full spec was reviewed by TAG here:
>>> https://github.com/w3ctag/design-reviews/issues/531
>>> No TAG review requested yet for the setMetadata method (during the
>>> Working Group discussion it was decided to use a constructor, but interest
>>> in the method version was recently revived).
>>>
>>>
>>> Other use cases:
>>>
>>> https://w3c.github.io/webrtc-nv-use-cases/#live-encoded-media
>>>
>>> https://w3c.github.io/webrtc-nv-use-cases/#stored-encoded-media
>>>
>>> https://w3c.github.io/webrtc-nv-use-cases/#auction
>>>
>>> TAG review statusPending
>>>
>>> Chromium Trial NameRTCEncodedFrameSetMetadata
>>>
>>> Origin Trial documentation link
>>> https://github.com/palak8669/webrtc-encoded-transform/blob/create-encoded-explainer/create-encoded-explainer.md
>>>
>>> WebFeature UseCounter nameRTCEncodedFrameSetMetadata
>>>
>>> Risks
>>>
>>>
>>> Interoperability and Compatibility
>>>
>>> Interoperability risk: There is always the risk that other browsers will
>>> not implement this feature. This risk is mitigated by alignment across
>>> browser vendors in the W3C WebRTC Working Group around the spec.
>>> Compatibility risk: This is a new feature intended to support new use
>>> cases. It introduces no breaking changes, so we do not expect any
>>> compatibility issues.
>>>
>>> *Gecko*: No signal
>>> However, they have shown interest in reviving the discussion for the
>>> setMetadata() method after achieving consensus on the Custom Codec
>>> negotiation API for WebRTC Encoded Transform.
>>>
>>> *WebKit*: No signal
>>>
>>> *Web developers*: Positive
>>>
>>> *Other signals*:
>>>
>>> Ergonomics
>>>
>>> This feature is an extension to WebRTC Encoded Transform, which itself
>>> is an extension to WebRTC/RTCPeerConnection.
>>>
>>>
>>> Activation
>>>
>>> No significant risks identified.
>>>
>>>
>>> Security
>>>
>>> No new security risks identified.
>>>
>>>
>>> WebView application risks
>>>
>>> Does this intent deprecate or change behavior of existing APIs, such
>>> that it has potentially high risk for Android WebView-based applications?
>>>
>>> No
>>>
>>>
>>> Goals for experimentation
>>>
>>> Determine if the proposed API properly supports the intended use case.
>>>
>>>
>>> Reason this experiment is being extended
>>>
>>> There are two reasons to request this extension: 1. This proposal
>>> initially started as a setMetadata() method on encoded frames, but the
>>> result of discussions in the W3C WebRTC Working Group was that introducing
>>> a constructor (instead of a method) was a better fit for the use cases for
>>> which there was consensus in the WG. After a few iterations over the
>>> constructor API shape, the WG achieved consensus recently and we have sent
>>> an Intent to Ship for that. However, the final version of the constructor
>>> only became available in M126 (the last milestone of the Origin Trial) and
>>> we would like to give developers a little more time to migrate to the
>>> shipped version of the API. 2. After achieving consensus on the constructor
>>> with custom metadata, a new use case has been discussed in the WG that has
>>> revived interest in the original setMetadata() proposal. The WG has
>>> achieved consensus on a new API for custom codec negotiation for which
>>> setMetadata() looks like a better fit than the

Re: [blink-dev] Intent to Extend Experiment: WebRTC encoded transform - Modify Metadata functions (setMetadata method)

2024-05-03 Thread Guido Urdaneta 
Yes, but since the main motivation is to let developers migrate to the
final version of the API (which changed in the last milestone of the
original OT), it would be acceptable to have a shorter extension. This is,
of course, assuming the I2S for the final version of the API is approved.

On Fri, May 3, 2024 at 2:39 PM Yoav Weiss (@Shopify) 
wrote:

> Do I understand correctly that you want to extend the OT for 3 more
> milestones, up to 129 (inclusive)?
>
>
>
> On Thu, May 2, 2024 at 2:45 PM Guido Urdaneta  wrote:
>
>> Contact emails...@chromium.org, gui...@chromium.org, agpa...@chromium.org
>>
>> Explainer
>> https://github.com/guidou/webrtc-extensions/blob/main/constructor-explainer.md
>>
>> Specification
>> https://w3c.github.io/webrtc-encoded-transform/#dom-rtcencodedvideoframe-constructor
>>
>> Summary
>>
>> Allow WebRTC Encoded Transform API to manipulate audio and video frame
>> metadata. Some WebRTC Encoded Transform use cases involve manipulation of
>> not only the payload of encoded video / audio frames but also its metadata.
>> For example, if a peer connection negotiates a custom codec and an encoded
>> transform is used to implement part or all of the the custom codec and
>> needs to set the output codec type as part of the metadata of the output
>> frame. See https://www.w3.org/2024/04/23-webrtc-minutes.html#t01
>>
>>
>>
>> Blink componentBlink>WebRTC
>> 
>>
>> TAG reviewThe original full spec was reviewed by TAG here:
>> https://github.com/w3ctag/design-reviews/issues/531
>> No TAG review requested yet for the setMetadata method (during the
>> Working Group discussion it was decided to use a constructor, but interest
>> in the method version was recently revived).
>>
>>
>> Other use cases:
>>
>> https://w3c.github.io/webrtc-nv-use-cases/#live-encoded-media
>>
>> https://w3c.github.io/webrtc-nv-use-cases/#stored-encoded-media
>>
>> https://w3c.github.io/webrtc-nv-use-cases/#auction
>>
>> TAG review statusPending
>>
>> Chromium Trial NameRTCEncodedFrameSetMetadata
>>
>> Origin Trial documentation link
>> https://github.com/palak8669/webrtc-encoded-transform/blob/create-encoded-explainer/create-encoded-explainer.md
>>
>> WebFeature UseCounter nameRTCEncodedFrameSetMetadata
>>
>> Risks
>>
>>
>> Interoperability and Compatibility
>>
>> Interoperability risk: There is always the risk that other browsers will
>> not implement this feature. This risk is mitigated by alignment across
>> browser vendors in the W3C WebRTC Working Group around the spec.
>> Compatibility risk: This is a new feature intended to support new use
>> cases. It introduces no breaking changes, so we do not expect any
>> compatibility issues.
>>
>> *Gecko*: No signal
>> However, they have shown interest in reviving the discussion for the
>> setMetadata() method after achieving consensus on the Custom Codec
>> negotiation API for WebRTC Encoded Transform.
>>
>> *WebKit*: No signal
>>
>> *Web developers*: Positive
>>
>> *Other signals*:
>>
>> Ergonomics
>>
>> This feature is an extension to WebRTC Encoded Transform, which itself is
>> an extension to WebRTC/RTCPeerConnection.
>>
>>
>> Activation
>>
>> No significant risks identified.
>>
>>
>> Security
>>
>> No new security risks identified.
>>
>>
>> WebView application risks
>>
>> Does this intent deprecate or change behavior of existing APIs, such that
>> it has potentially high risk for Android WebView-based applications?
>>
>> No
>>
>>
>> Goals for experimentation
>>
>> Determine if the proposed API properly supports the intended use case.
>>
>>
>> Reason this experiment is being extended
>>
>> There are two reasons to request this extension: 1. This proposal
>> initially started as a setMetadata() method on encoded frames, but the
>> result of discussions in the W3C WebRTC Working Group was that introducing
>> a constructor (instead of a method) was a better fit for the use cases for
>> which there was consensus in the WG. After a few iterations over the
>> constructor API shape, the WG achieved consensus recently and we have sent
>> an Intent to Ship for that. However, the final version of the constructor
>> only became available in M126 (the last milestone of the Origin Trial) and
>> we would like to give developers a little more time to migrate to the
>> shipped version of the API. 2. After achieving consensus on the constructor
>> with custom metadata, a new use case has been discussed in the WG that has
>> revived interest in the original setMetadata() proposal. The WG has
>> achieved consensus on a new API for custom codec negotiation for which
>> setMetadata() looks like a better fit than the constructor since it doesn't
>> require copying the payload of the encoded frame. So the WG might achieve
>> consensus on adding setMetadata() after all. See the resolution of
>> https://www.w3.org/2024/04/23-webrtc-minutes.html#t01 Since
>> setMetadata() might be added to the spec in addition to the

Re: [blink-dev] Intent to Extend Experiment: WebRTC encoded transform - Modify Metadata functions (setMetadata method)

2024-05-03 Thread Yoav Weiss (@Shopify) 
Do I understand correctly that you want to extend the OT for 3 more
milestones, up to 129 (inclusive)?



On Thu, May 2, 2024 at 2:45 PM Guido Urdaneta  wrote:

> Contact emails...@chromium.org, gui...@chromium.org, agpa...@chromium.org
>
> Explainer
> https://github.com/guidou/webrtc-extensions/blob/main/constructor-explainer.md
>
> Specification
> https://w3c.github.io/webrtc-encoded-transform/#dom-rtcencodedvideoframe-constructor
>
> Summary
>
> Allow WebRTC Encoded Transform API to manipulate audio and video frame
> metadata. Some WebRTC Encoded Transform use cases involve manipulation of
> not only the payload of encoded video / audio frames but also its metadata.
> For example, if a peer connection negotiates a custom codec and an encoded
> transform is used to implement part or all of the the custom codec and
> needs to set the output codec type as part of the metadata of the output
> frame. See https://www.w3.org/2024/04/23-webrtc-minutes.html#t01
>
>
>
> Blink componentBlink>WebRTC
> 
>
> TAG reviewThe original full spec was reviewed by TAG here:
> https://github.com/w3ctag/design-reviews/issues/531
> No TAG review requested yet for the setMetadata method (during the Working
> Group discussion it was decided to use a constructor, but interest in the
> method version was recently revived).
>
>
> Other use cases:
>
> https://w3c.github.io/webrtc-nv-use-cases/#live-encoded-media
>
> https://w3c.github.io/webrtc-nv-use-cases/#stored-encoded-media
>
> https://w3c.github.io/webrtc-nv-use-cases/#auction
>
> TAG review statusPending
>
> Chromium Trial NameRTCEncodedFrameSetMetadata
>
> Origin Trial documentation link
> https://github.com/palak8669/webrtc-encoded-transform/blob/create-encoded-explainer/create-encoded-explainer.md
>
> WebFeature UseCounter nameRTCEncodedFrameSetMetadata
>
> Risks
>
>
> Interoperability and Compatibility
>
> Interoperability risk: There is always the risk that other browsers will
> not implement this feature. This risk is mitigated by alignment across
> browser vendors in the W3C WebRTC Working Group around the spec.
> Compatibility risk: This is a new feature intended to support new use
> cases. It introduces no breaking changes, so we do not expect any
> compatibility issues.
>
> *Gecko*: No signal
> However, they have shown interest in reviving the discussion for the
> setMetadata() method after achieving consensus on the Custom Codec
> negotiation API for WebRTC Encoded Transform.
>
> *WebKit*: No signal
>
> *Web developers*: Positive
>
> *Other signals*:
>
> Ergonomics
>
> This feature is an extension to WebRTC Encoded Transform, which itself is
> an extension to WebRTC/RTCPeerConnection.
>
>
> Activation
>
> No significant risks identified.
>
>
> Security
>
> No new security risks identified.
>
>
> WebView application risks
>
> Does this intent deprecate or change behavior of existing APIs, such that
> it has potentially high risk for Android WebView-based applications?
>
> No
>
>
> Goals for experimentation
>
> Determine if the proposed API properly supports the intended use case.
>
>
> Reason this experiment is being extended
>
> There are two reasons to request this extension: 1. This proposal
> initially started as a setMetadata() method on encoded frames, but the
> result of discussions in the W3C WebRTC Working Group was that introducing
> a constructor (instead of a method) was a better fit for the use cases for
> which there was consensus in the WG. After a few iterations over the
> constructor API shape, the WG achieved consensus recently and we have sent
> an Intent to Ship for that. However, the final version of the constructor
> only became available in M126 (the last milestone of the Origin Trial) and
> we would like to give developers a little more time to migrate to the
> shipped version of the API. 2. After achieving consensus on the constructor
> with custom metadata, a new use case has been discussed in the WG that has
> revived interest in the original setMetadata() proposal. The WG has
> achieved consensus on a new API for custom codec negotiation for which
> setMetadata() looks like a better fit than the constructor since it doesn't
> require copying the payload of the encoded frame. So the WG might achieve
> consensus on adding setMetadata() after all. See the resolution of
> https://www.w3.org/2024/04/23-webrtc-minutes.html#t01 Since setMetadata()
> might be added to the spec in addition to the constructor, we would like to
> extend the trial to allow developers to continue experimenting with it.
>
>
> Ongoing technical constraints
>
> None
>
>
> Debuggability
>
> N/A
>
>
> Will this feature be supported on all six Blink platforms (Windows, Mac,
> Linux, ChromeOS, Android, and Android WebView)?Yes
>
> Is this feature fully tested by web-platform-tests
> 
> ?Yes
>
>
>