Hi Henrik,
Just wanted to double check that the latest plan is to revert the
experiment, correct? Asking because there was somewhat different comment on
the PR here
<https://github.com/twilio/twilio-video.js/pull/1970#issuecomment-1404596109>
.
At Twilio we really hope the experiment can be reverted asap given that it
is breaking for our customers and their end users in the wild. The adoption
of our new sdk release (with the above PR change) will take long time for
most of the customers as they need to 1) learn about the issue 2) pick-up
the new sdk version 3) roll out their applications with the new sdk
version. For some customers it takes months.
Basically the only way to really stop the breaking impact is to stop the
experiment and give some notice and time to adopt the changes.
I have to acknowledge it took surprisingly long time for us to discover
this breaking change and act on it. Holidays probably played a role here.
But we will do a retrospective internally to understand the reasons better
and to catch such issues sooner in the future.
On Friday, January 27, 2023 at 12:57:38 AM UTC-8 Henrik Boström wrote:
> I'll roll it back and file an intent to deprecate thread. It was a big
> mistake that I forgot to do that in the first place, and it's unfortunate
> that we have to roll this back. But I take having reached 50% Stable and
> only knowing for sure about Twilio as a positive sign (if this was very
> widespread we would have heard about it earlier than we did). Still it
> seems like Finch is a problem for Selenium test environments in that it's
> possible to run test code for several months without noticing a feature is
> on. That's a bit sad.
>
> On Thursday, January 26, 2023 at 8:46:19 PM UTC+1 Sudheer Boynapally wrote:
>
>> Is the experiment rolled back on January 25th ? or it is still at 50%
>> stable for M 109?
>>
>> On Thu, Jan 26, 2023 at 10:47 AM Alex Russell <sligh...@chromium.org>
>> wrote:
>>
>>> It appears this change is causing breakage at scale, including for our
>>> partners.
>>>
>>> Given how this is going, I'd like to suggest the following course of
>>> action:
>>>
>>> - Turn down the Finch rollout to 0% immediately
>>> - Revert the change in 111
>>> - File an intent-to-deprecate here ASAP w/ whatever data on compat
>>> we can gather (as it looks like usecounters won't work) so we can
>>> discuss a
>>> new plan to land this
>>>
>>> We have options that might be more suitable (e.g., reverse OT), or maybe
>>> the change just needs devrel in wider channels + time to breathe. Either
>>> way, backing off of this rollout seems like the right thing to do. In
>>> future, web-visible WebRTC changes also need to come through the main Blink
>>> intents process. Let's start down that path now.
>>>
>>> Curious for other API OWNERs feedback, so cc-ing
>>> blink-api-owners-discuss@.
>>>
>>> Best,
>>>
>>> Alex
>>>
>>> On Wed, Jan 25, 2023 at 10:04 PM Henrik Boström <hb...@google.com>
>>> wrote:
>>>
>>>> The track and stream removal experiment is at 50% Stable for M109. On
>>>> January 23rd it went from 10% to 50%. The intent is to ramp up to 100%.
>>>> Which experiment group you end up with (have or not have the deprecated
>>>> stats) are chosen with a dice roll every time the user restarts their
>>>> browser.
>>>>
>>>> In M111 (which is currently Canary) the removal is enabled-by-default
>>>> so in that version there is no dependency on getting finch configs pushed
>>>> anymore.
>>>>
>>>> On Thu, Jan 26, 2023 at 12:52 AM Sudheer Boynapally <sudhe...@gmail.com>
>>>> wrote:
>>>>
>>>>> Was there any change on January 23rd corresponding to this? like
>>>>> rolling out this deprecation?
>>>>>
>>>>> Thanks,
>>>>> On Wednesday, January 25, 2023 at 2:57:16 PM UTC-8 Sudheer Boynapally
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Is the deprecation of 'track' and 'stream' objects completed 100% on
>>>>>> all the versions of chrome v109 or any specific sub version of 109?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>>
>>>>>> On Tuesday, January 10, 2023 at 5:56:04 AM UTC-8 Henrik Boström wrote:
>>>>>>
>>>>>>> On Tue, Jan 10, 2023 at 8:51 AM Yoav Weiss <yoav...@chromium.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> +Henrik Boström - was there an intent sent for this removal? Any
>>>>>>>> form of developer communication?
>>>>>>>>
>>>>>>>
>>>>>>> There was developer communication dating as far back as July but I
>>>>>>> admit I had forgotten to send out a formal blink-dev intent to
>>>>>>> deprecate!
>>>>>>>
>>>>>>> - I should have done that.
>>>>>>>
>>>>>>> The getStats() API in question is not being deprecated, but the
>>>>>>> RTCStatsReport (an id-to-stats-object map) report will stop containing
>>>>>>> the
>>>>>>> stats object which were made obsolete in the spec several years ago due
>>>>>>> to
>>>>>>> the contents of these stats objects having been moved to other stats
>>>>>>> objects that are still being returned. Same values, different location.
>>>>>>> In
>>>>>>> other words, the report is being trimmed down by removing duplicate
>>>>>>> information. Stats processing code in an application is gated on stats
>>>>>>> type
>>>>>>> for knowing which metrics to look for on an individual stats object
>>>>>>> which
>>>>>>> should make this lower risk compared to other depracations. The
>>>>>>> motivation
>>>>>>> for this is performance optimizations (~40% report size reduction),
>>>>>>> technical debt reduction (-1400 LOC) and web compat ("track" does
>>>>>>> not exist in Firefox <https://webrtc-stats.callstats.io/verify/>).
>>>>>>>
>>>>>>> The communication channel used was WebRTC's official google group,
>>>>>>> discuss-webrtc <https://groups.google.com/g/discuss-webrtc>.
>>>>>>> History:
>>>>>>>
>>>>>>> - July 25, 2022 PSA
>>>>>>> <https://groups.google.com/g/discuss-webrtc/c/lfkFCY2CFco> announced
>>>>>>> the plan to deprecate at a milestone TBD. This was also the time
>>>>>>> where the
>>>>>>> "DEPRECATED_" prefix was added to the JavaScript-exposed stats
>>>>>>> object IDs,
>>>>>>> which made it into M106. The deprecation prefix is also visible in
>>>>>>> the
>>>>>>> chrome://webrtc-internals/ developer page when a page uses WebRTC.
>>>>>>> - There was another PSA on September 6, 2022
>>>>>>> <https://groups.google.com/g/discuss-webrtc/c/mmudYReualE> about
>>>>>>> other stats news with a reminder of the imminent stats deprecation.
>>>>>>> - The October 19, 2022 PSA
>>>>>>> <https://groups.google.com/g/discuss-webrtc/c/R7Mcux9RK6g> announced
>>>>>>> "track" stats being removed at 50% Canary.
>>>>>>> - The follow-up October 27, 2022 PSA
>>>>>>> <https://groups.google.com/g/discuss-webrtc/c/9c8ZjberEcI> announced
>>>>>>> it would also be removed at 50% Beta (where M109 Beta was released
>>>>>>> on
>>>>>>> December 1st). This PSA also clarifies that "The goal is to continue
>>>>>>> ramping it up on Stable when M109 is released".
>>>>>>> - Lastly we have yesterday's PSA
>>>>>>> <https://groups.google.com/g/discuss-webrtc/c/Bxrxr8gw08Y>
>>>>>>> announcing
>>>>>>> that the removal was advanced to 1% Stable which this conversation
>>>>>>> is a
>>>>>>> response to.
>>>>>>>
>>>>>>>
>>>>>>>> On Mon, Jan 9, 2023 at 9:42 PM Alex Russell <sligh...@chromium.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Thanks for adding blink-dev, Philipp. CC-ing the API OWNERs as
>>>>>>>>> this seems related to a pattern of breaking changes without Blink
>>>>>>>>> intents
>>>>>>>>> that we've been informed of by customers.
>>>>>>>>>
>>>>>>>>> Do I understand correctly that this deprecation is being managed
>>>>>>>>> via Finch for 109 Stable?
>>>>>>>>>
>>>>>>>>
>>>>>>> Yes, as to minimize risk of breakage the deprecation is managed via
>>>>>>> a Finch flag, which is currently 1% Stable + 50% Canary/Beta.
>>>>>>> To my knowledge, no issues have been reported since the rollout
>>>>>>> started in Canary in November, 2022 or "DEPRECATED_" prefix was added
>>>>>>> in
>>>>>>> July, 2022.
>>>>>>> This, combined with the fact that apps usually gate on type, is why
>>>>>>> I thought it would be safe to gently roll out further to Stable.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>>
>>>>>>>>> Alex
>>>>>>>>>
>>>>>>>>>
>>>>>>> On Tue, Jan 10, 2023 at 12:34 AM 'Aaron Boushley' via blink-dev <
>>>>>>> blin...@chromium.org> wrote:
>>>>>>>
>>>>>>>> Can you help me understand exactly which objects are being removed
>>>>>>>> here? We rely on `RTCPeerConnection.getStats()` although we pass in a
>>>>>>>> stream selector. We then iterate over the returned stats reports
>>>>>>>> looking
>>>>>>>> for ones containing the values we need.
>>>>>>>>
>>>>>>>
>>>>>>> The selector (be it an RTCRtpSender, RTCRtpReceiver or
>>>>>>> MediaStreamTrack) continues to work, it's just that the report no
>>>>>>> longer
>>>>>>> contains the removed stats objects.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Is this a removal of the stats objects that have the fixed ID of
>>>>>>>> "track" and "stream"?
>>>>>>>>
>>>>>>>
>>>>>>> It is the removal of the stats objects where .type == "track" or
>>>>>>> .type == "stream".
>>>>>>> In the spec this refers to dictionaries RTCMediaStreamTrackStats
>>>>>>> <https://w3c.github.io/webrtc-stats/#dom-rtcmediastreamtrackstats> and
>>>>>>> RTCMediaStreamStats
>>>>>>> <https://w3c.github.io/webrtc-stats/#dom-rtcmediastreamstats> which
>>>>>>> are part of the "Obsolete" section of the spec. See RTCStatsType
>>>>>>> <https://w3c.github.io/webrtc-stats/#rtcstatstype-str*> for
>>>>>>> complete list of stats object types.
>>>>>>>
>>>>>>> Regarding the track stats dictionary, the same metrics are still
>>>>>>> available, but you have to look at the non-deprecated locations:
>>>>>>> RTCOutboundRtpStreamStats
>>>>>>> <https://w3c.github.io/webrtc-stats/#outboundrtpstats-dict*> and
>>>>>>> RTCInboundRtpStreamStats
>>>>>>> <https://w3c.github.io/webrtc-stats/#dom-rtcinboundrtpstreamstats>
>>>>>>> dictionaries
>>>>>>> instead (type == "outbound-rtp" and type == "inbound-rtp"). See also
>>>>>>> the
>>>>>>> type "media-source" referenced from outbound-rtp.mediaSourceId.
>>>>>>>
>>>>>>>
>>>>>>>> Is there any more documentation I can look at beyond the 2
>>>>>>>> sentences above?
>>>>>>>>
>>>>>>>
>>>>>>> The spec <https://w3c.github.io/webrtc-stats/> as well as the
>>>>>>> implementation
>>>>>>> <https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/api/stats/rtcstats_objects.h>
>>>>>>> are
>>>>>>> good resources in general.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Aaron
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>> On Mon, Jan 9, 2023 at 11:30 AM 'Philipp Hancke' via blink-dev <
>>>>>>>>> blin...@chromium.org> wrote:
>>>>>>>>>
>>>>>>>>>> Henrik,
>>>>>>>>>>
>>>>>>>>>> while I understand that this has been ongoing for a while in
>>>>>>>>>> Canary
>>>>>>>>>> <https://groups.google.com/g/discuss-webrtc/c/R7Mcux9RK6g/m/KG8qe6AEBgAJ>
>>>>>>>>>>
>>>>>>>>>> and Beta
>>>>>>>>>> <https://groups.google.com/g/discuss-webrtc/c/9c8ZjberEcI/m/wbdQr77VBQAJ>
>>>>>>>>>> for
>>>>>>>>>> a while without anyone raising issues,
>>>>>>>>>> the timeline "tomorrow" (with ramp-up) comes as a surprise, in
>>>>>>>>>> particular considering practices like code freezes over the holidays.
>>>>>>>>>>
>>>>>>>>>> Can you explain why you think you don't need an intent to
>>>>>>>>>> deprecate on blink-dev (cc'd)?
>>>>>>>>>> (I agree they should go away but don't see a need for such an
>>>>>>>>>> aggressive timeline)
>>>>>>>>>>
>>>>>>>>>> Am Mo., 9. Jan. 2023 um 10:40 Uhr schrieb Henrik Boström <
>>>>>>>>>> hb...@webrtc.org>:
>>>>>>>>>>
>>>>>>>>>>> With the Stable release of M109 tomorrow, the deprecated "track"
>>>>>>>>>>> and "stream" stats objects returned by RTCPeerConnection.getStats()
>>>>>>>>>>> will no
>>>>>>>>>>> longer be available.
>>>>>>>>>>>
>>>>>>>>>>> This is unshipped at 1% Stable, but it will soon ramp up further
>>>>>>>>>>> with the goal of 100% unshipped in M109.
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>>
>>>>>>>>>>> ---
>>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>>> Google Groups "discuss-webrtc" group.
>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from
>>>>>>>>>>> it, send an email to discuss-webrt...@googlegroups.com.
>>>>>>>>>>> To view this discussion on the web visit
>>>>>>>>>>> https://groups.google.com/d/msgid/discuss-webrtc/0ebcadca-6d66-4f7c-b7e5-d2ac707a6e84n%40googlegroups.com
>>>>>>>>>>>
>>>>>>>>>>> <https://groups.google.com/d/msgid/discuss-webrtc/0ebcadca-6d66-4f7c-b7e5-d2ac707a6e84n%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>>>>>>>> .
>>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>> Google Groups "blink-dev" group.
>>>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>>>> send an email to blink-dev+...@chromium.org.
>>>>>>>>>> To view this discussion on the web visit
>>>>>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADxkKiJ-vkiyVTH%2BsSNOjv67zx0p7qLra1ePdGJS3JW3GTzfKQ%40mail.gmail.com
>>>>>>>>>>
>>>>>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADxkKiJ-vkiyVTH%2BsSNOjv67zx0p7qLra1ePdGJS3JW3GTzfKQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>>>>>>> .
>>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>>> Groups "blink-api-owners" group.
>>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>>> send an email to blink-api-owne...@chromium.org.
>>>>>>>>> To view this discussion on the web visit
>>>>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners/CAA44PQjegbaaorsT_r-iNtUAg2Gxa5f6WohHAaLeGw2tmBW%2BwA%40mail.gmail.com
>>>>>>>>>
>>>>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners/CAA44PQjegbaaorsT_r-iNtUAg2Gxa5f6WohHAaLeGw2tmBW%2BwA%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>>>>>> .
>>>>>>>>>
>>>>>>>>
>>
>> --
>>
>> Thanks,
>> Sudheer
>>
>
--
You received this message because you are subscribed to the Google Groups
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/a38bdad4-730c-4b2d-ad6e-12a5b7cef1c6n%40chromium.org.
