LGTM3 On Fri, Feb 10, 2023 at 6:31 PM Rick Byers <rby...@chromium.org> wrote:
> Sorry missed the link, details at > https://www.chromium.org/developers/enterprise-changes/ > > Rick > > On Fri, Feb 10, 2023 at 11:23 AM Rick Byers <rby...@chromium.org> wrote: > >> +1 to Philip's sentiment, thanks for putting the effort into this! Given >> that it got to 50% stable and this was the only issue reported and Twilio >> is happy with the plan, then I'm also reluctant to ask for more opt-outs. >> So LGTM2 >> >> But please circle back here if you hear of any other non-trivial >> breakage. If necessary we can postpone again with finch and add an >> enterprise opt-out. +1 to what Johnny says about enterprise policies being >> the only effective knob for some environments - we've heard of many cases >> where updating a software package is a multi-month process for enterprise >> customers of a software system, but where asking those customers to enable >> a policy in their fleet is relatively easy. It's generally not about >> software developed by the enterprise itself, but bought from a 3p who has a >> support contract. More details on the policy and purpose of it are here. >> >> Rick >> >> On Wed, Feb 8, 2023 at 5:54 PM Philip Jägenstedt <foo...@chromium.org> >> wrote: >> >>> LGTM1 for the plan to remove in 112 with reverse OT until 115. >>> >>> We discussed in today's meeting whether we should also have an >>> enterprise policy, but landed on not requesting it. The reason is that it's >>> quite possible that this change only breaks Twilio and nothing else, at >>> all. That's because of the shape of the API and that the change got to 50% >>> stable in M109 without other breakage being reported. An enterprise policy >>> would be the "belts and braces" approach, but there's also a cost to making >>> changes like these more laborious that has to be weighed against the risk. >>> >>> Removing things from the web platform is usually thankless work, but it >>> is important to reduce complexity and converge with other browsers >>> (interoperability) so I want to applaud the effort put into this 👏 >>> >>> On Wed, Feb 8, 2023 at 6:45 PM Johnny Stenback <jstenb...@chromium.org> >>> wrote: >>> >>>> Deprecation trials require changes to servers which an enterprise may >>>> not have access to or ability to affect changes to, whereas enterprise >>>> policies are entirely within the enterprise's control. I'd recommend >>>> reaching out to the enterprise team for their perspective on usage policies >>>> and perspective on whether a policy should be included here or not (I'm >>>> still of the opinion that it's in our interest to include one). >>>> >>>> Thanks, >>>> Johnny >>>> >>>> On Tue, Feb 7, 2023 at 2:36 AM Henrik Boström <h...@chromium.org> >>>> wrote: >>>> >>>>> I have the same concern as Harald regarding corporate policies. Why >>>>> not a Deprecation Origin Trial in that case for list of users and more >>>>> concrete timeline? >>>>> >>>>> On Tuesday, February 7, 2023 at 9:10:40 AM UTC+1 Harald Alvestrand >>>>> wrote: >>>>> >>>>>> Do we have trackable statistics on the usage of corporate policies? >>>>>> ie if nobody uses the policy in 2 milestones, can we detect that and >>>>>> decide that it is not needed and delete it, or will we be as unsure as we >>>>>> are now? >>>>>> >>>>>> >>>>>> On Mon, Feb 6, 2023 at 9:41 PM Mike Taylor <miketa...@chromium.org> >>>>>> wrote: >>>>>> >>>>>>> We don't know what we don't know, but it's not hard to imagine an >>>>>>> in-house enterprise WebRTC application that is using "stats" or "track". >>>>>>> Twilio is the breakage we know about (because a developer took the time >>>>>>> to >>>>>>> report a bug). Having a policy so an app continues to work while a fix >>>>>>> is >>>>>>> made is a good thing - and comes with the nice side effect of appearing >>>>>>> on >>>>>>> the Enterprise release notes, increasing awareness. >>>>>>> >>>>>>> On 2/4/23 3:28 AM, Harald Alvestrand wrote: >>>>>>> >>>>>>> What's the imagined scenario in which an enterprise policy would be >>>>>>> useful? >>>>>>> >>>>>>> The only place I could imagine it being relevant is if there exists >>>>>>> a WebRTC application that is only used within a single enterprise >>>>>>> (neither >>>>>>> hosting nor usage exists outside the enterprise), and that WebRTC >>>>>>> application depends on non-upgraded Twilio libraries. >>>>>>> >>>>>>> I don't know that we have evidence that such applications exist. >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, Feb 3, 2023 at 6:14 PM Mike Taylor <miketa...@chromium.org> >>>>>>> wrote: >>>>>>> >>>>>>>> I agree with Johnny that an enterprise policy would be useful, at >>>>>>>> least for a few milestones. >>>>>>>> >>>>>>>> On 1/30/23 5:16 AM, 'Harald Alvestrand' via blink-dev wrote: >>>>>>>> >>>>>>>> I'm not sure an enterprise policy is appropriate - I see the same >>>>>>>> problem with sunsetting the policy as with sunsetting the stat in >>>>>>>> general, >>>>>>>> and usage of enterprise policies is (as far as I know) far more opaque >>>>>>>> to >>>>>>>> us than origin trials or Finch feature usage. >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Jan 30, 2023 at 11:13 AM Henrik Boström <h...@chromium.org> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Friday, January 27, 2023 at 7:24:58 PM UTC+1 Johnny Stenback >>>>>>>>> wrote: >>>>>>>>> Is there an enterprise policy in place for this deprecation >>>>>>>>> already? If not, adding one seems appropriate given the challenges of >>>>>>>>> rolling out even simple fixes in some enterprise environments. >>>>>>>>> >>>>>>>>> One does not exist at the moment but I can add one >>>>>>>>> <https://chromium.googlesource.com/chromium/src/+/HEAD/docs/enterprise/add_new_policy.md> >>>>>>>>> . >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Johnny >>>>>>>>> >>>>>>>>> On Fri, Jan 27, 2023 at 5:16 AM Henrik Boström <h...@chromium.org> >>>>>>>>> wrote: >>>>>>>>> Delaying the enabled-by-default to M112 is fine by me but I'll >>>>>>>>> wait for a resolution here before taking action. Currently it is >>>>>>>>> enabled-by-default in Canary. >>>>>>>>> >>>>>>>>> On Friday, January 27, 2023 at 12:41:23 PM UTC+1 >>>>>>>>> philipp...@googlemail.com wrote: >>>>>>>>> Am Fr., 27. Jan. 2023 um 11:49 Uhr schrieb Henrik Boström < >>>>>>>>> h...@chromium.org>: >>>>>>>>> *Contact emails* >>>>>>>>> h...@chromium.org, h...@chromium.org >>>>>>>>> >>>>>>>>> *Background* >>>>>>>>> I attempted to remove this feature before but had forgotten to >>>>>>>>> file an intent to deprecate, background here >>>>>>>>> <https://groups.google.com/a/chromium.org/g/blink-dev/c/RsIktnGhHqw/> >>>>>>>>> . >>>>>>>>> >>>>>>>>> *Specification* >>>>>>>>> The getStats() API spec is here >>>>>>>>> <https://w3c.github.io/webrtc-stats/> and it contains all the >>>>>>>>> metrics. The deprecated metrics are also listed, but in the obsolete >>>>>>>>> section >>>>>>>>> <https://w3c.github.io/webrtc-stats/#obsolete-rtcmediastreamtrackstats-members>. >>>>>>>>> There's an open issue to remove obsolete metrics from the spec as >>>>>>>>> soon as >>>>>>>>> they are unshipped from modern browsers. This is considered a blocker >>>>>>>>> for >>>>>>>>> the document to reach Proposed Recommendation status. >>>>>>>>> >>>>>>>>> *Summary* >>>>>>>>> WebRTC is a set of JavaScript APIs (spec >>>>>>>>> <https://w3c.github.io/webrtc-pc/>) that allow real-time >>>>>>>>> communication between browsers. For the relevant metrics being >>>>>>>>> removed, >>>>>>>>> we're only talking about the WebRTC use case that is sending or >>>>>>>>> receiving >>>>>>>>> audio or video (typically Video Conferencing use cases), not the data >>>>>>>>> channel use cases that is a popular WebRTC use case, since data >>>>>>>>> channel >>>>>>>>> only use cases would never have any tracks/streams. >>>>>>>>> >>>>>>>>> RTCPeerConnection.getStats() returns a map of string-to-objects, >>>>>>>>> where each object is one of the dictionaries defined in the stats >>>>>>>>> spec. The >>>>>>>>> reason an app calls getStats() is mostly to report quality metrics >>>>>>>>> (send >>>>>>>>> and receive resolutions, bitrates, glitches, video QP, etc) which can >>>>>>>>> be >>>>>>>>> important for A/B experimentation. It can also be used in a way that >>>>>>>>> impacts app logic or even UX inside the app. Most common use case I >>>>>>>>> can >>>>>>>>> think of: poll getStats() at 10 Hz and render volume bars for each >>>>>>>>> participant based on volume levels from stats objects. >>>>>>>>> >>>>>>>>> The deprecation in question is to remove some stats objects that >>>>>>>>> were made obsolete several years ago: all metrics on the "track" >>>>>>>>> dictionary >>>>>>>>> have been moved to non-obsolete objects ("inbound-rtp", >>>>>>>>> "outbound-rtp", >>>>>>>>> "media-source"). Reasons for wanting to deprecate include: >>>>>>>>> >>>>>>>>> - Spec-compliance: needed for browser implementations to align >>>>>>>>> and for the spec to become Proposed Recommendation. >>>>>>>>> - Web compat: Firefox never implement "track" or "steam" >>>>>>>>> >>>>>>>>> <https://wpt.fyi/results/webrtc-stats/supported-stats.https.html?label=experimental&label=master&aligned> >>>>>>>>> due >>>>>>>>> to them being obsolete. >>>>>>>>> - Performance: the duplicated metrics make up ~40% of the >>>>>>>>> stats report size, which can be a significant number of bytes in >>>>>>>>> larger >>>>>>>>> meetings and it is common for apps to poll getStats() 10 times per >>>>>>>>> second. >>>>>>>>> - Tech debt: unblock removal of 1400 LOC. >>>>>>>>> >>>>>>>>> In the meantime, the obsolete metrics is duplicated in several >>>>>>>>> places of the stats report. >>>>>>>>> >>>>>>>>> *Risks* >>>>>>>>> *- Impossible to properly measure usage* >>>>>>>>> Because stats objects are exposed as JavaScript dictionaries, and >>>>>>>>> because apps have to iterate all objects of the stats report in order >>>>>>>>> to >>>>>>>>> find the ones they are interested in or if they just dump all the data >>>>>>>>> without filtering, there is no way to measure how big the dependency >>>>>>>>> is on >>>>>>>>> track in the real world. >>>>>>>>> >>>>>>>>> While we lack use counters, we have some positive signs: >>>>>>>>> >>>>>>>>> - Because Firefox does not have "track" or "stream" stats, any >>>>>>>>> app that can run on Firefox already exercises the paths of these >>>>>>>>> not >>>>>>>>> existing. >>>>>>>>> >>>>>>>>> >>>>>>>>> - An experiment to "unship deprecated metrics" has been >>>>>>>>> running at 50% Canary since October >>>>>>>>> >>>>>>>>> <https://groups.google.com/a/chromium.org/g/blink-dev/c/RsIktnGhHqw/m/3iqjODsMBwAJ>, >>>>>>>>> giving developers testing Canary a heads-up. Nobody complained >>>>>>>>> until the >>>>>>>>> experiment reached Stable. >>>>>>>>> - We got to 50% Stable in M109 and while we're in the process >>>>>>>>> of rolling it back now due to breaking twilio-video.js >>>>>>>>> <https://github.com/twilio/twilio-video.js/issues/1968>, it's >>>>>>>>> interesting to note that this is the only breakage we are aware of >>>>>>>>> (that >>>>>>>>> does not mean there aren't more breakages, but I believe this at >>>>>>>>> least says >>>>>>>>> something about the severity). >>>>>>>>> >>>>>>>>> *- Selenium et al typically starts browsers from fresh profiles >>>>>>>>> and hence does not know the finch trial seed* >>>>>>>>> The most likely explanation for breakage is not testing Canary or >>>>>>>>> test environments not having access to Finch experiments. This makes >>>>>>>>> the >>>>>>>>> behavior on Stable a surprise. >>>>>>>>> >>>>>>>>> *- To have a Reverse Origin Trial or not to have a Reverse Origin >>>>>>>>> Trial?* >>>>>>>>> Migrating should require so few lines of code (look for stats.type >>>>>>>>> == 'inbound-rtp' instead of stats.type == 'track', for example) that >>>>>>>>> it >>>>>>>>> seems to be a bigger hurdle for a developer to enroll in the trial >>>>>>>>> than to >>>>>>>>> simply fix their code. >>>>>>>>> >>>>>>>>> *- Compatiblity risk* >>>>>>>>> There is one particular metric out of all metrics that, if you run >>>>>>>>> Safari, does not exist on "inbound-rtp" yet. This can be a problem, >>>>>>>>> but >>>>>>>>> again is probably not a big problem because this particular metric was >>>>>>>>> never implemented on Firefox so apps already need to survive without >>>>>>>>> it, >>>>>>>>> and it is very easy to write a fallback path for the Safari case: >>>>>>>>> >>>>>>>>> let trackIdentifer = null; // In Firefox this will never be set >>>>>>>>> regardless. >>>>>>>>> if (inboundRtp.trackIdentifier) { >>>>>>>>> // Spec-compliant browser. >>>>>>>>> trackIdentifier = inboundRtp.trackIdentifier; >>>>>>>>> } else if (inboundRtp.trackId) { >>>>>>>>> // Fallback-path for Safari or 1+ year old Chromium browsers. >>>>>>>>> trackIdentifier = report.get(inboundRtp.trackId) >>>>>>>>> .trackIdentifier; >>>>>>>>> } >>>>>>>>> >>>>>>>>> *Proposal* >>>>>>>>> Rollback to 0% Stable but keep the "unship deprecated" experiment >>>>>>>>> at 50% Canary/Beta. Wait for Twilio to fix their issue and do another >>>>>>>>> rollout attempt. Keep a slower rollout pace next time. >>>>>>>>> >>>>>>>>> I see limited amount of value in a Reverse Origin Trial since it >>>>>>>>> appears to be more effort to register to the trial than to fix the >>>>>>>>> issue, >>>>>>>>> if you are affected. >>>>>>>>> >>>>>>>>> I do prefer to have the feature enabled-by-default in M111+ and >>>>>>>>> overwrite that default via Finch rather than the other way around as >>>>>>>>> to not >>>>>>>>> "turn off the fire alarm" for non-Finch testing environments. I >>>>>>>>> realize >>>>>>>>> that is not perfect (what if you run in a non-Finch environment) but >>>>>>>>> it >>>>>>>>> would reduce overall risk. >>>>>>>>> >>>>>>>>> Thank you Henrik. I agree with one suggestion: only do default-off >>>>>>>>> in M112+ (which is branching so you would just need to revert this >>>>>>>>> commit >>>>>>>>> <https://chromiumdash.appspot.com/commit/3a4d52f365df03413a856ea20366b36e8fb8ea0b> >>>>>>>>> on >>>>>>>>> the M111 branch). >>>>>>>>> This gives developers another month to update (which itself should >>>>>>>>> be quick) and then rolling it out to their customers and users (which >>>>>>>>> takes >>>>>>>>> time). >>>>>>>>> >>>>>>>>> I hope that the 50% rollout caused enough incidents (even though >>>>>>>>> you may never hear about some of them) to get the fixes in ASAP. >>>>>>>>> >>>>>>>>> cheers >>>>>>>>> >>>>>>>>> Philipp >>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "blink-dev" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to blink-dev+unsubscr...@chromium.org. >>>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink- >>>>>>>>> dev/5ecf1ea6-c16c-464a-b529-439e05e4feedn%40chromium.org >>>>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/5ecf1ea6-c16c-464a-b529-439e05e4feedn%40chromium.org?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "blink-dev" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to blink-dev+unsubscr...@chromium.org. >>>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink- >>>>>>>>> dev/6a0b5bb9-addd-4a56-b053-1429bbaabe2dn%40chromium.org >>>>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/6a0b5bb9-addd-4a56-b053-1429bbaabe2dn%40chromium.org?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "blink-dev" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to blink-dev+unsubscr...@chromium.org. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOqqYVG_P0ji6Z-n0DoCua4nzJZN7S48o23%3DUpx_ELerqphfUg%40mail.gmail.com >>>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOqqYVG_P0ji6Z-n0DoCua4nzJZN7S48o23%3DUpx_ELerqphfUg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "blink-dev" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to blink-dev+unsubscr...@chromium.org. >>>> To view this discussion on the web visit >>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACZRgz4SF%2BGDjoD8S%3DfYHgLYX6PxeTUj4hd%3DA1%2BS%2BeBD2ygtpQ%40mail.gmail.com >>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACZRgz4SF%2BGDjoD8S%3DfYHgLYX6PxeTUj4hd%3DA1%2BS%2BeBD2ygtpQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "blink-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to blink-dev+unsubscr...@chromium.org. >>> To view this discussion on the web visit >>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAARdPYcQdCft3PrYCEs93V35pEb%3DZy6%2BjfYNwOAojNPYdtzhKw%40mail.gmail.com >>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAARdPYcQdCft3PrYCEs93V35pEb%3DZy6%2BjfYNwOAojNPYdtzhKw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAL5BFfW7BWW2%3D-xQLOAEnRSuGM6edgUoTBg%3DHMztc6%3DMcZJoYA%40mail.gmail.com.
