Thanks for doing that work, Daniel! 0.015% effective breakage is way better than 0.25%, but it's still ~5x higher than what we're typically comfortable with. I'm wondering if folks have creative ideas on the outreach front - +Andre Bandarra <andre...@google.com> in particular
Otherwise, maybe it makes sense to finch this at 50% on Beta, Dev and Canary channels, to convince folks this is indeed coming? On Fri, Oct 21, 2022 at 1:40 PM Daniel Vogelheim <vogelh...@google.com> wrote: > On Wed, Oct 19, 2022 at 5:23 AM Yoav Weiss <yoavwe...@chromium.org> wrote: > >> Thanks for the detailed report!! >> >> It's great that we've managed to bring the usage down, but 0.25% is still >> too high for my comfort levels. >> Taking a manual survey of the major users seems like the right approach. >> I wonder if you could, on top of the top sites, also run a random survey of >> the bottom half of usage, to get a sense of breakage there? >> > > The long tail is long. :) Chromestatus offers a "Sample URLs" table for > each feature, so I took the top 50 sample URLs for > CrossOriginAccessBasedOnDocumentDomain > <https://chromestatus.com/metrics/feature/timeline/popularity/4171> [1] > and examined them manually, with & without Origin-Agent-Cluster on by > default. > > - 47 sites worked without any obvious problems. I usually examined the > main site and one page linked from the main site. > - 3 sites did not. Interestingly, one of them was another country domain > of the site I reported on in the "top 9" cases; and the other two were > different country domains of the same site. I guess one can now argue > whether I found 3 or only 2 sites that break. [2] > - If I assume Chromestatus URL sampling is vaguely proportional to page > views, then: 0.25% page views use the feature, 3 / 50 with visible issues > => 0.015% potential of problem page views. > > > [1] I'm not sure what their sampling method is; and in particular whether > it's stable and everyone gets the same list, or whether the random sample > is random every time. If it's relevant, I can provide the list of URLs I > used. > [2] I'm not sure if listing the sites publicly is desired, or even > permissible. One is a commercial site focused on sports results; the other > a non-commercial site focused on onscreen keyboards for different languages. > > >> On Mon, Oct 17, 2022 at 4:39 PM Daniel Vogelheim <vogelh...@google.com> >> wrote: >> >>> Hello all, >>> >>> It's been a while and 109 is coming up. As I'm preparing the >>> intent-to-ship for 109, I'd like to post an update on how the deprecation >>> is going: >>> >>> Current usage: Since announcing the deprecation, usage of >>> document.domain-enabled accesses have dropped by about 50%. >>> >>> - Feature stats: DocumentDomainEnabledCrossOriginAccess >>> <https://chromestatus.com/metrics/feature/timeline/popularity/2544> >>> >>> - Note that this *includes* usage when an Origin-Agent-Cluster header is >>> explicitly set, which is sustainable use that is not affected by the >>> deprecation. >>> >>> - CrossOriginAccessBasedOnDocumentDomain >>> <https://chromestatus.com/metrics/feature/timeline/popularity/4171> is >>> usage of document.domain enabled access, but only when based on the >>> Origin-Agent-Cluster's default (which is what this intent wants to change.) >>> This graph has the correct numbers for this intent; but makes long-term >>> trends harder to see because we only introduced the use counter *during* >>> the deprecation period. >>> >>> - So basically, usage has dropped form ~0.5% of page views ( >>> DocumentDomainEnabledCrossOriginAccess >>> <https://chromestatus.com/metrics/feature/timeline/popularity/2544> @ >>> Nov '21) to about ~0.25% of page views ( >>> CrossOriginAccessBasedOnDocumentDomain >>> <https://chromestatus.com/metrics/feature/timeline/popularity/4171> @ >>> Sept '22) >>> >>> When gathering the data for this post, I double-checked on a particular, >>> well-known media site that we had contacted about the deprecation during >>> the past months. I was surprised to notice that despite our outreach and >>> communication, they *still* use document.domain and document.domain >>> facilitated cross-origin access. But when taking a closer look, an >>> interesting find emerged: They are using document.domain setting to enable >>> auto-play of their media player, which is hosted on a separate domain. Our >>> advice was to use the 'autoplay' permission policy with permission >>> delegation instead. They are indeed doing so, but *in addition* to >>> document.domain setting. In other words, they opted for a conservative >>> implementation strategy where they auto-play their frame with two different >>> methods. When I load their page with document.domain setting disabled, it >>> works fine. That's a fine implementation strategy, but unfortunately it >>> mucks up our statistics since our use counters cannot know whether other >>> code exists to compensate for a failed document.domain facilitated access. >>> >>> When discussing this finding with another engineer, he suggested that >>> we're really interested in user-visible web breakage. Since I don't know >>> how to measure that directly, I manually looked at all top users of >>> document.domain and loaded each page with/without document.domain setting >>> to see if I could spot any difference. Document.domain usage - like the web >>> in general - is quite "top heavy": 9 sites account for about 50% of all >>> remaining dd usage. >>> >>> - 7 sites work without any discernible difference. (Caveat: Many use >>> languages I do not understand, which makes it difficult to spot subtle >>> differences in content. But to me, the sites looked and used the same, >>> regardless of document.domain setting. Caveat 2: One site requires a login, >>> so I could only really test the login page rather than their core >>> functionality.) >>> >>> - 1 site worked just the same, except for a pair of very extra fancy ad >>> frames that "framed" the main content left and right. The main content, >>> including in-page ads, seemed just fine, but the fancy ad frames were >>> missing. >>> >>> - 1 site was clearly missing content. >>> >>> For both of the last two, the console showed uncaught DOM exceptions for >>> a failed cross-domain access. What I suspect happens in the first case is >>> that during construction of the fancy ad frames an exception is thrown and >>> hence the frames aren't inserted in the page. In the second case something >>> similar happens, but when building up the main content. Or maybe before >>> building up the main content. Thus, that part of the main content is >>> missing. >>> >>> (We don't like broken web pages, so we reached out separately to the >>> owners of that last page on Friday. Their support has promised to put us in >>> contact with one of their developers which, as of this writing, hasn't >>> happened yet.) >>> >>> >>> On Fri, Jan 21, 2022 at 9:23 PM Yoav Weiss <yoavwe...@chromium.org> >>> wrote: >>> >>>> LGTM1 to deprecate under the following conditions: >>>> >>>> - As discussed, a 6 months deprecation period, as well as >>>> broad-scope and targeted outreach, that would hopefully bring usage >>>> down. >>>> - A well-crafted deprecation message that indicates the timeline, >>>> and at the same time indicates that we'll be responsive to community >>>> feedback (or a link to a blog post/documentation page that indicates >>>> the same) >>>> - Sending a separate intent for the actual removal at the end of >>>> the deprecation period, once the picture is a bit clearer. >>>> >>>> -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAL5BFfWK0vqSWrRsW_Fr_iW-1omFMsWSvExYwskLMd%2B1y%3DGnLw%40mail.gmail.com.
