On Thu, Jun 7, 2012 at 11:48 AM, Konstantin Tokarev <[email protected]> wrote: > > > 07.06.2012, 21:07, "Annie Sullivan" <[email protected]>: >> Oops, forgot to reply-all. >> >> On Thu, Jun 7, 2012 at 9:05 AM, Annie Sullivan <[email protected]> wrote: >> >>> On Wed, Jun 6, 2012 at 4:59 PM, Darin Adler <[email protected]> wrote: >>>> Our past experience with this sort of thing at Apple is that it led to >>>> bugs we didn't find until after we shipped "final" software because they >>>> did not show up during earlier testing. Since then, we've gone out of our >>>> way to avoid differences, since one of the main things we want to do with >>>> non-final builds is to approximate the final releases as closely as >>>> possible to get useful testing. >>>> >>>> To oversimplify, website developers notoriously make mistakes with these >>>> types of attributes doing things like version and OS checks, leading to >>>> website incompatibilities. >>> Yes, this is definitely a concern we have as well. >>>> Maybe you could make your case for the usefulness of the attribute? >>> The problem we're experiencing in Chrome is for sites that collect >>> usage data--it turns out there's a selection bias caused by users who >>> opt in to our canary, dev, and beta channels. >>> >>> The specific case I'm looking at right now is sites collecting >>> performance data from their user base. We'd love for these sites to be >>> able to report regressions they see in Chrome's performance as early >>> as possible. But it turns out users on different channels actually >>> show different performance characteristics. Beta users seem to have >>> faster machines, for example. > > Are you sure that this differences are statistically significant?
Yes. Adam >>> So in order to compare two versions of >>> Chrome, we need to compare data from users on the same release >>> channel. So we'd like sites who collect performance information to be >>> able to collect the build type in order to do that comparison. >>> >>> We considered a few alternatives: >>> 1) Adding a marker in the user agent string that indicates the >>> channel. The problem with this is that there is a lot of very fragile >>> code in the wild which parses user agent strings and breaks at the >>> slightest change. For example, code that parses Opera 10 as Opera 1. >>> 2) Updating the version number as Chrome advances from canary to dev >>> to beta to stable, or encoding the build type in one of the octets of >>> the version number. The problem with this is that there's a big >>> possibility of sites that do version checking accidentally turning off >>> features on some channels. There are also a lot of things that we *do* >>> want to track across release channels for a specific version, like >>> bugs. So changing the version number could cause confusion there. >>> 3) Sending an "x-buildtype" header. If we do this on every request, >>> it's a lot of bloat. We can't restrict it to requests that send a >>> corresponding "x-tell-me-the-buildtype" request header because most >>> metrics collection libraries send their metrics in an image request, >>> so they can't send custom headers. >> >> _______________________________________________ >> webkit-dev mailing list >> [email protected] >> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev > > -- > Regards, > Konstantin > _______________________________________________ > webkit-dev mailing list > [email protected] > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev _______________________________________________ webkit-dev mailing list [email protected] http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
