On 19 September 2017 at 04:54, Alex Davis <ada...@mozilla.com> wrote:
> Apparently we didn't land any client-side telemetry for sync until version >> 47 or so (i think?). that means that when we count users based on FxA (not >> sync) stats, we will see a longer tail of users with older versions. so >> when making this decision maybe we should pay attention to that? > > > This seems about right. Since Sync telemetry is client side, we won't have > the full history. > > @leif, I was reminded that we probably mostly care about multi-device > users. If users are on Fx <40 but don't sync between 2 devices, do we > really care? We wouldn't be providing much value to them. Any chance we can > break this down by multi-device users instead of devices? > > * We stop proactively testing on versions < X, but accept and fix reported >> bugs on that version >> * We wont fix bugs that only affect versions < Y, but don't prevent users >> from attempting to use them >> * We proactively error out when we detect a login attempt with version < Z >> > > Talking to Karlof in SF on Friday, he proposed something similar. Seems > like the second bullet would make the most sense. We stop fixing old > versions but don't prevent users from using them. This way we can be a bit > more aggressive with which version we want to support and it encourages > users to update. > One consideration here may be cases where we know it *definitely won't work*, e.g. the browser is too old to support a webchannel message that we now depend on. We should try to error out cleanly in such cases rather than showing e.g. a "working" spinner that just hangs around indefinitely. Ryan > > -- > Alex Davis // Mountain View > Product Manager // FxA & Sync > (415) 769-9247 > IRC & Slack: adavis > > On Mon, Sep 18, 2017 at 10:54 AM, Leif Oines <loi...@mozilla.com> wrote: > >> Thanks for running those numbers Mark. I've tried to recreate the same >> thing using the sync_summary_v2 derived dataset in re:dash. I suspect that >> since you're pulling raw pings perhaps we should trust those more... but I >> could be wrong. for comparison, below are the analogous numbers I'm getting >> from re:dash (https://sql.telemetry.mozilla.org/queries/38529). they are >> based on the entire sync_summary_v2 dataset for the past 7 days. >> >> version counts prop >> 48 1 0 >> 50 47,569 0.01 >> 51 53,285 0.01 >> 52 500,793 0.06 >> 53 136,974 0.02 >> 54 906,476 0.11 >> 55 5,987,420 0.76 >> 56 92,960 0.01 >> 57 19,784 0 >> >> so, according to these numbers versions 52 and up should cover 98.7% . >> that's pretty close to what Mark estimates. however I have a couple >> questions/comments: >> >> 1. Mark you said that your numbers are based on a 10% sample, yet they >> seem to be around the same order of magnitude as my numbers. as far as i >> know sync_summary_v2 is not a sampled dataset, so do we know what the >> difference there might be? >> 2. Apparently we didn't land any client-side telemetry for sync until >> version 47 or so (i think?). that means that when we count users based on >> FxA (not sync) stats, we will see a longer tail of users with older >> versions. so when making this decision maybe we should pay attention to >> that? >> >> with that in mind here are the comparable numbers for the FxA data. I >> omitted ios devices and versions with only 1 user: >> https://sql.telemetry.mozilla.org/queries/38604 >> >> version counts prop >> 40 and below 57,869.00 0.01 >> 41.00 15,443.00 0.00 >> 42.00 14,849.00 0.00 >> 43.00 37,472.00 0.00 >> 44.00 17,411.00 0.00 >> 45.00 55,595.00 0.01 >> 46.00 21,085.00 0.00 >> 47.00 108,109.00 0.01 >> 48.00 113,114.00 0.01 >> 49.00 68,451.00 0.01 >> 50.00 81,087.00 0.01 >> 51.00 82,610.00 0.01 >> 52.00 626,233.00 0.06 >> 53.00 207,903.00 0.02 >> 54.00 1,037,261.00 0.10 >> 55.00 7,741,729.00 0.74 >> 56.00 153,630.00 0.01 >> 57.00 32,955.00 0.00 >> >> Here, 52 and up covers only 93.5% of users. to cover 99% we would need to >> go back to 43. to cover ~97%, version 48. >> >> hope that helps! >> -Leif >> >> On Mon, Sep 18, 2017 at 12:55 AM, Mark Hammond <mhamm...@mozilla.com> >> wrote: >> >>> On 18/09/2017 10:20 AM, Ryan Kelly wrote: >>> > On 15 September 2017 at 10:35, Alex Davis <ada...@mozilla.com >>> > <mailto:ada...@mozilla.com>> wrote: >>> > >>> > If we were more aggressive and did 52, we'd cover 87.7% of >>> > users... which seems *too* aggressive but perhaps we can see if >>> > we can try to nudge people to upgrade first. >>> > >>> > >>> > I'd like to make a correction. It's 92.62% of users that are 52 or >>> > higher. >>> > >>> > >>> > That does seem aggressive, but we could also consider a nuanced >>> approach >>> > to phasing out support, because we have the following possibilities for >>> > what "removing support" means: >>> >>> I'm slightly skeptical of that data (although a quick look doesn't make >>> it clear there's anything wrong with it, if there actually is) - but my >>> understanding has always been that the uptake of new Firefox versions is >>> slower than we'd like, but generally successful within the 6 week cycle. >>> >>> By contrast, if I look at 10% of all desktop Firefoxes that submitted a >>> sync ping in the last 7 days I get: >>> >>> 49 - 1, 0.00 >>> 50 - 26527, 0.34 >>> 51 - 27602, 0.35 >>> 52 - 419814, 5.31 >>> 53 - 56468, 0.71 >>> 54 - 257171, 3.26 >>> 55 - 7002788, 88.63 >>> 56 - 83928, 1.06 >>> 57 - 26413, 0.33 >>> >>> so in that 10% sample, there was exactly 1 version 49. Looking at 52 >>> (current ESR) and up gives ~99.3% of all pings. >>> >>> https://gist.github.com/mhammond/46906eb3de269f3969e2fffc6dd801eb >>> >>> FWIW, going back 28 days (still a 10% sample) shows the same basic >>> pattern, although obviously 54 is higher due to it being the current >>> release for some of that period. >>> >>> 49 - 5, 0.00 >>> 50 - 109896, 0.36 >>> 51 - 117934, 0.39 >>> 52 - 1829602, 6.03 >>> 53 - 231241, 0.76 >>> 54 - 6847774, 22.58 >>> 55 - 20751445, 68.42 >>> 56 - 343482, 1.13 >>> 57 - 98950, 0.33 >>> >>> (this is the same gist as above, but ndays=28 instead of ndays=7) >>> >>> Mark >>> >>> >> >
_______________________________________________ Dev-fxacct mailing list Dev-fxacct@mozilla.org https://mail.mozilla.org/listinfo/dev-fxacct
