here's the same FxA-based table as above, but only for multi-device users. pretty similar - if anything, these people seem to be slightly more outdated https://sql.telemetry.mozilla.org/queries/38604
version count prop 40 and below 16,696.00 0.01 41.00 3,372.00 0.00 42.00 3,802.00 0.00 43.00 9,763.00 0.00 44.00 4,082.00 0.00 45.00 19,436.00 0.01 46.00 4,758.00 0.00 47.00 25,820.00 0.01 48.00 21,457.00 0.01 49.00 16,159.00 0.01 50.00 25,007.00 0.01 51.00 22,702.00 0.01 52.00 156,080.00 0.06 53.00 54,776.00 0.02 54.00 293,116.00 0.12 55.00 1,743,521.00 0.69 56.00 68,620.00 0.03 57.00 20,506.00 0.01 On Mon, Sep 18, 2017 at 11:54 AM, Alex Davis <ada...@mozilla.com> wrote: > Apparently we didn't land any client-side telemetry for sync until version >> 47 or so (i think?). that means that when we count users based on FxA (not >> sync) stats, we will see a longer tail of users with older versions. so >> when making this decision maybe we should pay attention to that? > > > This seems about right. Since Sync telemetry is client side, we won't have > the full history. > > @leif, I was reminded that we probably mostly care about multi-device > users. If users are on Fx <40 but don't sync between 2 devices, do we > really care? We wouldn't be providing much value to them. Any chance we can > break this down by multi-device users instead of devices? > > * We stop proactively testing on versions < X, but accept and fix reported >> bugs on that version >> * We wont fix bugs that only affect versions < Y, but don't prevent users >> from attempting to use them >> * We proactively error out when we detect a login attempt with version < Z >> > > Talking to Karlof in SF on Friday, he proposed something similar. Seems > like the second bullet would make the most sense. We stop fixing old > versions but don't prevent users from using them. This way we can be a bit > more aggressive with which version we want to support and it encourages > users to update. > > > -- > Alex Davis // Mountain View > Product Manager // FxA & Sync > (415) 769-9247 > IRC & Slack: adavis > > On Mon, Sep 18, 2017 at 10:54 AM, Leif Oines <loi...@mozilla.com> wrote: > >> Thanks for running those numbers Mark. I've tried to recreate the same >> thing using the sync_summary_v2 derived dataset in re:dash. I suspect that >> since you're pulling raw pings perhaps we should trust those more... but I >> could be wrong. for comparison, below are the analogous numbers I'm getting >> from re:dash (https://sql.telemetry.mozilla.org/queries/38529). they are >> based on the entire sync_summary_v2 dataset for the past 7 days. >> >> version counts prop >> 48 1 0 >> 50 47,569 0.01 >> 51 53,285 0.01 >> 52 500,793 0.06 >> 53 136,974 0.02 >> 54 906,476 0.11 >> 55 5,987,420 0.76 >> 56 92,960 0.01 >> 57 19,784 0 >> >> so, according to these numbers versions 52 and up should cover 98.7% . >> that's pretty close to what Mark estimates. however I have a couple >> questions/comments: >> >> 1. Mark you said that your numbers are based on a 10% sample, yet they >> seem to be around the same order of magnitude as my numbers. as far as i >> know sync_summary_v2 is not a sampled dataset, so do we know what the >> difference there might be? >> 2. Apparently we didn't land any client-side telemetry for sync until >> version 47 or so (i think?). that means that when we count users based on >> FxA (not sync) stats, we will see a longer tail of users with older >> versions. so when making this decision maybe we should pay attention to >> that? >> >> with that in mind here are the comparable numbers for the FxA data. I >> omitted ios devices and versions with only 1 user: >> https://sql.telemetry.mozilla.org/queries/38604 >> >> version counts prop >> 40 and below 57,869.00 0.01 >> 41.00 15,443.00 0.00 >> 42.00 14,849.00 0.00 >> 43.00 37,472.00 0.00 >> 44.00 17,411.00 0.00 >> 45.00 55,595.00 0.01 >> 46.00 21,085.00 0.00 >> 47.00 108,109.00 0.01 >> 48.00 113,114.00 0.01 >> 49.00 68,451.00 0.01 >> 50.00 81,087.00 0.01 >> 51.00 82,610.00 0.01 >> 52.00 626,233.00 0.06 >> 53.00 207,903.00 0.02 >> 54.00 1,037,261.00 0.10 >> 55.00 7,741,729.00 0.74 >> 56.00 153,630.00 0.01 >> 57.00 32,955.00 0.00 >> >> Here, 52 and up covers only 93.5% of users. to cover 99% we would need to >> go back to 43. to cover ~97%, version 48. >> >> hope that helps! >> -Leif >> >> On Mon, Sep 18, 2017 at 12:55 AM, Mark Hammond <mhamm...@mozilla.com> >> wrote: >> >>> On 18/09/2017 10:20 AM, Ryan Kelly wrote: >>> > On 15 September 2017 at 10:35, Alex Davis <ada...@mozilla.com >>> > <mailto:ada...@mozilla.com>> wrote: >>> > >>> > If we were more aggressive and did 52, we'd cover 87.7% of >>> > users... which seems *too* aggressive but perhaps we can see if >>> > we can try to nudge people to upgrade first. >>> > >>> > >>> > I'd like to make a correction. It's 92.62% of users that are 52 or >>> > higher. >>> > >>> > >>> > That does seem aggressive, but we could also consider a nuanced >>> approach >>> > to phasing out support, because we have the following possibilities for >>> > what "removing support" means: >>> >>> I'm slightly skeptical of that data (although a quick look doesn't make >>> it clear there's anything wrong with it, if there actually is) - but my >>> understanding has always been that the uptake of new Firefox versions is >>> slower than we'd like, but generally successful within the 6 week cycle. >>> >>> By contrast, if I look at 10% of all desktop Firefoxes that submitted a >>> sync ping in the last 7 days I get: >>> >>> 49 - 1, 0.00 >>> 50 - 26527, 0.34 >>> 51 - 27602, 0.35 >>> 52 - 419814, 5.31 >>> 53 - 56468, 0.71 >>> 54 - 257171, 3.26 >>> 55 - 7002788, 88.63 >>> 56 - 83928, 1.06 >>> 57 - 26413, 0.33 >>> >>> so in that 10% sample, there was exactly 1 version 49. Looking at 52 >>> (current ESR) and up gives ~99.3% of all pings. >>> >>> https://gist.github.com/mhammond/46906eb3de269f3969e2fffc6dd801eb >>> >>> FWIW, going back 28 days (still a 10% sample) shows the same basic >>> pattern, although obviously 54 is higher due to it being the current >>> release for some of that period. >>> >>> 49 - 5, 0.00 >>> 50 - 109896, 0.36 >>> 51 - 117934, 0.39 >>> 52 - 1829602, 6.03 >>> 53 - 231241, 0.76 >>> 54 - 6847774, 22.58 >>> 55 - 20751445, 68.42 >>> 56 - 343482, 1.13 >>> 57 - 98950, 0.33 >>> >>> (this is the same gist as above, but ndays=28 instead of ndays=7) >>> >>> Mark >>> >>> >> >
_______________________________________________ Sync-dev mailing list Sync-dev@mozilla.org https://mail.mozilla.org/listinfo/sync-dev
