Isn't this what Aurora (Alpha) builds are for?
It seems to me what you need is some way to communicate with users:
* Publish which features are new, what you are planning, and what kind
of testing and feedback you are looking for. -- Where should users look?
* A simple way for end users to provide this feedback (not a subscriber
mailing list!), which also allows some free text (not just checkboxen),
and (optionally) also allows you to ask the reporter back and engage in
an email exchange. -- Let users be heard
* Some outreach to let people know you want them to test this, and where
to find the news from point 1. An easy way would be articles on tech
news websites. They'd happily carry a story like "Firefox Android is
looking for feedback", and some of the users would happily engage. Beta
is fun! for them.
It seems to me that particularly points 1 and 2 are missing. At least I
wouldn't know where they are (aside from this mailing list).
Ben
Martyn Haigh wrote on 10.07.2015 18:33:
Hey people,
At the start of the year I wrote a bit of functionality called Tab
Queues which stemmed from a personal desire to do something different
in how people use Firefox. I spent a lot of time learning the codebase
(it was the biggest bit of work I've done on Fennec by myself since I
joined the team), creating patches, working out where I went wrong and
redoing patches.
The Mozilla workweek at Whistler recently got me thinking a lot about
how we use our time, and if we are using it in the best possible way
(hint: we've got no idea). We're at the stage where Tab Queue V1
looks ready to ship and what is troubling me is that we still have no
idea if this is a feature that people want rather than something we
see as 'neat'.
Really before a feature rides the trains I'd like to ship it to a
group of people who aren't our nightly users [1] and I'd like to get
feedback from them. We don't have a way of doing either of these
things at the moment as Beta is on lockdown for production ready code
and we just don't really have any way of getting actual feedback from
users other than usage stats from the telemetry data which is somewhat
hand-wavey. We're flying blind and it seems crazy to me that we often
put in months of work on a feature when we haven't even justified it's
existence with real users.
The way we operate at the moment in terms of letting features ride the
trains is great from a stability point of view, but not so great from
an experimentation / feedback point of view. It seems that we don't
know if people are going to like a feature until we ship it, by which
point it's too late to get constructive feedback. Apparently we have
an early Beta flag which we can use, but personally I don't understand
why it's there. Imagine if we shipped TQ behind this flag, users will
start using and at some point shortly after it'll just disappear as if
it were never there.
It's time to try something new. I've got some ideas:
* We introduce a new channel which runs alongside Beta but differs
in two ways:
1. Users sign up with the knowledge that this is our testing
ground and all that entails.
2. We build a feedback mechanism in to the channel, similar to
the heartbeat on desktop ("How do you feel about <feature x>"
- Happy face / Sad face with room for comments)
We ship features in this channel which may never see the light of day;
this channel doesn't necessarily follow the same release schedule as
Beta/Prod and doesn't represent a final product (although it'll mostly
be stable enough to use as a daily driver). Using this channel allows
users to test features and also provide feedback for these features,
giving them a voice as to what works or not within Fennec. It allows
us to prototype an idea without having to get it to production level
polish. We don't worry about localisation at the moment and instead
concentrate on getting ideas out and getting feedback in.
* Or we introduce a "Labs" section in Beta which enables users to
opt in to our testing ground. We then build and release features
to beta which are hidden behind flags. The downside of this is
that it wouldn't help in the case of TQ which requests a
permission bump as we can't currently request permissions on the
fly (come on Android M!). Also shipping stable code is important
and Beta is a great way of ensuring that our production code is
ready, I'm reluctant to mess with Beta too much.
Anyway, I really think that we need to work on validating ideas and
getting feedback and I'd love to kick off a discussion about
validating my thoughts (like what I did there?!)
What do you guys think?
Martyn
[1] I don't know how different our nightly users are from our
beta/prod users but because of the different delivery mechanism I'd
argue that they are more power users than the users who download from
the play store.
_______________________________________________
mobile-firefox-dev mailing list
[email protected]
https://mail.mozilla.org/listinfo/mobile-firefox-dev
_______________________________________________
mobile-firefox-dev mailing list
[email protected]
https://mail.mozilla.org/listinfo/mobile-firefox-dev
