You're right, Steve, and I agree.
I didn't mean to imply that a statistically accurate study is less
effective, or less rigorous. I ran such a survey for one company, and
we were very rigorous in ensuring our sampling was randomly
distributed, and had a lot of help from some brilliant statisticians
to ensure we picked the right people. We didn't learn much more than
we did from the first 30-odd participants. However, the weight of the
results meant much more to our executives, because the stats seemed
much more thorough to their thinking. I'd say I actually felt more
confident, knowing we had gone the extra mile.
Sorry about the slip. Just meant to say you can get pretty good
results with a small sampling, which is often as much as you need, and
as much as you have resources to test =]
Cheers =]
Bryan Minihan
On Oct 1, 2009, at 6:02 PM, Steve Baty wrote:
Sorry Bryan, but I need to call this out: "testing a small number of
representative users" as effective as "a lot of random users".
You give the impression that larger studies choose random users as
test participants. You'll find that testing sessions run to meet
statistical standards are required to select a representative sample
in a highly structured and formalised manner. They choose 'users at
random'; they don't choose random users. And the result is a much
more rigorous representation of your audience.
However, what happens on this large scale is not very different to
what we do on a small scale when choosing users from each persona.
This is a type of stratified random sample, and the way you select
the representative from each is likely to be a fairly random method.
None of which changes the point you were trying to make, which is
that smaller tests can be highly effective, and a much more
efficient use of your budget.
Regards
Steve
2009/10/2 Bryan Minihan <bjmini...@gmail.com>
Here's the link I've used before...from Jakob Nielsen. Argue his
credibility if you'd like, but in practice I've seen "testing a
small number
of representative users" as effective as "a lot of random users".
http://www.useit.com/alertbox/20000319.html
--
Steve 'Doc' Baty | Principal | Meld Consulting | P: +61 417 061 292
| E: steveb...@meld.com.au | Twitter: docbaty | Skype: steve_baty |
LinkedIn: www.linkedin.com/in/stevebaty
________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... disc...@ixda.org
Unsubscribe ................ http://www.ixda.org/unsubscribe
List Guidelines ............ http://www.ixda.org/guidelines
List Help .................. http://www.ixda.org/help
