This post has hit upon an interesting area of design assessment. There
have been books - many in fact - written about scale-based assessment
and measurement. I couldn't possibly touch on all the relevant points
here, but there are some pointers I can provide:
- If your goal is (as dru mentioned) to crosstab responses or generate
measures of central tendency such as means, medians, or modes, then a
scale approach is more useful than a binary "like/dislike" measure.
- There are always positivity biases in your data, depending on what
you're measuring. But there are a whole raft of other biases -
extremity bias, negativity bias, centrality bias - that can come into
play depending on what your respondents are rating, and even what
culture they're from. So you go with your measurement, try to identify
and account for the biases you may encounter, and interpret your data
in light of the context.
- As some of the responses alluded to, it's sometimes difficult to
decide what a "3" or a "1" *means* to each user. One way to address
this is to behaviorally anchor your response options. That is, do
something like (and this is just a humorous example...) "5 (I would
step over my own grandmother to buy this)", "4 (I would buy this if I
were walking by the store, but wouldn't go out of my way to purchase
it)", etc.
- I strongly caution against using scales with more than 5 response
options. And I personally use 4 options quite often, which has the
effect of removing the "neutral" option. The reasons are pretty
straightforward:
Fewer response options: The more response options you use, the harder
it is for your respondents to discriminate between any two adjacent
options. That is, on a 9-point scale, what's the diff. between a "3"
and a "4", a "4" and a "5", etc? Better to use fewer response options.
They're more easily discriminable.
Removing the neutral option: Research into attitude measurement has
shown that people can't *really* hold a neutral attitude. When people
choose the "neither agree nor disagree" (or "neither like nor
dislike") option, it's actually more an indication that they don't
*care* about the item you're asking them to rate. That's different
than neither liking or disliking. And if you don't know what you've
measured, it's impossible to interpret. Forcing people to take a stand
actually gets them more engaged in the question you're asking.
Just M2c, and of course ymmv.
-Paul
- - - - - - -
Paul Sherman, Principal, ShermanUX
UX Design | Research | Strategy
p...@shermanux.com
www.ShermanUX.com
+1.512.917.1942
- - - - - - -
On Nov 12, 2009, at 8:01 AM, R. Tan wrote:
I agree. So, you converted existing ratings and converted favorable
ones to
"like"?
On Thu, Nov 12, 2009 at 7:49 PM, Nik Lazell <
nicholas.laz...@realadventure.co.uk> wrote:
Hi Rih,
We have recently changed from a star rating system to a simple 'like'
functionality. Our findings showed users were more inclined to simply
say 'yes we liked that' - requiring less effort than consider and
justify a score out of 5.
Nik
-----Original Message-----
From: discuss-boun...@lists.interactiondesigners.com
[mailto:discuss-boun...@lists.interactiondesigners.com] On Behalf Of
R.
Tan
Sent: 11 November 2009 16:31
To: disc...@ixda.org
Subject: [IxDA Discuss] Rating vs "Like"
Hi Group,
Any thoughts on when it is best to use a rating system vs a simple
like/don't like voting feature? I currently need to decide which way
to
go
on a restaurant guide project in which the existing rating system is a
1-5
scale "star" (with no guide, higher stars = better).
Unless one has a known and universal explanation on how "stars" are
valued,
it seems that the rating system is flawed. What are your thoughts on
this?
Rih
________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... disc...@ixda.org
Unsubscribe ................ http://www.ixda.org/unsubscribe
List Guidelines ............ http://www.ixda.org/guidelines
List Help .................. http://www.ixda.org/help
________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... disc...@ixda.org
Unsubscribe ................ http://www.ixda.org/unsubscribe
List Guidelines ............ http://www.ixda.org/guidelines
List Help .................. http://www.ixda.org/help
________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... disc...@ixda.org
Unsubscribe ................ http://www.ixda.org/unsubscribe
List Guidelines ............ http://www.ixda.org/guidelines
List Help .................. http://www.ixda.org/help
