On Fri, Apr 26, 2013 at 5:15 PM, Rob Weir <robw...@apache.org> wrote:
> On Fri, Apr 26, 2013 at 6:38 PM, Kay Schenk <kay.sch...@gmail.com> wrote: > > On Fri, Apr 26, 2013 at 2:32 PM, Rob Weir <robw...@apache.org> wrote: > > > >> On Thu, Apr 25, 2013 at 8:18 AM, Rob Weir <robw...@apache.org> wrote: > >> > A quick update. We've been running for 18 hours, enough time to get > >> > around the globe. > >> > > >> > So far we have 1712 responses. > >> > > >> > >> 3072 responses now. > >> > >> I'd like to run the survey at least through the weekend, so we don't > >> bias against home users who might only have time to do the survey on > >> weekends. But I think we can wrap this up early next week. > >> > >> I've started to prototype the analysis scripts, using R, so when the > >> final data comes in I should be able quickly put out a report. So far > >> it looks like there will be around 5 logos that are bunched together > >> at the top. > >> > >> -Rob > >> > > > > I would think running it for at least a week would be good. > > > > > >> > >> > >> > It is probably worth thinking about what we want to do with all the > >> > data. I did an analysis of the results so far, and some quick > >> > thoughts: > >> > > >> > 1) The question asking for person's age appears to be useless. Unless > >> > we have an unusual number of users in their 70's , 80's and 90's, a > >> > large number of users are entering their birth year rather than their > >> > age. So I'm planning on just ignoring that field. (This is probably > >> > why you often see this done as a drop down list of years in other > >> > surveys, less error prone). > >> > > >> > 2) I will be able to produce a list of logos, sorted by average score > >> > and given the % breakdown of 1's, 2's, 3's, etc. Maybe even a > >> > histogram chart for each. > >> > > >> > 3) We're receiving a lot of comments on the individual logos, many > >> > more than I expected. So we can expect that for each logo there will > >> > be 200 or more text comments. I'm not sure what we want to do with > >> > these. Since this is an anonymous survey the users are quite blunt > >> > with their comments on the logos they did not prefer, using coarse > >> > language, etc., stuff that would be inappropriate if a list subscriber > >> > posted. It is not all constructive criticism. Some of it is quite > >> > harsh. On the other hand, there is a tremendous amount of valuable > >> > feedback as well, that we and the designers specifically might find > >> > useful. So what to do? > >> > > >> > a) Post it all, the good the bad and the ugly? > >> > > >> > b) Remove or censor only profanity and post the rest? > >> > > >> > c) Make feedback available only to the individual designers via email > >> > > >> > d) Publicly post the feedback only for the top 5 logos, share the rest > >> > privately with the individual designers. > >> > > > > I think probably c) and only if the designers request it. > > > > But, if we're looking to narrow down choices to 5, those who are more > > intimately involved in this process (marketing folks?), would probably > like > > to see the "constructive" comments for those. > > > > Another approach would be to have some volunteers read through the > comments and summarize in a paragraph, giving something like: > > "12 people thought that the font was too old-fashioned and suggested a > bolder font. Two people said that the number 4 was unlucky in > Chinese culture and should not be featured so predominately in the > logo. Four people wondered whether the logo would render well at > smaller image scales." > > In other words, take the raw comments and figure out the prominent > trends, like someone might do to summarize feedback raised in a > meeting or focus group. (Or those in the US might be reminded of > Zagat restaurant reviews) > > Since each logo has 200+ comments we might want to do this only for > the top 5 or so logos, I could probably volunteer to do this for one > of the logos, but not all of the top ones. But with enough help this > would be possible. > > -Rob > I'd be willing to summarize comments and note the main trends. I think that strikes a good balance between accepting all the input but not asking busy people to wade through a lot of material. Francis
