Firstly I strong agree with Graham's that we must make very clear on what we want to know from a survey. e.g. my first impression to Albino's original question list was that the purpose might be to know by which channels do our users get our product information and support... Later we added more purpose in and generated a long list in wiki. While my personal suggestion is that we may want to separate the long list into different surveys, because people who take the survey will have different interest because of their background. One survey should try to target to one certain group, for us to make the observation to certain targets that we set. It will not good if you take a survey and found 1/3 of the questions inside are not what you interested, or even understand... then you might want to quit from the survey, or generate some garbage answers by random selections... It is just my $0.02.
BTW, I noticed that in the question list, we didn't ask for people's career. We asked for the company, but not individual's. I suggest we add this question, since a developer is likely to have a quite different view to an executive... - Simon 2012/6/5 Kevin Grignon <kevingrignon...@gmail.com> > KG02 - see comments inline. > > On Tuesday, June 5, 2012, Graham Lauder wrote: > > > > KG01 - see comments inline. > > > > > > On Tue, Jun 5, 2012 at 8:26 AM, Graham Lauder <y...@apache.org > <javascript:;>> > > wrote: > > > > > > Hi. > > > > > > > > > > > > Questions relating to research! > > > > > > > > [....] > > > > > > > > > Perhaps the first survey we should conduct is a survey about what > > sort > > > > > of surveys our users would respond to. > > > > > > KG01 - Thanks for your feedback and interest in the user research > effort. > > > While I agree we could deploy different types of surveys to gather > > > different types of data, I feel that a survey of surveys might induce > > > premature survey fatigue. > > > > Survey fatique has already set in, that is not a new thing, that is > > recognisable simply by those surveys conducted by SUN. We haven't caused > > that, it is a factor of the modern marketing malaise. The cost of > > incentives > > these days, that one needs to hand out to get a significant sampling in a > > timely manner is huge. > > > KG02 - Agreed, risk of fatigue is a planning consideration. > > > > > > User research, especially surveys, consumes > > > people's time and energy. > > > > Indeed as I myself pointed out earlier in this thread > > > > > Rather, I propose we work from the other > > > direction. If the goal of the research activity is to gather data that > > will > > > help us build insight and drive informed design and development > > decisions, > > > then we should focus the surveys on the information we need to do > that. I > > > have captured some comments in the wiki discussion page. > > > > Indeed, however if the sample of respondents is ridiculously small, as > has > > historically been the case, then the data is useless. > > > KG02 - Perhaps, let's think positive :) > > > > > > You cannot use corporate methodologies in an open source environment. We > > have > > no ability to offer incentives, we therefore need to make the survey > > process > > as pleasant and enjoyable as possible or we need to find out from people > > what > > would encourage them to participate. > > > KG02 - Ok. I'm not advocating corporate or open source. I'm advocating that > we create surveys that 1) will deliver good data and 2) people will fill > out. > > > > > That requires research, I doubt it will require as big a sample as a UX > > survey > > but that is only because there are a limited number of answers needed. > > > > Every good research organisation I have worked with does short surveys to > > find > > out what they're doing right or wrong. > > > KG02 - indeed, a useful activity. > > > > For the most part they do these at the > > end of another survey, but that is because the group of respondents they > > are > > questioning will probably never do the same survey again. > > > KG02 - While surveys are common after usability evaluation session, nested > surveys are new to me. > > > > For us the problem > > has been getting respondents to finish. Lose them once and they won't > come > > back again and we will need to talk to our user community if not often, > at > > least regularly > > > KG02 - Consumability is a noted concern, and a valid goal. > > I would prefer to do things right first time up so people will happily > > respond > > to any surveys we need to put out. Remember that there are not only UX > > surveys to be done but Marketing as well. > > > KG02 - Indeed, that is why I placed a call for input from all disciplines. > > > > > We know already know two things that get people to complete surveys: > > Brevity and Fun. > > > > If we do a light hearted, quick survey that gives us the reasons that > > people > > will participate, I think that's a really good use of resources. > > > KG02 - Agreed. > > > > > The Surveys already put up are boring, generic and not likely to inspire > > people to complete them. > > > > OOo has a user base in the hundreds of millions a few hundred completions > > is > > not a sample. We need 10s of thousands of responses across scores of > > languages, to get a easonable sample. > > > > So first we need to figure out how to get that sample. > > > KG02 - Indeed a sustainable research strategy is important. Please capture > your thoughts on the wiki. > > > > > > Cheers > > G > > > > > > >
