Re: [UX] The Questions for users
On Wed, Jun 6, 2012 at 11:30 PM, Kevin Grignon kevingrignon...@gmail.com wrote: KG 01 - See comments inline On Wed, Jun 6, 2012 at 8:00 PM, Rob Weir robw...@apache.org wrote: Keep in mind that we already have a large recent survey of sorts, based on Google Analytics data from those who have visited the website and downloaded AOO. KG01 - Do we ask for email addresses of people who download. How might we engage downloaders to complete survey? When users download they are presented with a screen at SourceForge that invites them to sign up for our announcement mailing list, ooo-annnounce. We have over 8000 subscribers to that list. So it is just a small % of the 3 million+ downloads. We also have the ability to reach out to users via Twitter, Google+ Facebook, etc., So there are several ways we could get the word out. Another technique that we might be able to do is to offer the survey link to a random sample of those who download, say 1%. It won't tell us some of the detailed stuff, like whether they use AOO at home or at work, but there is more info available here than might be generally known. KG01 - Re-use is great. Can you send along a link to where I can review the specific data we capture? Yes, I think we can eliminate many questions form our demographic questions and pull the data from other sources, such as the download info. OK. I'll put together a report and send it it out to the list. For example: - what countries users are mainly from. Can also get detail to the level of what cities are most often downloading AOO. - what languages - what operating systems and versions they are using - what screen resolution they have - what browser they are using - if they found our website from searching Google, what were the most used search strings - if they came to our website via a link from another website, what were the most common referring sites - what social networking sites lead them most to the website - what pages on the website are most frequently read - what paths through the website most often lead to a download and any of these can be correlated against download conversion rate. So for example we can look at what % of visitors download AOO based on country, or language, or OS or browser or whatever. KG01 - Despite the overlap, I suspect that we will need to include some basic demographic questions in the user surveys to ensure we can correlate the results. For example, if users from a certain geography, or users of a certain role have issues, we need to assocaite their task prioritization and satisfaction ratings against their demographic data. Data from disparate data sources would not support such analysis. Right. Obviously this is not a replacement for a survey that looks at the habits and preferences of the user's in-application behavior. But this information is low hanging fruit that is based on data already collected. KG01 - Indeed, task assessment research is another category all together. -Rob On Tue, Jun 5, 2012 at 1:51 AM, Graham Lauder y...@apache.org wrote: KG01 - see comments inline. On Tue, Jun 5, 2012 at 8:26 AM, Graham Lauder y...@apache.org wrote: Hi. Questions relating to research! [] Perhaps the first survey we should conduct is a survey about what sort of surveys our users would respond to. KG01 - Thanks for your feedback and interest in the user research effort. While I agree we could deploy different types of surveys to gather different types of data, I feel that a survey of surveys might induce premature survey fatigue. Survey fatique has already set in, that is not a new thing, that is recognisable simply by those surveys conducted by SUN. We haven't caused that, it is a factor of the modern marketing malaise. The cost of incentives these days, that one needs to hand out to get a significant sampling in a timely manner is huge. User research, especially surveys, consumes people's time and energy. Indeed as I myself pointed out earlier in this thread Rather, I propose we work from the other direction. If the goal of the research activity is to gather data that will help us build insight and drive informed design and development decisions, then we should focus the surveys on the information we need to do that. I have captured some comments in the wiki discussion page. Indeed, however if the sample of respondents is ridiculously small, as has historically been the case, then the data is useless. You cannot use corporate methodologies in an open source environment. We have no ability to offer incentives, we therefore need to make the survey process as pleasant and enjoyable as possible or we need to find out from people what would encourage them to participate. That requires research, I doubt it will require as big a sample as a UX survey but that is only because there are a
Re: [UX] The Questions for users
On 7 June 2012 04:32, Kevin Grignon kevingrignon...@gmail.com wrote: Albino, Indeed, we have captured many good questions. I propose we start to share process and methology thoughts to move forward. Graham L. has some good suggestions. As for survey structure, I've been working on this online and will post my proposal tomorrow. I'm in training this afternoon, and won't have time to post until tomorrow am. Thanks for supporting this activity. Let's keep pushing. We need data to drive informed design and development decisions. I'd like to contribute one principle. Reduce the number of menu options in lists and move more to well-constructed dialogue boxes. As an example, I don't see the logic in having a Format main option and then Table next to it. Under Format, a Table entry that then leads to a dialogue for designing, inserting copying and deleting tables would be a lot better. Regards, Kevin On Wed, Jun 6, 2012 at 8:56 PM, Albino B Neto bino...@gmail.com wrote: Hi. We have many suggestions and questions for + of 40 minutes. I think it's time to define questions ? Albino -- Ian Ofqual Accredited IT Qualifications (The Schools ITQ) www.theINGOTs.org +44 (0)1827 305940 The Learning Machine Limited, Reg Office, 36 Ashby Road, Tamworth, Staffordshire, B79 8AQ. Reg No: 05560797, Registered in England and Wales.
Re: [UX] The Questions for users
Hi, I see the question list in UX wiki and I have some thoughts about the survey. For the question How old are you ? , my understanding is the purpose of this question is to see the age range of AOO users. I suggest we word the question like this: (Optional) If you don't mind, which age range are you in? A. ≤29 B. 30~39 C. 40~49 D. ≥50 I suggest we add a question to see what's the most important performance of office software when user use a office software and how user score these performances in AOO. 1. What's the most important performance of office software during your daily use? A.Design aesthetics B.Interface layout reasonable C.The efficiency ofdocument editing D. Software running speed 2. Please score to AOO for the following aspects performance based on your own use experience.(5 is very good, 1 is very bad) Design aesthetics 54321 Interface layout reasonable54321 The efficiency of document editing 54321 Software running speed 54321 What's your opinion and any thoughts? 2012/6/6 Kevin Grignon kevingrignon...@gmail.com KG01 - see comments inline. On Jun 6, 2012, at 3:01 AM, Dennis E. Hamilton dennis.hamil...@acm.org wrote: I am adding this here just because of the mention of career, profession, and employer. If I encountered a questionnaire that asked who my employer was, I would stop right there and not respond. It is an obvious asked-because-we-can useless question and impeaches the questionnaire source in my mind. If I encountered a questionnaire that asked for the nature of my employment (employed, retired, self-employed), I would respond to that. I would also answer a question about the industry my work was in. I would answer a question about the number of people in the organization where I work. I see no earthly purpose to knowing someone's employer and I don't believe it should be requested. KG01 - Agreed. Employer data provides no value to us and could be considered invasive. Role and industry data is more useful and valuable. As a reminder, the question candidates are a pool from which we can create the actual surveys. Final surveys will be subject to community approval. I'll add your comments to the wiki. - Dennis -Original Message- From: Shenfeng Liu [mailto:liush...@gmail.com] Sent: Monday, June 04, 2012 23:56 To: ooo-dev@incubator.apache.org Subject: Re: [UX] The Questions for users Firstly I strong agree with Graham's that we must make very clear on what we want to know from a survey. e.g. my first impression to Albino's original question list was that the purpose might be to know by which channels do our users get our product information and support... Later we added more purpose in and generated a long list in wiki. While my personal suggestion is that we may want to separate the long list into different surveys, because people who take the survey will have different interest because of their background. One survey should try to target to one certain group, for us to make the observation to certain targets that we set. It will not good if you take a survey and found 1/3 of the questions inside are not what you interested, or even understand... then you might want to quit from the survey, or generate some garbage answers by random selections... It is just my $0.02. BTW, I noticed that in the question list, we didn't ask for people's career. We asked for the company, but not individual's. I suggest we add this question, since a developer is likely to have a quite different view to an executive... - Simon 2012/6/5 Kevin Grignon kevingrignon...@gmail.com KG02 - see comments inline. On Tuesday, June 5, 2012, Graham Lauder wrote: KG01 - see comments inline. On Tue, Jun 5, 2012 at 8:26 AM, Graham Lauder y...@apache.org javascript:; wrote: Hi. Questions relating to research! [] Perhaps the first survey we should conduct is a survey about what sort of surveys our users would respond to. KG01 - Thanks for your feedback and interest in the user research effort. While I agree we could deploy different types of surveys to gather different types of data, I feel that a survey of surveys might induce premature survey fatigue. Survey fatique has already set in, that is not a new thing, that is recognisable simply by those surveys conducted by SUN. We haven't caused that, it is a factor of the modern marketing malaise. The cost of incentives these days, that one needs to hand out to get a significant sampling in a timely manner is huge. KG02 - Agreed, risk of fatigue is a planning consideration. User research, especially surveys, consumes people's time and energy. Indeed as I myself pointed out earlier in this thread Rather, I propose we work from the other direction
Re: [UX] The Questions for users
Keep in mind that we already have a large recent survey of sorts, based on Google Analytics data from those who have visited the website and downloaded AOO. It won't tell us some of the detailed stuff, like whether they use AOO at home or at work, but there is more info available here than might be generally known. For example: - what countries users are mainly from. Can also get detail to the level of what cities are most often downloading AOO. - what languages - what operating systems and versions they are using - what screen resolution they have - what browser they are using - if they found our website from searching Google, what were the most used search strings - if they came to our website via a link from another website, what were the most common referring sites - what social networking sites lead them most to the website - what pages on the website are most frequently read - what paths through the website most often lead to a download and any of these can be correlated against download conversion rate. So for example we can look at what % of visitors download AOO based on country, or language, or OS or browser or whatever. Obviously this is not a replacement for a survey that looks at the habits and preferences of the user's in-application behavior. But this information is low hanging fruit that is based on data already collected. -Rob On Tue, Jun 5, 2012 at 1:51 AM, Graham Lauder y...@apache.org wrote: KG01 - see comments inline. On Tue, Jun 5, 2012 at 8:26 AM, Graham Lauder y...@apache.org wrote: Hi. Questions relating to research! [] Perhaps the first survey we should conduct is a survey about what sort of surveys our users would respond to. KG01 - Thanks for your feedback and interest in the user research effort. While I agree we could deploy different types of surveys to gather different types of data, I feel that a survey of surveys might induce premature survey fatigue. Survey fatique has already set in, that is not a new thing, that is recognisable simply by those surveys conducted by SUN. We haven't caused that, it is a factor of the modern marketing malaise. The cost of incentives these days, that one needs to hand out to get a significant sampling in a timely manner is huge. User research, especially surveys, consumes people's time and energy. Indeed as I myself pointed out earlier in this thread Rather, I propose we work from the other direction. If the goal of the research activity is to gather data that will help us build insight and drive informed design and development decisions, then we should focus the surveys on the information we need to do that. I have captured some comments in the wiki discussion page. Indeed, however if the sample of respondents is ridiculously small, as has historically been the case, then the data is useless. You cannot use corporate methodologies in an open source environment. We have no ability to offer incentives, we therefore need to make the survey process as pleasant and enjoyable as possible or we need to find out from people what would encourage them to participate. That requires research, I doubt it will require as big a sample as a UX survey but that is only because there are a limited number of answers needed. Every good research organisation I have worked with does short surveys to find out what they're doing right or wrong. For the most part they do these at the end of another survey, but that is because the group of respondents they are questioning will probably never do the same survey again. For us the problem has been getting respondents to finish. Lose them once and they won't come back again and we will need to talk to our user community if not often, at least regularly I would prefer to do things right first time up so people will happily respond to any surveys we need to put out. Remember that there are not only UX surveys to be done but Marketing as well. We know already know two things that get people to complete surveys: Brevity and Fun. If we do a light hearted, quick survey that gives us the reasons that people will participate, I think that's a really good use of resources. The Surveys already put up are boring, generic and not likely to inspire people to complete them. OOo has a user base in the hundreds of millions a few hundred completions is not a sample. We need 10s of thousands of responses across scores of languages, to get a easonable sample. So first we need to figure out how to get that sample. Cheers G
Re: [UX] The Questions for users
Hi. We have many suggestions and questions for + of 40 minutes. I think it's time to define questions ? Albino
Re: [UX] The Questions for users
KG01 - See comments inline. On Wed, Jun 6, 2012 at 4:49 PM, Xin Li lxnice...@gmail.com wrote: Hi, I see the question list in UX wiki and I have some thoughts about the survey. For the question How old are you ? , my understanding is the purpose of this question is to see the age range of AOO users. I suggest we word the question like this: (Optional) If you don't mind, which age range are you in? A. ≤29 B. 30~39 C. 40~49 D. ≥50 KG01 - Indeed, when we select the final question, many questions will need to be refined. Some are, as you illustrate, too open ended. I suggest we add a question to see what's the most important performance of office software when user use a office software and how user score these performances in AOO. 1. What's the most important performance of office software during your daily use? A.Design aesthetics B.Interface layout reasonable C.The efficiency ofdocument editing D. Software running speed 2. Please score to AOO for the following aspects performance based on your own use experience.(5 is very good, 1 is very bad) Design aesthetics 54321 Interface layout reasonable54321 The efficiency of document editing 54321 Software running speed 54321 KG01 - Presenting top tasks and asking respondents to apply a prioritization and usability rating are core to our product feedback research. I'll share more thought on this in an upcoming email. What's your opinion and any thoughts? 2012/6/6 Kevin Grignon kevingrignon...@gmail.com KG01 - see comments inline. On Jun 6, 2012, at 3:01 AM, Dennis E. Hamilton dennis.hamil...@acm.org wrote: I am adding this here just because of the mention of career, profession, and employer. If I encountered a questionnaire that asked who my employer was, I would stop right there and not respond. It is an obvious asked-because-we-can useless question and impeaches the questionnaire source in my mind. If I encountered a questionnaire that asked for the nature of my employment (employed, retired, self-employed), I would respond to that. I would also answer a question about the industry my work was in. I would answer a question about the number of people in the organization where I work. I see no earthly purpose to knowing someone's employer and I don't believe it should be requested. KG01 - Agreed. Employer data provides no value to us and could be considered invasive. Role and industry data is more useful and valuable. As a reminder, the question candidates are a pool from which we can create the actual surveys. Final surveys will be subject to community approval. I'll add your comments to the wiki. - Dennis -Original Message- From: Shenfeng Liu [mailto:liush...@gmail.com] Sent: Monday, June 04, 2012 23:56 To: ooo-dev@incubator.apache.org Subject: Re: [UX] The Questions for users Firstly I strong agree with Graham's that we must make very clear on what we want to know from a survey. e.g. my first impression to Albino's original question list was that the purpose might be to know by which channels do our users get our product information and support... Later we added more purpose in and generated a long list in wiki. While my personal suggestion is that we may want to separate the long list into different surveys, because people who take the survey will have different interest because of their background. One survey should try to target to one certain group, for us to make the observation to certain targets that we set. It will not good if you take a survey and found 1/3 of the questions inside are not what you interested, or even understand... then you might want to quit from the survey, or generate some garbage answers by random selections... It is just my $0.02. BTW, I noticed that in the question list, we didn't ask for people's career. We asked for the company, but not individual's. I suggest we add this question, since a developer is likely to have a quite different view to an executive... - Simon 2012/6/5 Kevin Grignon kevingrignon...@gmail.com KG02 - see comments inline. On Tuesday, June 5, 2012, Graham Lauder wrote: KG01 - see comments inline. On Tue, Jun 5, 2012 at 8:26 AM, Graham Lauder y...@apache.org javascript:; wrote: Hi. Questions relating to research! [] Perhaps the first survey we should conduct is a survey about what sort of surveys our users would respond to. KG01 - Thanks for your feedback and interest in the user research effort. While I agree we could deploy different types of surveys to gather different types of data, I feel that a survey of surveys might induce premature survey fatigue. Survey fatique has already
Re: [UX] The Questions for users
KG 01 - See comments inline On Wed, Jun 6, 2012 at 8:00 PM, Rob Weir robw...@apache.org wrote: Keep in mind that we already have a large recent survey of sorts, based on Google Analytics data from those who have visited the website and downloaded AOO. KG01 - Do we ask for email addresses of people who download. How might we engage downloaders to complete survey? It won't tell us some of the detailed stuff, like whether they use AOO at home or at work, but there is more info available here than might be generally known. KG01 - Re-use is great. Can you send along a link to where I can review the specific data we capture? Yes, I think we can eliminate many questions form our demographic questions and pull the data from other sources, such as the download info. For example: - what countries users are mainly from. Can also get detail to the level of what cities are most often downloading AOO. - what languages - what operating systems and versions they are using - what screen resolution they have - what browser they are using - if they found our website from searching Google, what were the most used search strings - if they came to our website via a link from another website, what were the most common referring sites - what social networking sites lead them most to the website - what pages on the website are most frequently read - what paths through the website most often lead to a download and any of these can be correlated against download conversion rate. So for example we can look at what % of visitors download AOO based on country, or language, or OS or browser or whatever. KG01 - Despite the overlap, I suspect that we will need to include some basic demographic questions in the user surveys to ensure we can correlate the results. For example, if users from a certain geography, or users of a certain role have issues, we need to assocaite their task prioritization and satisfaction ratings against their demographic data. Data from disparate data sources would not support such analysis. Obviously this is not a replacement for a survey that looks at the habits and preferences of the user's in-application behavior. But this information is low hanging fruit that is based on data already collected. KG01 - Indeed, task assessment research is another category all together. -Rob On Tue, Jun 5, 2012 at 1:51 AM, Graham Lauder y...@apache.org wrote: KG01 - see comments inline. On Tue, Jun 5, 2012 at 8:26 AM, Graham Lauder y...@apache.org wrote: Hi. Questions relating to research! [] Perhaps the first survey we should conduct is a survey about what sort of surveys our users would respond to. KG01 - Thanks for your feedback and interest in the user research effort. While I agree we could deploy different types of surveys to gather different types of data, I feel that a survey of surveys might induce premature survey fatigue. Survey fatique has already set in, that is not a new thing, that is recognisable simply by those surveys conducted by SUN. We haven't caused that, it is a factor of the modern marketing malaise. The cost of incentives these days, that one needs to hand out to get a significant sampling in a timely manner is huge. User research, especially surveys, consumes people's time and energy. Indeed as I myself pointed out earlier in this thread Rather, I propose we work from the other direction. If the goal of the research activity is to gather data that will help us build insight and drive informed design and development decisions, then we should focus the surveys on the information we need to do that. I have captured some comments in the wiki discussion page. Indeed, however if the sample of respondents is ridiculously small, as has historically been the case, then the data is useless. You cannot use corporate methodologies in an open source environment. We have no ability to offer incentives, we therefore need to make the survey process as pleasant and enjoyable as possible or we need to find out from people what would encourage them to participate. That requires research, I doubt it will require as big a sample as a UX survey but that is only because there are a limited number of answers needed. Every good research organisation I have worked with does short surveys to find out what they're doing right or wrong. For the most part they do these at the end of another survey, but that is because the group of respondents they are questioning will probably never do the same survey again. For us the problem has been getting respondents to finish. Lose them once and they won't come back again and we will need to talk to our user community if not often, at least regularly I would prefer to do things right first time up so people will happily respond to any surveys we need to put out. Remember that there are
Re: [UX] The Questions for users
Albino, Indeed, we have captured many good questions. I propose we start to share process and methology thoughts to move forward. Graham L. has some good suggestions. As for survey structure, I've been working on this online and will post my proposal tomorrow. I'm in training this afternoon, and won't have time to post until tomorrow am. Thanks for supporting this activity. Let's keep pushing. We need data to drive informed design and development decisions. Regards, Kevin On Wed, Jun 6, 2012 at 8:56 PM, Albino B Neto bino...@gmail.com wrote: Hi. We have many suggestions and questions for + of 40 minutes. I think it's time to define questions ? Albino
Re: [UX] The Questions for users
KG02 - see comments inline. On Tuesday, June 5, 2012, Graham Lauder wrote: KG01 - see comments inline. On Tue, Jun 5, 2012 at 8:26 AM, Graham Lauder y...@apache.orgjavascript:; wrote: Hi. Questions relating to research! [] Perhaps the first survey we should conduct is a survey about what sort of surveys our users would respond to. KG01 - Thanks for your feedback and interest in the user research effort. While I agree we could deploy different types of surveys to gather different types of data, I feel that a survey of surveys might induce premature survey fatigue. Survey fatique has already set in, that is not a new thing, that is recognisable simply by those surveys conducted by SUN. We haven't caused that, it is a factor of the modern marketing malaise. The cost of incentives these days, that one needs to hand out to get a significant sampling in a timely manner is huge. KG02 - Agreed, risk of fatigue is a planning consideration. User research, especially surveys, consumes people's time and energy. Indeed as I myself pointed out earlier in this thread Rather, I propose we work from the other direction. If the goal of the research activity is to gather data that will help us build insight and drive informed design and development decisions, then we should focus the surveys on the information we need to do that. I have captured some comments in the wiki discussion page. Indeed, however if the sample of respondents is ridiculously small, as has historically been the case, then the data is useless. KG02 - Perhaps, let's think positive :) You cannot use corporate methodologies in an open source environment. We have no ability to offer incentives, we therefore need to make the survey process as pleasant and enjoyable as possible or we need to find out from people what would encourage them to participate. KG02 - Ok. I'm not advocating corporate or open source. I'm advocating that we create surveys that 1) will deliver good data and 2) people will fill out. That requires research, I doubt it will require as big a sample as a UX survey but that is only because there are a limited number of answers needed. Every good research organisation I have worked with does short surveys to find out what they're doing right or wrong. KG02 - indeed, a useful activity. For the most part they do these at the end of another survey, but that is because the group of respondents they are questioning will probably never do the same survey again. KG02 - While surveys are common after usability evaluation session, nested surveys are new to me. For us the problem has been getting respondents to finish. Lose them once and they won't come back again and we will need to talk to our user community if not often, at least regularly KG02 - Consumability is a noted concern, and a valid goal. I would prefer to do things right first time up so people will happily respond to any surveys we need to put out. Remember that there are not only UX surveys to be done but Marketing as well. KG02 - Indeed, that is why I placed a call for input from all disciplines. We know already know two things that get people to complete surveys: Brevity and Fun. If we do a light hearted, quick survey that gives us the reasons that people will participate, I think that's a really good use of resources. KG02 - Agreed. The Surveys already put up are boring, generic and not likely to inspire people to complete them. OOo has a user base in the hundreds of millions a few hundred completions is not a sample. We need 10s of thousands of responses across scores of languages, to get a easonable sample. So first we need to figure out how to get that sample. KG02 - Indeed a sustainable research strategy is important. Please capture your thoughts on the wiki. Cheers G
Re: [UX] The Questions for users
Firstly I strong agree with Graham's that we must make very clear on what we want to know from a survey. e.g. my first impression to Albino's original question list was that the purpose might be to know by which channels do our users get our product information and support... Later we added more purpose in and generated a long list in wiki. While my personal suggestion is that we may want to separate the long list into different surveys, because people who take the survey will have different interest because of their background. One survey should try to target to one certain group, for us to make the observation to certain targets that we set. It will not good if you take a survey and found 1/3 of the questions inside are not what you interested, or even understand... then you might want to quit from the survey, or generate some garbage answers by random selections... It is just my $0.02. BTW, I noticed that in the question list, we didn't ask for people's career. We asked for the company, but not individual's. I suggest we add this question, since a developer is likely to have a quite different view to an executive... - Simon 2012/6/5 Kevin Grignon kevingrignon...@gmail.com KG02 - see comments inline. On Tuesday, June 5, 2012, Graham Lauder wrote: KG01 - see comments inline. On Tue, Jun 5, 2012 at 8:26 AM, Graham Lauder y...@apache.org javascript:; wrote: Hi. Questions relating to research! [] Perhaps the first survey we should conduct is a survey about what sort of surveys our users would respond to. KG01 - Thanks for your feedback and interest in the user research effort. While I agree we could deploy different types of surveys to gather different types of data, I feel that a survey of surveys might induce premature survey fatigue. Survey fatique has already set in, that is not a new thing, that is recognisable simply by those surveys conducted by SUN. We haven't caused that, it is a factor of the modern marketing malaise. The cost of incentives these days, that one needs to hand out to get a significant sampling in a timely manner is huge. KG02 - Agreed, risk of fatigue is a planning consideration. User research, especially surveys, consumes people's time and energy. Indeed as I myself pointed out earlier in this thread Rather, I propose we work from the other direction. If the goal of the research activity is to gather data that will help us build insight and drive informed design and development decisions, then we should focus the surveys on the information we need to do that. I have captured some comments in the wiki discussion page. Indeed, however if the sample of respondents is ridiculously small, as has historically been the case, then the data is useless. KG02 - Perhaps, let's think positive :) You cannot use corporate methodologies in an open source environment. We have no ability to offer incentives, we therefore need to make the survey process as pleasant and enjoyable as possible or we need to find out from people what would encourage them to participate. KG02 - Ok. I'm not advocating corporate or open source. I'm advocating that we create surveys that 1) will deliver good data and 2) people will fill out. That requires research, I doubt it will require as big a sample as a UX survey but that is only because there are a limited number of answers needed. Every good research organisation I have worked with does short surveys to find out what they're doing right or wrong. KG02 - indeed, a useful activity. For the most part they do these at the end of another survey, but that is because the group of respondents they are questioning will probably never do the same survey again. KG02 - While surveys are common after usability evaluation session, nested surveys are new to me. For us the problem has been getting respondents to finish. Lose them once and they won't come back again and we will need to talk to our user community if not often, at least regularly KG02 - Consumability is a noted concern, and a valid goal. I would prefer to do things right first time up so people will happily respond to any surveys we need to put out. Remember that there are not only UX surveys to be done but Marketing as well. KG02 - Indeed, that is why I placed a call for input from all disciplines. We know already know two things that get people to complete surveys: Brevity and Fun. If we do a light hearted, quick survey that gives us the reasons that people will participate, I think that's a really good use of resources. KG02 - Agreed. The Surveys already put up are boring, generic and not likely to inspire people to complete them. OOo has a user base in the hundreds of millions a few hundred completions is not a sample. We need 10s of thousands of
Re: [UX] The Questions for users
KG03 - See comments inline. On Tue, Jun 5, 2012 at 2:56 PM, Shenfeng Liu liush...@gmail.com wrote: Firstly I strong agree with Graham's that we must make very clear on what we want to know from a survey. e.g. my first impression to Albino's original question list was that the purpose might be to know by which channels do our users get our product information and support... Later we added more purpose in and generated a long list in wiki. While my personal suggestion is that we may want to separate the long list into different surveys, because people who take the survey will have different interest because of their background. KG03 - Tailoring the survey content to keep the survey brief yet still a source of good data is a key priority. One survey should try to target to one certain group, for us to make the observation to certain targets that we set. It will not good if you take a survey and found 1/3 of the questions inside are not what you interested, or even understand... then you might want to quit from the survey, or generate some garbage answers by random selections... It is just my $0.02. BTW, I noticed that in the question list, we didn't ask for people's career. We asked for the company, but not individual's. I suggest we add this question, since a developer is likely to have a quite different view to an executive... KG03 - Actually, that is in there already. Question A50: Which best describes your job role? KG03 - Please share any specific data requirements that you or your development peers feel would help drive informed design and development decisions moving forward. Capture your thoughts on the wiki. See: http://wiki.services.openoffice.org/wiki/AOO_UX_Research_Surveys - Simon 2012/6/5 Kevin Grignon kevingrignon...@gmail.com KG02 - see comments inline. On Tuesday, June 5, 2012, Graham Lauder wrote: KG01 - see comments inline. On Tue, Jun 5, 2012 at 8:26 AM, Graham Lauder y...@apache.org javascript:; wrote: Hi. Questions relating to research! [] Perhaps the first survey we should conduct is a survey about what sort of surveys our users would respond to. KG01 - Thanks for your feedback and interest in the user research effort. While I agree we could deploy different types of surveys to gather different types of data, I feel that a survey of surveys might induce premature survey fatigue. Survey fatique has already set in, that is not a new thing, that is recognisable simply by those surveys conducted by SUN. We haven't caused that, it is a factor of the modern marketing malaise. The cost of incentives these days, that one needs to hand out to get a significant sampling in a timely manner is huge. KG02 - Agreed, risk of fatigue is a planning consideration. User research, especially surveys, consumes people's time and energy. Indeed as I myself pointed out earlier in this thread Rather, I propose we work from the other direction. If the goal of the research activity is to gather data that will help us build insight and drive informed design and development decisions, then we should focus the surveys on the information we need to do that. I have captured some comments in the wiki discussion page. Indeed, however if the sample of respondents is ridiculously small, as has historically been the case, then the data is useless. KG02 - Perhaps, let's think positive :) You cannot use corporate methodologies in an open source environment. We have no ability to offer incentives, we therefore need to make the survey process as pleasant and enjoyable as possible or we need to find out from people what would encourage them to participate. KG02 - Ok. I'm not advocating corporate or open source. I'm advocating that we create surveys that 1) will deliver good data and 2) people will fill out. That requires research, I doubt it will require as big a sample as a UX survey but that is only because there are a limited number of answers needed. Every good research organisation I have worked with does short surveys to find out what they're doing right or wrong. KG02 - indeed, a useful activity. For the most part they do these at the end of another survey, but that is because the group of respondents they are questioning will probably never do the same survey again. KG02 - While surveys are common after usability evaluation session, nested surveys are new to me. For us the problem has been getting respondents to finish. Lose them once and they won't come back again and we will need to talk to our user community if not often, at least regularly KG02 - Consumability is a noted concern, and a valid goal. I would prefer to do things right first time up so
Re: [UX] The Questions for users
2012/6/5 Kevin Grignon kevingrignon...@gmail.com KG03 - See comments inline. On Tue, Jun 5, 2012 at 2:56 PM, Shenfeng Liu liush...@gmail.com wrote: Firstly I strong agree with Graham's that we must make very clear on what we want to know from a survey. e.g. my first impression to Albino's original question list was that the purpose might be to know by which channels do our users get our product information and support... Later we added more purpose in and generated a long list in wiki. While my personal suggestion is that we may want to separate the long list into different surveys, because people who take the survey will have different interest because of their background. KG03 - Tailoring the survey content to keep the survey brief yet still a source of good data is a key priority. One survey should try to target to one certain group, for us to make the observation to certain targets that we set. It will not good if you take a survey and found 1/3 of the questions inside are not what you interested, or even understand... then you might want to quit from the survey, or generate some garbage answers by random selections... It is just my $0.02. BTW, I noticed that in the question list, we didn't ask for people's career. We asked for the company, but not individual's. I suggest we add this question, since a developer is likely to have a quite different view to an executive... KG03 - Actually, that is in there already. Question A50: Which best describes your job role? Found it! Thanks, Kevin! KG03 - Please share any specific data requirements that you or your development peers feel would help drive informed design and development decisions moving forward. Capture your thoughts on the wiki. See: http://wiki.services.openoffice.org/wiki/AOO_UX_Research_Surveys - Simon 2012/6/5 Kevin Grignon kevingrignon...@gmail.com KG02 - see comments inline. On Tuesday, June 5, 2012, Graham Lauder wrote: KG01 - see comments inline. On Tue, Jun 5, 2012 at 8:26 AM, Graham Lauder y...@apache.org javascript:; wrote: Hi. Questions relating to research! [] Perhaps the first survey we should conduct is a survey about what sort of surveys our users would respond to. KG01 - Thanks for your feedback and interest in the user research effort. While I agree we could deploy different types of surveys to gather different types of data, I feel that a survey of surveys might induce premature survey fatigue. Survey fatique has already set in, that is not a new thing, that is recognisable simply by those surveys conducted by SUN. We haven't caused that, it is a factor of the modern marketing malaise. The cost of incentives these days, that one needs to hand out to get a significant sampling in a timely manner is huge. KG02 - Agreed, risk of fatigue is a planning consideration. User research, especially surveys, consumes people's time and energy. Indeed as I myself pointed out earlier in this thread Rather, I propose we work from the other direction. If the goal of the research activity is to gather data that will help us build insight and drive informed design and development decisions, then we should focus the surveys on the information we need to do that. I have captured some comments in the wiki discussion page. Indeed, however if the sample of respondents is ridiculously small, as has historically been the case, then the data is useless. KG02 - Perhaps, let's think positive :) You cannot use corporate methodologies in an open source environment. We have no ability to offer incentives, we therefore need to make the survey process as pleasant and enjoyable as possible or we need to find out from people what would encourage them to participate. KG02 - Ok. I'm not advocating corporate or open source. I'm advocating that we create surveys that 1) will deliver good data and 2) people will fill out. That requires research, I doubt it will require as big a sample as a UX survey but that is only because there are a limited number of answers needed. Every good research organisation I have worked with does short surveys to find out what they're doing right or wrong. KG02 - indeed, a useful activity. For the most part they do these at the end of another survey, but that is because the group of respondents they are questioning will probably never do the same survey again. KG02 - While surveys are common after usability evaluation session, nested surveys are new to me. For us the problem has been getting respondents to finish. Lose them once and they won't
Re: [UX] The Questions for users
Hi. 2012/6/4 Kevin Grignon kevingrignon...@gmail.com: Let's review this pool of available questions and select the most useful for our survey. Join useful with the pleasent. :) Once we have a draft, let's call for reviewers, as their are some values we'll need to verify. Yeah! Albino
RE: [UX] The Questions for users
I am adding this here just because of the mention of career, profession, and employer. If I encountered a questionnaire that asked who my employer was, I would stop right there and not respond. It is an obvious asked-because-we-can useless question and impeaches the questionnaire source in my mind. If I encountered a questionnaire that asked for the nature of my employment (employed, retired, self-employed), I would respond to that. I would also answer a question about the industry my work was in. I would answer a question about the number of people in the organization where I work. I see no earthly purpose to knowing someone's employer and I don't believe it should be requested. - Dennis -Original Message- From: Shenfeng Liu [mailto:liush...@gmail.com] Sent: Monday, June 04, 2012 23:56 To: ooo-dev@incubator.apache.org Subject: Re: [UX] The Questions for users Firstly I strong agree with Graham's that we must make very clear on what we want to know from a survey. e.g. my first impression to Albino's original question list was that the purpose might be to know by which channels do our users get our product information and support... Later we added more purpose in and generated a long list in wiki. While my personal suggestion is that we may want to separate the long list into different surveys, because people who take the survey will have different interest because of their background. One survey should try to target to one certain group, for us to make the observation to certain targets that we set. It will not good if you take a survey and found 1/3 of the questions inside are not what you interested, or even understand... then you might want to quit from the survey, or generate some garbage answers by random selections... It is just my $0.02. BTW, I noticed that in the question list, we didn't ask for people's career. We asked for the company, but not individual's. I suggest we add this question, since a developer is likely to have a quite different view to an executive... - Simon 2012/6/5 Kevin Grignon kevingrignon...@gmail.com KG02 - see comments inline. On Tuesday, June 5, 2012, Graham Lauder wrote: KG01 - see comments inline. On Tue, Jun 5, 2012 at 8:26 AM, Graham Lauder y...@apache.org javascript:; wrote: Hi. Questions relating to research! [] Perhaps the first survey we should conduct is a survey about what sort of surveys our users would respond to. KG01 - Thanks for your feedback and interest in the user research effort. While I agree we could deploy different types of surveys to gather different types of data, I feel that a survey of surveys might induce premature survey fatigue. Survey fatique has already set in, that is not a new thing, that is recognisable simply by those surveys conducted by SUN. We haven't caused that, it is a factor of the modern marketing malaise. The cost of incentives these days, that one needs to hand out to get a significant sampling in a timely manner is huge. KG02 - Agreed, risk of fatigue is a planning consideration. User research, especially surveys, consumes people's time and energy. Indeed as I myself pointed out earlier in this thread Rather, I propose we work from the other direction. If the goal of the research activity is to gather data that will help us build insight and drive informed design and development decisions, then we should focus the surveys on the information we need to do that. I have captured some comments in the wiki discussion page. Indeed, however if the sample of respondents is ridiculously small, as has historically been the case, then the data is useless. KG02 - Perhaps, let's think positive :) You cannot use corporate methodologies in an open source environment. We have no ability to offer incentives, we therefore need to make the survey process as pleasant and enjoyable as possible or we need to find out from people what would encourage them to participate. KG02 - Ok. I'm not advocating corporate or open source. I'm advocating that we create surveys that 1) will deliver good data and 2) people will fill out. That requires research, I doubt it will require as big a sample as a UX survey but that is only because there are a limited number of answers needed. Every good research organisation I have worked with does short surveys to find out what they're doing right or wrong. KG02 - indeed, a useful activity. For the most part they do these at the end of another survey, but that is because the group of respondents they are questioning will probably never do the same survey again. KG02 - While surveys are common after usability evaluation session, nested surveys are new to me. For us the problem has been getting respondents to finish. Lose them once and they won't come back again and we will need
Re: [UX] The Questions for users
KG01 - see comments inline. On Jun 6, 2012, at 3:01 AM, Dennis E. Hamilton dennis.hamil...@acm.org wrote: I am adding this here just because of the mention of career, profession, and employer. If I encountered a questionnaire that asked who my employer was, I would stop right there and not respond. It is an obvious asked-because-we-can useless question and impeaches the questionnaire source in my mind. If I encountered a questionnaire that asked for the nature of my employment (employed, retired, self-employed), I would respond to that. I would also answer a question about the industry my work was in. I would answer a question about the number of people in the organization where I work. I see no earthly purpose to knowing someone's employer and I don't believe it should be requested. KG01 - Agreed. Employer data provides no value to us and could be considered invasive. Role and industry data is more useful and valuable. As a reminder, the question candidates are a pool from which we can create the actual surveys. Final surveys will be subject to community approval. I'll add your comments to the wiki. - Dennis -Original Message- From: Shenfeng Liu [mailto:liush...@gmail.com] Sent: Monday, June 04, 2012 23:56 To: ooo-dev@incubator.apache.org Subject: Re: [UX] The Questions for users Firstly I strong agree with Graham's that we must make very clear on what we want to know from a survey. e.g. my first impression to Albino's original question list was that the purpose might be to know by which channels do our users get our product information and support... Later we added more purpose in and generated a long list in wiki. While my personal suggestion is that we may want to separate the long list into different surveys, because people who take the survey will have different interest because of their background. One survey should try to target to one certain group, for us to make the observation to certain targets that we set. It will not good if you take a survey and found 1/3 of the questions inside are not what you interested, or even understand... then you might want to quit from the survey, or generate some garbage answers by random selections... It is just my $0.02. BTW, I noticed that in the question list, we didn't ask for people's career. We asked for the company, but not individual's. I suggest we add this question, since a developer is likely to have a quite different view to an executive... - Simon 2012/6/5 Kevin Grignon kevingrignon...@gmail.com KG02 - see comments inline. On Tuesday, June 5, 2012, Graham Lauder wrote: KG01 - see comments inline. On Tue, Jun 5, 2012 at 8:26 AM, Graham Lauder y...@apache.org javascript:; wrote: Hi. Questions relating to research! [] Perhaps the first survey we should conduct is a survey about what sort of surveys our users would respond to. KG01 - Thanks for your feedback and interest in the user research effort. While I agree we could deploy different types of surveys to gather different types of data, I feel that a survey of surveys might induce premature survey fatigue. Survey fatique has already set in, that is not a new thing, that is recognisable simply by those surveys conducted by SUN. We haven't caused that, it is a factor of the modern marketing malaise. The cost of incentives these days, that one needs to hand out to get a significant sampling in a timely manner is huge. KG02 - Agreed, risk of fatigue is a planning consideration. User research, especially surveys, consumes people's time and energy. Indeed as I myself pointed out earlier in this thread Rather, I propose we work from the other direction. If the goal of the research activity is to gather data that will help us build insight and drive informed design and development decisions, then we should focus the surveys on the information we need to do that. I have captured some comments in the wiki discussion page. Indeed, however if the sample of respondents is ridiculously small, as has historically been the case, then the data is useless. KG02 - Perhaps, let's think positive :) You cannot use corporate methodologies in an open source environment. We have no ability to offer incentives, we therefore need to make the survey process as pleasant and enjoyable as possible or we need to find out from people what would encourage them to participate. KG02 - Ok. I'm not advocating corporate or open source. I'm advocating that we create surveys that 1) will deliver good data and 2) people will fill out. That requires research, I doubt it will require as big a sample as a UX survey but that is only because there are a limited number of answers needed. Every good research organisation I have worked with does short surveys to find out what they're doing right or wrong. KG02 - indeed, a useful activity
Re: [UX] The Questions for users
Hi. Questions relating to research! We aren't in order to adjust. Sorry long text. Legend: - Questions --Response options. *** - How old are you ? - What S.O you use ? --Linux --Mac --Windows --other [what] - Use of Linux ? Wich graphic interface: --GNOME --KDE --Lxde --Others [what] - Where do you use Apache OpenOffice? --Home --Office --Company --Telecenter --Others [whatl] -Where to get support ? --Manuals --Mailling list --Search --Friends --Others [what] -Where do you want to get support ? --Manuals --Mailing list --Search --Friends --Others [whatl] -How often do you use Apahce OpenOffice ? --Sometimes --With frequency --Daily - How much time you spend on the computer ? --30 min to 1 hora --1h to 3h --3h to 5h --More of 8h - As you consider using the computer ? --Beginner --intermediate --Advanced (expert) -How important computer for you: --unimportant --insignificant --Very Important -How do you consider a nice software? --With enough buttons --Buttons significant --Buttons simple and agile --Results --Buttons and good visual meanings. *** Accepted reviews and more questions. Each survey has to generate information that guides and informs decisions. For each question we should ask ourselves, why do we need to know this? What is the reason for this query. What advance - enhancement - feature will be generated or affected by a majority response for one particular selection. So each survey therefore, has to have a specific reason for being. We have come to a point where there are a number of possibilities. We need user input to steer us down the right road. For this questionairre we have not decided what these parameters are, we have the cart before the horse. Our users do us a huge favour by participating in a survey, for them it is a substantial cost in time and effort for no percievable reward. While it is necessary obviously to think of the value of the survey to the project we need to think about our users and what encourages them to take part and how do we make the survey a pleasurable experience for them. This is a statistic that is a warning about not considering our respondents. The old OOo had a registration thing at first run. It directed the user to a user survey. It was long and complex and badly targeted. Several hundred million downloads resulted in a few tens of thousands respondents, an almost unmeasurable fraction of one percent response. Perhaps the first survey we should conduct is a survey about what sort of surveys our users would respond to. Cheers GL PS I have made a few comments on the wiki page with regard to the questions there. Albino
Re: [UX] The Questions for users
Hi. Questions relating to research! We aren't in order to adjust. Sorry long text. Legend: - Questions --Response options. *** - How old are you ? - What S.O you use ? --Linux --Mac --Windows --other [what] - Use of Linux ? Wich graphic interface: --GNOME --KDE --Lxde --Others [what] - Where do you use Apache OpenOffice? --Home --Office --Company --Telecenter --Others [whatl] -Where to get support ? --Manuals --Mailling list --Search --Friends --Others [what] -Where do you want to get support ? --Manuals --Mailing list --Search --Friends --Others [whatl] -How often do you use Apahce OpenOffice ? --Sometimes --With frequency --Daily - How much time you spend on the computer ? --30 min to 1 hora --1h to 3h --3h to 5h --More of 8h - As you consider using the computer ? --Beginner --intermediate --Advanced (expert) -How important computer for you: --unimportant --insignificant --Very Important -How do you consider a nice software? --With enough buttons --Buttons significant --Buttons simple and agile --Results --Buttons and good visual meanings. *** Accepted reviews and more questions. Each survey has to generate information that guides and informs decisions. For each question we should ask ourselves, why do we need to know this? What is the reason for this query. What advance - enhancement - feature will be generated or affected by a majority response for one particular selection. So each survey therefore, has to have a specific reason for being. We have come to a point where there are a number of possibilities. We need user input to steer us down the right road. For this questionairre we have not decided what these parameters are, we have the cart before the horse. Our users do us a huge favour by participating in a survey, for them it is a substantial cost in time and effort for no percievable reward. While it is necessary obviously to think of the value of the survey to the project we need to think about our users and what encourages them to take part and how do we make the survey a pleasurable experience for them. This is a statistic that is a warning about not considering our respondents. The old OOo had a registration thing at first run. It directed the user to a user survey. It was long and complex and badly targeted. Several hundred million downloads resulted in a few tens of thousands respondents, an almost unmeasurable fraction of one percent response. Perhaps the first survey we should conduct is a survey about what sort of surveys our users would respond to. Cheers GL PS I have made a few comments on the wiki page with regard to the questions there. Albino
Re: [UX] The Questions for users
Hi. Questions relating to research! [] Perhaps the first survey we should conduct is a survey about what sort of surveys our users would respond to. Cheers GL PS I have made a few comments on the wiki page with regard to the questions there. Discussion on Delivery Methodologies Unfortunately probably our best way of reaching OOo users, the OOo mail redirect, has been taken away (Although the value of that list could be debatable) so we have to come up with different methods. We have basically two demographics that we have simple access to: New Users and Experienced Users. New User contact: At Download, from a redirect from the download site At install: a link to a survey to do while AOO is installing Immediately Post install Experienced user contact: As for the New User Via the announce list from the Home Page (Add in another link I am willing to help with AOO User Research) At upgrade as part of the process This in addition to posting on all lists obviously It should go without saying that the user should be able to opt-out at any point Discuss either here or on the wiki http://wiki.services.openoffice.org/wiki/Talk:AOO_UX_Research_Surveys Cheers GL
Re: [UX] The Questions for users
KG01 - see comments inline. On Tue, Jun 5, 2012 at 8:26 AM, Graham Lauder y...@apache.org wrote: Hi. Questions relating to research! [] Perhaps the first survey we should conduct is a survey about what sort of surveys our users would respond to. KG01 - Thanks for your feedback and interest in the user research effort. While I agree we could deploy different types of surveys to gather different types of data, I feel that a survey of surveys might induce premature survey fatigue. User research, especially surveys, consumes people's time and energy. Rather, I propose we work from the other direction. If the goal of the research activity is to gather data that will help us build insight and drive informed design and development decisions, then we should focus the surveys on the information we need to do that. I have captured some comments in the wiki discussion page. http://wiki.services.openoffice.org/wiki/Talk:AOO_UX_Research_Surveys Cheers GL PS I have made a few comments on the wiki page with regard to the questions there. Discussion on Delivery Methodologies Unfortunately probably our best way of reaching OOo users, the OOo mail redirect, has been taken away (Although the value of that list could be debatable) so we have to come up with different methods. We have basically two demographics that we have simple access to: New Users and Experienced Users. New User contact: At Download, from a redirect from the download site At install: a link to a survey to do while AOO is installing Immediately Post install Experienced user contact: As for the New User Via the announce list from the Home Page (Add in another link I am willing to help with AOO User Research) At upgrade as part of the process This in addition to posting on all lists obviously It should go without saying that the user should be able to opt-out at any point Discuss either here or on the wiki http://wiki.services.openoffice.org/wiki/Talk:AOO_UX_Research_Surveys Cheers GL
Re: [UX] The Questions for users
Additional comments inline. See KG02. On Tue, Jun 5, 2012 at 10:49 AM, Kevin Grignon kevingrignon...@gmail.comwrote: KG01 - see comments inline. On Tue, Jun 5, 2012 at 8:26 AM, Graham Lauder y...@apache.org wrote: Hi. Questions relating to research! [] Perhaps the first survey we should conduct is a survey about what sort of surveys our users would respond to. KG01 - Thanks for your feedback and interest in the user research effort. While I agree we could deploy different types of surveys to gather different types of data, I feel that a survey of surveys might induce premature survey fatigue. User research, especially surveys, consumes people's time and energy. Rather, I propose we work from the other direction. If the goal of the research activity is to gather data that will help us build insight and drive informed design and development decisions, then we should focus the surveys on the information we need to do that. I have captured some comments in the wiki discussion page. http://wiki.services.openoffice.org/wiki/Talk:AOO_UX_Research_Surveys Cheers GL PS I have made a few comments on the wiki page with regard to the questions there. Discussion on Delivery Methodologies Unfortunately probably our best way of reaching OOo users, the OOo mail redirect, has been taken away (Although the value of that list could be debatable) so we have to come up with different methods. We have basically two demographics that we have simple access to: New Users and Experienced Users. New User contact: At Download, from a redirect from the download site At install: a link to a survey to do while AOO is installing Immediately Post install Experienced user contact: As for the New User Via the announce list from the Home Page (Add in another link I am willing to help with AOO User Research) At upgrade as part of the process This in addition to posting on all lists obviously It should go without saying that the user should be able to opt-out at any point Discuss either here or on the wiki http://wiki.services.openoffice.org/wiki/Talk:AOO_UX_Research_Surveys KG02 - Graham, I can log into and edit the page, but cannot login to or edit the comments. Thoughts? Cheers GL
Re: [UX] The Questions for users
Additional comments inline. See KG02. On Tue, Jun 5, 2012 at 10:49 AM, Kevin Grignon kevingrignon...@gmail.comwrote: KG01 - see comments inline. On Tue, Jun 5, 2012 at 8:26 AM, Graham Lauder y...@apache.org wrote: Hi. Questions relating to research! [] Perhaps the first survey we should conduct is a survey about what sort of surveys our users would respond to. KG01 - Thanks for your feedback and interest in the user research effort. While I agree we could deploy different types of surveys to gather different types of data, I feel that a survey of surveys might induce premature survey fatigue. User research, especially surveys, consumes people's time and energy. Rather, I propose we work from the other direction. If the goal of the research activity is to gather data that will help us build insight and drive informed design and development decisions, then we should focus the surveys on the information we need to do that. I have captured some comments in the wiki discussion page. http://wiki.services.openoffice.org/wiki/Talk:AOO_UX_Research_Surveys Cheers GL PS I have made a few comments on the wiki page with regard to the questions there. Discussion on Delivery Methodologies Unfortunately probably our best way of reaching OOo users, the OOo mail redirect, has been taken away (Although the value of that list could be debatable) so we have to come up with different methods. We have basically two demographics that we have simple access to: New Users and Experienced Users. New User contact: At Download, from a redirect from the download site At install: a link to a survey to do while AOO is installing Immediately Post install Experienced user contact: As for the New User Via the announce list from the Home Page (Add in another link I am willing to help with AOO User Research) At upgrade as part of the process This in addition to posting on all lists obviously It should go without saying that the user should be able to opt-out at any point Discuss either here or on the wiki http://wiki.services.openoffice.org/wiki/Talk:AOO_UX_Research_Surveys KG02 - Graham, I can log into and edit the page, but cannot login to or edit the comments. Thoughts? Mediawiki sometimes plays silly buggers with login. If you are logged but your ID is not at the top of the page then look for the View Source tag click that and it seems to take you to the editable page Cheers G Cheers GL
Re: [UX] The Questions for users
KG01 - see comments inline. On Tue, Jun 5, 2012 at 8:26 AM, Graham Lauder y...@apache.org wrote: Hi. Questions relating to research! [] Perhaps the first survey we should conduct is a survey about what sort of surveys our users would respond to. KG01 - Thanks for your feedback and interest in the user research effort. While I agree we could deploy different types of surveys to gather different types of data, I feel that a survey of surveys might induce premature survey fatigue. Survey fatique has already set in, that is not a new thing, that is recognisable simply by those surveys conducted by SUN. We haven't caused that, it is a factor of the modern marketing malaise. The cost of incentives these days, that one needs to hand out to get a significant sampling in a timely manner is huge. User research, especially surveys, consumes people's time and energy. Indeed as I myself pointed out earlier in this thread Rather, I propose we work from the other direction. If the goal of the research activity is to gather data that will help us build insight and drive informed design and development decisions, then we should focus the surveys on the information we need to do that. I have captured some comments in the wiki discussion page. Indeed, however if the sample of respondents is ridiculously small, as has historically been the case, then the data is useless. You cannot use corporate methodologies in an open source environment. We have no ability to offer incentives, we therefore need to make the survey process as pleasant and enjoyable as possible or we need to find out from people what would encourage them to participate. That requires research, I doubt it will require as big a sample as a UX survey but that is only because there are a limited number of answers needed. Every good research organisation I have worked with does short surveys to find out what they're doing right or wrong. For the most part they do these at the end of another survey, but that is because the group of respondents they are questioning will probably never do the same survey again. For us the problem has been getting respondents to finish. Lose them once and they won't come back again and we will need to talk to our user community if not often, at least regularly I would prefer to do things right first time up so people will happily respond to any surveys we need to put out. Remember that there are not only UX surveys to be done but Marketing as well. We know already know two things that get people to complete surveys: Brevity and Fun. If we do a light hearted, quick survey that gives us the reasons that people will participate, I think that's a really good use of resources. The Surveys already put up are boring, generic and not likely to inspire people to complete them. OOo has a user base in the hundreds of millions a few hundred completions is not a sample. We need 10s of thousands of responses across scores of languages, to get a easonable sample. So first we need to figure out how to get that sample. Cheers G
Re: [UX] The Questions for users
Hi. Thanks for suggestions, anyone else. Let's add them. Albino
Re: [UX] The Questions for users
Hi, I hope you can have draft which reflects suggestions so far. I am too lazy to go through all previous mails to figure out what it looks like now.. Ps, I just go through previous mails. Shall uses' expectation/suggestion be added to this list? - Stability - More functions - Easy to use - Better Development Guide/Experience through UNO API (so on) Thanks a lot for your great work/effort for it! Regards 2012/6/3 Albino B Neto bino...@gmail.com Hi. Thanks for suggestions, anyone else. Let's add them. Albino
Re: [UX] The Questions for users
On Fri, Jun 1, 2012 at 6:42 PM, Albino B Neto bino...@gmail.com wrote: Hi. Questions relating to research! We aren't in order to adjust. Sorry long text. Legend: - Questions --Response options. Will this question be international? or is it local? An international survey might have questions about country and language. *** - How old are you ? Maybe ask for sex? - What S.O you use ? Is this a single choice or multiple choice question? Many people use more than one. So if the intent is a single choice, you might word the question as: what is your primary operating system? --Linux --Mac --Windows --other [what] - Use of Linux ? Wich graphic interface: --GNOME --KDE --Lxde --Others [what] - Where do you use Apache OpenOffice? Again, is this single choice or multiple To me the choices Office and Company seem to be the same. So there is a risk in the survey that users will randomly split between those two choices. Better to have a smaller number of choices that are clearly different, like: Home, Work, School, Other. --Home --Office --Company --Telecenter --Others [whatl] -Where to get support ? --Manuals --Mailling list --Search --Friends --Others [what] -Where do you want to get support ? --Manuals --Mailing list --Search --Friends --Others [whatl] -How often do you use Apahce OpenOffice ? The daily choice is quantifiable. with frequency is not. I don't know if it is more or less than daily. --Sometimes --With frequency --Daily - How much time you spend on the computer ? per day --30 min to 1 hora --1h to 3h --3h to 5h --More of 8h - As you consider using the computer ? --Beginner --intermediate --Advanced (expert) -How important computer for you: --unimportant --insignificant --Very Important -How do you consider a nice software? This might be a good place to ask a series of questions along the lines of: How important are each of the following to you. Score 1-5, where 1 is not important at all and 5 is very important 1. Easy to use without reading manual 2. Interoperability with Microsoft file formats 3. Open source, built and supported by a volunteer community 4. Is available in my language 5. Speed 6. Availability of free document templates 7. Availability of extensions And so on. The nice thing about this approach is you can then correlate the responses for each of these questions with the demographic info (how often do you use your computer, where do you use AOO, etc.) That is pretty much the standard form of an opinion survey: 1) A section of demographic questions that are factual statements about the user 2) A section of opinion questions The analysis can then correlate the opinions to the demographics, and say things like, Users who used AOO more than once a day were most concerned about performance, but those who use it only once a month are more concerned about usability. --With enough buttons --Buttons significant --Buttons simple and agile --Results --Buttons and good visual meanings. *** Accepted reviews and more questions. Albino
Re: [UX] The Questions for users
Hi, some notes inside. Albino B Neto schrieb: Hi. Questions relating to research! We aren't in order to adjust. Sorry long text. Legend: - Questions --Response options. *** - How old are you ? Why this question? - What S.O you use ? --Linux --Mac --Windows --other [what] The user might use different OS depending where he works, for example Windows on the job and Linux at home. Do you want to know the OS of the machine the user is currently using or the machine he is mostly using? - Use of Linux ? Wich graphic interface: --GNOME --KDE --Lxde --Others [what] We want to know, whether Win98 is still used. So after answer Windows a question for the type of Windows should follow. - Where do you use Apache OpenOffice? --Home --Office --Company --Telecenter --Others [whatl] What is the difference between 'Office' and 'Company'? What is a 'Telecenter'? I miss 'puplic PC in school, university, or library'. Again, do you mean the machine the user currently uses or the machine he is mostly using? -Where to get support ? --Manuals --Mailling list --Search --Friends --Others [what] 'Forum' is missing. -Where do you want to get support ? --Manuals --Mailing list --Search --Friends --Others [whatl] 'Forum' is missing. -How often do you use Apahce OpenOffice ? --Sometimes --With frequency --Daily What is 'Sometimes' and 'With frequency'? What about a question When did you use AOO the last time?' - How much time you spend on the computer ? --30 min to 1 hora --1h to 3h --3h to 5h --More of 8h There might be a large difference between workdays and weekend. - As you consider using the computer ? --Beginner --intermediate --Advanced (expert) -How important computer for you: --unimportant --insignificant --Very Important Both will mostly give the middle answer. Try to find four answers. What do you want to learn from these questions? -How do you consider a nice software? --With enough buttons --Buttons significant --Buttons simple and agile --Results --Buttons and good visual meanings. *** Why the focus on buttons? You might ask which other software the user thinks is 'nice' and why. Accepted reviews and more questions. Questions for the tasks, which where solved with Apache OpenOffice. Question how the commands are used: menu via mouse/touch pad, buttons via mouse, keyboard short cuts, menu via keyboard Some questions make no sense, if the user installs Apache OpenOffice the first time. So a switch might be necessary. Kind regards Regina
