Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 03:34, Zeeshan Ali (Khattak) zeesha...@gnome.orgwrote: [snip] Maybe they all lied? Don't you think it is a bit early to speculate on results? (...) Overall I can see already one clear result, even before the poll has started: We do not know who is using GNOME. Maybe this needs reflexion ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Thu, Aug 18, 2011 at 19:53, Stormy Peters sto...@gnome.org wrote: All the questions after this assume a knowledge of GNOME and how our community works. That's fine if you are polling developers. If you are polling average users, then I think it's not worth asking. Another issue that I don't think has been raised yet (sorry if it has, this thread reached the limits a while ago) is that the changes we make to GNOME aren't intended just for existing GNU/Linux users or people with similar interests. My understanding is that we want GNOME to be useful to people that haven't been attracted yet to any of the existing free desktops (most of the people in this world). Somewhat related, you need to take into account people's natural resistance to change. A factor that isn't equally relevant when surveying future users as it is when surveying present ones. Regards, Tomeu ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Rethinking GnomeGoals
Hello hackers GnomeGoals have been rather stale lately. Im willing to invest some time in bringing it up to date. Right now it seems like the included modules is a mix of old definitions (desktop, platform etc). Some modules are even deprecated by now. Would it be feasible and advantageous to move the GnomeGoals to a structure based on the new module definitions like suites-core, apps, world? ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 1:20 AM, Zeeshan Ali (Khattak) zeesha...@gnome.org wrote: On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras felipe.contre...@gmail.com wrote: Nothing is ever perfect, but having at least some results is better than nothing. Since you have repeated this assertion a few times, I must ask: What if the results are all wrong and we don't have any way of knowing that? Would those results still be better than nothing in your opinion? What do you mean by all wrong? Let's assume that the results show that 1000 people are not happy with GNOME. How can that be wrong? 1000 people responded that, the results were not somehow altered, or boycotted, the results are the results, and that's that. ... 'Wrong' in social research typically means that your results lack validity: that you think the data is measuring one thing (eg. 'GNOME users' happiness with GNOME 3') but is in fact measuring something else. When you do survey research, you have to be certain that one person understands the questions in the same way that another person does. Looking at your questionnaire, that won't be the case. An example: === 02. Overall, how happy are you with GNOME? === (single choice) * unhappy * not so happy * happy * very happy * completely ecstatic Different people will understand the words GNOME/happy/very happy/ecstatic in different ways. Some might think 'GNOME' is their distro (including the lower levels of the stack), some that it's their 'shell', others that it's all their GUI software [1]. Likewise, 'happy' will be thought of differently by different people (a very odd word to include in a questionnaire, if you don't mind me saying): there are a vast range of expectations and usage patterns in relation to desktop computers, all of which will affect how people respond. Someone could tick 'unhappy' but by most measures have had a perfectly satisfactory experience. You've also got the representativeness problem. Your sample will inevitably be unrepresentative, probably highly so. Even if 100% of your *unrepresentative sample* tick the unhappy box, that doesn't tell you much about your target population: you might just have sampled every 'unhappy' GNOME user that's out there. tl;dr version: your survey results will be misleading. We already have a wealth of data about peoples' experiences with GNOME 3 and are working to address the issues that are being raised. It's great that you want to help, but this survey really won't be useful. Allan [1] GNOME's place in the stack means that you can't really do satisfaction surveys on it. This is one reason why GNOME is a more difficult research topic than, say, Git. -- IRC: aday on irc.gnome.org Blog: http://afaikblog.wordpress.com/ ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Rethinking GnomeGoals
On 19 August 2011 00:05, Djohn Heist djohnhe...@yahoo.com wrote: Hello hackers Helllo Djohn, Would it be feasible and advantageous to move the GnomeGoals to a structure based on the new module definitions like suites-core, apps, world? Indeed, It would really great if you can help with this. I think a good start would be the template file [1] Thanks for your work! [1] https://live.gnome.org/GnomeGoals/Template -- Javier Jardón Cabezas ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 3:34 AM, Zeeshan Ali (Khattak) zeesha...@gnome.org wrote: On Fri, Aug 19, 2011 at 2:09 AM, Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 1:20 AM, Zeeshan Ali (Khattak) zeesha...@gnome.org wrote: On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras felipe.contre...@gmail.com wrote: Nothing is ever perfect, but having at least some results is better than nothing. Since you have repeated this assertion a few times, I must ask: What if the results are all wrong and we don't have any way of knowing that? Would those results still be better than nothing in your opinion? What do you mean by all wrong? Let's assume that the results show that 1000 people are not happy with GNOME. How can that be wrong? Maybe they all lied? Maybe people who are satisfied do not want to or have time to take part in surveys and you only get people who are not happy into the survey? In which case, the results may show results that are not correct. i-e a significantly large number of participant say that they are very unhappy with GNOME but what if that number is nothing compared to the number of people who are very much satisfied with GNOME? I didn't say this so far because it might sound like I am trying to make a joke but since you still insist on your assertions about the survey, I feel I must say this: How do you know people in general like to participate in surveys? It is my observation that most people do not like to do that, unless they have something to complain about. Now this observation of mine could very well be wrong but how do we know that? Do we do a survey to find out if people like to participate in surveys? Are you serious? That totally and completely speculative and unrealistic. Have you ever participated in making a survey? I have, as I have explained, for the Git survey. In my experience, only the people that want to help in some way do spend the amount of time required to fill the survey. But again, as I said, if there's no survey on Earth you could trust, just ignore the results. Results by themselves cannot hurt you. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 1:14 PM, Allan Day allanp...@gmail.com wrote: Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 1:20 AM, Zeeshan Ali (Khattak) zeesha...@gnome.org wrote: On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras felipe.contre...@gmail.com wrote: Nothing is ever perfect, but having at least some results is better than nothing. Since you have repeated this assertion a few times, I must ask: What if the results are all wrong and we don't have any way of knowing that? Would those results still be better than nothing in your opinion? What do you mean by all wrong? Let's assume that the results show that 1000 people are not happy with GNOME. How can that be wrong? 1000 people responded that, the results were not somehow altered, or boycotted, the results are the results, and that's that. ... 'Wrong' in social research typically means that your results lack validity: that you think the data is measuring one thing (eg. 'GNOME users' happiness with GNOME 3') but is in fact measuring something else. When you do survey research, you have to be certain that one person understands the questions in the same way that another person does. Looking at your questionnaire, that won't be the case. An example: === 02. Overall, how happy are you with GNOME? === (single choice) * unhappy * not so happy * happy * very happy * completely ecstatic Different people will understand the words GNOME/happy/very happy/ecstatic in different ways. Some might think 'GNOME' is their distro (including the lower levels of the stack), Which is why we ask more question to understand their level of geekness. That should help the make correlations; the people that use a terminal all the time more likely know that GNOME is just the DE. The people that don't have much experience might be confusing GNOME with the distribution. Likewise, 'happy' will be thought of differently by different people (a very odd word to include in a questionnaire, if you don't mind me saying): I think everyone understands the word happy. That is what is used in Git user survey, and seems to be doing the job just fine. In any case, if you have suggestions that don't have these problems, feel free to share them. You've also got the representativeness problem. Your sample will inevitably be unrepresentative, probably highly so. Even if 100% of your *unrepresentative sample* tick the unhappy box, that doesn't tell you much about your target population: you might just have sampled every 'unhappy' GNOME user that's out there. If you can identify the bias, that's not a huge problem. tl;dr version: your survey results will be misleading. No, the results would not be misleading; the *analysis* of the results might. But different people can analyze them in different ways. The important thing is to get *some* results. We already have a wealth of data about peoples' experiences with GNOME 3 and are working to address the issues that are being raised. It's great that you want to help, but this survey really won't be useful. Where? I haven't seen any. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 1:17 PM, Felipe Contreras felipe.contre...@gmail.com wrote: Are you serious? That totally and completely speculative and unrealistic. Have you ever participated in making a survey? I have, as I have explained, for the Git survey. In my experience, only the people that want to help in some way do spend the amount of time required to fill the survey. Could you at least make the answer options less emotional? Like exchange happy for satisfied etc. I don't remember answering ecstatic in the Git survey but that could be my bad memory. -- Patryk Zawadzki I solve problems. ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 2:25 PM, Patryk Zawadzki pat...@pld-linux.org wrote: On Fri, Aug 19, 2011 at 1:17 PM, Felipe Contreras felipe.contre...@gmail.com wrote: Are you serious? That totally and completely speculative and unrealistic. Have you ever participated in making a survey? I have, as I have explained, for the Git survey. In my experience, only the people that want to help in some way do spend the amount of time required to fill the survey. Could you at least make the answer options less emotional? Like exchange happy for satisfied etc. I don't remember answering ecstatic in the Git survey but that could be my bad memory. That's a reasonable alternative. How about pleased? Any other people have an opinion? -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 2:17 PM, Felipe Contreras felipe.contre...@gmail.com wrote: I didn't say this so far because it might sound like I am trying to make a joke but since you still insist on your assertions about the survey, I feel I must say this: How do you know people in general like to participate in surveys? It is my observation that most people do not like to do that, unless they have something to complain about. Now this observation of mine could very well be wrong but how do we know that? Do we do a survey to find out if people like to participate in surveys? Are you serious? That totally and completely speculative and unrealistic. What is speculative? I made it very clear that it is *my* observation and *if* it is correct, the results of this survey may very well be wrong. Do you have any evidence that suggests that my observations above are incorrect? Have you ever participated in making a survey? No I have not but that does not necessarily mean what I said is incorrect and could just be ignored by pointing to examples of other surveys. If other people are ignoring an important issue, doesn't mean we should do the same. I have, as I have explained, for the Git survey. In my experience, only the people that want to help in some way do spend the amount of time required to fill the survey. As I have explained to you many times before, git's user-base is mostly (if not all) geeks and those geeks know where the mailing-list is and be able to access the survey easily. Still, I am a geek and a very happy user of git but I didn't even know about the existence of this survey until you told me. Even then, I didn't care to participate. I am pretty sure I would have cared to participate if I had something to complain about its current or planned features. GNOME's user-base consists of people who do not even know what GNOME is so many of them will not be able to participate, especially if they are happy users. In short, example of git surveys are quite irrelevant here. But again, as I said, if there's no survey on Earth you could trust, just ignore the results. Results by themselves cannot hurt you. In this case those results will really hurt since then you will have some numbers to back-up your claim of GNOME 3 is completely unusable. *If* your motivation for this survey has remained the same, you'll spread a lot of negative propaganda (which you already did even when you didn't have any numbers) and many people will just say Oh, people don't like this gnome 3 thingie, must be shit and will stay away from it. Even if you don't do that, there is many others who will use this data in that way. -- Regards, Zeeshan Ali (Khattak) FSF member#5124 ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 14:33, Felipe Contreras felipe.contre...@gmail.comwrote: On Fri, Aug 19, 2011 at 2:25 PM, Patryk Zawadzki pat...@pld-linux.org wrote: On Fri, Aug 19, 2011 at 1:17 PM, Felipe Contreras felipe.contre...@gmail.com wrote: Are you serious? That totally and completely speculative and unrealistic. Have you ever participated in making a survey? I have, as I have explained, for the Git survey. In my experience, only the people that want to help in some way do spend the amount of time required to fill the survey. Could you at least make the answer options less emotional? Like exchange happy for satisfied etc. I don't remember answering ecstatic in the Git survey but that could be my bad memory. That's a reasonable alternative. How about pleased? Any other people have an opinion? satisfied is good -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 10:12 AM, Luc Pionchon pionchon@gmail.com wrote: On Fri, Aug 19, 2011 at 03:34, Zeeshan Ali (Khattak) zeesha...@gnome.org wrote: [snip] Maybe they all lied? Don't you think it is a bit early to speculate on results? (...) Overall I can see already one clear result, even before the poll has started: We do not know who is using GNOME. Maybe this needs reflexion No, in the current scenario that is actually a success story. Maybe this will change when GNOME OS becomes a reality. -- Regards, Zeeshan Ali (Khattak) FSF member#5124 ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 13:14, Allan Day allanp...@gmail.com wrote: We already have a wealth of data about peoples' experiences with GNOME 3 Allan, this is interesting, what is the main pointer to access this data? ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 2:40 PM, Zeeshan Ali (Khattak) zeesha...@gnome.org wrote: On Fri, Aug 19, 2011 at 2:17 PM, Felipe Contreras felipe.contre...@gmail.com wrote: I didn't say this so far because it might sound like I am trying to make a joke but since you still insist on your assertions about the survey, I feel I must say this: How do you know people in general like to participate in surveys? It is my observation that most people do not like to do that, unless they have something to complain about. Now this observation of mine could very well be wrong but how do we know that? Do we do a survey to find out if people like to participate in surveys? Are you serious? That totally and completely speculative and unrealistic. What is speculative? I made it very clear that it is *my* observation and *if* it is correct, the results of this survey may very well be wrong. Do you have any evidence that suggests that my observations above are incorrect? Do you even know what speculation means? to make an inference based on inconclusive evidence; to surmise or conjecture [1] You don't have any evidence how often does this happens in real surveys, if at all. It's all based on conjecture. Have you ever participated in making a survey? No I have not but that does not necessarily mean what I said is incorrect and could just be ignored by pointing to examples of other surveys. If other people are ignoring an important issue, doesn't mean we should do the same. You are again going off-tracks. Let's go back to the point. You said: What if the results are all wrong and we don't have any way of knowing that? Would those results still be better than nothing in your opinion? You can ignore the results. Problem solved, is it not? I have, as I have explained, for the Git survey. In my experience, only the people that want to help in some way do spend the amount of time required to fill the survey. As I have explained to you many times before, git's user-base is mostly (if not all) geeks and those geeks know where the mailing-list is and be able to access the survey easily. Still, I am a geek and a very happy user of git but I didn't even know about the existence of this survey until you told me. Even then, I didn't care to participate. I am pretty sure I would have cared to participate if I had something to complain about its current or planned features. GNOME's user-base consists of people who do not even know what GNOME is so many of them will not be able to participate, especially if they are happy users. In short, example of git surveys are quite irrelevant here. So what if that's true? (I don't think so) At least I have a data-point of experience with surveys, you can discard it all you want, but what makes your speculation based on imaginary notions somehow more valid that my experience in real-world scenarios? At best you can say that they are both equally useless (I don't think so). The world is no filled with Zeeshans. Most people fill surveys truthfully. If you think otherwise, you can ignore the results. But again, as I said, if there's no survey on Earth you could trust, just ignore the results. Results by themselves cannot hurt you. In this case those results will really hurt since then you will have some numbers to back-up your claim of GNOME 3 is completely unusable. *If* your motivation for this survey has remained the same, you'll spread a lot of negative propaganda (which you already did even when you didn't have any numbers) and many people will just say Oh, people don't like this gnome 3 thingie, must be shit and will stay away from it. Even if you don't do that, there is many others who will use this data in that way. Aha, so that's what you are afraid. This survey will happen with or without GNOME's blessing. It would be in GNOME's best interest to improve the survey to get more useful results, and so far, I think many people have done so. However, at this point it's clear that you are not interested in improving the survey, all you are doing is making imaginary claims that lead to a dead-end; all surveys are pointless, because the answers might be lies. There's no way to go forward from there. If you have some *suggestions* how to improve the survey to avoid whatever issues you see, then say so, otherwise I'll not explain any more why this is flawed thinking that leads nowhere. Cheers. [1] http://en.wiktionary.org/wiki/speculate -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 1:14 PM, Allan Day allanp...@gmail.com wrote: Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 1:20 AM, Zeeshan Ali (Khattak) zeesha...@gnome.org wrote: On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras felipe.contre...@gmail.com wrote: ... Different people will understand the words GNOME/happy/very happy/ecstatic in different ways. Some might think 'GNOME' is their distro (including the lower levels of the stack), Which is why we ask more question to understand their level of geekness. That should help the make correlations; the people that use a terminal all the time more likely know that GNOME is just the DE. The people that don't have much experience might be confusing GNOME with the distribution. 'Geekness' is not the only thing that will affect people's understandings, and you haven't adequately measured that anyway. Plus that doesn't do anything to deal with the problem of what people understand by 'GNOME'. Likewise, 'happy' will be thought of differently by different people (a very odd word to include in a questionnaire, if you don't mind me saying): I think everyone understands the word happy. /ME wipes a mouthful of coffee from my monitor Then you haven't read enough of the survey research literature. ... In any case, if you have suggestions that don't have these problems, feel free to share them. My suggestion would be to give up entirely or to rethink the premise of your research. The latter is what I'd have advised when I was working as a research consultant, or what I would have told one of my students when I used to teach this stuff, for that matter. You've also got the representativeness problem. Your sample will inevitably be unrepresentative, probably highly so. Even if 100% of your *unrepresentative sample* tick the unhappy box, that doesn't tell you much about your target population: you might just have sampled every 'unhappy' GNOME user that's out there. If you can identify the bias, that's not a huge problem. So tell me - how will you accurately compensate for the effects of self-selection bias? What kinds of claims will you make about representativeness? tl;dr version: your survey results will be misleading. No, the results would not be misleading; the *analysis* of the results might. But different people can analyze them in different ways. The important thing is to get *some* results. It seems bizarre to suggest that research data is valid irrespective of how it is gathered. If your questionnaire does not provide valid measurements no amount of analysis can compensate. We already have a wealth of data about peoples' experiences with GNOME 3 and are working to address the issues that are being raised. It's great that you want to help, but this survey really won't be useful. Where? I haven't seen any. We've had incredible amounts of feedback; most (if not all) of which has been read, and which does get taken seriously. I also know that those of us who are influencing the design of GNOME 3 take a strong interest in peoples' experiences with it and ask them questions (that's certainly what I do). There's also a small series of user tests last I did Christmas, the results of which have been fed into the development process. Believe me, that is more than enough to be going on for now. (Some more user testing would be useful at some point in the future, though.) Allan -- IRC: aday on irc.gnome.org Blog: http://afaikblog.wordpress.com/ ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 3:17 PM, Allan Day allanp...@gmail.com wrote: Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 1:14 PM, Allan Day allanp...@gmail.com wrote: Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 1:20 AM, Zeeshan Ali (Khattak) zeesha...@gnome.org wrote: On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras felipe.contre...@gmail.com wrote: ... Different people will understand the words GNOME/happy/very happy/ecstatic in different ways. Some might think 'GNOME' is their distro (including the lower levels of the stack), Which is why we ask more question to understand their level of geekness. That should help the make correlations; the people that use a terminal all the time more likely know that GNOME is just the DE. The people that don't have much experience might be confusing GNOME with the distribution. 'Geekness' is not the only thing that will affect people's understandings, and you haven't adequately measured that anyway. Plus that doesn't do anything to deal with the problem of what people understand by 'GNOME'. It's easy to throw empty criticism. Provide *suggestions*. Likewise, 'happy' will be thought of differently by different people (a very odd word to include in a questionnaire, if you don't mind me saying): I think everyone understands the word happy. /ME wipes a mouthful of coffee from my monitor Then you haven't read enough of the survey research literature. That doesn't change the fact that everyone understands the word happy. ... In any case, if you have suggestions that don't have these problems, feel free to share them. My suggestion would be to give up entirely or to rethink the premise of your research. The latter is what I'd have advised when I was working as a research consultant, or what I would have told one of my students when I used to teach this stuff, for that matter. That's not helpful. If you are such a master, surely you can come up with a totally brand new user survey that is order of magnitude better. That would be greatly appreciated. You've also got the representativeness problem. Your sample will inevitably be unrepresentative, probably highly so. Even if 100% of your *unrepresentative sample* tick the unhappy box, that doesn't tell you much about your target population: you might just have sampled every 'unhappy' GNOME user that's out there. If you can identify the bias, that's not a huge problem. So tell me - how will you accurately compensate for the effects of self-selection bias? What kinds of claims will you make about representativeness? What would *you* do? tl;dr version: your survey results will be misleading. No, the results would not be misleading; the *analysis* of the results might. But different people can analyze them in different ways. The important thing is to get *some* results. It seems bizarre to suggest that research data is valid irrespective of how it is gathered. If your questionnaire does not provide valid measurements no amount of analysis can compensate. You can thrown an analysis saying all this data is crap if that makes you happier, but this survey won't eat babies. We already have a wealth of data about peoples' experiences with GNOME 3 and are working to address the issues that are being raised. It's great that you want to help, but this survey really won't be useful. Where? I haven't seen any. We've had incredible amounts of feedback; most (if not all) of which has been read, and which does get taken seriously. I also know that those of us who are influencing the design of GNOME 3 take a strong interest in peoples' experiences with it and ask them questions (that's certainly what I do). There's also a small series of user tests last I did Christmas, the results of which have been fed into the development process. Believe me, that is more than enough to be going on for now. (Some more user testing would be useful at some point in the future, though.) For a professor you should know better. I want the data. Anyway, I am going to ignore your comments, unless you provide some *suggestions* for improvement. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 4:08 PM, Andy Wingo wi...@pobox.com wrote: On Fri 19 Aug 2011 13:33, Felipe Contreras felipe.contre...@gmail.com writes: That's a reasonable alternative. How about pleased? Any other people have an opinion? You present yourself as reasonable by adjusting on the small points, but you ignore the feedback of greater importance. My opinion is that you are not the right person to lead an effort to gather feedback on GNOME. Is there anyone in the universe able to create a user survey worthy of GNOME? Can you convince him of doing so? In the meantime, this is the best that we have. I will continue listening for constructive feedback, but comments such as this is not good, you are doing it wrong, it's impossible, lead to nowhere. Besides, as Alan Cox said, it doesn't have to be perfect, like software, we can learn from the mistakes of the 2011 survey, and make a better one for 2012. Can we not? -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri 19 Aug 2011 13:33, Felipe Contreras felipe.contre...@gmail.com writes: That's a reasonable alternative. How about pleased? Any other people have an opinion? You present yourself as reasonable by adjusting on the small points, but you ignore the feedback of greater importance. My opinion is that you are not the right person to lead an effort to gather feedback on GNOME. The Git survey, AFAIU, was done _with_ the git developers. This one, if you manage to bully it through, will be _in spite of_ the GNOME developers. It will not have the effect you desire. Andy -- http://wingolog.org/ ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 2:13 PM, Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 4:08 PM, Andy Wingo wi...@pobox.com wrote: On Fri 19 Aug 2011 13:33, Felipe Contreras felipe.contre...@gmail.com writes: That's a reasonable alternative. How about pleased? Any other people have an opinion? You present yourself as reasonable by adjusting on the small points, but you ignore the feedback of greater importance. My opinion is that you are not the right person to lead an effort to gather feedback on GNOME. Is there anyone in the universe able to create a user survey worthy of GNOME? Can you convince him of doing so? Gathering feedback does not necessarily require an online user survey. As stated, for a project which currently targets, among others, users who do not care what parts of their operating system can be labelled GNOME a survey is not a very reliable way of gathering feedback. Have you ever tried to explain, to a person who doesn't have an interest in software, what GNOME actually is? In the meantime, this is the best that we have. I will continue listening for constructive feedback, but comments such as this is not good, you are doing it wrong, it's impossible, lead to nowhere. Besides, as Alan Cox said, it doesn't have to be perfect, like software, we can learn from the mistakes of the 2011 survey, and make a better one for 2012. Can we not? I urge you to consider the fact that if the majority of people subscribed to desktop-devel-list don't have faith in idea of an online user survey, an online user survey is probably not going to much have effect on the views of the people who contribute to the discussions on desktop-devel-list, and since most of the GNOME community read desktop-devel-list you can probably extend this to all of the other GNOME mailing lists and IRC channels as well. Sam ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On 19 August 2011 14:13, Felipe Contreras felipe.contre...@gmail.com wrote: Is there anyone in the universe able to create a user survey worthy of GNOME? Can you convince him of doing so? Do your survey with the questions you want, and come to your own conclusions. Blog about them if you want. You could even convince a distribution to include a popup with a link, although I think that's insane. Just don't tell people that it's from the GNOME project, in any way authorized or blessed by the ruling cabal[1] or developers. I'm pretty sure the majority of the people actually working on GNOME 3.2 don't want a survey at all. Sorry to be blunt. Richard. [1] http://ftp.gnome.org/conspiracy/ ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On 08/19/2011 09:13 PM, Felipe Contreras wrote: On Fri, Aug 19, 2011 at 4:08 PM, Andy Wingo wi...@pobox.com wrote: On Fri 19 Aug 2011 13:33, Felipe Contreras felipe.contre...@gmail.com writes: That's a reasonable alternative. How about pleased? Any other people have an opinion? You present yourself as reasonable by adjusting on the small points, but you ignore the feedback of greater importance. My opinion is that you are not the right person to lead an effort to gather feedback on GNOME. Is there anyone in the universe able to create a user survey worthy of GNOME? Can you convince him of doing so? In the meantime, this is the best that we have. I will continue listening for constructive feedback, but comments such as this is not good, you are doing it wrong, it's impossible, lead to nowhere. Besides, as Alan Cox said, it doesn't have to be perfect, like software, we can learn from the mistakes of the 2011 survey, and make a better one for 2012. Can we not? Hi again, I actually thought that Andy's feedback was constructive. Your approach, motives and way to handle this discussion are questionable. You have obviously failed to convince the GNOME community and GNOME developers that your survey would be useful, and I'm afraid nobody feels like taking leadership on the project neither. A lot of people have already told you that enough feedback has been gathered at this time. I doubt you will get much endorsement or help around here anymore (I could of course be wrong on that last point). To tell you the truth I have been involved in trying to run a survey for GNOME 2 years ago IIRC (and I think it's a recurrent project - you can find some old pages on our wiki) with a group of other people, and we reached the same conclusion: a survey will not help GNOME to get better. I obviously had very different motives (and GNOME 3 was not around). So if you want to help GNOME maybe you should discuss further with the design team and see how to contribute in a positive manner. I hope you will find my comments helpful as that's what I am trying to convey (help to someone who seems full of energy to do something to improve GNOME - and we need people like this). All the best. Fred ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Le vendredi 19 août 2011 à 16:08 +0300, Felipe Contreras a écrit : It's easy to throw empty criticism. Provide *suggestions*. Well, here’s a suggestion: since nobody knows how to address the correct target population or how to interpret the results, I suggest to spend our time fixing bugs instead. Cheers, -- .''`. Josselin Mouette : :' : `. `' `- ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, 2011-08-19 at 16:08 +0300, Felipe Contreras wrote: Likewise, 'happy' will be thought of differently by different people (a very odd word to include in a questionnaire, if you don't mind me saying): I think everyone understands the word happy. /ME wipes a mouthful of coffee from my monitor Then you haven't read enough of the survey research literature. That doesn't change the fact that everyone understands the word happy. Not necessary. Just to give an example - there is strong cultural influence how do you respond to simple question 'How are you'. In some cultures it is impolite to answer better then 'so so' and the normal answer is somehow along lines 'it could be worst, it could be better'. On the other hand the correct response in English is usually 'great' or 'fine' (to quote my teacher 'even if your house is burnt and your dog is terminally ill'). I have been warned to avoid 'standard' 'so so' response as I will be perceived as either impolite or after some large disaster because what I really meant was 'great'. (Somehow less directly related but also illustrates the problem of tricky words - in my native language friend means what in English is understood by close friend and many people whom I would call in English friend I would call in Polish acquaintance. Even though I know the difference I am less inclined to call people friends as in my mental model they are described by word 'acquaintance'). Of course this is 'just' cultural bias caused by people not being native speakers of English. You need to add individual bias. In each case it adds more and more 'noise' to survey. tl;dr version: your survey results will be misleading. No, the results would not be misleading; the *analysis* of the results might. But different people can analyze them in different ways. The important thing is to get *some* results. It seems bizarre to suggest that research data is valid irrespective of how it is gathered. If your questionnaire does not provide valid measurements no amount of analysis can compensate. You can thrown an analysis saying all this data is crap if that makes you happier, but this survey won't eat babies. I would argue that incorrect data (misinformation) is worst case then no data at all. Regards signature.asc Description: This is a digitally signed message part ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Gathering feedback does not necessarily require an online user survey. As stated, for a project which currently targets, among others, users who do not care what parts of their operating system can be labelled GNOME a survey is not a very reliable way of gathering feedback. Have you ever tried to explain, to a person who doesn't have an interest in software, what GNOME actually is? Yes - my MBA research was into Linux desktops some years ago and did involve looking at end users attitudes. The quick summary from then would be: Most users used the desktop they got by default (whether because they didn't know to to switch or were never annoyed enough to bother I didn't have time to find out) The managers wanted a system that was a free exact clone of windows look/feel because change was expensive (training, lost time etc) The technies in the organisation often inflicted their personal desktop preference on the entire company. If I wanted to look at the Gnome 3 is crap assertion I think I would tackle it a bit differently as so much online updating is going on nowdays. Collect statistics from a few Fedora and other mirror sites, correlate downloads together by IP/time and other evidence, and look at how many of them download which desktops or combination of desktops. Repeat this over time and plot graphs. Distro popularity shifts may also provide some evidence for this. The trouble is while that will tell you about movement and popularity it will not tell you why. So it's a way to evaluate the claim Gnome 3 is crap loads of people are changing or holding back on updating their desktop but it's not going to answer useful things. There is a bit of value in knowing if lots of people hate or love Gnome 3, but the real value is knowing how it could be better for users, and counting downloads won't do that. And if real non-technical end users are like the ones I dealt with then asking them probably won't help either. Particularly in the business world to many of them at the time Gnone was 'click on this splodge in the morning to write letters' 'click on that thing in the corner to turn it off'. They are not decision makers either - impress their boss 8) The more interested and technically motivated people on the other hand can tell you stuff, power users particularly. They tell you stuff that reflects a particular use and understanding case though. Similarly you can learn an enormous amount by seeing what people are struggling with and what they do to the desktop - eg the various 'how to fix Gnome 3' pages tell you a lot about what people wanted and which is non-obvious for configuration. They are also from people who liked it enough to persevere so made an effort. I urge you to consider the fact that if the majority of people subscribed to desktop-devel-list don't have faith in idea of an online user survey, an online user survey is probably not going to much have effect on the views of the people who contribute to the discussions on desktop-devel-list, and since most of the GNOME community read desktop-devel-list you can probably extend this to all of the other GNOME mailing lists and IRC channels as well. Some days I think Miguel got the Ximian monkey dead right, except that there should have been three of them. Alan ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 5:02 PM, Frederic Muller fr...@gnome.org wrote: On 08/19/2011 09:13 PM, Felipe Contreras wrote: On Fri, Aug 19, 2011 at 4:08 PM, Andy Wingo wi...@pobox.com wrote: On Fri 19 Aug 2011 13:33, Felipe Contreras felipe.contre...@gmail.com writes: That's a reasonable alternative. How about pleased? Any other people have an opinion? You present yourself as reasonable by adjusting on the small points, but you ignore the feedback of greater importance. My opinion is that you are not the right person to lead an effort to gather feedback on GNOME. Is there anyone in the universe able to create a user survey worthy of GNOME? Can you convince him of doing so? In the meantime, this is the best that we have. I will continue listening for constructive feedback, but comments such as this is not good, you are doing it wrong, it's impossible, lead to nowhere. Besides, as Alan Cox said, it doesn't have to be perfect, like software, we can learn from the mistakes of the 2011 survey, and make a better one for 2012. Can we not? I actually thought that Andy's feedback was constructive. Really? Then you can take that feedback and translate it into concrete actions, right? To me, that sounds as useful as the advice of some academic that says that no software should ever be deployed without strong static analysis. He might be right, but if he is not offering himself to do the job he is proposing, what's the point? IOW; Talk is cheap, show me the code. Your approach, motives and way to handle this discussion are questionable. You have obviously failed to convince the GNOME community and GNOME developers that your survey would be useful, and I'm afraid nobody feels like taking leadership on the project neither. Yeah, I failed at an impossible task, maybe. A lot of people have already told you that enough feedback has been gathered at this time. I doubt you will get much endorsement or help around here anymore (I could of course be wrong on that last point). To tell you the truth I have been involved in trying to run a survey for GNOME 2 years ago IIRC (and I think it's a recurrent project - you can find some old pages on our wiki) with a group of other people, and we reached the same conclusion: a survey will not help GNOME to get better. I obviously had very different motives (and GNOME 3 was not around). I strongly disagree. You can't know that until you actually *try*. Why are you so afraid to try? So if you want to help GNOME maybe you should discuss further with the design team and see how to contribute in a positive manner. All I am trying to do is get some user feedback. Without such feedback I doubt any kind of discussion on the design would be fruitful, because it all be dismissed based on assumptions and wishful thinking. Cheers. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 5:42 PM, Josselin Mouette j...@debian.org wrote: Le vendredi 19 août 2011 à 16:08 +0300, Felipe Contreras a écrit : It's easy to throw empty criticism. Provide *suggestions*. Well, here’s a suggestion: since nobody knows how to address the correct target population or how to interpret the results, I suggest to spend our time fixing bugs instead. Yes, because we are absolutely and positively certain that fixing these bugs is exactly what GNOME users want. There is no possibility that they want something else, or that the prioritization is not ideal. By definition, whatever GNOME does, is what the users want, and to suggest otherwise is heresy. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 4:55 PM, Sam Thursfield sss...@gmail.com wrote: On Fri, Aug 19, 2011 at 2:13 PM, Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 4:08 PM, Andy Wingo wi...@pobox.com wrote: On Fri 19 Aug 2011 13:33, Felipe Contreras felipe.contre...@gmail.com writes: That's a reasonable alternative. How about pleased? Any other people have an opinion? You present yourself as reasonable by adjusting on the small points, but you ignore the feedback of greater importance. My opinion is that you are not the right person to lead an effort to gather feedback on GNOME. Is there anyone in the universe able to create a user survey worthy of GNOME? Can you convince him of doing so? Gathering feedback does not necessarily require an online user survey. Indeed, do you have a better suggestion? As stated, for a project which currently targets, among others, users who do not care what parts of their operating system can be labelled GNOME a survey is not a very reliable way of gathering feedback. Have you ever tried to explain, to a person who doesn't have an interest in software, what GNOME actually is? Again, do you have a suggestion to get feedback in a more useful way? In the meantime, this is the best that we have. I will continue listening for constructive feedback, but comments such as this is not good, you are doing it wrong, it's impossible, lead to nowhere. Besides, as Alan Cox said, it doesn't have to be perfect, like software, we can learn from the mistakes of the 2011 survey, and make a better one for 2012. Can we not? I urge you to consider the fact that if the majority of people subscribed to desktop-devel-list don't have faith in idea of an online user survey, an online user survey is probably not going to much have effect on the views of the people who contribute to the discussions on desktop-devel-list, and since most of the GNOME community read desktop-devel-list you can probably extend this to all of the other GNOME mailing lists and IRC channels as well. So the status quo, where there are absolutely no numbers whatsoever is preferred. Any attempt to gather quantifiable feedback is discouraged. IOW; the GNOME community does not care about what users have to say at all. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 4:56 PM, Richard Hughes hughsi...@gmail.com wrote: On 19 August 2011 14:13, Felipe Contreras felipe.contre...@gmail.com wrote: Is there anyone in the universe able to create a user survey worthy of GNOME? Can you convince him of doing so? Do your survey with the questions you want, and come to your own conclusions. Blog about them if you want. You could even convince a distribution to include a popup with a link, although I think that's insane. Just don't tell people that it's from the GNOME project, in any way authorized or blessed by the ruling cabal[1] or developers. I'm pretty sure the majority of the people actually working on GNOME 3.2 don't want a survey at all. Sorry to be blunt. No, thanks for the direct feedback. So basically you are saying there's no way any survey of any quality would be blessed by the GNOME community. That certainly clarifies things. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Thu, Aug 18, 2011 at 10:37:33PM +0100, Alan Cox wrote: Doing nothing achieves nothing, doing something achieves learning. You may well not learn what you intended but you will learn something including quite possibly how to do future surveys better. Any survey that isn't a carefully controlled randomly selected sample of users doesn't result in learning. It results in data that forms some sort of rorschach blot. Everyone will see what they want to see. Those who believe that Gnome 3 is a step back will point out that the majority of responses are negative. Those who believe it's a step forward will point out that happy users are going to be far less inclined to respond. There's no way whatsoever to determine how representative the responses are, and so there's no way whatsoever to learn anything about the population. All we'd learn is that some users like Gnome 3 and some users don't, and that's something we *already know*. So we'd gain nothing, but we'd guarantee another huge set of arguments which would themselves also tell us nothing. I'm not saying its necessarily a great approach but it's vastly superior to people sitting around picking holes in the idea until it never happens. I disagree. Doing something that sucks more time and energy away from development without actually telling us anything in return is worse than that not happening. Felipe is obviously free to do whatever he wants, but there's no benefit in Gnome itself participating in any way. If we want to find out what our users think then the only way to do that is to have professional involvement and a random sample set. -- Matthew Garrett | mj...@srcf.ucam.org ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 7:24 PM, Matthew Garrett mj...@srcf.ucam.org wrote: On Thu, Aug 18, 2011 at 10:37:33PM +0100, Alan Cox wrote: Doing nothing achieves nothing, doing something achieves learning. You may well not learn what you intended but you will learn something including quite possibly how to do future surveys better. Any survey that isn't a carefully controlled randomly selected sample of users doesn't result in learning. Unless the biases are identified, which we are trying to do. Moreover, I have tried to push the idea to have an automatic notification, which would maximize the number of responders, and thus increase the randomization. But apparent this idea is not welcome. So, ideas to improve the randomization are dismissed, and then you say without randomization, the survey is not useful. IOW; you are intentionally deadlocking the proposal. It results in data that forms some sort of rorschach blot. It might if you look at it as a whole, but you can try to dissect it. Everyone will see what they want to see. Those who believe that Gnome 3 is a step back will point out that the majority of responses are negative. Those who believe it's a step forward will point out that happy users are going to be far less inclined to respond. There's no way whatsoever to determine how representative the responses are, and so there's no way whatsoever to learn anything about the population. All we'd learn is that some users like Gnome 3 and some users don't, and that's something we *already know*. So we'd gain nothing, but we'd guarantee another huge set of arguments which would themselves also tell us nothing. That's an assumption. What if we get 10 million responses? Would you still claim that the results are not representative? I think only *after* getting the results you would be able to say anything about it's representativeness. Something more realistic, say you get at least 300 responses that don't have any geek bias, that would be more than enough to make some statistically significant conclusions. I'm not saying its necessarily a great approach but it's vastly superior to people sitting around picking holes in the idea until it never happens. I disagree. Doing something that sucks more time and energy away from development without actually telling us anything in return is worse than that not happening. Felipe is obviously free to do whatever he wants, but there's no benefit in Gnome itself participating in any way. If we want to find out what our users think then the only way to do that is to have professional involvement and a random sample set. This is not sucking any time and energy from anybody, I just need access to the server that has limesurvey installed, or somebody else can do that (can't take that much time), I would contact all the relevant news sites and make the relevant posts in social media. All that that is needed from GNOME people is a blessing. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, 2011-08-19 at 19:42 +0300, Felipe Contreras wrote: On Fri, Aug 19, 2011 at 4:56 PM, Richard Hughes hughsi...@gmail.com wrote: On 19 August 2011 14:13, Felipe Contreras felipe.contre...@gmail.com wrote: Is there anyone in the universe able to create a user survey worthy of GNOME? Can you convince him of doing so? Do your survey with the questions you want, and come to your own conclusions. Blog about them if you want. You could even convince a distribution to include a popup with a link, although I think that's insane. Just don't tell people that it's from the GNOME project, in any way authorized or blessed by the ruling cabal[1] or developers. I'm pretty sure the majority of the people actually working on GNOME 3.2 don't want a survey at all. Sorry to be blunt. No, thanks for the direct feedback. So basically you are saying there's no way any survey of any quality would be blessed by the GNOME community. That certainly clarifies things. It seems obvious from most responses here that there are not very many people within the GNOME community that think that this sort of a survey would be beneficial, and worry that it may even be counter-productive. In response to this realization, you have apparently shifted into outrage mode. You pretend that it is impossible to simultaneously care about what users while also opposing a user survey that has no hope of being a representative sample of users. It is possible for well-meaning people to come to different conclusions on the best methods for achieving a certain goal. It seems that most people here don't agree with your methods. Please accept the fact that this does not mean that they hate users, despite your attempts to conflate the two things. You are free to proceed with your survey on your own. Others are free to not wish to join you. It's that simple. Can you please stop the faux outrage? jonner ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Any survey that isn't a carefully controlled randomly selected sample of users doesn't result in learning. It results in data that forms some You need truely or reasonably random samples for certain kinds of activities and analysis in particularly quantitative analysis when you want to perform p tests and the like. You don't need it in order to learn merely to generate statistical proofs and those are often quite useless anyway. Proviing gnome 3 is great/indifferent/sucks doesn't have much value. You do not need it for explorative learning. Small children do not need to open a statistically valid sample of randomly chosen doors to learn about doors ! I for one would not be surprised if a lot of responses were not more positive than some seem to think. There has been time for people to use it and adjust and apply the fixes. Even odder there is no Gnome fork. If as I hear 'Gnome 3 is hated by technical people' and there are enough who care there ought to be a Gnome fork by now. But what do we have - exde, dead, turned into a one page rant and no code Mate - described by phoronix as The Mate Desktop Environment fork of GNOME2 was started by an Arch Linux user back in June, but it hasn't yet gained too much traction and is mostly just talked about on various forums around the web. which about sums it up. sort of rorschach blot. Everyone will see what they want to see. Those who believe that Gnome 3 is a step back will point out that the majority of responses are negative. Those who believe it's a step forward will point out that happy users are going to be far less inclined to respond. You seem to be assuming the results and that the only question of interest is does gnome 3 suck. nothing, but we'd guarantee another huge set of arguments which would themselves also tell us nothing. Those will tell you a lot if someone analyses them. Again you may not be able to do formal mathematical tests on them but so what. If we want to find out what our users think then the only way to do that is to have professional involvement and a random sample set. Of course, and the only way to produce a kernel or desktop is to hire professionals to do it for you no doubt. Alan ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 10:40 AM, Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 4:55 PM, Sam Thursfield sss...@gmail.com wrote: Gathering feedback does not necessarily require an online user survey. Indeed, do you have a better suggestion? There are several other ways to get feedback. For example, user testing. I'm sure all the major distributions have done some user testing. Most large companies have a whole user testing team/group. I'm not a user testing expert but it involves giving people (both new and experienced) tasks to do, watching how they do it (without interfering) and then asking them about their experience. I know people who have successfully used http://www.usertesting.com/ for web sites. I don't know if a similar, inexpensive option exists for desktop software or not. Stormy ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 8:25 PM, Stormy Peters sto...@gnome.org wrote: On Fri, Aug 19, 2011 at 10:40 AM, Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 4:55 PM, Sam Thursfield sss...@gmail.com wrote: Gathering feedback does not necessarily require an online user survey. Indeed, do you have a better suggestion? There are several other ways to get feedback. For example, user testing. I'm sure all the major distributions have done some user testing. Most large companies have a whole user testing team/group. And where are the results? Without evidence it's only wishful thinking. I'm not a user testing expert but it involves giving people (both new and experienced) tasks to do, watching how they do it (without interfering) and then asking them about their experience. I know people who have successfully used http://www.usertesting.com/ for web sites. I don't know if a similar, inexpensive option exists for desktop software or not. Right, so nobody is going to do this. Is there any better suggestion that would actually be implemented? -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 8:15 PM, Jonathon Jongsma jonat...@quotidian.org wrote: On Fri, 2011-08-19 at 19:42 +0300, Felipe Contreras wrote: On Fri, Aug 19, 2011 at 4:56 PM, Richard Hughes hughsi...@gmail.com wrote: On 19 August 2011 14:13, Felipe Contreras felipe.contre...@gmail.com wrote: Is there anyone in the universe able to create a user survey worthy of GNOME? Can you convince him of doing so? Do your survey with the questions you want, and come to your own conclusions. Blog about them if you want. You could even convince a distribution to include a popup with a link, although I think that's insane. Just don't tell people that it's from the GNOME project, in any way authorized or blessed by the ruling cabal[1] or developers. I'm pretty sure the majority of the people actually working on GNOME 3.2 don't want a survey at all. Sorry to be blunt. No, thanks for the direct feedback. So basically you are saying there's no way any survey of any quality would be blessed by the GNOME community. That certainly clarifies things. It seems obvious from most responses here that there are not very many people within the GNOME community that think that this sort of a survey would be beneficial, and worry that it may even be counter-productive. In response to this realization, you have apparently shifted into outrage mode. You pretend that it is impossible to simultaneously care about what users while also opposing a user survey that has no hope of being a representative sample of users. You might say you do, and you might even believe so, but if your actions demonstrate otherwise, perhaps you do not. If the GNOME community really cared about what users have to say, and this survey indeed does not have any hope of having a representative sample of users (I disagree), then wouldn't they take the reins and do it properly? It is possible for well-meaning people to come to different conclusions on the best methods for achieving a certain goal. Yes, whenever I have a disagreement on a method to develop some software, I just go ahead and do it that way, and then say; see? this is how it should be done. Saying you are wrong is easy, anybody can do that. It seems that most people here don't agree with your methods. Please accept the fact that this does not mean that they hate users, despite your attempts to conflate the two things. I would, if they went ahead with the right methods and got some user feedback, if not in the form of a survey, in any method. You are free to proceed with your survey on your own. Others are free to not wish to join you. It's that simple. Can you please stop the faux outrage? Sure, I just wanted to make things clear. In fact, if they cared about user feedback, there would be some numbers available somewhere, and I wouldn't have to do this. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On 19 August 2011 18:42, Felipe Contreras felipe.contre...@gmail.com wrote: Sure, I just wanted to make things clear. In fact, if they cared about user feedback, there would be some numbers available somewhere, and I wouldn't have to do this. We're not asking you to do anything. Please just run the poll on your personal blog and stop getting aggressive with developers on this mailing list. Thanks, Richard. ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Really ought to stay out of this thread but there is one point that is important to address below. On Fri, Aug 19, 2011 at 7:17 AM, Felipe Contreras felipe.contre...@gmail.com wrote: But again, as I said, if there's no survey on Earth you could trust, just ignore the results. Results by themselves cannot hurt you. This isn't right. Poorly understood results lead you draw incorrect conclusions which lead you astray. Or at minimum cause the marketing team undue stress trying to explain that the results don't make any sense. Really, I don't understand why anyone would want to go through the trouble if the results aren't *useful*. Now, if you want to do something productive I encourage you to work with the guidance of Allan and others. Jon ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Thu, Aug 18, 2011 at 10:04 PM, Olav Vitters o...@vitters.nl wrote: On Thu, Aug 18, 2011 at 08:03:45PM +0300, Felipe Contreras wrote: I can only think of one reason why somebody would provide criticism without suggestions for improvement... 1. Because they cannot think of a good suggestion. Then surely I cannot be blamed for not coming with one either. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 8:50 PM, Richard Hughes hughsi...@gmail.com wrote: On 19 August 2011 18:42, Felipe Contreras felipe.contre...@gmail.com wrote: Sure, I just wanted to make things clear. In fact, if they cared about user feedback, there would be some numbers available somewhere, and I wouldn't have to do this. We're not asking you to do anything. I am not suggesting you are. Please just run the poll on your personal blog and stop getting aggressive with developers on this mailing list. I am not being aggressive. All I am asking is for clarification; is there *anything* I could do to make the survey more acceptable to you guys, or are you opposed to the very idea of having a user survey blessed by GNOME? -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 08:14:25PM +0300, Felipe Contreras wrote: On Fri, Aug 19, 2011 at 7:24 PM, Matthew Garrett mj...@srcf.ucam.org wrote: Any survey that isn't a carefully controlled randomly selected sample of users doesn't result in learning. Unless the biases are identified, which we are trying to do. You can only identify the biases if you already know the population, and you can only know the population if you've got a random sample set to begin with. Moreover, I have tried to push the idea to have an automatic notification, which would maximize the number of responders, and thus increase the randomization. But apparent this idea is not welcome. It doesn't help. The people most likely to respond to an irritating popup that disrupts their work are people who already feel that gnome 3 is an irritating piece of software that disrupts their work. You can't get a random sample in-band. So, ideas to improve the randomization are dismissed, and then you say without randomization, the survey is not useful. IOW; you are intentionally deadlocking the proposal. I am saying that your results aren't useful unless your sample is random. I don't know of a good way to obtain a representative sample. Everyone will see what they want to see. Those who believe that Gnome 3 is a step back will point out that the majority of responses are negative. Those who believe it's a step forward will point out that happy users are going to be far less inclined to respond. There's no way whatsoever to determine how representative the responses are, and so there's no way whatsoever to learn anything about the population. All we'd learn is that some users like Gnome 3 and some users don't, and that's something we *already know*. So we'd gain nothing, but we'd guarantee another huge set of arguments which would themselves also tell us nothing. That's an assumption. What if we get 10 million responses? Would you still claim that the results are not representative? Yes, because you have no idea how big the population is. Maybe 10 million is the total population and it's representative. Maybe it's 50% of the population, disproportionately biased towards those of a given prior opinion. You can't know. I think only *after* getting the results you would be able to say anything about it's representativeness. Something more realistic, say you get at least 300 responses that don't have any geek bias, that would be more than enough to make some statistically significant conclusions. It really wouldn't. I disagree. Doing something that sucks more time and energy away from development without actually telling us anything in return is worse than that not happening. Felipe is obviously free to do whatever he wants, but there's no benefit in Gnome itself participating in any way. If we want to find out what our users think then the only way to do that is to have professional involvement and a random sample set. This is not sucking any time and energy from anybody, I just need access to the server that has limesurvey installed, or somebody else can do that (can't take that much time), I would contact all the relevant news sites and make the relevant posts in social media. All that that is needed from GNOME people is a blessing. The sucking of time and energy would come from the argument over the results afterwards. -- Matthew Garrett | mj...@srcf.ucam.org ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
(Resend: Managed to leave d-d-l off Cc: by accident) On Fri, Aug 19, 2011 at 06:15:03PM +0100, Alan Cox wrote: Any survey that isn't a carefully controlled randomly selected sample of users doesn't result in learning. It results in data that forms some You need truely or reasonably random samples for certain kinds of activities and analysis in particularly quantitative analysis when you want to perform p tests and the like. You don't need it in order to learn merely to generate statistical proofs and those are often quite useless anyway. Proviing gnome 3 is great/indifferent/sucks doesn't have much value. You do not need it for explorative learning. Small children do not need to open a statistically valid sample of randomly chosen doors to learn about doors ! I am all for making it easier for people to give feedback about Gnome, but presenting it as a survey gives a strong implication that the results are meaningful as an aggregate rather than as a collection of anecdotes. If we want to hear form users, let's make it easy for users to talk to us. A survey isn't the way to achieve that. sort of rorschach blot. Everyone will see what they want to see. Those who believe that Gnome 3 is a step back will point out that the majority of responses are negative. Those who believe it's a step forward will point out that happy users are going to be far less inclined to respond. You seem to be assuming the results and that the only question of interest is does gnome 3 suck. I'm assuming that the sort of people who are going to go to the effort of filling out a survey are likely to be closer to the population discussing things on lwn than the population of usres in general. That may be entirely untrue! But if we get the opposite results then it still doesn't tell us anything that's actually true, and it's still an opportunity to argue the issue rather than focus on making software better. If we want to find out what our users think then the only way to do that is to have professional involvement and a random sample set. Of course, and the only way to produce a kernel or desktop is to hire professionals to do it for you no doubt. If you went back to 1991 and wanted a production-quality kernel within a year, Linux probably wouldn't be your starting point. There'd be a learning process involved with setting up a professional-quality survey team, and the first few attempts would be pretty buggy. We'd get there in time, but until then... -- Matthew Garrett | mj...@srcf.ucam.org ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 12:12 PM, Felipe Contreras felipe.contre...@gmail.com wrote: I am not being aggressive. All I am asking is for clarification; is there *anything* I could do to make the survey more acceptable to you guys, or are you opposed to the very idea of having a user survey blessed by GNOME? Your answers sound aggressive to me but I think that's totally understandable given all the negative feedback. I gave my feedback. I'd want the survey to be much more detailed. What do you think about this menu option on Cheese seems like it would give more feedback than do you like GNOME? But I do not have time to help come up with the questions, so I agree with many folks that say you'll have to take the feedback you've gotten and move forward. Giving feedback does not mean providing alternatives or working on the project. It's easy to give feedback. It's much harder and more time consuming to incorporate that feedback. You asked for feedback, you got some. If you want those people's approval, then you'll probably have to incorporate that feedback. If you aren't planning on incorporating it, then it's probably best to stop insisting that people need to provide alternatives if they give negative feedback. Obviously, you don't need everyone's approval to move forward. Rarely does any project get 100% approval. How you move forward, how much feedback you want and how you use that feedback is up to you. Good luck! Stormy ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 21:20, Matthew Garrett mj...@srcf.ucam.org wrote: The people most likely to respond to an irritating popup that disrupts their work are people who ... ... do not use GNOME 3. GNOME 3 is designed to reduce distraction and interruption and to put you in control. Our new notifications system subtly presents messages and will save them until you are ready for them, and the GNOME 3 panel has been styled so that it is part of the background, not the foreground. These changes allow you to focus on your creative tasks. I hope this brings a bit of humor in this thread. ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 14:46, Luc Pionchon pionchon@gmail.com wrote: On Fri, Aug 19, 2011 at 13:14, Allan Day allanp...@gmail.com wrote: We already have a wealth of data about peoples' experiences with GNOME 3 Allan, this is interesting, what is the main pointer to access this data? Allan, you may have missed it in this epic thread; I am sincerely interested in reading this data, would you be kind enough to point me at it? Luc ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v2)
On Mon, 2011-08-01 at 18:35 +0300, Felipe Contreras wrote: What do you think? Keep in mind that Gnome 3 just hasn't been around for very long. Right now Gnome 3 is most likely only being used by technical people, Linux enthusiasts, etc. - it has not trickled down to end users yet. We may have found the top, obvious problems in Gnome 3, but not the long tail of bugs. I think the survey would be much more useful with free-form answers, instead of fixed options. That way you may be able to eliminate one level of indirection (oh, most people said 'somewhat', now let's make another survey to find out why). While fixed options let you do amazing web hackery, free-form answers lead to more insightful results. Back in 2006 I did a survey of the Gnome deployments (i.e. somewhere around the middle of the 2.x series). These questions were posted to various places: https://live.gnome.org/GnomeMarketing/QuestionsForDeployments And this is the final report with the results: http://people.gnome.org/~federico/docs/gnome-deployments-2006/index.html (At the end of the report there are links to the raw results that people sent in. Download them and read them; they are short and quite interesting. Where the links say, primates.ximian.com, please replace that with people.gnome.org - I can't re-generate the HTML report with that change just now, unfortunately.) Let me tell you about some things I learned from that survey: * Free-form answers work really well. It *will* take you some time to read them and see how to categorize or weigh them, but it's worth the effort. * Being able to publish the raw results really helps; this way other people can help you extract statistics. Do ask permission from the respondents to post their replies so that other people can study them. * That survey probably gave too much importance to the system administrators themselves - not surprisingly, better admin tools was the most requested thing. However, the survey *did* get us good insight into end-user's problems. * Those pie charts in the report make no sense. Apologies for chartjunk. It may be very valuable to ask the heads of deployments what they think of Gnome 3 so far, even if they haven't evaluated it yet on their users - these people have a very good sense of what will work well for people in the real world. By the way, the main page for the deployments is this (I don't know how up-to-date it is): https://live.gnome.org/GnomeMarketing/GnomeDeployments About two years after that survey got published, some people had the idea of making it periodic - unfortunately I lost track of them and of that effort. You may want to look around for them; they'll have interesting thoughts, I'm sure. Federico ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 9:20 PM, Matthew Garrett mj...@srcf.ucam.org wrote: On Fri, Aug 19, 2011 at 08:14:25PM +0300, Felipe Contreras wrote: On Fri, Aug 19, 2011 at 7:24 PM, Matthew Garrett mj...@srcf.ucam.org wrote: Any survey that isn't a carefully controlled randomly selected sample of users doesn't result in learning. Unless the biases are identified, which we are trying to do. You can only identify the biases if you already know the population, and you can only know the population if you've got a random sample set to begin with. That's not true. You might need that if you want to account for *all* the biases, which nobody can do anyway. What most people do is try to figure them out, chances are you might be missing some of the biases, but hopefully the unidentified misrepresented group won't be that big anyway, and thus wouldn't affect so much the analysis. If it turns out that a significant bias is not identified beforehand, that can be tackled in the next survey in 2012. Moreover, I have tried to push the idea to have an automatic notification, which would maximize the number of responders, and thus increase the randomization. But apparent this idea is not welcome. It doesn't help. The people most likely to respond to an irritating popup that disrupts their work are people who already feel that gnome 3 is an irritating piece of software that disrupts their work. You can't get a random sample in-band. It doesn't help? It does randomize the sample more, doesn't it? Maybe it's not perfectly randomized, but nothing can ever be perfect. So, ideas to improve the randomization are dismissed, and then you say without randomization, the survey is not useful. IOW; you are intentionally deadlocking the proposal. I am saying that your results aren't useful unless your sample is random. I don't know of a good way to obtain a representative sample. There's no such thing as 0% random, or 100% random, all we can thrive for is to increase the randomness. And I already explained that non-random samples are already useful if you can identify the biases. Everyone will see what they want to see. Those who believe that Gnome 3 is a step back will point out that the majority of responses are negative. Those who believe it's a step forward will point out that happy users are going to be far less inclined to respond. There's no way whatsoever to determine how representative the responses are, and so there's no way whatsoever to learn anything about the population. All we'd learn is that some users like Gnome 3 and some users don't, and that's something we *already know*. So we'd gain nothing, but we'd guarantee another huge set of arguments which would themselves also tell us nothing. That's an assumption. What if we get 10 million responses? Would you still claim that the results are not representative? Yes, because you have no idea how big the population is. Maybe 10 million is the total population and it's representative. Maybe it's 50% of the population, disproportionately biased towards those of a given prior opinion. You can't know. Do you have any idea what is the likelihood of that happening? Try throwing a dice 10 times and always getting 1-3. Even if the dice is rigged, it's very unlikely. It gets exponentially less likely 1 million times. I think only *after* getting the results you would be able to say anything about it's representativeness. Something more realistic, say you get at least 300 responses that don't have any geek bias, that would be more than enough to make some statistically significant conclusions. It really wouldn't. Yes it would. Check Cochran's formulas. 300 unbiased responses gives you already good statistical power, after a certain point it doesn't matter much what is the total population; 10m, 30m, 1m. The likelihood that would would get 300 unbiased responses all pointing to the wrong direction is almost nothing, in fact a few dozens would do (if they are truly random). There are some simple calculators online: http://www.surveysystem.com/sscalc.htm But yeah, since there's going to be bias, you need more. I disagree. Doing something that sucks more time and energy away from development without actually telling us anything in return is worse than that not happening. Felipe is obviously free to do whatever he wants, but there's no benefit in Gnome itself participating in any way. If we want to find out what our users think then the only way to do that is to have professional involvement and a random sample set. This is not sucking any time and energy from anybody, I just need access to the server that has limesurvey installed, or somebody else can do that (can't take that much time), I would contact all the relevant news sites and make the relevant posts in social media. All that that is needed from GNOME people is a blessing. The sucking of time and energy would come from the argument over
Re: GNOME user survey 2011 (v4)
On 19 August 2011 20:26, Felipe Contreras felipe.contre...@gmail.com wrote: ...To me GNOME is hitting everything in the room as it's going forward, and saying; I'm fine, I know where I'm going... To me, the sun is shining through the windows of a freshly redecorated room. If you have specific problems with GNOME, file bugs and discuss things with maintainers and designers. It'll be way more effective at changing things than writing a survey. Richard. ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 9:28 PM, Matthew Garrett mj...@srcf.ucam.org wrote: (Resend: Managed to leave d-d-l off Cc: by accident) On Fri, Aug 19, 2011 at 06:15:03PM +0100, Alan Cox wrote: Any survey that isn't a carefully controlled randomly selected sample of users doesn't result in learning. It results in data that forms some You need truely or reasonably random samples for certain kinds of activities and analysis in particularly quantitative analysis when you want to perform p tests and the like. You don't need it in order to learn merely to generate statistical proofs and those are often quite useless anyway. Proviing gnome 3 is great/indifferent/sucks doesn't have much value. You do not need it for explorative learning. Small children do not need to open a statistically valid sample of randomly chosen doors to learn about doors ! I am all for making it easier for people to give feedback about Gnome, but presenting it as a survey gives a strong implication that the results are meaningful as an aggregate rather than as a collection of anecdotes. If we want to hear form users, let's make it easy for users to talk to us. A survey isn't the way to achieve that. Again, any better suggestions? I tried many of them back in 2007, and got nowhere, I think a user survey is the best one we've got. sort of rorschach blot. Everyone will see what they want to see. Those who believe that Gnome 3 is a step back will point out that the majority of responses are negative. Those who believe it's a step forward will point out that happy users are going to be far less inclined to respond. You seem to be assuming the results and that the only question of interest is does gnome 3 suck. I'm assuming that the sort of people who are going to go to the effort of filling out a survey are likely to be closer to the population discussing things on lwn than the population of usres in general. That may be entirely untrue! But if we get the opposite results then it still doesn't tell us anything that's actually true, and it's still an opportunity to argue the issue rather than focus on making software better. We most likely are going to be able to identify that bias. Let's make some wild guesses; 50% of the people that use GNOME 3 like it, and 50% don't. Of that amount, 90% seem to be geeks. In the remaining 10%, the people that use GNOME 2 show 80% happiness, and of GNOME 3 it's only 60%. But you still don't think there's any value in there, fair enough. Then we dig through that 40% subset that didn't like GNOME 3 and take a look at their comments, and we find Very strange, Can't get used to it, and things like that. At that point we might want to see if they left an email to contact them, and then try to gather more detailed feedback. I think there's a chance that this survey could tell us something that's actually true. If we want to find out what our users think then the only way to do that is to have professional involvement and a random sample set. Of course, and the only way to produce a kernel or desktop is to hire professionals to do it for you no doubt. If you went back to 1991 and wanted a production-quality kernel within a year, Linux probably wouldn't be your starting point. There'd be a learning process involved with setting up a professional-quality survey team, and the first few attempts would be pretty buggy. We'd get there in time, but until then... Until then it's better to have nothing? -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 10:47 PM, Richard Hughes hughsi...@gmail.com wrote: On 19 August 2011 20:26, Felipe Contreras felipe.contre...@gmail.com wrote: ...To me GNOME is hitting everything in the room as it's going forward, and saying; I'm fine, I know where I'm going... To me, the sun is shining through the windows of a freshly redecorated room. Either you are hallucinating, or you are not getting the analogy. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 10:53:46PM +0300, Felipe Contreras wrote: If you went back to 1991 and wanted a production-quality kernel within a year, Linux probably wouldn't be your starting point. There'd be a learning process involved with setting up a professional-quality survey team, and the first few attempts would be pretty buggy. We'd get there in time, but until then... Until then it's better to have nothing? It's better to have no data than to have misleading data. -- Matthew Garrett | mj...@srcf.ucam.org ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 9:34 PM, Stormy Peters sto...@gnome.org wrote: On Fri, Aug 19, 2011 at 12:12 PM, Felipe Contreras felipe.contre...@gmail.com wrote: I am not being aggressive. All I am asking is for clarification; is there *anything* I could do to make the survey more acceptable to you guys, or are you opposed to the very idea of having a user survey blessed by GNOME? Your answers sound aggressive to me but I think that's totally understandable given all the negative feedback. Perhaps you are not used to straight-forward communication. I'm not trying to aggravate anyone. I gave my feedback. I'd want the survey to be much more detailed. What do you think about this menu option on Cheese seems like it would give more feedback than do you like GNOME? But I do not have time to help come up with the questions, so I agree with many folks that say you'll have to take the feedback you've gotten and move forward. Trying to do that would create a huge survey that most likely most people will not even try to answer. If somebody really detests certain menu option on Cheese, I'm sure they'll let you know in the comments box. Who knows, maybe it turns out the part that most people are not satisfied with is the documentation, those kinds of results might trigger some interesting debate. Or maybe you are right, and we wouldn't not get anything useful, but at least we would have some ideas for the next survey. Giving feedback does not mean providing alternatives or working on the project. It's easy to give feedback. It's much harder and more time consuming to incorporate that feedback. You asked for feedback, you got some. If you want those people's approval, then you'll probably have to incorporate that feedback. I have incorporated all the feedback that can be incorporated. The rest is too vague, or not actionable. What do *you* think must absolutely be changed in the survey? If you aren't planning on incorporating it, then it's probably best to stop insisting that people need to provide alternatives if they give negative feedback. Huh? That's a very broad statement. Let's be clear, I have not turned away any feedback. Let's analyze for example the claims by Allan Day: --- When you do survey research, you have to be certain that one person understands the questions in the same way that another person does. Generally yes. Is that achievable in all the questions in this survey? Probably not (would love to hear some suggestions otherwise). Which why some other questions are asked to determine the people that might be thinking in other terms. (I already explained that) You've also got the representativeness problem. Your sample will inevitably be unrepresentative, probably highly so. Says who? What if we get 10 million answers? That would be such a big chunk of the total population that this problem is not a big deal. Or what if there's a notification app embedded in GNOME 3.2. That would not only maximize the reponders, but also maximize the randomness. Wouldn't it? (I already explained that) your survey results will be misleading That's very useful. Now, how about some ideas to make the results less misleading? --- What exactly do you want me to do with that feedback? (aside from what I have already done) I am all ears. Obviously, you don't need everyone's approval to move forward. Rarely does any project get 100% approval. How you move forward, how much feedback you want and how you use that feedback is up to you. I want the approval of the GNOME community, and I am willing to accept all suggestions for improvement in order to get that. So, what should I do? -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 10:26:08PM +0300, Felipe Contreras wrote: Yes, because you have no idea how big the population is. Maybe 10 million is the total population and it's representative. Maybe it's 50% of the population, disproportionately biased towards those of a given prior opinion. You can't know. Do you have any idea what is the likelihood of that happening? Try throwing a dice 10 times and always getting 1-3. Even if the dice is rigged, it's very unlikely. It gets exponentially less likely 1 million times. That's clearly wrong. If you have a bucket of red balls and blue balls and you draw 10 million balls, and you find that you drew 6 million red balls and 4 million blue balls, what does that tell you? If you're sampling randomly it tells you that there are more red balls than blue balls. If you're subconsciously preferring to pick up red balls then it tells you nothing. So we need to avoid subconsciously picking red balls, which means we need to pick users randomly which is something we can't do with a voluntary survey. Cochran's formulas don't apply here because you're not picking your sample set at random. -- Matthew Garrett | mj...@srcf.ucam.org ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, 19 Aug 2011 21:05:26 +0100 Matthew Garrett mj...@srcf.ucam.org wrote: On Fri, Aug 19, 2011 at 10:53:46PM +0300, Felipe Contreras wrote: If you went back to 1991 and wanted a production-quality kernel within a year, Linux probably wouldn't be your starting point. There'd be a learning process involved with setting up a professional-quality survey team, and the first few attempts would be pretty buggy. We'd get there in time, but until then... Until then it's better to have nothing? It's better to have no data than to have misleading data. It's better to have no desktop than one that might not be production quality ? Same argument, same problem. PS data is never misleading. It's presentation maybe misleading but the data is just bits. I do think the comments on more open and why fill in the box type questions are on the button for the reasons expressed about sample size, randomness and what it would be useful to learn. Or perhaps rerun Federico's survey ? Alan ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 11:14 PM, Matthew Garrett mj...@srcf.ucam.org wrote: On Fri, Aug 19, 2011 at 10:26:08PM +0300, Felipe Contreras wrote: Yes, because you have no idea how big the population is. Maybe 10 million is the total population and it's representative. Maybe it's 50% of the population, disproportionately biased towards those of a given prior opinion. You can't know. Do you have any idea what is the likelihood of that happening? Try throwing a dice 10 times and always getting 1-3. Even if the dice is rigged, it's very unlikely. It gets exponentially less likely 1 million times. That's clearly wrong. If you have a bucket of red balls and blue balls and you draw 10 million balls, and you find that you drew 6 million red balls and 4 million blue balls, what does that tell you? If you're sampling randomly it tells you that there are more red balls than blue balls. If you're subconsciously preferring to pick up red balls then it tells you nothing. So we need to avoid subconsciously picking red balls, which means we need to pick users randomly which is something we can't do with a voluntary survey. Cochran's formulas don't apply here because you're not picking your sample set at random. That's a very bad example. An example closer to reality would be that color is indeed the bias, but we are not interested in the color, but the size of the balls. After the survey, we find out that overall, red balls are bigger than blue balls. Fortunately we don't care about the proportion of blue vs red balls in the total population, we only care about blue balls, so, we only consider the size of those. In the GNOME case, the color of the balls corresponds to the bias we want to identify; like geekness, and the size is the actual thing we are interested on, which is their happiness. We only care about non geeks (blue balls), as many GNOME people have stated, the real target users are the ones that don't even know what is GNOME. Now, if what you are worried about is the self-selection bias, we can add a new question Why are you taking this survey? with the option Somebody is pushing me, and encourage people to push their relatives/colleagues/friends to fill the survey (just like a professional firm would, except crowd-sourced). Then, for external validity, you only consider the results of the people that answered Somebody is pushing me (they don't have self-selection bias). -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Where is the data?
I'm sorry I couldn't read through the whole GNOME Survey v4 thread, but it was just too long. What I read though is that data was collected and extists. Now I'd like to simply ask: where is it? Where we developers can find real cold numbers backing out the designs we're asked to implement? Where we developers can find hard facts proving the NOTABUG and the WONTFIX we mark in the most questioned and hot issues? I'm not a designer, so I may not understand all the papers you provide in your support, and I may not understand what are the rules and laws of Human Computer Interaction, as you call it. But I understand numbers, and would be convinced by seeing that 66% percent of people find this method of working more productive, or 3 out 5 tested users where able to discover the functionality without guidance, or all 8 people interviewed did not use the feature just removed. Note the small numbers: I know that user testing is limited in scope, and I know that reaching out to a broader audience would require money that the foundation doesn't have. But if you insist in saying that all necessary feedback was already acquired, you need to make it public somewhere. You need to back your decisions on market share, not technical purity, because that is our goal (imho). I continously read of people complaining about GNOME 3, around the web and in gnome-shell-list (because, unfortunately, much of the complaint is against the shell way), and the answers, when provided, are just repeating the same design assertions, to the point that some subscribers are fed up of writing them. As a specific example of unscientific user testing, I got a friend of mine to try GNOME 3 at the desktop summit, and when it was time to shutdown he just asked me, because he found no way and he thought it was a bug. I'm sorry but I didn't have any explanations, so I just said the designers said so, and similarly I had none when some KDE hackers asked me on the same problem. I know that what I write, following the guidelines and the mockups, is right. But people providing feedback don't always agree with that, and if myself cannot understand the reason, how can I explain to them? I understand that some features in 3.0 were like design experiments, because we have the whole 3.* cycle to improve. But if the results of those experiments (that is, people's feedback) is not analyzed thoroughly, how can we be sure that the design is right? Or on the other hand, how can I see that the feedback is listened to, if decisions are never reverted? Sorry for this long mail, and sorry for contributing to the desktop-devel noise, but I've been waiting to ask these questions for too long. I hope that a striking and factual answer will avoid lengthy discussion. Giovanni signature.asc Description: This is a digitally signed message part ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Where is the data?
On 19 August 2011 19:00, Giovanni Campagna scampa.giova...@gmail.com wrote: As a specific example of unscientific user testing, I got a friend of mine to try GNOME 3 at the desktop summit, and when it was time to shutdown he just asked me, because he found no way and he thought it was a bug. I'm sorry but I didn't have any explanations, so I just said the designers said so, and similarly I had none when some KDE hackers asked me on the same problem. I know that what I write, following the guidelines and the mockups, is right. But people providing feedback don't always agree with that, and if myself cannot understand the reason, how can I explain to them? I understand that some features in 3.0 were like design experiments, because we have the whole 3.* cycle to improve. But if the results of those experiments (that is, people's feedback) is not analyzed thoroughly, how can we be sure that the design is right? Or on the other hand, how can I see that the feedback is listened to, if decisions are never reverted? The Alt modifier to get to Shutdown options in the usermenu is clearly wrong. As far as I can see, that Alt modifier is used only once in the entirety of GNOME Core and is definitely not a common interaction anywhere in the software world. Thus it is completely non-discoverable. It is impossible for anyone to ever figure out that functionality unless they were told it was there. Core functionality like powering off a computer should not be an easter egg. I know that this has been discussed at length on the bug report and elsewhere but the designers have so far refused to revert this bug/feature so that normal people can use the usermenu effectively. I believe Debian has interest in shipping a non-hidden Power Off button and I'd expect Ubuntu's GNOME Shell package would follow that example. Jeremy ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Felipe Contreras felipe.contreras at gmail.com writes: That doesn't change the fact that everyone understands the word happy. http://www.ted.com/talks/daniel_kahneman_the_riddle_of_experience_vs_memory.html ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
