Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 5:58 PM, Maciej Marcin Piechotka uzytkown...@gmail.com wrote: On Fri, 2011-08-19 at 16:08 +0300, Felipe Contreras wrote: Likewise, 'happy' will be thought of differently by different people (a very odd word to include in a questionnaire, if you don't mind me saying): I think everyone understands the word happy. /ME wipes a mouthful of coffee from my monitor Then you haven't read enough of the survey research literature. That doesn't change the fact that everyone understands the word happy. Not necessary. Just to give an example - there is strong cultural influence how do you respond to simple question 'How are you'. In some cultures it is impolite to answer better then 'so so' and the normal answer is somehow along lines 'it could be worst, it could be better'. On the other hand the correct response in English is usually 'great' or 'fine' (to quote my teacher 'even if your house is burnt and your dog is terminally ill'). I have been warned to avoid 'standard' 'so so' response as I will be perceived as either impolite or after some large disaster because what I really meant was 'great'. (Somehow less directly related but also illustrates the problem of tricky words - in my native language friend means what in English is understood by close friend and many people whom I would call in English friend I would call in Polish acquaintance. Even though I know the difference I am less inclined to call people friends as in my mental model they are described by word 'acquaintance'). Of course this is 'just' cultural bias caused by people not being native speakers of English. You need to add individual bias. In each case it adds more and more 'noise' to survey. Yes, words can mean different things depending on the cultural background, but that's not the case for happy. In my experience all cultures know exactly what you mean when you say you are happy with something. If you have evidence to the contrary for this precise word, please say so, otherwise it's just a faulty generalization. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Sat, Aug 20, 2011 at 6:24 AM, Benjamin Otte o...@gnome.org wrote: Felipe Contreras felipe.contreras at gmail.com writes: That doesn't change the fact that everyone understands the word happy. http://www.ted.com/talks/daniel_kahneman_the_riddle_of_experience_vs_memory.html Yes, I have seen that video before. However, not once in that video is he suggesting that people are confused about what happy means. When somebody says he is happy with his vacation, he knows exactly what he means, and so do we. What Kahneman is saying is that we are talking to remembering self, that's all. You can't ask the experiencing self anything, only the remembering self can answer something in a survey. Now, here's the funny part, this is similar to the reason why you need a survey; you need some data, some numbers. Because when you ask the question Is GNOME improving? or Do people like GNOME 3?, all you have are recollections of experiences which can be completely distorted from the reality, only with data you can have some degree of certainty. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Thu, Aug 18, 2011 at 23:21, Stormy Peters sto...@gnome.org wrote: Of course, maybe I'm wrong. Perhaps the average user of Linux/GNOME does know what GNOME is, knows how to contact the GNOME team Maybe this is an interesting point to know. Also from a branding point of view. Is it important (or not) that users know that, this is GNOME? That it is free software? That it is made by an international collaborative community? That the community can be contacted? That feedback and contribution is welcome? (shall it be even just a typo in a doc, or a love message) I feel it somehow wrong if a large proportion of users would just vaguely know they are using linux. ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 03:34, Zeeshan Ali (Khattak) zeesha...@gnome.orgwrote: [snip] Maybe they all lied? Don't you think it is a bit early to speculate on results? (...) Overall I can see already one clear result, even before the poll has started: We do not know who is using GNOME. Maybe this needs reflexion ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Thu, Aug 18, 2011 at 19:53, Stormy Peters sto...@gnome.org wrote: All the questions after this assume a knowledge of GNOME and how our community works. That's fine if you are polling developers. If you are polling average users, then I think it's not worth asking. Another issue that I don't think has been raised yet (sorry if it has, this thread reached the limits a while ago) is that the changes we make to GNOME aren't intended just for existing GNU/Linux users or people with similar interests. My understanding is that we want GNOME to be useful to people that haven't been attracted yet to any of the existing free desktops (most of the people in this world). Somewhat related, you need to take into account people's natural resistance to change. A factor that isn't equally relevant when surveying future users as it is when surveying present ones. Regards, Tomeu ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 1:20 AM, Zeeshan Ali (Khattak) zeesha...@gnome.org wrote: On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras felipe.contre...@gmail.com wrote: Nothing is ever perfect, but having at least some results is better than nothing. Since you have repeated this assertion a few times, I must ask: What if the results are all wrong and we don't have any way of knowing that? Would those results still be better than nothing in your opinion? What do you mean by all wrong? Let's assume that the results show that 1000 people are not happy with GNOME. How can that be wrong? 1000 people responded that, the results were not somehow altered, or boycotted, the results are the results, and that's that. ... 'Wrong' in social research typically means that your results lack validity: that you think the data is measuring one thing (eg. 'GNOME users' happiness with GNOME 3') but is in fact measuring something else. When you do survey research, you have to be certain that one person understands the questions in the same way that another person does. Looking at your questionnaire, that won't be the case. An example: === 02. Overall, how happy are you with GNOME? === (single choice) * unhappy * not so happy * happy * very happy * completely ecstatic Different people will understand the words GNOME/happy/very happy/ecstatic in different ways. Some might think 'GNOME' is their distro (including the lower levels of the stack), some that it's their 'shell', others that it's all their GUI software [1]. Likewise, 'happy' will be thought of differently by different people (a very odd word to include in a questionnaire, if you don't mind me saying): there are a vast range of expectations and usage patterns in relation to desktop computers, all of which will affect how people respond. Someone could tick 'unhappy' but by most measures have had a perfectly satisfactory experience. You've also got the representativeness problem. Your sample will inevitably be unrepresentative, probably highly so. Even if 100% of your *unrepresentative sample* tick the unhappy box, that doesn't tell you much about your target population: you might just have sampled every 'unhappy' GNOME user that's out there. tl;dr version: your survey results will be misleading. We already have a wealth of data about peoples' experiences with GNOME 3 and are working to address the issues that are being raised. It's great that you want to help, but this survey really won't be useful. Allan [1] GNOME's place in the stack means that you can't really do satisfaction surveys on it. This is one reason why GNOME is a more difficult research topic than, say, Git. -- IRC: aday on irc.gnome.org Blog: http://afaikblog.wordpress.com/ ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 3:34 AM, Zeeshan Ali (Khattak) zeesha...@gnome.org wrote: On Fri, Aug 19, 2011 at 2:09 AM, Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 1:20 AM, Zeeshan Ali (Khattak) zeesha...@gnome.org wrote: On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras felipe.contre...@gmail.com wrote: Nothing is ever perfect, but having at least some results is better than nothing. Since you have repeated this assertion a few times, I must ask: What if the results are all wrong and we don't have any way of knowing that? Would those results still be better than nothing in your opinion? What do you mean by all wrong? Let's assume that the results show that 1000 people are not happy with GNOME. How can that be wrong? Maybe they all lied? Maybe people who are satisfied do not want to or have time to take part in surveys and you only get people who are not happy into the survey? In which case, the results may show results that are not correct. i-e a significantly large number of participant say that they are very unhappy with GNOME but what if that number is nothing compared to the number of people who are very much satisfied with GNOME? I didn't say this so far because it might sound like I am trying to make a joke but since you still insist on your assertions about the survey, I feel I must say this: How do you know people in general like to participate in surveys? It is my observation that most people do not like to do that, unless they have something to complain about. Now this observation of mine could very well be wrong but how do we know that? Do we do a survey to find out if people like to participate in surveys? Are you serious? That totally and completely speculative and unrealistic. Have you ever participated in making a survey? I have, as I have explained, for the Git survey. In my experience, only the people that want to help in some way do spend the amount of time required to fill the survey. But again, as I said, if there's no survey on Earth you could trust, just ignore the results. Results by themselves cannot hurt you. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 1:14 PM, Allan Day allanp...@gmail.com wrote: Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 1:20 AM, Zeeshan Ali (Khattak) zeesha...@gnome.org wrote: On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras felipe.contre...@gmail.com wrote: Nothing is ever perfect, but having at least some results is better than nothing. Since you have repeated this assertion a few times, I must ask: What if the results are all wrong and we don't have any way of knowing that? Would those results still be better than nothing in your opinion? What do you mean by all wrong? Let's assume that the results show that 1000 people are not happy with GNOME. How can that be wrong? 1000 people responded that, the results were not somehow altered, or boycotted, the results are the results, and that's that. ... 'Wrong' in social research typically means that your results lack validity: that you think the data is measuring one thing (eg. 'GNOME users' happiness with GNOME 3') but is in fact measuring something else. When you do survey research, you have to be certain that one person understands the questions in the same way that another person does. Looking at your questionnaire, that won't be the case. An example: === 02. Overall, how happy are you with GNOME? === (single choice) * unhappy * not so happy * happy * very happy * completely ecstatic Different people will understand the words GNOME/happy/very happy/ecstatic in different ways. Some might think 'GNOME' is their distro (including the lower levels of the stack), Which is why we ask more question to understand their level of geekness. That should help the make correlations; the people that use a terminal all the time more likely know that GNOME is just the DE. The people that don't have much experience might be confusing GNOME with the distribution. Likewise, 'happy' will be thought of differently by different people (a very odd word to include in a questionnaire, if you don't mind me saying): I think everyone understands the word happy. That is what is used in Git user survey, and seems to be doing the job just fine. In any case, if you have suggestions that don't have these problems, feel free to share them. You've also got the representativeness problem. Your sample will inevitably be unrepresentative, probably highly so. Even if 100% of your *unrepresentative sample* tick the unhappy box, that doesn't tell you much about your target population: you might just have sampled every 'unhappy' GNOME user that's out there. If you can identify the bias, that's not a huge problem. tl;dr version: your survey results will be misleading. No, the results would not be misleading; the *analysis* of the results might. But different people can analyze them in different ways. The important thing is to get *some* results. We already have a wealth of data about peoples' experiences with GNOME 3 and are working to address the issues that are being raised. It's great that you want to help, but this survey really won't be useful. Where? I haven't seen any. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 1:17 PM, Felipe Contreras felipe.contre...@gmail.com wrote: Are you serious? That totally and completely speculative and unrealistic. Have you ever participated in making a survey? I have, as I have explained, for the Git survey. In my experience, only the people that want to help in some way do spend the amount of time required to fill the survey. Could you at least make the answer options less emotional? Like exchange happy for satisfied etc. I don't remember answering ecstatic in the Git survey but that could be my bad memory. -- Patryk Zawadzki I solve problems. ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 2:25 PM, Patryk Zawadzki pat...@pld-linux.org wrote: On Fri, Aug 19, 2011 at 1:17 PM, Felipe Contreras felipe.contre...@gmail.com wrote: Are you serious? That totally and completely speculative and unrealistic. Have you ever participated in making a survey? I have, as I have explained, for the Git survey. In my experience, only the people that want to help in some way do spend the amount of time required to fill the survey. Could you at least make the answer options less emotional? Like exchange happy for satisfied etc. I don't remember answering ecstatic in the Git survey but that could be my bad memory. That's a reasonable alternative. How about pleased? Any other people have an opinion? -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 2:17 PM, Felipe Contreras felipe.contre...@gmail.com wrote: I didn't say this so far because it might sound like I am trying to make a joke but since you still insist on your assertions about the survey, I feel I must say this: How do you know people in general like to participate in surveys? It is my observation that most people do not like to do that, unless they have something to complain about. Now this observation of mine could very well be wrong but how do we know that? Do we do a survey to find out if people like to participate in surveys? Are you serious? That totally and completely speculative and unrealistic. What is speculative? I made it very clear that it is *my* observation and *if* it is correct, the results of this survey may very well be wrong. Do you have any evidence that suggests that my observations above are incorrect? Have you ever participated in making a survey? No I have not but that does not necessarily mean what I said is incorrect and could just be ignored by pointing to examples of other surveys. If other people are ignoring an important issue, doesn't mean we should do the same. I have, as I have explained, for the Git survey. In my experience, only the people that want to help in some way do spend the amount of time required to fill the survey. As I have explained to you many times before, git's user-base is mostly (if not all) geeks and those geeks know where the mailing-list is and be able to access the survey easily. Still, I am a geek and a very happy user of git but I didn't even know about the existence of this survey until you told me. Even then, I didn't care to participate. I am pretty sure I would have cared to participate if I had something to complain about its current or planned features. GNOME's user-base consists of people who do not even know what GNOME is so many of them will not be able to participate, especially if they are happy users. In short, example of git surveys are quite irrelevant here. But again, as I said, if there's no survey on Earth you could trust, just ignore the results. Results by themselves cannot hurt you. In this case those results will really hurt since then you will have some numbers to back-up your claim of GNOME 3 is completely unusable. *If* your motivation for this survey has remained the same, you'll spread a lot of negative propaganda (which you already did even when you didn't have any numbers) and many people will just say Oh, people don't like this gnome 3 thingie, must be shit and will stay away from it. Even if you don't do that, there is many others who will use this data in that way. -- Regards, Zeeshan Ali (Khattak) FSF member#5124 ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 14:33, Felipe Contreras felipe.contre...@gmail.comwrote: On Fri, Aug 19, 2011 at 2:25 PM, Patryk Zawadzki pat...@pld-linux.org wrote: On Fri, Aug 19, 2011 at 1:17 PM, Felipe Contreras felipe.contre...@gmail.com wrote: Are you serious? That totally and completely speculative and unrealistic. Have you ever participated in making a survey? I have, as I have explained, for the Git survey. In my experience, only the people that want to help in some way do spend the amount of time required to fill the survey. Could you at least make the answer options less emotional? Like exchange happy for satisfied etc. I don't remember answering ecstatic in the Git survey but that could be my bad memory. That's a reasonable alternative. How about pleased? Any other people have an opinion? satisfied is good -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 10:12 AM, Luc Pionchon pionchon@gmail.com wrote: On Fri, Aug 19, 2011 at 03:34, Zeeshan Ali (Khattak) zeesha...@gnome.org wrote: [snip] Maybe they all lied? Don't you think it is a bit early to speculate on results? (...) Overall I can see already one clear result, even before the poll has started: We do not know who is using GNOME. Maybe this needs reflexion No, in the current scenario that is actually a success story. Maybe this will change when GNOME OS becomes a reality. -- Regards, Zeeshan Ali (Khattak) FSF member#5124 ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 13:14, Allan Day allanp...@gmail.com wrote: We already have a wealth of data about peoples' experiences with GNOME 3 Allan, this is interesting, what is the main pointer to access this data? ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 2:40 PM, Zeeshan Ali (Khattak) zeesha...@gnome.org wrote: On Fri, Aug 19, 2011 at 2:17 PM, Felipe Contreras felipe.contre...@gmail.com wrote: I didn't say this so far because it might sound like I am trying to make a joke but since you still insist on your assertions about the survey, I feel I must say this: How do you know people in general like to participate in surveys? It is my observation that most people do not like to do that, unless they have something to complain about. Now this observation of mine could very well be wrong but how do we know that? Do we do a survey to find out if people like to participate in surveys? Are you serious? That totally and completely speculative and unrealistic. What is speculative? I made it very clear that it is *my* observation and *if* it is correct, the results of this survey may very well be wrong. Do you have any evidence that suggests that my observations above are incorrect? Do you even know what speculation means? to make an inference based on inconclusive evidence; to surmise or conjecture [1] You don't have any evidence how often does this happens in real surveys, if at all. It's all based on conjecture. Have you ever participated in making a survey? No I have not but that does not necessarily mean what I said is incorrect and could just be ignored by pointing to examples of other surveys. If other people are ignoring an important issue, doesn't mean we should do the same. You are again going off-tracks. Let's go back to the point. You said: What if the results are all wrong and we don't have any way of knowing that? Would those results still be better than nothing in your opinion? You can ignore the results. Problem solved, is it not? I have, as I have explained, for the Git survey. In my experience, only the people that want to help in some way do spend the amount of time required to fill the survey. As I have explained to you many times before, git's user-base is mostly (if not all) geeks and those geeks know where the mailing-list is and be able to access the survey easily. Still, I am a geek and a very happy user of git but I didn't even know about the existence of this survey until you told me. Even then, I didn't care to participate. I am pretty sure I would have cared to participate if I had something to complain about its current or planned features. GNOME's user-base consists of people who do not even know what GNOME is so many of them will not be able to participate, especially if they are happy users. In short, example of git surveys are quite irrelevant here. So what if that's true? (I don't think so) At least I have a data-point of experience with surveys, you can discard it all you want, but what makes your speculation based on imaginary notions somehow more valid that my experience in real-world scenarios? At best you can say that they are both equally useless (I don't think so). The world is no filled with Zeeshans. Most people fill surveys truthfully. If you think otherwise, you can ignore the results. But again, as I said, if there's no survey on Earth you could trust, just ignore the results. Results by themselves cannot hurt you. In this case those results will really hurt since then you will have some numbers to back-up your claim of GNOME 3 is completely unusable. *If* your motivation for this survey has remained the same, you'll spread a lot of negative propaganda (which you already did even when you didn't have any numbers) and many people will just say Oh, people don't like this gnome 3 thingie, must be shit and will stay away from it. Even if you don't do that, there is many others who will use this data in that way. Aha, so that's what you are afraid. This survey will happen with or without GNOME's blessing. It would be in GNOME's best interest to improve the survey to get more useful results, and so far, I think many people have done so. However, at this point it's clear that you are not interested in improving the survey, all you are doing is making imaginary claims that lead to a dead-end; all surveys are pointless, because the answers might be lies. There's no way to go forward from there. If you have some *suggestions* how to improve the survey to avoid whatever issues you see, then say so, otherwise I'll not explain any more why this is flawed thinking that leads nowhere. Cheers. [1] http://en.wiktionary.org/wiki/speculate -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 1:14 PM, Allan Day allanp...@gmail.com wrote: Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 1:20 AM, Zeeshan Ali (Khattak) zeesha...@gnome.org wrote: On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras felipe.contre...@gmail.com wrote: ... Different people will understand the words GNOME/happy/very happy/ecstatic in different ways. Some might think 'GNOME' is their distro (including the lower levels of the stack), Which is why we ask more question to understand their level of geekness. That should help the make correlations; the people that use a terminal all the time more likely know that GNOME is just the DE. The people that don't have much experience might be confusing GNOME with the distribution. 'Geekness' is not the only thing that will affect people's understandings, and you haven't adequately measured that anyway. Plus that doesn't do anything to deal with the problem of what people understand by 'GNOME'. Likewise, 'happy' will be thought of differently by different people (a very odd word to include in a questionnaire, if you don't mind me saying): I think everyone understands the word happy. /ME wipes a mouthful of coffee from my monitor Then you haven't read enough of the survey research literature. ... In any case, if you have suggestions that don't have these problems, feel free to share them. My suggestion would be to give up entirely or to rethink the premise of your research. The latter is what I'd have advised when I was working as a research consultant, or what I would have told one of my students when I used to teach this stuff, for that matter. You've also got the representativeness problem. Your sample will inevitably be unrepresentative, probably highly so. Even if 100% of your *unrepresentative sample* tick the unhappy box, that doesn't tell you much about your target population: you might just have sampled every 'unhappy' GNOME user that's out there. If you can identify the bias, that's not a huge problem. So tell me - how will you accurately compensate for the effects of self-selection bias? What kinds of claims will you make about representativeness? tl;dr version: your survey results will be misleading. No, the results would not be misleading; the *analysis* of the results might. But different people can analyze them in different ways. The important thing is to get *some* results. It seems bizarre to suggest that research data is valid irrespective of how it is gathered. If your questionnaire does not provide valid measurements no amount of analysis can compensate. We already have a wealth of data about peoples' experiences with GNOME 3 and are working to address the issues that are being raised. It's great that you want to help, but this survey really won't be useful. Where? I haven't seen any. We've had incredible amounts of feedback; most (if not all) of which has been read, and which does get taken seriously. I also know that those of us who are influencing the design of GNOME 3 take a strong interest in peoples' experiences with it and ask them questions (that's certainly what I do). There's also a small series of user tests last I did Christmas, the results of which have been fed into the development process. Believe me, that is more than enough to be going on for now. (Some more user testing would be useful at some point in the future, though.) Allan -- IRC: aday on irc.gnome.org Blog: http://afaikblog.wordpress.com/ ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 3:17 PM, Allan Day allanp...@gmail.com wrote: Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 1:14 PM, Allan Day allanp...@gmail.com wrote: Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 1:20 AM, Zeeshan Ali (Khattak) zeesha...@gnome.org wrote: On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras felipe.contre...@gmail.com wrote: ... Different people will understand the words GNOME/happy/very happy/ecstatic in different ways. Some might think 'GNOME' is their distro (including the lower levels of the stack), Which is why we ask more question to understand their level of geekness. That should help the make correlations; the people that use a terminal all the time more likely know that GNOME is just the DE. The people that don't have much experience might be confusing GNOME with the distribution. 'Geekness' is not the only thing that will affect people's understandings, and you haven't adequately measured that anyway. Plus that doesn't do anything to deal with the problem of what people understand by 'GNOME'. It's easy to throw empty criticism. Provide *suggestions*. Likewise, 'happy' will be thought of differently by different people (a very odd word to include in a questionnaire, if you don't mind me saying): I think everyone understands the word happy. /ME wipes a mouthful of coffee from my monitor Then you haven't read enough of the survey research literature. That doesn't change the fact that everyone understands the word happy. ... In any case, if you have suggestions that don't have these problems, feel free to share them. My suggestion would be to give up entirely or to rethink the premise of your research. The latter is what I'd have advised when I was working as a research consultant, or what I would have told one of my students when I used to teach this stuff, for that matter. That's not helpful. If you are such a master, surely you can come up with a totally brand new user survey that is order of magnitude better. That would be greatly appreciated. You've also got the representativeness problem. Your sample will inevitably be unrepresentative, probably highly so. Even if 100% of your *unrepresentative sample* tick the unhappy box, that doesn't tell you much about your target population: you might just have sampled every 'unhappy' GNOME user that's out there. If you can identify the bias, that's not a huge problem. So tell me - how will you accurately compensate for the effects of self-selection bias? What kinds of claims will you make about representativeness? What would *you* do? tl;dr version: your survey results will be misleading. No, the results would not be misleading; the *analysis* of the results might. But different people can analyze them in different ways. The important thing is to get *some* results. It seems bizarre to suggest that research data is valid irrespective of how it is gathered. If your questionnaire does not provide valid measurements no amount of analysis can compensate. You can thrown an analysis saying all this data is crap if that makes you happier, but this survey won't eat babies. We already have a wealth of data about peoples' experiences with GNOME 3 and are working to address the issues that are being raised. It's great that you want to help, but this survey really won't be useful. Where? I haven't seen any. We've had incredible amounts of feedback; most (if not all) of which has been read, and which does get taken seriously. I also know that those of us who are influencing the design of GNOME 3 take a strong interest in peoples' experiences with it and ask them questions (that's certainly what I do). There's also a small series of user tests last I did Christmas, the results of which have been fed into the development process. Believe me, that is more than enough to be going on for now. (Some more user testing would be useful at some point in the future, though.) For a professor you should know better. I want the data. Anyway, I am going to ignore your comments, unless you provide some *suggestions* for improvement. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 4:08 PM, Andy Wingo wi...@pobox.com wrote: On Fri 19 Aug 2011 13:33, Felipe Contreras felipe.contre...@gmail.com writes: That's a reasonable alternative. How about pleased? Any other people have an opinion? You present yourself as reasonable by adjusting on the small points, but you ignore the feedback of greater importance. My opinion is that you are not the right person to lead an effort to gather feedback on GNOME. Is there anyone in the universe able to create a user survey worthy of GNOME? Can you convince him of doing so? In the meantime, this is the best that we have. I will continue listening for constructive feedback, but comments such as this is not good, you are doing it wrong, it's impossible, lead to nowhere. Besides, as Alan Cox said, it doesn't have to be perfect, like software, we can learn from the mistakes of the 2011 survey, and make a better one for 2012. Can we not? -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri 19 Aug 2011 13:33, Felipe Contreras felipe.contre...@gmail.com writes: That's a reasonable alternative. How about pleased? Any other people have an opinion? You present yourself as reasonable by adjusting on the small points, but you ignore the feedback of greater importance. My opinion is that you are not the right person to lead an effort to gather feedback on GNOME. The Git survey, AFAIU, was done _with_ the git developers. This one, if you manage to bully it through, will be _in spite of_ the GNOME developers. It will not have the effect you desire. Andy -- http://wingolog.org/ ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 2:13 PM, Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 4:08 PM, Andy Wingo wi...@pobox.com wrote: On Fri 19 Aug 2011 13:33, Felipe Contreras felipe.contre...@gmail.com writes: That's a reasonable alternative. How about pleased? Any other people have an opinion? You present yourself as reasonable by adjusting on the small points, but you ignore the feedback of greater importance. My opinion is that you are not the right person to lead an effort to gather feedback on GNOME. Is there anyone in the universe able to create a user survey worthy of GNOME? Can you convince him of doing so? Gathering feedback does not necessarily require an online user survey. As stated, for a project which currently targets, among others, users who do not care what parts of their operating system can be labelled GNOME a survey is not a very reliable way of gathering feedback. Have you ever tried to explain, to a person who doesn't have an interest in software, what GNOME actually is? In the meantime, this is the best that we have. I will continue listening for constructive feedback, but comments such as this is not good, you are doing it wrong, it's impossible, lead to nowhere. Besides, as Alan Cox said, it doesn't have to be perfect, like software, we can learn from the mistakes of the 2011 survey, and make a better one for 2012. Can we not? I urge you to consider the fact that if the majority of people subscribed to desktop-devel-list don't have faith in idea of an online user survey, an online user survey is probably not going to much have effect on the views of the people who contribute to the discussions on desktop-devel-list, and since most of the GNOME community read desktop-devel-list you can probably extend this to all of the other GNOME mailing lists and IRC channels as well. Sam ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On 19 August 2011 14:13, Felipe Contreras felipe.contre...@gmail.com wrote: Is there anyone in the universe able to create a user survey worthy of GNOME? Can you convince him of doing so? Do your survey with the questions you want, and come to your own conclusions. Blog about them if you want. You could even convince a distribution to include a popup with a link, although I think that's insane. Just don't tell people that it's from the GNOME project, in any way authorized or blessed by the ruling cabal[1] or developers. I'm pretty sure the majority of the people actually working on GNOME 3.2 don't want a survey at all. Sorry to be blunt. Richard. [1] http://ftp.gnome.org/conspiracy/ ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On 08/19/2011 09:13 PM, Felipe Contreras wrote: On Fri, Aug 19, 2011 at 4:08 PM, Andy Wingo wi...@pobox.com wrote: On Fri 19 Aug 2011 13:33, Felipe Contreras felipe.contre...@gmail.com writes: That's a reasonable alternative. How about pleased? Any other people have an opinion? You present yourself as reasonable by adjusting on the small points, but you ignore the feedback of greater importance. My opinion is that you are not the right person to lead an effort to gather feedback on GNOME. Is there anyone in the universe able to create a user survey worthy of GNOME? Can you convince him of doing so? In the meantime, this is the best that we have. I will continue listening for constructive feedback, but comments such as this is not good, you are doing it wrong, it's impossible, lead to nowhere. Besides, as Alan Cox said, it doesn't have to be perfect, like software, we can learn from the mistakes of the 2011 survey, and make a better one for 2012. Can we not? Hi again, I actually thought that Andy's feedback was constructive. Your approach, motives and way to handle this discussion are questionable. You have obviously failed to convince the GNOME community and GNOME developers that your survey would be useful, and I'm afraid nobody feels like taking leadership on the project neither. A lot of people have already told you that enough feedback has been gathered at this time. I doubt you will get much endorsement or help around here anymore (I could of course be wrong on that last point). To tell you the truth I have been involved in trying to run a survey for GNOME 2 years ago IIRC (and I think it's a recurrent project - you can find some old pages on our wiki) with a group of other people, and we reached the same conclusion: a survey will not help GNOME to get better. I obviously had very different motives (and GNOME 3 was not around). So if you want to help GNOME maybe you should discuss further with the design team and see how to contribute in a positive manner. I hope you will find my comments helpful as that's what I am trying to convey (help to someone who seems full of energy to do something to improve GNOME - and we need people like this). All the best. Fred ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Le vendredi 19 août 2011 à 16:08 +0300, Felipe Contreras a écrit : It's easy to throw empty criticism. Provide *suggestions*. Well, here’s a suggestion: since nobody knows how to address the correct target population or how to interpret the results, I suggest to spend our time fixing bugs instead. Cheers, -- .''`. Josselin Mouette : :' : `. `' `- ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, 2011-08-19 at 16:08 +0300, Felipe Contreras wrote: Likewise, 'happy' will be thought of differently by different people (a very odd word to include in a questionnaire, if you don't mind me saying): I think everyone understands the word happy. /ME wipes a mouthful of coffee from my monitor Then you haven't read enough of the survey research literature. That doesn't change the fact that everyone understands the word happy. Not necessary. Just to give an example - there is strong cultural influence how do you respond to simple question 'How are you'. In some cultures it is impolite to answer better then 'so so' and the normal answer is somehow along lines 'it could be worst, it could be better'. On the other hand the correct response in English is usually 'great' or 'fine' (to quote my teacher 'even if your house is burnt and your dog is terminally ill'). I have been warned to avoid 'standard' 'so so' response as I will be perceived as either impolite or after some large disaster because what I really meant was 'great'. (Somehow less directly related but also illustrates the problem of tricky words - in my native language friend means what in English is understood by close friend and many people whom I would call in English friend I would call in Polish acquaintance. Even though I know the difference I am less inclined to call people friends as in my mental model they are described by word 'acquaintance'). Of course this is 'just' cultural bias caused by people not being native speakers of English. You need to add individual bias. In each case it adds more and more 'noise' to survey. tl;dr version: your survey results will be misleading. No, the results would not be misleading; the *analysis* of the results might. But different people can analyze them in different ways. The important thing is to get *some* results. It seems bizarre to suggest that research data is valid irrespective of how it is gathered. If your questionnaire does not provide valid measurements no amount of analysis can compensate. You can thrown an analysis saying all this data is crap if that makes you happier, but this survey won't eat babies. I would argue that incorrect data (misinformation) is worst case then no data at all. Regards signature.asc Description: This is a digitally signed message part ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Gathering feedback does not necessarily require an online user survey. As stated, for a project which currently targets, among others, users who do not care what parts of their operating system can be labelled GNOME a survey is not a very reliable way of gathering feedback. Have you ever tried to explain, to a person who doesn't have an interest in software, what GNOME actually is? Yes - my MBA research was into Linux desktops some years ago and did involve looking at end users attitudes. The quick summary from then would be: Most users used the desktop they got by default (whether because they didn't know to to switch or were never annoyed enough to bother I didn't have time to find out) The managers wanted a system that was a free exact clone of windows look/feel because change was expensive (training, lost time etc) The technies in the organisation often inflicted their personal desktop preference on the entire company. If I wanted to look at the Gnome 3 is crap assertion I think I would tackle it a bit differently as so much online updating is going on nowdays. Collect statistics from a few Fedora and other mirror sites, correlate downloads together by IP/time and other evidence, and look at how many of them download which desktops or combination of desktops. Repeat this over time and plot graphs. Distro popularity shifts may also provide some evidence for this. The trouble is while that will tell you about movement and popularity it will not tell you why. So it's a way to evaluate the claim Gnome 3 is crap loads of people are changing or holding back on updating their desktop but it's not going to answer useful things. There is a bit of value in knowing if lots of people hate or love Gnome 3, but the real value is knowing how it could be better for users, and counting downloads won't do that. And if real non-technical end users are like the ones I dealt with then asking them probably won't help either. Particularly in the business world to many of them at the time Gnone was 'click on this splodge in the morning to write letters' 'click on that thing in the corner to turn it off'. They are not decision makers either - impress their boss 8) The more interested and technically motivated people on the other hand can tell you stuff, power users particularly. They tell you stuff that reflects a particular use and understanding case though. Similarly you can learn an enormous amount by seeing what people are struggling with and what they do to the desktop - eg the various 'how to fix Gnome 3' pages tell you a lot about what people wanted and which is non-obvious for configuration. They are also from people who liked it enough to persevere so made an effort. I urge you to consider the fact that if the majority of people subscribed to desktop-devel-list don't have faith in idea of an online user survey, an online user survey is probably not going to much have effect on the views of the people who contribute to the discussions on desktop-devel-list, and since most of the GNOME community read desktop-devel-list you can probably extend this to all of the other GNOME mailing lists and IRC channels as well. Some days I think Miguel got the Ximian monkey dead right, except that there should have been three of them. Alan ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 5:02 PM, Frederic Muller fr...@gnome.org wrote: On 08/19/2011 09:13 PM, Felipe Contreras wrote: On Fri, Aug 19, 2011 at 4:08 PM, Andy Wingo wi...@pobox.com wrote: On Fri 19 Aug 2011 13:33, Felipe Contreras felipe.contre...@gmail.com writes: That's a reasonable alternative. How about pleased? Any other people have an opinion? You present yourself as reasonable by adjusting on the small points, but you ignore the feedback of greater importance. My opinion is that you are not the right person to lead an effort to gather feedback on GNOME. Is there anyone in the universe able to create a user survey worthy of GNOME? Can you convince him of doing so? In the meantime, this is the best that we have. I will continue listening for constructive feedback, but comments such as this is not good, you are doing it wrong, it's impossible, lead to nowhere. Besides, as Alan Cox said, it doesn't have to be perfect, like software, we can learn from the mistakes of the 2011 survey, and make a better one for 2012. Can we not? I actually thought that Andy's feedback was constructive. Really? Then you can take that feedback and translate it into concrete actions, right? To me, that sounds as useful as the advice of some academic that says that no software should ever be deployed without strong static analysis. He might be right, but if he is not offering himself to do the job he is proposing, what's the point? IOW; Talk is cheap, show me the code. Your approach, motives and way to handle this discussion are questionable. You have obviously failed to convince the GNOME community and GNOME developers that your survey would be useful, and I'm afraid nobody feels like taking leadership on the project neither. Yeah, I failed at an impossible task, maybe. A lot of people have already told you that enough feedback has been gathered at this time. I doubt you will get much endorsement or help around here anymore (I could of course be wrong on that last point). To tell you the truth I have been involved in trying to run a survey for GNOME 2 years ago IIRC (and I think it's a recurrent project - you can find some old pages on our wiki) with a group of other people, and we reached the same conclusion: a survey will not help GNOME to get better. I obviously had very different motives (and GNOME 3 was not around). I strongly disagree. You can't know that until you actually *try*. Why are you so afraid to try? So if you want to help GNOME maybe you should discuss further with the design team and see how to contribute in a positive manner. All I am trying to do is get some user feedback. Without such feedback I doubt any kind of discussion on the design would be fruitful, because it all be dismissed based on assumptions and wishful thinking. Cheers. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 5:42 PM, Josselin Mouette j...@debian.org wrote: Le vendredi 19 août 2011 à 16:08 +0300, Felipe Contreras a écrit : It's easy to throw empty criticism. Provide *suggestions*. Well, here’s a suggestion: since nobody knows how to address the correct target population or how to interpret the results, I suggest to spend our time fixing bugs instead. Yes, because we are absolutely and positively certain that fixing these bugs is exactly what GNOME users want. There is no possibility that they want something else, or that the prioritization is not ideal. By definition, whatever GNOME does, is what the users want, and to suggest otherwise is heresy. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 4:55 PM, Sam Thursfield sss...@gmail.com wrote: On Fri, Aug 19, 2011 at 2:13 PM, Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 4:08 PM, Andy Wingo wi...@pobox.com wrote: On Fri 19 Aug 2011 13:33, Felipe Contreras felipe.contre...@gmail.com writes: That's a reasonable alternative. How about pleased? Any other people have an opinion? You present yourself as reasonable by adjusting on the small points, but you ignore the feedback of greater importance. My opinion is that you are not the right person to lead an effort to gather feedback on GNOME. Is there anyone in the universe able to create a user survey worthy of GNOME? Can you convince him of doing so? Gathering feedback does not necessarily require an online user survey. Indeed, do you have a better suggestion? As stated, for a project which currently targets, among others, users who do not care what parts of their operating system can be labelled GNOME a survey is not a very reliable way of gathering feedback. Have you ever tried to explain, to a person who doesn't have an interest in software, what GNOME actually is? Again, do you have a suggestion to get feedback in a more useful way? In the meantime, this is the best that we have. I will continue listening for constructive feedback, but comments such as this is not good, you are doing it wrong, it's impossible, lead to nowhere. Besides, as Alan Cox said, it doesn't have to be perfect, like software, we can learn from the mistakes of the 2011 survey, and make a better one for 2012. Can we not? I urge you to consider the fact that if the majority of people subscribed to desktop-devel-list don't have faith in idea of an online user survey, an online user survey is probably not going to much have effect on the views of the people who contribute to the discussions on desktop-devel-list, and since most of the GNOME community read desktop-devel-list you can probably extend this to all of the other GNOME mailing lists and IRC channels as well. So the status quo, where there are absolutely no numbers whatsoever is preferred. Any attempt to gather quantifiable feedback is discouraged. IOW; the GNOME community does not care about what users have to say at all. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 4:56 PM, Richard Hughes hughsi...@gmail.com wrote: On 19 August 2011 14:13, Felipe Contreras felipe.contre...@gmail.com wrote: Is there anyone in the universe able to create a user survey worthy of GNOME? Can you convince him of doing so? Do your survey with the questions you want, and come to your own conclusions. Blog about them if you want. You could even convince a distribution to include a popup with a link, although I think that's insane. Just don't tell people that it's from the GNOME project, in any way authorized or blessed by the ruling cabal[1] or developers. I'm pretty sure the majority of the people actually working on GNOME 3.2 don't want a survey at all. Sorry to be blunt. No, thanks for the direct feedback. So basically you are saying there's no way any survey of any quality would be blessed by the GNOME community. That certainly clarifies things. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Thu, Aug 18, 2011 at 10:37:33PM +0100, Alan Cox wrote: Doing nothing achieves nothing, doing something achieves learning. You may well not learn what you intended but you will learn something including quite possibly how to do future surveys better. Any survey that isn't a carefully controlled randomly selected sample of users doesn't result in learning. It results in data that forms some sort of rorschach blot. Everyone will see what they want to see. Those who believe that Gnome 3 is a step back will point out that the majority of responses are negative. Those who believe it's a step forward will point out that happy users are going to be far less inclined to respond. There's no way whatsoever to determine how representative the responses are, and so there's no way whatsoever to learn anything about the population. All we'd learn is that some users like Gnome 3 and some users don't, and that's something we *already know*. So we'd gain nothing, but we'd guarantee another huge set of arguments which would themselves also tell us nothing. I'm not saying its necessarily a great approach but it's vastly superior to people sitting around picking holes in the idea until it never happens. I disagree. Doing something that sucks more time and energy away from development without actually telling us anything in return is worse than that not happening. Felipe is obviously free to do whatever he wants, but there's no benefit in Gnome itself participating in any way. If we want to find out what our users think then the only way to do that is to have professional involvement and a random sample set. -- Matthew Garrett | mj...@srcf.ucam.org ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 7:24 PM, Matthew Garrett mj...@srcf.ucam.org wrote: On Thu, Aug 18, 2011 at 10:37:33PM +0100, Alan Cox wrote: Doing nothing achieves nothing, doing something achieves learning. You may well not learn what you intended but you will learn something including quite possibly how to do future surveys better. Any survey that isn't a carefully controlled randomly selected sample of users doesn't result in learning. Unless the biases are identified, which we are trying to do. Moreover, I have tried to push the idea to have an automatic notification, which would maximize the number of responders, and thus increase the randomization. But apparent this idea is not welcome. So, ideas to improve the randomization are dismissed, and then you say without randomization, the survey is not useful. IOW; you are intentionally deadlocking the proposal. It results in data that forms some sort of rorschach blot. It might if you look at it as a whole, but you can try to dissect it. Everyone will see what they want to see. Those who believe that Gnome 3 is a step back will point out that the majority of responses are negative. Those who believe it's a step forward will point out that happy users are going to be far less inclined to respond. There's no way whatsoever to determine how representative the responses are, and so there's no way whatsoever to learn anything about the population. All we'd learn is that some users like Gnome 3 and some users don't, and that's something we *already know*. So we'd gain nothing, but we'd guarantee another huge set of arguments which would themselves also tell us nothing. That's an assumption. What if we get 10 million responses? Would you still claim that the results are not representative? I think only *after* getting the results you would be able to say anything about it's representativeness. Something more realistic, say you get at least 300 responses that don't have any geek bias, that would be more than enough to make some statistically significant conclusions. I'm not saying its necessarily a great approach but it's vastly superior to people sitting around picking holes in the idea until it never happens. I disagree. Doing something that sucks more time and energy away from development without actually telling us anything in return is worse than that not happening. Felipe is obviously free to do whatever he wants, but there's no benefit in Gnome itself participating in any way. If we want to find out what our users think then the only way to do that is to have professional involvement and a random sample set. This is not sucking any time and energy from anybody, I just need access to the server that has limesurvey installed, or somebody else can do that (can't take that much time), I would contact all the relevant news sites and make the relevant posts in social media. All that that is needed from GNOME people is a blessing. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, 2011-08-19 at 19:42 +0300, Felipe Contreras wrote: On Fri, Aug 19, 2011 at 4:56 PM, Richard Hughes hughsi...@gmail.com wrote: On 19 August 2011 14:13, Felipe Contreras felipe.contre...@gmail.com wrote: Is there anyone in the universe able to create a user survey worthy of GNOME? Can you convince him of doing so? Do your survey with the questions you want, and come to your own conclusions. Blog about them if you want. You could even convince a distribution to include a popup with a link, although I think that's insane. Just don't tell people that it's from the GNOME project, in any way authorized or blessed by the ruling cabal[1] or developers. I'm pretty sure the majority of the people actually working on GNOME 3.2 don't want a survey at all. Sorry to be blunt. No, thanks for the direct feedback. So basically you are saying there's no way any survey of any quality would be blessed by the GNOME community. That certainly clarifies things. It seems obvious from most responses here that there are not very many people within the GNOME community that think that this sort of a survey would be beneficial, and worry that it may even be counter-productive. In response to this realization, you have apparently shifted into outrage mode. You pretend that it is impossible to simultaneously care about what users while also opposing a user survey that has no hope of being a representative sample of users. It is possible for well-meaning people to come to different conclusions on the best methods for achieving a certain goal. It seems that most people here don't agree with your methods. Please accept the fact that this does not mean that they hate users, despite your attempts to conflate the two things. You are free to proceed with your survey on your own. Others are free to not wish to join you. It's that simple. Can you please stop the faux outrage? jonner ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Any survey that isn't a carefully controlled randomly selected sample of users doesn't result in learning. It results in data that forms some You need truely or reasonably random samples for certain kinds of activities and analysis in particularly quantitative analysis when you want to perform p tests and the like. You don't need it in order to learn merely to generate statistical proofs and those are often quite useless anyway. Proviing gnome 3 is great/indifferent/sucks doesn't have much value. You do not need it for explorative learning. Small children do not need to open a statistically valid sample of randomly chosen doors to learn about doors ! I for one would not be surprised if a lot of responses were not more positive than some seem to think. There has been time for people to use it and adjust and apply the fixes. Even odder there is no Gnome fork. If as I hear 'Gnome 3 is hated by technical people' and there are enough who care there ought to be a Gnome fork by now. But what do we have - exde, dead, turned into a one page rant and no code Mate - described by phoronix as The Mate Desktop Environment fork of GNOME2 was started by an Arch Linux user back in June, but it hasn't yet gained too much traction and is mostly just talked about on various forums around the web. which about sums it up. sort of rorschach blot. Everyone will see what they want to see. Those who believe that Gnome 3 is a step back will point out that the majority of responses are negative. Those who believe it's a step forward will point out that happy users are going to be far less inclined to respond. You seem to be assuming the results and that the only question of interest is does gnome 3 suck. nothing, but we'd guarantee another huge set of arguments which would themselves also tell us nothing. Those will tell you a lot if someone analyses them. Again you may not be able to do formal mathematical tests on them but so what. If we want to find out what our users think then the only way to do that is to have professional involvement and a random sample set. Of course, and the only way to produce a kernel or desktop is to hire professionals to do it for you no doubt. Alan ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 10:40 AM, Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 4:55 PM, Sam Thursfield sss...@gmail.com wrote: Gathering feedback does not necessarily require an online user survey. Indeed, do you have a better suggestion? There are several other ways to get feedback. For example, user testing. I'm sure all the major distributions have done some user testing. Most large companies have a whole user testing team/group. I'm not a user testing expert but it involves giving people (both new and experienced) tasks to do, watching how they do it (without interfering) and then asking them about their experience. I know people who have successfully used http://www.usertesting.com/ for web sites. I don't know if a similar, inexpensive option exists for desktop software or not. Stormy ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 8:25 PM, Stormy Peters sto...@gnome.org wrote: On Fri, Aug 19, 2011 at 10:40 AM, Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 4:55 PM, Sam Thursfield sss...@gmail.com wrote: Gathering feedback does not necessarily require an online user survey. Indeed, do you have a better suggestion? There are several other ways to get feedback. For example, user testing. I'm sure all the major distributions have done some user testing. Most large companies have a whole user testing team/group. And where are the results? Without evidence it's only wishful thinking. I'm not a user testing expert but it involves giving people (both new and experienced) tasks to do, watching how they do it (without interfering) and then asking them about their experience. I know people who have successfully used http://www.usertesting.com/ for web sites. I don't know if a similar, inexpensive option exists for desktop software or not. Right, so nobody is going to do this. Is there any better suggestion that would actually be implemented? -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 8:15 PM, Jonathon Jongsma jonat...@quotidian.org wrote: On Fri, 2011-08-19 at 19:42 +0300, Felipe Contreras wrote: On Fri, Aug 19, 2011 at 4:56 PM, Richard Hughes hughsi...@gmail.com wrote: On 19 August 2011 14:13, Felipe Contreras felipe.contre...@gmail.com wrote: Is there anyone in the universe able to create a user survey worthy of GNOME? Can you convince him of doing so? Do your survey with the questions you want, and come to your own conclusions. Blog about them if you want. You could even convince a distribution to include a popup with a link, although I think that's insane. Just don't tell people that it's from the GNOME project, in any way authorized or blessed by the ruling cabal[1] or developers. I'm pretty sure the majority of the people actually working on GNOME 3.2 don't want a survey at all. Sorry to be blunt. No, thanks for the direct feedback. So basically you are saying there's no way any survey of any quality would be blessed by the GNOME community. That certainly clarifies things. It seems obvious from most responses here that there are not very many people within the GNOME community that think that this sort of a survey would be beneficial, and worry that it may even be counter-productive. In response to this realization, you have apparently shifted into outrage mode. You pretend that it is impossible to simultaneously care about what users while also opposing a user survey that has no hope of being a representative sample of users. You might say you do, and you might even believe so, but if your actions demonstrate otherwise, perhaps you do not. If the GNOME community really cared about what users have to say, and this survey indeed does not have any hope of having a representative sample of users (I disagree), then wouldn't they take the reins and do it properly? It is possible for well-meaning people to come to different conclusions on the best methods for achieving a certain goal. Yes, whenever I have a disagreement on a method to develop some software, I just go ahead and do it that way, and then say; see? this is how it should be done. Saying you are wrong is easy, anybody can do that. It seems that most people here don't agree with your methods. Please accept the fact that this does not mean that they hate users, despite your attempts to conflate the two things. I would, if they went ahead with the right methods and got some user feedback, if not in the form of a survey, in any method. You are free to proceed with your survey on your own. Others are free to not wish to join you. It's that simple. Can you please stop the faux outrage? Sure, I just wanted to make things clear. In fact, if they cared about user feedback, there would be some numbers available somewhere, and I wouldn't have to do this. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On 19 August 2011 18:42, Felipe Contreras felipe.contre...@gmail.com wrote: Sure, I just wanted to make things clear. In fact, if they cared about user feedback, there would be some numbers available somewhere, and I wouldn't have to do this. We're not asking you to do anything. Please just run the poll on your personal blog and stop getting aggressive with developers on this mailing list. Thanks, Richard. ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Really ought to stay out of this thread but there is one point that is important to address below. On Fri, Aug 19, 2011 at 7:17 AM, Felipe Contreras felipe.contre...@gmail.com wrote: But again, as I said, if there's no survey on Earth you could trust, just ignore the results. Results by themselves cannot hurt you. This isn't right. Poorly understood results lead you draw incorrect conclusions which lead you astray. Or at minimum cause the marketing team undue stress trying to explain that the results don't make any sense. Really, I don't understand why anyone would want to go through the trouble if the results aren't *useful*. Now, if you want to do something productive I encourage you to work with the guidance of Allan and others. Jon ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Thu, Aug 18, 2011 at 10:04 PM, Olav Vitters o...@vitters.nl wrote: On Thu, Aug 18, 2011 at 08:03:45PM +0300, Felipe Contreras wrote: I can only think of one reason why somebody would provide criticism without suggestions for improvement... 1. Because they cannot think of a good suggestion. Then surely I cannot be blamed for not coming with one either. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 8:50 PM, Richard Hughes hughsi...@gmail.com wrote: On 19 August 2011 18:42, Felipe Contreras felipe.contre...@gmail.com wrote: Sure, I just wanted to make things clear. In fact, if they cared about user feedback, there would be some numbers available somewhere, and I wouldn't have to do this. We're not asking you to do anything. I am not suggesting you are. Please just run the poll on your personal blog and stop getting aggressive with developers on this mailing list. I am not being aggressive. All I am asking is for clarification; is there *anything* I could do to make the survey more acceptable to you guys, or are you opposed to the very idea of having a user survey blessed by GNOME? -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 08:14:25PM +0300, Felipe Contreras wrote: On Fri, Aug 19, 2011 at 7:24 PM, Matthew Garrett mj...@srcf.ucam.org wrote: Any survey that isn't a carefully controlled randomly selected sample of users doesn't result in learning. Unless the biases are identified, which we are trying to do. You can only identify the biases if you already know the population, and you can only know the population if you've got a random sample set to begin with. Moreover, I have tried to push the idea to have an automatic notification, which would maximize the number of responders, and thus increase the randomization. But apparent this idea is not welcome. It doesn't help. The people most likely to respond to an irritating popup that disrupts their work are people who already feel that gnome 3 is an irritating piece of software that disrupts their work. You can't get a random sample in-band. So, ideas to improve the randomization are dismissed, and then you say without randomization, the survey is not useful. IOW; you are intentionally deadlocking the proposal. I am saying that your results aren't useful unless your sample is random. I don't know of a good way to obtain a representative sample. Everyone will see what they want to see. Those who believe that Gnome 3 is a step back will point out that the majority of responses are negative. Those who believe it's a step forward will point out that happy users are going to be far less inclined to respond. There's no way whatsoever to determine how representative the responses are, and so there's no way whatsoever to learn anything about the population. All we'd learn is that some users like Gnome 3 and some users don't, and that's something we *already know*. So we'd gain nothing, but we'd guarantee another huge set of arguments which would themselves also tell us nothing. That's an assumption. What if we get 10 million responses? Would you still claim that the results are not representative? Yes, because you have no idea how big the population is. Maybe 10 million is the total population and it's representative. Maybe it's 50% of the population, disproportionately biased towards those of a given prior opinion. You can't know. I think only *after* getting the results you would be able to say anything about it's representativeness. Something more realistic, say you get at least 300 responses that don't have any geek bias, that would be more than enough to make some statistically significant conclusions. It really wouldn't. I disagree. Doing something that sucks more time and energy away from development without actually telling us anything in return is worse than that not happening. Felipe is obviously free to do whatever he wants, but there's no benefit in Gnome itself participating in any way. If we want to find out what our users think then the only way to do that is to have professional involvement and a random sample set. This is not sucking any time and energy from anybody, I just need access to the server that has limesurvey installed, or somebody else can do that (can't take that much time), I would contact all the relevant news sites and make the relevant posts in social media. All that that is needed from GNOME people is a blessing. The sucking of time and energy would come from the argument over the results afterwards. -- Matthew Garrett | mj...@srcf.ucam.org ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
(Resend: Managed to leave d-d-l off Cc: by accident) On Fri, Aug 19, 2011 at 06:15:03PM +0100, Alan Cox wrote: Any survey that isn't a carefully controlled randomly selected sample of users doesn't result in learning. It results in data that forms some You need truely or reasonably random samples for certain kinds of activities and analysis in particularly quantitative analysis when you want to perform p tests and the like. You don't need it in order to learn merely to generate statistical proofs and those are often quite useless anyway. Proviing gnome 3 is great/indifferent/sucks doesn't have much value. You do not need it for explorative learning. Small children do not need to open a statistically valid sample of randomly chosen doors to learn about doors ! I am all for making it easier for people to give feedback about Gnome, but presenting it as a survey gives a strong implication that the results are meaningful as an aggregate rather than as a collection of anecdotes. If we want to hear form users, let's make it easy for users to talk to us. A survey isn't the way to achieve that. sort of rorschach blot. Everyone will see what they want to see. Those who believe that Gnome 3 is a step back will point out that the majority of responses are negative. Those who believe it's a step forward will point out that happy users are going to be far less inclined to respond. You seem to be assuming the results and that the only question of interest is does gnome 3 suck. I'm assuming that the sort of people who are going to go to the effort of filling out a survey are likely to be closer to the population discussing things on lwn than the population of usres in general. That may be entirely untrue! But if we get the opposite results then it still doesn't tell us anything that's actually true, and it's still an opportunity to argue the issue rather than focus on making software better. If we want to find out what our users think then the only way to do that is to have professional involvement and a random sample set. Of course, and the only way to produce a kernel or desktop is to hire professionals to do it for you no doubt. If you went back to 1991 and wanted a production-quality kernel within a year, Linux probably wouldn't be your starting point. There'd be a learning process involved with setting up a professional-quality survey team, and the first few attempts would be pretty buggy. We'd get there in time, but until then... -- Matthew Garrett | mj...@srcf.ucam.org ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 12:12 PM, Felipe Contreras felipe.contre...@gmail.com wrote: I am not being aggressive. All I am asking is for clarification; is there *anything* I could do to make the survey more acceptable to you guys, or are you opposed to the very idea of having a user survey blessed by GNOME? Your answers sound aggressive to me but I think that's totally understandable given all the negative feedback. I gave my feedback. I'd want the survey to be much more detailed. What do you think about this menu option on Cheese seems like it would give more feedback than do you like GNOME? But I do not have time to help come up with the questions, so I agree with many folks that say you'll have to take the feedback you've gotten and move forward. Giving feedback does not mean providing alternatives or working on the project. It's easy to give feedback. It's much harder and more time consuming to incorporate that feedback. You asked for feedback, you got some. If you want those people's approval, then you'll probably have to incorporate that feedback. If you aren't planning on incorporating it, then it's probably best to stop insisting that people need to provide alternatives if they give negative feedback. Obviously, you don't need everyone's approval to move forward. Rarely does any project get 100% approval. How you move forward, how much feedback you want and how you use that feedback is up to you. Good luck! Stormy ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 21:20, Matthew Garrett mj...@srcf.ucam.org wrote: The people most likely to respond to an irritating popup that disrupts their work are people who ... ... do not use GNOME 3. GNOME 3 is designed to reduce distraction and interruption and to put you in control. Our new notifications system subtly presents messages and will save them until you are ready for them, and the GNOME 3 panel has been styled so that it is part of the background, not the foreground. These changes allow you to focus on your creative tasks. I hope this brings a bit of humor in this thread. ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 14:46, Luc Pionchon pionchon@gmail.com wrote: On Fri, Aug 19, 2011 at 13:14, Allan Day allanp...@gmail.com wrote: We already have a wealth of data about peoples' experiences with GNOME 3 Allan, this is interesting, what is the main pointer to access this data? Allan, you may have missed it in this epic thread; I am sincerely interested in reading this data, would you be kind enough to point me at it? Luc ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 9:20 PM, Matthew Garrett mj...@srcf.ucam.org wrote: On Fri, Aug 19, 2011 at 08:14:25PM +0300, Felipe Contreras wrote: On Fri, Aug 19, 2011 at 7:24 PM, Matthew Garrett mj...@srcf.ucam.org wrote: Any survey that isn't a carefully controlled randomly selected sample of users doesn't result in learning. Unless the biases are identified, which we are trying to do. You can only identify the biases if you already know the population, and you can only know the population if you've got a random sample set to begin with. That's not true. You might need that if you want to account for *all* the biases, which nobody can do anyway. What most people do is try to figure them out, chances are you might be missing some of the biases, but hopefully the unidentified misrepresented group won't be that big anyway, and thus wouldn't affect so much the analysis. If it turns out that a significant bias is not identified beforehand, that can be tackled in the next survey in 2012. Moreover, I have tried to push the idea to have an automatic notification, which would maximize the number of responders, and thus increase the randomization. But apparent this idea is not welcome. It doesn't help. The people most likely to respond to an irritating popup that disrupts their work are people who already feel that gnome 3 is an irritating piece of software that disrupts their work. You can't get a random sample in-band. It doesn't help? It does randomize the sample more, doesn't it? Maybe it's not perfectly randomized, but nothing can ever be perfect. So, ideas to improve the randomization are dismissed, and then you say without randomization, the survey is not useful. IOW; you are intentionally deadlocking the proposal. I am saying that your results aren't useful unless your sample is random. I don't know of a good way to obtain a representative sample. There's no such thing as 0% random, or 100% random, all we can thrive for is to increase the randomness. And I already explained that non-random samples are already useful if you can identify the biases. Everyone will see what they want to see. Those who believe that Gnome 3 is a step back will point out that the majority of responses are negative. Those who believe it's a step forward will point out that happy users are going to be far less inclined to respond. There's no way whatsoever to determine how representative the responses are, and so there's no way whatsoever to learn anything about the population. All we'd learn is that some users like Gnome 3 and some users don't, and that's something we *already know*. So we'd gain nothing, but we'd guarantee another huge set of arguments which would themselves also tell us nothing. That's an assumption. What if we get 10 million responses? Would you still claim that the results are not representative? Yes, because you have no idea how big the population is. Maybe 10 million is the total population and it's representative. Maybe it's 50% of the population, disproportionately biased towards those of a given prior opinion. You can't know. Do you have any idea what is the likelihood of that happening? Try throwing a dice 10 times and always getting 1-3. Even if the dice is rigged, it's very unlikely. It gets exponentially less likely 1 million times. I think only *after* getting the results you would be able to say anything about it's representativeness. Something more realistic, say you get at least 300 responses that don't have any geek bias, that would be more than enough to make some statistically significant conclusions. It really wouldn't. Yes it would. Check Cochran's formulas. 300 unbiased responses gives you already good statistical power, after a certain point it doesn't matter much what is the total population; 10m, 30m, 1m. The likelihood that would would get 300 unbiased responses all pointing to the wrong direction is almost nothing, in fact a few dozens would do (if they are truly random). There are some simple calculators online: http://www.surveysystem.com/sscalc.htm But yeah, since there's going to be bias, you need more. I disagree. Doing something that sucks more time and energy away from development without actually telling us anything in return is worse than that not happening. Felipe is obviously free to do whatever he wants, but there's no benefit in Gnome itself participating in any way. If we want to find out what our users think then the only way to do that is to have professional involvement and a random sample set. This is not sucking any time and energy from anybody, I just need access to the server that has limesurvey installed, or somebody else can do that (can't take that much time), I would contact all the relevant news sites and make the relevant posts in social media. All that that is needed from GNOME people is a blessing. The sucking of time and energy would come from the argument over
Re: GNOME user survey 2011 (v4)
On 19 August 2011 20:26, Felipe Contreras felipe.contre...@gmail.com wrote: ...To me GNOME is hitting everything in the room as it's going forward, and saying; I'm fine, I know where I'm going... To me, the sun is shining through the windows of a freshly redecorated room. If you have specific problems with GNOME, file bugs and discuss things with maintainers and designers. It'll be way more effective at changing things than writing a survey. Richard. ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 9:28 PM, Matthew Garrett mj...@srcf.ucam.org wrote: (Resend: Managed to leave d-d-l off Cc: by accident) On Fri, Aug 19, 2011 at 06:15:03PM +0100, Alan Cox wrote: Any survey that isn't a carefully controlled randomly selected sample of users doesn't result in learning. It results in data that forms some You need truely or reasonably random samples for certain kinds of activities and analysis in particularly quantitative analysis when you want to perform p tests and the like. You don't need it in order to learn merely to generate statistical proofs and those are often quite useless anyway. Proviing gnome 3 is great/indifferent/sucks doesn't have much value. You do not need it for explorative learning. Small children do not need to open a statistically valid sample of randomly chosen doors to learn about doors ! I am all for making it easier for people to give feedback about Gnome, but presenting it as a survey gives a strong implication that the results are meaningful as an aggregate rather than as a collection of anecdotes. If we want to hear form users, let's make it easy for users to talk to us. A survey isn't the way to achieve that. Again, any better suggestions? I tried many of them back in 2007, and got nowhere, I think a user survey is the best one we've got. sort of rorschach blot. Everyone will see what they want to see. Those who believe that Gnome 3 is a step back will point out that the majority of responses are negative. Those who believe it's a step forward will point out that happy users are going to be far less inclined to respond. You seem to be assuming the results and that the only question of interest is does gnome 3 suck. I'm assuming that the sort of people who are going to go to the effort of filling out a survey are likely to be closer to the population discussing things on lwn than the population of usres in general. That may be entirely untrue! But if we get the opposite results then it still doesn't tell us anything that's actually true, and it's still an opportunity to argue the issue rather than focus on making software better. We most likely are going to be able to identify that bias. Let's make some wild guesses; 50% of the people that use GNOME 3 like it, and 50% don't. Of that amount, 90% seem to be geeks. In the remaining 10%, the people that use GNOME 2 show 80% happiness, and of GNOME 3 it's only 60%. But you still don't think there's any value in there, fair enough. Then we dig through that 40% subset that didn't like GNOME 3 and take a look at their comments, and we find Very strange, Can't get used to it, and things like that. At that point we might want to see if they left an email to contact them, and then try to gather more detailed feedback. I think there's a chance that this survey could tell us something that's actually true. If we want to find out what our users think then the only way to do that is to have professional involvement and a random sample set. Of course, and the only way to produce a kernel or desktop is to hire professionals to do it for you no doubt. If you went back to 1991 and wanted a production-quality kernel within a year, Linux probably wouldn't be your starting point. There'd be a learning process involved with setting up a professional-quality survey team, and the first few attempts would be pretty buggy. We'd get there in time, but until then... Until then it's better to have nothing? -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 10:47 PM, Richard Hughes hughsi...@gmail.com wrote: On 19 August 2011 20:26, Felipe Contreras felipe.contre...@gmail.com wrote: ...To me GNOME is hitting everything in the room as it's going forward, and saying; I'm fine, I know where I'm going... To me, the sun is shining through the windows of a freshly redecorated room. Either you are hallucinating, or you are not getting the analogy. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 10:53:46PM +0300, Felipe Contreras wrote: If you went back to 1991 and wanted a production-quality kernel within a year, Linux probably wouldn't be your starting point. There'd be a learning process involved with setting up a professional-quality survey team, and the first few attempts would be pretty buggy. We'd get there in time, but until then... Until then it's better to have nothing? It's better to have no data than to have misleading data. -- Matthew Garrett | mj...@srcf.ucam.org ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 9:34 PM, Stormy Peters sto...@gnome.org wrote: On Fri, Aug 19, 2011 at 12:12 PM, Felipe Contreras felipe.contre...@gmail.com wrote: I am not being aggressive. All I am asking is for clarification; is there *anything* I could do to make the survey more acceptable to you guys, or are you opposed to the very idea of having a user survey blessed by GNOME? Your answers sound aggressive to me but I think that's totally understandable given all the negative feedback. Perhaps you are not used to straight-forward communication. I'm not trying to aggravate anyone. I gave my feedback. I'd want the survey to be much more detailed. What do you think about this menu option on Cheese seems like it would give more feedback than do you like GNOME? But I do not have time to help come up with the questions, so I agree with many folks that say you'll have to take the feedback you've gotten and move forward. Trying to do that would create a huge survey that most likely most people will not even try to answer. If somebody really detests certain menu option on Cheese, I'm sure they'll let you know in the comments box. Who knows, maybe it turns out the part that most people are not satisfied with is the documentation, those kinds of results might trigger some interesting debate. Or maybe you are right, and we wouldn't not get anything useful, but at least we would have some ideas for the next survey. Giving feedback does not mean providing alternatives or working on the project. It's easy to give feedback. It's much harder and more time consuming to incorporate that feedback. You asked for feedback, you got some. If you want those people's approval, then you'll probably have to incorporate that feedback. I have incorporated all the feedback that can be incorporated. The rest is too vague, or not actionable. What do *you* think must absolutely be changed in the survey? If you aren't planning on incorporating it, then it's probably best to stop insisting that people need to provide alternatives if they give negative feedback. Huh? That's a very broad statement. Let's be clear, I have not turned away any feedback. Let's analyze for example the claims by Allan Day: --- When you do survey research, you have to be certain that one person understands the questions in the same way that another person does. Generally yes. Is that achievable in all the questions in this survey? Probably not (would love to hear some suggestions otherwise). Which why some other questions are asked to determine the people that might be thinking in other terms. (I already explained that) You've also got the representativeness problem. Your sample will inevitably be unrepresentative, probably highly so. Says who? What if we get 10 million answers? That would be such a big chunk of the total population that this problem is not a big deal. Or what if there's a notification app embedded in GNOME 3.2. That would not only maximize the reponders, but also maximize the randomness. Wouldn't it? (I already explained that) your survey results will be misleading That's very useful. Now, how about some ideas to make the results less misleading? --- What exactly do you want me to do with that feedback? (aside from what I have already done) I am all ears. Obviously, you don't need everyone's approval to move forward. Rarely does any project get 100% approval. How you move forward, how much feedback you want and how you use that feedback is up to you. I want the approval of the GNOME community, and I am willing to accept all suggestions for improvement in order to get that. So, what should I do? -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 10:26:08PM +0300, Felipe Contreras wrote: Yes, because you have no idea how big the population is. Maybe 10 million is the total population and it's representative. Maybe it's 50% of the population, disproportionately biased towards those of a given prior opinion. You can't know. Do you have any idea what is the likelihood of that happening? Try throwing a dice 10 times and always getting 1-3. Even if the dice is rigged, it's very unlikely. It gets exponentially less likely 1 million times. That's clearly wrong. If you have a bucket of red balls and blue balls and you draw 10 million balls, and you find that you drew 6 million red balls and 4 million blue balls, what does that tell you? If you're sampling randomly it tells you that there are more red balls than blue balls. If you're subconsciously preferring to pick up red balls then it tells you nothing. So we need to avoid subconsciously picking red balls, which means we need to pick users randomly which is something we can't do with a voluntary survey. Cochran's formulas don't apply here because you're not picking your sample set at random. -- Matthew Garrett | mj...@srcf.ucam.org ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, 19 Aug 2011 21:05:26 +0100 Matthew Garrett mj...@srcf.ucam.org wrote: On Fri, Aug 19, 2011 at 10:53:46PM +0300, Felipe Contreras wrote: If you went back to 1991 and wanted a production-quality kernel within a year, Linux probably wouldn't be your starting point. There'd be a learning process involved with setting up a professional-quality survey team, and the first few attempts would be pretty buggy. We'd get there in time, but until then... Until then it's better to have nothing? It's better to have no data than to have misleading data. It's better to have no desktop than one that might not be production quality ? Same argument, same problem. PS data is never misleading. It's presentation maybe misleading but the data is just bits. I do think the comments on more open and why fill in the box type questions are on the button for the reasons expressed about sample size, randomness and what it would be useful to learn. Or perhaps rerun Federico's survey ? Alan ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 11:14 PM, Matthew Garrett mj...@srcf.ucam.org wrote: On Fri, Aug 19, 2011 at 10:26:08PM +0300, Felipe Contreras wrote: Yes, because you have no idea how big the population is. Maybe 10 million is the total population and it's representative. Maybe it's 50% of the population, disproportionately biased towards those of a given prior opinion. You can't know. Do you have any idea what is the likelihood of that happening? Try throwing a dice 10 times and always getting 1-3. Even if the dice is rigged, it's very unlikely. It gets exponentially less likely 1 million times. That's clearly wrong. If you have a bucket of red balls and blue balls and you draw 10 million balls, and you find that you drew 6 million red balls and 4 million blue balls, what does that tell you? If you're sampling randomly it tells you that there are more red balls than blue balls. If you're subconsciously preferring to pick up red balls then it tells you nothing. So we need to avoid subconsciously picking red balls, which means we need to pick users randomly which is something we can't do with a voluntary survey. Cochran's formulas don't apply here because you're not picking your sample set at random. That's a very bad example. An example closer to reality would be that color is indeed the bias, but we are not interested in the color, but the size of the balls. After the survey, we find out that overall, red balls are bigger than blue balls. Fortunately we don't care about the proportion of blue vs red balls in the total population, we only care about blue balls, so, we only consider the size of those. In the GNOME case, the color of the balls corresponds to the bias we want to identify; like geekness, and the size is the actual thing we are interested on, which is their happiness. We only care about non geeks (blue balls), as many GNOME people have stated, the real target users are the ones that don't even know what is GNOME. Now, if what you are worried about is the self-selection bias, we can add a new question Why are you taking this survey? with the option Somebody is pushing me, and encourage people to push their relatives/colleagues/friends to fill the survey (just like a professional firm would, except crowd-sourced). Then, for external validity, you only consider the results of the people that answered Somebody is pushing me (they don't have self-selection bias). -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Felipe Contreras felipe.contreras at gmail.com writes: That doesn't change the fact that everyone understands the word happy. http://www.ted.com/talks/daniel_kahneman_the_riddle_of_experience_vs_memory.html ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Wed, Aug 17, 2011 at 11:12 PM, Jason D. Clinton m...@jasonclinton.com wrote: On Wed, Aug 17, 2011 at 10:23, Felipe Contreras felipe.contre...@gmail.com wrote: Here's the fourth version of the survey, only tiny minor changes, it seems it's stabilized as there isn't many more comments. Shall we start planning the deployment? Who can get it into the site? Can I have access? How about an application that pops notifications similar to this one? Would such a thing be accepted? You haven't yet addressed the problems that I pointed out. You haven't yet provided any suggestion. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Thu, Aug 18, 2011 at 11:57 AM, Felipe Contreras felipe.contre...@gmail.com wrote: On Wed, Aug 17, 2011 at 11:12 PM, Jason D. Clinton m...@jasonclinton.com wrote: How about an application that pops notifications similar to this one? Would such a thing be accepted? You haven't yet addressed the problems that I pointed out. You haven't yet provided any suggestion. You are the one who proposed the idea of this survey and (more importantly) insist that a useful survey is possible for GNOME so *you* must address all criticism/concerns if you want us to take your proposal seriously. Ignoring input from others just because they don't have a solution for the problem(s) they point out isn't going to lead anywhere. -- Regards, Zeeshan Ali (Khattak) FSF member#5124 ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Thu, Aug 18, 2011 at 7:28 PM, Zeeshan Ali (Khattak) zeesha...@gnome.org wrote: On Thu, Aug 18, 2011 at 11:57 AM, Felipe Contreras felipe.contre...@gmail.com wrote: On Wed, Aug 17, 2011 at 11:12 PM, Jason D. Clinton m...@jasonclinton.com wrote: How about an application that pops notifications similar to this one? Would such a thing be accepted? You haven't yet addressed the problems that I pointed out. You haven't yet provided any suggestion. You are the one who proposed the idea of this survey and (more importantly) insist that a useful survey is possible for GNOME so *you* must address all criticism/concerns if you want us to take your proposal seriously. Ignoring input from others just because they don't have a solution for the problem(s) they point out isn't going to lead anywhere. So, your idea of input is this can be improved. What kind of input is that? If you have a concrete suggestions for improvement, I'm all ears. Or do you want me to make random modifications until Jason likes one of them? So far, *everybody* that has raised concerns has at least tried to provide some suggestions, except Jason, who apparently for some reason wants me to do his thinking for himself. I would gladly try that, if that is going to move this thing forward, but I think it would be much more productive if he did that, if at least because it would shorten the cycle between: how about this? nope, this? nah... I can only think of one reason why somebody would provide criticism without suggestions for improvement... Also, I thought this was *GNOME* user survey, not mine. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Hi Felipe, all. While I've been watching this all go by, I've not jumped in largely due to being crazy-busy. Sorry about that! While the crazy-busy situation has not yet changed, I couldn't help but notice this: === 01. Which of the following images best resemble your desktop? === (image selection) - GNOME 2 - GNOME 3 - Unity - KDE I assume these images will have sufficient accessible descriptions associated with them for users who are blind. In that same spirit: Shall we start planning the deployment? I have no opinion on this other than to say should deployment planning indeed begin, would you mind pinging the Accessibility team along the way? I'm sure it will all be fine and accessible, but I would hate to make that assumption only for us to find out, upon deployment, that something was missed regarding the survey instrument, the notifications, etc. Thanks! Take care. --joanie ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Hi Felipe, Thanks for trying to get feedback from users. This is something that is really hard to do. It might also be worth contacting a company that does this professionally to see if they can help us. I do not think you will be able to do very much with the answers to the questions you ask below. It's going to be a lot of work for data that is not useful. Let me try to explain. On Wed, Aug 17, 2011 at 9:23 AM, Felipe Contreras felipe.contre...@gmail.com wrote: Hi, Here's the fourth version of the survey, only tiny minor changes, it seems it's stabilized as there isn't many more comments. Shall we start planning the deployment? Who can get it into the site? Can I have access? Where are we deploying it? How are we going to get people to take it? How about an application that pops notifications similar to this one? Would such a thing be accepted? Within GNOME? Would the distros agree to ship it? === 01. Which of the following images best resemble your desktop? === (image selection) - GNOME 2 - GNOME 3 - Unity - KDE I think if we want to get average users, most of them are not going to know what GNOME is. I love GNOME and I've been using GNOME for years and working with GNOME, and I still don't really know what all is GNOME on my desktop. I think the questions will have to be much more specific. === 02. Overall, how happy are you with GNOME? === (single choice) * unhappy * not so happy * happy * very happy * completely ecstatic If people tell you they are happy or unhappy with GNOME, what are you going to do with that? If they are unhappy what are you going to fix? If they are happy, what did they like? The color, the menus, the windows, the apps, ...??? === 03. Where do you run GNOME? === (multiple choice, with other) === 04. Which GNOME version(s) are you using? === (multiple choice, with other) === 05. How long have you been using GNOME? (years) === (numeric) The previous 3 questions are only useful if it will somehow help you understand the other answers better. === 06. How do you compare your current GNOME version with the version from one year ago? === (single choice) * better * no changes * worse * cannot say They aren't going to know what's GNOME versus what's the distro. And maybe they like the help better and the menus less. Or maybe it's missing their favorite feature. This is way too vague ... === 07. Does GNOME do what you want? === (single choice) * Everything * Mostly * Somewhat * Barely * Not at all If they say not at all, what are we missing? === 08. How happy are you with GNOME in regards to == (matrix) Columns: unhappy / not so happy / happy / very happy / completely ecstatic + ease of use + documentation + language availability + accessibility + community This gets closer but it still way too high level. === 09. Which other desktop environments have you used in recent years? == (multiple choice, with other) + KDE + Unity + XFCE + LXDE + Enlightenment + other (please specify) Are you planning on polling just the open source community? Nobody else is going to have any idea what these are. They would be more likely to know Windows and Mac. But I'm not sure that this question is useful for us to act on. === 10. How many years of experience do you have using computers? === (numeric) Does it matter? === 11. How often do you use terminal/console? == (single choice) * What is that? * When I have no other option * I can't live without them * Is there anything else? All the questions after this assume a knowledge of GNOME and how our community works. That's fine if you are polling developers. If you are polling average users, then I think it's not worth asking. Stormy === 12. Have you contributed to the GNOME project? === (single choice) * Yes * No === 13. Have you contacted the GNOME team? === (single choice) * Yes, successfully * Yes, unsuccessfully * No, I don't know how * No, never had the need === 14. If you could change three things in GNOME, what would they be? === (free form) === 15. Do you have any comments or suggestions for the GNOME team? === (free form) -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Thu, Aug 18, 2011 at 08:03:45PM +0300, Felipe Contreras wrote: I can only think of one reason why somebody would provide criticism without suggestions for improvement... 1. Because they cannot think of a good suggestion. Let's keep this constructive or otherwise don't cc desktop-devel-list. -- Regards, Olav ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
I do not think you will be able to do very much with the answers to the questions you ask below. It's going to be a lot of work for data that is not useful. Let me try to explain. I thhink there is a better way to do this Felipe should do it without the Gnome oligarchy and then put the findings up on Linux Weekly News. That way we'll learn something, if not everything and it can no longer be stalled forever by bickering. And then well it's up to people if they listen, what they do with the data and how they follow it up. Sure the results will need reviewing with a little car - but thats true of any survey even one you paid through the nose for, in fact often more so because they more you pay the harder some of them will work to make sure you get the answers they think you want to hear 8) Plus the resulting debate may well answer even more questions than the survey ever did... Alan ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Thu, Aug 18, 2011 at 1:20 PM, Alan Cox a...@lxorguk.ukuu.org.uk wrote: And then well it's up to people if they listen, what they do with the data and how they follow it up. Sure the results will need reviewing with a little car The answers are so vague that you are not going to be able to follow up on them. So they are unhappy with GNOME. Then what? Plus the resulting debate may well answer even more questions than the survey ever did... If you care about debating and learning from our core group of dedicated supporters, yes. If you care about average users, well, I doubt you'll learn much from them this way. The questions and answers are just too vague and those people are likely not reading LWN so they won't be able to follow up with us that way. Of course, maybe I'm wrong. Perhaps the average user of Linux/GNOME does know what GNOME is, knows how to contact the GNOME team and can tell you what version of GNOME they are using. And if they do, what is the survey going to tell you? That they do or don't like GNOME? And how long they have been using GNOME? What are we going to do with that information? Before any survey, you should know how you are going to use the information so that you can be sure to ask the right questions. Stormy ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Thu, Aug 18, 2011 at 8:33 PM, Joanmarie Diggs joan...@gnome.org wrote: While I've been watching this all go by, I've not jumped in largely due to being crazy-busy. Sorry about that! While the crazy-busy situation has not yet changed, I couldn't help but notice this: No worries, that's why the survey is not scheduled any time soon. === 01. Which of the following images best resemble your desktop? === (image selection) - GNOME 2 - GNOME 3 - Unity - KDE I assume these images will have sufficient accessible descriptions associated with them for users who are blind. Well, I don't know such descriptions would be. We would need somebody that has actually used GNOME (and KDE) through this interface, and even then I'm not sure it's possible. In that same spirit: Shall we start planning the deployment? I have no opinion on this other than to say should deployment planning indeed begin, would you mind pinging the Accessibility team along the way? I'm sure it will all be fine and accessible, but I would hate to make that assumption only for us to find out, upon deployment, that something was missed regarding the survey instrument, the notifications, etc. How to do that? I assume cross-posting to multiple lists is discouraged. But yes, it would be great to get their feedback. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Hi Felipe. On Thu, 2011-08-18 at 23:50 +0300, Felipe Contreras wrote: On Thu, Aug 18, 2011 at 8:33 PM, Joanmarie Diggs joan...@gnome.org wrote: [...] === 01. Which of the following images best resemble your desktop? === (image selection) - GNOME 2 - GNOME 3 - Unity - KDE I assume these images will have sufficient accessible descriptions associated with them for users who are blind. Well, I don't know such descriptions would be. We would need somebody that has actually used GNOME (and KDE) through this interface, and even then I'm not sure it's possible. FWIW, I suspect most users who are blind and would be responding to such a survey are very much aware of exactly what desktop environment they are using and could identify it by name even without the benefit of a multiple-choice question. So the alternative text I'd probably use is something like: - GNOME 2 - GNOME 3 - Unity - KDE Having said that, it seems that the stage you're in right now is still one of figuring out exact language for all the questions, how best to get meaningful data, and how to go about deploying it. Thus what is the real point I'd like you to take away now about this issue is simply: If you present an image, you'll need an alternative, text-based description to make that image accessible for users who are blind. Please don't forget the description. :-) When you reach the point where you need the exact language for such descriptions, please come hang out with the accessibility team and we'll help you sort it out. In that same spirit: Shall we start planning the deployment? I have no opinion on this other than to say should deployment planning indeed begin, would you mind pinging the Accessibility team along the way? I'm sure it will all be fine and accessible, but I would hate to make that assumption only for us to find out, upon deployment, that something was missed regarding the survey instrument, the notifications, etc. How to do that? I assume cross-posting to multiple lists is discouraged. But yes, it would be great to get their feedback. Well, on behalf of the Accessibility team, I invite you to join us in #a11y on IRC any time you have questions. If you prefer a mailing list, the gnome-accessibility-list reaches just about all of the team members. Thanks again! Take care. --joanie ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Hi, On Thu, Aug 18, 2011 at 8:53 PM, Stormy Peters sto...@gnome.org wrote: Thanks for trying to get feedback from users. This is something that is really hard to do. It might also be worth contacting a company that does this professionally to see if they can help us. I do not think you will be able to do very much with the answers to the questions you ask below. It's going to be a lot of work for data that is not useful. Let me try to explain. On Wed, Aug 17, 2011 at 9:23 AM, Felipe Contreras felipe.contre...@gmail.com wrote: Here's the fourth version of the survey, only tiny minor changes, it seems it's stabilized as there isn't many more comments. Shall we start planning the deployment? Who can get it into the site? Can I have access? Where are we deploying it? How are we going to get people to take it? Have you read the original thread? http://thread.gmane.org/gmane.comp.gnome.desktop/45432/focus=45456 I originally suggested questionpro.com, as it was the best option I could find. Later on Frederic Muller suggested to use limesurvey which supposedly it's installed in GNOME servers. As for getting people to take it, I was suggesting the usual info distribution channels; blogs, planets, online magazines, twitter, Google+, facebook, etc. In addition to that I suggested a new software component that would read information from the Internet, and pop up a notification to the user, the user would be able to disable these notifications easily, of course. Perhaps for GNOME 3.2. But I didn't hear a lot of encouragement for that idea. How about an application that pops notifications similar to this one? Would such a thing be accepted? Within GNOME? Would the distros agree to ship it? === 01. Which of the following images best resemble your desktop? === (image selection) - GNOME 2 - GNOME 3 - Unity - KDE I think if we want to get average users, most of them are not going to know what GNOME is. That is the purpose of this question, they don't have to know, they just select the image that resembles what they are using. I love GNOME and I've been using GNOME for years and working with GNOME, and I still don't really know what all is GNOME on my desktop. I think the questions will have to be much more specific. How do you make this question more specific? They are images, you just have to select one. === 02. Overall, how happy are you with GNOME? === (single choice) * unhappy * not so happy * happy * very happy * completely ecstatic If people tell you they are happy or unhappy with GNOME, what are you going to do with that? If they are unhappy what are you going to fix? If they are happy, what did they like? The color, the menus, the windows, the apps, ...??? Baby steps, first, let's get the results. If they are happy, great, not much to do, if they are unhappy, well, then some further actions after this survey might be needed. Having said that, we might find some clues in the rest of the survey, as this question is somewhat split into multiple groups later on, and in the worst case scenario, there's the free-form comments. === 03. Where do you run GNOME? === (multiple choice, with other) === 04. Which GNOME version(s) are you using? === (multiple choice, with other) === 05. How long have you been using GNOME? (years) === (numeric) The previous 3 questions are only useful if it will somehow help you understand the other answers better. Also the previous one. Maybe people don't like GNOME 2 that much, but they do like GNOME 3... === 06. How do you compare your current GNOME version with the version from one year ago? === (single choice) * better * no changes * worse * cannot say They aren't going to know what's GNOME versus what's the distro. And maybe they like the help better and the menus less. Or maybe it's missing their favorite feature. This is way too vague ... The purpose is to get some sense of progress. Say, the respondents using GNOME 2 are answering worse a lot, it might be worth investigating what might have been those changes. How would you make this question less vague? === 07. Does GNOME do what you want? === (single choice) * Everything * Mostly * Somewhat * Barely * Not at all If they say not at all, what are we missing? Again, then you might want to take further actions beyond this survey. Most likely though, you would be able to find some correlations between this question to other ones. Maybe it's only people with a lot of experience with computers that would answer in such way. We would know only after getting the results. === 08. How happy are you with GNOME in regards to == (matrix) Columns: unhappy / not so happy / happy / very happy / completely ecstatic + ease of use + documentation + language availability + accessibility + community This gets closer but it still way too high level. At least is something. If you have a better idea, why not share it?
Re: GNOME user survey 2011 (v4)
The answers are so vague that you are not going to be able to follow up on them. So they are unhappy with GNOME. Then what? Then at the very least you've got some picture of what is going on and you can try and trigger discussion about why people are unhappy (or indeed happy). Of course, maybe I'm wrong. Perhaps the average user of Linux/GNOME does know what GNOME is, knows how to contact the GNOME team and can tell you what version of GNOME they are using. And if they do, what is the survey There are going to be a large number of users whose viewpoint is essentially don't care, how you measure them is hard in pretty much any situation. A truely random sample of Gnome users will be hard to get by any approach. going to tell you? That they do or don't like GNOME? And how long they have been using GNOME? What are we going to do with that information? Use it to work out what questions it would be interesting to ask next year ? Look at what shows up in terms of additional comments. Look at the discussion around it, drop in the odd 'Why ?' question of your own. Use it to kickstart a secondary debate on the gnome site. (And btw while they won't read LWN people will link to it and discuss it in other places too) There is a second thing here too IMHO. The questions that could be asked and fixing them are currently buried in the debate. I can't see how progress will be made on picking questions usefully until someone moves from trying to achieve consensus to picking what they think is best based upon the resposes and just doing it regardless of whether each question is considered wrong by 5% of the people in the debate. Before any survey, you should know how you are going to use the information so that you can be sure to ask the right questions. So I could equally have said Why release Gnome 3.0, we know it isn't perfect and there are wrong things. Releasing it was better than stasis, it provided a learning experience that will make 3.2 much better I am sure. Doing nothing achieves nothing, doing something achieves learning. You may well not learn what you intended but you will learn something including quite possibly how to do future surveys better. I'm not saying its necessarily a great approach but it's vastly superior to people sitting around picking holes in the idea until it never happens. Right now this seems to be in blocking mode, and blocking a volunteer off to do stuff and see what happens be it code or otherwise is usually the wrong thing to do. Sure -t here is a good case for not describing it in any way that suggests its GNOME foundation endorsed or driven. Gnome grew from a comically clueless 0.1 tarball, surveys can do likewise. Alan ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Thu, Aug 18, 2011 at 11:21 PM, Stormy Peters sto...@gnome.org wrote: On Thu, Aug 18, 2011 at 1:20 PM, Alan Cox a...@lxorguk.ukuu.org.uk wrote: And then well it's up to people if they listen, what they do with the data and how they follow it up. Sure the results will need reviewing with a little car The answers are so vague that you are not going to be able to follow up on them. So they are unhappy with GNOME. Then what? This is a simple survey, not some kind of magical questionnaire. Just having the information that users are unhappy is valuable already. Plus, there's the free-form suggestion text. Plus the resulting debate may well answer even more questions than the survey ever did... If you care about debating and learning from our core group of dedicated supporters, yes. If you care about average users, well, I doubt you'll learn much from them this way. The questions and answers are just too vague and those people are likely not reading LWN so they won't be able to follow up with us that way. Well, we don't know what the average GNOME user looks like, do we? What we do know is that is the target user, and we have tried to identify them with the question in the survey. Supposing we get a few thousands of respondents, and only 10% qualify as normal users, even then, if you filter those answers, that should give you statistically significant information about the whole target user-base. Oh, and maybe a lot of people don't read LWN, but they don't have too, just like anything viral, the link to the survey would spread, and eventually at least few normal users are bound to have a geek friend that would show them the link. Of course, maybe I'm wrong. Perhaps the average user of Linux/GNOME does know what GNOME is, knows how to contact the GNOME team and can tell you what version of GNOME they are using. And if they do, what is the survey going to tell you? That they do or don't like GNOME? And how long they have been using GNOME? What are we going to do with that information? Suppose you are right, and we do get that useless (not in my books) information, what is the damage? Suppose however that we do find something useful there. I think it's totally worth trying. Before any survey, you should know how you are going to use the information so that you can be sure to ask the right questions. Not necessarily. Again, asking a useless question doesn't hurt anybody. Of course, if you have better questions, those should be prioritized over the ones that have less chance of being fruitful. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 12:37 AM, Alan Cox a...@lxorguk.ukuu.org.uk wrote: Use it to work out what questions it would be interesting to ask next year ? Look at what shows up in terms of additional comments. Look at the discussion around it, drop in the odd 'Why ?' question of your own. Use it to kickstart a secondary debate on the gnome site. Exactly. I forgot this crucial point. This has been the case in the Git survey; after years of doing it, we have found that some questions were missing (by looking at the comments box), and that some questions were not really providing much. Nothing is ever perfect, but having at least some results is better than nothing. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras felipe.contre...@gmail.com wrote: Nothing is ever perfect, but having at least some results is better than nothing. Since you have repeated this assertion a few times, I must ask: What if the results are all wrong and we don't have any way of knowing that? Would those results still be better than nothing in your opinion? -- Regards, Zeeshan Ali (Khattak) FSF member#5124 ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, 19 Aug 2011 01:20:53 +0300 Zeeshan Ali (Khattak) zeesha...@gnome.org wrote: On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras felipe.contre...@gmail.com wrote: Nothing is ever perfect, but having at least some results is better than nothing. Since you have repeated this assertion a few times, I must ask: What if the results are all wrong and we don't have any way of knowing that? Would those results still be better than nothing in your opinion? In that hypothetical case possibly not. But that isn't really likely to be the case even with a bad survey, especially if you start looking at how people used the open comments and asking why questions, or looking at the debate it triggers. You can learn things even by asserting a position and seeing the responses you get. Alan ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 1:20 AM, Zeeshan Ali (Khattak) zeesha...@gnome.org wrote: On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras felipe.contre...@gmail.com wrote: Nothing is ever perfect, but having at least some results is better than nothing. Since you have repeated this assertion a few times, I must ask: What if the results are all wrong and we don't have any way of knowing that? Would those results still be better than nothing in your opinion? What do you mean by all wrong? Let's assume that the results show that 1000 people are not happy with GNOME. How can that be wrong? 1000 people responded that, the results were not somehow altered, or boycotted, the results are the results, and that's that. The *conclusions* based on the analysis of the results might be wrong, but that wouldn't be a problem of the survey, and if you are so afraid of that, you can ignore the results of the survey completely. I for one think the survey already has enough mechanisms to determine biases, and therefore come up with conclusions with a reasonable degree of certainty. But I wonder, can you come up with some example of bad results to the answers proposed here, and why exactly we wouldn't know they are bad? -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 2:09 AM, Felipe Contreras felipe.contre...@gmail.com wrote: On Fri, Aug 19, 2011 at 1:20 AM, Zeeshan Ali (Khattak) zeesha...@gnome.org wrote: On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras felipe.contre...@gmail.com wrote: Nothing is ever perfect, but having at least some results is better than nothing. Since you have repeated this assertion a few times, I must ask: What if the results are all wrong and we don't have any way of knowing that? Would those results still be better than nothing in your opinion? What do you mean by all wrong? Let's assume that the results show that 1000 people are not happy with GNOME. How can that be wrong? Maybe they all lied? Maybe people who are satisfied do not want to or have time to take part in surveys and you only get people who are not happy into the survey? In which case, the results may show results that are not correct. i-e a significantly large number of participant say that they are very unhappy with GNOME but what if that number is nothing compared to the number of people who are very much satisfied with GNOME? I didn't say this so far because it might sound like I am trying to make a joke but since you still insist on your assertions about the survey, I feel I must say this: How do you know people in general like to participate in surveys? It is my observation that most people do not like to do that, unless they have something to complain about. Now this observation of mine could very well be wrong but how do we know that? Do we do a survey to find out if people like to participate in surveys? -- Regards, Zeeshan Ali (Khattak) FSF member#5124 ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Wed, Aug 17, 2011 at 5:23 PM, Felipe Contreras felipe.contre...@gmail.com wrote: Hi, Here's the fourth version of the survey, only tiny minor changes, it seems it's stabilized as there isn't many more comments. Shall we start planning the deployment? Who can get it into the site? Can I have access? How about an application that pops notifications similar to this one? Would such a thing be accepted? Cheers. === 01. Which of the following images best resemble your desktop? === (image selection) - GNOME 2 - GNOME 3 - Unity - KDE === 02. Overall, how happy are you with GNOME? === (single choice) As Bastien pointed out already, in question#1 you are reaching out to users who use GNOME without knowing so but in the rest of the questions, you are not. I think this survey isn't going to be useful to us if you don't include such users in it as GNOME 3 is very much targeted for this type of users. Before you ask, no I don't know how you can do that. -- Regards, Zeeshan Ali (Khattak) FSF member#5124 ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Wed, Aug 17, 2011 at 6:42 PM, Zeeshan Ali (Khattak) zeesha...@gnome.org wrote: On Wed, Aug 17, 2011 at 5:23 PM, Felipe Contreras felipe.contre...@gmail.com wrote: Here's the fourth version of the survey, only tiny minor changes, it seems it's stabilized as there isn't many more comments. Shall we start planning the deployment? Who can get it into the site? Can I have access? How about an application that pops notifications similar to this one? Would such a thing be accepted? Cheers. === 01. Which of the following images best resemble your desktop? === (image selection) - GNOME 2 - GNOME 3 - Unity - KDE === 02. Overall, how happy are you with GNOME? === (single choice) As Bastien pointed out already, in question#1 you are reaching out to users who use GNOME without knowing so but in the rest of the questions, you are not. I think this survey isn't going to be useful to us if you don't include such users in it as GNOME 3 is very much targeted for this type of users. Before you ask, no I don't know how you can do that. Huh? What makes you think so? Let's suppose the user has absolutely no idea what is GNOME, so (s)he is not clicking on a GNOME user survey link, but gets a notification from the desktop (which I'm not sure is going to happen), or something like that. The title of the page will still be GNOME desktop user survey, so what is the user going to thing? Hmm, looks like I'm using this GNOME thing. First question, click on the image that best resembles the desktop, the image says GNOME 3, well, certainly looks like I'm using that GNOME thing. There's a tendency here. Even ignoring all that, the second question Overall, how happy are you with GNOME? is still targeted at these people, even if they don't know what GNOME is, they know the survey is about the desktop they are using, what else would it be about? The fist question makes sure they have the right thing in mind. Sure, some of the rest of the questions are not applicable, but that doesn't mean these kind of users are not targeted, at least they will be able to answer 02, 07, 10, and probably 03, and that should certainly be at least better than nothing. Maybe those question can in the beginning, and perhaps even split into two parts. There's really not much that can be done in order to target these users better. Cheers. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Wed, Aug 17, 2011 at 10:23, Felipe Contreras felipe.contre...@gmail.com wrote: Here's the fourth version of the survey, only tiny minor changes, it seems it's stabilized as there isn't many more comments. Shall we start planning the deployment? Who can get it into the site? Can I have access? How about an application that pops notifications similar to this one? Would such a thing be accepted? You haven't yet addressed the problems that I pointed out. ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
