Re: Where is the data?
2011/8/20 Olav Vitters :
> On Sat, Aug 20, 2011 at 06:46:15PM +0200, Giovanni Campagna wrote:
>> Still, if bug #123456 is more voted than bug #654321, it may be worth
>> dedicating some more design hours to the former than to the latter,
>> investigating the reasons, providing evidence, doing tests, etc.
>> Then, maybe the bug will still be closed NOTABUG/WONTFIX, but it will be
>> given a response proportional to the number of people reporting it.
>
> Because voting just gets a small number of people involved, results in
> distractions ("why isn't this fixed yet; has XXX votes")
Just ignore them if you don't care about them.
> , only technical
> users get involved and that it just is not seen by any significant
> amount of users I will not enable voting. It does more harm than good.
Whoa! Wait a second. How do you jump from "only technical users get
involved" to "it does more harm than good".
Technical users are still users, right? If you get 0 votes, that at
least gives you something, specially compared to another bug report
that has 1000 votes. At the end of the day you might decide to go for
the one that has 0 votes, because it's easier, but at least you know,
there's that bug report over there that more than a couple people
*definitely* care about.
> This is also on bugzilla.mozilla.org. Most voted bug at one point had
> 700 or so votes. That is about 1000 times off from what I find
> noteworthy.
Again, you can ignore the votes if you want, or you can assign your
own scale, but the point is, that a vote with 700 or so votes is
certainly more likely to be important that one with 0 votes.
Personally, I have had great success with bug reporting voting on my
projects, and seems to match pretty much the feedback I receive
through other methods.
But I'm not going to try to convince you, I'm sure there's no
objective measure that would change your mind.
--
Felipe Contreras
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Where is the data?
On Sat, Aug 20, 2011 at 8:03 PM, Shaun McCance wrote: > On Sat, 2011-08-20 at 18:09 +0300, Felipe Contreras wrote: >> On Sat, Aug 20, 2011 at 6:02 PM, Shaun McCance wrote: >> > On Sat, 2011-08-20 at 14:43 +0200, Tomasz Torcz wrote: >> >> I think his objections were justified. There is really no raw data >> >> at those URLs. >> > >> > Except Allan never claimed he was providing raw data. In fact, >> > he explicitly said that he does not do write-ups of user tests. >> >> So what's the point of replying to a mail asking specifically for data? > > When somebody asks me for something I don't have, I usually > respond telling them so, and explaining why I don't have it. > I think it's rude to ignore people. Well, that's true, but he claimed there was "lots of data". I think that was an exaggeration. >> > I also do user tests when working on the help. I also don't do >> > write-ups. I fix problems or I pass information on to those who >> > are in a position to fix the problems. >> > >> > Just because I don't publish reports doesn't mean I don't do >> > user tests. And the constant assertions that nobody is looking >> > at feedback are getting a bit insulting. >> >> User tests, like surveys, are not perfect and can be both misleading, >> and not significant enough. > > You're right, of course. All methods have flaws. But user testing, > at least, gives results that are personal and actionable. When I > get results like "all 5 users were uncertain where to click when > instructed to click the 'user menu'", I know what to do. I have > no idea what to do with "63% of respondents report they are less > happy than they were a year ago". Yes, that kind of user testing gives you clear short-term actions, but it doesn't tell you if people happy with the whole system overall. It doesn't tell you if you are missing something essential, nor does it tell you when you are bleeding user-base. But you missed my point, my point is that they can be improved through *collaboration*, you publicize them, and people make suggestion to improve them. >> If those tests are to be taken seriously, they should be published so >> that they can be scrutinized, otherwise they are not evidence of >> anything, not to the rest of the world. > > I agree there are problems with transparency. A lot of things > get done on IRC, because high-bandwidth communication is great > for rapid development. I've been a strong supporter of public > logs for IRC. I think we should discuss ways to better record > what we do and the decisions we make. Indeed. > But, I don't want to be in a situation where we have to wait > for committees to scrutinize data and approve proposals before > we can make changes. That sounds like an awful project to write > software for. I don't think anybody is proposing that. All people are looking for is some public record. Just by doing that you might realize that perhaps you didn't have such strong reasons to switch to something as much as you were thinking, after all. Also, people could tell you; you are misinterpreting the results _there_. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Where is the data?
On Sat, 2011-08-20 at 18:42 +0200, Giovanni Campagna wrote: > Il giorno sab, 20/08/2011 alle 11.02 -0400, Shaun McCance ha scritto: > > I also do user tests when working on the help. I also don't do > > write-ups. I fix problems or I pass information on to those who > > are in a position to fix the problems. > > Uhm... Until today, I saw nothing concerning results of user testing, > including in the parts of code written by me, and in the bugs assigned > to me. Which either means we're doing too few tests, or they're > communicated badly to the larger community. Or both, probably. Both. I don't think the notion that we could use more tests is controversial, nor the notion that we could communicate better. But the implication throughout the way too many emails in the last week is that nobody is doing anything. That's just plain false, and I'm sure you can understand how that would upset those of us who do user testing. > In either case, we may need to stop and consider how to improve this > situation. > For example, at the desktop summit, the first keynote, about large > companies and open sources, emphasized the great amount of user testing > continuously undertaken. This could be employed by the project and > reused to evaluate design and implementation, but pnly if those results > are made available outside the companies. No doubt. In the past, we've benefited greatly from results of user tests done by companies like Sun and Novell. A constructive conversation about how we can better partner with companies to get more user testing would be, I think, welcomed by the community. > > Just because I don't publish reports doesn't mean I don't do > > user tests. And the constant assertions that nobody is looking > > at feedback are getting a bit insulting. > > Well, if you don't publish anything, other people cannot know your doing > them, and someone, like me, will wrongly end up thinking that you're > doing nothing. > I'm sorry if this is annoying you, but you cannot expect people from > outside, that don't know you, to trust your work, without seeing the end > results of it. I can't speak for anybody else you've worked with, but I've never shared results with you because I've never asked you to change anything. If somebody asks you to change something in software that you maintain, you absolutely have the right to demand they justify the change. Maintainers have the final authority in GNOME. They always have. I make decisions every single day about the direction of the software I maintain, and about the structure, contents, and tone of our help and documentation. People usually trust me, because I've been at this for a long time now. If I had to make a formal justification for all those decisions, I don't think I'd get anything done at all. I know you're just trying to make great free software like the rest of us, and that you don't want to stifle the development. We could use more transparency, but not more red tape. But right now, I think people have serious mailing list fatigue. We might get more done if we circulate proposals off-list. -- Shaun ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Where is the data?
On Sat, Aug 20, 2011 at 06:46:15PM +0200, Giovanni Campagna wrote:
> Still, if bug #123456 is more voted than bug #654321, it may be worth
> dedicating some more design hours to the former than to the latter,
> investigating the reasons, providing evidence, doing tests, etc.
> Then, maybe the bug will still be closed NOTABUG/WONTFIX, but it will be
> given a response proportional to the number of people reporting it.
Because voting just gets a small number of people involved, results in
distractions ("why isn't this fixed yet; has XXX votes"), only technical
users get involved and that it just is not seen by any significant
amount of users I will not enable voting. It does more harm than good.
This is also on bugzilla.mozilla.org. Most voted bug at one point had
700 or so votes. That is about 1000 times off from what I find
noteworthy.
--
Regards,
Olav
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Where is the data?
On Sat, 2011-08-20 at 18:09 +0300, Felipe Contreras wrote: > On Sat, Aug 20, 2011 at 6:02 PM, Shaun McCance wrote: > > On Sat, 2011-08-20 at 14:43 +0200, Tomasz Torcz wrote: > >> I think his objections were justified. There is really no raw data > >> at those URLs. > > > > Except Allan never claimed he was providing raw data. In fact, > > he explicitly said that he does not do write-ups of user tests. > > So what's the point of replying to a mail asking specifically for data? When somebody asks me for something I don't have, I usually respond telling them so, and explaining why I don't have it. I think it's rude to ignore people. > > I also do user tests when working on the help. I also don't do > > write-ups. I fix problems or I pass information on to those who > > are in a position to fix the problems. > > > > Just because I don't publish reports doesn't mean I don't do > > user tests. And the constant assertions that nobody is looking > > at feedback are getting a bit insulting. > > User tests, like surveys, are not perfect and can be both misleading, > and not significant enough. You're right, of course. All methods have flaws. But user testing, at least, gives results that are personal and actionable. When I get results like "all 5 users were uncertain where to click when instructed to click the 'user menu'", I know what to do. I have no idea what to do with "63% of respondents report they are less happy than they were a year ago". > If those tests are to be taken seriously, they should be published so > that they can be scrutinized, otherwise they are not evidence of > anything, not to the rest of the world. I agree there are problems with transparency. A lot of things get done on IRC, because high-bandwidth communication is great for rapid development. I've been a strong supporter of public logs for IRC. I think we should discuss ways to better record what we do and the decisions we make. But, I don't want to be in a situation where we have to wait for committees to scrutinize data and approve proposals before we can make changes. That sounds like an awful project to write software for. -- Shaun ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Where is the data?
Il giorno sab, 20/08/2011 alle 16.50 +0100, Allan Day ha scritto: > [...] > > >> > Where we developers can find hard facts proving the NOTABUG and the > >> > WONTFIX we mark in the most questioned and hot issues? > >> > >> You can always mark a bug with the ui-review keyword if you want a > >> second opinion. > > > > You misunderstood me. I didn't say that I don't know when to close, I > > said I don't know how to explain to the users why I'm closing. You need > > facts to prove your points, or people won't understand and refuse to > > agree. > > Oh right, sorry. > > There is evidence available to back up the current design but it's not > all super hard and scientific. If you want a better justification for > certain decisions, just ask the designers to add to the documentation > on the wiki. Maybe we'll be able to do more user testing one day too. Well, that's sounds good to me. I'll remember it next time someone complains. (Which in turn means that I'll ask for evidence myself, if I find something I don't agree with) > >> > I'm not a designer, so I may not understand all the papers you provide > >> > in your support, and I may not understand what are the rules and laws of > >> > Human Computer Interaction, as you call it. But I understand numbers, > >> > and would be convinced by seeing that 66% percent of people find this > >> > method of working more productive, or 3 out 5 tested users where able to > >> > discover the functionality without guidance, or all 8 people interviewed > >> > did not use the feature just removed. > >> ... > >> > >> More user testing would be a good thing, and that might provide some > >> of the numbers that you crave. In the mean time, we're not operating > >> in the dark however: we can tell a lot from a combination of the UX > >> literature, dog fooding, feed back from users and comparison with what > >> other OSs/DEs/whatever are doing. It's not numerical data but it is > >> data all the same. > > > > Usual example: the shutdown button. There is no UX literature proving > > that "suspend is the right way to shutdown a system". Other systems > > (Windows, Mac OS, KDE) are keeping power off as the primary method. Feed > > back from users is far from enthusiastic. So... is there anything that > > proves your points? > > The best examples of this kind of behaviour are mobile phones and tablets, > imo. GNOME is a desktop (as in "good old PC") environment, not a mobile one. But that's a specific issue, and this is not the right place to discuss it. > > As an example, it is said that the user wants to focus one specific task > > at time, and thus the taskbar was removed and all task switching moved > > to the overview. I can concede that the premise is correct, but it is > > hardly self evident that the overview helps with this, given that a task > > often involves more than one window and more than one workspace (which > > forces the user to alt-tab to avoid the "overview distraction"). > > I *really* don't have the time to properly discuss these issues with > you right now, I'm afraid... :) I think the main thing is that > launching isn't always visible, and that that changes the experience > for the better. (Yay for more baseless assertion! ;) ) Don't worry, I didn't want to discuss it here. (If I wanted, I would have opened a thread in gnome-shell-list :D ). It was just to show that sometimes the designer documents and wiki pages are not easily understandable or agreeable with by the common bug reporter. > >> I thought that the responses to the top ideas > >> on Ubuntu brainstorm were a good example of how this can be done, > >> actually [5]. Doing that kind of thing requires time and effort, of > >> course... > > > > I see. Well, we could ask the feedback reporters to do the work. After > > all, it is in their interest to have the developers focused on the > > problems. > > Or we could have voting in bugzilla. I know many people are against to > > this, but it would immediately show the hottest issue, that surely > > require reconsideration by the design team. Plus it would avoid +1 > > comments that spam our mailboxes. > > The main thing would be to visibly demonstrate serious consideration > of the most popular suggestions. There are a few different ways that > those suggestions could be made; it'd be interesting to evaluate the > different possible approaches. Uhm... How technically impractical would it be for GNOME to run a blueprints system similar to Ubuntu's? It was shown to work positively for them. > Hope that helps; sorry if it doesn't. Collaborating, and discussing constructively, always helps. Especially if done on Saturday, so we can keep work days for writing code. Thanks a lot for your answers! Giovanni signature.asc Description: This is a digitally signed message part ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Where is the data?
Il giorno sab, 20/08/2011 alle 15.48 +0100, Rui Tiago Cação Matos ha scritto: > On 20 August 2011 14:18, Giovanni Campagna wrote: > > Or we could have voting in bugzilla. I know many people are against to > > this, but it would immediately show the hottest issue, that surely > > require reconsideration by the design team. > > No. It would only tell you that a bunch of users who don't like a > particular design decision were lured into voting on bugzilla from > some random web forum. In particular, it wouldn't tell you *anything* > about all the users who approve the decision or, most probably, aren't > even aware of it because they don't spend all their time discussing > their computer's UI. Still, if bug #123456 is more voted than bug #654321, it may be worth dedicating some more design hours to the former than to the latter, investigating the reasons, providing evidence, doing tests, etc. Then, maybe the bug will still be closed NOTABUG/WONTFIX, but it will be given a response proportional to the number of people reporting it. Giovanni signature.asc Description: This is a digitally signed message part ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Where is the data?
Il giorno sab, 20/08/2011 alle 11.02 -0400, Shaun McCance ha scritto: > On Sat, 2011-08-20 at 14:43 +0200, Tomasz Torcz wrote: > > On Sat, Aug 20, 2011 at 02:38:55PM +0200, Olav Vitters wrote: > > > On Sat, Aug 20, 2011 at 02:28:54PM +0300, Felipe Contreras wrote: > > > > So, no data there. > > > > > > > There's no data there. > > > > > > > No data there either. > > > > > > > I have read that whole thread; there's no data there either, just > > > > about of explanations that can be reduced to; it's sleek. > > > > > > > No data here either. > > > > > > > Here's some data, but I don't see any connection to GNOME 3 design. > > > > > > > No data here either. > > > > > > This is not kindergarten. I've set your moderation bit. > > > > I think his objections were justified. There is really no raw data > > at those URLs. > > Except Allan never claimed he was providing raw data. In fact, > he explicitly said that he does not do write-ups of user tests. > > I also do user tests when working on the help. I also don't do > write-ups. I fix problems or I pass information on to those who > are in a position to fix the problems. Uhm... Until today, I saw nothing concerning results of user testing, including in the parts of code written by me, and in the bugs assigned to me. Which either means we're doing too few tests, or they're communicated badly to the larger community. Or both, probably. In either case, we may need to stop and consider how to improve this situation. For example, at the desktop summit, the first keynote, about large companies and open sources, emphasized the great amount of user testing continuously undertaken. This could be employed by the project and reused to evaluate design and implementation, but pnly if those results are made available outside the companies. > Just because I don't publish reports doesn't mean I don't do > user tests. And the constant assertions that nobody is looking > at feedback are getting a bit insulting. Well, if you don't publish anything, other people cannot know your doing them, and someone, like me, will wrongly end up thinking that you're doing nothing. I'm sorry if this is annoying you, but you cannot expect people from outside, that don't know you, to trust your work, without seeing the end results of it. Remember, this is not only about changing what is implemented because of feedback, but also about providing us with the necessary weapons to fight feedback that would bring us on the wrong way. If we're not all convinced of what we do, the outside appearance is of a ruling cabal and a mass of people blindly following. If we're all united, instead, we'll stand on our strong arguments and defend the design against all detractors. (provided it is the right design, of course) Giovanni signature.asc Description: This is a digitally signed message part ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Where is the data?
More selective answers... :) On Sat, Aug 20, 2011 at 2:18 PM, Giovanni Campagna wrote: > Il giorno sab, 20/08/2011 alle 12.04 +0100, Allan Day ha scritto: > This is the first time I see the amount of users tested, and the exact > tasks involved. I think it should go somewhere in the wiki, especially > alongside the exact results (what was trickier? why?) > Thanks anyway! I really did want to write a proper report but I never found the time. Jon, Jimmac and Owen got a quick run down on the results and there's a bunch of bugs that I filed as a result of the testing. >> > Where we developers can find hard facts proving the NOTABUG and the >> > WONTFIX we mark in the most questioned and hot issues? >> >> You can always mark a bug with the ui-review keyword if you want a >> second opinion. > > You misunderstood me. I didn't say that I don't know when to close, I > said I don't know how to explain to the users why I'm closing. You need > facts to prove your points, or people won't understand and refuse to > agree. Oh right, sorry. There is evidence available to back up the current design but it's not all super hard and scientific. If you want a better justification for certain decisions, just ask the designers to add to the documentation on the wiki. Maybe we'll be able to do more user testing one day too. >> > I'm not a designer, so I may not understand all the papers you provide >> > in your support, and I may not understand what are the rules and laws of >> > Human Computer Interaction, as you call it. But I understand numbers, >> > and would be convinced by seeing that 66% percent of people find this >> > method of working more productive, or 3 out 5 tested users where able to >> > discover the functionality without guidance, or all 8 people interviewed >> > did not use the feature just removed. >> ... >> >> More user testing would be a good thing, and that might provide some >> of the numbers that you crave. In the mean time, we're not operating >> in the dark however: we can tell a lot from a combination of the UX >> literature, dog fooding, feed back from users and comparison with what >> other OSs/DEs/whatever are doing. It's not numerical data but it is >> data all the same. > > Usual example: the shutdown button. There is no UX literature proving > that "suspend is the right way to shutdown a system". Other systems > (Windows, Mac OS, KDE) are keeping power off as the primary method. Feed > back from users is far from enthusiastic. So... is there anything that > proves your points? The best examples of this kind of behaviour are mobile phones and tablets, imo. > As an example, it is said that the user wants to focus one specific task > at time, and thus the taskbar was removed and all task switching moved > to the overview. I can concede that the premise is correct, but it is > hardly self evident that the overview helps with this, given that a task > often involves more than one window and more than one workspace (which > forces the user to alt-tab to avoid the "overview distraction"). I *really* don't have the time to properly discuss these issues with you right now, I'm afraid... :) I think the main thing is that launching isn't always visible, and that that changes the experience for the better. (Yay for more baseless assertion! ;) ) >> I thought that the responses to the top ideas >> on Ubuntu brainstorm were a good example of how this can be done, >> actually [5]. Doing that kind of thing requires time and effort, of >> course... > > I see. Well, we could ask the feedback reporters to do the work. After > all, it is in their interest to have the developers focused on the > problems. > Or we could have voting in bugzilla. I know many people are against to > this, but it would immediately show the hottest issue, that surely > require reconsideration by the design team. Plus it would avoid +1 > comments that spam our mailboxes. The main thing would be to visibly demonstrate serious consideration of the most popular suggestions. There are a few different ways that those suggestions could be made; it'd be interesting to evaluate the different possible approaches. Hope that helps; sorry if it doesn't. Allan -- IRC: aday on irc.gnome.org Blog: http://afaikblog.wordpress.com/ ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Where is the data?
On Sat, Aug 20, 2011 at 6:02 PM, Shaun McCance wrote: > On Sat, 2011-08-20 at 14:43 +0200, Tomasz Torcz wrote: >> I think his objections were justified. There is really no raw data >> at those URLs. > > Except Allan never claimed he was providing raw data. In fact, > he explicitly said that he does not do write-ups of user tests. So what's the point of replying to a mail asking specifically for data? > I also do user tests when working on the help. I also don't do > write-ups. I fix problems or I pass information on to those who > are in a position to fix the problems. > > Just because I don't publish reports doesn't mean I don't do > user tests. And the constant assertions that nobody is looking > at feedback are getting a bit insulting. User tests, like surveys, are not perfect and can be both misleading, and not significant enough. If those tests are to be taken seriously, they should be published so that they can be scrutinized, otherwise they are not evidence of anything, not to the rest of the world. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Where is the data?
On Sat, 2011-08-20 at 14:43 +0200, Tomasz Torcz wrote: > On Sat, Aug 20, 2011 at 02:38:55PM +0200, Olav Vitters wrote: > > On Sat, Aug 20, 2011 at 02:28:54PM +0300, Felipe Contreras wrote: > > > So, no data there. > > > > > There's no data there. > > > > > No data there either. > > > > > I have read that whole thread; there's no data there either, just > > > about of explanations that can be reduced to; it's sleek. > > > > > No data here either. > > > > > Here's some data, but I don't see any connection to GNOME 3 design. > > > > > No data here either. > > > > This is not kindergarten. I've set your moderation bit. > > I think his objections were justified. There is really no raw data > at those URLs. Except Allan never claimed he was providing raw data. In fact, he explicitly said that he does not do write-ups of user tests. I also do user tests when working on the help. I also don't do write-ups. I fix problems or I pass information on to those who are in a position to fix the problems. Just because I don't publish reports doesn't mean I don't do user tests. And the constant assertions that nobody is looking at feedback are getting a bit insulting. -- Shaun ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Where is the data?
On 20 August 2011 14:18, Giovanni Campagna wrote: > Or we could have voting in bugzilla. I know many people are against to > this, but it would immediately show the hottest issue, that surely > require reconsideration by the design team. No. It would only tell you that a bunch of users who don't like a particular design decision were lured into voting on bugzilla from some random web forum. In particular, it wouldn't tell you *anything* about all the users who approve the decision or, most probably, aren't even aware of it because they don't spend all their time discussing their computer's UI. Rui ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Where is the data?
Il giorno sab, 20/08/2011 alle 12.04 +0100, Allan Day ha scritto: > Hey Giovanni, > > I'm being selective in my responses here... > > Giovanni Campagna wrote: > > I'm sorry I couldn't read through the whole GNOME Survey v4 thread, but > > it was just too long. What I read though is that data was collected and > > extists. > > Now I'd like to simply ask: where is it? > > Where we developers can find real cold numbers backing out the designs > > we're asked to implement? > > I wrote that we already have lots of data on peoples' experiences with > GNOME 3. Is that what you mean? There's a brief summary of that in the > final paragraph of my previous email to the list [1]. > > I never found the time to do a proper write up of the user testing I > did on the shell. It was brief and ad hoc; I can tell you that the > five participants in my study were all able to complete the tasks that > were set for them though, which included basic things like launching > applications, switching windows, changing the desktop background, > responding to notifications and changing the volume via the system > settings area. They obviously found some things trickier than others, > but they could do everything I asked them to do. This is the first time I see the amount of users tested, and the exact tasks involved. I think it should go somewhere in the wiki, especially alongside the exact results (what was trickier? why?) Thanks anyway! > > Where we developers can find hard facts proving the NOTABUG and the > > WONTFIX we mark in the most questioned and hot issues? > > You can always mark a bug with the ui-review keyword if you want a > second opinion. You misunderstood me. I didn't say that I don't know when to close, I said I don't know how to explain to the users why I'm closing. You need facts to prove your points, or people won't understand and refuse to agree. > > I'm not a designer, so I may not understand all the papers you provide > > in your support, and I may not understand what are the rules and laws of > > Human Computer Interaction, as you call it. But I understand numbers, > > and would be convinced by seeing that 66% percent of people find this > > method of working more productive, or 3 out 5 tested users where able to > > discover the functionality without guidance, or all 8 people interviewed > > did not use the feature just removed. > ... > > More user testing would be a good thing, and that might provide some > of the numbers that you crave. In the mean time, we're not operating > in the dark however: we can tell a lot from a combination of the UX > literature, dog fooding, feed back from users and comparison with what > other OSs/DEs/whatever are doing. It's not numerical data but it is > data all the same. Usual example: the shutdown button. There is no UX literature proving that "suspend is the right way to shutdown a system". Other systems (Windows, Mac OS, KDE) are keeping power off as the primary method. Feed back from users is far from enthusiastic. So... is there anything that proves your points? (The nice feature of numerical data is that it's unquestionable. Anything else can be disagreed, which leads to discussion and hundred comments long bug reports) > > I know that what I write, following the guidelines and the mockups, is > > right. But people providing feedback don't always agree with that, and > > if myself cannot understand the reason, how can I explain to them? > ... > > The basic features of the shell design are explained on the wiki [2]. > It's helpful to have a more thorough explanation for more > controversial design decisions though, such as the ones that Owen [3] > and me [4] provided for window buttons. Just ask if you want more > information on a particular issue. The design, as explained in the wiki, is a set of assertions. I am sure that UX literature contains the explanation for why those assertions are right, but as they stand, we can place a [citation needed] tag on them. Plus there is nothing there explaining why current implementation is the best way to achieve those design goals. As an example, it is said that the user wants to focus one specific task at time, and thus the taskbar was removed and all task switching moved to the overview. I can concede that the premise is correct, but it is hardly self evident that the overview helps with this, given that a task often involves more than one window and more than one workspace (which forces the user to alt-tab to avoid the "overview distraction"). The explanation Owen gave on the minimize/maximize issue on the other hand was complete, provided real world examples and showed the problems of the previous design. For comparison, at the desktop summit I listened to a talk by the same person on the shutdown stuff, and I was not convinced at all, because I did not get the "why" of the basic premise on which it was all set. > > I understand that some features in 3.0 were like "design experiments", > > because we have the
Re: Where is the data?
On Sat, Aug 20, 2011 at 02:43:28PM +0200, Tomasz Torcz wrote: > I think his objections were justified. There is really no raw data > at those URLs. My problem is not with objections. It is about the way of expressing those objections. -- Regards, Olav ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Where is the data?
On Sat, Aug 20, 2011 at 02:38:55PM +0200, Olav Vitters wrote: > On Sat, Aug 20, 2011 at 02:28:54PM +0300, Felipe Contreras wrote: > > So, no data there. > > > There's no data there. > > > No data there either. > > > I have read that whole thread; there's no data there either, just > > about of explanations that can be reduced to; it's sleek. > > > No data here either. > > > Here's some data, but I don't see any connection to GNOME 3 design. > > > No data here either. > > This is not kindergarten. I've set your moderation bit. I think his objections were justified. There is really no raw data at those URLs. -- Tomasz TorczThere exists no separation between gods and men: xmpp: zdzich...@chrome.pl one blends softly casual into the other. ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Where is the data?
On Sat, Aug 20, 2011 at 02:28:54PM +0300, Felipe Contreras wrote: > So, no data there. > There's no data there. > No data there either. > I have read that whole thread; there's no data there either, just > about of explanations that can be reduced to; it's sleek. > No data here either. > Here's some data, but I don't see any connection to GNOME 3 design. > No data here either. This is not kindergarten. I've set your moderation bit. -- Regards, Olav ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Where is the data?
Nguyen Thai Ngoc Duy wrote: > On Sat, Aug 20, 2011 at 6:04 PM, Allan Day wrote: >> I never found the time to do a proper write up of the user testing I >> did on the shell. It was brief and ad hoc; I can tell you that the >> five participants in my study were all able to complete the tasks that >> were set for them though, which included basic things like launching >> applications, switching windows, changing the desktop background, >> responding to notifications and changing the volume via the system >> settings area. They obviously found some things trickier than others, >> but they could do everything I asked them to do. > > While that covers feature exploration, I don't think it covers > usability over a long period of time of use, once users know what > gnome3 provides: given a workflow, how efficient is window switching, > what is often used, for example. That's correct. My small study had a narrow scope. > Have you also done such user > testing? That's not exactly the kind of thing a volunteer can do in their spare time. ;) It would require non-trivial financial backing to do a study of medium to long-term usage. Allan -- IRC: aday on irc.gnome.org Blog: http://afaikblog.wordpress.com/ ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Where is the data?
On Sat, Aug 20, 2011 at 6:04 PM, Allan Day wrote: > I never found the time to do a proper write up of the user testing I > did on the shell. It was brief and ad hoc; I can tell you that the > five participants in my study were all able to complete the tasks that > were set for them though, which included basic things like launching > applications, switching windows, changing the desktop background, > responding to notifications and changing the volume via the system > settings area. They obviously found some things trickier than others, > but they could do everything I asked them to do. While that covers feature exploration, I don't think it covers usability over a long period of time of use, once users know what gnome3 provides: given a workflow, how efficient is window switching, what is often used, for example.. Have you also done such user testing? -- Duy ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Where is the data?
On Sat, Aug 20, 2011 at 2:04 PM, Allan Day wrote: > I never found the time to do a proper write up of the user testing I > did on the shell. It was brief and ad hoc; I can tell you that the > five participants in my study were all able to complete the tasks that > were set for them though, which included basic things like launching > applications, switching windows, changing the desktop background, > responding to notifications and changing the volume via the system > settings area. They obviously found some things trickier than others, > but they could do everything I asked them to do. So, no data there. > [1] > http://mail.gnome.org/archives/desktop-devel-list/2011-August/msg00123.html There's no data there. > [2] https://live.gnome.org/GnomeShell/Design/ No data there either. > [3] > http://mail.gnome.org/archives/gnome-shell-list/2011-February/msg00192.html I have read that whole thread; there's no data there either, just about of explanations that can be reduced to; it's sleek. > [4] http://afaikblog.wordpress.com/2011/03/01/where-did-the-buttons-go/ No data here either. > [5] > http://mdzlog.alcor.net/2010/12/10/ubuntu-brainstorm-top-10-for-december-2010/ Here's some data, but I don't see any connection to GNOME 3 design. > [6] https://live.gnome.org/ThreePointOne/Features/FixAnnoyingThings No data here either. -- Felipe Contreras ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Where is the data?
Hey Giovanni, I'm being selective in my responses here... Giovanni Campagna wrote: > I'm sorry I couldn't read through the whole GNOME Survey v4 thread, but > it was just too long. What I read though is that data was collected and > extists. > Now I'd like to simply ask: where is it? > Where we developers can find real cold numbers backing out the designs > we're asked to implement? I wrote that we already have lots of data on peoples' experiences with GNOME 3. Is that what you mean? There's a brief summary of that in the final paragraph of my previous email to the list [1]. I never found the time to do a proper write up of the user testing I did on the shell. It was brief and ad hoc; I can tell you that the five participants in my study were all able to complete the tasks that were set for them though, which included basic things like launching applications, switching windows, changing the desktop background, responding to notifications and changing the volume via the system settings area. They obviously found some things trickier than others, but they could do everything I asked them to do. > Where we developers can find hard facts proving the NOTABUG and the > WONTFIX we mark in the most questioned and hot issues? You can always mark a bug with the ui-review keyword if you want a second opinion. > I'm not a designer, so I may not understand all the papers you provide > in your support, and I may not understand what are the rules and laws of > Human Computer Interaction, as you call it. But I understand numbers, > and would be convinced by seeing that 66% percent of people find this > method of working more productive, or 3 out 5 tested users where able to > discover the functionality without guidance, or all 8 people interviewed > did not use the feature just removed. ... More user testing would be a good thing, and that might provide some of the numbers that you crave. In the mean time, we're not operating in the dark however: we can tell a lot from a combination of the UX literature, dog fooding, feed back from users and comparison with what other OSs/DEs/whatever are doing. It's not numerical data but it is data all the same. > I know that what I write, following the guidelines and the mockups, is > right. But people providing feedback don't always agree with that, and > if myself cannot understand the reason, how can I explain to them? ... The basic features of the shell design are explained on the wiki [2]. It's helpful to have a more thorough explanation for more controversial design decisions though, such as the ones that Owen [3] and me [4] provided for window buttons. Just ask if you want more information on a particular issue. > I understand that some features in 3.0 were like "design experiments", > because we have the whole 3.* cycle to improve. But if the results of > those experiments (that is, people's feedback) is not analyzed > thoroughly, how can we be sure that the design is right? Feedback does get read and gets weighed up against the other evidence that is available. I think it would be great if we had a more visible process there, however. I thought that the responses to the top ideas on Ubuntu brainstorm were a good example of how this can be done, actually [5]. Doing that kind of thing requires time and effort, of course... Also, we have been tracking the more minor niggles that people have reported [6]. > Or on the other > hand, how can I see that the feedback is listened to, if decisions are > never reverted? The shell design has never been set in concrete. Many things were changed during the design and development process and I'm sure there will be more changes in the future. I think it's too early to expect significant changes to the 3.0 design though. Allan [1] http://mail.gnome.org/archives/desktop-devel-list/2011-August/msg00123.html [2] https://live.gnome.org/GnomeShell/Design/ [3] http://mail.gnome.org/archives/gnome-shell-list/2011-February/msg00192.html [4] http://afaikblog.wordpress.com/2011/03/01/where-did-the-buttons-go/ [5] http://mdzlog.alcor.net/2010/12/10/ubuntu-brainstorm-top-10-for-december-2010/ [6] https://live.gnome.org/ThreePointOne/Features/FixAnnoyingThings -- IRC: aday on irc.gnome.org Blog: http://afaikblog.wordpress.com/ ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Where is the data?
On 19 August 2011 19:00, Giovanni Campagna wrote: > As a specific example of unscientific user testing, I got a friend of > mine to try GNOME 3 at the desktop summit, and when it was time to > shutdown he just asked me, because he found no way and he thought it was > a bug. I'm sorry but I didn't have any explanations, so I just said "the > designers said so", and similarly I had none when some KDE hackers asked > me on the same problem. > I know that what I write, following the guidelines and the mockups, is > right. But people providing feedback don't always agree with that, and > if myself cannot understand the reason, how can I explain to them? > I understand that some features in 3.0 were like "design experiments", > because we have the whole 3.* cycle to improve. But if the results of > those experiments (that is, people's feedback) is not analyzed > thoroughly, how can we be sure that the design is right? Or on the other > hand, how can I see that the feedback is listened to, if decisions are > never reverted? The Alt modifier to get to Shutdown options in the usermenu is clearly wrong. As far as I can see, that Alt modifier is used only once in the entirety of GNOME Core and is definitely not a common interaction anywhere in the software world. Thus it is completely non-discoverable. It is impossible for anyone to ever figure out that functionality unless they were told it was there. Core functionality like powering off a computer should not be an easter egg. I know that this has been discussed at length on the bug report and elsewhere but the designers have so far refused to revert this bug/feature so that normal people can use the usermenu effectively. I believe Debian has interest in shipping a non-hidden Power Off button and I'd expect Ubuntu's GNOME Shell package would follow that example. Jeremy ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Where is the data?
I'm sorry I couldn't read through the whole GNOME Survey v4 thread, but it was just too long. What I read though is that data was collected and extists. Now I'd like to simply ask: where is it? Where we developers can find real cold numbers backing out the designs we're asked to implement? Where we developers can find hard facts proving the NOTABUG and the WONTFIX we mark in the most questioned and hot issues? I'm not a designer, so I may not understand all the papers you provide in your support, and I may not understand what are the rules and laws of Human Computer Interaction, as you call it. But I understand numbers, and would be convinced by seeing that 66% percent of people find this method of working more productive, or 3 out 5 tested users where able to discover the functionality without guidance, or all 8 people interviewed did not use the feature just removed. Note the small numbers: I know that user testing is limited in scope, and I know that reaching out to a broader audience would require money that the foundation doesn't have. But if you insist in saying that all necessary feedback was already acquired, you need to make it public somewhere. You need to back your decisions on market share, not technical purity, because that is our goal (imho). I continously read of people complaining about GNOME 3, around the web and in gnome-shell-list (because, unfortunately, much of the complaint is against "the shell way"), and the answers, when provided, are just repeating the same design assertions, to the point that some subscribers are fed up of writing them. As a specific example of unscientific user testing, I got a friend of mine to try GNOME 3 at the desktop summit, and when it was time to shutdown he just asked me, because he found no way and he thought it was a bug. I'm sorry but I didn't have any explanations, so I just said "the designers said so", and similarly I had none when some KDE hackers asked me on the same problem. I know that what I write, following the guidelines and the mockups, is right. But people providing feedback don't always agree with that, and if myself cannot understand the reason, how can I explain to them? I understand that some features in 3.0 were like "design experiments", because we have the whole 3.* cycle to improve. But if the results of those experiments (that is, people's feedback) is not analyzed thoroughly, how can we be sure that the design is right? Or on the other hand, how can I see that the feedback is listened to, if decisions are never reverted? Sorry for this long mail, and sorry for contributing to the desktop-devel noise, but I've been waiting to ask these questions for too long. I hope that a striking and factual answer will avoid lengthy discussion. Giovanni signature.asc Description: This is a digitally signed message part ___ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list
