Re: [Wikimedia-l] Upcoming Survey, Feedback requested, and Office Hour
Dear Jan-Bart, Unfortunately the exact wordings was I'll point out also that there are zero real-world implications for the survey results. Because we all agree that there is now such thing like a zero real-world impact survey, we really hope that the raw results of this survey will be made as public as possible (privacy issue), and that in the future , survey including question about WMF partners (chapters are not the only ones) will be done since the very beginning in collaboration with all the partners involved. sincerely Charles ___ Charles ANDRES, Chairman Wikimedia CH – Association for the advancement of free knowledge – www.wikimedia.ch Skype: charles.andres.wmch IRC://irc.freenode.net/wikimedia-ch Le 12 sept. 2012 à 16:14, Jan-Bart de Vreede jdevre...@wikimedia.org a écrit : Hey So I might have missed some mails on this thread (perhaps because they were not posted on this public list) but I highly doubt that Sue perform surveys that do not have a real-world impact on our operations. I know that the results of the previous surveys were used in several discussions (including at a board level) in order to provide more insight…. On the other hand, using these surveys to gain more insight is not the same as using them to hold each other accountable which is sometimes easy to do. Every survey (and questions) has a lots of interpretation magic which can easily lead you astray, but I don't have to tell you (the community) this :) Jan-Bart On 10 Sep 2012, at 23:01, Delphine Ménard notafi...@gmail.com wrote: Contrarily to Sue, I do think that these surveys (should) have a real-world impact and (should) keep us all on our toes, fine tuned to the critisicism, needs and wishes of the editors of the WIkimedia projects. As such I expect us to make sure that we do get as precise a picture as possible of what those are. ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
Re: [Wikimedia-l] Upcoming Survey, Feedback requested, and Office Hour
On 9/12/12 4:14 PM, Jan-Bart de Vreede wrote: Hey So I might have missed some mails on this thread (perhaps because they were not posted on this public list) but I highly doubt that Sue perform surveys that do not have a real-world impact on our operations. I know that the results of the previous surveys were used in several discussions (including at a board level) in order to provide more insight…. Indeed. I agree. These surveys do have real-world impact, which is why we objected to a survey asking people from all over the world how they would rate Wikimedia Chapters activities when 1) there is likely no chapter in their country 2) they may have no idea that a wikimedia chapter is for example Wikimedia Washington DC or Wikimedia Israel 3) all chapters are collectively considered regardless of individual differences And since you comment on that specific sentence I'll point out also that there are zero real-world implications for the survey results. I'd like to clarify that these exact words come from Sue herself in an email sent on the 10th of September on internal-l. I am glad to read that you disagree with that statement and recognize that there is real-world impact. (did not want to comment any further on that problematic survey, but wanted to attribute statement properly) Flo On the other hand, using these surveys to gain more insight is not the same as using them to hold each other accountable which is sometimes easy to do. Every survey (and questions) has a lots of interpretation magic which can easily lead you astray, but I don't have to tell you (the community) this :) Jan-Bart On 10 Sep 2012, at 23:01, Delphine Ménard notafishz-re5jqeeqqe8avxtiumw...@public.gmane.org wrote: Contrarily to Sue, I do think that these surveys (should) have a real-world impact and (should) keep us all on our toes, fine tuned to the critisicism, needs and wishes of the editors of the WIkimedia projects. As such I expect us to make sure that we do get as precise a picture as possible of what those are. ___ Wikimedia-l mailing list wikimedia-l-rusutvdil2icgmh+5r0dm0b+6bgkl...@public.gmane.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
Re: [Wikimedia-l] Upcoming Survey, Feedback requested, and Office Hour
Hey So someone sent me the internal-l mail and I do think that the zero real-world thing is taken out of context here. But a few points 1) Lets not have these discussions on internal-l, there is no reason to not have those in public 2) See 1) 3) I am sure that the data set allows us to see chapter's individual responses, depending on whether or not we know the country (I would figure we do?) 4) Doesn't every survey contain questions that don't apply to the whole responding audience? Jan-Bart On 14 sep. 2012, at 17:05, Florence Devouard anthe...@yahoo.com wrote: On 9/12/12 4:14 PM, Jan-Bart de Vreede wrote: Hey So I might have missed some mails on this thread (perhaps because they were not posted on this public list) but I highly doubt that Sue perform surveys that do not have a real-world impact on our operations. I know that the results of the previous surveys were used in several discussions (including at a board level) in order to provide more insight…. Indeed. I agree. These surveys do have real-world impact, which is why we objected to a survey asking people from all over the world how they would rate Wikimedia Chapters activities when 1) there is likely no chapter in their country 2) they may have no idea that a wikimedia chapter is for example Wikimedia Washington DC or Wikimedia Israel 3) all chapters are collectively considered regardless of individual differences And since you comment on that specific sentence I'll point out also that there are zero real-world implications for the survey results. I'd like to clarify that these exact words come from Sue herself in an email sent on the 10th of September on internal-l. I am glad to read that you disagree with that statement and recognize that there is real-world impact. (did not want to comment any further on that problematic survey, but wanted to attribute statement properly) Flo On the other hand, using these surveys to gain more insight is not the same as using them to hold each other accountable which is sometimes easy to do. Every survey (and questions) has a lots of interpretation magic which can easily lead you astray, but I don't have to tell you (the community) this :) Jan-Bart On 10 Sep 2012, at 23:01, Delphine Ménard notafishz-re5jqeeqqe8avxtiumw...@public.gmane.org wrote: Contrarily to Sue, I do think that these surveys (should) have a real-world impact and (should) keep us all on our toes, fine tuned to the critisicism, needs and wishes of the editors of the WIkimedia projects. As such I expect us to make sure that we do get as precise a picture as possible of what those are. ___ Wikimedia-l mailing list wikimedia-l-rusutvdil2icgmh+5r0dm0b+6bgkl...@public.gmane.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
Re: [Wikimedia-l] Upcoming Survey, Feedback requested, and Office Hour
On 14/09/12 18:08, Jan-bart de Vreede wrote: 4) Doesn't every survey contain questions that don't apply to the whole responding audience? No, because the answer would not be terribly useful. In all surveys I have seen in my work life [statistics work -- I don't design much surveys myself, but reply to a lot of them from different horizons :-], this does never happen. Typically, the filtering for a question about chapters would be something like this (simplified and written quickly): 1) Do you know any organization active within the Wikimedia movement ? (free text fields allow people to enter names) 2) This page contain a list of organizations active within the Wikimedia movement; please tick all the ones you know (even if you have only heard the name) (a list follows, with maybe WMF, Local chapters in general, and a list of individual chapters) 3) People are then asked to rate and/or comment each entity that was mentioned under (1) or (2). This way, not only do we avoid having people give their opinion on a topic they've never heard about, but we also get two different levels of knowledge on the topic (either the user knew it well enough to list the name, or he had to be reminded). Frédéric ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
Re: [Wikimedia-l] Upcoming Survey, Feedback requested, and Office Hour
Hello Tilman, On Sun, Sep 9, 2012 at 7:24 PM, Tilman Bayer tba...@wikimedia.org wrote: Hi Delphine, On Tue, Aug 21, 2012 at 6:57 AM, Delphine Ménard notafi...@gmail.com wrote: On Tue, Aug 21, 2012 at 3:32 PM, Tilman Bayer tba...@wikimedia.org wrote: ... Still, I was aware that there had been some objections to that question by chapter representatives (which I don't assume have to do with the fact that respondents rated chapters' performance lower than that of other entities in the two previous surveys), and looked into these concerns while the present questionnaire was prepared; I also reached out to one of the critics in person at Wikimania. But I still haven't seen a compelling argument why the way the question is asked should be biased against chapters. The argument that the opinion of Wikimedians who live in countries without chapter should not count seems weak to me, e.g. because the projects that the work of chapters aims to support are international, and because the question asked about chapters in general, not one particular chapter. That is not the argument I was trying to make (ie. voices of Wikimedians in a country without chapter don't count). Rather, there is a long list of things the Foundation does, where people are asked whether they knew about it, or not. And after that, right when people have been made aware of everything the Foundation does, they are asked to rate the work of the Foundation. The same question about the chapters comes after absolutely nothing has been said about chapter work, which, I believe, does introduce a bias. In short, people are being asked to rate something they *at this point in the survey* have an idea about (for the WMF) although they might have had no idea about it before starting the survey. All I'm asking is that we review the context in which this question is being asked so results make more sense. OK, after some other people also remarked that preceding this question by other questions which conveyed quite some information about the Foundation's activities but not about the chapters' activities. we have now rearranged the questions so that this is no longer the case. This is a bit of a compromise regarding the structuring of the questionnaire into sections, but fortunately it could be done without invalidating existing translations or changing the variables of the resulting dataset. Thanks. I will not hide that I am still not sure whether we don't now have two out of context questions instead of just one, but I guess it's what we could do for this round, so thank you for doing this. I sincerely hope that we can all together revisit this part of the survey to give results that can be used by all of us to increase satisfaction and performance in the future. Contrarily to Sue, I do think that these surveys (should) have a real-world impact and (should) keep us all on our toes, fine tuned to the critisicism, needs and wishes of the editors of the WIkimedia projects. As such I expect us to make sure that we do get as precise a picture as possible of what those are. Best, Delphine -- @notafish NB. This gmail address is used for mailing lists. Personal emails will get lost. Intercultural musings: Ceci n'est pas une endive - http://blog.notanendive.org Photos with simple eyes: notaphoto - http://photo.notafish.org ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
Re: [Wikimedia-l] Upcoming Survey, Feedback requested, and Office Hour
Hi, sorry to fish out a very old message, but since the survey is about to go live, I would like to share some concerns I have about it. Unfortunately, I was on holidays as this announcement came out, so couldn't do it earlier. I find the idea of an editor's survey to be extremely important, since it is (among other) a good indicator of how the editing community perceives the atmosphere in the projects, the evolution of the software and such things. However, I feel this survey is a bit of a missed opportunity on different aspects. There is a mix of feedback about the projects and the community and satisfaction about the WMF, which does not, in my opinion, quite fit together. I find we should separate those things so as to keep people free of personal opinions about what the organisation may or may not do for/with them and let them focus better on their editor's experience as such. Moreover, this would allow for more questions about editing, maybe a short presentation of new tools, rating them etc. which seems to be quite absent from this questionnaire. For example, I would love to see a question in the technology part about whether people want/edit from their mobile device, or if they are familiar with the mobile apps and use them, that kind of stuff. (rationale given for taking these questions out was length of the survey, but I think these things are much more relevant to the well-editing of the contributors than how well they rate the WMF work). Not to mention that trying to get some feedback from sister projects would be good also (Commons is already a good first step). If, however, we're going to mix editor's experience and satisfaction about Wikimedia, I am cruelly missing any kind of feedback question about the work of the chapters and/or other organisations or groups in the Wikimedia Universe that would give people the right scope about what is happening in a more offline kind of way. Of course, we could do a separate survey for chapters, but if we're truly an international movement, then all Wikimedia entities that support/interact with the community probably would benefit from being put in the same bag in order to fine tune their support and help for the Wikimedia communities. Sue in Washington at Wikimania pulled out the results of one question that was asked in the editor's survey about whether people were satisfied about the work of the Foundation and the work of the chapters. She underlined herself that the results to this question were probably difficult to interpret out of the box since at no point in the survey was there a question about whether people were aware that there was a chapter in their country, which would have qulified the results a bit. This was already quite criticized last time, yet the question is still there, unchanged, and without more context than it had last time. (under FINAL THOUGHTS). In short people are asked to rate the Foundation about everything it does, while the chapters are never mentionned, and then people are asked to rate the work of both. Interesting way to look at it. We're doing much better, but there is still some English Wikipedia centrism in Question F2 for example. ;-) : How well do you believe the Foundation supports: English Wikipedia? Wikipedia sites in other languages? Finally, what I regret most, is that so little time was allocated to reviewing this survey collaboratively. There was about 20 days to review, comment, give feedback, translate etc. and this in the mist of the summer holiday for a big part of our community. When I see such processes as the ToS revision [1] conducted successfully allowing for 120 days of discussion, and such surveys rushed through the summer, when their data will probably be used to decide much of the strategic orientation of the next year, I'm thinking we're missing out on trying to collect data that is truly relevant to the work we do and that helps us to review our orientations and adjust how well we support our editing community. Best, Delphine [1] http://meta.wikimedia.org/wiki/Talk:Terms_of_use On Sat, Jul 28, 2012 at 7:28 AM, Christine Moellenberndt christine...@gmail.com wrote: *Hi everyone, It's been a bit since I last emailed this list (or any list, for that matter!)... you may remember me, I worked at the Foundation last year in the Community Department, working with Philippe on any number of issues, as well as with the OTRS team. I've come back to work on a short term project with the Foundation, and I have to say it's great to be back! (and a great break from my Master's thesis!) We're getting ready to run the next version of the Editor Survey, for August 2012. This will be the third incarnation we've run since 2011. As with the prior incarnations of the survey, we'll be looking at a variety of topics, this time with the goal of not only understanding your needs and pressing issues while interacting with fellow editors, but also focusing on editors' satisfaction with the
Re: [Wikimedia-l] Upcoming Survey, Feedback requested, and Office Hour
On Fri, Jul 27, 2012 at 10:28 PM, Christine Moellenberndt christine...@gmail.com wrote: *Hi everyone, It's been a bit since I last emailed this list (or any list, for that matter!)... you may remember me, I worked at the Foundation last year in the Community Department, working with Philippe on any number of issues, as well as with the OTRS team. I've come back to work on a short term project with the Foundation, and I have to say it's great to be back! (and a great break from my Master's thesis!) We're getting ready to run the next version of the Editor Survey, for August 2012. This will be the third incarnation we've run since 2011. As with the prior incarnations of the survey, we'll be looking at a variety of topics, this time with the goal of not only understanding your needs and pressing issues while interacting with fellow editors, but also focusing on editors' satisfaction with the work of the Foundation. The last time we ran an editor survey, it was completed by over 6,000 respondents. When you break that down, it means that each minute of time demanded by the survey corresponds to 100 hours of Wikipedians' time. We want to make sure that this time is spent wisely, ensuring that the questions we have are worded clearly, don't cause confusion, and will generate meaningful answers. So we'd like to ask you to take a look at the survey, and give us feedback on the questions. You can find them here: https://meta.wikimedia.org/**wiki/Research:Wikipedia_** Editor_Survey_August_2012/**Questionshttps://meta.wikimedia.org/wiki/Research:Wikipedia_Editor_Survey_August_2012/Questions https://meta.wikimedia.org/w/**index.php?title=Research:** Wikipedia_Editor_Survey_**August_2012/Questionshttps://meta.wikimedia.org/w/index.php?title=Research:Wikipedia_Editor_Survey_August_2012/Questions ... and please leave your feedback on the talk page there so we can keep the discussion in one place :) You can find out more information about the survey here: https://meta.wikimedia.org/**wiki/Research:Wikipedia_** Editor_Survey_August_2012https://meta.wikimedia.org/wiki/Research:Wikipedia_Editor_Survey_August_2012 Also, we are planning an IRC Office Hour on the survey, this **Tuesday, July 31 at 1700 UTC.** (See https://meta.wikimedia.org/** wiki/IRC_office_hoursforhttps://meta.wikimedia.org/wiki/IRC_office_hoursforgeneral information about IRC Office hours) Just a quick reminder that this is happening two hours from now. I know there has been some discussion about offering Office Hours in a broader range of times, and I know this time may not be the greatest for some... but this was the best time we could find currently. Thanks everyone!! -Christine Wikimedia Foundation* __**_ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.**org Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/**mailman/listinfo/wikimedia-lhttps://lists.wikimedia.org/mailman/listinfo/wikimedia-l -- Tilman Bayer Senior Operations Analyst (Movement Communications) Wikimedia Foundation IRC (Freenode): HaeB ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
[Wikimedia-l] Upcoming Survey, Feedback requested, and Office Hour
*Hi everyone, It's been a bit since I last emailed this list (or any list, for that matter!)... you may remember me, I worked at the Foundation last year in the Community Department, working with Philippe on any number of issues, as well as with the OTRS team. I've come back to work on a short term project with the Foundation, and I have to say it's great to be back! (and a great break from my Master's thesis!) We're getting ready to run the next version of the Editor Survey, for August 2012. This will be the third incarnation we've run since 2011. As with the prior incarnations of the survey, we'll be looking at a variety of topics, this time with the goal of not only understanding your needs and pressing issues while interacting with fellow editors, but also focusing on editors' satisfaction with the work of the Foundation. The last time we ran an editor survey, it was completed by over 6,000 respondents. When you break that down, it means that each minute of time demanded by the survey corresponds to 100 hours of Wikipedians' time. We want to make sure that this time is spent wisely, ensuring that the questions we have are worded clearly, don't cause confusion, and will generate meaningful answers. So we'd like to ask you to take a look at the survey, and give us feedback on the questions. You can find them here: https://meta.wikimedia.org/wiki/Research:Wikipedia_Editor_Survey_August_2012/Questions https://meta.wikimedia.org/w/index.php?title=Research:Wikipedia_Editor_Survey_August_2012/Questions ... and please leave your feedback on the talk page there so we can keep the discussion in one place :) You can find out more information about the survey here: https://meta.wikimedia.org/wiki/Research:Wikipedia_Editor_Survey_August_2012 Also, we are planning an IRC Office Hour on the survey, this **Tuesday, July 31 at 1700 UTC.** (See https://meta.wikimedia.org/wiki/IRC_office_hoursfor general information about IRC Office hours) I know there has been some discussion about offering Office Hours in a broader range of times, and I know this time may not be the greatest for some... but this was the best time we could find currently. Thanks everyone!! -Christine Wikimedia Foundation* ___ Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
