Re: [Wiki-research-l] Wiki Research Journal? - Conference vs. journal publication
I think it basically is a different publication model. You probably could have two track (one for final, the other for working papers) so as to address the disciplines which rely on journals as final outlets. Best, Dj 20 lis 2012 20:26, Jodi Schneider jschnei...@pobox.com napisał(a): Hi Dariusz, For reusing the paper unchanged this is indeed a problem. Journal could be added to the list of mentioned reuse venues--but this still wouldn't imply that the entirety of the paper could be used without change, I suspect. For ACM conferences, there are two types of papers: - archival - non-archival Archival papers are published in the ACM digital library, in the conference proceedings, and are considered final research products to be cited. Non-archival papers are not considered final research products and (as far as I know) don't require copyright transfer. Aside from making papers destined to be used VERBATIM without change in journals 'non-archival', how could this be addressed? -Jodi On Thu, Nov 8, 2012 at 6:16 PM, Dariusz Jemielniak dar...@alk.edu.plwrote: hi Jodi, the conferences I attend or follow (e.g. EGOS, AoM, APROS. SFAA) afaik do not typically require signed copyright notices at all, and if they do, the copyright is granted specifically for publishing in the proceedings, and legally resembles a license more, than a full copyright transfer. The problem with your copyright form, as I read it, is that ACM receives and retains all rights (which may be not welcome by some journal publishers, and you never know where eventually you're going to try to publish, so why risk?), and also that it does not specify journal articles as acceptable forms of future reusing the paper. In fact, the copyright form is very similar to the forms used by journals (many of which allow publishing the article on a personal page or in future book works, if proper attribution is provided). This also may indicate to some possible attendants that WikiSym conference publication is an alternative to a journal publication. In my field a conference paper does not count as a publication and is usually treated as a way of improving the paper before submitting it to a journal. Thus the possibility that you're going to use the paper almost verbatim in a journal submission is quite real. These differences possibly may result in WikiSym, as a conference and only in some disciplines, being less popular than it deserves. best, dariusz On Thu, Nov 8, 2012 at 6:59 PM, Jodi Schneider jschnei...@pobox.comwrote: Hi Dariusz, This is interesting, because if we can articulate problems in the copyright notice, we may be able to fix them. Currently, for WikiSym, the ACM Publications copyright form for proceedings is used: *http://www.acm.org/publications/CopyReleaseProc-9-12.pdf* This includes: * The right to reuse any portion of the Work, without fee, in future works of the Author's (or Author's Employer's) own, including books, lectures and presentations in all media, provided that the ACM citation, notice of the Copyright and the ACM DOI are included ... * The right to revise the work. What is a typical copyright notice for conferences you *do* publish in? What rights do you need? In computer science, typically a journal publication arising in part from a conference paper would include new results, or several papers would be combined. -Jodi On Thu, Nov 8, 2012 at 5:23 PM, Dariusz Jemielniak dar...@alk.edu.plwrote: This is why conferences such as WikiSym are not very attractive for my field, as they require some copyright transfer (which may effectively make publishing in the final destination difficult). -- __ dr hab. Dariusz Jemielniak profesor zarządzania kierownik katedry Zarządzania Międzynarodowego i centrum badawczego CROW Akademia Leona Koźmińskiego http://www.crow.alk.edu.pl ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal? - Conference vs. journal publication
Can we not just label which of our accepted papers are archival? It seems that some disciplines assume Journal == Archival and Conference != Archival. This is apparently inaccurate in other disciplines so there must be some reason we don't just note which papers are archived and which are not. I don't see it though. -Aaron On Wed, Nov 21, 2012 at 8:00 AM, Dariusz Jemielniak dar...@alk.edu.plwrote: I think it basically is a different publication model. You probably could have two track (one for final, the other for working papers) so as to address the disciplines which rely on journals as final outlets. Best, Dj 20 lis 2012 20:26, Jodi Schneider jschnei...@pobox.com napisał(a): Hi Dariusz, For reusing the paper unchanged this is indeed a problem. Journal could be added to the list of mentioned reuse venues--but this still wouldn't imply that the entirety of the paper could be used without change, I suspect. For ACM conferences, there are two types of papers: - archival - non-archival Archival papers are published in the ACM digital library, in the conference proceedings, and are considered final research products to be cited. Non-archival papers are not considered final research products and (as far as I know) don't require copyright transfer. Aside from making papers destined to be used VERBATIM without change in journals 'non-archival', how could this be addressed? -Jodi On Thu, Nov 8, 2012 at 6:16 PM, Dariusz Jemielniak dar...@alk.edu.plwrote: hi Jodi, the conferences I attend or follow (e.g. EGOS, AoM, APROS. SFAA) afaik do not typically require signed copyright notices at all, and if they do, the copyright is granted specifically for publishing in the proceedings, and legally resembles a license more, than a full copyright transfer. The problem with your copyright form, as I read it, is that ACM receives and retains all rights (which may be not welcome by some journal publishers, and you never know where eventually you're going to try to publish, so why risk?), and also that it does not specify journal articles as acceptable forms of future reusing the paper. In fact, the copyright form is very similar to the forms used by journals (many of which allow publishing the article on a personal page or in future book works, if proper attribution is provided). This also may indicate to some possible attendants that WikiSym conference publication is an alternative to a journal publication. In my field a conference paper does not count as a publication and is usually treated as a way of improving the paper before submitting it to a journal. Thus the possibility that you're going to use the paper almost verbatim in a journal submission is quite real. These differences possibly may result in WikiSym, as a conference and only in some disciplines, being less popular than it deserves. best, dariusz On Thu, Nov 8, 2012 at 6:59 PM, Jodi Schneider jschnei...@pobox.comwrote: Hi Dariusz, This is interesting, because if we can articulate problems in the copyright notice, we may be able to fix them. Currently, for WikiSym, the ACM Publications copyright form for proceedings is used: *http://www.acm.org/publications/CopyReleaseProc-9-12.pdf* This includes: * The right to reuse any portion of the Work, without fee, in future works of the Author's (or Author's Employer's) own, including books, lectures and presentations in all media, provided that the ACM citation, notice of the Copyright and the ACM DOI are included ... * The right to revise the work. What is a typical copyright notice for conferences you *do* publish in? What rights do you need? In computer science, typically a journal publication arising in part from a conference paper would include new results, or several papers would be combined. -Jodi On Thu, Nov 8, 2012 at 5:23 PM, Dariusz Jemielniak dar...@alk.edu.plwrote: This is why conferences such as WikiSym are not very attractive for my field, as they require some copyright transfer (which may effectively make publishing in the final destination difficult). -- __ dr hab. Dariusz Jemielniak profesor zarządzania kierownik katedry Zarządzania Międzynarodowego i centrum badawczego CROW Akademia Leona Koźmińskiego http://www.crow.alk.edu.pl ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal? - Conference vs. journal publication
Jodi, You have a good point about the ease of changing WikiSym and opening a new track, but I wonder why an author might choose to submit a working paper and pay for travel to a conference if they'll end up submitting the final product to a place where they receive credit anyway. Since WikiSym is intended to be a terminal publication venue, I'd like that all who submit (and are accepted) get the appropriate amount of credit for their work, but I understand that it would be naive to expect the world to be able/willing to quickly change. -Aaron On Wed, Nov 21, 2012 at 10:00 AM, Jodi Schneider jschnei...@pobox.comwrote: Hi Aaron, I think Dariusz' suggestion is that we add a conference track for non-archival working papers from social scientists. It is much faster for WikiSym to change than for the recognition environment of all social scientists to change. Even if we mark papers as archival the 'conference' label may still be a downside. What do you think? -Jodi On Wed, Nov 21, 2012 at 2:38 PM, Aaron Halfaker aaron.halfa...@gmail.comwrote: Can we not just label which of our accepted papers are archival? It seems that some disciplines assume Journal == Archival and Conference != Archival. This is apparently inaccurate in other disciplines so there must be some reason we don't just note which papers are archived and which are not. I don't see it though. -Aaron On Wed, Nov 21, 2012 at 8:00 AM, Dariusz Jemielniak dar...@alk.edu.plwrote: I think it basically is a different publication model. You probably could have two track (one for final, the other for working papers) so as to address the disciplines which rely on journals as final outlets. Best, Dj 20 lis 2012 20:26, Jodi Schneider jschnei...@pobox.com napisał(a): Hi Dariusz, For reusing the paper unchanged this is indeed a problem. Journal could be added to the list of mentioned reuse venues--but this still wouldn't imply that the entirety of the paper could be used without change, I suspect. For ACM conferences, there are two types of papers: - archival - non-archival Archival papers are published in the ACM digital library, in the conference proceedings, and are considered final research products to be cited. Non-archival papers are not considered final research products and (as far as I know) don't require copyright transfer. Aside from making papers destined to be used VERBATIM without change in journals 'non-archival', how could this be addressed? -Jodi On Thu, Nov 8, 2012 at 6:16 PM, Dariusz Jemielniak dar...@alk.edu.plwrote: hi Jodi, the conferences I attend or follow (e.g. EGOS, AoM, APROS. SFAA) afaik do not typically require signed copyright notices at all, and if they do, the copyright is granted specifically for publishing in the proceedings, and legally resembles a license more, than a full copyright transfer. The problem with your copyright form, as I read it, is that ACM receives and retains all rights (which may be not welcome by some journal publishers, and you never know where eventually you're going to try to publish, so why risk?), and also that it does not specify journal articles as acceptable forms of future reusing the paper. In fact, the copyright form is very similar to the forms used by journals (many of which allow publishing the article on a personal page or in future book works, if proper attribution is provided). This also may indicate to some possible attendants that WikiSym conference publication is an alternative to a journal publication. In my field a conference paper does not count as a publication and is usually treated as a way of improving the paper before submitting it to a journal. Thus the possibility that you're going to use the paper almost verbatim in a journal submission is quite real. These differences possibly may result in WikiSym, as a conference and only in some disciplines, being less popular than it deserves. best, dariusz On Thu, Nov 8, 2012 at 6:59 PM, Jodi Schneider jschnei...@pobox.comwrote: Hi Dariusz, This is interesting, because if we can articulate problems in the copyright notice, we may be able to fix them. Currently, for WikiSym, the ACM Publications copyright form for proceedings is used: *http://www.acm.org/publications/CopyReleaseProc-9-12.pdf* This includes: * The right to reuse any portion of the Work, without fee, in future works of the Author's (or Author's Employer's) own, including books, lectures and presentations in all media, provided that the ACM citation, notice of the Copyright and the ACM DOI are included ... * The right to revise the work. What is a typical copyright notice for conferences you *do* publish in? What rights do you need? In computer science, typically a journal publication arising in part from a conference paper would include new results, or several papers would be combined. -Jodi On Thu, Nov 8, 2012 at 5:23 PM, Dariusz
Re: [Wiki-research-l] Wiki Research Journal? - Conference vs. journal publication
Hi Dariusz, For reusing the paper unchanged this is indeed a problem. Journal could be added to the list of mentioned reuse venues--but this still wouldn't imply that the entirety of the paper could be used without change, I suspect. For ACM conferences, there are two types of papers: - archival - non-archival Archival papers are published in the ACM digital library, in the conference proceedings, and are considered final research products to be cited. Non-archival papers are not considered final research products and (as far as I know) don't require copyright transfer. Aside from making papers destined to be used VERBATIM without change in journals 'non-archival', how could this be addressed? -Jodi On Thu, Nov 8, 2012 at 6:16 PM, Dariusz Jemielniak dar...@alk.edu.plwrote: hi Jodi, the conferences I attend or follow (e.g. EGOS, AoM, APROS. SFAA) afaik do not typically require signed copyright notices at all, and if they do, the copyright is granted specifically for publishing in the proceedings, and legally resembles a license more, than a full copyright transfer. The problem with your copyright form, as I read it, is that ACM receives and retains all rights (which may be not welcome by some journal publishers, and you never know where eventually you're going to try to publish, so why risk?), and also that it does not specify journal articles as acceptable forms of future reusing the paper. In fact, the copyright form is very similar to the forms used by journals (many of which allow publishing the article on a personal page or in future book works, if proper attribution is provided). This also may indicate to some possible attendants that WikiSym conference publication is an alternative to a journal publication. In my field a conference paper does not count as a publication and is usually treated as a way of improving the paper before submitting it to a journal. Thus the possibility that you're going to use the paper almost verbatim in a journal submission is quite real. These differences possibly may result in WikiSym, as a conference and only in some disciplines, being less popular than it deserves. best, dariusz On Thu, Nov 8, 2012 at 6:59 PM, Jodi Schneider jschnei...@pobox.comwrote: Hi Dariusz, This is interesting, because if we can articulate problems in the copyright notice, we may be able to fix them. Currently, for WikiSym, the ACM Publications copyright form for proceedings is used: *http://www.acm.org/publications/CopyReleaseProc-9-12.pdf* This includes: * The right to reuse any portion of the Work, without fee, in future works of the Author's (or Author's Employer's) own, including books, lectures and presentations in all media, provided that the ACM citation, notice of the Copyright and the ACM DOI are included ... * The right to revise the work. What is a typical copyright notice for conferences you *do* publish in? What rights do you need? In computer science, typically a journal publication arising in part from a conference paper would include new results, or several papers would be combined. -Jodi On Thu, Nov 8, 2012 at 5:23 PM, Dariusz Jemielniak dar...@alk.edu.plwrote: This is why conferences such as WikiSym are not very attractive for my field, as they require some copyright transfer (which may effectively make publishing in the final destination difficult). -- __ dr hab. Dariusz Jemielniak profesor zarządzania kierownik katedry Zarządzania Międzynarodowego i centrum badawczego CROW Akademia Leona Koźmińskiego http://www.crow.alk.edu.pl ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal? - Why?
My answer is the fact that many of us are reading this mailing list, reading papers in various draft and final forms that people are writing, discussing the topic, etc. I see a community forming here. A journal would seem a natural evolution of that. I dont think the editorial team has to be expert in everything in itself; it might need to be able to find reviewers in everything though. Kerry _ From: wiki-research-l-boun...@lists.wikimedia.org [mailto:wiki-research-l-boun...@lists.wikimedia.org] On Behalf Of Brian Keegan Sent: Friday, 9 November 2012 8:35 AM To: Research into Wikimedia content and communities Subject: Re: [Wiki-research-l] Wiki Research Journal? - Why? I keep coming back to this same question Aaron's raised as well. Wiki is obviously the glue holding everything thematically as well as logistically together in the proposals I've seen here-to-for, but it seems nigh-impossible to assemble an editorial board that is simultaneously open and qualified to reviewing submissions that almost certainly cover the gamut from journalism and media studies, computer and information sciences, complex and network sciences, sociology and organizational behavior, business and economics, legal and policy studies, education and outreach. Any single issue risks incoherence including articles across all these fields and the possibility of having rotating special issues dedicated to any single domain for this Wiki-journal to ensure some coherence would seem to suggest simply organizing a special issue in pre-existing journals. It comes down to this: someone needs to clearly articulate why active wiki-researchers like myself should take the risk of publishing our research in a new journal when we potentially have higher-impact journals and better-tailored special issues as alternative and ready outlets. On Thu, Nov 8, 2012 at 5:19 PM, Aaron Halfaker aaron.halfa...@gmail.com wrote: So, if I can re-ignite and re-frame the original question I posed, * Why do we need a wiki journal if there are already high impact journals that are receptive to high quality wiki studies? -Aaron On Thu, Nov 8, 2012 at 4:11 PM, Manuel Palomo Duarte manuel.pal...@uca.es wrote: Nice post, Kerry. Let me add that the citation rates are calculated using the cites in reputated journals already indexed ... 2012/11/8 Kerry Raymond kerry.raym...@gmail.com Actually the reputation of journals is usually derived from its impact factor https://en.wikipedia.org/wiki/Impact_factor which is all about citation rates rather than acceptance/rejection rates. Acceptance rates are sometimes used for newer journals as citation rates arent available. But it doesnt follow that a new journal must reject reasonable papers in order to achieve some desired acceptance rate. A new journal (properly advertised) will probably attract a lot of papers that have been rejected elsewhere so you probably end up with plenty of worthy-of-rejection material. There is no way to get an immediate great reputation for a new journal. But I think a clear focus on topic, a hard-working international editorial team, and a firm but fair reviewing process and reviewers will yield good-quality papers and will attract more good quality papers in response Kerry _ From: wiki-research-l-boun...@lists.wikimedia.org [mailto:wiki-research-l-boun...@lists.wikimedia.org] On Behalf Of Aaron Halfaker Sent: Friday, 9 November 2012 1:51 AM To: Research into Wikimedia content and communities Subject: Re: [Wiki-research-l] Wiki Research Journal? - Why? Highly rated is an interesting property. One of the ways that a publication venue becomes highly rated is by being highly restrictive. In fact, the primary measurement of the quality of a publication venue is the acceptance rate of that conference. WikiSym is not considered highly rated because a high proportion of the submitted papers are accepted. Would a wiki journal be more restrictive in order to gain a highly rated status? I think it's interesting to ask why WikiSym needs improvement and why attendance has been falling. If a WikiSym is a wiki conference that is struggling to maintain participation, how might a wiki journal surmount such trouble? Assuming that the answer to my question above is yes, the wiki-journal would be more restrictive, how would such a journal gather more submissions than an established conference like WikiSym -- enough to both produce regular issues and maintain a high rejection rate? -Aaron On Thu, Nov 8, 2012 at 9:12 AM, Joe Corneli holtzerman...@gmail.com wrote: On Thu, Nov 8, 2012 at 3:02 PM, Aaron Halfaker aaron.halfa...@gmail.com wrote: To state it plainly, why do we need yet another publication venue specific to wiki software? I think people want a highly rated publication venue. Also, «The reason why WikiSym is changing is for the same reason. People are not going to the conference
Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review
Even more, you can easily identify the authors because usually they include references to their previous publications to build the new hypothesis ... 2012/11/8 Adam Jenkins adam.jenk...@gmail.com Most of my reviewing for conference and journals was double blind, although the effectiveness of it was always a bit questionable, as in many cases you can, as a reviewer, identify the author from the style and argument. This tends to get worse in specialised areas, where the pool of researchers is necessarily small. Adam On 7 November 2012 06:16, Kerry Raymond kerry.raym...@gmail.com wrote: I cannot speak for other disciplines but double-blind is not standard in Computer Science. ** ** Kerry ** ** ** ** ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPIFM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139 Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede contener información confidencial, dirigida a un destinatario y objetivo específico. Si usted no es el destinatario del mismo le pido disculpas, y le pido que elimine este correo, evitando cualquier divulgación, copia o distribución de su contenido, así como desarrollar o ejecutar cualquier acción basada en el mismo. -- Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and objective. In case you are not the addressee of the same, I apologize. And I ask you to delete this mail, and not to resend, copy or distribute its content, as well as develop or execute any action based on the same. ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review
I don't thnk opening peer reviewing would be a good idea. Reviewer must keep unknown, or she could suffer preasures (even bribes) from authors. In my opinion only the editor must communicate with the reviewers 2012/11/8 koltzenb...@w4w.net agree, ... so it is up to you as a reviewer what you do with your blindness :-) doesn't this point in the direction of - plus - is + ? I mean: why not do open peer reviewing? Claudia On Thu, 8 Nov 2012 09:43:39 +0100, Manuel Palomo Duarte wrote Even more, you can easily identify the authors because usually they include references to their previous publications to build the new hypothesis ... 2012/11/8 Adam Jenkins adam.jenk...@gmail.com Most of my reviewing for conference and journals was double blind, although the effectiveness of it was always a bit questionable, as in many cases you can, as a reviewer, identify the author from the style and argument. This tends to get worse in specialised areas, where the pool of researchers is necessarily small. Adam On 7 November 2012 06:16, Kerry Raymond kerry.raym...@gmail.com wrote: I cannot speak for other disciplines but double-blind is not standard in Computer Science. ** ** Kerry ** ** ** ** ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPIFM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139 Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede contener información confidencial, dirigida a un destinatario y objetivo específico. Si usted no es el destinatario del mismo le pido disculpas, y le pido que elimine este correo, evitando cualquier divulgación, copia o distribución de su contenido, así como desarrollar o ejecutar cualquier acción basada en el mismo. -- Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and objective. In case you are not the addressee of the same, I apologize. And I ask you to delete this mail, and not to resend, copy or distribute its content, as well as develop or execute any action based on the same. thanks cheers, Claudia koltzenb...@w4w.net ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPIFM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139 Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede contener información confidencial, dirigida a un destinatario y objetivo específico. Si usted no es el destinatario del mismo le pido disculpas, y le pido que elimine este correo, evitando cualquier divulgación, copia o distribución de su contenido, así como desarrollar o ejecutar cualquier acción basada en el mismo. -- Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and objective. In case you are not the addressee of the same, I apologize. And I ask you to delete this mail, and not to resend, copy or distribute its content, as well as develop or execute any action based on the same. ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review
I don't agree. I a hard argument can be considered by some people as a preasure, while other could not. In fact, what's the gain in knowing who is reviewing a paper? 2012/11/8 koltzenb...@w4w.net well, any attempts at pressures or bribes could easily be made known, couldn't they? On Thu, 8 Nov 2012 09:56:35 +0100, Manuel Palomo Duarte wrote I don't thnk opening peer reviewing would be a good idea. Reviewer must keep unknown, or she could suffer preasures (even bribes) from authors. In my opinion only the editor must communicate with the reviewers 2012/11/8 koltzenb...@w4w.net agree, ... so it is up to you as a reviewer what you do with your blindness :-) doesn't this point in the direction of - plus - is + ? I mean: why not do open peer reviewing? Claudia On Thu, 8 Nov 2012 09:43:39 +0100, Manuel Palomo Duarte wrote Even more, you can easily identify the authors because usually they include references to their previous publications to build the new hypothesis ... 2012/11/8 Adam Jenkins adam.jenk...@gmail.com Most of my reviewing for conference and journals was double blind, although the effectiveness of it was always a bit questionable, as in many cases you can, as a reviewer, identify the author from the style and argument. This tends to get worse in specialised areas, where the pool of researchers is necessarily small. Adam On 7 November 2012 06:16, Kerry Raymond kerry.raym...@gmail.com wrote: I cannot speak for other disciplines but double-blind is not standard in Computer Science. ** ** Kerry ** ** ** ** ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPIFM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139 Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede contener información confidencial, dirigida a un destinatario y objetivo específico. Si usted no es el destinatario del mismo le pido disculpas, y le pido que elimine este correo, evitando cualquier divulgación, copia o distribución de su contenido, así como desarrollar o ejecutar cualquier acción basada en el mismo. -- Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and objective. In case you are not the addressee of the same, I apologize. And I ask you to delete this mail, and not to resend, copy or distribute its content, as well as develop or execute any action based on the same. thanks cheers, Claudia koltzenb...@w4w.net ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPIFM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139 Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede contener información confidencial, dirigida a un destinatario y objetivo específico. Si usted no es el destinatario del mismo le pido disculpas, y le pido que elimine este correo, evitando cualquier divulgación, copia o distribución de su contenido, así como desarrollar o ejecutar cualquier acción basada en el mismo. -- Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and objective. In case you are not the addressee of the same, I apologize. And I ask you to delete this mail, and not to resend, copy or distribute its content, as well as develop or execute any action based on the same. thanks cheers, Claudia koltzenb...@w4w.net ___
Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review
Manuel asks: In fact, what's the gain in knowing who is reviewing a paper? let us look at this from another angle, maybe: As reviewers in open reviewing we get a chance of becoming more aware of our own inclinations in the face of public visibility vis-a-vis objectivity, well-reflected arguments and more transparency in general. Q: Why should authors of research have to bow to any authority that is hiding its identity and tendencies? actually, so far I have heard no convincing arguments why in the age of open Wikis any reviewer's identity should stay behind closed doors. Maybe you have another argument to convince me? Claudia On Thu, 8 Nov 2012 10:29:25 +0100, Manuel Palomo Duarte wrote I don't agree. I a hard argument can be considered by some people as a preasure, while other could not. In fact, what's the gain in knowing who is reviewing a paper? 2012/11/8 koltzenb...@w4w.net well, any attempts at pressures or bribes could easily be made known, couldn't they? On Thu, 8 Nov 2012 09:56:35 +0100, Manuel Palomo Duarte wrote I don't thnk opening peer reviewing would be a good idea. Reviewer must keep unknown, or she could suffer preasures (even bribes) from authors. In my opinion only the editor must communicate with the reviewers 2012/11/8 koltzenb...@w4w.net agree, ... so it is up to you as a reviewer what you do with your blindness :-) doesn't this point in the direction of - plus - is + ? I mean: why not do open peer reviewing? Claudia On Thu, 8 Nov 2012 09:43:39 +0100, Manuel Palomo Duarte wrote Even more, you can easily identify the authors because usually they include references to their previous publications to build the new hypothesis ... 2012/11/8 Adam Jenkins adam.jenk...@gmail.com Most of my reviewing for conference and journals was double blind, although the effectiveness of it was always a bit questionable, as in many cases you can, as a reviewer, identify the author from the style and argument. This tends to get worse in specialised areas, where the pool of researchers is necessarily small. Adam On 7 November 2012 06:16, Kerry Raymond kerry.raym...@gmail.com wrote: I cannot speak for other disciplines but double-blind is not standard in Computer Science. ** ** Kerry ** ** ** ** ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPIFM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139 Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede contener información confidencial, dirigida a un destinatario y objetivo específico. Si usted no es el destinatario del mismo le pido disculpas, y le pido que elimine este correo, evitando cualquier divulgación, copia o distribución de su contenido, así como desarrollar o ejecutar cualquier acción basada en el mismo. -- Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and objective. In case you are not the addressee of the same, I apologize. And I ask you to delete this mail, and not to resend, copy or distribute its content, as well as develop or execute any action based on the same. thanks cheers, Claudia koltzenb...@w4w.net ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPIFM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139 Aviso legal: Este
Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review
no. Also, academic world may be quite small in some disciplines. If a reviewer knows that s/he may be evaluated by the author some time in the future (be it in a journal review, or possibly also in career promotion reviews, too) s/he will be definitely motivated not to report any major flaws, especially if the reviewed author is a big shot. Single blind review is a must. In my view, the advantages of double blind review are also important, even if practically the actual anonymity may not always be preserved. In double blind review the bottom line is that it sometimes may and up as a single blind review, but in many cases it does not and then it is more fair. best, dj On Thu, Nov 8, 2012 at 10:08 AM, koltzenb...@w4w.net wrote: well, any attempts at pressures or bribes could easily be made known, couldn't they? On Thu, 8 Nov 2012 09:56:35 +0100, Manuel Palomo Duarte wrote I don't thnk opening peer reviewing would be a good idea. Reviewer must keep unknown, or she could suffer preasures (even bribes) from authors. In my opinion only the editor must communicate with the reviewers 2012/11/8 koltzenb...@w4w.net agree, ... so it is up to you as a reviewer what you do with your blindness :-) doesn't this point in the direction of - plus - is + ? I mean: why not do open peer reviewing? Claudia On Thu, 8 Nov 2012 09:43:39 +0100, Manuel Palomo Duarte wrote Even more, you can easily identify the authors because usually they include references to their previous publications to build the new hypothesis ... 2012/11/8 Adam Jenkins adam.jenk...@gmail.com Most of my reviewing for conference and journals was double blind, although the effectiveness of it was always a bit questionable, as in many cases you can, as a reviewer, identify the author from the style and argument. This tends to get worse in specialised areas, where the pool of researchers is necessarily small. Adam On 7 November 2012 06:16, Kerry Raymond kerry.raym...@gmail.com wrote: I cannot speak for other disciplines but double-blind is not standard in Computer Science. ** ** Kerry ** ** ** ** ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPIFM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139 Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede contener información confidencial, dirigida a un destinatario y objetivo específico. Si usted no es el destinatario del mismo le pido disculpas, y le pido que elimine este correo, evitando cualquier divulgación, copia o distribución de su contenido, así como desarrollar o ejecutar cualquier acción basada en el mismo. -- Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and objective. In case you are not the addressee of the same, I apologize. And I ask you to delete this mail, and not to resend, copy or distribute its content, as well as develop or execute any action based on the same. thanks cheers, Claudia koltzenb...@w4w.net ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPIFM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139 Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede contener información confidencial, dirigida a un destinatario y objetivo específico. Si usted no es el destinatario del mismo le pido disculpas, y le pido que elimine este correo, evitando cualquier divulgación, copia o distribución de su contenido, así
Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review
hm, sadly enough I must agree that you seem to be raising important real-life points, Dariusz. But am I getting you correctly that you think that major flaws will only be pointed out in a review if the reviewer can officially stay anonymous? in your experience, Dariusz, does this mean reviewers feel fine in placing tons of trust in the editors and their helphands who organize the review not to tell authors who was their most brilliant reviewer? anyway, I think we need a reviewing system where fair and open criticism can flourish. in my view of the matter, there will be no one-size-fits-all because self-organized communities do have a multicultural tendency to self-organize :-) one example system that we night also discuss and try out for the Wiki Research Journal is a combination of open and closed peer review: see ACP who, in a highly specialised community, do 8 weeks of post publication public discussion http://www.atmospheric-chemistry-and- physics.net/review/review_process_and_interactive_public_discussion.html let me suggest we now go to the wiki page http://meta.wikimedia.org/wiki/Wiki_Research_Ideas and continue our debate on the peer review model in a pro/contra/undecided style there and keep in mind that we are not talking about a traditional journal here but about a new research journal about Wikis and about research done by using Wikis http://meta.wikimedia.org/wiki/Wiki_Research_Ideas/Volunteers see you, Claudia koltzenb...@w4w.net On Thu, 8 Nov 2012 10:44:13 +0100, Dariusz Jemielniak wrote no. Also, academic world may be quite small in some disciplines. If a reviewer knows that s/he may be evaluated by the author some time in the future (be it in a journal review, or possibly also in career promotion reviews, too) s/he will be definitely motivated not to report any major flaws, especially if the reviewed author is a big shot. Single blind review is a must. In my view, the advantages of double blind review are also important, even if practically the actual anonymity may not always be preserved. In double blind review the bottom line is that it sometimes may and up as a single blind review, but in many cases it does not and then it is more fair. best, dj On Thu, Nov 8, 2012 at 10:08 AM, koltzenb...@w4w.net wrote: well, any attempts at pressures or bribes could easily be made known, couldn't they? On Thu, 8 Nov 2012 09:56:35 +0100, Manuel Palomo Duarte wrote I don't thnk opening peer reviewing would be a good idea. Reviewer must keep unknown, or she could suffer preasures (even bribes) from authors. In my opinion only the editor must communicate with the reviewers 2012/11/8 koltzenb...@w4w.net agree, ... so it is up to you as a reviewer what you do with your blindness :-) doesn't this point in the direction of - plus - is + ? I mean: why not do open peer reviewing? Claudia On Thu, 8 Nov 2012 09:43:39 +0100, Manuel Palomo Duarte wrote Even more, you can easily identify the authors because usually they include references to their previous publications to build the new hypothesis ... 2012/11/8 Adam Jenkins adam.jenk...@gmail.com Most of my reviewing for conference and journals was double blind, although the effectiveness of it was always a bit questionable, as in many cases you can, as a reviewer, identify the author from the style and argument. This tends to get worse in specialised areas, where the pool of researchers is necessarily small. Adam On 7 November 2012 06:16, Kerry Raymond kerry.raym...@gmail.com wrote: I cannot speak for other disciplines but double-blind is not standard in Computer Science. ** ** Kerry ** ** ** ** ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPIFM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139 Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede contener información confidencial, dirigida a un destinatario y objetivo específico.
Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review
hi, On Thu, Nov 8, 2012 at 11:17 AM, koltzenb...@w4w.net wrote: hm, sadly enough I must agree that you seem to be raising important real-life points, Dariusz. But am I getting you correctly that you think that major flaws will only be pointed out in a review if the reviewer can officially stay anonymous? well, not only officially, but also practically. It is an important ethical responsibility of the managing editor to ensure anonymity of the reviewers, so that they can be honest. in your experience, Dariusz, does this mean reviewers feel fine in placing tons of trust in the editors and their helphands who organize the review not to tell authors who was their most brilliant reviewer? Yes, that is my experience. In fact, I have never seen the editor revealing the reviewer's identity. I have heard of one such instance, when the author discovered the reviewer's identity simply because of the high praises the reviewer was making for his work, and because of pushing the reviewer's works as suggested needed literature, but the only reaction from the editor was matter-of-factly allowing not to incorporate these suggestions from the review and excluding the reviewer from further process. and keep in mind that we are not talking about a traditional journal here but about a new research journal about Wikis and about research done by using Wikis that's true and experimenting with the format is a good idea! I think, for example, that publishing reviews and responses to them, and allowing commenting on them is a good idea. I'm quite convinced though that the anonymity of reviewers helps. Of course, it can be probably be also played with and tested. best, dj ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review
If the question is only how to set up a journal then I wonder if this should be taking place off-list, since that's not really a wiki research question. If it is a question about how to set up a journal that specifically meshes with the socio-technical patterns used by wiki communities, then of course it is appropriate for discussion here. (And obviously I think it's the latter!) the question may also be how to set up a journal relevant for research specific for wiki-communities, that stands a chance of becoming the leading journal (ranked, listed, prestigious, etc.) in some related fields and then questions on which traditional academic practices of a journal are necessary, and which are optional, obviously is both important, and of interest for this list. Just saying. dj ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
[Wiki-research-l] Wiki Research Journal? - Why?
User:Arided added the following to http://meta.wikimedia.org/wiki/Wiki_Research_Ideas The field of wiki studies exists but there is no dedicated journal. This is a problem to be solved. There is an academic/industry wiki studies conference called WikiSym. Also, there is Wikimania, a more wiki-like, less academic conference for wiki studies and technologies. Why do we also need a wiki journal? What needs would such a journal fulfill that the conferences do not? To state it plainly, why do we need yet another publication venue specific to wiki software? -Aaron ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal? - Why?
hi Aaron, I think that the rejection-rate principle does not apply to the highly rated criterion for journals, when JCR/ISI (the only ranking that matters at present) criteria are considered. The key and predominant criterion is the number of citations in the journals, which are already in the ranking. Keep in mind that in some disciplines conference paper do not matter AT ALL (they are not counted as anything in career advancement). One source of competitive advantage of a wiki-centered journal is its specialized focus. Both writers and readers on wiki-phenomena are likely to consider a wiki-specialized journal as a good venue of publishing/reading. Also, with our community as a driving force, it is conceivable that the journal would have a relatively high readership (and consequently, citation numbers). best, dj On Thu, Nov 8, 2012 at 4:51 PM, Aaron Halfaker aaron.halfa...@gmail.comwrote: Highly rated is an interesting property. One of the ways that a publication venue becomes highly rated is by being highly restrictive. In fact, the primary measurement of the quality of a publication venue is the acceptance rate of that conference. WikiSym is not considered highly rated because a high proportion of the submitted papers are accepted. Would a wiki journal be more restrictive in order to gain a highly rated status? I think it's interesting to ask why WikiSym needs improvement and why attendance has been falling. If a WikiSym is a wiki conference that is struggling to maintain participation, how might a wiki journal surmount such trouble? Assuming that the answer to my question above is yes, the wiki-journal would be more restrictive, how would such a journal gather more submissions than an established conference like WikiSym -- enough to both produce regular issues and maintain a high rejection rate? -Aaron On Thu, Nov 8, 2012 at 9:12 AM, Joe Corneli holtzerman...@gmail.comwrote: On Thu, Nov 8, 2012 at 3:02 PM, Aaron Halfaker aaron.halfa...@gmail.com wrote: To state it plainly, why do we need yet another publication venue specific to wiki software? I think people want a highly rated publication venue. Also, The reason why WikiSym is changing is for the same reason. People are not going to the conference! I think the attendance has been below 100 for some time now. That's not a sustainable number for the amount of work that goes into organizing a conference. But what you're saying suggests that maybe work should be done to improve existing venues rather than creating a new one. ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- __ dr hab. Dariusz Jemielniak profesor zarządzania kierownik katedry Zarządzania Międzynarodowego i centrum badawczego CROW Akademia Leona Koźmińskiego http://www.crow.alk.edu.pl ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal? - Why?
Dariusz, you make a good point about the criterion for ranking journals, but my point still stands that you wnn't have a high quality set of papers without strict criteria for rejection. I've reviewed enough papers to know what tends to get rejected. I don't see how a such a specialized focus as beneficial or our community as a particularly strong force for driving citations. Surely WikiSym has an equally specialized focus and the same community behind it. As for disciplines that do not count conference papers, I cannot comment because my discipline (Computer Science) looks at top tier conference publications in a similar way to journal publications. However, I'd argue that anyone who does not value a publication purely because the venue is called a conference regardless of the impact/restrictiveness is making a mistake. I've seen people include the acceptance rates on their CV to avoid this situation. -Aaron On Thu, Nov 8, 2012 at 10:04 AM, Dariusz Jemielniak dar...@alk.edu.plwrote: hi Aaron, I think that the rejection-rate principle does not apply to the highly rated criterion for journals, when JCR/ISI (the only ranking that matters at present) criteria are considered. The key and predominant criterion is the number of citations in the journals, which are already in the ranking. Keep in mind that in some disciplines conference paper do not matter AT ALL (they are not counted as anything in career advancement). One source of competitive advantage of a wiki-centered journal is its specialized focus. Both writers and readers on wiki-phenomena are likely to consider a wiki-specialized journal as a good venue of publishing/reading. Also, with our community as a driving force, it is conceivable that the journal would have a relatively high readership (and consequently, citation numbers). best, dj On Thu, Nov 8, 2012 at 4:51 PM, Aaron Halfaker aaron.halfa...@gmail.comwrote: Highly rated is an interesting property. One of the ways that a publication venue becomes highly rated is by being highly restrictive. In fact, the primary measurement of the quality of a publication venue is the acceptance rate of that conference. WikiSym is not considered highly rated because a high proportion of the submitted papers are accepted. Would a wiki journal be more restrictive in order to gain a highly rated status? I think it's interesting to ask why WikiSym needs improvement and why attendance has been falling. If a WikiSym is a wiki conference that is struggling to maintain participation, how might a wiki journal surmount such trouble? Assuming that the answer to my question above is yes, the wiki-journal would be more restrictive, how would such a journal gather more submissions than an established conference like WikiSym -- enough to both produce regular issues and maintain a high rejection rate? -Aaron On Thu, Nov 8, 2012 at 9:12 AM, Joe Corneli holtzerman...@gmail.comwrote: On Thu, Nov 8, 2012 at 3:02 PM, Aaron Halfaker aaron.halfa...@gmail.com wrote: To state it plainly, why do we need yet another publication venue specific to wiki software? I think people want a highly rated publication venue. Also, The reason why WikiSym is changing is for the same reason. People are not going to the conference! I think the attendance has been below 100 for some time now. That's not a sustainable number for the amount of work that goes into organizing a conference. But what you're saying suggests that maybe work should be done to improve existing venues rather than creating a new one. ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- __ dr hab. Dariusz Jemielniak profesor zarządzania kierownik katedry Zarządzania Międzynarodowego i centrum badawczego CROW Akademia Leona Koźmińskiego http://www.crow.alk.edu.pl ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal? - Conference vs. journal publication
I can't speak for every field, but at least for my own field of information systems, where conferences count for zero, at least among the most research-intensive universities: Counting conference publications or not is in no way a judgment either way of the quality of the papers. In information systems, it is well known that some high-quality conferences (such as ICIS, HICSS and AoM-OCIS) regularly field higher quality papers than many journals. However, such publication often counted as zero in promotion and tenure considerations. What is going on is that in our field (and I suspect also in similar fields) *conferences are not considered terminal publication outlets--only journals are*. That is, when you present a paper in a conference, even when it is published in the proceedings, you are expected to later publish a significantly revised and significantly extended version of that paper in a journal article (and I would guess that in 90% of the time, this is what happens, at least for high-quality papers). A high-quality conference paper is expected to yield a high-quality journal article. Thus, *to avoid double-counting*, conference publications are ignored in promotion and tenure considerations. From what I understand, in fields like computer science where conferences are terminal publication outlets (that is, conference papers are often not republished in journals), then it naturally makes sense that the conference papers should be considered the measure of a researcher's productive quality. ~ Chitu Aaron Halfaker a écrit : As for disciplines that do not count conference papers, I cannot comment because my discipline (Computer Science) looks at top tier conference publications in a similar way to journal publications. However, I'd argue that anyone who does not value a publication purely because the venue is called a "conference" regardless of the impact/restrictiveness is making a mistake. I've seen people include the acceptance rates on their CV to avoid this situation. -Aaron ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal? - Conference vs. journal publication
hi Jodi, the conferences I attend or follow (e.g. EGOS, AoM, APROS. SFAA) afaik do not typically require signed copyright notices at all, and if they do, the copyright is granted specifically for publishing in the proceedings, and legally resembles a license more, than a full copyright transfer. The problem with your copyright form, as I read it, is that ACM receives and retains all rights (which may be not welcome by some journal publishers, and you never know where eventually you're going to try to publish, so why risk?), and also that it does not specify journal articles as acceptable forms of future reusing the paper. In fact, the copyright form is very similar to the forms used by journals (many of which allow publishing the article on a personal page or in future book works, if proper attribution is provided). This also may indicate to some possible attendants that WikiSym conference publication is an alternative to a journal publication. In my field a conference paper does not count as a publication and is usually treated as a way of improving the paper before submitting it to a journal. Thus the possibility that you're going to use the paper almost verbatim in a journal submission is quite real. These differences possibly may result in WikiSym, as a conference and only in some disciplines, being less popular than it deserves. best, dariusz On Thu, Nov 8, 2012 at 6:59 PM, Jodi Schneider jschnei...@pobox.com wrote: Hi Dariusz, This is interesting, because if we can articulate problems in the copyright notice, we may be able to fix them. Currently, for WikiSym, the ACM Publications copyright form for proceedings is used: *http://www.acm.org/publications/CopyReleaseProc-9-12.pdf* This includes: * The right to reuse any portion of the Work, without fee, in future works of the Author's (or Author's Employer's) own, including books, lectures and presentations in all media, provided that the ACM citation, notice of the Copyright and the ACM DOI are included ... * The right to revise the work. What is a typical copyright notice for conferences you *do* publish in? What rights do you need? In computer science, typically a journal publication arising in part from a conference paper would include new results, or several papers would be combined. -Jodi On Thu, Nov 8, 2012 at 5:23 PM, Dariusz Jemielniak dar...@alk.edu.plwrote: This is why conferences such as WikiSym are not very attractive for my field, as they require some copyright transfer (which may effectively make publishing in the final destination difficult). -- __ dr hab. Dariusz Jemielniak profesor zarządzania kierownik katedry Zarządzania Międzynarodowego i centrum badawczego CROW Akademia Leona Koźmińskiego http://www.crow.alk.edu.pl ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs.single-blind review
I guess the scenario you want to protect against is this. Reviewer is Junior Researcher, the author is a Head of School. Next year Junior Researcher applies for job at that school and doesn't get it or applies for a grant or Kerry -Original Message- From: wiki-research-l-boun...@lists.wikimedia.org [mailto:wiki-research-l-boun...@lists.wikimedia.org] On Behalf Of koltzenb...@w4w.net Sent: Thursday, 8 November 2012 7:41 PM To: Research into Wikimedia content and communities Subject: Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs.single-blind review Manuel asks: In fact, what's the gain in knowing who is reviewing a paper? let us look at this from another angle, maybe: As reviewers in open reviewing we get a chance of becoming more aware of our own inclinations in the face of public visibility vis-a-vis objectivity, well-reflected arguments and more transparency in general. Q: Why should authors of research have to bow to any authority that is hiding its identity and tendencies? actually, so far I have heard no convincing arguments why in the age of open Wikis any reviewer's identity should stay behind closed doors. Maybe you have another argument to convince me? Claudia On Thu, 8 Nov 2012 10:29:25 +0100, Manuel Palomo Duarte wrote I don't agree. I a hard argument can be considered by some people as a preasure, while other could not. In fact, what's the gain in knowing who is reviewing a paper? 2012/11/8 koltzenb...@w4w.net well, any attempts at pressures or bribes could easily be made known, couldn't they? On Thu, 8 Nov 2012 09:56:35 +0100, Manuel Palomo Duarte wrote I don't thnk opening peer reviewing would be a good idea. Reviewer must keep unknown, or she could suffer preasures (even bribes) from authors. In my opinion only the editor must communicate with the reviewers 2012/11/8 koltzenb...@w4w.net agree, ... so it is up to you as a reviewer what you do with your blindness :-) doesn't this point in the direction of - plus - is + ? I mean: why not do open peer reviewing? Claudia On Thu, 8 Nov 2012 09:43:39 +0100, Manuel Palomo Duarte wrote Even more, you can easily identify the authors because usually they include references to their previous publications to build the new hypothesis ... 2012/11/8 Adam Jenkins adam.jenk...@gmail.com Most of my reviewing for conference and journals was double blind, although the effectiveness of it was always a bit questionable, as in many cases you can, as a reviewer, identify the author from the style and argument. This tends to get worse in specialised areas, where the pool of researchers is necessarily small. Adam On 7 November 2012 06:16, Kerry Raymond kerry.raym...@gmail.com wrote: I cannot speak for other disciplines but double-blind is not standard in Computer Science. ** ** Kerry ** ** ** ** ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPIFM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139 Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede contener información confidencial, dirigida a un destinatario y objetivo específico. Si usted no es el destinatario del mismo le pido disculpas, y le pido que elimine este correo, evitando cualquier divulgación, copia o distribución de su contenido, así como desarrollar o ejecutar cualquier acción basada en el mismo. -- Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and objective. In case you are not the addressee of the same, I apologize. And I ask you to delete this mail, and not to resend, copy or distribute its content, as well as develop or execute any action based on the same. thanks cheers, Claudia koltzenb...@w4w.net ___ Wiki-research-l mailing list
Re: [Wiki-research-l] Wiki Research Journal? - Why?
I don't have much time at the moment for a proper response, but I wanted to point you to the Research Index on meta: http://meta.wikimedia.org/wiki/Research I've personally cataloged ongoing experiments in this space and reviewed the work of others. See also http://meta.wikimedia.org/wiki/Category:Projects_reviewed_by_RCom and check the talk pages for discussions. -Aaron On Thu, Nov 8, 2012 at 2:10 PM, Joe Corneli holtzerman...@gmail.com wrote: On Thu, Nov 8, 2012 at 4:38 PM, koltzenb...@w4w.net wrote: Answer 2: articles are not submitted to the journal's editors but written openly on the journals' platform (and then maybe sent to a review process elsewhere as well as opening up to public review here) My answer would be like your Answer 2 above. Let me be clear that what I envision would be more like a research hub than a journal -- but in the end, it would of course include papers that could be cited (and that could be noted down on contributors' CVs). But not all contributions would have to be like that. If we extended the scope quite broadly, it would be like Wikipedia, but without the 'no original research' clause. We'd presumably want some other rule, about focusing on high quality research. I might also go further: Answer 2a: The platform itself could be a target for experiment by contributors. So, while we could start with a standard MediaWiki installation and standard papers, the journal could also review papers plus experiments. The experiment could take place with extensions to the basic MediaWiki installation, or in some other attached wiki. (In mathematics, there's a journal called Experimental Mathematics which captures a similar sort of spirit.) ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal? - Why?
Actually the reputation of journals is usually derived from its impact factor https://en.wikipedia.org/wiki/Impact_factor which is all about citation rates rather than acceptance/rejection rates. Acceptance rates are sometimes used for newer journals as citation rates aren't available. But it doesn't follow that a new journal must reject reasonable papers in order to achieve some desired acceptance rate. A new journal (properly advertised) will probably attract a lot of papers that have been rejected elsewhere so you probably end up with plenty of worthy-of-rejection material. There is no way to get an immediate great reputation for a new journal. But I think a clear focus on topic, a hard-working international editorial team, and a firm but fair reviewing process and reviewers will yield good-quality papers and will attract more good quality papers in response Kerry _ From: wiki-research-l-boun...@lists.wikimedia.org [mailto:wiki-research-l-boun...@lists.wikimedia.org] On Behalf Of Aaron Halfaker Sent: Friday, 9 November 2012 1:51 AM To: Research into Wikimedia content and communities Subject: Re: [Wiki-research-l] Wiki Research Journal? - Why? Highly rated is an interesting property. One of the ways that a publication venue becomes highly rated is by being highly restrictive. In fact, the primary measurement of the quality of a publication venue is the acceptance rate of that conference. WikiSym is not considered highly rated because a high proportion of the submitted papers are accepted. Would a wiki journal be more restrictive in order to gain a highly rated status? I think it's interesting to ask why WikiSym needs improvement and why attendance has been falling. If a WikiSym is a wiki conference that is struggling to maintain participation, how might a wiki journal surmount such trouble? Assuming that the answer to my question above is yes, the wiki-journal would be more restrictive, how would such a journal gather more submissions than an established conference like WikiSym -- enough to both produce regular issues and maintain a high rejection rate? -Aaron On Thu, Nov 8, 2012 at 9:12 AM, Joe Corneli holtzerman...@gmail.com wrote: On Thu, Nov 8, 2012 at 3:02 PM, Aaron Halfaker aaron.halfa...@gmail.com wrote: To state it plainly, why do we need yet another publication venue specific to wiki software? I think people want a highly rated publication venue. Also, The reason why WikiSym is changing is for the same reason. People are not going to the conference! I think the attendance has been below 100 for some time now. That's not a sustainable number for the amount of work that goes into organizing a conference. But what you're saying suggests that maybe work should be done to improve existing venues rather than creating a new one. ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal? - Why?
Nice post, Kerry. Let me add that the citation rates are calculated using the cites in reputated journals already indexed ... 2012/11/8 Kerry Raymond kerry.raym...@gmail.com Actually the reputation of journals is usually derived from its impact factor ** ** https://en.wikipedia.org/wiki/Impact_factor ** ** which is all about citation rates rather than acceptance/rejection rates.* *** ** ** Acceptance rates are sometimes used for newer journals as citation rates aren’t available. But it doesn’t follow that a new journal must reject reasonable papers in order to achieve some desired acceptance rate. A new journal (properly advertised) will probably attract a lot of papers that have been rejected elsewhere so you probably end up with plenty of worthy-of-rejection material. ** ** There is no way to get an immediate “great reputation” for a new journal. But I think a clear focus on topic, a hard-working international editorial team, and a firm but fair reviewing process and reviewers will yield good-quality papers and will attract more good quality papers in response* *** ** ** Kerry ** ** ** ** -- *From:* wiki-research-l-boun...@lists.wikimedia.org [mailto: wiki-research-l-boun...@lists.wikimedia.org] *On Behalf Of *Aaron Halfaker *Sent:* Friday, 9 November 2012 1:51 AM *To:* Research into Wikimedia content and communities *Subject:* Re: [Wiki-research-l] Wiki Research Journal? - Why? ** ** Highly rated is an interesting property. One of the ways that a publication venue becomes highly rated is by being highly restrictive. In fact, the primary measurement of the quality of a publication venue is the acceptance rate of that conference. ** ** WikiSym is not considered highly rated because a high proportion of the submitted papers are accepted. Would a wiki journal be more restrictive in order to gain a highly rated status? ** ** I think it's interesting to ask why WikiSym needs improvement and why attendance has been falling. If a WikiSym is a wiki conference that is struggling to maintain participation, how might a wiki journal surmount such trouble? Assuming that the answer to my question above is yes, the wiki-journal would be more restrictive, how would such a journal gather more submissions than an established conference like WikiSym -- enough to both produce regular issues and maintain a high rejection rate? ** ** -Aaron ** ** On Thu, Nov 8, 2012 at 9:12 AM, Joe Corneli holtzerman...@gmail.com wrote: On Thu, Nov 8, 2012 at 3:02 PM, Aaron Halfaker aaron.halfa...@gmail.com wrote: To state it plainly, why do we need yet another publication venue specific to wiki software? I think people want a highly rated publication venue. Also, «The reason why WikiSym is changing is for the same reason. People are not going to the conference! I think the attendance has been below 100 for some time now. That's not a sustainable number for the amount of work that goes into organizing a conference.» But what you're saying suggests that maybe work should be done to improve existing venues rather than creating a new one. ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ** ** ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPIFM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139 Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede contener información confidencial, dirigida a un destinatario y objetivo específico. Si usted no es el destinatario del mismo le pido disculpas, y le pido que elimine este correo, evitando cualquier divulgación, copia o distribución de su contenido, así como desarrollar o ejecutar cualquier acción basada en el mismo. -- Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and objective. In case you are not the addressee of the same, I apologize. And I ask you to delete this mail, and not to resend, copy or distribute its content, as well as develop or execute any action based on the same. ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal? - Why?
So, if I can re-ignite and re-frame the original question I posed, - Why do we need a wiki journal if there are already high impact journals that are receptive to high quality wiki studies? -Aaron On Thu, Nov 8, 2012 at 4:11 PM, Manuel Palomo Duarte manuel.pal...@uca.eswrote: Nice post, Kerry. Let me add that the citation rates are calculated using the cites in reputated journals already indexed ... 2012/11/8 Kerry Raymond kerry.raym...@gmail.com Actually the reputation of journals is usually derived from its impact factor ** ** https://en.wikipedia.org/wiki/Impact_factor ** ** which is all about citation rates rather than acceptance/rejection rates. ** ** Acceptance rates are sometimes used for newer journals as citation rates aren’t available. But it doesn’t follow that a new journal must reject reasonable papers in order to achieve some desired acceptance rate. A new journal (properly advertised) will probably attract a lot of papers that have been rejected elsewhere so you probably end up with plenty of worthy-of-rejection material. ** ** There is no way to get an immediate “great reputation” for a new journal. But I think a clear focus on topic, a hard-working international editorial team, and a firm but fair reviewing process and reviewers will yield good-quality papers and will attract more good quality papers in response ** ** Kerry ** ** ** ** -- *From:* wiki-research-l-boun...@lists.wikimedia.org [mailto: wiki-research-l-boun...@lists.wikimedia.org] *On Behalf Of *Aaron Halfaker *Sent:* Friday, 9 November 2012 1:51 AM *To:* Research into Wikimedia content and communities *Subject:* Re: [Wiki-research-l] Wiki Research Journal? - Why? ** ** Highly rated is an interesting property. One of the ways that a publication venue becomes highly rated is by being highly restrictive. In fact, the primary measurement of the quality of a publication venue is the acceptance rate of that conference. ** ** WikiSym is not considered highly rated because a high proportion of the submitted papers are accepted. Would a wiki journal be more restrictive in order to gain a highly rated status? ** ** I think it's interesting to ask why WikiSym needs improvement and why attendance has been falling. If a WikiSym is a wiki conference that is struggling to maintain participation, how might a wiki journal surmount such trouble? Assuming that the answer to my question above is yes, the wiki-journal would be more restrictive, how would such a journal gather more submissions than an established conference like WikiSym -- enough to both produce regular issues and maintain a high rejection rate? ** ** -Aaron ** ** On Thu, Nov 8, 2012 at 9:12 AM, Joe Corneli holtzerman...@gmail.com wrote: On Thu, Nov 8, 2012 at 3:02 PM, Aaron Halfaker aaron.halfa...@gmail.com wrote: To state it plainly, why do we need yet another publication venue specific to wiki software? I think people want a highly rated publication venue. Also, «The reason why WikiSym is changing is for the same reason. People are not going to the conference! I think the attendance has been below 100 for some time now. That's not a sustainable number for the amount of work that goes into organizing a conference.» But what you're saying suggests that maybe work should be done to improve existing venues rather than creating a new one. ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ** ** ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPIFM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139 Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede contener información confidencial, dirigida a un destinatario y objetivo específico. Si usted no es el destinatario del mismo le pido disculpas, y le pido que elimine este correo, evitando cualquier divulgación, copia o distribución de su contenido, así como desarrollar o ejecutar cualquier acción basada en el mismo. -- Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and objective. In case you are not the addressee of the same, I apologize. And I ask you to delete this mail, and not to resend, copy
Re: [Wiki-research-l] Wiki Research Journal? - Why?
MHO: only if they don't review wiki studies properly ... 2012/11/8 Aaron Halfaker aaron.halfa...@gmail.com So, if I can re-ignite and re-frame the original question I posed, - Why do we need a wiki journal if there are already high impact journals that are receptive to high quality wiki studies? -Aaron On Thu, Nov 8, 2012 at 4:11 PM, Manuel Palomo Duarte manuel.pal...@uca.es wrote: Nice post, Kerry. Let me add that the citation rates are calculated using the cites in reputated journals already indexed ... 2012/11/8 Kerry Raymond kerry.raym...@gmail.com Actually the reputation of journals is usually derived from its impact factor ** ** https://en.wikipedia.org/wiki/Impact_factor ** ** which is all about citation rates rather than acceptance/rejection rates. ** ** Acceptance rates are sometimes used for newer journals as citation rates aren’t available. But it doesn’t follow that a new journal must reject reasonable papers in order to achieve some desired acceptance rate. A new journal (properly advertised) will probably attract a lot of papers that have been rejected elsewhere so you probably end up with plenty of worthy-of-rejection material. ** ** There is no way to get an immediate “great reputation” for a new journal. But I think a clear focus on topic, a hard-working international editorial team, and a firm but fair reviewing process and reviewers will yield good-quality papers and will attract more good quality papers in response ** ** Kerry ** ** ** ** -- *From:* wiki-research-l-boun...@lists.wikimedia.org [mailto: wiki-research-l-boun...@lists.wikimedia.org] *On Behalf Of *Aaron Halfaker *Sent:* Friday, 9 November 2012 1:51 AM *To:* Research into Wikimedia content and communities *Subject:* Re: [Wiki-research-l] Wiki Research Journal? - Why? ** ** Highly rated is an interesting property. One of the ways that a publication venue becomes highly rated is by being highly restrictive. In fact, the primary measurement of the quality of a publication venue is the acceptance rate of that conference. ** ** WikiSym is not considered highly rated because a high proportion of the submitted papers are accepted. Would a wiki journal be more restrictive in order to gain a highly rated status? ** ** I think it's interesting to ask why WikiSym needs improvement and why attendance has been falling. If a WikiSym is a wiki conference that is struggling to maintain participation, how might a wiki journal surmount such trouble? Assuming that the answer to my question above is yes, the wiki-journal would be more restrictive, how would such a journal gather more submissions than an established conference like WikiSym -- enough to both produce regular issues and maintain a high rejection rate? ** ** -Aaron ** ** On Thu, Nov 8, 2012 at 9:12 AM, Joe Corneli holtzerman...@gmail.com wrote: On Thu, Nov 8, 2012 at 3:02 PM, Aaron Halfaker aaron.halfa...@gmail.com wrote: To state it plainly, why do we need yet another publication venue specific to wiki software? I think people want a highly rated publication venue. Also, «The reason why WikiSym is changing is for the same reason. People are not going to the conference! I think the attendance has been below 100 for some time now. That's not a sustainable number for the amount of work that goes into organizing a conference.» But what you're saying suggests that maybe work should be done to improve existing venues rather than creating a new one. ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ** ** ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPIFM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139 Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede contener información confidencial, dirigida a un destinatario y objetivo específico. Si usted no es el destinatario del mismo le pido disculpas, y le pido que elimine este correo, evitando cualquier divulgación, copia o distribución de su contenido, así como desarrollar o ejecutar cualquier acción basada en el mismo. -- Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and objective. In case you
Re: [Wiki-research-l] Wiki Research Journal? - Why?
I keep coming back to this same question Aaron's raised as well. Wiki is obviously the glue holding everything thematically as well as logistically together in the proposals I've seen here-to-for, but it seems nigh-impossible to assemble an editorial board that is simultaneously open and qualified to reviewing submissions that almost certainly cover the gamut from journalism and media studies, computer and information sciences, complex and network sciences, sociology and organizational behavior, business and economics, legal and policy studies, education and outreach. Any single issue risks incoherence including articles across all these fields and the possibility of having rotating special issues dedicated to any single domain for this Wiki-journal to ensure some coherence would seem to suggest simply organizing a special issue in pre-existing journals. It comes down to this: someone needs to clearly articulate why active wiki-researchers like myself should take the risk of publishing our research in a new journal when we potentially have higher-impact journals and better-tailored special issues as alternative and ready outlets. On Thu, Nov 8, 2012 at 5:19 PM, Aaron Halfaker aaron.halfa...@gmail.comwrote: So, if I can re-ignite and re-frame the original question I posed, - Why do we need a wiki journal if there are already high impact journals that are receptive to high quality wiki studies? -Aaron On Thu, Nov 8, 2012 at 4:11 PM, Manuel Palomo Duarte manuel.pal...@uca.es wrote: Nice post, Kerry. Let me add that the citation rates are calculated using the cites in reputated journals already indexed ... 2012/11/8 Kerry Raymond kerry.raym...@gmail.com Actually the reputation of journals is usually derived from its impact factor ** ** https://en.wikipedia.org/wiki/Impact_factor ** ** which is all about citation rates rather than acceptance/rejection rates. ** ** Acceptance rates are sometimes used for newer journals as citation rates aren’t available. But it doesn’t follow that a new journal must reject reasonable papers in order to achieve some desired acceptance rate. A new journal (properly advertised) will probably attract a lot of papers that have been rejected elsewhere so you probably end up with plenty of worthy-of-rejection material. ** ** There is no way to get an immediate “great reputation” for a new journal. But I think a clear focus on topic, a hard-working international editorial team, and a firm but fair reviewing process and reviewers will yield good-quality papers and will attract more good quality papers in response ** ** Kerry ** ** ** ** -- *From:* wiki-research-l-boun...@lists.wikimedia.org [mailto: wiki-research-l-boun...@lists.wikimedia.org] *On Behalf Of *Aaron Halfaker *Sent:* Friday, 9 November 2012 1:51 AM *To:* Research into Wikimedia content and communities *Subject:* Re: [Wiki-research-l] Wiki Research Journal? - Why? ** ** Highly rated is an interesting property. One of the ways that a publication venue becomes highly rated is by being highly restrictive. In fact, the primary measurement of the quality of a publication venue is the acceptance rate of that conference. ** ** WikiSym is not considered highly rated because a high proportion of the submitted papers are accepted. Would a wiki journal be more restrictive in order to gain a highly rated status? ** ** I think it's interesting to ask why WikiSym needs improvement and why attendance has been falling. If a WikiSym is a wiki conference that is struggling to maintain participation, how might a wiki journal surmount such trouble? Assuming that the answer to my question above is yes, the wiki-journal would be more restrictive, how would such a journal gather more submissions than an established conference like WikiSym -- enough to both produce regular issues and maintain a high rejection rate? ** ** -Aaron ** ** On Thu, Nov 8, 2012 at 9:12 AM, Joe Corneli holtzerman...@gmail.com wrote: On Thu, Nov 8, 2012 at 3:02 PM, Aaron Halfaker aaron.halfa...@gmail.com wrote: To state it plainly, why do we need yet another publication venue specific to wiki software? I think people want a highly rated publication venue. Also, «The reason why WikiSym is changing is for the same reason. People are not going to the conference! I think the attendance has been below 100 for some time now. That's not a sustainable number for the amount of work that goes into organizing a conference.» But what you're saying suggests that maybe work should be done to improve existing venues rather than creating a new one. ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal? - Why?
Some serious deliberation on identity and boundaries is also necessary. WikiSym in recent years has been criticized (fairly in my eyes as an author an PC member) as having significantly shifted from wiki-development and professional implementation to academic (English) Wikipedia studies. Is this just about Wikipedia, or MediaWiki, or any wiki? Will studies using non-wiki open collaboration and peer-production systems like crowdsourcing, citizen science, remixing, FLOSS development, etc. be allowed? There's a thousand slippery slopes absent a clear identity, mission, and goal. And to crucially re-iterate again, what is the competitive advantage of having a journal of wiki-studies when every field from legal studies to complex systems is clamoring to incorporate wiki research to serve their agendas shifting towards social, participatory, open, big approaches? I remain convinced that organizing wiki-scholars to edit special issues, perhaps even incorporating wiki-like processes into the review processes themselves to the extent editorial boards are open to it, will be far more fruitful use of scarce academic time and interest. On Thu, Nov 8, 2012 at 5:34 PM, Brian Keegan bkee...@northwestern.eduwrote: I keep coming back to this same question Aaron's raised as well. Wiki is obviously the glue holding everything thematically as well as logistically together in the proposals I've seen here-to-for, but it seems nigh-impossible to assemble an editorial board that is simultaneously open and qualified to reviewing submissions that almost certainly cover the gamut from journalism and media studies, computer and information sciences, complex and network sciences, sociology and organizational behavior, business and economics, legal and policy studies, education and outreach. Any single issue risks incoherence including articles across all these fields and the possibility of having rotating special issues dedicated to any single domain for this Wiki-journal to ensure some coherence would seem to suggest simply organizing a special issue in pre-existing journals. It comes down to this: someone needs to clearly articulate why active wiki-researchers like myself should take the risk of publishing our research in a new journal when we potentially have higher-impact journals and better-tailored special issues as alternative and ready outlets. On Thu, Nov 8, 2012 at 5:19 PM, Aaron Halfaker aaron.halfa...@gmail.comwrote: So, if I can re-ignite and re-frame the original question I posed, - Why do we need a wiki journal if there are already high impact journals that are receptive to high quality wiki studies? -Aaron On Thu, Nov 8, 2012 at 4:11 PM, Manuel Palomo Duarte manuel.pal...@uca.es wrote: Nice post, Kerry. Let me add that the citation rates are calculated using the cites in reputated journals already indexed ... 2012/11/8 Kerry Raymond kerry.raym...@gmail.com Actually the reputation of journals is usually derived from its impact factor ** ** https://en.wikipedia.org/wiki/Impact_factor ** ** which is all about citation rates rather than acceptance/rejection rates. ** ** Acceptance rates are sometimes used for newer journals as citation rates aren’t available. But it doesn’t follow that a new journal must reject reasonable papers in order to achieve some desired acceptance rate. A new journal (properly advertised) will probably attract a lot of papers that have been rejected elsewhere so you probably end up with plenty of worthy-of-rejection material. ** ** There is no way to get an immediate “great reputation” for a new journal. But I think a clear focus on topic, a hard-working international editorial team, and a firm but fair reviewing process and reviewers will yield good-quality papers and will attract more good quality papers in response ** ** Kerry ** ** ** ** -- *From:* wiki-research-l-boun...@lists.wikimedia.org [mailto: wiki-research-l-boun...@lists.wikimedia.org] *On Behalf Of *Aaron Halfaker *Sent:* Friday, 9 November 2012 1:51 AM *To:* Research into Wikimedia content and communities *Subject:* Re: [Wiki-research-l] Wiki Research Journal? - Why? ** ** Highly rated is an interesting property. One of the ways that a publication venue becomes highly rated is by being highly restrictive. In fact, the primary measurement of the quality of a publication venue is the acceptance rate of that conference. ** ** WikiSym is not considered highly rated because a high proportion of the submitted papers are accepted. Would a wiki journal be more restrictive in order to gain a highly rated status? ** ** I think it's interesting to ask why WikiSym needs improvement and why attendance has been falling. If a WikiSym is a wiki conference that is struggling to maintain participation, how might a wiki journal surmount such trouble? Assuming
Re: [Wiki-research-l] Wiki Research Journal? - Why?
On Thu, Nov 8, 2012 at 10:34 PM, Brian Keegan bkee...@northwestern.edu wrote: It seems nigh-impossible to assemble an editorial board that is simultaneously open and qualified to reviewing submissions that almost certainly cover the gamut from journalism and media studies, computer and information sciences, complex and network sciences, sociology and organizational behavior, business and economics, legal and policy studies, education and outreach. This is why I think a wiki for research would be so cool. Again, just imagine Wikipedia without the no original research restriction. Why would people contribute to a wiki Research Hub? is very different from Why would people contribute to a Wiki Studies journal? People from the fields you mentioned might have many (different) reasons for participating in cutting edge, massively multiauthor, and/or highly cross-disciplinary work ON a wiki. As for where they publish in the end, that would presumably be up to them. However, it would also be relatively easy create a collection of overlay journals on top of the wiki research hub, with individual review boards who were qualified to deal with particular selections of topics (E.g. the Wiki Journal of Journalism and Media Studies, the Wiki Journal of Computer and Information Sciences, etc.) It seems to me that if we built support for research practices in general, support for research publication practices would follow in due course. ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review
The woman discrimination is something the journal should care about. Any idea on how to face it? 2012/11/6 Chitu Okoli chitu.ok...@concordia.ca Actually, I think it is more reasonable to use double-blind by default unless authors request single-blind. If single-blind were the default, it would be difficult to request double-blind as exceptions: * If there is a big name researcher who wants to take advantage of his/her reputation, he/she would not request double-blind. * If there is a big name researcher who is modest and does not think highly of himself/herself, he/she would not request double-blind. * If there is a minority or woman researcher afraid of discrimination, if he/she requested double-blind, the reviewers would reasonably guess that the author(s) are minorities or women. Thus, I think double-blind as a default for everyone with single-blind as special exception would be the more practical and fairer general policy. With the increase of preprints and working papers (e.g. arXiv and SSRN), I think author anonymity is becoming increasingly impractical. In any case, these comments mainly apply to social science journals; I still think that single-blind makes more sense for computer science journals. ~ Chitu Kerry Raymond a écrit : I think a compromise position is to use single-blind unless the authors request double-blind and are therefore prepared to undertake the ridiculous gymnastics required. Certainly (in computer science) I would agree that it is very hard for any established researcher publishing in their normal field to successfully disguise their identify. Sent from my iPad On 05/11/2012, at 8:30 AM, Chitu Okoli chitu.ok...@concordia.ca chitu.ok...@concordia.ca wrote: Although in theory double-blind review is superior to single-blind, in practice double-blind vs. single-blind review has very little to do with journal quality. It is much more a matter of disciplinary culture. (Single-blind: authors don't know who the reviewers are, but reviewers do know who authors are; Double-blind: neither authors nor reviewers know who each other are) Yes, I am certainly aware of studies that show that double-blind reviewing does indeed reduce the bias towards publishing famous researchers and reduces the bias against publishing work by minority researchers and women. So, I believe that double-blind reviewing is indeed meaningful. However, my general observation is that the decision for a journal to be double-blind or single-blind is mainly a matter of disciplinary tradition. To make a very gross generalization, social science journals are generally double-blind, whereas natural science and mathematical science journals are generally single-blind. This observation is very relevant to this discussion, because the wiki-research-l community sits in between this divide. My perception is that 90% of people who post on this list are in information systems (double-blind), computer science (single-blind) or information science (either double- or single-blind). If we can accept that single-blind journals can be considered as high-quality as well, then I feel a journal with wikified peer review would do much better being single-blind, especially if its subject matter is wiki-related topics. I understand that one of the primary reasons many journals decide against going double-blind is because of the ridiculous gymnastics that have to be undertaken in many cases to try to hide authors' identity. In computer science, where many researchers make their programs available on the Web, and much of their research concerns particular websites that they have developed, double-blinding would often be impossible if attempted--reviewers couldn't properly evaluate the work without knowing who created it. I think this is very much the case for a wiki-based peer review system for much wiki-related research. Of course, the official reviewers should remain anonymous. (I know that open peer review has often been proposed--authors and reviewers know each others' identities--and has even been experimented with on several occasions, but it does not seem to be a quality improvement--I can post citations if anyone is interested.) It is much easier to hide the identity of the official reviewers than it is to hide that of the authors, and the benefits of single-blinding are substantial and proven. ~ Chitu ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPIFM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34)
Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review
just out of curiosity, what could be the reasonable expected purposes for requesting a single-blind review instead of a standard double-blind in your model? best, dj On Tue, Nov 6, 2012 at 3:56 AM, Chitu Okoli chitu.ok...@concordia.cawrote: Actually, I think it is more reasonable to use double-blind by default unless authors request single-blind. If single-blind were the default, it would be difficult to request double-blind as exceptions: * If there is a big name researcher who wants to take advantage of his/her reputation, he/she would not request double-blind. * If there is a big name researcher who is modest and does not think highly of himself/herself, he/she would not request double-blind. * If there is a minority or woman researcher afraid of discrimination, if he/she requested double-blind, the reviewers would reasonably guess that the author(s) are minorities or women. Thus, I think double-blind as a default for everyone with single-blind as special exception would be the more practical and fairer general policy. With the increase of preprints and working papers (e.g. arXiv and SSRN), I think author anonymity is becoming increasingly impractical. In any case, these comments mainly apply to social science journals; I still think that single-blind makes more sense for computer science journals. ~ Chitu Kerry Raymond a écrit : I think a compromise position is to use single-blind unless the authors request double-blind and are therefore prepared to undertake the ridiculous gymnastics required. Certainly (in computer science) I would agree that it is very hard for any established researcher publishing in their normal field to successfully disguise their identify. Sent from my iPad On 05/11/2012, at 8:30 AM, Chitu Okoli chitu.ok...@concordia.ca chitu.ok...@concordia.ca wrote: Although in theory double-blind review is superior to single-blind, in practice double-blind vs. single-blind review has very little to do with journal quality. It is much more a matter of disciplinary culture. (Single-blind: authors don't know who the reviewers are, but reviewers do know who authors are; Double-blind: neither authors nor reviewers know who each other are) Yes, I am certainly aware of studies that show that double-blind reviewing does indeed reduce the bias towards publishing famous researchers and reduces the bias against publishing work by minority researchers and women. So, I believe that double-blind reviewing is indeed meaningful. However, my general observation is that the decision for a journal to be double-blind or single-blind is mainly a matter of disciplinary tradition. To make a very gross generalization, social science journals are generally double-blind, whereas natural science and mathematical science journals are generally single-blind. This observation is very relevant to this discussion, because the wiki-research-l community sits in between this divide. My perception is that 90% of people who post on this list are in information systems (double-blind), computer science (single-blind) or information science (either double- or single-blind). If we can accept that single-blind journals can be considered as high-quality as well, then I feel a journal with wikified peer review would do much better being single-blind, especially if its subject matter is wiki-related topics. I understand that one of the primary reasons many journals decide against going double-blind is because of the ridiculous gymnastics that have to be undertaken in many cases to try to hide authors' identity. In computer science, where many researchers make their programs available on the Web, and much of their research concerns particular websites that they have developed, double-blinding would often be impossible if attempted--reviewers couldn't properly evaluate the work without knowing who created it. I think this is very much the case for a wiki-based peer review system for much wiki-related research. Of course, the official reviewers should remain anonymous. (I know that open peer review has often been proposed--authors and reviewers know each others' identities--and has even been experimented with on several occasions, but it does not seem to be a quality improvement--I can post citations if anyone is interested.) It is much easier to hide the identity of the official reviewers than it is to hide that of the authors, and the benefits of single-blinding are substantial and proven. ~ Chitu ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- __ dr hab. Dariusz Jemielniak profesor zarządzania kierownik katedry Zarządzania Międzynarodowego i centrum badawczego CROW Akademia Leona Koźmińskiego http://www.crow.alk.edu.pl
Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review
Here are a few scenarios: * The research topic concerns a public website. The website identifies the authors. The paper makes no sense without explicitly identifying the website. Thus, authors should be able to request single-blind review. Note that this scenario very much applies to this entire discussion of a new research journal that uses wiki-based research development. I don't know if you caught Kerry Raymond's comment on this thread (I copy it below), which explains this point very succinctly. * Authors have posted a working paper which has been on the web for a long time, and is known to most researchers in that field of interest (i.e. most potential and qualified reviewers for the peer-reviewed version). In this case, I would think that reviewers should not be excluded for no reason other than they know the authors' identity. One of the most backward policies I've ever seen related to this is JIBS's policy to protect double-blind review: "Authors should also not post their submitted manuscript (including working papers and prior drafts) on websites where it could be easily discovered by potential reviewers." [1] Apparently, they consider double-blind review a more sacred ideal than early dissemination of research through working papers. * The research critically involves a multimedia artifact, such as a video, that cannot be easily be submitted as supporting materials for peer review. The video is better posted on a website. Here's a case of requested "gymnastics" I've seen in order to protect double-blind peer review even in such cases: "We ask each author to create his/her own account with an open access provider of choice (e.g., linked video could be hosted in Vimeo or YouTube). Please use a pseudo user name in order to maintain anonymity during the review process." [2] Although I do believe in the benefits of double-blind review (I'll send a separate post with a few citations), in my own research I am increasingly confronted with the fact that new approaches to research that favour openness and mass collaboration are fundamentally in conflict with the idea of anonymity in the identity of the authors of a manuscript submitted for peer review. Personally, I prefer to forge ahead with innovative modes of research conduct, even if double-blind review is sacrificed. For me, a perfect compromise is to default to double-blind, but fall back to single-blind when the nature of the research project calls for it. ~ Chitu [1] http://www.palgrave-journals.com/jibs/author_instructions.html#Ethical-guidelines [2] http://icis2011.aisnet.org/Paper_Submission.html#B Dariusz Jemielniak a crit: just out of curiosity, what could be the reasonable expected purposes for requesting a single-blind review instead of a standard double-blind in your model? best, dj On Tue, Nov 6, 2012 at 3:56 AM, Chitu Okoli chitu.ok...@concordia.ca wrote: Actually, I think it is more reasonable to use double-blind by default unless authors request single-blind. If single-blind were the default, it would be difficult to request double-blind as exceptions: * If there is a "big name" researcher who wants to take advantage of his/her reputation, he/she would not request double-blind. * If there is a "big name" researcher who is modest and does not think highly of himself/herself, he/she would not request double-blind. * If there is a minority or woman researcher afraid of discrimination, if he/she requested double-blind, the reviewers would reasonably guess that the author(s) are minorities or women. Thus, I think double-blind as a default for everyone with single-blind as special exception would be the more practical and fairer general policy. With the increase of preprints and working papers (e.g. arXiv and SSRN), I think author anonymity is becoming increasingly impractical. In any case, these comments mainly apply to social science journals; I still think that single-blind makes more sense for computer science journals. ~ Chitu Kerry Raymond a crit: I would note that the use of 1) would render double-blind irrelevant in 2). We would all know ... On 06/11/2012, at 6:05 AM, "Kerry
Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review
Here are some references about the pros and cons of double-blind peer review: * Book: Editorial Peer Review: Its Strengths Weaknesses by Ann C. Weller [1]. This book covers not only double-blind peer review, but empirical studies about all kinds of peer review (including open peer review, where even the reviewers are not anonymous). This book, and papers it summarizes, is my primary source of information on this topic. If your library doesn't have it, you could ask them to get it (that's how I got a hold of it for myself). * Nature magazine report on an international survey about peer review [2]. Highlights pertinent to this discussion: - "Support [for double-blind peer review] is highest with those who have experienced it (the humanities and social sciences) or *where it is perceived to do the most good (among female authors)*. The least enthusiastic group is editors." - "The one bright light in favour of double-blind peer review is the measured reduction in bias against authors with female first names (shown in numerous studies, such as ref. 4). This suggests that authors submitting papers to traditionally minded journals should include the given names of authors only on the final, published version." - "The double-blind approach is predicated on a culture in which manuscripts-in-progress are kept secret. This is true for the most part in the life sciences. But some physical sciences, such as high-energy physics, share preprints extensively through arXiv, an online repository. *Thus, double-blind peer review is at odds with another 'force for good' in the academic world: the open sharing of information.* The PRC survey found that highly competitive fields (such as neuroscience) or those with larger commercial or applied interests (such as materials science and chemical engineering) were the most enthusiastic about double-blinding, whereas fields with more of a tradition for openness (astronomy and mathematics) were decidedly less supportive." * Two open discussions on Nature magazine blogs about double-blind peer review from 2005 [3] and 2008 [4]. The 2008 discussion was in response to the editorial mentioned above. ~ Chitu [1] http://www.amazon.ca/Editorial-Peer-Review-Strengths-Weaknesses/dp/1573871001 [2] http://www.nature.com/nature/journal/v451/n7179/full/451605b.html [3] http://blogs.nature.com/actionpotential/2005/12/doubleblind_peer_review.html [4] http://blogs.nature.com/peer-to-peer/2008/02/working_doubleblind.html Manuel Palomo Duarte a crit: The woman discrimination is something the journal should care about. Any idea on how to face it? ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review
well, then I think I basically disagree on this one. I think that the fact that the authors CAN be identified after doing some more or less advanced research, does not mean that the reviewers are going to actively seek to break their anonymity (in fact, I'd assume this would be discouraged by most journal policies, and there are many traditional research projects where identifying the authors after some investigation is possible). Double-blind review is a process which is sustained and secured by good-faith participants (both the authors and the reviewers, too). Even if the authors can be guessed with some probability just from the references list, it does not mean that eliminating all elements of doubt serves a good purpose. I, for that matter, would rather avoid checking SSRN/Academia/wiki for the authors' names, to protect the rules of the game, and I would report that i may know the authors if I had known about their project from before hand. best, dj On Tue, Nov 6, 2012 at 3:48 PM, Chitu Okoli chitu.ok...@concordia.cawrote: Here are a few scenarios: * The research topic concerns a public website. The website identifies the authors. The paper makes no sense without explicitly identifying the website. Thus, authors should be able to request single-blind review. Note that this scenario very much applies to this entire discussion of a new research journal that uses wiki-based research development. I don't know if you caught Kerry Raymond's comment on this thread (I copy it below), which explains this point very succinctly. * Authors have posted a working paper which has been on the web for a long time, and is known to most researchers in that field of interest (i.e. most potential and qualified reviewers for the peer-reviewed version). In this case, I would think that reviewers should not be excluded for no reason other than they know the authors' identity. One of the most backward policies I've ever seen related to this is JIBS's policy to protect double-blind review: Authors should also not post their submitted manuscript (including working papers and prior drafts) on websites where it could be easily discovered by potential reviewers. [1] Apparently, they consider double-blind review a more sacred ideal than early dissemination of research through working papers. * The research critically involves a multimedia artifact, such as a video, that cannot be easily be submitted as supporting materials for peer review. The video is better posted on a website. Here's a case of requested gymnastics I've seen in order to protect double-blind peer review even in such cases: We ask each author to create his/her own account with an open access provider of choice (e.g., linked video could be hosted in Vimeo or YouTube). Please use a pseudo user name in order to maintain anonymity during the review process. [2] Although I do believe in the benefits of double-blind review (I'll send a separate post with a few citations), in my own research I am increasingly confronted with the fact that new approaches to research that favour openness and mass collaboration are fundamentally in conflict with the idea of anonymity in the identity of the authors of a manuscript submitted for peer review. Personally, I prefer to forge ahead with innovative modes of research conduct, even if double-blind review is sacrificed. For me, a perfect compromise is to default to double-blind, but fall back to single-blind when the nature of the research project calls for it. ~ Chitu [1] http://www.palgrave-journals.com/jibs/author_instructions.html#Ethical-guidelines [2] http://icis2011.aisnet.org/Paper_Submission.html#B Dariusz Jemielniak a écrit : just out of curiosity, what could be the reasonable expected purposes for requesting a single-blind review instead of a standard double-blind in your model? best, dj On Tue, Nov 6, 2012 at 3:56 AM, Chitu Okoli chitu.ok...@concordia.cawrote: Actually, I think it is more reasonable to use double-blind by default unless authors request single-blind. If single-blind were the default, it would be difficult to request double-blind as exceptions: * If there is a big name researcher who wants to take advantage of his/her reputation, he/she would not request double-blind. * If there is a big name researcher who is modest and does not think highly of himself/herself, he/she would not request double-blind. * If there is a minority or woman researcher afraid of discrimination, if he/she requested double-blind, the reviewers would reasonably guess that the author(s) are minorities or women. Thus, I think double-blind as a default for everyone with single-blind as special exception would be the more practical and fairer general policy. With the increase of preprints and working papers (e.g. arXiv and SSRN), I think author anonymity is becoming increasingly impractical. In any case, these comments mainly apply to social science
Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review
I cannot speak for other disciplines but double-blind is not standard in Computer Science. Kerry ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal?
On Mon, Nov 5, 2012 at 12:57 AM, Ward Cunningham w...@c2.com wrote: I wonder if a better place to innovate might be in the conduct of research, rather than the reporting, review and publication of research? +1* Regarding the existing conversation, if we want a journal, we need to ask what the purpose is. I'd highly recommend Jason Priem's notion of the decoupled journal [1][2]. Jason points out that journals have been used for four main purposes, historically: 1. Registration 2. Archiving 3. Dissemination 4. Certification Decoupling these functions is the way forward for scholarly communication. And it's already been happening -- with ArXiV, SSRN, Math Overflow, ... and new ways of measuring research impact [3]. So which function(s) matter most to us? We can ask: (1) What can wikis do for the registration, archiving, and/or dissemination functions, better than existing technologies? (2) How can wikis contribute to altmetrics [3] used for certification functions? (3) How can we as a community surface the most interesting and powerful research? What technologies do we need? What social habits do we need? (4) And, finally, how can our answers to #3 contribute to the certification we value? (Prestige, publications counting for tenure, ...) I think rather than trying to create a high profile, high impact traditional journal, if we focused on these and similar questions, we would both move wiki research forward, and drive scientific communication itself forward. -Jodi * of course, reporting and doing research aren't an either/or -- they're closely related and one drives the other [1] Jason Priem at Purdue: video http://youtu.be/OM22JuiWYgE slides https://docs.google.com/present/view?id=ddfg787c_362f465q2g5 I've written a short summary here: http://jodischneider.com/blog/2012/11/04/altmetrics-can-help-surface-quality-content-jason-priem-on-the-decoupled-journal-as-the-achievable-future-of-scholarly-communication/ [2] Also a draft article called Decoupling the scholarly journal by Jason Priem and Bradley M. Hemminger, under review for the Frontiers in Computational Neuroscience special issue Beyond open access: visions for open evaluation of scientific papers by post-publication peer review https://docs.google.com/document/d/1xDOy9GXXrUFc9TUIR2C470DTau8JEgZ9k-SMNIx5pb8/edit?hl=en_USamp;authkey=CMeCqOYD [3] http://altmetrics.org ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal?
+1 Jodi! I agree it would be great to experiment on-site as you suggest Claudia On Mon, 5 Nov 2012 10:39:34 +, Jodi Schneider wrote On Mon, Nov 5, 2012 at 12:57 AM, Ward Cunningham w...@c2.com wrote: I wonder if a better place to innovate might be in the conduct of research, rather than the reporting, review and publication of research? +1* Regarding the existing conversation, if we want a journal, we need to ask what the purpose is. I'd highly recommend Jason Priem's notion of the decoupled journal [1][2]. Jason points out that journals have been used for four main purposes, historically: 1. Registration 2. Archiving 3. Dissemination 4. Certification Decoupling these functions is the way forward for scholarly communication. And it's already been happening -- with ArXiV, SSRN, Math Overflow, ... and new ways of measuring research impact [3]. So which function(s) matter most to us? We can ask: (1) What can wikis do for the registration, archiving, and/or dissemination functions, better than existing technologies? (2) How can wikis contribute to altmetrics [3] used for certification functions? (3) How can we as a community surface the most interesting and powerful research? What technologies do we need? What social habits do we need? (4) And, finally, how can our answers to #3 contribute to the certification we value? (Prestige, publications counting for tenure, ...) I think rather than trying to create a high profile, high impact traditional journal, if we focused on these and similar questions, we would both move wiki research forward, and drive scientific communication itself forward. -Jodi * of course, reporting and doing research aren't an either/or -- they're closely related and one drives the other [1] Jason Priem at Purdue: video http://youtu.be/OM22JuiWYgE slides https://docs.google.com/present/view?id=ddfg787c_362f465q2g5 I've written a short summary here: http://jodischneider.com/blog/2012/11/04/altmetrics-can-help-surface- quality-content-jason-priem-on-the-decoupled-journal-as-the-achievable- future-of-scholarly-communication/ [2] Also a draft article called Decoupling the scholarly journal by Jason Priem and Bradley M. Hemminger, under review for the Frontiers in Computational Neuroscience special issue Beyond open access: visions for open evaluation of scientific papers by post-publication peer review https://docs.google.com/document/d/1xDOy9GXXrUFc9TUIR2C470DTau8JEgZ9k-SMNIx5pb8/edit? hl=en_USauthkey=CMeCqOYD [3] http://altmetrics.org thanks cheers, Claudia koltzenb...@w4w.net ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal?
As a side consideration, I think that science is elitist, today. Obviously, there are some required rules to assure and assess what is sciencie and what is not, but we have the opportunity to open science to the world. Until now people just consume science. We are in a historical moment to welcome and engage the entire world in science: proposing stuff to be researched, developing tools, extracting and curating data, and checking and peer-reviewing facts. In the same way writing the world memory is a task for all us (Wikipedia), sciencie follows the same principles. Open science, open research, open review, open data. 2012/11/5 Jodi Schneider jschnei...@pobox.com On Mon, Nov 5, 2012 at 12:57 AM, Ward Cunningham w...@c2.com wrote: I wonder if a better place to innovate might be in the conduct of research, rather than the reporting, review and publication of research? +1* Regarding the existing conversation, if we want a journal, we need to ask what the purpose is. I'd highly recommend Jason Priem's notion of the decoupled journal [1][2]. Jason points out that journals have been used for four main purposes, historically: 1. Registration 2. Archiving 3. Dissemination 4. Certification Decoupling these functions is the way forward for scholarly communication. And it's already been happening -- with ArXiV, SSRN, Math Overflow, ... and new ways of measuring research impact [3]. So which function(s) matter most to us? We can ask: (1) What can wikis do for the registration, archiving, and/or dissemination functions, better than existing technologies? (2) How can wikis contribute to altmetrics [3] used for certification functions? (3) How can we as a community surface the most interesting and powerful research? What technologies do we need? What social habits do we need? (4) And, finally, how can our answers to #3 contribute to the certification we value? (Prestige, publications counting for tenure, ...) I think rather than trying to create a high profile, high impact traditional journal, if we focused on these and similar questions, we would both move wiki research forward, and drive scientific communication itself forward. -Jodi * of course, reporting and doing research aren't an either/or -- they're closely related and one drives the other [1] Jason Priem at Purdue: video http://youtu.be/OM22JuiWYgE slides https://docs.google.com/present/view?id=ddfg787c_362f465q2g5 I've written a short summary here: http://jodischneider.com/blog/2012/11/04/altmetrics-can-help-surface-quality-content-jason-priem-on-the-decoupled-journal-as-the-achievable-future-of-scholarly-communication/ [2] Also a draft article called Decoupling the scholarly journal by Jason Priem and Bradley M. Hemminger, under review for the Frontiers in Computational Neuroscience special issue Beyond open access: visions for open evaluation of scientific papers by post-publication peer review https://docs.google.com/document/d/1xDOy9GXXrUFc9TUIR2C470DTau8JEgZ9k-SMNIx5pb8/edit?hl=en_USamp;authkey=CMeCqOYDhttps://docs.google.com/document/d/1xDOy9GXXrUFc9TUIR2C470DTau8JEgZ9k-SMNIx5pb8/edit?hl=en_USauthkey=CMeCqOYD [3] http://altmetrics.org ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- Emilio J. Rodríguez-Posada http://LibreFind.org - The wiki search engine ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal?
On Mon, Nov 5, 2012 at 1:59 AM, Pierre-Carl Langlais langlais.qo...@gmail.com wrote: I may sound a bit overractive, but can't we do both? I would easily imagine the following two-way system: *A wiki-laboratory, which hosts quick and less quick projects in progress. That could also include some reactive analysis by social scientists to recent wikimedian and collaborative knowledge phenomena. *A wiki-journal, that should give a definitive and quotable form to the preceding researches and enlarge the discussions to wider disciplinary debates. By publishing external submissions it could give some new food for thought to the wiki-laboratory. The relationships between the two structures would be a relevant analogy to the mutually-rewarding dialogue between the practical and theoretical side of scientific research. I think we should be aware of two interesting countervailing trends. (1) Observe that we are STILL having this conversation on a mailing list, despite the existence of a wiki page that is in theory devoted to exploring precisely these issues. (http://meta.wikimedia.org/wiki/Wiki_Research_Ideas) It would appear that working on a wiki is extra work compared to working on a mailing list (at least for this exploratory phase of the conversation). That's not meant to be a normative judgment, it just points to the need for a broader awareness of what wiki-like research might look like. In particular, it may also look like correspondence. So, let's imagine that the wiki-laboratory was already prototyped and tested on usenet. What should our next steps be? (2) Observe that the wiki journal idea is connected with a particular research obsession (for some), namely finding experts. Of course, in order to find an expert, the expert must first have been created or manufactured. In a positive light, this means education. In a negative light, it means regimes for producing stratification and alienation. Both, of course, exist already. So again the question is one of next steps, not something de novo. My sense is that that practicality is usually abhorred (it's expensive, mundane, and you can't get credit for it), whereas theory is strongly preferred (it's powerful, efficient, and it can go on your CV). In my view, the only sensible solution is to dissolve the (imagined) separation between practice and theory, and look instead at the actual practices of researchers and knowledge workers, trying to support them (i.e. us), in what we actually do. ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal?
Following the discussion of yesterday, I have enhanced a bit the design draft: http://meta.wikimedia.org/wiki/Wiki_Research_Ideas/Design It now includes a specific thema for each issue. For instance, I have chosen « Wikipedia Verifiability ». In order to visualize what kind of content we could get, I have replaced the Lorem ipsum stuff with « fake » summaries. As you can see, the journal may include both social science analyses (cf. the quellenkritik to wikipedia thing), and computer science experiment (the device from wikisource). Those who are interesting in taking part to the project may add their names here : http://meta.wikimedia.org/wiki/Wiki_Research_Ideas/Volunteers Kind regards, As a side consideration, I think that science is elitist, today. Obviously, there are some required rules to assure and assess what is sciencie and what is not, but we have the opportunity to open science to the world. Until now people just consume science. We are in a historical moment to welcome and engage the entire world in science: proposing stuff to be researched, developing tools, extracting and curating data, and checking and peer-reviewing facts. In the same way writing the world memory is a task for all us (Wikipedia), sciencie follows the same principles. Open science, open research, open review, open data. 2012/11/5 Jodi Schneider jschnei...@pobox.com On Mon, Nov 5, 2012 at 12:57 AM, Ward Cunningham w...@c2.com wrote: I wonder if a better place to innovate might be in the conduct of research, rather than the reporting, review and publication of research? +1* Regarding the existing conversation, if we want a journal, we need to ask what the purpose is. I'd highly recommend Jason Priem's notion of the decoupled journal [1][2]. Jason points out that journals have been used for four main purposes, historically: Registration Archiving Dissemination Certification Decoupling these functions is the way forward for scholarly communication. And it's already been happening -- with ArXiV, SSRN, Math Overflow, ... and new ways of measuring research impact [3]. So which function(s) matter most to us? We can ask: (1) What can wikis do for the registration, archiving, and/or dissemination functions, better than existing technologies? (2) How can wikis contribute to altmetrics [3] used for certification functions? (3) How can we as a community surface the most interesting and powerful research? What technologies do we need? What social habits do we need? (4) And, finally, how can our answers to #3 contribute to the certification we value? (Prestige, publications counting for tenure, ...) I think rather than trying to create a high profile, high impact traditional journal, if we focused on these and similar questions, we would both move wiki research forward, and drive scientific communication itself forward. -Jodi * of course, reporting and doing research aren't an either/or -- they're closely related and one drives the other [1] Jason Priem at Purdue: video http://youtu.be/OM22JuiWYgE slides https://docs.google.com/present/view?id=ddfg787c_362f465q2g5 I've written a short summary here: http://jodischneider.com/blog/2012/11/04/altmetrics-can-help-surface-quality-content-jason-priem-on-the-decoupled-journal-as-the-achievable-future-of-scholarly-communication/ [2] Also a draft article called Decoupling the scholarly journal by Jason Priem and Bradley M. Hemminger, under review for the Frontiers in Computational Neuroscience special issue Beyond open access: visions for open evaluation of scientific papers by post- publication peer review https://docs.google.com/document/d/1xDOy9GXXrUFc9TUIR2C470DTau8JEgZ9k-SMNIx5pb8/edit?hl=en_USamp;authkey=CMeCqOYD [3] http://altmetrics.org ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- Emilio J. Rodríguez-Posada http://LibreFind.org - The wiki search engine ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal?
I think two things can be done in parallel. 1. Allow folks to create descriptions of research in progress on the wiki, which can be progressively updated. This enables others to make suggestions on methodology, give feedback on drafts of papers and so forth. Open and collaborative and experimental in the meta-sense. Clearly many on this list desire to experiment with new ways of working. 2. Have a more formal traditional review process, so that the journal mets the criteria for reputable that is important for people's CVs, tenure, promotion and so forth. As much as many of this don't like this way of working, it is the reality for earning your salary. I don't think it should be a requirement that you engage in 1) before engaging in 2). But I would like to believe that people who engage in 1) are far more likely to be have an easy acceptance through 2) because their work has already had the benefit of many eyes. Where I think this plan is likely to come unstuck relates to the question of authorship of the final papers for the purposes of 2). If someone feels that they have made a lot of contribution to the research through 1) they might feel entitled to author rights. In this regard, it is worth reminding ourselves of the Vancouver protocol, which many journals either mandate or will resort to in the event of authorship disputes: http://www.authorder.com/index.php?option=com_contentview=articleid=28Ite mid=47 (a short version) http://www.icmje.org/urm_full.pdf (a long version see pages 2-3) An open way of working will clearly enable people to contribute to the research and the writing in ways that might make them eligible to be an author under the Vancouver protocol. The argument will tend to hinge on the question of whether the contributions were substantial. But I don't think this concern is a reason not to enable more collaborative ways of researching. It's just something that everyone should be aware of up-front, that opening your research up to input from others might mean you have to add some co-authors to your work if they make substantial contributions. Kerry ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal?
Good points, Kerry, sup! dj On Mon, Nov 5, 2012 at 9:05 PM, Kerry Raymond kerry.raym...@gmail.comwrote: I think two things can be done in parallel. 1. Allow folks to create descriptions of research in progress on the wiki, which can be progressively updated. This enables others to make suggestions on methodology, give feedback on drafts of papers and so forth. Open and collaborative and experimental in the meta-sense. Clearly many on this list desire to experiment with new ways of working. 2. Have a more formal traditional review process, so that the journal mets the criteria for reputable that is important for people's CVs, tenure, promotion and so forth. As much as many of this don't like this way of working, it is the reality for earning your salary. I don't think it should be a requirement that you engage in 1) before engaging in 2). But I would like to believe that people who engage in 1) are far more likely to be have an easy acceptance through 2) because their work has already had the benefit of many eyes. Where I think this plan is likely to come unstuck relates to the question of authorship of the final papers for the purposes of 2). If someone feels that they have made a lot of contribution to the research through 1) they might feel entitled to author rights. In this regard, it is worth reminding ourselves of the Vancouver protocol, which many journals either mandate or will resort to in the event of authorship disputes: http://www.authorder.com/index.php?option=com_contentview=articleid=28Ite mid=47 (a short version) http://www.icmje.org/urm_full.pdf (a long version see pages 2-3) An open way of working will clearly enable people to contribute to the research and the writing in ways that might make them eligible to be an author under the Vancouver protocol. The argument will tend to hinge on the question of whether the contributions were substantial. But I don't think this concern is a reason not to enable more collaborative ways of researching. It's just something that everyone should be aware of up-front, that opening your research up to input from others might mean you have to add some co-authors to your work if they make substantial contributions. Kerry ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- __ dr hab. Dariusz Jemielniak profesor zarządzania kierownik katedry Zarządzania Międzynarodowego i centrum badawczego CROW Akademia Leona Koźmińskiego http://www.crow.alk.edu.pl ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal?
I equally see the journal as a way to bridge research as it is, and research as it's likely to become. The main work the editing committee is liable to do is social: enforce usual procedures, keep in touch with the ISI and other certification centers, find partnerships with scholar institutions, ensure communication toward scientific and general public. Is such a work inessential? I am not so sure. During the OA Week meeting at the UNESCO, we all came to agree on one point: the main (if not the sole) obstacle to the generalization of new scientific practices consists in current, unquestioned, hard to get rid off habits. Designing some kind of intermediary between pioneering initiatives and mainstream reflexes may have some relevancy, not in the long-term perspective, but now. The journal may have therefore two valuable objectives: *It practices an extended version of open access. The mere facts of publishing peer-reviewing debates and getting outsiders to take a small, albeit definite, part in the writing and reviewing process are already quite unusual novelties. If the journal becomes a success wide enough to encourage other journals in imitating it, I think the whole project would have been rewarding. *It's a good way to recruit new scholars, especially for the wiki- laboratory. I know a lot of French academics who are interested in Wikis. Yet they mainly issue solitary analysis without taking part to wider wiki-research. Le 5 nov. 12 à 19:35, Ward Cunningham a écrit : Wiki started with almost no schema and remains light in this regard today. This favor's innovation while it complicates interpretation. My own suspicion is that wiki's low-schema is better suited for the laboratory than the journal, at least as a journal is currently conceived. Wiki may be well suited for a refactored journal, simply because it makes experiments easy, and this is what makes Jason's premise apropos. I'm agreeing with you on the refactoring factor. Wiki is a place where people act rather than say. It may not be the most convenient structure to host a journal, although Wikipedia has already imported some tools and behaviors from scientific journals: footnotes, FA/GA reviewing and so on. It is a powerful device to reveal social and textual interactions. As such, it seems to have all the requirements to experiment extended open access, by showing the elaboration of editorial decisions, peer-reviewing discussions, corrections, works in progress from a proposal to a complete article… PCL___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review
Actually, I think it is more reasonable to use double-blind by default unless authors request single-blind. If single-blind were the default, it would be difficult to request double-blind as exceptions: * If there is a "big name" researcher who wants to take advantage of his/her reputation, he/she would not request double-blind. * If there is a "big name" researcher who is modest and does not think highly of himself/herself, he/she would not request double-blind. * If there is a minority or woman researcher afraid of discrimination, if he/she requested double-blind, the reviewers would reasonably guess that the author(s) are minorities or women. Thus, I think double-blind as a default for everyone with single-blind as special exception would be the more practical and fairer general policy. With the increase of preprints and working papers (e.g. arXiv and SSRN), I think author anonymity is becoming increasingly impractical. In any case, these comments mainly apply to social science journals; I still think that single-blind makes more sense for computer science journals. ~ Chitu Kerry Raymond a écrit : I think a compromise position is to use single-blind unless the authors request double-blind and are therefore prepared to undertake the "ridiculous gymnastics" required. Certainly (in computer science) I would agree that it is very hard for any established researcher publishing in their normal field to successfully disguise their identify. Sent from my iPad On 05/11/2012, at 8:30 AM, "Chitu Okoli" chitu.ok...@concordia.ca wrote: Although in theory double-blind review is superior to single-blind, in practice double-blind vs. single-blind review has very little to do with journal quality. It is much more a matter of disciplinary culture. (Single-blind: authors don't know who the reviewers are, but reviewers do know who authors are; Double-blind: neither authors nor reviewers know who each other are) Yes, I am certainly aware of studies that show that double-blind reviewing does indeed reduce the bias towards publishing famous researchers and reduces the bias against publishing work by minority researchers and women. So, I believe that double-blind reviewing is indeed meaningful. However, my general observation is that the decision for a journal to be double-blind or single-blind is mainly a matter of disciplinary tradition. To make a very gross generalization, social science journals are generally double-blind, whereas natural science and mathematical science journals are generally single-blind. This observation is very relevant to this discussion, because the wiki-research-l community sits in between this divide. My perception is that 90% of people who post on this list are in information systems (double-blind), computer science (single-blind) or information science (either double- or single-blind). If we can accept that single-blind journals can be considered as high-quality as well, then I feel a journal with wikified peer review would do much better being single-blind, especially if its subject matter is wiki-related topics. I understand that one of the primary reasons many journals decide against going double-blind is because of the ridiculous gymnastics that have to be undertaken in many cases to try to hide authors' identity. In computer science, where many researchers make their programs available on the Web, and much of their research concerns particular websites that they have developed, double-blinding would often be impossible if attempted--reviewers couldn't properly evaluate the work without knowing who created it. I think this is very much the case for a wiki-based peer review system for much wiki-related research. Of course, the official reviewers should remain anonymous. (I know that open peer review has often been proposed--authors and reviewers know each others' identities--and has even been experimented with on several occasions, but it does not seem to be a quality improvement--I can post citations if anyone is interested.) It is much easier to hide the identity of the official reviewers than it is to hide that of the authors, and the benefits of single-blinding are substantial and proven. ~ Chitu ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal?
I would note that the use of 1) would render double-blind irrelevant in 2). We would all know ... Sent from my iPad On 06/11/2012, at 6:05 AM, Kerry Raymond kerry.raym...@gmail.com wrote: I think two things can be done in parallel. 1. Allow folks to create descriptions of research in progress on the wiki, which can be progressively updated. This enables others to make suggestions on methodology, give feedback on drafts of papers and so forth. Open and collaborative and experimental in the meta-sense. Clearly many on this list desire to experiment with new ways of working. 2. Have a more formal traditional review process, so that the journal mets the criteria for reputable that is important for people's CVs, tenure, promotion and so forth. As much as many of this don't like this way of working, it is the reality for earning your salary. I don't think it should be a requirement that you engage in 1) before engaging in 2). But I would like to believe that people who engage in 1) are far more likely to be have an easy acceptance through 2) because their work has already had the benefit of many eyes. Where I think this plan is likely to come unstuck relates to the question of authorship of the final papers for the purposes of 2). If someone feels that they have made a lot of contribution to the research through 1) they might feel entitled to author rights. In this regard, it is worth reminding ourselves of the Vancouver protocol, which many journals either mandate or will resort to in the event of authorship disputes: http://www.authorder.com/index.php?option=com_contentview=articleid=28Ite mid=47 (a short version) http://www.icmje.org/urm_full.pdf (a long version see pages 2-3) An open way of working will clearly enable people to contribute to the research and the writing in ways that might make them eligible to be an author under the Vancouver protocol. The argument will tend to hinge on the question of whether the contributions were substantial. But I don't think this concern is a reason not to enable more collaborative ways of researching. It's just something that everyone should be aware of up-front, that opening your research up to input from others might mean you have to add some co-authors to your work if they make substantial contributions. Kerry ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal?
I'll first say that I've never been on an editorial board, so my comments might be somewhat limited. Like my students, I learn best when I'm shown where I'm mistaken, so I would like to learn from you all! On one hand, I agree that readership is very important. On the other hand, I don't believe it is the most critical issue. I think the most critical matter in starting a new journal is to *attract high-quality submissions* --that is, to get researchers who do high quality work to submit some of their best research to the journal. If that can be achieved, then a high readership is virtually guaranteed. Other benefits should readily follow: high citations, ISI listing, etc. I've been reading all the comments on this thread carefully, and I see two distinct issues that are being mixed into one: (1) a new journal dedicated to research about wikis, and maybe also about related phenomena; and (2) a journal with new kind of wiki-based peer review system. I think it is best to rather keep these issues distinct. First, why do we need a new journal dedicated to wiki research? I would think we don't want a new journal that publishes mainly low-quality research; we want a new journal that publishes high-quality research. So, is there such a need for wiki researchers? That is, do publishers of high-quality wiki-related research have a hard time finding high-quality journal outlets to publish their research? Based on the excellent wiki-based research published in a wide variety of journals, I don't think such a problem exists. So, why would a researcher with high-quality wiki research risk publishing their hard work in a new, unproven, even experimental journal? In my case, I have tenure, so I might consider taking such risks. However, many of my colleagues are working towards establishing their research careers, and I would definitely advise them against publishing their best work in anything but proven journals. My point is not that a new journal cannot attract high-quality research; rather, my suspicion is that it can do so only if it is filling a void for high-quality research on topics that are difficult to publish in existing high-quality outlets. I'm yet to see this issue addressed in this discussion. Second, concerning a new kind of wikified peer review: I think that such an experiment is very much worthwhile and should be attempted. However, from a scientific perspective, an experiment to test phenomenon W (wikified peer review) should control for all other possibly confounding phenomena to make sure that the end result is an accurate reflection of a proper test of phenomenon W. In this case, the risks of a new journal with a poorly justified research focus (as I argued above) is a major confound that blurs the results of testing for W. In short, I think the best way to test wikified peer review is to work with an existing journal that has already established its viability and ability to attract high-quality submissions. The Journal of Peer Production has been mentioned as a target candidate, and their description of their peer review philosophy indicates that they might be quite open to such an experiment, if not with all papers, at least with some: http://peerproduction.net/peer-review/process/. However, it is still a new journal, and doesn't seem to have yet reached the state of releasing regular issues, so its newness might yet be a confound for testing W. ~ Chitu Dariusz Jemielniak a crit: actually, with our community, it is not. What other journals die for, we have sort of provided. This is why a Wiki journal may have a better chance than others, but only if it is prepared with the academic career paths and full proper code of conduct nuances considered (double-blind scholarly peer review, proper editorial board, PDFs with page numbers, etc.). dj On Sun, Nov 4, 2012 at 7:14 PM, Ed H. Chi c...@acm.org wrote: There has been a lot of talk about how to start a journal. The real issue in starting a journal is not the editorial board, or the way it is published, or whether it will gather the citation impact. The real issue is READERSHIP. If you can get people to read the journal, then it will have editors wanting to serve the journal, and it will gather citation impact. The reason why WikiSym is changing is for the same reason. People are not going to the conference! I think the attendance has been
Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review
Although in theory double-blind review is superior to single-blind, in practice double-blind vs. single-blind review has very little to do with journal quality. It is much more a matter of disciplinary culture. (Single-blind: authors don't know who the reviewers are, but reviewers do know who authors are; Double-blind: neither authors nor reviewers know who each other are) Yes, I am certainly aware of studies that show that double-blind reviewing does indeed reduce the bias towards publishing famous researchers and reduces the bias against publishing work by minority researchers and women. So, I believe that double-blind reviewing is indeed meaningful. However, my general observation is that the decision for a journal to be double-blind or single-blind is mainly a matter of disciplinary tradition. To make a very gross generalization, social science journals are generally double-blind, whereas natural science and mathematical science journals are generally single-blind. This observation is very relevant to this discussion, because the wiki-research-l community sits in between this divide. My perception is that 90% of people who post on this list are in information systems (double-blind), computer science (single-blind) or information science (either double- or single-blind). If we can accept that single-blind journals can be considered as high-quality as well, then I feel a journal with wikified peer review would do much better being single-blind, especially if its subject matter is wiki-related topics. I understand that one of the primary reasons many journals decide against going double-blind is because of the ridiculous gymnastics that have to be undertaken in many cases to try to hide authors' identity. In computer science, where many researchers make their programs available on the Web, and much of their research concerns particular websites that they have developed, double-blinding would often be impossible if attempted--reviewers couldn't properly evaluate the work without knowing who created it. I think this is very much the case for a wiki-based peer review system for much wiki-related research. Of course, the official reviewers should remain anonymous. (I know that open peer review has often been proposed--authors and reviewers know each others' identities--and has even been experimented with on several occasions, but it does not seem to be a quality improvement--I can post citations if anyone is interested.) It is much easier to hide the identity of the official reviewers than it is to hide that of the authors, and the benefits of single-blinding are substantial and proven. ~ Chitu Dariusz Jemielniak a crit: actually, with our community, it is not. What other journals die for, we have sort of provided. This is why a Wiki journal may have a better chance than others, but only if it is prepared with the academic career paths and full proper code of conduct nuances considered (double-blind scholarly peer review, proper editorial board, PDFs with page numbers, etc.). dj ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal?
I wonder if a better place to innovate might be in the conduct of research, rather than the reporting, review and publication of research? While wiki speeds collaboration within a community, the research literature favors long-lasting contributions outside the community. Wiki or wiki-like collaboration tools might be better applied to stimulating and conducting high-quality research that will then usefully feed existing publications. I assume here that Wikipedia and the like have been sufficiently successful to raise more research questions and to enable more research methods than this community has time and/or ability to pursue. (If topics were scarce then they might be usefully hoarded.) Of course there are many ways to share methods and directions and lots have already been applied in this community. However, I get the impression that results still take months or years to produce. Aside: I have built a data mining tool and methodology, Exploratory Parsing, that can read all of Wikipedia in 10 seconds for a useful notion of read. I have also created a Federated Wiki that promotes wiki-like sharing without need for a common vision or agreed social norms. I intend to combine the two to make an experiment manager where setups are easily shared and where methodological improvements and/or research redirections easily build on other's work in progress. What would our environment have to be like before our collaborative cycle time is reduced to days? On Nov 4, 2012, at 2:27 PM, Chitu Okoli wrote: On one hand, I agree that readership is very important. On the other hand, I don't believe it is the most critical issue. I think the most critical matter in starting a new journal is to *attract high-quality submissions* --that is, to get researchers who do high quality work to submit some of their best research to the journal. If that can be achieved, then a high readership is virtually guaranteed. Other benefits should readily follow: high citations, ISI listing, etc. I've been reading all the comments on this thread carefully, and I see two distinct issues that are being mixed into one: (1) a new journal dedicated to research about wikis, and maybe also about related phenomena; and (2) a journal with new kind of wiki-based peer review system. I think it is best to rather keep these issues distinct. ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal?
Wondering further, I recently became acquainted with the Sage Bioinformatics Synapse platform: https://synapse.sagebase.org/ by way of a keynote at the O'Reilly Strata Rx conference by the Sage president, Stephen Friend; http://www.youtube.com/watch?v=m4Pvq4sldbQ A later talk (apparently not online) by Erich Huang made it clear that their goal is to move research to the open source, open access arena. Erich spoke in terms of putting everything from research journals, to data, to software used for analysis online (e.g. at GitHub), making it available for continuous peer review, evaluation, forking and evolution, to augment the way science is communicated. That notion seems very much in the spirit of Ward's Wiki Way. I am still investigating Synapse. I hope to know much more about it soon. Jack On Sun, Nov 4, 2012 at 4:57 PM, Ward Cunningham w...@c2.com wrote: I wonder if a better place to innovate might be in the conduct of research, rather than the reporting, review and publication of research? While wiki speeds collaboration within a community, the research literature favors long-lasting contributions outside the community. Wiki or wiki-like collaboration tools might be better applied to stimulating and conducting high-quality research that will then usefully feed existing publications. I assume here that Wikipedia and the like have been sufficiently successful to raise more research questions and to enable more research methods than this community has time and/or ability to pursue. (If topics were scarce then they might be usefully hoarded.) Of course there are many ways to share methods and directions and lots have already been applied in this community. However, I get the impression that results still take months or years to produce. Aside: I have built a data mining tool and methodology, Exploratory Parsing, that can read all of Wikipedia in 10 seconds for a useful notion of read. I have also created a Federated Wiki that promotes wiki-like sharing without need for a common vision or agreed social norms. I intend to combine the two to make an experiment manager where setups are easily shared and where methodological improvements and/or research redirections easily build on other's work in progress. What would our environment have to be like before our collaborative cycle time is reduced to days? On Nov 4, 2012, at 2:27 PM, Chitu Okoli wrote: On one hand, I agree that readership is very important. On the other hand, I don't believe it is the most critical issue. I think the most critical matter in starting a new journal is to *attract high-quality submissions* --that is, to get researchers who do high quality work to submit some of their best research to the journal. If that can be achieved, then a high readership is virtually guaranteed. Other benefits should readily follow: high citations, ISI listing, etc. I've been reading all the comments on this thread carefully, and I see two distinct issues that are being mixed into one: (1) a new journal dedicated to research about wikis, and maybe also about related phenomena; and (2) a journal with new kind of wiki-based peer review system. I think it is best to rather keep these issues distinct. ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal?
On Sun, Nov 4, 2012 at 4:57 PM, Ward Cunningham w...@c2.com wrote: Aside: I have built a data mining tool and methodology, Exploratory Parsing, that can read all of Wikipedia in 10 seconds for a useful notion of read. I have also created a Federated Wiki that promotes wiki-like sharing without need for a common vision or agreed social norms. I intend to combine the two to make an experiment manager where setups are easily shared and where methodological improvements and/or research redirections easily build on other's work in progress. What would our environment have to be like before our collaborative cycle time is reduced to days? I would highly encourage researchers interested in collaborative systems to take a look at Federated Wiki. Collaboration among experts is a very interesting potential use case, considering the way the wiki handles data and visualization, as well as the git-like way is allows for collaboration. -- Steven Walling https://wikimediafoundation.org/ ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal?
Le 5 nov. 2012 à 01:57, Ward Cunningham w...@c2.com a écrit : I wonder if a better place to innovate might be in the conduct of research, rather than the reporting, review and publication of research? While wiki speeds collaboration within a community, the research literature favors long-lasting contributions outside the community. Wiki or wiki-like collaboration tools might be better applied to stimulating and conducting high-quality research that will then usefully feed existing publications. I may sound a bit overractive, but can't we do both? I would easily imagine the following two-way system: *A wiki-laboratory, which hosts quick and less quick projects in progress. That could also include some reactive analysis by social scientists to recent wikimedian and collaborative knowledge phenomena. *A wiki-journal, that should give a definitive and quotable form to the preceding researches and enlarge the discussions to wider disciplinary debates. By publishing external submissions it could give some new food for thought to the wiki-laboratory. The relationships between the two structures would be a relevant analogy to the mutually-rewarding dialogue between the practical and theoretical side of scientific research. On Nov 4, 2012, at 2:27 PM, Chitu Okoli wrote: On one hand, I agree that readership is very important. On the other hand, I don't believe it is the most critical issue. I think the most critical matter in starting a new journal is to *attract high-quality submissions* --that is, to get researchers who do high quality work to submit some of their best research to the journal. If that can be achieved, then a high readership is virtually guaranteed. Other benefits should readily follow: high citations, ISI listing, etc. I've been reading all the comments on this thread carefully, and I see two distinct issues that are being mixed into one: (1) a new journal dedicated to research about wikis, and maybe also about related phenomena; and (2) a journal with new kind of wiki-based peer review system. I think it is best to rather keep these issues distinct. ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review
I think a compromise position is to use single-blind unless the authors request double-blind and are therefore prepared to undertake the ridiculous gymnastics required. Certainly (in computer science) I would agree that it is very hard for any established researcher publishing in their normal field to successfully disguise their identify. Sent from my iPad On 05/11/2012, at 8:30 AM, Chitu Okoli chitu.ok...@concordia.ca wrote: Although in theory double-blind review is superior to single-blind, in practice double-blind vs. single-blind review has very little to do with journal quality. It is much more a matter of disciplinary culture. (Single-blind: authors don't know who the reviewers are, but reviewers do know who authors are; Double-blind: neither authors nor reviewers know who each other are) Yes, I am certainly aware of studies that show that double-blind reviewing does indeed reduce the bias towards publishing famous researchers and reduces the bias against publishing work by minority researchers and women. So, I believe that double-blind reviewing is indeed meaningful. However, my general observation is that the decision for a journal to be double-blind or single-blind is mainly a matter of disciplinary tradition. To make a very gross generalization, social science journals are generally double-blind, whereas natural science and mathematical science journals are generally single-blind. This observation is very relevant to this discussion, because the wiki-research-l community sits in between this divide. My perception is that 90% of people who post on this list are in information systems (double-blind), computer science (single-blind) or information science (either double- or single-blind). If we can accept that single-blind journals can be considered as high-quality as well, then I feel a journal with wikified peer review would do much better being single-blind, especially if its subject matter is wiki-related topics. I understand that one of the primary reasons many journals decide against going double-blind is because of the ridiculous gymnastics that have to be undertaken in many cases to try to hide authors' identity. In computer science, where many researchers make their programs available on the Web, and much of their research concerns particular websites that they have developed, double-blinding would often be impossible if attempted--reviewers couldn't properly evaluate the work without knowing who created it. I think this is very much the case for a wiki-based peer review system for much wiki-related research. Of course, the official reviewers should remain anonymous. (I know that open peer review has often been proposed--authors and reviewers know each others' identities--and has even been experimented with on several occasions, but it does not seem to be a quality improvement--I can post citations if anyone is interested.) It is much easier to hide the identity of the official reviewers than it is to hide that of the authors, and the benefits of single-blinding are substantial and proven. ~ Chitu Dariusz Jemielniak a écrit : actually, with our community, it is not. What other journals die for, we have sort of provided. This is why a Wiki journal may have a better chance than others, but only if it is prepared with the academic career paths and full proper code of conduct nuances considered (double-blind scholarly peer review, proper editorial board, PDFs with page numbers, etc.). dj ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
