On 08/12/2013 05:57 AM, Andrei Alexandrescu wrote: > On 8/11/13 4:45 PM, Joseph Rushton Wakeling wrote: >> On Sunday, 11 August 2013 at 23:37:28 UTC, Andrei Alexandrescu wrote: >>> That's an odd thing to say seeing as a lot of CS academic research is >>> ten years ahead of the industry. >> >> I would personally venture to say that the publication practises of >> academia in general and CS in particular have many destructive and >> damaging aspects, and that industry-academia gap might be narrowed quite >> a bit if these were addressed. > > Could be improved, sure. Destructive and damaging - I'd be curious for some > substantiation.
In the case of CS in particular, the publication system is different from much of academia because it's so strongly based around conferences and conference proceedings. I'd say that's damaging in several ways. First, it means people write to the submission deadline rather than to their work having reached a satisfactory point of readiness. All other activities grind to a halt in the run-up to major conference deadlines -- you see students and postdocs in particular pulling all-nighters in order to make sure that everything gets done in time. Besides the health implications of that, such a last-minute rush has plenty of scope for making mistakes or introducing errors, errors that will be in the permanent academic record with little scope for correction (conference proceedings generally don't carry errata). There are also more direct sources of bias -- e.g. if the work is based on user surveys, the chances are all the people in the lab _not_ working towards a paper deadline will be shanghaied into completing those surveys, disrupting their own work and also ensuring that the results are based on a very skewed selection of the population. This pressure to deliver on deadline something that will be accepted by the conference can also lead to quite a superficial approach to the existing literature, with references skimmed quickly in order to find any random phrase that may support the current piece of work (even though on closer reading it may actually indicate the opposite). The second source of damage comes via the conference review process. Because conferences are all-or-nothing affairs -- you get accepted or you don't -- there's a strong tendency to submit multiple papers presenting different facets of essentially the same work to multiple different conferences, just to ensure that _something_ gets accepted. That means overwork both for the authors (who have to write all those extra papers) and also for conference referees, who have to deal with the resulting excess of papers. Reviewers are also working to deadlines, and with a lot of papers to assess in a short space of time (which is very disruptive to their other work), that can lead to snap and very superficial judgements. If there's a discrepancy in the amount of work that has to be done -- e.g. a "yes" means just a "yes", but a "no" means having to write a detailed report explaining why -- that can lead to accepting papers simply to lessen the workload. There are also financial aspects -- because most conferences (understandably) won't accept papers unless at least one author comes to present, it means that authors' ability to publish their work can be constrained by their labs' ability to fund travel, accommodation and conference fees rather than by the quality of what they've done. And finally, when all is done and dusted, the proceedings of conferences are almost invariably then locked up behind a publisher paywall, despite the fact that almost all the document preparation work is done by authors and conference volunteers. How many tech businesses can afford the annual subscriptions to digital libraries? (I'm thinking small startups here.) I suppose you could say that many of these issues are personal/professional failings of individual researchers or labs, but in my experience these failings are driven by the pressure to publish conference papers, and young researchers are pretty much trained to follow these working practices in order to succeed. What particularly frustrates me about this particular situation is that the justification for the current system -- that computer science is too fast-moving for journal publication to keep up with the latest results -- simply doesn't hold water in an age of electronic publication. It's habit and professional career structures, rather than the interests of research communication, that maintain the current system. I could go on, but I think these examples will serve as substantiation. :-)