Dear Joe, All, I find all points correct, with one "doubt" on 5. TPC Meeting.
I find that the best practice is a substantial on-line discussion among the reviewers rather than a TPC Meeting whose cost for the conference (specially small ones) is really high. A very good example here can be taken from P2P. Infocom tends to be less effective, with many reviewers not actively involved in trying to make a real discussion. If there has not been a preparation with discussion of different reviews, TPC meetings often results in huge effort without a substantial improvement of the reviews' quality. My 2c Renato On 5/29/13 8:05 PM, Joe Touch wrote: > Hi, all, > > As part of the ComSoc technical cosponsorship (TCS) process, TCs are > supposed to nominate at least two members of the TPC who will monitor > the review process. > > However, there doesn't appear to be any guidelines for providing > feedback on that process. > > I've drafted the following, which I hope will open a discussion on this > issue. If it evolves into something useful, perhaps it can be posted on > the TC websites for use by those appointed to monitor TC-endorsed TCS'd > meetings. > > NB: I've cross-posted this to TCCC, ITC, and TCHSN, which are where I > participate primarily; if any other TC has suggestions, please take the > discussion to the TCCC list if possible. > > Thanks, > > Joe > > ----------------------------------------------------------------- > > Rating system: > EXCELLENT best-practice to be aspired to > AVERAGE acceptable practice > DEFICIENT cause for concern for ComSoc involvement > > 1. TPC participation invitation E/A/D > > E = before first Call for Papers (CFP) issued > A = before CFP submissions due > D = after CFP submissions due > > 2. involvement in CFP promotion E/A/D > > E = invited to forward CFP and submit > A = invited to submit > D = neither > > 3. paper assignment for review E/A/D > > E = invited to select papers based on expertise and > abstracts/titles > A = invited to select based on topic area > D = not invited to select > > NB: "everyone reviews all" = E > > 4. paper review format E/A/D > > E = includes rank, feedback for author, and private > feedback for TPC discussion > A = includes rank and author feedback > D = includes only rank > > 5. TPC meeting E/A/D > > E = in-person meeting with support for remote > A = in-person with no remote support or only telecon or e-mail > D = no meeting > > 6. paper review process E/A/D > > E = considers average rank AND outlier info, discussion points > also based on natural 'gap' in evaluation > A = considers average rank based on natural gap in evaluation > D = considers rank only > > 7. paper reviews returned E/A/D > > E = >=3 substantive reviews returned with rank and > comments for the authors > A = >=3 substantive reviews returned with rank and > at least a rationale for rejects > D = <3 reviews for some papers, reviews not returned at all, > or only rank provided > > 7. paper accept rate E/A/D > > E = <=50%, based on natural gap in paper evaluation > A = <=50%, not based on 'gap' > D = >50% > > ------------------------------------------------------ > _______________________________________________ > IEEE Communications Society Tech. Committee on Computer Communications > (TCCC) - for discussions on computer networking and communication. > [email protected] > https://lists.cs.columbia.edu/cucslists/listinfo/tccc > _______________________________________________ IEEE Communications Society Tech. Committee on Computer Communications (TCCC) - for discussions on computer networking and communication. [email protected] https://lists.cs.columbia.edu/cucslists/listinfo/tccc
