How many people have signed up to be reviewers?

I don't think that scheduling is part of the review process and that can be done by the person/team "organizing" ApacheCon on behalf of the PMC.

To me review is looking at content for
- relevance
- quality of the presentations (suggest fixes to content, English, graphics, etc.)
This should result in a consensus score
- Perfect - ready for prime time
- Needs minor changes as documented by the reviewers
- Great topic but needs more work - perhaps a reviewer could volunteer to work with the presenter to get it ready if chosen
- Not recommended for topic or content reasons

The reviewers could also make non-binding recommendations about the balance between topics - marketing(why Cloudstack), Operations/implementation, Technical details, Roadmap, etc. based on what they have seen.

This should be used by the organizers to make the choices and organize the program. The organizers have the final say on the choice of presentations and schedule

Reviewers are there to help the process not control it.

I would be worried that you do not have enough reviewers rather than too many.
Then the work falls on the PMC and organizers.

When planning meetings, I would recommend that you clearly separate the roles and only invite the reviewers to the meetings about review. Get the list of presentation to present to the reviewers and decide if there are any instructions that you want to give to reviewers. I would recommend that you keep the organizing group small. Membership should be set by the PMC and should be people that are committed to the ApacheCon project and have the time. The committee can request help for specific tasks from others in the community who are not on the committee.

I would also recommend that organizers do not do reviews. They should read the finalists but if they do reviews, there may be a suggestion of favouring presentations that they reviewed. It also ensures that the organizers are not getting heat from rejected presenters - "it is the reviewers fault you did not get selected".

My advice is to get as many reviewers as you can so that no one is essential and each reviewer has a limited number of presentations to review but each presentation gets reviewed by multiple people. Also bear in mind that not all reviewers have the same ability to review each presentation. Reviews should be anonymous and only the summary comments given to the presenter. Reviewers of a presentation should be able to discuss the presentation during the review to make sure that reviewers do not feel isolated or get lost when they hit content that they don't understand fully.



Ron


On 01/04/2018 12:20 AM, Tutkowski, Mike wrote:
Thanks for the feedback, Will!

I agree with the approach you outlined.

Thanks for being so involved in the process! Let’s chat with Giles once he’s 
back to see if we can get your questions answered.

On Mar 31, 2018, at 10:14 PM, Will Stevens <wstev...@cloudops.com> wrote:

In the past the committee was chosen as a relatively small group in order
to make it easier to manage feedback.  In order to make it fair to everyone
in the community, I would suggest that instead of doing it with a small
group, we do it out in the open on a scheduled call.

We will have to get a list of the talks that are CloudStack specific from
ApacheCon, but that should be possible.

Once we have the talks selected, then a smaller number of us can work on
setting up the actual ordering and the details.

I have been quite involved so far.  Giles and I have been organizing the
sponsors, website and dealing with ApacheCon so far.  Obviously, Mike is
also working on this as well.

I think we are headed in the right direction on this.

Cheers,

Will

On Mar 31, 2018 11:49 PM, "Tutkowski, Mike" <mike.tutkow...@netapp.com>
wrote:

Hi Ron,

I am definitely open to working this however makes the most sense.

It looks like Will’s e-mail indicates that the process I suggested has been
followed in the past (which is how I recall, as well).

Let’s make sure I understood Will correctly.

Will – Are you, in fact, indicating that what I was suggesting is how we
have reviewed the CFP in the past? If so, are you able to address Ron’s
concerns?

Also, Will – I am not sure about a hackathon. Let’s chat with Giles once
he’s back from vacation since he’s been the most involved with organizing
the CloudStack track within ApacheCon.

Thanks!

Mike


On 3/31/18, 9:00 PM, "Ron Wheeler" <rwhee...@artifact-software.com> wrote:

    I am not sure about your concern in that case.
    I am not sure why people not interested in Cloudstack would volunteer as
    reviewers and want to pick bad presentations.

    I would be more worried that there are not enough good presentations
    proposed rather than some meritorious presentation will get rejected due
    to "outsiders" voting it down in favour of less useful presentations.

    It may be tricky to get balance if that means taking "bad" proposals
    that can not be fixed that cover topics that are in areas that are not
    otherwise covered at the expense of great presentations that are in
    areas with many choices.

    We should wait to see how many presentations have to be rejected and the
    number of reviewers before getting too exercised over the loyalty of
    reviewers.

    Getting more reviewers is likely the most effective way to see that a
    wider range of topics is covered.

    Ron

    On 31/03/2018 7:15 PM, Tutkowski, Mike wrote:
Hi Ron,

 From what I understand, the CloudStack proposals will be mixed in
with all of the ApacheCon proposals.
In the past when I’ve participated in these CloudStack panels to
review proposals, we had to compare each proposal against the others to
arrive at a balance of topics (i.e. not all networking focused, not all
XenServer focused, etc.) and to suggest improvements for proposals that we
did not accept for other reasons.
 From what I understand (but Giles can comment further on this), we
have a track at ApacheCon and will need to fill it with X number of
presentations. To do this, it seems like a CloudStack-focused panel would
be a good approach, but I am definitely open to another approach. We don’t
want to exclude anyone (in or out of the CloudStack Community) who might
like to provide input. Anyone who is interested would, of course, be free
to join us in combing through the proposals.
We don’t need to get started on this right away. The CFP just closed
yesterday. Let’s wait for feedback from Giles (who is currently on
vacation) and go from there.
Thanks!
Mike

On 3/31/18, 6:59 AM, "Ron Wheeler" <rwhee...@artifact-software.com>
wrote:
     Is this a real concern?
     Why would a large number of Apache contributors who are not
interested
     in Cloudstack (enough to outvote those "part of the Cloudstack
     community") get involved as reviewers

     Reviewing involves some commitment of time so I am hard pressed
to guess
     why some Apache contributor would volunteer to do the work in
order to
     veto a presentation that they have not yet seen or have no
interest in
     seeing.

     Are we guaranteed a fixed number of hours of presentations or is
the
     review process part of the allocation of overall time?

     On what basis can some group veto a presentation?
     That would seem to be a very strong action and I would hope that
it
     requires a strong reason.

     OTOH if a large??? number of Apache contributors (regardless of
their
     affiliation) say that a presentation has serious issues or very
limited
     interest, that would seem to be a red flag that the presentation
     requires improvement or needs to be dropped in favour of another
     Cloudstack presentation, if it can not be fixed.

     We should also be aware that this is an opportunity to "market"
     Cloudstack to the broader Apache community.
     Outside reviewers might have valuable input into how
presentations can
     attract new adopters or be clearer to the broader DevOps
community.
     We also need to remember that we do have an active community and
other
     opportunities during the year to present presentations that do
not get
     selected for this conference.

     If their is a real fear that a lot of "outsiders" are going to
disrupt
     the review process, a more reasonable response would seem to be
to get
     more reviewers from the community.

     I have volunteered already.

     Ron

     On 30/03/2018 11:11 PM, Tutkowski, Mike wrote:
Hi Rafael,

It’s a little bit tricky in our particular situation. Allow me
to explain:
As you are likely aware, the CloudStack Collaboration
Conference will be held as a track in the larger ApacheCon conference in
Montreal this coming September.
It is true, as you say, that anyone who wishes to do so can
contribute to reviewing the CFP for ApacheCon.
What is a bit of a concern, however, is that we might get
certain CloudStack CFP proposals vetoed by people who are not, per se, a
part of our community.
That being the case, I have contacted the organizers for
ApacheCon to see if there is some way we can section off the CloudStack CFP
from the larger ApacheCon CFP for review purposes.
Assuming we can do this, the panel that I am proposing here
would handle this review task.
I hope that helps clarify the situation.

Thanks!
Mike

On 3/30/18, 8:38 AM, "Rafael Weingärtner" <
rafaelweingart...@gmail.com> wrote:
     Are we going to have a separated review process?

     I thought anybody could go here [1] and apply for a
reviewer position and
     start reviewing. Well, that is what I did. I have already
reviewed some
     CloudStack proposals (of course I did not review mines).
After asking to
     review presentations, Rich has giving me access to the
system. I thought
     everybody interest in helping was going to do the same.

     [1]
https://cfp.apachecon.com/conference.html?apachecon-north-america-2018

     On Wed, Mar 28, 2018 at 4:05 AM, Swen - swen.io <
m...@swen.io> wrote:
Hi Mike,

congrats!

I can help sort through presentations.

Best regards,
Swen

-----Ursprüngliche Nachricht-----
Von: Tutkowski, Mike [mailto:mike.tutkow...@netapp.com]
Gesendet: Dienstag, 27. März 2018 21:40
An: dev@cloudstack.apache.org;
us...@cloudstack.apache.org
Betreff: Committee to Sort through CCC Presentation
Submissions
Hi everyone,

As you may be aware, this coming September in Montreal,
the CloudStack
Community will be hosting the CloudStack Collaboration
Conference:
http://ca.cloudstackcollab.org/

Even though the event is six months away, we are on a
tight schedule with
regards to the Call For Participation (CFP):

https://www.apachecon.com/acna18/schedule.html

If you are interested in submitting a talk, please do
so before March 30th.
That being said, as usual, we will have need of a small
committee to sort
through these presentation submissions.

If you are interested in helping out in this process,
please reply to this
message.

Thanks!
Mike




     --
     Rafael Weingärtner




--
Ron Wheeler
President
Artifact Software Inc
email: rwhee...@artifact-software.com
skype: ronaldmwheeler
phone: 866-970-2435, ext 102

Reply via email to