We still need to manage the review process and make sure that it is adequately staffed.

The allocation of presentations to reviewers has to be managed to be sure that the reviewers have the support that they need to do a proper review and that the reviews get done.

Ron


On 05/04/2018 11:45 AM, Tutkowski, Mike wrote:
Perfect…then, unless anyone has other opinions they’d like to share on the 
topic, let’s follow that approach.

On 4/5/18, 9:43 AM, "Rafael Weingärtner" <rafaelweingart...@gmail.com> wrote:

     That is exactly it.
On Thu, Apr 5, 2018 at 12:37 PM, Tutkowski, Mike <mike.tutkow...@netapp.com>
     wrote:
> Hi Rafael,
     >
     > I think as long as we (the CloudStack Community) have the final say on 
how
     > we fill our allotted slots in the CloudStack track of ApacheCon in
     > Montreal, then it’s perfectly fine for us to leverage Apache’s normal
     > review process to gather all the feedback from the larger Apache 
Community.
     >
     > As you say, we could wait for the feedback to come in via that mechanism
     > and then, as per Will’s earlier comments, we could advertise on our 
users@
     > and dev@ mailing lists when we plan to get together for a call and make
     > final decisions on the CFP.
     >
     > Is that, in fact, what you were thinking, Rafael?
     >
     > Talk to you soon,
     > Mike
     >
     > On 4/4/18, 2:58 PM, "Rafael Weingärtner" <rafaelweingart...@gmail.com>
     > wrote:
     >
     >     I think everybody that “raised their hands here” already signed up to
     >     review.
     >
     >     Mike, what about if we only gathered the reviews from Apache main
     > review
     >     system, and then we use that to decide which presentations will get 
in
     >     CloudStack tracks? Then, we reduce the work on our side (we also 
remove
     >     bias…). I do believe that the review from other peers from Apache
     > community
     >     (even the one outside from our small community) will be fair and
     > technical
     >     (meaning, without passion and or favoritism).
     >
     >     Having said that, I think we only need a small group of PMCs to 
gather
     > the
     >     results and out of the best ranked proposals, we pick the ones to our
     >     tracks.
     >
     >     What do you (Mike) and others think?
     >
     >
     >     On Tue, Apr 3, 2018 at 5:07 PM, Tutkowski, Mike <
     > mike.tutkow...@netapp.com>
     >     wrote:
     >
     >     > Hi Ron,
     >     >
     >     > I don’t actually have insight into how many people have currently
     > signed
     >     > up online to be CFP reviewers for ApacheCon. At present, I’m only
     > aware of
     >     > those who have responded to this e-mail chain.
     >     >
     >     > We should be able to find out more in the coming weeks. We’re still
     > quite
     >     > early in the process.
     >     >
     >     > Thanks for your feedback,
     >     > Mike
     >     >
     >     > On 4/1/18, 9:18 AM, "Ron Wheeler" <rwhee...@artifact-software.com>
     > wrote:
     >     >
     >     >     How many people have signed up to be reviewers?
     >     >
     >     >     I don't think that scheduling is part of the review process and
     > that
     >     > can
     >     >     be done by the person/team "organizing" ApacheCon on behalf of
     > the PMC.
     >     >
     >     >     To me review is looking at content for
     >     >     - relevance
     >     >     - quality of the presentations (suggest fixes to content,
     > English,
     >     >     graphics, etc.)
     >     >     This should result in a consensus score
     >     >     - Perfect - ready for prime time
     >     >     - Needs minor changes as documented by the reviewers
     >     >     - Great topic but needs more work - perhaps a reviewer could
     > volunteer
     >     >     to work with the presenter to get it ready if chosen
     >     >     - Not recommended for topic or content reasons
     >     >
     >     >     The reviewers could also make non-binding recommendations about
     > the
     >     >     balance between topics - marketing(why Cloudstack),
     >     >     Operations/implementation, Technical details, Roadmap, etc.
     > based on
     >     >     what they have seen.
     >     >
     >     >     This should be used by the organizers to make the choices and
     > organize
     >     >     the program.
     >     >     The organizers have the final say on the choice of 
presentations
     > and
     >     >     schedule
     >     >
     >     >     Reviewers are there to help the process not control it.
     >     >
     >     >     I would be worried that you do not have enough reviewers rather
     > than
     >     > too
     >     >     many.
     >     >     Then the work falls on the PMC and organizers.
     >     >
     >     >     When planning meetings, I would recommend that you clearly
     > separate the
     >     >     roles and only invite the reviewers to the meetings about
     > review. Get
     >     >     the list of presentation to present to the reviewers and decide
     > if
     >     > there
     >     >     are any instructions that you want to give to reviewers.
     >     >     I would recommend that you keep the organizing group small.
     > Membership
     >     >     should be set by the PMC and should be people that are 
committed
     > to the
     >     >     ApacheCon project and have the time. The committee can request
     > help for
     >     >     specific tasks from others in the community who are not on the
     >     > committee.
     >     >
     >     >     I would also recommend that organizers do not do reviews. They
     > should
     >     >     read the finalists but if they do reviews, there may be a
     > suggestion of
     >     >     favouring presentations that they reviewed. It also ensures 
that
     > the
     >     >     organizers are not getting heat from rejected presenters - "it
     > is the
     >     >     reviewers fault you did not get selected".
     >     >
     >     >     My advice is to get as many reviewers as you can so that no one
     > is
     >     >     essential and each reviewer has a limited number of
     > presentations to
     >     >     review but each presentation gets reviewed by multiple people.
     > Also
     >     > bear
     >     >     in mind that not all reviewers have the same ability to review
     > each
     >     >     presentation.
     >     >     Reviews should be anonymous and only the summary comments given
     > to the
     >     >     presenter. Reviewers of a presentation should be able to 
discuss
     > the
     >     >     presentation during the review to make sure that reviewers do
     > not feel
     >     >     isolated or get lost when they hit content that they don't
     > understand
     >     > fully.
     >     >
     >     >
     >     >
     >     >     Ron
     >     >
     >     >
     >     >     On 01/04/2018 12:20 AM, Tutkowski, Mike wrote:
     >     >     > Thanks for the feedback, Will!
     >     >     >
     >     >     > I agree with the approach you outlined.
     >     >     >
     >     >     > Thanks for being so involved in the process! Let’s chat with
     > Giles
     >     > once he’s back to see if we can get your questions answered.
     >     >     >
     >     >     >> On Mar 31, 2018, at 10:14 PM, Will Stevens <
     > wstev...@cloudops.com>
     >     > wrote:
     >     >     >>
     >     >     >> In the past the committee was chosen as a relatively small
     > group in
     >     > order
     >     >     >> to make it easier to manage feedback.  In order to make it
     > fair to
     >     > everyone
     >     >     >> in the community, I would suggest that instead of doing it
     > with a
     >     > small
     >     >     >> group, we do it out in the open on a scheduled call.
     >     >     >>
     >     >     >> We will have to get a list of the talks that are CloudStack
     >     > specific from
     >     >     >> ApacheCon, but that should be possible.
     >     >     >>
     >     >     >> Once we have the talks selected, then a smaller number of us
     > can
     >     > work on
     >     >     >> setting up the actual ordering and the details.
     >     >     >>
     >     >     >> I have been quite involved so far.  Giles and I have been
     >     > organizing the
     >     >     >> sponsors, website and dealing with ApacheCon so far.
     > Obviously,
     >     > Mike is
     >     >     >> also working on this as well.
     >     >     >>
     >     >     >> I think we are headed in the right direction on this.
     >     >     >>
     >     >     >> Cheers,
     >     >     >>
     >     >     >> Will
     >     >     >>
     >     >     >> On Mar 31, 2018 11:49 PM, "Tutkowski, Mike" <
     >     > mike.tutkow...@netapp.com>
     >     >     >> wrote:
     >     >     >>
     >     >     >> Hi Ron,
     >     >     >>
     >     >     >> I am definitely open to working this however makes the most
     > sense.
     >     >     >>
     >     >     >> It looks like Will’s e-mail indicates that the process I
     > suggested
     >     > has been
     >     >     >> followed in the past (which is how I recall, as well).
     >     >     >>
     >     >     >> Let’s make sure I understood Will correctly.
     >     >     >>
     >     >     >> Will – Are you, in fact, indicating that what I was
     > suggesting is
     >     > how we
     >     >     >> have reviewed the CFP in the past? If so, are you able to
     > address
     >     > Ron’s
     >     >     >> concerns?
     >     >     >>
     >     >     >> Also, Will – I am not sure about a hackathon. Let’s chat 
with
     > Giles
     >     > once
     >     >     >> he’s back from vacation since he’s been the most involved 
with
     >     > organizing
     >     >     >> the CloudStack track within ApacheCon.
     >     >     >>
     >     >     >> Thanks!
     >     >     >>
     >     >     >> Mike
     >     >     >>
     >     >     >>
     >     >     >> On 3/31/18, 9:00 PM, "Ron Wheeler" <
     > rwhee...@artifact-software.com>
     >     > wrote:
     >     >     >>
     >     >     >>     I am not sure about your concern in that case.
     >     >     >>     I am not sure why people not interested in Cloudstack
     > would
     >     > volunteer as
     >     >     >>     reviewers and want to pick bad presentations.
     >     >     >>
     >     >     >>     I would be more worried that there are not enough good
     >     > presentations
     >     >     >>     proposed rather than some meritorious presentation will
     > get
     >     > rejected due
     >     >     >>     to "outsiders" voting it down in favour of less useful
     >     > presentations.
     >     >     >>
     >     >     >>     It may be tricky to get balance if that means taking 
"bad"
     >     > proposals
     >     >     >>     that can not be fixed that cover topics that are in 
areas
     > that
     >     > are not
     >     >     >>     otherwise covered at the expense of great presentations
     > that
     >     > are in
     >     >     >>     areas with many choices.
     >     >     >>
     >     >     >>     We should wait to see how many presentations have to be
     >     > rejected and the
     >     >     >>     number of reviewers before getting too exercised over 
the
     >     > loyalty of
     >     >     >>     reviewers.
     >     >     >>
     >     >     >>     Getting more reviewers is likely the most effective way
     > to see
     >     > that a
     >     >     >>     wider range of topics is covered.
     >     >     >>
     >     >     >>     Ron
     >     >     >>
     >     >     >>>     On 31/03/2018 7:15 PM, Tutkowski, Mike wrote:
     >     >     >>> Hi Ron,
     >     >     >>>
     >     >     >>>  From what I understand, the CloudStack proposals will be
     > mixed in
     >     >     >> with all of the ApacheCon proposals.
     >     >     >>> In the past when I’ve participated in these CloudStack
     > panels to
     >     >     >> review proposals, we had to compare each proposal against 
the
     >     > others to
     >     >     >> arrive at a balance of topics (i.e. not all networking
     > focused, not
     >     > all
     >     >     >> XenServer focused, etc.) and to suggest improvements for
     > proposals
     >     > that we
     >     >     >> did not accept for other reasons.
     >     >     >>>  From what I understand (but Giles can comment further on
     > this), we
     >     >     >> have a track at ApacheCon and will need to fill it with X
     > number of
     >     >     >> presentations. To do this, it seems like a 
CloudStack-focused
     > panel
     >     > would
     >     >     >> be a good approach, but I am definitely open to another
     > approach.
     >     > We don’t
     >     >     >> want to exclude anyone (in or out of the CloudStack
     > Community) who
     >     > might
     >     >     >> like to provide input. Anyone who is interested would, of
     > course,
     >     > be free
     >     >     >> to join us in combing through the proposals.
     >     >     >>> We don’t need to get started on this right away. The CFP 
just
     >     > closed
     >     >     >> yesterday. Let’s wait for feedback from Giles (who is
     > currently on
     >     >     >> vacation) and go from there.
     >     >     >>> Thanks!
     >     >     >>> Mike
     >     >     >>>
     >     >     >>> On 3/31/18, 6:59 AM, "Ron Wheeler" <
     > rwhee...@artifact-software.com
     >     > >
     >     >     >> wrote:
     >     >     >>>      Is this a real concern?
     >     >     >>>      Why would a large number of Apache contributors who 
are
     > not
     >     >     >> interested
     >     >     >>>      in Cloudstack (enough to outvote those "part of the
     > Cloudstack
     >     >     >>>      community") get involved as reviewers
     >     >     >>>
     >     >     >>>      Reviewing involves some commitment of time so I am 
hard
     >     > pressed
     >     >     >> to guess
     >     >     >>>      why some Apache contributor would volunteer to do the
     > work in
     >     >     >> order to
     >     >     >>>      veto a presentation that they have not yet seen or 
have
     > no
     >     >     >> interest in
     >     >     >>>      seeing.
     >     >     >>>
     >     >     >>>      Are we guaranteed a fixed number of hours of
     > presentations or
     >     > is
     >     >     >> the
     >     >     >>>      review process part of the allocation of overall time?
     >     >     >>>
     >     >     >>>      On what basis can some group veto a presentation?
     >     >     >>>      That would seem to be a very strong action and I would
     > hope
     >     > that
     >     >     >> it
     >     >     >>>      requires a strong reason.
     >     >     >>>
     >     >     >>>      OTOH if a large??? number of Apache contributors
     > (regardless
     >     > of
     >     >     >> their
     >     >     >>>      affiliation) say that a presentation has serious 
issues
     > or
     >     > very
     >     >     >> limited
     >     >     >>>      interest, that would seem to be a red flag that the
     >     > presentation
     >     >     >>>      requires improvement or needs to be dropped in favour 
of
     >     > another
     >     >     >>>      Cloudstack presentation, if it can not be fixed.
     >     >     >>>
     >     >     >>>      We should also be aware that this is an opportunity to
     >     > "market"
     >     >     >>>      Cloudstack to the broader Apache community.
     >     >     >>>      Outside reviewers might have valuable input into how
     >     >     >> presentations can
     >     >     >>>      attract new adopters or be clearer to the broader 
DevOps
     >     >     >> community.
     >     >     >>>      We also need to remember that we do have an active
     > community
     >     > and
     >     >     >> other
     >     >     >>>      opportunities during the year to present presentations
     > that do
     >     >     >> not get
     >     >     >>>      selected for this conference.
     >     >     >>>
     >     >     >>>      If their is a real fear that a lot of "outsiders" are
     > going to
     >     >     >> disrupt
     >     >     >>>      the review process, a more reasonable response would
     > seem to
     >     > be
     >     >     >> to get
     >     >     >>>      more reviewers from the community.
     >     >     >>>
     >     >     >>>      I have volunteered already.
     >     >     >>>
     >     >     >>>      Ron
     >     >     >>>
     >     >     >>>>      On 30/03/2018 11:11 PM, Tutkowski, Mike wrote:
     >     >     >>>> Hi Rafael,
     >     >     >>>>
     >     >     >>>> It’s a little bit tricky in our particular situation. 
Allow
     > me
     >     >     >> to explain:
     >     >     >>>> As you are likely aware, the CloudStack Collaboration
     >     >     >> Conference will be held as a track in the larger ApacheCon
     >     > conference in
     >     >     >> Montreal this coming September.
     >     >     >>>> It is true, as you say, that anyone who wishes to do so 
can
     >     >     >> contribute to reviewing the CFP for ApacheCon.
     >     >     >>>> What is a bit of a concern, however, is that we might get
     >     >     >> certain CloudStack CFP proposals vetoed by people who are
     > not, per
     >     > se, a
     >     >     >> part of our community.
     >     >     >>>> That being the case, I have contacted the organizers for
     >     >     >> ApacheCon to see if there is some way we can section off the
     >     > CloudStack CFP
     >     >     >> from the larger ApacheCon CFP for review purposes.
     >     >     >>>> Assuming we can do this, the panel that I am proposing 
here
     >     >     >> would handle this review task.
     >     >     >>>> I hope that helps clarify the situation.
     >     >     >>>>
     >     >     >>>> Thanks!
     >     >     >>>> Mike
     >     >     >>>>
     >     >     >>>> On 3/30/18, 8:38 AM, "Rafael Weingärtner" <
     >     >     >> rafaelweingart...@gmail.com> wrote:
     >     >     >>>>      Are we going to have a separated review process?
     >     >     >>>>
     >     >     >>>>      I thought anybody could go here [1] and apply for a
     >     >     >> reviewer position and
     >     >     >>>>      start reviewing. Well, that is what I did. I have
     > already
     >     >     >> reviewed some
     >     >     >>>>      CloudStack proposals (of course I did not review
     > mines).
     >     >     >> After asking to
     >     >     >>>>      review presentations, Rich has giving me access to 
the
     >     >     >> system. I thought
     >     >     >>>>      everybody interest in helping was going to do the 
same.
     >     >     >>>>
     >     >     >>>>      [1]
     >     >     >> https://cfp.apachecon.com/conference.html?apachecon-
     >     > north-america-2018
     >     >     >>>>
     >     >     >>>>      On Wed, Mar 28, 2018 at 4:05 AM, Swen - swen.io <
     >     >     >> m...@swen.io> wrote:
     >     >     >>>>> Hi Mike,
     >     >     >>>>>
     >     >     >>>>> congrats!
     >     >     >>>>>
     >     >     >>>>> I can help sort through presentations.
     >     >     >>>>>
     >     >     >>>>> Best regards,
     >     >     >>>>> Swen
     >     >     >>>>>
     >     >     >>>>> -----Ursprüngliche Nachricht-----
     >     >     >>>>> Von: Tutkowski, Mike [mailto:mike.tutkow...@netapp.com]
     >     >     >>>>> Gesendet: Dienstag, 27. März 2018 21:40
     >     >     >>>>> An: dev@cloudstack.apache.org;
     >     >     >> us...@cloudstack.apache.org
     >     >     >>>>> Betreff: Committee to Sort through CCC Presentation
     >     >     >> Submissions
     >     >     >>>>> Hi everyone,
     >     >     >>>>>
     >     >     >>>>> As you may be aware, this coming September in Montreal,
     >     >     >> the CloudStack
     >     >     >>>>> Community will be hosting the CloudStack Collaboration
     >     >     >> Conference:
     >     >     >>>>> http://ca.cloudstackcollab.org/
     >     >     >>>>>
     >     >     >>>>> Even though the event is six months away, we are on a
     >     >     >> tight schedule with
     >     >     >>>>> regards to the Call For Participation (CFP):
     >     >     >>>>>
     >     >     >>>>> https://www.apachecon.com/acna18/schedule.html
     >     >     >>>>>
     >     >     >>>>> If you are interested in submitting a talk, please do
     >     >     >> so before March 30th.
     >     >     >>>>> That being said, as usual, we will have need of a small
     >     >     >> committee to sort
     >     >     >>>>> through these presentation submissions.
     >     >     >>>>>
     >     >     >>>>> If you are interested in helping out in this process,
     >     >     >> please reply to this
     >     >     >>>>> message.
     >     >     >>>>>
     >     >     >>>>> Thanks!
     >     >     >>>>> Mike
     >     >     >>>>>
     >     >     >>>>>
     >     >     >>>>>
     >     >     >>>>
     >     >     >>>>      --
     >     >     >>>>      Rafael Weingärtner
     >     >     >>>>
     >     >     >>>>
     >     >     >>>
     >     >
     >     >     --
     >     >     Ron Wheeler
     >     >     President
     >     >     Artifact Software Inc
     >     >     email: rwhee...@artifact-software.com
     >     >     skype: ronaldmwheeler
     >     >     phone: 866-970-2435, ext 102
     >     >
     >     >
     >     >
     >     >
     >
     >
     >     --
     >     Rafael Weingärtner
     >
     >
     >
--
     Rafael Weingärtner

--
Ron Wheeler
President
Artifact Software Inc
email: rwhee...@artifact-software.com
skype: ronaldmwheeler
phone: 866-970-2435, ext 102

Reply via email to