An update on this: After adding the initial maintainer list, we got feedback to
add more maintainers for some components, so we added four others (Josh Rosen
for core API, Mark Hamstra for scheduler, Shivaram Venkataraman for MLlib and
Xiangrui Meng for Python). We also decided to lower the
.nabble.com/VOTE-Designating-maintainers-for-some-Spark-components-tp9115p9281.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
Thanks everyone for voting on this. With all of the PMC votes being for, the
vote passes, but there were some concerns that I wanted to address for everyone
who brought them up, as well as in the wording we will use for this policy.
First, like every Apache project, Spark follows the Apache
+1 (binding)
I see this as a way to increase transparency and efficiency around a
process that already informally exists, with benefits to both new
contributors and committers. For new contributors, it makes clear who they
should ping about a pending patch. For committers, it's a good reference
-1 (not binding, +1 for maintainer, -1 for sign off)
Agree with Greg and Vinod. In the beginning, everything is better
(more efficient, more focus), but after some time, fighting begins.
Code style is the most hot topic to fight (we already saw it in some
PRs). If two committers (one of them is
+1 (binding)
I agree with the proposal that it just formalizes what we have been
doing till now, and will increase the efficiency and focus of the
review process.
To address Davies' concern, I agree coding style is often a hot topic
of contention. But that is just an indication that our
Sorry for my last email, I misunderstood the proposal here, all the
committer still have equal -1 to all the code changes.
Also, as mentioned in the proposal, the sign off only happens to
public API and architect, something like discussion about code style
things are still the same.
So, I'd
+1 (binding)
On 8 Nov 2014 07:26, Davies Liu dav...@databricks.com wrote:
Sorry for my last email, I misunderstood the proposal here, all the
committer still have equal -1 to all the code changes.
Also, as mentioned in the proposal, the sign off only happens to
public API and architect,
+1 (binding)
Ankur http://www.ankurdave.com/
On Wed, Nov 5, 2014 at 5:31 PM, Matei Zaharia matei.zaha...@gmail.com
wrote:
I'd like to formally call a [VOTE] on this model, to last 72 hours. The
[VOTE] will end on Nov 8, 2014 at 6 PM PST.
+1 (binding)
For tickets which span across multiple components, will it need to be
approved by all maintainers? For example, I'm working on the Python
bindings of GraphX where code is added to both Python and GraphX modules.
Thanks,
-Kushal.
On Thu, Nov 6, 2014 at 12:02 AM, Ankur Dave
+1 overall
also +1 to Sandy's suggestion to getting build maintainers as well.
On Wed, Nov 5, 2014 at 7:57 PM, Sandy Ryza sandy.r...@cloudera.com wrote:
This seems like a good idea.
An area that wasn't listed, but that I think could strongly benefit from
maintainers, is the build. Having
+1 (binding)
On Thu, Nov 6, 2014 at 4:02 PM, Ankur Dave ankurd...@gmail.com wrote:
+1 (binding)
Ankur http://www.ankurdave.com/
On Wed, Nov 5, 2014 at 5:31 PM, Matei Zaharia matei.zaha...@gmail.com
wrote:
I'd like to formally call a [VOTE] on this model, to last 72 hours. The
[VOTE]
Matei,
I saw that you're listed as a maintainer for ~6 different subcomponents,
and on over half of those, you're only the 2nd person. My concern is that
you would be stretched thin and maybe wouldn't be able to work as a back
up on all of those subcomponents. Are you planning on adding more
+1.
Tom
On Wednesday, November 5, 2014 9:21 PM, Matei Zaharia
matei.zaha...@gmail.com wrote:
BTW, my own vote is obviously +1 (binding).
Matei
On Nov 5, 2014, at 5:31 PM, Matei Zaharia matei.zaha...@gmail.com wrote:
Hi all,
I wanted to share a discussion we've been having on
+1
Sean
On Nov 5, 2014, at 6:32 PM, Matei Zaharia matei.zaha...@gmail.com wrote:
Hi all,
I wanted to share a discussion we've been having on the PMC list, as well as
call for an official vote on it on a public list. Basically, as the Spark
project scales up, we need to define a model to
+1
The app to track PRs based on component is a great idea...
On Thu, Nov 6, 2014 at 8:47 AM, Sean McNamara sean.mcnam...@webtrends.com
wrote:
+1
Sean
On Nov 5, 2014, at 6:32 PM, Matei Zaharia matei.zaha...@gmail.com wrote:
Hi all,
I wanted to share a discussion we've been having on
+1 (binding)
—
Sent from Mailbox
On Thu, Nov 6, 2014 at 6:52 PM, Debasish Das debasish.da...@gmail.com
wrote:
+1
The app to track PRs based on component is a great idea...
On Thu, Nov 6, 2014 at 8:47 AM, Sean McNamara sean.mcnam...@webtrends.com
wrote:
+1
Sean
On Nov 5, 2014, at 6:32
+1 (binding).
(our pull request browsing tool is open-source, by the way; contributions
welcome: https://github.com/databricks/spark-pr-dashboard)
On Thu, Nov 6, 2014 at 9:28 AM, Nick Pentreath nick.pentre...@gmail.com
wrote:
+1 (binding)
—
Sent from Mailbox
On Thu, Nov 6, 2014 at 6:52
Hi Matei,
Good call on scaling the project itself. Identifying domain experts in
different areas is a good thing. But I have some questions about the
implementation. Here's my understanding of the proposal:
(1) The PMC votes on a list of components and their maintainers. Changes to
that list
Hi BC,
The point is exactly to ensure that the maintainers have looked at each patch
to that component and consider it to fit consistently into its architecture.
The issue is not about rogue committers, it's about making sure that changes
don't accidentally sneak in that we want to roll back,
On Thu, Nov 6, 2014 at 11:25 AM, Matei Zaharia matei.zaha...@gmail.com
wrote:
snip
Ultimately, the core motivation is that the project has grown to the point
where it's hard to expect every committer to have full understanding of
every component. Some committers know a ton about systems but
I think new committers might or might not be maintainers (it would
depend on the PMC vote). I don't think it would affect what you could
merge, you can merge in any part of the source tree, you just need to
get sign off if you want to touch a public API or make major
architectural changes. Most
In Cloudstack, I believe one becomes a maintainer first for a subset of
modules, before he/she becomes a proven maintainter who has commit rights on
the entire source tree.
So would it make sense to go that route, and have committers voted in as
maintainers for certain parts of the
-1 (non-binding)
This is an idea that runs COMPLETELY counter to the Apache Way, and is
to be severely frowned up. This creates *unequal* ownership of the
codebase.
Each Member of the PMC should have *equal* rights to all areas of the
codebase until their purview. It should not be subjected to
Hey Greg,
Regarding subversion - I think the reference is to partial vs full
committers here:
https://subversion.apache.org/docs/community-guide/roles.html
- Patrick
On Thu, Nov 6, 2014 at 4:18 PM, Greg Stein gst...@gmail.com wrote:
-1 (non-binding)
This is an idea that runs COMPLETELY
In fact, if you look at the subversion commiter list, the majority of
people here have commit access only for particular areas of the
project:
http://svn.apache.org/repos/asf/subversion/trunk/COMMITTERS
On Thu, Nov 6, 2014 at 4:26 PM, Patrick Wendell pwend...@gmail.com wrote:
Hey Greg,
+1 (non-binding) [for original process proposal]
Greg, the first time I've seen the word ownership on this thread is in
your message. The first time the word lead has appeared in this thread is
in your message as well. I don't think that was the intent. The PMC and
Committers have a
Partial committers are people invited to work on a particular area, and
they do not require sign-off to work on that area. They can get a sign-off
and commit outside that area. That approach doesn't compare to this
proposal.
Full committers are PMC members. As each PMC member is responsible for
So I don't understand, Greg, are the partial committers committers, or are they
not? Spark also has a PMC, but our PMC currently consists of all committers (we
decided not to have a differentiation when we left the incubator). I see the
Subversion partial committers listed as committers on
PMC [1] is responsible for oversight and does not designate partial or full
committer. There are projects where all committers become PMC and others
where PMC is reserved for committers with the most merit (and willingness
to take on the responsibility of project oversight, releases, etc...).
It looks like the difference between the proposed Spark model and the
CloudStack / SVN model is:
* In the former, maintainers / partial committers are a way of centralizing
oversight over particular components among committers
* In the latter, maintainers / partial committers are a way of giving
My 2 cents:
Spark since pre-Apache days has been the most friendly and welcoming open
source project I've seen, and that's reflected in its success.
It seems pretty obvious to me that, for example, Michael should be looking
at major changes to the SQL codebase. I trust him to do that in a way
I'm actually going to change my non-binding to +0 for the proposal as-is.
I overlooked some parts of the original proposal that, when reading over
them again, do not sit well with me. one of the maintainers needs to sign
off on each patch to the component, as Greg has pointed out, does seem to
[ I'm going to try and pull a couple thread directions into this one, to
avoid explosion :-) ]
On Thu, Nov 6, 2014 at 6:44 PM, Corey Nolet cjno...@gmail.com wrote:
Note: I'm going to use you generically; I understand you [Corey] are not
a PMC member, at this time.
+1 (non-binding) [for original
On Thu, Nov 6, 2014 at 7:28 PM, Sandy Ryza sandy.r...@cloudera.com wrote:
It looks like the difference between the proposed Spark model and the
CloudStack / SVN model is:
* In the former, maintainers / partial committers are a way of
centralizing oversight over particular components among
Alright, Greg, I think I understand how Subversion's model is different, which
is that the PMC members are all full committers. However, I still think that
the model proposed here is purely organizational (how the PMC and committers
organize themselves), and in no way changes peoples' ownership
[last reply for tonite; let others read; and after the next drink or three,
I shouldn't be replying...]
On Thu, Nov 6, 2014 at 11:38 PM, Matei Zaharia matei.zaha...@gmail.com
wrote:
Alright, Greg, I think I understand how Subversion's model is different,
which is that the PMC members are all
Greg,
Thanks a lot for commenting on this, but I feel we are splitting hairs
here. Matei did mention -1, followed by or give feedback. The original
process outlined by Matei was exactly about review, rather than fighting.
Nobody wants to spend their energy fighting. Everybody is doing it to
With the maintainer model, the process is as follows:
- Any committer could review the patch and merge it, but they would need to
forward it to me (or another core API maintainer) to make sure we also approve
- At any point during this process, I could come in and -1 it, or give
feedback
Hi all,
I wanted to share a discussion we've been having on the PMC list, as well as
call for an official vote on it on a public list. Basically, as the Spark
project scales up, we need to define a model to make sure there is still great
oversight of key components (in particular internal
Hi Matei,
Definitely in favor of moving into this model for exactly the reasons
you mentioned.
From the module list though, the module that I'm mostly involved with
and is not listed is the Mesos integration piece.
I believe we also need a maintainer for Mesos, and I wonder if there
is someone
+1 (binding)
On Wed, Nov 5, 2014 at 5:33 PM, Matei Zaharia matei.zaha...@gmail.com
wrote:
BTW, my own vote is obviously +1 (binding).
Matei
On Nov 5, 2014, at 5:31 PM, Matei Zaharia matei.zaha...@gmail.com
wrote:
Hi all,
I wanted to share a discussion we've been having on the PMC
+1 (binding)
We are already doing this implicitly. In my experience, this can create
longer term personal commitment, which usually leads to better design
decisions if somebody knows they would need to look after something for a
while.
On Wed, Nov 5, 2014 at 5:33 PM, Matei Zaharia
+1, with a question
Will these maintainers have a cleanup for those pending PRs upon we start to
apply this model? there are some patches always being there but haven’t been
merged, some of which are periodically maintained (rebase, ping , etc….), the
others are just phased out
Best,
--
Hi Tim,
We can definitely add one for that if the component grows larger or becomes
harder to maintain. The main reason I didn't propose one is that the Mesos
integration is actually a lot simpler than YARN at the moment, partly because
we support several YARN versions that have incompatible
This seems like a good idea.
An area that wasn't listed, but that I think could strongly benefit from
maintainers, is the build. Having consistent oversight over Maven, SBT,
and dependencies would allow us to avoid subtle breakages.
Component maintainers have come up several times within the
I'm a +1 on this as well, I think it will be a useful model as we
scale the project in the future and recognizes some informal process
we have now.
To respond to Sandy's comment: for changes that fall in between the
component boundaries or are straightforward, my understanding of this
model is
+1
2014-11-05 18:08 GMT-08:00 Patrick Wendell pwend...@gmail.com:
I'm a +1 on this as well, I think it will be a useful model as we
scale the project in the future and recognizes some informal process
we have now.
To respond to Sandy's comment: for changes that fall in between the
+1, Sounds good.
Now I know whom to ping for what, even if I did not follow the whole
history of the project very carefully.
Prashant Sharma
On Thu, Nov 6, 2014 at 7:01 AM, Matei Zaharia matei.zaha...@gmail.com
wrote:
Hi all,
I wanted to share a discussion we've been having on the PMC
+1 (binding)
On Wed, Nov 5, 2014 at 6:29 PM, Nicholas Chammas nicholas.cham...@gmail.com
wrote:
+1 on this proposal.
On Wed, Nov 5, 2014 at 8:55 PM, Nan Zhu zhunanmcg...@gmail.com wrote:
Will these maintainers have a cleanup for those pending PRs upon we start
to apply this model?
I
+1 (binding)
On Wed, Nov 5, 2014 at 7:52 PM, Mark Hamstra m...@clearstorydata.com wrote:
+1 (binding)
On Wed, Nov 5, 2014 at 6:29 PM, Nicholas Chammas nicholas.cham...@gmail.com
wrote:
+1 on this proposal.
On Wed, Nov 5, 2014 at 8:55 PM, Nan Zhu zhunanmcg...@gmail.com wrote:
Will these
+1
发自我的 iPhone
在 2014年11月5日，20:06，Denny Lee denny.g@gmail.com 写道：
+1 great idea.
On Wed, Nov 5, 2014 at 20:04 Xiangrui Meng men...@gmail.com wrote:
+1 (binding)
On Wed, Nov 5, 2014 at 7:52 PM, Mark Hamstra m...@clearstorydata.com
wrote:
+1 (binding)
On Wed, Nov 5, 2014 at
+1 since this is already the de facto model we are using.
On Thu, Nov 6, 2014 at 12:40 PM, Wangfei (X) wangf...@huawei.com wrote:
+1
发自我的 iPhone
在 2014年11月5日，20:06，Denny Lee denny.g@gmail.com 写道：
+1 great idea.
On Wed, Nov 5, 2014 at 20:04 Xiangrui Meng men...@gmail.com wrote:
Great idea! +1
— Jeremy
-
jeremyfreeman.net
@thefreemanlab
On Nov 5, 2014, at 11:48 PM, Timothy Chen tnac...@gmail.com wrote:
Matei that makes sense, +1 (non-binding)
Tim
On Wed, Nov 5, 2014 at 8:46 PM, Cheng Lian lian.cs@gmail.com wrote:
+1 since this is
+1, that definitely will speeds up the PR reviewing / merging.
-Original Message-
From: Cheng Lian [mailto:lian.cs@gmail.com]
Sent: Thursday, November 6, 2014 12:46 PM
To: dev
Subject: Re: [VOTE] Designating maintainers for some Spark components
+1 since this is already the de facto
+1 Great idea!
--
View this message in context:
http://apache-spark-developers-list.1001551.n3.nabble.com/VOTE-Designating-maintainers-for-some-Spark-components-tp9115p9142.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com
+1, It makes sense!
- Kousuke
(2014/11/05 17:31), Matei Zaharia wrote:
Hi all,
I wanted to share a discussion we've been having on the PMC list, as well as
call for an official vote on it on a public list. Basically, as the Spark
project scales up, we need to define a model to make sure
+1, sounds good.
On Wed, Nov 5, 2014 at 9:19 PM, Kousuke Saruta saru...@oss.nttdata.co.jp
wrote:
+1, It makes sense!
- Kousuke
(2014/11/05 17:31), Matei Zaharia wrote:
Hi all,
I wanted to share a discussion we've been having on the PMC list, as well
as call for an official vote on it
+1 it make more focus and more consistence.
Yours, Xuefeng Wu 吴雪峰 敬上
On 2014年11月6日, at 上午9:31, Matei Zaharia matei.zaha...@gmail.com wrote:
Hi all,
I wanted to share a discussion we've been having on the PMC list, as well as
call for an official vote on it on a public list.
Several people asked about having maintainers review the PR queue for their
modules regularly, and I like that idea. We have a new tool now to help with
that in https://spark-prs.appspot.com.
In terms of the set of open PRs itself, it is large but note that there are
also 2800 *closed* PRs,
+1
Cheers!
Manoj.
On Thu, Nov 6, 2014 at 12:51 PM, Matei Zaharia matei.zaha...@gmail.com
wrote:
Several people asked about having maintainers review the PR queue for
their modules regularly, and I like that idea. We have a new tool now to
help with that in https://spark-prs.appspot.com.
In
+1
Liquan
On Wed, Nov 5, 2014 at 11:32 PM, Manoj Babu manoj...@gmail.com wrote:
+1
Cheers!
Manoj.
On Thu, Nov 6, 2014 at 12:51 PM, Matei Zaharia matei.zaha...@gmail.com
wrote:
Several people asked about having maintainers review the PR queue for
their modules regularly, and I like
62 matches
Mail list logo
