So just to clarify a few things in case people didn't read the entire thread in the PR, the discussion is what is the criteria for a blocker and really my concerns are what people are using as criteria for not marking a jira as a blocker. The only thing we have documented to mark a jira as a blocker is for correctness issues: http://spark.apache.org/contributing.html. And really I think that is initially mark it as a blocker to bring attention to it.The final decision as to whether something is a blocker is up to the PMC who votes on whether a release passes. I think it would be impossible to properly define what a blocker is with strict rules. Personally from this thread I would like to make sure committers and PMC members aren't saying its a block for reasons other then the actual impact the jira has and if its at all in question it should be brought to the PMC's attention for a vote. I agree with others that if its during an RC it should be talked about on the RC thread. A few specific things that were said that I disagree with are: - its not a blocker because it was also an issue in the last release (meaning feature release). ie the bug was introduced in 2.2 and now we are doing 2.4 so its automatically not a blocker. This to me is just wrong. Lots of things are not found immediately, or aren't reported immediately. Now I do believe the timeframe its been in there does affect the decision on the impact but just making the decision on this to me is to strict. - Committers and PMC members should not be saying its not a blocker because they personally or their company doesn't care about this feature or api, or state that the Spark project as a whole doesn't care about this feature unless that was specifically voted on at the project level. They need to follow the api compatibility we have documented. This is really a broader issue then just marking a jira, it goes to anything checked in and perhaps need to be a separate thread.
For the verbiage of what a regression is, it seems like that should be defined by our versioning documents. It states what we do in maintenance, feature, and major releases (http://spark.apache.org/versioning-policy.html), if its not defined by that we probably need to clarify. There was a good example we might want to clarify about things like scala or java compatibility in feature releases. Obviously this is my opinion and its here for everyone to discuss and come to a consensus on. Tom On Wednesday, October 24, 2018, 2:09:49 PM CDT, Sean Owen <sro...@gmail.com> wrote: Shifting this to dev@. See the PR https://github.com/apache/spark/pull/22144 for more context. There will be no objective, complete definition of blocker, or even regression or correctness issue. Many cases are clear, some are not. We can draw up more guidelines, and feel free to open PRs against the 'contributing' doc. But in general these are the same consensus-driven decisions we negotiate all the time. What isn't said that should be is that there is a cost to not releasing. Keep in mind we have, also, decided on a 'release train' cadence. That does properly change the calculus about what's a blocker; the right decision could change within even a week. I wouldn't mind some verbiage around what a regression is. Since the last minor release? We can VOTE on anything we like, but we already VOTE on the release. Weirdly, technically, the release vote criteria is simple majority, FWIW: http://www.apache.org/legal/release-policy.html#release-approval Yes, actually, it is only the PMC's votes that literally matter. Those votes are, surely, based on input from others too. But that is actually working as intended. Let's understand statements like "X is not a blocker" to mean "I don't think that X is a blocker". Interpretations not proclamations, backed up by reasons, not all of which are appeals to policy and precedent. I find it hard to argue about these in the abstract, because I believe it's already widely agreed, and written down in ASF policy, that nobody makes decisions unilaterally. Done, yes. Practically speaking, the urgent issue is the 2.4 release. I don't see process failures here that need fixing or debate. I do think those outstanding issues merit technical discussion. The outcome will be a tradeoff of some subjective issues, not read off of a policy sheet, and will entail tradeoffs. Let's speak freely about those technical issues and try to find the consensus position. On Wed, Oct 24, 2018 at 12:21 PM Mark Hamstra <notificati...@github.com> wrote: Thanks @tgravescs for your latest posts -- they've saved me from posting something similar in many respects but more strongly worded. What is bothering me (not just in the discussion of this PR, but more broadly) is that we have individuals making declarative statements about whether something can or can't block a release, or that something "is not that important to Spark at this point", etc. -- things for which there is no supporting PMC vote or declared policy. It may be your opinion, @cloud-fan , that Hive compatibility should no longer be important to the Apache Spark project, and I have no problem with you expressing your opinion on the matter. That may even be the internal policy at your employer, I don't know. But you are just not in a position on your own to make this declaration for the Apache Spark project. I don't mean to single you out, @cloud-fan , as the only offender, since this isn't a unique instance. For example, heading into a recent release we also saw individual declarations that the data correctness issue caused by the shuffle replay partitioning issue was not a blocker because it was not a regression or that it was not significant enough to alter the release schedule. Rather, my point is that things like release schedules, the declaration of release candidates, labeling JIRA tickets with "blocker", and de facto or even declared policy on regressions and release blockers are just tools in the service of the PMC. If, as was the case with the shuffle data correctness issue, PMC members think that the issue must be fixed before the next release, then release schedules, RC-status, other individuals' perceptions of importance to the project or of policy ultimately don't matter -- only the vote of the PMC does. What is concerning me is that, instead of efforts to persuade the PMC members that something should not block the next release or should not be important to the project, I am seeing flat declarations that an issue is not a blocker or not important. That may serve to stifle work to immediately fix a bug, or to discourage other contributions, but I can assure that trying to make the PMC serve the tools instead of the other way around won't serve to persuade at least some PMC members on how they should vote. Sorry, I guess I can't avoid wording things strongly after all. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
