Amos, Who started the word "meaningful" is not important. Release discussion will not be judgement of how meaningful/major/big the contributions are.
CI problem i have describe about R interpreter PR [1] is not related with any other contribution that we're trying to release. CI test does not have any known false negative, and we force contributor rerun the test until false positive disappear. So logically, we can guarantee that every contribution has passed the CI test correctly. Also Testis being done not only by CI but also all Zeppelin users. If users see serious problem in the release candidate, they'll block the release vote during release candidate verification. Hope this make you feel more confident about the code we're trying to release. Thanks, moon [1] http://apache-zeppelin-incubating-dev-mailing-list.75694.x6.nabble.com/GitHub-incubator-zeppelin-pull-request-R-Interpreter-for-Zeppelin-tp956p4623.html On Tue, Dec 29, 2015 at 10:53 PM Amos B. Elberg <amos.elb...@gmail.com> wrote: > You did reply to both. Let me try to clarify the problem with CI: > > The problem is *not* that particular PRs cause instability at runtime. > > The problem with CI is that if CI is not working properly, then *we can’t > know* whether PRs will cause instability. Or what that connects to > Zeppelin will break. > > CI is our own standard of testability. > > It is very common in organizations that they establish a standard of > reliability. But, when things become difficult, or there is a problem with > the standard, the organization comes under pressure to bend or flex the > standard. > > In my experience, when organizations violate their own standards for the > sake of expedience, it is a recipe for trouble 100% of the time. > > — > > I just reviewed the changes Jongyoul posted. One of them relates to a bug > that was reported in September that has become an issue at Twitter. That > seems to me to be a justification for a “hot fix” to the bug. > > I still don’t see any justification for a release though. > > > From: moon soo Lee <m...@apache.org> > Reply: dev@zeppelin.incubator.apache.org < > dev@zeppelin.incubator.apache.org> > Date: December 30, 2015 at 1:36:58 AM > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org> > Subject: Re: [DISCUSS] Release 0.5.6-incubating > > Amos, > > If i summarize why you against 0.5.6-incubating release, > > * CI is not working > * Does not have meaningful or major features to be released > > these two, right? I replied answer for both of them > > Here > > http://apache-zeppelin-incubating-dev-mailing-list.75694.x6.nabble.com/DISCUSS-Release-0-5-6-incubating-tp4728p4763.html > and here > > http://apache-zeppelin-incubating-dev-mailing-list.75694.x6.nabble.com/DISCUSS-Release-0-5-6-incubating-tp4728p4765.html > > I'm listening and respect your opinion. Please check my reply and tell me > if you have different opinion, but please include REASON WHY you think in > that way otherwise it's hard to understand what you're thinking. > > Thanks, > moon > > > On Tue, Dec 29, 2015 at 10:18 PM Amos B. Elberg <amos.elb...@gmail.com> > wrote: > > > Do you want me to explain the commits after 0.5.5 in details? > > I want you to provide an example of any feature that justifies the effort > > that will be put into making a release, delaying 0.6 and CI and > everything > > else, and rebasing the outstanding major PRs. > > > > I will settle for *one* example. Just one! > > > > And what is your answer that why minor release has a important feature > and > > what the difference between major and minor is? > > My view is that a “minor” release is one that doesn’t require changes in > > code built against the release other than recompiling. “Major” means > > people have to work to update their code because of the release. > > > > I don't know why you oppose a new minor release including minor bug > fixes. > > I’m not even sure these count as “bug fixes” :p A change to the shading > > of a window so it matches other windows is nice, but its hardly a “bug > > fix.” > > > > Anyway I don’t think this release will really be limited to UI and > “minor” > > changes. I think there will be changes to the core code — like the 1.6 PR > > — that will actually be problems disguised as minor changes. And i don’t > > think we can test for that without CI. > > > > And What kind of aspects are less maintainable between 0.5.5 and 0.5.6? > > The fact of the change is what makes it less maintainable! > > > > And what kind of fixes makes Zeppelin less stable? > > The *codebase* is definitely less stable. > > > > Do you believe that some PR is unstable because of failing CI? > > > > Since CI is failing, how do I know if any PRs are stable or not? > > > > > > From: Jongyoul Lee <jongy...@gmail.com> > > Reply: Jongyoul Lee <jongy...@gmail.com> > > Date: December 30, 2015 at 1:05:55 AM > > To: Amos B. Elberg <amos.elb...@gmail.com> > > CC: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org > > > > Subject: Re: [DISCUSS] Release 0.5.6-incubating > > > > Do you want me to explain the commits after 0.5.5 in details? And what is > > your answer that why minor release has a important feature and what the > > difference between major and minor is? I also think it's good to fix > > version up for ignite but this is not a major feature. I don't know why > > you oppose a new minor release including minor bug fixes. And What kind > of > > aspects are less maintainable between 0.5.5 and 0.5.6? If 0.5.6 is less > > maintainable, we should revert that commit because it's harmful to > > Zeppelin. And what kind of fixes makes Zeppelin less stable? I would like > > to show me that commit number or issue number. And finally, Moon admitted > > CI had some flakey tests and have tried to remove or fix that tests. Do > you > > believe that some PR is unstable because of failing CI? > > > > On Wed, Dec 30, 2015 at 2:42 PM, Amos B. Elberg <amos.elb...@gmail.com> > > wrote: > > A codebase that often changes in ways that break other code is an > unstable > > codebase, by definition. > > > > I don’t think it will be more stable at runtime, especially since CI > isn’t > > working. > > > > It definitely won’t be more maintainable. The key problematic code is > > still in. > > > > Other than Spark 1.6 and Ignite, I don’t see any reason at all for a > 0.5.6 > > release. (Konstantin was right — it is good for Apache releases to > > maintain version compatibility with new versions of other Apache > software. > > That is Apache projects helping each other.) > > > > What feature do you feel justifies a 0.5.6 release? What feature other > > than 1.6 and Ignite does anyone feel justifies a 0.5.6 release? > > > > From: Jongyoul Lee <jongy...@gmail.com> > > Reply: Jongyoul Lee <jongy...@gmail.com> > > Date: December 30, 2015 at 12:32:01 AM > > > > To: Amos B. Elberg <amos.elb...@gmail.com> > > CC: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org > > > > Subject: Re: [DISCUSS] Release 0.5.6-incubating > > > > Amos, > > > > I don't think we have a strict plan for making a minor release and we > have > > a roadmap for major release. And ignite and Spark 1.6 is not a key > feature > > of 0.5.6. Konstantin just wanted to be merged that contribution if that > > voting is finished until we make a release. And Spark 1.6 is on going. As > > you told, we are an Apache project. 0.5.6 will be stable and > maintainable. > > If 0.5.6 has an experimental features, I don't agree to make a release. > > 0.5.6 will be more stable version of 0.5.5. And of course, the most > people > > like more stable version. Isn't it enough? > > > > Regards, > > Jongyoul > > > > On Wed, Dec 30, 2015 at 2:17 PM, Amos B. Elberg <amos.elb...@gmail.com> > > wrote: > > My suggestion is that we do a 0.5.6 that just has the bare minimal > changes > > necessary for Spark 1.6 and Ignite and nothing else. > > > > That way we provide “must have” features while minimizing risk. > > > > Other than that, yes: I think we should keep our current release plan and > > not make a release for “nice to have” changes until CI is fixed. > > > > The main purpose of making a new minor release should be whether already > > merged features are meaningful to make a minor release even if any major > > issues are on going, isn't it? > > > > I’m not sure that I understand what you are asking. > > > > We have a planned 0.6 release. We just did an unplanned “minor” 0.5.5 > > release. It feels like only a few weeks ago. I voted for it because it > > seemed that it would stabilize the codebase and provide a maintainable > > interim foundation. > > > > I do not think any of the features since 0.5.5 are “meaningful” enough to > > justify changing the release plan. Not even close. I think it is rare > > that any off-roadmap “nice to have” feature would ever be a good reason > to > > change a release plan. Especially when our CI “house” is not in order. > > > > We’re an Apache project — we need to be stable, maintainable, reliable, > > predictable. > > > > Is there any merged PR that is so important it can’t wait for 0.6? > > > > From: Jongyoul Lee <jongy...@gmail.com> > > Reply: Jongyoul Lee <jongy...@gmail.com> > > Date: December 29, 2015 at 11:54:35 PM > > To: Amos B. Elberg <amos.elb...@gmail.com> > > CC: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org > > > > > > Subject: Re: [DISCUSS] Release 0.5.6-incubating > > > > Okay, Amos, > > > > Do you propose Zeppelin should not have another release before fix CI > > issue? I think even though CI has some problems, another minors fixes is > > meaningful to make a new minor release. Do you agree with that? Or don't > > you agree that it's enough? The main purpose of making a new minor > release > > should be whether already merged features are meaningful to make a minor > > release even if any major issues are on going, isn't it? > > > > Regards, > > Jongyoul > > > > On Wed, Dec 30, 2015 at 1:35 PM, Amos B. Elberg <amos.elb...@gmail.com> > > wrote: > > Hah! > > > > I promise you, an hour after a 0.5.6 comes out, I will have emails asking > > me when I will support 0.5.6, even if no-one actually needs any 0.5.6 > > changes or even knows what they are! > > > > I want to be clear though: My primary issue for 0.5.6 is not whether to > > merge the R interpreter. > > > > My issues are I think we need to fix CI in general, and I’m loathe to > have > > more releases with that dammed spark-under-zeppelin code, which is the > root > > of many other issues. > > > > > > From: Jongyoul Lee <jongy...@gmail.com> > > Reply: Jongyoul Lee <jongy...@gmail.com> > > Date: December 29, 2015 at 11:21:00 PM > > To: Amos B. Elberg <amos.elb...@gmail.com>, > > dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org> > > > > Subject: Re: [DISCUSS] Release 0.5.6-incubating > > > > Okay, I understand your situation. If you rebased your PR from master, > you > > can rebased your PR only once but I also know why you had to do that. I > > think R is a roadmap for 0.6.0 and you'd better skip rebasing 0.5.6. How > > about you? > > > > On Wed, Dec 30, 2015 at 1:12 PM, Amos B. Elberg <amos.elb...@gmail.com> > > wrote: > > Jongyoul - the reason we have to rebase twice is that the changes in > > zeppelin-master will break the R interpreter. > > > > So I’ll have to rebase once so that I’m based off of 0.5.6 and people can > > use the code. Then rebase again for 0.6.0. > > > > Remember, I have a user base I need to support — there are a lot of > people > > using the R interpreter now. So its not just a PR where I can ignore it > > until its ready to merge. > > > > The changes have already broken the shiro PR apparently quite often. > > > > I made a “release” of the R Interpreter just so I could stop rebasing > > against Zeppelin master. I spent > 60 hours dealing with this for the > > changes leading up to 0.5 and 0.5.5. > > > > From: Jongyoul Lee <jongy...@gmail.com> > > Reply: Jongyoul Lee <jongy...@gmail.com> > > Date: December 29, 2015 at 11:08:36 PM > > To: Amos B. Elberg <amos.elb...@gmail.com> > > > > Subject: Re: [DISCUSS] Release 0.5.6-incubating > > > > I don't know why you should rebased twice. If you can make a PR from > > current master, almost changes merged without rebasing and if a committer > > asks to rebase your PR, this is because you and another contributors > > changes the same codes and another contributions is merged before your > PR. > > In specific R case, Moon want you to rebase because he tries to fix the > > testing codes so rebasing your PR will accepts their changes. In most > case, > > contributors don't need to rebase their PR before merge because committer > > commit someone's PR by doing cherry-pick. We also felt sorry that you > were > > bothered by testing issue and Moon is fighting to fix the testing infra. > > However all of PRs shouldn't be rebased. > > > > On Wed, Dec 30, 2015 at 12:59 PM, Amos B. Elberg <amos.elb...@gmail.com> > > wrote: > > I think there is no big pain because whole changes to be merged > > into 0.5.6 will be also merged into 0.6.0. > > > > If we make another release now, the PRs will have to be rebased *twice*, > > once for 0.5.6, and once again for 0.6. > > > > Also - since this is now the second e-mail from a committer to do the > same > > thing — is there a reason you guys are leaving R out of the agenda for > the > > next release? I understood the PR had been accepted and was pending only > > Moon fixing the testing infrastructure. > > > > > > From: Jongyoul Lee <jongy...@gmail.com> > > Reply: dev@zeppelin.incubator.apache.org < > > dev@zeppelin.incubator.apache.org> > > Date: December 29, 2015 at 10:56:33 PM > > > > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org > > > > Subject: Re: [DISCUSS] Release 0.5.6-incubating > > > > Good idea! > > > > 0.5.6 is a minor release thus fixing minor bugs and typos is enough. But > I > > also think 0.6.0 should have major changes like supporting spark 1.6 and > > Shiro security and improving testing infra. And concerning rebasing and > > committing, I think there is no big pain because whole changes to be > merged > > into 0.5.6 will be also merged into 0.6.0. > > > > JL > > > > On Wed, Dec 30, 2015 at 12:32 PM, Amos B. Elberg <amos.elb...@gmail.com> > > wrote: > > > > > I don’t want to come off as the naysayer here, but I think the effort > > that > > > would go into a release would be better spent on the testing > > infrastructure > > > and outstanding PRs. > > > > > > If we feel we need a release for 1.6 and Ignite, I suggest we make a > > > release that *only* includes the absolute minimal changes required to > do > > > that. > > > > > > There was one PR for 1.6 support that didn’t really work and is going > to > > > break anything built against the current codebase. Except for a change > in > > > the name of one method called by one class, all of the changes seem to > > > involve support for spark-under-zeppelin, which is something we want to > > > take out. > > > > > > We also don’t currently have a working testing framework. A lot of PRs > > > have been committed under the “ignore travis its broken” theory. I’m > > > loathe to make a release that hasn’t really been tested, and it doesn’t > > > seem we’re in a position to do that. > > > > > > While there have been a lot of merged PRs, I don’t think any of them > are > > > on-roadmap. They mostly seem to be very minor, like fixing typos and > > > changing which text box gets highlighted. Those are important things, > of > > > course, but not major enough to justify the effort involved in a > release. > > > > > > Another release will not make it easier to integrate the larger PRs. It > > > will have the opposite effect. Developers will have to rebase against > > > whatever gets broken by 1.6 and other changes. > > > > > > I suggest we wait to do a significant release until we can take out the > > > legacy spark-under-zeppelin code that has caused so many issues, have a > > > working testing framework, and integrate the major outstanding PRs. > > > > > > So, again, if we want a release, I suggest we include the absolute > > minimum > > > changes necessary for 1.6 and Ignite, and perhaps call it an interim or > > > maintenance release. > > > > > > > > > From: Konstantin Boudnik <c...@apache.org> > > > Reply: dev@zeppelin.incubator.apache.org < > > > dev@zeppelin.incubator.apache.org>, dev@zeppelin.incubator.apache.org > < > > > dev@zeppelin.incubator.apache.org> > > > Date: December 29, 2015 at 10:05:36 PM > > > To: dev@zeppelin.incubator.apache.org < > dev@zeppelin.incubator.apache.org > > > > > > Subject: Re: [DISCUSS] Release 0.5.6-incubating > > > > > > Good idea! BTW, Apache Ignite is voting right now on 1.5.0.final - > would > > > make > > > sense to add this to the latest release of Zeppelin. I will open a JIRA > > and > > > supply a patch for it, if there's no objections. > > > > > > Cos > > > > > > On Wed, Dec 30, 2015 at 03:00AM, moon soo Lee wrote: > > > > Hi folks, > > > > > > > > How about we make release 0.5.6-incubating? > > > > > > > > Since last release, more than 100 pull requests are merged and more > > than > > > 80 > > > > issues are resolved. > > > > > > > > It's including new interpreters, a lot of new features and > improvement > > on > > > > GUI with much improved stability thanks to lots of bug fixes. > > > > > > > > Also it's great time to have a Zeppelin release that support Spark > 1.6 > > ( > > > > ZEPPELIN-395), which about to be released. > > > > > > > > Once we branch for 0.5.6-incubating release, it's more safe to make > > large > > > > code base change such as "Added Shiro security" ( > > > > https://github.com/apache/incubator-zeppelin/pull/53) and many other > > > > planned feature in 0.6.0 roadmap, that will require lots of internal > > API > > > > updates. > > > > > > > > What do you think? > > > > > > > > Thanks, > > > > moon > > > > > > > > > > > -- > > 이종열, Jongyoul Lee, 李宗烈 > > http://madeng.net > > > > > > > > -- > > 이종열, Jongyoul Lee, 李宗烈 > > http://madeng.net > > > > > > > > -- > > 이종열, Jongyoul Lee, 李宗烈 > > http://madeng.net > > > > > > > > -- > > 이종열, Jongyoul Lee, 李宗烈 > > http://madeng.net > > > > > > > > -- > > 이종열, Jongyoul Lee, 李宗烈 > > http://madeng.net > > > > > > > > -- > > 이종열, Jongyoul Lee, 李宗烈 > > http://madeng.net > > >
