+1 for 0.5.6 release with current improvements/fixes.
On Wed, Dec 30, 2015 at 4:10 PM, moon soo Lee <m...@apache.org> wrote: > Amos, > > Who started the word "meaningful" is not important. > Release discussion will not be judgement of how meaningful/major/big the > contributions are. > > CI problem i have describe about R interpreter PR [1] is not related with > any other contribution that we're trying to release. > > CI test does not have any known false negative, and we force contributor > rerun the test until false positive disappear. So logically, we can > guarantee that every contribution has passed the CI test correctly. > > Also Testis being done not only by CI but also all Zeppelin users. > If users see serious problem in the release candidate, they'll block the > release vote during release candidate verification. > > Hope this make you feel more confident about the code we're trying to > release. > > Thanks, > moon > > [1] > > http://apache-zeppelin-incubating-dev-mailing-list.75694.x6.nabble.com/GitHub-incubator-zeppelin-pull-request-R-Interpreter-for-Zeppelin-tp956p4623.html > > > On Tue, Dec 29, 2015 at 10:53 PM Amos B. Elberg <amos.elb...@gmail.com> > wrote: > > > You did reply to both. Let me try to clarify the problem with CI: > > > > The problem is *not* that particular PRs cause instability at runtime. > > > > The problem with CI is that if CI is not working properly, then *we can’t > > know* whether PRs will cause instability. Or what that connects to > > Zeppelin will break. > > > > CI is our own standard of testability. > > > > It is very common in organizations that they establish a standard of > > reliability. But, when things become difficult, or there is a problem > with > > the standard, the organization comes under pressure to bend or flex the > > standard. > > > > In my experience, when organizations violate their own standards for the > > sake of expedience, it is a recipe for trouble 100% of the time. > > > > — > > > > I just reviewed the changes Jongyoul posted. One of them relates to a > bug > > that was reported in September that has become an issue at Twitter. That > > seems to me to be a justification for a “hot fix” to the bug. > > > > I still don’t see any justification for a release though. > > > > > > From: moon soo Lee <m...@apache.org> > > Reply: dev@zeppelin.incubator.apache.org < > > dev@zeppelin.incubator.apache.org> > > Date: December 30, 2015 at 1:36:58 AM > > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org > > > > Subject: Re: [DISCUSS] Release 0.5.6-incubating > > > > Amos, > > > > If i summarize why you against 0.5.6-incubating release, > > > > * CI is not working > > * Does not have meaningful or major features to be released > > > > these two, right? I replied answer for both of them > > > > Here > > > > > http://apache-zeppelin-incubating-dev-mailing-list.75694.x6.nabble.com/DISCUSS-Release-0-5-6-incubating-tp4728p4763.html > > and here > > > > > http://apache-zeppelin-incubating-dev-mailing-list.75694.x6.nabble.com/DISCUSS-Release-0-5-6-incubating-tp4728p4765.html > > > > I'm listening and respect your opinion. Please check my reply and tell me > > if you have different opinion, but please include REASON WHY you think in > > that way otherwise it's hard to understand what you're thinking. > > > > Thanks, > > moon > > > > > > On Tue, Dec 29, 2015 at 10:18 PM Amos B. Elberg <amos.elb...@gmail.com> > > wrote: > > > > > Do you want me to explain the commits after 0.5.5 in details? > > > I want you to provide an example of any feature that justifies the > effort > > > that will be put into making a release, delaying 0.6 and CI and > > everything > > > else, and rebasing the outstanding major PRs. > > > > > > I will settle for *one* example. Just one! > > > > > > And what is your answer that why minor release has a important feature > > and > > > what the difference between major and minor is? > > > My view is that a “minor” release is one that doesn’t require changes > in > > > code built against the release other than recompiling. “Major” means > > > people have to work to update their code because of the release. > > > > > > I don't know why you oppose a new minor release including minor bug > > fixes. > > > I’m not even sure these count as “bug fixes” :p A change to the shading > > > of a window so it matches other windows is nice, but its hardly a “bug > > > fix.” > > > > > > Anyway I don’t think this release will really be limited to UI and > > “minor” > > > changes. I think there will be changes to the core code — like the 1.6 > PR > > > — that will actually be problems disguised as minor changes. And i > don’t > > > think we can test for that without CI. > > > > > > And What kind of aspects are less maintainable between 0.5.5 and 0.5.6? > > > The fact of the change is what makes it less maintainable! > > > > > > And what kind of fixes makes Zeppelin less stable? > > > The *codebase* is definitely less stable. > > > > > > Do you believe that some PR is unstable because of failing CI? > > > > > > Since CI is failing, how do I know if any PRs are stable or not? > > > > > > > > > From: Jongyoul Lee <jongy...@gmail.com> > > > Reply: Jongyoul Lee <jongy...@gmail.com> > > > Date: December 30, 2015 at 1:05:55 AM > > > To: Amos B. Elberg <amos.elb...@gmail.com> > > > CC: dev@zeppelin.incubator.apache.org < > dev@zeppelin.incubator.apache.org > > > > > > Subject: Re: [DISCUSS] Release 0.5.6-incubating > > > > > > Do you want me to explain the commits after 0.5.5 in details? And what > is > > > your answer that why minor release has a important feature and what the > > > difference between major and minor is? I also think it's good to fix > > > version up for ignite but this is not a major feature. I don't know why > > > you oppose a new minor release including minor bug fixes. And What kind > > of > > > aspects are less maintainable between 0.5.5 and 0.5.6? If 0.5.6 is less > > > maintainable, we should revert that commit because it's harmful to > > > Zeppelin. And what kind of fixes makes Zeppelin less stable? I would > like > > > to show me that commit number or issue number. And finally, Moon > admitted > > > CI had some flakey tests and have tried to remove or fix that tests. Do > > you > > > believe that some PR is unstable because of failing CI? > > > > > > On Wed, Dec 30, 2015 at 2:42 PM, Amos B. Elberg <amos.elb...@gmail.com > > > > > wrote: > > > A codebase that often changes in ways that break other code is an > > unstable > > > codebase, by definition. > > > > > > I don’t think it will be more stable at runtime, especially since CI > > isn’t > > > working. > > > > > > It definitely won’t be more maintainable. The key problematic code is > > > still in. > > > > > > Other than Spark 1.6 and Ignite, I don’t see any reason at all for a > > 0.5.6 > > > release. (Konstantin was right — it is good for Apache releases to > > > maintain version compatibility with new versions of other Apache > > software. > > > That is Apache projects helping each other.) > > > > > > What feature do you feel justifies a 0.5.6 release? What feature other > > > than 1.6 and Ignite does anyone feel justifies a 0.5.6 release? > > > > > > From: Jongyoul Lee <jongy...@gmail.com> > > > Reply: Jongyoul Lee <jongy...@gmail.com> > > > Date: December 30, 2015 at 12:32:01 AM > > > > > > To: Amos B. Elberg <amos.elb...@gmail.com> > > > CC: dev@zeppelin.incubator.apache.org < > dev@zeppelin.incubator.apache.org > > > > > > Subject: Re: [DISCUSS] Release 0.5.6-incubating > > > > > > Amos, > > > > > > I don't think we have a strict plan for making a minor release and we > > have > > > a roadmap for major release. And ignite and Spark 1.6 is not a key > > feature > > > of 0.5.6. Konstantin just wanted to be merged that contribution if that > > > voting is finished until we make a release. And Spark 1.6 is on going. > As > > > you told, we are an Apache project. 0.5.6 will be stable and > > maintainable. > > > If 0.5.6 has an experimental features, I don't agree to make a release. > > > 0.5.6 will be more stable version of 0.5.5. And of course, the most > > people > > > like more stable version. Isn't it enough? > > > > > > Regards, > > > Jongyoul > > > > > > On Wed, Dec 30, 2015 at 2:17 PM, Amos B. Elberg <amos.elb...@gmail.com > > > > > wrote: > > > My suggestion is that we do a 0.5.6 that just has the bare minimal > > changes > > > necessary for Spark 1.6 and Ignite and nothing else. > > > > > > That way we provide “must have” features while minimizing risk. > > > > > > Other than that, yes: I think we should keep our current release plan > and > > > not make a release for “nice to have” changes until CI is fixed. > > > > > > The main purpose of making a new minor release should be whether > already > > > merged features are meaningful to make a minor release even if any > major > > > issues are on going, isn't it? > > > > > > I’m not sure that I understand what you are asking. > > > > > > We have a planned 0.6 release. We just did an unplanned “minor” 0.5.5 > > > release. It feels like only a few weeks ago. I voted for it because it > > > seemed that it would stabilize the codebase and provide a maintainable > > > interim foundation. > > > > > > I do not think any of the features since 0.5.5 are “meaningful” enough > to > > > justify changing the release plan. Not even close. I think it is rare > > > that any off-roadmap “nice to have” feature would ever be a good reason > > to > > > change a release plan. Especially when our CI “house” is not in order. > > > > > > We’re an Apache project — we need to be stable, maintainable, reliable, > > > predictable. > > > > > > Is there any merged PR that is so important it can’t wait for 0.6? > > > > > > From: Jongyoul Lee <jongy...@gmail.com> > > > Reply: Jongyoul Lee <jongy...@gmail.com> > > > Date: December 29, 2015 at 11:54:35 PM > > > To: Amos B. Elberg <amos.elb...@gmail.com> > > > CC: dev@zeppelin.incubator.apache.org < > dev@zeppelin.incubator.apache.org > > > > > > > > > Subject: Re: [DISCUSS] Release 0.5.6-incubating > > > > > > Okay, Amos, > > > > > > Do you propose Zeppelin should not have another release before fix CI > > > issue? I think even though CI has some problems, another minors fixes > is > > > meaningful to make a new minor release. Do you agree with that? Or > don't > > > you agree that it's enough? The main purpose of making a new minor > > release > > > should be whether already merged features are meaningful to make a > minor > > > release even if any major issues are on going, isn't it? > > > > > > Regards, > > > Jongyoul > > > > > > On Wed, Dec 30, 2015 at 1:35 PM, Amos B. Elberg <amos.elb...@gmail.com > > > > > wrote: > > > Hah! > > > > > > I promise you, an hour after a 0.5.6 comes out, I will have emails > asking > > > me when I will support 0.5.6, even if no-one actually needs any 0.5.6 > > > changes or even knows what they are! > > > > > > I want to be clear though: My primary issue for 0.5.6 is not whether to > > > merge the R interpreter. > > > > > > My issues are I think we need to fix CI in general, and I’m loathe to > > have > > > more releases with that dammed spark-under-zeppelin code, which is the > > root > > > of many other issues. > > > > > > > > > From: Jongyoul Lee <jongy...@gmail.com> > > > Reply: Jongyoul Lee <jongy...@gmail.com> > > > Date: December 29, 2015 at 11:21:00 PM > > > To: Amos B. Elberg <amos.elb...@gmail.com>, > > > dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org> > > > > > > Subject: Re: [DISCUSS] Release 0.5.6-incubating > > > > > > Okay, I understand your situation. If you rebased your PR from master, > > you > > > can rebased your PR only once but I also know why you had to do that. I > > > think R is a roadmap for 0.6.0 and you'd better skip rebasing 0.5.6. > How > > > about you? > > > > > > On Wed, Dec 30, 2015 at 1:12 PM, Amos B. Elberg <amos.elb...@gmail.com > > > > > wrote: > > > Jongyoul - the reason we have to rebase twice is that the changes in > > > zeppelin-master will break the R interpreter. > > > > > > So I’ll have to rebase once so that I’m based off of 0.5.6 and people > can > > > use the code. Then rebase again for 0.6.0. > > > > > > Remember, I have a user base I need to support — there are a lot of > > people > > > using the R interpreter now. So its not just a PR where I can ignore it > > > until its ready to merge. > > > > > > The changes have already broken the shiro PR apparently quite often. > > > > > > I made a “release” of the R Interpreter just so I could stop rebasing > > > against Zeppelin master. I spent > 60 hours dealing with this for the > > > changes leading up to 0.5 and 0.5.5. > > > > > > From: Jongyoul Lee <jongy...@gmail.com> > > > Reply: Jongyoul Lee <jongy...@gmail.com> > > > Date: December 29, 2015 at 11:08:36 PM > > > To: Amos B. Elberg <amos.elb...@gmail.com> > > > > > > Subject: Re: [DISCUSS] Release 0.5.6-incubating > > > > > > I don't know why you should rebased twice. If you can make a PR from > > > current master, almost changes merged without rebasing and if a > committer > > > asks to rebase your PR, this is because you and another contributors > > > changes the same codes and another contributions is merged before your > > PR. > > > In specific R case, Moon want you to rebase because he tries to fix the > > > testing codes so rebasing your PR will accepts their changes. In most > > case, > > > contributors don't need to rebase their PR before merge because > committer > > > commit someone's PR by doing cherry-pick. We also felt sorry that you > > were > > > bothered by testing issue and Moon is fighting to fix the testing > infra. > > > However all of PRs shouldn't be rebased. > > > > > > On Wed, Dec 30, 2015 at 12:59 PM, Amos B. Elberg < > amos.elb...@gmail.com> > > > wrote: > > > I think there is no big pain because whole changes to be merged > > > into 0.5.6 will be also merged into 0.6.0. > > > > > > If we make another release now, the PRs will have to be rebased > *twice*, > > > once for 0.5.6, and once again for 0.6. > > > > > > Also - since this is now the second e-mail from a committer to do the > > same > > > thing — is there a reason you guys are leaving R out of the agenda for > > the > > > next release? I understood the PR had been accepted and was pending > only > > > Moon fixing the testing infrastructure. > > > > > > > > > From: Jongyoul Lee <jongy...@gmail.com> > > > Reply: dev@zeppelin.incubator.apache.org < > > > dev@zeppelin.incubator.apache.org> > > > Date: December 29, 2015 at 10:56:33 PM > > > > > > To: dev@zeppelin.incubator.apache.org < > dev@zeppelin.incubator.apache.org > > > > > > Subject: Re: [DISCUSS] Release 0.5.6-incubating > > > > > > Good idea! > > > > > > 0.5.6 is a minor release thus fixing minor bugs and typos is enough. > But > > I > > > also think 0.6.0 should have major changes like supporting spark 1.6 > and > > > Shiro security and improving testing infra. And concerning rebasing and > > > committing, I think there is no big pain because whole changes to be > > merged > > > into 0.5.6 will be also merged into 0.6.0. > > > > > > JL > > > > > > On Wed, Dec 30, 2015 at 12:32 PM, Amos B. Elberg < > amos.elb...@gmail.com> > > > wrote: > > > > > > > I don’t want to come off as the naysayer here, but I think the effort > > > that > > > > would go into a release would be better spent on the testing > > > infrastructure > > > > and outstanding PRs. > > > > > > > > If we feel we need a release for 1.6 and Ignite, I suggest we make a > > > > release that *only* includes the absolute minimal changes required to > > do > > > > that. > > > > > > > > There was one PR for 1.6 support that didn’t really work and is going > > to > > > > break anything built against the current codebase. Except for a > change > > in > > > > the name of one method called by one class, all of the changes seem > to > > > > involve support for spark-under-zeppelin, which is something we want > to > > > > take out. > > > > > > > > We also don’t currently have a working testing framework. A lot of > PRs > > > > have been committed under the “ignore travis its broken” theory. I’m > > > > loathe to make a release that hasn’t really been tested, and it > doesn’t > > > > seem we’re in a position to do that. > > > > > > > > While there have been a lot of merged PRs, I don’t think any of them > > are > > > > on-roadmap. They mostly seem to be very minor, like fixing typos and > > > > changing which text box gets highlighted. Those are important things, > > of > > > > course, but not major enough to justify the effort involved in a > > release. > > > > > > > > Another release will not make it easier to integrate the larger PRs. > It > > > > will have the opposite effect. Developers will have to rebase against > > > > whatever gets broken by 1.6 and other changes. > > > > > > > > I suggest we wait to do a significant release until we can take out > the > > > > legacy spark-under-zeppelin code that has caused so many issues, > have a > > > > working testing framework, and integrate the major outstanding PRs. > > > > > > > > So, again, if we want a release, I suggest we include the absolute > > > minimum > > > > changes necessary for 1.6 and Ignite, and perhaps call it an interim > or > > > > maintenance release. > > > > > > > > > > > > From: Konstantin Boudnik <c...@apache.org> > > > > Reply: dev@zeppelin.incubator.apache.org < > > > > dev@zeppelin.incubator.apache.org>, > dev@zeppelin.incubator.apache.org > > < > > > > dev@zeppelin.incubator.apache.org> > > > > Date: December 29, 2015 at 10:05:36 PM > > > > To: dev@zeppelin.incubator.apache.org < > > dev@zeppelin.incubator.apache.org > > > > > > > > Subject: Re: [DISCUSS] Release 0.5.6-incubating > > > > > > > > Good idea! BTW, Apache Ignite is voting right now on 1.5.0.final - > > would > > > > make > > > > sense to add this to the latest release of Zeppelin. I will open a > JIRA > > > and > > > > supply a patch for it, if there's no objections. > > > > > > > > Cos > > > > > > > > On Wed, Dec 30, 2015 at 03:00AM, moon soo Lee wrote: > > > > > Hi folks, > > > > > > > > > > How about we make release 0.5.6-incubating? > > > > > > > > > > Since last release, more than 100 pull requests are merged and more > > > than > > > > 80 > > > > > issues are resolved. > > > > > > > > > > It's including new interpreters, a lot of new features and > > improvement > > > on > > > > > GUI with much improved stability thanks to lots of bug fixes. > > > > > > > > > > Also it's great time to have a Zeppelin release that support Spark > > 1.6 > > > ( > > > > > ZEPPELIN-395), which about to be released. > > > > > > > > > > Once we branch for 0.5.6-incubating release, it's more safe to make > > > large > > > > > code base change such as "Added Shiro security" ( > > > > > https://github.com/apache/incubator-zeppelin/pull/53) and many > other > > > > > planned feature in 0.6.0 roadmap, that will require lots of > internal > > > API > > > > > updates. > > > > > > > > > > What do you think? > > > > > > > > > > Thanks, > > > > > moon > > > > > > > > > > > > > > > > -- > > > 이종열, Jongyoul Lee, 李宗烈 > > > http://madeng.net > > > > > > > > > > > > -- > > > 이종열, Jongyoul Lee, 李宗烈 > > > http://madeng.net > > > > > > > > > > > > -- > > > 이종열, Jongyoul Lee, 李宗烈 > > > http://madeng.net > > > > > > > > > > > > -- > > > 이종열, Jongyoul Lee, 李宗烈 > > > http://madeng.net > > > > > > > > > > > > -- > > > 이종열, Jongyoul Lee, 李宗烈 > > > http://madeng.net > > > > > > > > > > > > -- > > > 이종열, Jongyoul Lee, 李宗烈 > > > http://madeng.net > > > > > >
