Amos "When it tries that, in some spark configurations it can't verify that the cluster is up, so it never runs its tests."
--> Precisely, once every time I looked into the detailed Travis log, one of the reason the cluster is not up was that the Travis server/VM/container has not enough resource to start the cluster so it gets delayed and the cluster status check just fails after some timeout. "But I can't tell, and its hard to do anything about because I can't reproduce the issue outside of CI" --> Same problem here, some times some of my PRs are just red because of random failure, the only way to make it green is to *repeatedly* force-push until it passes green. I also remarked that when I force push during morning time UTC, I rarely have random failure. Starting from 6PM UTC, pushing a PR has a lot more chance to fail randomly. And it corresponds to US West Coast waking up and start using Travis infrastructure. At least it's my assumption, I don't have clear evidence to support it because I don't have access to Travis internal metrics anyway. The only thing I can tell is PR has few random failure when West Coast is sleeping :) On Wed, Mar 30, 2016 at 6:37 PM, Amos Elberg <amos.elb...@gmail.com> wrote: > DuyHai - I'm not sure we're talking about exactly the same thing. > > There is one issue which is that CI just sort of randomly fails, for > example last night many times when it was supposed to download Spark, the > spark .tar.gz file failed on checksum. This was just random. I was able > to resolve the random fails by repeatedly force-pushing, as you describe. > > What I'm describing is a bit different, I think. For many of the tests of > Zeppelin-spark integration, the Zeppelin test infrastructure has to launch > a spark cluster with a clean and sane configuration. That's what's failing > here. What happens is, the test class calls the function to launch the > cluster then, before running the actual tests, tries to verify that the > cluster is running. When it tries that, in some spark configurations it > can't verify that the cluster is up, so it never runs its tests. > > I was playing around with this last night, and its an interesting failure. > When I copy the tests verbatim out of a test class that works into the test > class that fails, the test class *still* fails. I think what may be > happening is that it may not be properly shutting spark down after other > tests, so it remains open but falls into an error state. But I can't tell, > and its hard to do anything about because I can't reproduce the issue > outside of CI, so the only way I can test anything is by pushing a new > build and force-pushing past the random failures, so I can see what fails. > So - I think you may see my dilemma here. > > On Wed, Mar 30, 2016 at 12:13 PM, DuyHai Doan <doanduy...@gmail.com> > wrote: > > > Amos > > > > "It's also not correct to say that the test class even "fails" -- what's > > happening is that the testing infrastructure for this class fails to > load." > > --> This is the behavior I have observed many many times > > > > Being a heavy user of Travis (see my other open source project > > www.achilles.io) I can tell you that random failure due to hardware > under > > heavy load (so not being able to start a process for testing) is VERY > VERY > > common. > > > > Usually, when US west coast wakes up, the servers gets busy and tests > start > > to fail because not enough resources > > > > Did you try to "force push" on the PR208 with the removed Spark test > > several times to trigger CI again ? I know that this trick has always > > worked for me until now > > > > > > > > On Wed, Mar 30, 2016 at 5:18 PM, moon soo Lee <m...@apache.org> wrote: > > > > > Amos, > > > > > > Please respect the community consensus [1] and author of 702 and people > > > collaborated in 702. They're all community members. > > > > > > Like i summarized > > > > > > > > > http://apache-zeppelin-incubating-dev-mailing-list.75694.x6.nabble.com/DISCUSS-Back-to-PRs-208-702-tp7691p7787.html > > > , > > > no one disagree on 702. > > > > > > And please remember, it's open community. We want to collaborative > work. > > > Hope you're not attempting prevent other people making contribution to > > the > > > project. > > > > > > Thanks, > > > moon > > > > > > [1] > > > > > > > > > http://apache-zeppelin-incubating-dev-mailing-list.75694.x6.nabble.com/R-interpreter-in-Zeppelin-further-steps-tp6967.html > > > > > > On Wed, Mar 30, 2016 at 7:54 AM Amos B. Elberg <amos.elb...@gmail.com> > > > wrote: > > > > > > > Moon - no-one has supported your view. The community in fact has > > > > overwhelmingly rejected it. Nobody prefers 702. Nobody agrees with > you. > > > > > > > > You are simply personally obstructing this, because of your personal > > > > animosity -- and you have been for months. > > > > > > > > It's time for you to step out of this --- if you think you can try to > > > lead > > > > a community open source project while ignoring the community to > > railroad > > > > your views and push your personal agenda, you should ask the mentors > > what > > > > happens to open source projects when the committers ignore the > > community. > > > > > > > > > On Mar 30, 2016, at 10:25 AM, moon soo Lee <m...@apache.org> > wrote: > > > > > > > > > > Amos, > > > > > > > > > > Please get familiarize with yourself more about contribution and > > review > > > > > process. > > > > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-zeppelin/blob/master/CONTRIBUTING.md#the-review-process > > > > > > > > > > It's not ready while PPMC really made no +1 vote for 208 for last > > > couple > > > > of > > > > > months while it's breaking CI. > > > > > > > > > > Consensus is, we need R interpreter. Some people prefer > > implementation > > > of > > > > > 208 and some people prefer implementation of 702. > > > > > > > > > > No consensus of merging the code that breaks CI. > > > > > Please do not impose your personal desires on the others. > > > > > > > > > > Anyway, i think you're very close to making 208 CI green. > > > > > Hope you can make it anytime soon. > > > > > > > > > > Thanks, > > > > > moon > > > > > > > > > >> On Tue, Mar 29, 2016 at 5:42 PM Amos Elberg < > amos.elb...@gmail.com> > > > > wrote: > > > > >> > > > > >> "Merge them when they're ready" doesn't work --- 208 has been > ready > > > for > > > > six > > > > >> months. Meanwhile, Moon has been feverishly making commits to > 702, > > > and > > > > >> declared it "ready to merge" over the weekend, even though no-one > > had > > > > >> tested it. > > > > >> > > > > >> That is the exact reason why this thread was started. > > > > >> > > > > >> What I've asked for consensus on is that 208 *IS* ready. That is > > what > > > > >> numerous people have already supported. The only person who says > > that > > > > it > > > > >> isn't, is Moon. > > > > >> > > > > >> I am fine with Tom's suggestion. > > > > >> > > > > >> But "merge them when Moon says they're ready"? The community has > > been > > > > >> saying 208 is ready for *months*. It is literally one individual > > who > > > > has > > > > >> prevented this from being merged in all of that time. > > > > >> > > > > >> I will explain to Tom off-list what's going on with CI; he's new > to > > > all > > > > of > > > > >> this. > > > > >> > > > > >> > > > > >> On Tue, Mar 29, 2016 at 7:54 PM, Jeff Steinmetz < > > > > >> jeffrey.steinm...@gmail.com > > > > >>> wrote: > > > > >> > > > > >>> +1 RE: Merge 208 and/or 702 as they're ready - so zeppelin can > > > > benefit > > > > >>> from the merits of both approaches. > > > > >>> That’s been my understanding as well, as discussed in this [1] > > > thread. > > > > >>> > > > > >>> > > > > >>> +1 on Tom’s comments as well. > > > > >>> Hoping for no more arguing in this dev list - so we can get back > to > > > our > > > > >>> regularly scheduled positive ASF contribution spirit. > > > > >>> > > > > >>> Best, > > > > >>> Jeff > > > > >>> > > > > >>> [1] > > > > >> > > > > > > > > > > http://apache-zeppelin-incubating-dev-mailing-list.75694.x6.nabble.com/R-interpreter-in-Zeppelin-further-steps-tp6967.html > > > > >>> > > > > >>> > > > > >>> > > > > >>>> On 3/29/16, 4:35 PM, "moon soo Lee" <m...@apache.org> wrote: > > > > >>>> > > > > >>>> My position is merge 208 and/or 702 as they're ready. So > zeppelin > > > can > > > > >> take > > > > >>>> both merits. > > > > >>>> > > > > >>>> I've seen some people +1 on 208 in this thread. And i'm clearly > +1 > > > for > > > > >>>> merge both, and some other people are +1 for merge both in > > previous > > > > >>>> thread[1]. And Jeff provided very good technical merits of two. > > And > > > no > > > > >> -1 > > > > >>>> on 208, 702. > > > > >>>> > > > > >>>> So i think plan on merge 208 and 702 is well aligned with > > community > > > > >>> desire. > > > > >>>> > > > > >>>> That's my understanding. > > > > >>>> > > > > >>>> Now, can you explain why do you think people disagree on this > > > > position? > > > > >>>> > > > > >>>> Thanks, > > > > >>>> moon > > > > >>>> > > > > >>>> [1] > > > > >> > > > > > > > > > > http://apache-zeppelin-incubating-dev-mailing-list.75694.x6.nabble.com/R-interpreter-in-Zeppelin-further-steps-tp6967.html > > > > >>>> > > > > >>>>> On Tue, Mar 29, 2016 at 4:00 PM Amos Elberg < > > amos.elb...@gmail.com > > > > > > > > >>>> wrote: > > > > >>>> > > > > >>>>> No Moon - You've got it backwards. No-one supports *your* > > > position. > > > > >>>>> > > > > >>>>> *You* are ignoring the community and attempting to impose your > > will > > > > on > > > > >>>>> everyone else. > > > > >>>>> > > > > >>>>> This is the fifth time we've had a thread about this, and the > > fifth > > > > >> time > > > > >>>>> its come out the same way. > > > > >>>>> > > > > >>>>> On Tue, Mar 29, 2016 at 6:55 PM, moon soo Lee <m...@apache.org > > > > > > >> wrote: > > > > >>>>> > > > > >>>>>>> > > > > >>>>>>> Moon --- People disagree with you. > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> Amos, disagreeing on any opinion is fine but you're not > > > representing > > > > >>> all > > > > >>>>>> people in the community. > > > > >>>>>> > > > > >>>>>> So you'll need to explain a) who disagree on b) what and c) > > where > > > > >>> other > > > > >>>>>> people find those disagreement. > > > > >>>>>> Otherwise, it's going to be considered you're just trying to > > > impose > > > > >>> your > > > > >>>>>> personal desires on the others. > > > > >>>>>> > > > > >>>>>> So could you answer a), b) and c) ? > > > > >>>>>> > > > > >>>>>> Thanks, > > > > >>>>>> moon > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> On Tue, Mar 29, 2016 at 12:00 PM Amos B. Elberg < > > > > >>> amos.elb...@gmail.com> > > > > >>>>>> wrote: > > > > >>>>>> > > > > >>>>>>> Moon --- People disagree with you. Rather than keep going > > > > >>>>> back-and-forth > > > > >>>>>>> about it, I started this discussion to clear up any question > > > about > > > > >>> the > > > > >>>>>>> sense of the community. > > > > >>>>>>> > > > > >>>>>>> This is the apache way. You have said many times, "community > > > > >> before > > > > >>>>>> code." > > > > >>>>>>> > > > > >>>>>>> How many more people do you need to hear from? How many more > > > > >>>>> discussion > > > > >>>>>>> threads saying the same thing do you need to see? > > > > >>>>>>> > > > > >>>>>>>> On Mar 29, 2016, at 2:50 PM, moon soo Lee <m...@apache.org> > > > > >>> wrote: > > > > >>>>>>>> > > > > >>>>>>>> Hi, > > > > >>>>>>>> > > > > >>>>>>>> Answers inline. > > > > >>>>>>>> > > > > >>>>>>>>> On Tue, Mar 29, 2016 at 11:08 AM Amos Elberg < > > > > >>> amos.elb...@gmail.com > > > > >>>>>> > > > > >>>>>>> wrote: > > > > >>>>>>>>> > > > > >>>>>>>>> Kos & Moon -- > > > > >>>>>>>>> > > > > >>>>>>>>> The gist of this thread, is that people disagree with what > > > > >> Moon > > > > >>>>> has > > > > >>>>>>> said > > > > >>>>>>>>> regarding code quality, whether 208 breaks CI (and if so, > > why), > > > > >>> and > > > > >>>>>>> whether > > > > >>>>>>>>> its appropriate to merge 702, as Moon has proposed. > > > > >>>>>>>> Like Kos mentioned, please do not impose your personal > desires > > > > >> on > > > > >>> the > > > > >>>>>>>> others. You don't need to try define people agree on > something > > > > >> or > > > > >>>>>>> disagree > > > > >>>>>>>> on something. > > > > >>>>>>>> > > > > >>>>>>>> People have different opinions. Just let people express > their > > > > >>> opinion > > > > >>>>>>>> themselves. > > > > >>>>>>>> > > > > >>>>>>>> Can you do that? > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>>> Since this saga started, we've had 5 threads to get the > > sense > > > > >>> of > > > > >>>>> the > > > > >>>>>>>>> community on what to do. All of those came out the same > way. > > > > >>> More > > > > >>>>>>> than a > > > > >>>>>>>>> dozen people have asked for the same thing. > > > > >>>>>>>> Isn't it time to just get this done so we can all move on? > > > > >>>>>>>>> > > > > >>>>>>>>> (If Moon believes there's a real CI issue here, I have no > > doubt > > > > >>> that > > > > >>>>>> it > > > > >>>>>>>>> will be solved an hour after merge --- as Moon undertook to > > do > > > > >>> back > > > > >>>>> in > > > > >>>>>>>>> December.) > > > > >>>>>>>> I have no good technical reason to merge single PR that does > > not > > > > >>> pass > > > > >>>>>> CI > > > > >>>>>>>> and not merge all other PR that also does not pass CI. > > > > >>>>>>>> > > > > >>>>>>>> As i explained in previous email, it's more like problem of > > > > >>> policy. > > > > >>>>> If > > > > >>>>>>> you > > > > >>>>>>>> have good technical reason to change the policy, please > start > > a > > > > >>>>>>> discussion. > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> Thanks, > > > > >>>>>>>> moon > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>>> > > > > >>>>>>>>> > > > > >>>>>>>>> > > > > >>>>>>>>>> On Tue, Mar 29, 2016 at 1:53 PM, moon soo Lee < > > > > >> m...@apache.org> > > > > >>>>>> wrote: > > > > >>>>>>>>>> > > > > >>>>>>>>>> Hi, > > > > >>>>>>>>>> > > > > >>>>>>>>>> Regarding CI test about 208, > > > > >>>>>>>>>> > > > > >>>>>>>>>> Zeppelin have several profiles for CI test. Each profile > > tests > > > > >>>>>> Zeppelin > > > > >>>>>>>>>> with different Spark Version. Also it different profiles > > > > >>> different > > > > >>>>>>> level > > > > >>>>>>>>> of > > > > >>>>>>>>>> tests (integration test using selenium). > > > > >>>>>>>>>> > > > > >>>>>>>>>> Current status of 208 in CI test is, passing single > profile, > > > > >>> fails > > > > >>>>>> all > > > > >>>>>>>>>> other profiles. Which is exactly the same status that i > have > > > > >>> helped > > > > >>>>>> 208 > > > > >>>>>>>>> few > > > > >>>>>>>>>> months ago by the way. see. > > > > >> > > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-173423103 > > > > >>>>>>>>>> > > > > >>>>>>>>>> 208 has some code interacts with Spark. And 7 CI profile > out > > > > >> of > > > > >>> 8 > > > > >>>>> are > > > > >>>>>>> for > > > > >>>>>>>>>> test code against various version of Spark. While Zeppelin > > > > >> used > > > > >>> to > > > > >>>>>>>>> supports > > > > >>>>>>>>>> multiple version of Spark, from range of 1.1 ~ 1.6. > > > > >>>>>>>>>> > > > > >>>>>>>>>> SparkInterpreter (scala) > > > > >>>>>>>>>> PySparkInterpreter (python) > > > > >>>>>>>>>> SqlInterpreter (spark sql) > > > > >>>>>>>>>> > > > > >>>>>>>>>> supports all versions of spark in the profile (pyspark > > > > >> supports > > > > >>>>> from > > > > >>>>>>>>> 1.2). > > > > >>>>>>>>>> I think it's very strait forward to expect the same > quality > > > > >> for > > > > >>> R > > > > >>>>>>>>>> interpreter. > > > > >>>>>>>>>> > > > > >>>>>>>>>> I can suggest two possible way, > > > > >>>>>>>>>> > > > > >>>>>>>>>> - Keep working on make all profile of CI green. While 208 > > > > >>> already > > > > >>>>>>> passes > > > > >>>>>>>>>> one profile and test in all other profiles are the same > but > > > > >> only > > > > >>>>>>> against > > > > >>>>>>>>>> different spark version, it won't be that difficult to > make > > it > > > > >>> pass > > > > >>>>>> all > > > > >>>>>>>>>> other profile. > > > > >>>>>>>>>> - Or activate 208 only for spark 1.6 and mark/document > which > > > > >> is > > > > >>>>>> minimum > > > > >>>>>>>>>> version requirement of spark. Like Pyspark interpreter did > > > > >>>>> (requires > > > > >>>>>>>>> spark > > > > >>>>>>>>>> 1.2 or newer). > > > > >>>>>>>>>> > > > > >>>>>>>>>> > > > > >>>>>>>>>> Regarding code merge policy, > > > > >>>>>>>>>> > > > > >>>>>>>>>> Zeppelin community had been make sure CI pass before merge > > in > > > > >> to > > > > >>>>>>> master, > > > > >>>>>>>>>> since it's beginning, until now. That's i believe another > > > > >>> consensus > > > > >>>>>>> that > > > > >>>>>>>>> we > > > > >>>>>>>>>> believed we have in the community. > > > > >>>>>>>>>> > > > > >>>>>>>>>> That's only reason keep spoken why 208 is not merged for > > last > > > > >> 4 > > > > >>>>>> months. > > > > >>>>>>>>>> And only reason for all other PR forced to make CI green > > > > >> before > > > > >>> it > > > > >>>>>>> get's > > > > >>>>>>>>>> merged. > > > > >>>>>>>>>> > > > > >>>>>>>>>> Personally i think not breaking master branch is valuable > > > > >> while > > > > >>>>> that > > > > >>>>>>>>> makes > > > > >>>>>>>>>> any contributor start contribution safely at any point > from > > > > >>> master > > > > >>>>>>>>> branch. > > > > >>>>>>>>>> And users who want to try latest community work can safely > > > > >> test > > > > >>>>>>> Zeppelin > > > > >>>>>>>>>> from master branch. > > > > >>>>>>>>>> > > > > >>>>>>>>>> But if anyone think Zeppelin community need to discuss > about > > > > >> it, > > > > >>>>>> please > > > > >>>>>>>>>> start a discussion. I'm happy to see discussion happens. > > > > >>>>>>>>>> > > > > >>>>>>>>>> Thanks, > > > > >>>>>>>>>> moon > > > > >>>>>>>>>> > > > > >>>>>>>>>> > > > > >>>>>>>>>> On Tue, Mar 29, 2016 at 9:31 AM Konstantin Boudnik < > > > > >>> c...@apache.org > > > > >>>>>> > > > > >>>>>>>>> wrote: > > > > >>>>>>>>>> > > > > >>>>>>>>>>> hmm.... that's getting weird again. So, far I failed to > > see: > > > > >>>>>>>>>>> - CI issues being addressed. If the consensus of the > > > > >> community > > > > >>> to > > > > >>>>>>>>> merge > > > > >>>>>>>>>> in > > > > >>>>>>>>>>> something, break the CI and collect the technical debts > - > > > > >>> that's > > > > >>>>>>>>> fine > > > > >>>>>>>>>>> and > > > > >>>>>>>>>>> that's your choice (I am not here to pass the judgement > on > > > > >>> the > > > > >>>>>>>>> quality > > > > >>>>>>>>>>> of > > > > >>>>>>>>>>> the code) > > > > >>>>>>>>>>> - a consensus to keep anyone away from _anything_ in the > > > > >>> project > > > > >>>>>>>>>> matters. > > > > >>>>>>>>>>> Please do not impose your personal desires on the > others. > > > > >>> While > > > > >>>>>>>>> you're > > > > >>>>>>>>>>> entitled to express them (free speech and all that), no > > one > > > > >>> is > > > > >>>>>>>>>> entitled > > > > >>>>>>>>>>> to > > > > >>>>>>>>>>> listen, less oblige by it (based on the same principles > of > > > > >>>>>>>>> individual > > > > >>>>>>>>>>> rights). > > > > >>>>>>>>>>> > > > > >>>>>>>>>>> So, please keep it civil and find a way to improve the > > code, > > > > >> if > > > > >>>>>>> needed, > > > > >>>>>>>>>>> and get > > > > >>>>>>>>>>> it in once the committers are satisfied with its quality. > > > > >>>>>>>>>>> > > > > >>>>>>>>>>> Cos > > > > >>>>>>>>>>> > > > > >>>>>>>>>>>> On Tue, Mar 29, 2016 at 11:51AM, Amos B. Elberg wrote: > > > > >>>>>>>>>>>> Moon - no. That is the opposite of what people are > saying. > > > > >>>>>>>>>>>> > > > > >>>>>>>>>>>> I started this thread because I feel that you are > > > > >> disregarding > > > > >>>>> the > > > > >>>>>>>>>>> consensus > > > > >>>>>>>>>>>> of the community. > > > > >>>>>>>>>>>> > > > > >>>>>>>>>>>> The thread asks for two things - 208 to be merged > without > > > > >>> further > > > > >>>>>>>>>> delay, > > > > >>>>>>>>>>> and > > > > >>>>>>>>>>>> for you to stay out of the issue of R interpreters > > entirely. > > > > >>> 702 > > > > >>>>>> can > > > > >>>>>>>>>> be > > > > >>>>>>>>>>>> addressed after 208 is merged. > > > > >>>>>>>>>>>> > > > > >>>>>>>>>>>> How many more people do you need to hear from? > > > > >>>>>>>>>>>> > > > > >>>>>>>>>>>>> On Mar 29, 2016, at 5:40 AM, moon soo Lee < > > m...@apache.org > > > > >>> > > > > >>>>>> wrote: > > > > >>>>>>>>>>>>> > > > > >>>>>>>>>>>>> Hi folks, > > > > >>>>>>>>>>>>> > > > > >>>>>>>>>>>>> I didn't see anyone disagreement merge 208 and/or 702 > in > > > > >> this > > > > >>>>>>>>> thread > > > > >>>>>>>>>>> and > > > > >>>>>>>>>>>>> previous thread [1], as they're ready. while they both > > have > > > > >>>>>>>>> technical > > > > >>>>>>>>>>>>> merits as Jeff summarized really well. > > > > >>>>>>>>>>>>> > > > > >>>>>>>>>>>>> Now i can see 208 finally made some progress on CI [2]. > > > > >> Hope > > > > >>>>>>>>> continue > > > > >>>>>>>>>>> the > > > > >>>>>>>>>>>>> work and make CI green. Also I can see 702 is trying to > > > > >>>>> finishing > > > > >>>>>>>>> up > > > > >>>>>>>>>>> and > > > > >>>>>>>>>>>>> waiting for CI become green. > > > > >>>>>>>>>>>>> > > > > >>>>>>>>>>>>> I don't want to merge something that breaks CI. If > then, > > it > > > > >>>>>> becomes > > > > >>>>>>>>>>> make > > > > >>>>>>>>>>>>> very difficult to verify all other contributions. Other > > > > >>>>>>>>> contributions > > > > >>>>>>>>>>> are > > > > >>>>>>>>>>>>> as important as these two. Hope community can > understand > > > > >>> that. > > > > >>>>>>>>>>>>> > > > > >>>>>>>>>>>>> Considering recent progress of both contributions, i > > expect > > > > >>>>>> they'll > > > > >>>>>>>>>> be > > > > >>>>>>>>>>>>> ready anytime soon. And then we can finally merge them. > > > > >>>>>>>>>>>>> > > > > >>>>>>>>>>>>> About merging 702, 208 contributions, does this sounds > > > > >> clear? > > > > >>>>>>>>>>>>> > > > > >>>>>>>>>>>>> If they're both merged, It's possible to improve both > > > > >>>>> RInterpreter > > > > >>>>>>>>> by > > > > >>>>>>>>>>>>> taking each others advantage. Therefore, no reason to > > worry > > > > >>> at > > > > >>>>>> this > > > > >>>>>>>>>>> point > > > > >>>>>>>>>>>>> about which one is better, which one has advantages, > > which > > > > >>> one > > > > >>>>>> will > > > > >>>>>>>>>>> merge > > > > >>>>>>>>>>>>> before the other, etc. Both have pros and cons and both > > > > >> will > > > > >>>>> help > > > > >>>>>>>>>>> Zeppelin > > > > >>>>>>>>>>>>> thankfully. > > > > >>>>>>>>>>>>> > > > > >>>>>>>>>>>>> Thanks, > > > > >>>>>>>>>>>>> moon > > > > >>>>>>>>>>>>> > > > > >>>>>>>>>>>>> [1] > > > > >> > > > > > > > > > > http://apache-zeppelin-incubating-dev-mailing-list.75694.x6.nabble.com/R-interpreter-in-Zeppelin-further-steps-tp6967.html > > > > >>>>>>>>>>>>> [2] > > > > >> > > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-202682652 > > > > >>>>>>>>>>>>> > > > > >>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> On Tue, Mar 29, 2016 at 1:45 AM enzo < > > > > >>>>>>>>>> e...@smartinsightsfromdata.com> > > > > >>>>>>>>>>> wrote: > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> I am looking forward to see 208 merged, *soon* please. > > > > >>> From my > > > > >>>>>>>>>> tests > > > > >>>>>>>>>>> it > > > > >>>>>>>>>>>>>> seems that this should be the priority. > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> I think 702 has merits (but I’ve used it less) and > > > > >> deserves > > > > >>> to > > > > >>>>> be > > > > >>>>>>>>>>> merged > > > > >>>>>>>>>>>>>> too once ready. > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> Ultimately after a period of "real road” testing we > > will > > > > >> be > > > > >>>>> able > > > > >>>>>>>>> to > > > > >>>>>>>>>>>>>> understand what we really need. > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> E.g. from past discussions I am not convinced that > > either > > > > >> PR > > > > >>>>>>>>> would, > > > > >>>>>>>>>>>>>> as-it-is, support fully the needs of a multi-user > > > > >> Zeppelin > > > > >>>>>> Server > > > > >>>>>>>>>>> approach > > > > >>>>>>>>>>>>>> (something similar to RStudio Server Professional to > get > > > > >> an > > > > >>>>>> idea). > > > > >>>>>>>>>> A > > > > >>>>>>>>>>>>>> period of use and gradual evolution (and possibly > > merge?) > > > > >>> will > > > > >>>>> be > > > > >>>>>>>>>>> required. > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> The sooner we start the better. > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> Enzo > > > > >>>>>>>>>>>>>> e...@smartinsightsfromdata.com > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>> On 29 Mar 2016, at 07:08, Jeff Steinmetz < > > > > >>>>>>>>>>> jeffrey.steinm...@gmail.com> > > > > >>>>>>>>>>>>>>> wrote: > > > > >>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>> I’m not affiliated to either author nor have any > > > > >>> attachment to > > > > >>>>>> an > > > > >>>>>>>>>>>>>> specific outcome - and happy to continue being as > > > > >> objective > > > > >>> and > > > > >>>>>>>>>>> unbiased as > > > > >>>>>>>>>>>>>> possible. > > > > >>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>> I would say they now have different philosophical > > > > >>> approaches > > > > >>>>> (as > > > > >>>>>>>>> of > > > > >>>>>>>>>>> the > > > > >>>>>>>>>>>>>> March 23rd merge of datalayer#7 to 702). > > > > >>>>>>>>>>>>>>> I agree with Amos Elberg that 702 has changed > > directions > > > > >> a > > > > >>> few > > > > >>>>>>>>>> times. > > > > >>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>> Re: commits to 702 by Leemoonsoo on March 23 via > > > > >>> datalayer#7: > > > > >>>>>>>>>>>>>>> I found the current state of the 702 PR to be > succinct, > > > > >> in > > > > >>>>>> terms > > > > >>>>>>>>>> of > > > > >>>>>>>>>>>>>> it’s code base, via its use of the SparkR dependency - > > > > >>> which is > > > > >>>>>>>>>>> already > > > > >>>>>>>>>>>>>> baked into spark distribution. > > > > >>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>> The 702 code base appears to be easier to maintain > > using > > > > >>> this > > > > >>>>>>>>>>> approach > > > > >>>>>>>>>>>>>> (less code, no rscala source, no BSD licensing > additions > > > > >>>>>> required, > > > > >>>>>>>>>>> easier > > > > >>>>>>>>>>>>>> to read). > > > > >>>>>>>>>>>>>>> 702 packages correctly with -Pbuild-distr as > expected - > > > > >>> i.e. > > > > >>>>> it > > > > >>>>>>>>>> works > > > > >>>>>>>>>>>>>> out of gate from the distribution directory. > > > > >>>>>>>>>>>>>>> The build profile -Psparkr worked as expected, and > the > > > > >>>>> addition > > > > >>>>>>>>> of > > > > >>>>>>>>>>> this > > > > >>>>>>>>>>>>>> profile felt intuitive to me. > > > > >>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>> Myself and a colleague that uses R extensively > noticed > > > > >> (as > > > > >>>>> Amos > > > > >>>>>>>>>>> Elberg > > > > >>>>>>>>>>>>>> reminded us): > > > > >>>>>>>>>>>>>>> 208 handles passing arrays and other data types > between > > > > >>> scala > > > > >>>>> & > > > > >>>>>> R > > > > >>>>>>>>>>> more > > > > >>>>>>>>>>>>>> gracefully than 702. > > > > >>>>>>>>>>>>>>> 208 handles the output of intermediate R calls more > > > > >>> gracefully > > > > >>>>>>>>> than > > > > >>>>>>>>>>> 702. > > > > >>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>> Beyond that: > > > > >>>>>>>>>>>>>>> 208 Requires SPARK_HOME to be set or the interpreter > > > > >> hangs > > > > >>>>> with > > > > >>>>>>>>> no > > > > >>>>>>>>>>>>>> error. It’s been mentioned by the 208 author that the > > > > >>>>>> requirement > > > > >>>>>>>>>> to > > > > >>>>>>>>>>> set > > > > >>>>>>>>>>>>>> SPARK_HOME is a feature. I think we could revisit > this > > > > >>>>>> assumption > > > > >>>>>>>>>>> now that > > > > >>>>>>>>>>>>>> I see how 702 handles this with defaults via a > graceful > > > > >>>>> fallback. > > > > >>>>>>>>>>>>>>> 702 works fine with zero configuration, I.e for those > > > > >> that > > > > >>>>> want > > > > >>>>>>>>> to > > > > >>>>>>>>>>> test > > > > >>>>>>>>>>>>>> locally with no separate spark distribution installed > > > > >>>>> (SPARK_HOME > > > > >>>>>>>>>>> does not > > > > >>>>>>>>>>>>>> need to be set). > > > > >>>>>>>>>>>>>>> 702 having just an %r interpreter, and having it as > > part > > > > >> of > > > > >>>>> the > > > > >>>>>>>>>> spark > > > > >>>>>>>>>>>>>> interpreter (same subdirectory) feels like a cleaner > > > > >>> approach > > > > >>>>>>>>> (this > > > > >>>>>>>>>> is > > > > >>>>>>>>>>>>>> arguably a philosophical difference again). > > > > >>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>> It feels like an exhaustive list of > > > > >>>>> `.z.show.googleVis(Motion)` > > > > >>>>>>>>>> type > > > > >>>>>>>>>>>>>> calls in 208 could bloom into unnecessary code > > maintenance > > > > >>>>>>>>> overhead > > > > >>>>>>>>>>> and > > > > >>>>>>>>>>>>>> required additions every time a new chart library is > > > > >>>>> introduced, > > > > >>>>>>>>> vs. > > > > >>>>>>>>>>> a more > > > > >>>>>>>>>>>>>> generic show method. Perhaps a follow on improvement > > post > > > > >>>>> merge > > > > >>>>>>>>>>> (applies > > > > >>>>>>>>>>>>>> to both PRs). > > > > >>>>>>>>>>>>>>> This same chart rendering works in 702 with > > > > >> `print(Motion, > > > > >>>>>>>>>>> tag='chart’)` > > > > >>>>>>>>>>>>>> which isn’t necessarily better or worse, again, a > > > > >> different > > > > >>>>>>>>>>> philosophical > > > > >>>>>>>>>>>>>> approach. > > > > >>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>> They both have merit in different regards. It’s > > doesn’t > > > > >>> feel > > > > >>>>>>>>>>>>>> appropriate to make a broad statement that "no-one > > > > >> supported > > > > >>>>>> 702”. > > > > >>>>>>>>>>>>>>> If I had a magic wand, it would be a hybrid of the > two > > > > >>>>>>>>> approaches. > > > > >>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>> I look forward to continuing the review of each PR > > > > >>>>> individually > > > > >>>>>>>>> or > > > > >>>>>>>>>>> both > > > > >>>>>>>>>>>>>> collaboratively. > > > > >>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>> Regards, > > > > >>>>>>>>>>>>>>> Jeff > > > > >>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>> On 3/28/16, 4:13 PM, "Sourav Mazumder" < > > > > >>>>>>>>>> sourav.mazumde...@gmail.com > > > > >>>>>>>>>>>> > > > > >>>>>>>>>>>>>>> wrote: > > > > >>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>> All said and done we had enough discussion on this > > point > > > > >>> for > > > > >>>>>>>>> many > > > > >>>>>>>>>>> months > > > > >>>>>>>>>>>>>>>> now. As far as I know, 208 is the PR which > > > > >>> community/people > > > > >>>>>>>>> have > > > > >>>>>>>>>>> so far > > > > >>>>>>>>>>>>>>>> used mostly and successfully (at least me and > whoever > > I > > > > >>>>>>>>> introduced > > > > >>>>>>>>>>> to > > > > >>>>>>>>>>>>>> 208 > > > > >>>>>>>>>>>>>>>> for SparkR support). I thought it was going to be > > > > >> merged a > > > > >>>>> long > > > > >>>>>>>>>> time > > > > >>>>>>>>>>>>>> ago. > > > > >>>>>>>>>>>>>>>> May be what will make sense is to first integrate > the > > > > >> 208. > > > > >>>>>>>>> After > > > > >>>>>>>>>>> that, > > > > >>>>>>>>>>>>>> a > > > > >>>>>>>>>>>>>>>> new PR can be created on that and if 702 has > anything > > > > >>> extra > > > > >>>>>> then > > > > >>>>>>>>>>> that > > > > >>>>>>>>>>>>>>>> feature can be added. > > > > >>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>> Regards, > > > > >>>>>>>>>>>>>>>> Sourav > > > > >>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>> On Mon, Mar 28, 2016 at 12:37 AM, Eran Witkon < > > > > >>>>>>>>>> eranwit...@gmail.com > > > > >>>>>>>>>>>> > > > > >>>>>>>>>>>>>> wrote: > > > > >>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>> @Elberg, If I were you I would ask myself why isn't > > the > > > > >>>>>>>>> community > > > > >>>>>>>>>>>>>> taking > > > > >>>>>>>>>>>>>>>>> part in this debate? > > > > >>>>>>>>>>>>>>>>> Personally I prefer a team player as a contributor > > over > > > > >>> the > > > > >>>>>>>>> best > > > > >>>>>>>>>>>>>> developer. > > > > >>>>>>>>>>>>>>>>> just my 2c > > > > >>>>>>>>>>>>>>>>> Eran > > > > >>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>> On Mon, 28 Mar 2016 at 09:52 Amos B. Elberg < > > > > >>>>>>>>>> amos.elb...@gmail.com > > > > >>>>>>>>>>>> > > > > >>>>>>>>>>>>>> wrote: > > > > >>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>> Moon - I opened this discussion so it could take > > place > > > > >>> with > > > > >>>>>>>>> the > > > > >>>>>>>>>>>>>> community > > > > >>>>>>>>>>>>>>>>>> as a whole, not just you. > > > > >>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>> Suffice it to say, I disagree with every one of > the > > > > >>>>> technical > > > > >>>>>>>>>>> claims > > > > >>>>>>>>>>>>>>>>>> you've just made, and I don't trust your intent. > > > > >>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>> Let the community process happen. > > > > >>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>> On Mar 28, 2016, at 2:47 AM, moon soo Lee < > > > > >>>>> m...@apache.org> > > > > >>>>>>>>>>> wrote: > > > > >>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>> Hi, > > > > >>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>> Simply put, > > > > >>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>> - 702 and/or 208 will can merged as they're > ready. > > > > >> [1] > > > > >>>>>>>>>>>>>>>>>>> - 208 will not be merged while it does not pass > CI. > > > > >> If > > > > >>> you > > > > >>>>>>>>>> think > > > > >>>>>>>>>>> code > > > > >>>>>>>>>>>>>>>>> in > > > > >>>>>>>>>>>>>>>>>>> 208 is not a problem but CI itself or other part > of > > > > >>>>> Zeppelin > > > > >>>>>>>>> is > > > > >>>>>>>>>>>>>>>>> problem, > > > > >>>>>>>>>>>>>>>>>>> then that particular problem be fixed before > merge > > > > >> 208. > > > > >>>>>>>>>>>>>>>>>>> - 702 has proper integration test [2] > > > > >>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>> I'm not sure why you're so hard at devaluating > 702. > > > > >>>>>>>>>>>>>>>>>>> 702 is not something you need to beat and win. > 702 > > is > > > > >>>>>>>>> something > > > > >>>>>>>>>>> you > > > > >>>>>>>>>>>>>>>>> need > > > > >>>>>>>>>>>>>>>>>> to > > > > >>>>>>>>>>>>>>>>>>> help / learn / collaborate. > > > > >>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>> Will you able to show your ability to collaborate > > > > >> with > > > > >>>>> other > > > > >>>>>>>>>>>>>> community > > > > >>>>>>>>>>>>>>>>>>> members? > > > > >>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>> Thanks, > > > > >>>>>>>>>>>>>>>>>>> moon > > > > >>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>> [1] > > > > >> > > > > > > > > > > http://apache-zeppelin-incubating-dev-mailing-list.75694.x6.nabble.com/R-interpreter-in-Zeppelin-further-steps-tp6967.html > > > > >>>>>>>>>>>>>>>>>>> [2] > > > > >> > > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/702/files#diff-64a9440e811c5fba6ac1b61157fa6912R87 > > > > >>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>>> On Sun, Mar 27, 2016 at 7:11 PM Amos Elberg < > > > > >>>>>>>>>>> amos.elb...@gmail.com> > > > > >>>>>>>>>>>>>>>>>> wrote: > > > > >>>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>>> I am saddened to have to start this thread > > *again*. > > > > >>>>> While > > > > >>>>>> I > > > > >>>>>>>>>>> thought > > > > >>>>>>>>>>>>>>>>> we > > > > >>>>>>>>>>>>>>>>>> had > > > > >>>>>>>>>>>>>>>>>>>> reached consensus on this, several times over, > > > > >>> apparently > > > > >>>>>>>>> some > > > > >>>>>>>>>>>>>> people > > > > >>>>>>>>>>>>>>>>>>>> disagree. I hope this will be the last time. > > > > >>>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>>> With this thread, I am asking the community to > > reach > > > > >>>>>>>>> consensus > > > > >>>>>>>>>>> (1) > > > > >>>>>>>>>>>>>>>>> That > > > > >>>>>>>>>>>>>>>>>> 208 > > > > >>>>>>>>>>>>>>>>>>>> should be merged this week, without further > delay; > > > > >> and > > > > >>>>> (2) > > > > >>>>>>>>>> That > > > > >>>>>>>>>>> Moon > > > > >>>>>>>>>>>>>>>>> Lee > > > > >>>>>>>>>>>>>>>>>>>> Soo and Felix Cheung take no further part in the > > > > >>>>>> discussions > > > > >>>>>>>>>> of > > > > >>>>>>>>>>> 208 > > > > >>>>>>>>>>>>>>>>> and > > > > >>>>>>>>>>>>>>>>>>>> 702. > > > > >>>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>>> This PR has been pending since August. It has > been > > > > >>>>> stalled > > > > >>>>>>>>>> that > > > > >>>>>>>>>>>>>> entire > > > > >>>>>>>>>>>>>>>>>> time > > > > >>>>>>>>>>>>>>>>>>>> for no technical reason. > > > > >>>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>>> We reached agreement to merge 208 in November, > > again > > > > >>> in > > > > >>>>>>>>>>> December, > > > > >>>>>>>>>>>>>> and > > > > >>>>>>>>>>>>>>>>>> again > > > > >>>>>>>>>>>>>>>>>>>> in February -- when Moon agreed to stay out of > the > > > > >>> issue. > > > > >>>>>>>>> At > > > > >>>>>>>>>>> that > > > > >>>>>>>>>>>>>>>>>> point, > > > > >>>>>>>>>>>>>>>>>>>> Alex, I, and others, began working on it, and > > > > >>> appeared to > > > > >>>>>> be > > > > >>>>>>>>>>> making > > > > >>>>>>>>>>>>>>>>>>>> substantial progress. > > > > >>>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>>> And then Alex just stopped. Instead, he > commenced > > > > >> the > > > > >>>>>>>>> thread > > > > >>>>>>>>>>> saying > > > > >>>>>>>>>>>>>>>>>> that a > > > > >>>>>>>>>>>>>>>>>>>> consensus had to be reached on 208 and 702. > Until > > > > >>> that > > > > >>>>>>>>> point, > > > > >>>>>>>>>>>>>>>>>> essentially > > > > >>>>>>>>>>>>>>>>>>>> no-one had paid attention to 702. In the > > discussion > > > > >>> that > > > > >>>>>>>>>>> followed, > > > > >>>>>>>>>>>>>> we > > > > >>>>>>>>>>>>>>>>>>>> reached a consensus to merge 208 as soon as > > > > >> possible. > > > > >>>>>> After > > > > >>>>>>>>>> the > > > > >>>>>>>>>>>>>>>>> thread > > > > >>>>>>>>>>>>>>>>>> had > > > > >>>>>>>>>>>>>>>>>>>> died, Alex asked if anyone had additional > > comments, > > > > >>> and > > > > >>>>>> Moon > > > > >>>>>>>>>>>>>> popped-in > > > > >>>>>>>>>>>>>>>>>> to > > > > >>>>>>>>>>>>>>>>>>>> insist that both PRs be merged. Again, no-one > > > > >>> supported > > > > >>>>>>>>> 702. > > > > >>>>>>>>>>> At > > > > >>>>>>>>>>>>>> all. > > > > >>>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>>> Each time I said "we had a consensus before, > does > > > > >>> anyone > > > > >>>>>>>>> want > > > > >>>>>>>>>> to > > > > >>>>>>>>>>>>>>>>> change > > > > >>>>>>>>>>>>>>>>>>>> it," Alex or Moon steered the discussion away. > > The > > > > >>> final > > > > >>>>>>>>> vote > > > > >>>>>>>>>>> was > > > > >>>>>>>>>>>>>> not > > > > >>>>>>>>>>>>>>>>>> to > > > > >>>>>>>>>>>>>>>>>>>> merge 702 or merge "both" -- it was to treat > them > > as > > > > >>>>> normal > > > > >>>>>>>>>> PRs. > > > > >>>>>>>>>>>>>>>>>> (Although > > > > >>>>>>>>>>>>>>>>>>>> one person did want both merged simultaneously.) > > > > >> That > > > > >>>>>> would > > > > >>>>>>>>>>> mean > > > > >>>>>>>>>>>>>>>>>>>> completing 208 on its merits and then evaluating > > > > >> 702. > > > > >>>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>>> At the time, I objected to the discussion, > > because I > > > > >>>>>> thought > > > > >>>>>>>>>> the > > > > >>>>>>>>>>>>>> whole > > > > >>>>>>>>>>>>>>>>>>>> thing was a contrived excuse for Moon to reject > > 208 > > > > >> by > > > > >>>>>>>>> pushing > > > > >>>>>>>>>>> 702. > > > > >>>>>>>>>>>>>>>>>> That > > > > >>>>>>>>>>>>>>>>>>>> is exactly what he is now seeking to do. > > > > >>>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>>> *Status of 208 & 702* > > > > >>>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>>> PR 208 has been feature-complete and testable > > since > > > > >>> early > > > > >>>>>>>>>>> September. > > > > >>>>>>>>>>>>>>>>> It > > > > >>>>>>>>>>>>>>>>>>>> has been adopted by more than 1000 users, who I > > have > > > > >>> been > > > > >>>>>>>>>>> supporting > > > > >>>>>>>>>>>>>>>>> for > > > > >>>>>>>>>>>>>>>>>>>> more than six months. The code has not > undergone > > > > >> any > > > > >>>>> major > > > > >>>>>>>>>>> changes > > > > >>>>>>>>>>>>>>>>>> since > > > > >>>>>>>>>>>>>>>>>>>> September. There are no known bugs, and no > > > > >> outstanding > > > > >>>>>>>>> feature > > > > >>>>>>>>>>>>>>>>> requests > > > > >>>>>>>>>>>>>>>>>>>> that can be satisfied without major changes to > the > > > > >>>>> Zeppelin > > > > >>>>>>>>>>>>>>>>>> architecture. > > > > >>>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>>> 208 does *not* fail CI. 208 includes extensive > > unit > > > > >>>>> tests > > > > >>>>>>>>> of > > > > >>>>>>>>>>> the > > > > >>>>>>>>>>>>>>>>>> R-Spark > > > > >>>>>>>>>>>>>>>>>>>> integration because this turned out to get > broken > > by > > > > >>>>>> changes > > > > >>>>>>>>>> in > > > > >>>>>>>>>>>>>>>>> Zeppelin > > > > >>>>>>>>>>>>>>>>>>>> often. Because CI is unable at present to > > provide a > > > > >>>>>>>>>> consistent > > > > >>>>>>>>>>>>>>>>>>>> environment, 208's *OWN UNIT TESTS*, which pass > > when > > > > >>> run > > > > >>>>> on > > > > >>>>>>>>> an > > > > >>>>>>>>>>>>>>>>> ordinary > > > > >>>>>>>>>>>>>>>>>>>> machine, fail when run on CI. > > > > >>>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>>> 208 does need a push for compatibility with a > > > > >> recently > > > > >>>>>>>>> adopted > > > > >>>>>>>>>>> PR -- > > > > >>>>>>>>>>>>>>>>>> that > > > > >>>>>>>>>>>>>>>>>>>> is work I've essentially completed, but have not > > > > >>> pushed. > > > > >>>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>>> PR 702 is a re-design based on 208 -- not just > > > > >>>>>> architecture, > > > > >>>>>>>>>> but > > > > >>>>>>>>>>>>>> right > > > > >>>>>>>>>>>>>>>>>> down > > > > >>>>>>>>>>>>>>>>>>>> to the choice of demo images, which were taken > > from > > > > >>> 208's > > > > >>>>>>>>>>>>>>>>> documentation. > > > > >>>>>>>>>>>>>>>>>>>> In fact, 702 has had been re-engineered several > > > > >> times > > > > >>> to > > > > >>>>>>>>> more > > > > >>>>>>>>>>>>>> closely > > > > >>>>>>>>>>>>>>>>>>>> conform to 208's architecture and feature set. > > But > > > > >>> 702 > > > > >>>>>>>>> still > > > > >>>>>>>>>>>>>> remains > > > > >>>>>>>>>>>>>>>>>>>> feature-incomplete -- it cannot handle the range > > of > > > > >>>>>>>>>>> visualizations, > > > > >>>>>>>>>>>>>> R > > > > >>>>>>>>>>>>>>>>>>>> classes, etc., that 208 can. It is not stable > > code, > > > > >>> and > > > > >>>>>>>>> shows > > > > >>>>>>>>>> no > > > > >>>>>>>>>>>>>> signs > > > > >>>>>>>>>>>>>>>>>> of > > > > >>>>>>>>>>>>>>>>>>>> stabilizing any time soon. > > > > >>>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>>> No-one has adopted 702. It has changed > radically, > > > > >>>>>>>>>>> fundamentally, at > > > > >>>>>>>>>>>>>>>>>> least > > > > >>>>>>>>>>>>>>>>>>>> 4 times over the past two months since it was > > > > >>> submitted. > > > > >>>>>>>>> One > > > > >>>>>>>>>> of > > > > >>>>>>>>>>>>>> those > > > > >>>>>>>>>>>>>>>>>>>> changes was only days ago. > > > > >>>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>>> 702 also has no proper tests, which is the > excuse > > > > >> for > > > > >>> not > > > > >>>>>>>>>>> merging > > > > >>>>>>>>>>>>>> 208. > > > > >>>>>>>>>>>>>>>>>> 702 > > > > >>>>>>>>>>>>>>>>>>>> has things labelled "tests," but they don't > > actually > > > > >>>>>> attempt > > > > >>>>>>>>>> to > > > > >>>>>>>>>>>>>>>>> connect > > > > >>>>>>>>>>>>>>>>>> to > > > > >>>>>>>>>>>>>>>>>>>> R or Spark, which are the things that break and > > > > >> which > > > > >>>>>>>>>> therefore > > > > >>>>>>>>>>> need > > > > >>>>>>>>>>>>>>>>>>>> testing. > > > > >>>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>>> *** > > > > >>>>>>>>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>>>>>>>> I would like credit for my own work and design. > I > > > > >>> think I > > > > >>>>>>>>> have > > > > >>>>>>>>>>> more > > > > >>>>>>>>>>>>>>>>> than > > > > >>>>>>>>>>>>>>>>>>>> earned that. > > > > >> > > > > > > > > > >
