Done. Olaf
> Am 24.04.2016 um 11:25 schrieb Robert Metzger <[email protected]>: > > Can somebody give me Contributor permissions in the Bigtop JIRA? > I can not comment to issues anymore because of the JIRA spam lockdown :( > > On Thu, Apr 21, 2016 at 11:16 AM, Robert Metzger <[email protected]> > wrote: > >> Thank you for your responses. We use shading for two main purposes: >> - Relocating dependencies >> - building fat-jars / uber jars >> >> I understand that it doesn't make sense to deploy fat-jars in a controlled >> environment like bigtop. The dist jar of Flink is currently ~80MB, so its >> not as bad as with the others you've mentioned. >> Sadly, the build process of Flink is pretty complicated and has a lot of >> implications, so changing this is certainly not trivial. >> But its definitively on our TODO list to address the issue. >> >> Regarding the dependency relocation: We sadly have to do this because of >> projects like Guava, which do not provide stable interfaces across releases. >> Flink is shading away Hadoop's guava so that it doesn't conflict with our >> guava. We in turn also shade away our guava dependency so that our users >> can use whatever guava version they want to use. >> >> I'll keep building Flink from source for the bigtop packaging. >> >> >> >> On Wed, Apr 20, 2016 at 11:32 PM, Konstantin Boudnik <[email protected]> >> wrote: >> >>> +1 what Olaf said! We already see monstrosities that Spark builds and >>> provides >>> (close to 400MB binary blobs) or Zeppelin, which goes over half a gig >>> (sic!) >>> >>> We are producing the stack for very exact specifications, so can afford >>> removing a bunch of redistributed bits and pieces, as well to avoid the >>> shading nonsense. Build time is a very small price to pay to get clean >>> deployment. >>> >>> Cos >>> >>> On Wed, Apr 20, 2016 at 08:16PM, Olaf Flebbe wrote: >>>> Hi, >>>> >>>> Yep, we like to rebuild from source. AFAIK shading dependencies has to >>> do >>>> with uber jars. Since we do not use uber jars, can we please have simple >>>> jars, since we use a packaging system to do the deployment? >>>> >>>> Regards, >>>> Olaf >>>> >>>> >>>>> Am 20.04.2016 um 16:24 schrieb Robert Metzger <[email protected]>: >>>>> >>>>> Hi, >>>>> >>>>> I'm currently looking into the existing pull request for the Flink >>>>> integration [1]. >>>>> While doing that, I wondered if its a requirement from Bigtop to build >>>>> Flink from source when building the rpm / deb packages? >>>>> Not building from source would speed up the build a lot and it would >>> ensure >>>>> that the binary build is verified by the Flink community (there are >>> some >>>>> issues with maven dependency shading when building Flink with maven >>> 3.3.x) >>>>> >>>>> >>>>> >>>>> [1] https://github.com/apache/bigtop/pull/93/files >>>>> >>>>> On Wed, Apr 6, 2016 at 11:16 AM, Maximilian Michels <[email protected]> >>> wrote: >>>>> >>>>>> Hi Konstantin, hi Jay, >>>>>> >>>>>> Thanks for the replies and the prompt progress on the pull request. >>> It >>>>>> looks like we will have a Flink RPM package in Bigtop very soon. For >>>>>> the remaining integration we can open follow-up pull requests. >>>>>> >>>>>> If anything comes up while finishing up the work, please ping me! >>>>>> >>>>>> Cheers, >>>>>> Max >>>>>> >>>>>> On Tue, Apr 5, 2016 at 6:01 PM, jay vyas < >>> [email protected]> >>>>>> wrote: >>>>>>> Yup , we're getting there ! I tend to this sometimes in after work >>> hours, >>>>>>> I'll call the grad students tonite, and see if they want to make a >>> final >>>>>>> push this wknd on it, i can help them. >>>>>>> >>>>>>> On Tue, Apr 5, 2016 at 11:56 AM, Konstantin Boudnik <[email protected] >>>> >>>>>> wrote: >>>>>>> >>>>>>>> Hi Maximilian. >>>>>>>> >>>>>>>> Indeed. the saga of getting Flink into the Apache Bigdata stack >>> has a >>>>>> long >>>>>>>> history ;) It's good to see it's finally converges. Your proposal >>> of >>>>>>>> breaking >>>>>>>> the existing PR in peces certainly makes sense! That's how we >>> prefer to >>>>>> do >>>>>>>> things as well - smaller changes are easier to check, fix, and even >>>>>> revert >>>>>>>> if >>>>>>>> needed. >>>>>>>> >>>>>>>> Another thing: we normally would expect to have a single commit for >>>>>> JIRA, >>>>>>>> so >>>>>>>> it would make sense to squash (rebase) the existing PR into smaller >>>>>> number >>>>>>>> of >>>>>>>> commits. Otherwise, it is pretty hard to navigate through all of >>> them. >>>>>>>> >>>>>>>> Please don't hesitate to ping the list shall you need any >>> assistance. >>>>>>>> Regards, >>>>>>>> Cos >>>>>>>> >>>>>>>> On Tue, Apr 05, 2016 at 11:25AM, Maximilian Michels wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I'm an Apache Flink committer and I'd be very happy to see Flink >>> enter >>>>>>>>> Bigtop. We have seen quite some interest in Bigtop in the Flink >>>>>>>>> community. I've been checking out Bhupendra Singh's pull request >>> which >>>>>>>>> followed this thread: https://github.com/apache/bigtop/pull/93 >>>>>>>>> >>>>>>>>> The packaging of Flink remains one of the biggest hurdles for >>> people >>>>>>>>> who want to install and run Flink on a cluster. Thus, that was my >>> main >>>>>>>>> focus when reviewing the PR. Apart from a few issues I found, the >>> pull >>>>>>>>> request looks good. It would be great if we could bring it into a >>>>>>>>> mergeable state. >>>>>>>>> >>>>>>>>> I wonder if it makes sense to break this pull request into several >>>>>>>>> pull requests? For example, one for the packaging, one for the >>> puppet >>>>>>>>> scripts, and another one for the smoke tests. That could make >>>>>>>>> reviewing of the changes easier and people could already try out >>>>>>>>> incremental changes. I'd be happy to help out with the packaging >>> and >>>>>>>>> scripting. >>>>>>>>> >>>>>>>>> What do you think about that? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Max >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> jay vyas >>>>>> >>>> >>> >>> >>> >>
signature.asc
Description: Message signed with OpenPGP using GPGMail
