+1 what Olaf said! We already see monstrosities that Spark builds and provides (close to 400MB binary blobs) or Zeppelin, which goes over half a gig (sic!)
We are producing the stack for very exact specifications, so can afford removing a bunch of redistributed bits and pieces, as well to avoid the shading nonsense. Build time is a very small price to pay to get clean deployment. Cos On Wed, Apr 20, 2016 at 08:16PM, Olaf Flebbe wrote: > Hi, > > Yep, we like to rebuild from source. AFAIK shading dependencies has to do > with uber jars. Since we do not use uber jars, can we please have simple > jars, since we use a packaging system to do the deployment? > > Regards, > Olaf > > > > Am 20.04.2016 um 16:24 schrieb Robert Metzger <[email protected]>: > > > > Hi, > > > > I'm currently looking into the existing pull request for the Flink > > integration [1]. > > While doing that, I wondered if its a requirement from Bigtop to build > > Flink from source when building the rpm / deb packages? > > Not building from source would speed up the build a lot and it would ensure > > that the binary build is verified by the Flink community (there are some > > issues with maven dependency shading when building Flink with maven 3.3.x) > > > > > > > > [1] https://github.com/apache/bigtop/pull/93/files > > > > On Wed, Apr 6, 2016 at 11:16 AM, Maximilian Michels <[email protected]> wrote: > > > >> Hi Konstantin, hi Jay, > >> > >> Thanks for the replies and the prompt progress on the pull request. It > >> looks like we will have a Flink RPM package in Bigtop very soon. For > >> the remaining integration we can open follow-up pull requests. > >> > >> If anything comes up while finishing up the work, please ping me! > >> > >> Cheers, > >> Max > >> > >> On Tue, Apr 5, 2016 at 6:01 PM, jay vyas <[email protected]> > >> wrote: > >>> Yup , we're getting there ! I tend to this sometimes in after work hours, > >>> I'll call the grad students tonite, and see if they want to make a final > >>> push this wknd on it, i can help them. > >>> > >>> On Tue, Apr 5, 2016 at 11:56 AM, Konstantin Boudnik <[email protected]> > >> wrote: > >>> > >>>> Hi Maximilian. > >>>> > >>>> Indeed. the saga of getting Flink into the Apache Bigdata stack has a > >> long > >>>> history ;) It's good to see it's finally converges. Your proposal of > >>>> breaking > >>>> the existing PR in peces certainly makes sense! That's how we prefer to > >> do > >>>> things as well - smaller changes are easier to check, fix, and even > >> revert > >>>> if > >>>> needed. > >>>> > >>>> Another thing: we normally would expect to have a single commit for > >> JIRA, > >>>> so > >>>> it would make sense to squash (rebase) the existing PR into smaller > >> number > >>>> of > >>>> commits. Otherwise, it is pretty hard to navigate through all of them. > >>>> > >>>> Please don't hesitate to ping the list shall you need any assistance. > >>>> Regards, > >>>> Cos > >>>> > >>>> On Tue, Apr 05, 2016 at 11:25AM, Maximilian Michels wrote: > >>>>> Hi, > >>>>> > >>>>> I'm an Apache Flink committer and I'd be very happy to see Flink enter > >>>>> Bigtop. We have seen quite some interest in Bigtop in the Flink > >>>>> community. I've been checking out Bhupendra Singh's pull request which > >>>>> followed this thread: https://github.com/apache/bigtop/pull/93 > >>>>> > >>>>> The packaging of Flink remains one of the biggest hurdles for people > >>>>> who want to install and run Flink on a cluster. Thus, that was my main > >>>>> focus when reviewing the PR. Apart from a few issues I found, the pull > >>>>> request looks good. It would be great if we could bring it into a > >>>>> mergeable state. > >>>>> > >>>>> I wonder if it makes sense to break this pull request into several > >>>>> pull requests? For example, one for the packaging, one for the puppet > >>>>> scripts, and another one for the smoke tests. That could make > >>>>> reviewing of the changes easier and people could already try out > >>>>> incremental changes. I'd be happy to help out with the packaging and > >>>>> scripting. > >>>>> > >>>>> What do you think about that? > >>>>> > >>>>> Thanks, > >>>>> Max > >>>> > >>> > >>> > >>> > >>> -- > >>> jay vyas > >> >
signature.asc
Description: Digital signature
