Actually I think I remember Dr. Cos saying, around the time we started working with bigtop, that he went through their poms with a fine toothed comb and helped them get everything in order. Maybe we could ask him to help us out.
________________________________ From: Dmitriy Lyubimov <dlie...@gmail.com> Sent: Tuesday, September 6, 2016 8:24:29 PM To: dev@mahout.apache.org Subject: Re: Mahout distro Size I dunno. they build shaded assembly artifact it seems and are happy with this approach. It would seem we'd just need the legacy deps in a similar case. On Tue, Sep 6, 2016 at 4:48 PM, Andrew Palumbo <ap....@outlook.com> wrote: > bq. > > 4: other projects do something too. spark (at least it used to) to produce > tons of lib-managed deps as the result of its build, they probably still > have? > > > Do you mean using something like Spark's dependency resolver? > > ________________________________ > From: Dmitriy Lyubimov <dlie...@gmail.com> > Sent: Tuesday, September 6, 2016 4:46:24 PM > To: dev@mahout.apache.org > Subject: Re: Mahout distro Size > > 2 + 1 > 3 + 1 > > 4: other projects do something too. spark (at least it used to) to produce > tons of lib-managed deps as the result of its build, they probably still > have? > > On the other hand, the samsara only dependencies are really light. backends > are really always "provided", and the rest of it is fairly small enough not > to be an issue either way. but we probably definitely should drop local > support for MR stuff (MR local mode didn't work correctly anyway, last time > I checked) > > On Tue, Sep 6, 2016 at 1:33 PM, Andrew Palumbo <ap....@outlook.com> wrote: > > > The current apache-mahout-distribution-0.12.2.tar.gz<http://mirror. > > stjschools.org/public/apache/mahout/0.12.2/apache-mahout- > > distribution-0.12.2.tar.gz> is 224M. we need to look for ways to get this > > size down. > > > > 1. A few Possibilities: > > > > 2. Drop h2o (binary only) from Distro? (18M - unused) > > > > 3. MAHOUT-1865<https://issues.apache.org/jira/browse/MAHOUT-1865>: > > Remove Hadoop 1 support. could save us some space. > > > > 4. MAHOUT-1706<https://issues.apache.org/jira/browse/MAHOUT-1706>: > > Remove dependency jars from /lib in mahout binary distribution. Should > also > > save space. > > > > 5. Having dropped support for MAHOUT_LOCAL we can now likely set a lot > > of dependencies to <provided> scope, we can revisit: MAHOUT-1705< > > https://issues.apache.org/jira/browse/MAHOUT-1705>: Verify dependencies > > in job jar for mahout-examples. > > > > * 16M ./lib/hadoop > > > > * 85M ./lib/ > > > > * Many of the jars in /lib/ and possibly /lib/hadoop are > already > > packaged into the mahout-examples jar and adding them to the classpath > from > > /lib/ is therefore redundant. As well many may be provided. > > >
