+1
> On Jan 1, 2015, at 3:39 PM, "ASF GitHub Bot (JIRA)" <[email protected]> wrote: > > > [ > https://issues.apache.org/jira/browse/MAHOUT-1636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14262681#comment-14262681 > ] > > ASF GitHub Bot commented on MAHOUT-1636: > ---------------------------------------- > > Github user pferrel commented on the pull request: > > https://github.com/apache/mahout/pull/69#issuecomment-68500838 > > Go for it > > >> Class dependencies for the spark module are put in a job.jar, which is very >> inefficient >> --------------------------------------------------------------------------------------- >> >> Key: MAHOUT-1636 >> URL: https://issues.apache.org/jira/browse/MAHOUT-1636 >> Project: Mahout >> Issue Type: Bug >> Components: spark >> Affects Versions: 1.0-snapshot >> Reporter: Pat Ferrel >> Assignee: Ted Dunning >> Fix For: 1.0-snapshot >> >> >> using a maven plugin and an assembly job.xml a job.jar is created with all >> dependencies including transitive ones. This job.jar is in >> mahout/spark/target and is included in the classpath when a Spark job is >> run. This allows dependency classes to be found at runtime but the job.jar >> include a great deal of things not needed that are duplicates of classes >> found in the main mrlegacy job.jar. If the job.jar is removed, drivers will >> not find needed classes. A better way needs to be implemented for including >> class dependencies. >> I'm not sure what that better way is so am leaving the assembly alone for >> now. Whoever picks up this Jira will have to remove it after deciding on a >> better method. > > > > -- > This message was sent by Atlassian JIRA > (v6.3.4#6332)
