[ https://issues.apache.org/jira/browse/SPARK-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13987699#comment-13987699 ]
Guoqiang Li edited comment on SPARK-1698 at 5/2/14 2:32 PM: ------------------------------------------------------------ [~srowen] In [The PR 598|https://github.com/apache/spark/pull/598] #1,#2,#4 do not occur and #3 is very easy to find. The directory structure of spark similar to hadoop 2.3.0. There are three subcomponents: core,examples,hive,The directory structure of spark: {code} +- SPARK_HOME | +- bin | \- * | +- sbin | \- * | +- RELEASE | +- conf | \- * | +- python | \- * | +- share | \- spark | +- core | +- lib | \- *.jar | +- spark-core*.jar | +- spark-repl*.jar | +- spark-yarn*.jar | +- spark-bagel*.jar | +- spark-graphx*.jar | +- spark-sql*.jar | +- spark-catalyst*.jar | +- spark-mllib*.jar | +- spark-streaming*.jar | +- hive | +- lib | \- *.jar | \- spark-hive*.jar | +- examples | +- lib | \- *.jar | \- spark-examples*.jar {code} was (Author: gq): [~srowen] In [The PR 598|https://github.com/apache/spark/pull/598] #1,#2,#4 do not occur and #3 is very easy to find. The directory structure of Spark similar to hadoop 2.3.0. There are three subcomponents: core,examples,hive,their path is share/spark/core,share/spark/examples,share/spark/hive. their dependency path is share/spark/core/lib,share/spark/examples/lib,share/spark/hive/lib > Improve spark integration > ------------------------- > > Key: SPARK-1698 > URL: https://issues.apache.org/jira/browse/SPARK-1698 > Project: Spark > Issue Type: Improvement > Components: Build, Deploy > Reporter: Guoqiang Li > Assignee: Guoqiang Li > Fix For: 1.0.0 > > > Use the shade plugin to create a big JAR with all the dependencies can cause > a few problems > 1. Missing jar's meta information > 2. Some file is covered, eg: plugin.xml > 3. Different versions of a jar may co-exist > 4. Too big, java 6 does not support -- This message was sent by Atlassian JIRA (v6.2#6252)