[ 
https://issues.apache.org/jira/browse/SPARK-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13987699#comment-13987699
 ] 

Guoqiang Li edited comment on SPARK-1698 at 5/2/14 2:32 PM:
------------------------------------------------------------

[~srowen]
In [The PR 598|https://github.com/apache/spark/pull/598] #1,#2,#4 do not occur 
and #3 is very easy to find.
The directory structure of spark similar to hadoop 2.3.0.
There are three subcomponents: core,examples,hive,The directory structure of 
spark: 
{code}
+- SPARK_HOME
|  +- bin
|     \- *
|  +- sbin
|     \- *
|  +- RELEASE
|  +- conf
|     \- *
|  +- python
|     \- *
|  +- share
|     \- spark
|        +- core
|           +- lib
|              \- *.jar
|           +- spark-core*.jar
|           +- spark-repl*.jar
|           +- spark-yarn*.jar
|           +- spark-bagel*.jar
|           +- spark-graphx*.jar
|           +- spark-sql*.jar
|           +- spark-catalyst*.jar
|           +- spark-mllib*.jar
|           +- spark-streaming*.jar
|        +- hive
|           +- lib
|              \- *.jar
|           \- spark-hive*.jar
|        +- examples
|           +- lib
|              \- *.jar
|           \- spark-examples*.jar
{code}


was (Author: gq):
[~srowen]
In [The PR 598|https://github.com/apache/spark/pull/598] #1,#2,#4 do not occur 
and #3 is very easy to find.
The directory structure of Spark similar to hadoop 2.3.0.
There are three subcomponents: core,examples,hive,their path is 
share/spark/core,share/spark/examples,share/spark/hive. their dependency path 
is share/spark/core/lib,share/spark/examples/lib,share/spark/hive/lib

> Improve spark integration
> -------------------------
>
>                 Key: SPARK-1698
>                 URL: https://issues.apache.org/jira/browse/SPARK-1698
>             Project: Spark
>          Issue Type: Improvement
>          Components: Build, Deploy
>            Reporter: Guoqiang Li
>            Assignee: Guoqiang Li
>             Fix For: 1.0.0
>
>
> Use the shade plugin to create a big JAR with all the dependencies can cause 
> a few problems
> 1. Missing jar's meta information
> 2. Some file is covered, eg: plugin.xml
> 3. Different versions of a jar may co-exist
> 4. Too big, java 6 does not support



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Reply via email to