[ 
https://issues.apache.org/jira/browse/SPARK-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

José Manuel Abuín Mosquera updated SPARK-2753:
----------------------------------------------

    Description: 
Hi all,

this is my first sent issue, I googled and searche dinto the Spark code and 
arrived here.

When passing as argument to --archives a tar.gz or a .zip file, Spark uploads 
it to the distributed cache, but it is not uncompressing it.

According the documentation, it is supposed to uncompress it, is this a bug??

Launching command is:

/opt/spark-1.0.1/bin/spark-submit --class ProlnatSpark --master yarn-cluster 
--num-executors 32 --driver-library-path /opt/hadoop/hadoop-2.2.0/lib/native/ 
--driver-memory 390m --executor-memory 890m --executor-cores 1 
--archives=Diccionarios.tar.gz --verbose ProlnatSpark.jar 
Wikipedias/WikipediaPlain.txt saidaWikipediaSpark

In files 
/yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala and 
/yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnableUtil.scala
 doesn't seem to uncompress the files.

I hope this helps, thank you very much :)

  was:
Hi all,

this is my first sent issue, I googled and searche dinto the Spark code and 
arrived here.

When passing as argument to --archives a tar.gz or a .zip file, Spark uploads 
it to the distributed cache, but it is not uncompressing it.

According the documentation, it is supposed to uncompress it, is this a bug??

Launching command is:

/opt/spark-1.0.1/bin/spark-submit --class ProlnatSpark --master yarn-cluster 
--num-executors 32 --driver-library-path /opt/hadoop/hadoop-2.2.0/lib/native/ 
--driver-memory 390m --executor-memory 890m --executor-cores 1 
--archives=Diccionarios.tar.gz --verbose ProlnatSpark.jar 
Wikipedias/WikipediaPlain.txt saidaWikipediaSpark

In files 
/yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala and 
/yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnableUtil.scala
 doesn't seem to uncompress the files.

I hope this helps, than you very much :)


> Is it supposed --archives option in yarn cluster mode to uncompress file?
> -------------------------------------------------------------------------
>
>                 Key: SPARK-2753
>                 URL: https://issues.apache.org/jira/browse/SPARK-2753
>             Project: Spark
>          Issue Type: Bug
>          Components: YARN
>    Affects Versions: 1.0.0
>         Environment: CentOS release 6.5 (64 bits) and Hadoop 2.2.0
>            Reporter: José Manuel Abuín Mosquera
>              Labels: archives, cache, distributed, yarn
>
> Hi all,
> this is my first sent issue, I googled and searche dinto the Spark code and 
> arrived here.
> When passing as argument to --archives a tar.gz or a .zip file, Spark uploads 
> it to the distributed cache, but it is not uncompressing it.
> According the documentation, it is supposed to uncompress it, is this a bug??
> Launching command is:
> /opt/spark-1.0.1/bin/spark-submit --class ProlnatSpark --master yarn-cluster 
> --num-executors 32 --driver-library-path /opt/hadoop/hadoop-2.2.0/lib/native/ 
> --driver-memory 390m --executor-memory 890m --executor-cores 1 
> --archives=Diccionarios.tar.gz --verbose ProlnatSpark.jar 
> Wikipedias/WikipediaPlain.txt saidaWikipediaSpark
> In files 
> /yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala and 
> /yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnableUtil.scala
>  doesn't seem to uncompress the files.
> I hope this helps, thank you very much :)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Reply via email to