[ 
https://issues.apache.org/jira/browse/SPARK-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989642#comment-13989642
 ] 

Sean Owen commented on SPARK-1718:
----------------------------------

This is the same issue as https://issues.apache.org/jira/browse/SPARK-1520 
right?

> pyspark doesn't work with assembly jar containing over 65536 files/dirs built 
> on redhat 
> ----------------------------------------------------------------------------------------
>
>                 Key: SPARK-1718
>                 URL: https://issues.apache.org/jira/browse/SPARK-1718
>             Project: Spark
>          Issue Type: Bug
>          Components: PySpark
>    Affects Versions: 1.0.0
>            Reporter: Thomas Graves
>
> Recently pyspark was ported to yarn (pr 30), but when I went to try it I 
> couldn't get it work.  I was building on a redhat 6 box.  I figured out that 
> if the assembly jar file contained over 65536 files/directories then it 
> wouldn't work.  If I unjarred the assembly and removed some stuff to get it 
> under 65536 and jarred it back up, then it would work.     
> It appears to only be an issue when building on a redhat box as I can build 
> on my mac and it works just fine there.   



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Reply via email to