[ https://issues.apache.org/jira/browse/SPARK-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989642#comment-13989642 ]
Sean Owen commented on SPARK-1718: ---------------------------------- This is the same issue as https://issues.apache.org/jira/browse/SPARK-1520 right? > pyspark doesn't work with assembly jar containing over 65536 files/dirs built > on redhat > ---------------------------------------------------------------------------------------- > > Key: SPARK-1718 > URL: https://issues.apache.org/jira/browse/SPARK-1718 > Project: Spark > Issue Type: Bug > Components: PySpark > Affects Versions: 1.0.0 > Reporter: Thomas Graves > > Recently pyspark was ported to yarn (pr 30), but when I went to try it I > couldn't get it work. I was building on a redhat 6 box. I figured out that > if the assembly jar file contained over 65536 files/directories then it > wouldn't work. If I unjarred the assembly and removed some stuff to get it > under 65536 and jarred it back up, then it would work. > It appears to only be an issue when building on a redhat box as I can build > on my mac and it works just fine there. -- This message was sent by Atlassian JIRA (v6.2#6252)