[ https://issues.apache.org/jira/browse/SPARK-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14502695#comment-14502695 ]
Sean Owen commented on SPARK-7009: ---------------------------------- Let's see if I remember this correctly: Java 7 supports zip64, so there's no problem if building/running with Java 7+ only. Some (early) Java 6 won't read zip64 correctly though. I think the implicit workaround there was to update to a later Java 6, since it doesn't affect most releases. Java 6 has some *different* hacky extension to zip that lets it read/write more than 65K files though, which means that weirdly Java 6-built assemblies might work on old Java 6 after all. I think we only officially support the zip64 version. Implicitly, actually, early Java 6 doesn't necessarily work with Spark. So... does this end up helping this weird situation if Ant is only making zip64 archives? (Nice that this doesn't actually involve adding an Ant script) > Build assembly JAR via ant to avoid zip64 problems > -------------------------------------------------- > > Key: SPARK-7009 > URL: https://issues.apache.org/jira/browse/SPARK-7009 > Project: Spark > Issue Type: Improvement > Components: Build > Affects Versions: 1.3.0 > Environment: Java 7+ > Reporter: Steve Loughran > Original Estimate: 2h > Remaining Estimate: 2h > > SPARK-1911 shows the problem that JDK7+ is using zip64 to build large JARs; a > format incompatible with Java and pyspark. > Provided the total number of .class files+resources is <64K, ant can be used > to make the final JAR instead, perhaps by unzipping the maven-generated JAR > then rezipping it with zip64=never, before publishing the artifact via maven. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org