[
https://issues.apache.org/jira/browse/SPARK-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14009295#comment-14009295
]
Yin Huai commented on SPARK-1935:
---------------------------------
Thanks, [~rxin]. Let me add more info.
Commands I used:
{code}
mvn clean -DskipTests clean package
{code}
{code}
sbt/sbt assembly
{code}
You can also check the pre-built Hadoop 1 package which has the 1.3 codec.
There are a few methods in the class of Base64 that were introduced with 1.4
(http://commons.apache.org/proper/commons-codec/apidocs/org/apache/commons/codec/binary/Base64.html).
I noticed the problem when Hive was calling
{code}
public static byte[] decodeBase64(String base64String)
{code}
> Explicitly add commons-codec 1.4 as a dependency
> ------------------------------------------------
>
> Key: SPARK-1935
> URL: https://issues.apache.org/jira/browse/SPARK-1935
> Project: Spark
> Issue Type: Bug
> Components: Build
> Affects Versions: 0.9.1
> Reporter: Yin Huai
> Priority: Minor
>
> Right now, commons-codec is a transitive dependency. When Spark is built by
> maven for Hadoop 1, jets3t 0.7.1 will pull in commons-codec 1.3 which is an
> older version (Hadoop 1.0.4 depends on 1.4). This older version can cause
> problems because 1.4 introduces incompatible changes and new methods.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
