[
https://issues.apache.org/jira/browse/SPARK-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15399089#comment-15399089
]
Adam Roberts commented on SPARK-16769:
--------------------------------------
I can do that, cautious that Hive might be a problem here (commons.httpclient
is mentioned as a dependency in sql/hive/pom.xml but removing it doesn't cause
any of the Spark tests to fail - granted I didn't clean my m2 dir though). In
the Hive code there's one use of commons.httpclient and that's in
hive.ql.parse.LoadSemanticAnalyzer
I'm wondering if we're actually stuck with this library if we want to use Spark
with Hive (presumably because Hive hasn't moved up to use httpclient 4.x
instead of this commons.httpclient 3.x version)
I listed the classes for each library mentioning hive and don't see
commons.httpclient even if it's listed as a dependency in the main pom or in
sql/hive/pom.xml
I checked
hive-beeline-1.2.1.spark2.jar, hive-cli-1.2.1.spark2.jar,
hive-jdbc-1.2.1.spark2.jar, hive-metastore-1.2.1.spark2.jar,
spark-hive_2.11-2.1.0-SNAPSHOT.jar,
spark-hive-thriftserver_2.11-2.1.0-SNAPSHOT.jar
> httpclient classic dependency - potentially a patch required?
> -------------------------------------------------------------
>
> Key: SPARK-16769
> URL: https://issues.apache.org/jira/browse/SPARK-16769
> Project: Spark
> Issue Type: Improvement
> Components: Build
> Affects Versions: 1.6.2, 2.0.0
> Environment: All Spark versions, any environment
> Reporter: Adam Roberts
> Priority: Minor
>
> In our jars folder for Spark we provide a jar with a CVE
> https://www.versioneye.com/java/commons-httpclient:commons-httpclient/3.1.
> CVE-2012-5783
> This paper outlines the problem
> www.cs.utexas.edu/~shmat/shmat_ccs12.pdf
> My question is: do we need to ship this version as well or is it only used
> for tests? Is it a patched version? I plan to run without this dependency and
> if there are NoClassDefFound problems I'll add <scope>test</scope> so we
> don't ship it (downloading it in the first place is bad enough though)
> Note that this is valid for all versions, suggesting it be raised to a
> critical if Spark functionality is depending on it because of what the pdf
> I've linked to mentions
> Here is the jar being included:
> ls $SPARK_HOME/jars | grep "httpclient"
> commons-httpclient-3.1.jar
> httpclient-4.5.2.jar
> The first jar potentially contains the security issue, could be a patched
> version, need to verify. SHA1 sum for this jar is
> 964cd74171f427720480efdec40a7c7f6e58426a
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
