[
https://issues.apache.org/jira/browse/SPARK-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15399388#comment-15399388
]
Adam Roberts commented on SPARK-16769:
--------------------------------------
I was aiming to to build and provide a patched commons.httpclient 3 for
distribution to Maven central as I'd rather not risk compatibility issues for
our users but looks like the patch outlined here
https://issues.apache.org/jira/browse/HTTPCLIENT-613 is aimed at the 4.x line
instead - and I see no equivalent class. Given how old this item is, I'm not
surprised this isn't something obvious we can do..
So in order to remedy this one if we DO really need it, we'd need to remove
that one line of code in Hive that's still reliant on it and look out for a
Hive release with the fix in (or we'd do it in the forked version you've
mentioned in order to have control instead of waiting on the next 1.x or 2.x
Hive release)
Then we can remove the dependency altogether except for jets3t which uses it
everywhere, see
https://github.com/hyperic/jets3t/search?utf8=%E2%9C%93&q=httpclient - unless
we drop the Hadoop 2.2 line or bump up the jets3t version for that (no idea
about compatibility though).
> httpclient classic dependency - potentially a patch required?
> -------------------------------------------------------------
>
> Key: SPARK-16769
> URL: https://issues.apache.org/jira/browse/SPARK-16769
> Project: Spark
> Issue Type: Improvement
> Components: Build
> Affects Versions: 1.6.2, 2.0.0
> Environment: All Spark versions, any environment
> Reporter: Adam Roberts
> Priority: Minor
>
> In our jars folder for Spark we provide a jar with a CVE
> https://www.versioneye.com/java/commons-httpclient:commons-httpclient/3.1.
> CVE-2012-5783
> This paper outlines the problem
> www.cs.utexas.edu/~shmat/shmat_ccs12.pdf
> My question is: do we need to ship this version as well or is it only used
> for tests? Is it a patched version? I plan to run without this dependency and
> if there are NoClassDefFound problems I'll add <scope>test</scope> so we
> don't ship it (downloading it in the first place is bad enough though)
> Note that this is valid for all versions, suggesting it be raised to a
> critical if Spark functionality is depending on it because of what the pdf
> I've linked to mentions
> Here is the jar being included:
> ls $SPARK_HOME/jars | grep "httpclient"
> commons-httpclient-3.1.jar
> httpclient-4.5.2.jar
> The first jar potentially contains the security issue, could be a patched
> version, need to verify. SHA1 sum for this jar is
> 964cd74171f427720480efdec40a7c7f6e58426a
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
