[
https://issues.apache.org/jira/browse/HADOOP-18090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved HADOOP-18090.
-------------------------------------
Fix Version/s: 3.5.0
Resolution: Fixed
> Exclude com/jcraft/jsch classes from being shaded/relocated
> -----------------------------------------------------------
>
> Key: HADOOP-18090
> URL: https://issues.apache.org/jira/browse/HADOOP-18090
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 3.3.1
> Reporter: mkv
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.5.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Spark 3.2.0 transitively introduces hadoop-client-api and
> hadoop-client-runtime dependencies.
> When we create a SFTPFileSystem instance
> (org.apache.hadoop.fs.sftp.SFTPFileSystem) it tries to load the relocated
> classes from _com.jcraft.jsch_ package.
> The filesystem instance creation fails with error:
> {code:java}
> java.lang.ClassNotFoundException:
> org.apache.hadoop.shaded.com.jcraft.jsch.SftpException
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) {code}
> Excluding client from transitive load of spark and directly using
> hadoop-common/hadoop-client is the way its working for us.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
