[
https://issues.apache.org/jira/browse/SPARK-30272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17069933#comment-17069933
]
Jorge Machado commented on SPARK-30272:
---------------------------------------
Hey Sean,
This seems still to make problems for example:
{code:java}
java.lang.NoClassDefFoundError:
com/google/common/util/concurrent/internal/InternalFutureFailureAccess
java.lang.NoClassDefFoundError:
com/google/common/util/concurrent/internal/InternalFutureFailureAccess at
java.lang.ClassLoader.defineClass1(Native Method) at
java.lang.ClassLoader.defineClass(ClassLoader.java:757) at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at
java.net.URLClassLoader.defineClass(URLClassLoader.java:468) at
java.net.URLClassLoader.access$100(URLClassLoader.java:74) at
java.net.URLClassLoader$1.run(URLClassLoader.java:369) at
java.net.URLClassLoader$1.run(URLClassLoader.java:363) at
java.security.AccessController.doPrivileged(Native Method) at
java.net.URLClassLoader.findClass(URLClassLoader.java:362) at
java.lang.ClassLoader.loadClass(ClassLoader.java:419) at
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at
java.lang.ClassLoader.loadClass(ClassLoader.java:352) at
java.lang.ClassLoader.defineClass1(Native Method) at
java.lang.ClassLoader.defineClass(ClassLoader.java:757) at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at
java.net.URLClassLoader.defineClass(URLClassLoader.java:468) at
java.net.URLClassLoader.access$100(URLClassLoader.java:74) at
java.net.URLClassLoader$1.run(URLClassLoader.java:369) at
java.net.URLClassLoader$1.run(URLClassLoader.java:363) at
java.security.AccessController.doPrivileged(Native Method) at
java.net.URLClassLoader.findClass(URLClassLoader.java:362) at
java.lang.ClassLoader.loadClass(ClassLoader.java:419) at
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at
java.lang.ClassLoader.loadClass(ClassLoader.java:352) at
java.lang.ClassLoader.defineClass1(Native Method) at
java.lang.ClassLoader.defineClass(ClassLoader.java:757) at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at
java.net.URLClassLoader.defineClass(URLClassLoader.java:468) at
java.net.URLClassLoader.access$100(URLClassLoader.java:74) at
java.net.URLClassLoader$1.run(URLClassLoader.java:369) at
java.net.URLClassLoader$1.run(URLClassLoader.java:363) at
java.security.AccessController.doPrivileged(Native Method) at
java.net.URLClassLoader.findClass(URLClassLoader.java:362) at
java.lang.ClassLoader.loadClass(ClassLoader.java:419) at
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at
java.lang.ClassLoader.loadClass(ClassLoader.java:352) at
com.google.common.cache.LocalCache$LoadingValueReference.<init>(LocalCache.java:3472)
at
com.google.common.cache.LocalCache$LoadingValueReference.<init>(LocalCache.java:3476)
at
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2134)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) at
com.google.common.cache.LocalCache.get(LocalCache.java:3951) at
com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
at org.apache.hadoop.security.Groups.getGroups(Groups.java:228) at
org.apache.hadoop.security.UserGroupInformation.getGroups(UserGroupInformation.java:1588)
at
org.apache.hadoop.security.UserGroupInformation.getPrimaryGroupName(UserGroupInformation.java:1453)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.<init>(AzureBlobFileSystemStore.java:147)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.initialize(AzureBlobFileSystem.java:104)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3303) at
org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124) at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3352) at
org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3320) at
org.apache.hadoop.fs.FileSystem.get(FileSystem.java:479) at
org.apache.hadoop.fs.Path.getFileSystem(Path.java:365) at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.setInputPaths(FileInputFormat.java:522)
at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.setInputPaths(FileInputFormat.java:491)
at
org.apache.spark.SparkContext.$anonfun$newAPIHadoopFile$2(SparkContext.scala:1219)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.SparkContext.withScope(SparkContext.scala:757) at
org.apache.spark.SparkContext.newAPIHadoopFile(SparkContext.scala:1207) at
org.apache.spark.api.java.JavaSparkContext.newAPIHadoopFile(JavaSparkContext.scala:484)
{code}
I still see a lot of references to guava 14 on master is this normal ? Sorry
for the question...
> Remove usage of Guava that breaks in Guava 27
> ---------------------------------------------
>
> Key: SPARK-30272
> URL: https://issues.apache.org/jira/browse/SPARK-30272
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core, SQL
> Affects Versions: 3.0.0
> Reporter: Sean R. Owen
> Assignee: Sean R. Owen
> Priority: Major
> Fix For: 3.0.0
>
>
> Background:
> https://issues.apache.org/jira/browse/SPARK-29250
> https://github.com/apache/spark/pull/25932
> Hadoop 3.2.1 will update Guava from 11 to 27. There are a number of methods
> that changed between those releases, typically just a rename, but, means one
> set of code can't work with both, while we want to work with Hadoop 2.x and
> 3.x. Among them:
> - Objects.toStringHelper was moved to MoreObjects; we can just use the
> Commons Lang3 equivalent
> - Objects.hashCode etc were renamed; use java.util.Objects equivalents
> - MoreExecutors.sameThreadExecutor() became directExecutor(); for same-thread
> execution we can use a dummy implementation of ExecutorService / Executor
> - TypeToken.isAssignableFrom become isSupertypeOf; work around with reflection
> There is probably more to the Guava issue than just this change, but it will
> make Spark itself work with more versions and reduce our exposure to Guava
> along the way anyway.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
