[jira] [Reopened] (SPARK-36548) Throwing NoClassDefFoundError for Logging$class
[ https://issues.apache.org/jira/browse/SPARK-36548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sadagopan kalyanaraman reopened SPARK-36548: > Throwing NoClassDefFoundError for Logging$class > --- > > Key: SPARK-36548 > URL: https://issues.apache.org/jira/browse/SPARK-36548 > Project: Spark > Issue Type: Bug > Components: Kubernetes, Spark Core >Affects Versions: 3.1.1 >Reporter: sadagopan kalyanaraman >Priority: Major > > I'm getting NoClassDefFoundError. is it removed in 3.1.1? > > https://spark.apache.org/docs/3.1.1/api/java/org/apache/spark/internal/Logging.html > > *https://spark.apache.org/docs/2.4.7/api/java/org/apache/spark/internal/Logging.html* > > Exception in thread "main" java.util.ServiceConfigurationError: > org.apache.spark.sql.sources.DataSourceRegister: Provider > com.google.cloud.spark.bigquery.BigQueryRelationProvider could not be > instantiatedException in thread "main" java.util.ServiceConfigurationError: > org.apache.spark.sql.sources.DataSourceRegister: Provider > com.google.cloud.spark.bigquery.BigQueryRelationProvider could not be > instantiated at java.base/java.util.ServiceLoader.fail(Unknown Source) at > java.base/java.util.ServiceLoader$ProviderImpl.newInstance(Unknown Source) at > java.base/java.util.ServiceLoader$ProviderImpl.get(Unknown Source) at > java.base/java.util.ServiceLoader$3.next(Unknown Source) at > scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:44) at > scala.collection.Iterator.foreach(Iterator.scala:941) at > scala.collection.Iterator.foreach$(Iterator.scala:941) at > scala.collection.AbstractIterator.foreach(Iterator.scala:1429) at > scala.collection.IterableLike.foreach(IterableLike.scala:74) at > scala.collection.IterableLike.foreach$(IterableLike.scala:73) at > scala.collection.AbstractIterable.foreach(Iterable.scala:56) at > scala.collection.TraversableLike.filterImpl(TraversableLike.scala:255) at > scala.collection.TraversableLike.filterImpl$(TraversableLike.scala:249) at > scala.collection.AbstractTraversable.filterImpl(Traversable.scala:108) at > scala.collection.TraversableLike.filter(TraversableLike.scala:347) at > scala.collection.TraversableLike.filter$(TraversableLike.scala:347) at > scala.collection.AbstractTraversable.filter(Traversable.scala:108) at > org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:659) > at > org.apache.spark.sql.streaming.DataStreamReader.loadInternal(DataStreamReader.scala:209) > at > org.apache.spark.sql.streaming.DataStreamReader.load(DataStreamReader.scala:195) > at > com.xyz.xyz.xyz.AbstractCdcMessageToSqJob.runJob(AbstractCdcMessageToSqJob.java:94) > at > com.xyz.xyz.xyz.CdcMessageToSqlBillerJob.main(CdcMessageToSqlBillerJob.java:30) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown > Source) at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown > Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at > org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) > at > org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:951) > at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) at > org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) at > org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) at > org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1030) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1039) at > org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)Caused by: > java.lang.NoClassDefFoundError: org/apache/spark/internal/Logging$class at > com.google.cloud.spark.bigquery.BigQueryUtilScala$.(BigQueryUtil.scala:34) > at > com.google.cloud.spark.bigquery.BigQueryUtilScala$.(BigQueryUtil.scala) > at > com.google.cloud.spark.bigquery.BigQueryRelationProvider.(BigQueryRelationProvider.scala:43) > at > com.google.cloud.spark.bigquery.BigQueryRelationProvider.(BigQueryRelationProvider.scala:50) > at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown > Source) at > java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown > Source) at java.base/java.lang.reflect.Constructor.newInstance(Unknown > Source) ... 33 moreCaused by: java.lang.ClassNotFoundException: > org.apache.spark.internal.Logging$class at > java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source) at >
[jira] [Comment Edited] (SPARK-36548) Throwing NoClassDefFoundError for Logging$class
[ https://issues.apache.org/jira/browse/SPARK-36548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17407481#comment-17407481 ] sadagopan kalyanaraman edited comment on SPARK-36548 at 8/31/21, 3:43 PM: -- [~hyukjin.kwon] why was this moved to resolved? was it fixed was (Author: sadagopan): [~gurwls223], why was this moved to resolved? was it fixed > Throwing NoClassDefFoundError for Logging$class > --- > > Key: SPARK-36548 > URL: https://issues.apache.org/jira/browse/SPARK-36548 > Project: Spark > Issue Type: Bug > Components: Kubernetes, Spark Core >Affects Versions: 3.1.1 >Reporter: sadagopan kalyanaraman >Priority: Major > > I'm getting NoClassDefFoundError. is it removed in 3.1.1? > > https://spark.apache.org/docs/3.1.1/api/java/org/apache/spark/internal/Logging.html > > *https://spark.apache.org/docs/2.4.7/api/java/org/apache/spark/internal/Logging.html* > > Exception in thread "main" java.util.ServiceConfigurationError: > org.apache.spark.sql.sources.DataSourceRegister: Provider > com.google.cloud.spark.bigquery.BigQueryRelationProvider could not be > instantiatedException in thread "main" java.util.ServiceConfigurationError: > org.apache.spark.sql.sources.DataSourceRegister: Provider > com.google.cloud.spark.bigquery.BigQueryRelationProvider could not be > instantiated at java.base/java.util.ServiceLoader.fail(Unknown Source) at > java.base/java.util.ServiceLoader$ProviderImpl.newInstance(Unknown Source) at > java.base/java.util.ServiceLoader$ProviderImpl.get(Unknown Source) at > java.base/java.util.ServiceLoader$3.next(Unknown Source) at > scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:44) at > scala.collection.Iterator.foreach(Iterator.scala:941) at > scala.collection.Iterator.foreach$(Iterator.scala:941) at > scala.collection.AbstractIterator.foreach(Iterator.scala:1429) at > scala.collection.IterableLike.foreach(IterableLike.scala:74) at > scala.collection.IterableLike.foreach$(IterableLike.scala:73) at > scala.collection.AbstractIterable.foreach(Iterable.scala:56) at > scala.collection.TraversableLike.filterImpl(TraversableLike.scala:255) at > scala.collection.TraversableLike.filterImpl$(TraversableLike.scala:249) at > scala.collection.AbstractTraversable.filterImpl(Traversable.scala:108) at > scala.collection.TraversableLike.filter(TraversableLike.scala:347) at > scala.collection.TraversableLike.filter$(TraversableLike.scala:347) at > scala.collection.AbstractTraversable.filter(Traversable.scala:108) at > org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:659) > at > org.apache.spark.sql.streaming.DataStreamReader.loadInternal(DataStreamReader.scala:209) > at > org.apache.spark.sql.streaming.DataStreamReader.load(DataStreamReader.scala:195) > at > com.xyz.xyz.xyz.AbstractCdcMessageToSqJob.runJob(AbstractCdcMessageToSqJob.java:94) > at > com.xyz.xyz.xyz.CdcMessageToSqlBillerJob.main(CdcMessageToSqlBillerJob.java:30) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown > Source) at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown > Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at > org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) > at > org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:951) > at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) at > org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) at > org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) at > org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1030) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1039) at > org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)Caused by: > java.lang.NoClassDefFoundError: org/apache/spark/internal/Logging$class at > com.google.cloud.spark.bigquery.BigQueryUtilScala$.(BigQueryUtil.scala:34) > at > com.google.cloud.spark.bigquery.BigQueryUtilScala$.(BigQueryUtil.scala) > at > com.google.cloud.spark.bigquery.BigQueryRelationProvider.(BigQueryRelationProvider.scala:43) > at > com.google.cloud.spark.bigquery.BigQueryRelationProvider.(BigQueryRelationProvider.scala:50) > at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown > Source) at > java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown > Source) at
[jira] [Commented] (SPARK-36548) Throwing NoClassDefFoundError for Logging$class
[ https://issues.apache.org/jira/browse/SPARK-36548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17407481#comment-17407481 ] sadagopan kalyanaraman commented on SPARK-36548: [~gurwls223], why was this moved to resolved? was it fixed > Throwing NoClassDefFoundError for Logging$class > --- > > Key: SPARK-36548 > URL: https://issues.apache.org/jira/browse/SPARK-36548 > Project: Spark > Issue Type: Bug > Components: Kubernetes, Spark Core >Affects Versions: 3.1.1 >Reporter: sadagopan kalyanaraman >Priority: Major > > I'm getting NoClassDefFoundError. is it removed in 3.1.1? > > https://spark.apache.org/docs/3.1.1/api/java/org/apache/spark/internal/Logging.html > > *https://spark.apache.org/docs/2.4.7/api/java/org/apache/spark/internal/Logging.html* > > Exception in thread "main" java.util.ServiceConfigurationError: > org.apache.spark.sql.sources.DataSourceRegister: Provider > com.google.cloud.spark.bigquery.BigQueryRelationProvider could not be > instantiatedException in thread "main" java.util.ServiceConfigurationError: > org.apache.spark.sql.sources.DataSourceRegister: Provider > com.google.cloud.spark.bigquery.BigQueryRelationProvider could not be > instantiated at java.base/java.util.ServiceLoader.fail(Unknown Source) at > java.base/java.util.ServiceLoader$ProviderImpl.newInstance(Unknown Source) at > java.base/java.util.ServiceLoader$ProviderImpl.get(Unknown Source) at > java.base/java.util.ServiceLoader$3.next(Unknown Source) at > scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:44) at > scala.collection.Iterator.foreach(Iterator.scala:941) at > scala.collection.Iterator.foreach$(Iterator.scala:941) at > scala.collection.AbstractIterator.foreach(Iterator.scala:1429) at > scala.collection.IterableLike.foreach(IterableLike.scala:74) at > scala.collection.IterableLike.foreach$(IterableLike.scala:73) at > scala.collection.AbstractIterable.foreach(Iterable.scala:56) at > scala.collection.TraversableLike.filterImpl(TraversableLike.scala:255) at > scala.collection.TraversableLike.filterImpl$(TraversableLike.scala:249) at > scala.collection.AbstractTraversable.filterImpl(Traversable.scala:108) at > scala.collection.TraversableLike.filter(TraversableLike.scala:347) at > scala.collection.TraversableLike.filter$(TraversableLike.scala:347) at > scala.collection.AbstractTraversable.filter(Traversable.scala:108) at > org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:659) > at > org.apache.spark.sql.streaming.DataStreamReader.loadInternal(DataStreamReader.scala:209) > at > org.apache.spark.sql.streaming.DataStreamReader.load(DataStreamReader.scala:195) > at > com.xyz.xyz.xyz.AbstractCdcMessageToSqJob.runJob(AbstractCdcMessageToSqJob.java:94) > at > com.xyz.xyz.xyz.CdcMessageToSqlBillerJob.main(CdcMessageToSqlBillerJob.java:30) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown > Source) at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown > Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at > org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) > at > org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:951) > at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) at > org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) at > org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) at > org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1030) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1039) at > org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)Caused by: > java.lang.NoClassDefFoundError: org/apache/spark/internal/Logging$class at > com.google.cloud.spark.bigquery.BigQueryUtilScala$.(BigQueryUtil.scala:34) > at > com.google.cloud.spark.bigquery.BigQueryUtilScala$.(BigQueryUtil.scala) > at > com.google.cloud.spark.bigquery.BigQueryRelationProvider.(BigQueryRelationProvider.scala:43) > at > com.google.cloud.spark.bigquery.BigQueryRelationProvider.(BigQueryRelationProvider.scala:50) > at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown > Source) at > java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown > Source) at java.base/java.lang.reflect.Constructor.newInstance(Unknown > Source) ... 33 moreCaused by: java.lang.ClassNotFoundException: > org.apache.spark.internal.Logging$class at >
[jira] [Comment Edited] (SPARK-36548) Throwing NoClassDefFoundError for Logging$class
[ https://issues.apache.org/jira/browse/SPARK-36548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17402394#comment-17402394 ] sadagopan kalyanaraman edited comment on SPARK-36548 at 8/20/21, 7:37 PM: -- [~sarutak] i agree on the documentation but the no class def error? was (Author: sadagopan): [~sarutak] i agree on the doc but the no class def error? > Throwing NoClassDefFoundError for Logging$class > --- > > Key: SPARK-36548 > URL: https://issues.apache.org/jira/browse/SPARK-36548 > Project: Spark > Issue Type: Bug > Components: Kubernetes, Spark Core >Affects Versions: 3.1.1 >Reporter: sadagopan kalyanaraman >Priority: Major > > I'm getting NoClassDefFoundError. is it removed in 3.1.1? > > https://spark.apache.org/docs/3.1.1/api/java/org/apache/spark/internal/Logging.html > > *https://spark.apache.org/docs/2.4.7/api/java/org/apache/spark/internal/Logging.html* > > Exception in thread "main" java.util.ServiceConfigurationError: > org.apache.spark.sql.sources.DataSourceRegister: Provider > com.google.cloud.spark.bigquery.BigQueryRelationProvider could not be > instantiatedException in thread "main" java.util.ServiceConfigurationError: > org.apache.spark.sql.sources.DataSourceRegister: Provider > com.google.cloud.spark.bigquery.BigQueryRelationProvider could not be > instantiated at java.base/java.util.ServiceLoader.fail(Unknown Source) at > java.base/java.util.ServiceLoader$ProviderImpl.newInstance(Unknown Source) at > java.base/java.util.ServiceLoader$ProviderImpl.get(Unknown Source) at > java.base/java.util.ServiceLoader$3.next(Unknown Source) at > scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:44) at > scala.collection.Iterator.foreach(Iterator.scala:941) at > scala.collection.Iterator.foreach$(Iterator.scala:941) at > scala.collection.AbstractIterator.foreach(Iterator.scala:1429) at > scala.collection.IterableLike.foreach(IterableLike.scala:74) at > scala.collection.IterableLike.foreach$(IterableLike.scala:73) at > scala.collection.AbstractIterable.foreach(Iterable.scala:56) at > scala.collection.TraversableLike.filterImpl(TraversableLike.scala:255) at > scala.collection.TraversableLike.filterImpl$(TraversableLike.scala:249) at > scala.collection.AbstractTraversable.filterImpl(Traversable.scala:108) at > scala.collection.TraversableLike.filter(TraversableLike.scala:347) at > scala.collection.TraversableLike.filter$(TraversableLike.scala:347) at > scala.collection.AbstractTraversable.filter(Traversable.scala:108) at > org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:659) > at > org.apache.spark.sql.streaming.DataStreamReader.loadInternal(DataStreamReader.scala:209) > at > org.apache.spark.sql.streaming.DataStreamReader.load(DataStreamReader.scala:195) > at > com.xyz.xyz.xyz.AbstractCdcMessageToSqJob.runJob(AbstractCdcMessageToSqJob.java:94) > at > com.xyz.xyz.xyz.CdcMessageToSqlBillerJob.main(CdcMessageToSqlBillerJob.java:30) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown > Source) at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown > Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at > org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) > at > org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:951) > at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) at > org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) at > org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) at > org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1030) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1039) at > org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)Caused by: > java.lang.NoClassDefFoundError: org/apache/spark/internal/Logging$class at > com.google.cloud.spark.bigquery.BigQueryUtilScala$.(BigQueryUtil.scala:34) > at > com.google.cloud.spark.bigquery.BigQueryUtilScala$.(BigQueryUtil.scala) > at > com.google.cloud.spark.bigquery.BigQueryRelationProvider.(BigQueryRelationProvider.scala:43) > at > com.google.cloud.spark.bigquery.BigQueryRelationProvider.(BigQueryRelationProvider.scala:50) > at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown > Source) at > java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown > Source) at
[jira] [Commented] (SPARK-36548) Throwing NoClassDefFoundError for Logging$class
[ https://issues.apache.org/jira/browse/SPARK-36548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17402394#comment-17402394 ] sadagopan kalyanaraman commented on SPARK-36548: [~sarutak] i agree on the doc but the no class def error? > Throwing NoClassDefFoundError for Logging$class > --- > > Key: SPARK-36548 > URL: https://issues.apache.org/jira/browse/SPARK-36548 > Project: Spark > Issue Type: Bug > Components: Kubernetes, Spark Core >Affects Versions: 3.1.1 >Reporter: sadagopan kalyanaraman >Priority: Major > > I'm getting NoClassDefFoundError. is it removed in 3.1.1? > > https://spark.apache.org/docs/3.1.1/api/java/org/apache/spark/internal/Logging.html > > *https://spark.apache.org/docs/2.4.7/api/java/org/apache/spark/internal/Logging.html* > > Exception in thread "main" java.util.ServiceConfigurationError: > org.apache.spark.sql.sources.DataSourceRegister: Provider > com.google.cloud.spark.bigquery.BigQueryRelationProvider could not be > instantiatedException in thread "main" java.util.ServiceConfigurationError: > org.apache.spark.sql.sources.DataSourceRegister: Provider > com.google.cloud.spark.bigquery.BigQueryRelationProvider could not be > instantiated at java.base/java.util.ServiceLoader.fail(Unknown Source) at > java.base/java.util.ServiceLoader$ProviderImpl.newInstance(Unknown Source) at > java.base/java.util.ServiceLoader$ProviderImpl.get(Unknown Source) at > java.base/java.util.ServiceLoader$3.next(Unknown Source) at > scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:44) at > scala.collection.Iterator.foreach(Iterator.scala:941) at > scala.collection.Iterator.foreach$(Iterator.scala:941) at > scala.collection.AbstractIterator.foreach(Iterator.scala:1429) at > scala.collection.IterableLike.foreach(IterableLike.scala:74) at > scala.collection.IterableLike.foreach$(IterableLike.scala:73) at > scala.collection.AbstractIterable.foreach(Iterable.scala:56) at > scala.collection.TraversableLike.filterImpl(TraversableLike.scala:255) at > scala.collection.TraversableLike.filterImpl$(TraversableLike.scala:249) at > scala.collection.AbstractTraversable.filterImpl(Traversable.scala:108) at > scala.collection.TraversableLike.filter(TraversableLike.scala:347) at > scala.collection.TraversableLike.filter$(TraversableLike.scala:347) at > scala.collection.AbstractTraversable.filter(Traversable.scala:108) at > org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:659) > at > org.apache.spark.sql.streaming.DataStreamReader.loadInternal(DataStreamReader.scala:209) > at > org.apache.spark.sql.streaming.DataStreamReader.load(DataStreamReader.scala:195) > at > com.xyz.xyz.xyz.AbstractCdcMessageToSqJob.runJob(AbstractCdcMessageToSqJob.java:94) > at > com.xyz.xyz.xyz.CdcMessageToSqlBillerJob.main(CdcMessageToSqlBillerJob.java:30) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown > Source) at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown > Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at > org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) > at > org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:951) > at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) at > org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) at > org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) at > org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1030) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1039) at > org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)Caused by: > java.lang.NoClassDefFoundError: org/apache/spark/internal/Logging$class at > com.google.cloud.spark.bigquery.BigQueryUtilScala$.(BigQueryUtil.scala:34) > at > com.google.cloud.spark.bigquery.BigQueryUtilScala$.(BigQueryUtil.scala) > at > com.google.cloud.spark.bigquery.BigQueryRelationProvider.(BigQueryRelationProvider.scala:43) > at > com.google.cloud.spark.bigquery.BigQueryRelationProvider.(BigQueryRelationProvider.scala:50) > at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown > Source) at > java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown > Source) at java.base/java.lang.reflect.Constructor.newInstance(Unknown > Source) ... 33 moreCaused by: java.lang.ClassNotFoundException: > org.apache.spark.internal.Logging$class at >
[jira] [Updated] (SPARK-36548) Throwing NoClassDefFoundError for Logging$class
[ https://issues.apache.org/jira/browse/SPARK-36548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sadagopan kalyanaraman updated SPARK-36548: --- Description: I'm getting NoClassDefFoundError. is it removed in 3.1.1? https://spark.apache.org/docs/3.1.1/api/java/org/apache/spark/internal/Logging.html *https://spark.apache.org/docs/2.4.7/api/java/org/apache/spark/internal/Logging.html* Exception in thread "main" java.util.ServiceConfigurationError: org.apache.spark.sql.sources.DataSourceRegister: Provider com.google.cloud.spark.bigquery.BigQueryRelationProvider could not be instantiatedException in thread "main" java.util.ServiceConfigurationError: org.apache.spark.sql.sources.DataSourceRegister: Provider com.google.cloud.spark.bigquery.BigQueryRelationProvider could not be instantiated at java.base/java.util.ServiceLoader.fail(Unknown Source) at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(Unknown Source) at java.base/java.util.ServiceLoader$ProviderImpl.get(Unknown Source) at java.base/java.util.ServiceLoader$3.next(Unknown Source) at scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:44) at scala.collection.Iterator.foreach(Iterator.scala:941) at scala.collection.Iterator.foreach$(Iterator.scala:941) at scala.collection.AbstractIterator.foreach(Iterator.scala:1429) at scala.collection.IterableLike.foreach(IterableLike.scala:74) at scala.collection.IterableLike.foreach$(IterableLike.scala:73) at scala.collection.AbstractIterable.foreach(Iterable.scala:56) at scala.collection.TraversableLike.filterImpl(TraversableLike.scala:255) at scala.collection.TraversableLike.filterImpl$(TraversableLike.scala:249) at scala.collection.AbstractTraversable.filterImpl(Traversable.scala:108) at scala.collection.TraversableLike.filter(TraversableLike.scala:347) at scala.collection.TraversableLike.filter$(TraversableLike.scala:347) at scala.collection.AbstractTraversable.filter(Traversable.scala:108) at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:659) at org.apache.spark.sql.streaming.DataStreamReader.loadInternal(DataStreamReader.scala:209) at org.apache.spark.sql.streaming.DataStreamReader.load(DataStreamReader.scala:195) at com.xyz.xyz.xyz.AbstractCdcMessageToSqJob.runJob(AbstractCdcMessageToSqJob.java:94) at com.xyz.xyz.xyz.CdcMessageToSqlBillerJob.main(CdcMessageToSqlBillerJob.java:30) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:951) at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1030) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1039) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)Caused by: java.lang.NoClassDefFoundError: org/apache/spark/internal/Logging$class at com.google.cloud.spark.bigquery.BigQueryUtilScala$.(BigQueryUtil.scala:34) at com.google.cloud.spark.bigquery.BigQueryUtilScala$.(BigQueryUtil.scala) at com.google.cloud.spark.bigquery.BigQueryRelationProvider.(BigQueryRelationProvider.scala:43) at com.google.cloud.spark.bigquery.BigQueryRelationProvider.(BigQueryRelationProvider.scala:50) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source) ... 33 moreCaused by: java.lang.ClassNotFoundException: org.apache.spark.internal.Logging$class at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source) at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown Source) at java.base/java.lang.ClassLoader.loadClass(Unknown Source) ... 41 more was: I'm getting NoClassDefFoundError. is it removed in 3.1.1? Exception in thread "main" java.util.ServiceConfigurationError: org.apache.spark.sql.sources.DataSourceRegister: Provider com.google.cloud.spark.bigquery.BigQueryRelationProvider could not be instantiatedException in thread "main" java.util.ServiceConfigurationError: org.apache.spark.sql.sources.DataSourceRegister: Provider
[jira] [Created] (SPARK-36548) Throwing NoClassDefFoundError for Logging$class
sadagopan kalyanaraman created SPARK-36548: -- Summary: Throwing NoClassDefFoundError for Logging$class Key: SPARK-36548 URL: https://issues.apache.org/jira/browse/SPARK-36548 Project: Spark Issue Type: Bug Components: Kubernetes, Spark Core Affects Versions: 3.1.1 Reporter: sadagopan kalyanaraman I'm getting NoClassDefFoundError. is it removed in 3.1.1? Exception in thread "main" java.util.ServiceConfigurationError: org.apache.spark.sql.sources.DataSourceRegister: Provider com.google.cloud.spark.bigquery.BigQueryRelationProvider could not be instantiatedException in thread "main" java.util.ServiceConfigurationError: org.apache.spark.sql.sources.DataSourceRegister: Provider com.google.cloud.spark.bigquery.BigQueryRelationProvider could not be instantiated at java.base/java.util.ServiceLoader.fail(Unknown Source) at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(Unknown Source) at java.base/java.util.ServiceLoader$ProviderImpl.get(Unknown Source) at java.base/java.util.ServiceLoader$3.next(Unknown Source) at scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:44) at scala.collection.Iterator.foreach(Iterator.scala:941) at scala.collection.Iterator.foreach$(Iterator.scala:941) at scala.collection.AbstractIterator.foreach(Iterator.scala:1429) at scala.collection.IterableLike.foreach(IterableLike.scala:74) at scala.collection.IterableLike.foreach$(IterableLike.scala:73) at scala.collection.AbstractIterable.foreach(Iterable.scala:56) at scala.collection.TraversableLike.filterImpl(TraversableLike.scala:255) at scala.collection.TraversableLike.filterImpl$(TraversableLike.scala:249) at scala.collection.AbstractTraversable.filterImpl(Traversable.scala:108) at scala.collection.TraversableLike.filter(TraversableLike.scala:347) at scala.collection.TraversableLike.filter$(TraversableLike.scala:347) at scala.collection.AbstractTraversable.filter(Traversable.scala:108) at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:659) at org.apache.spark.sql.streaming.DataStreamReader.loadInternal(DataStreamReader.scala:209) at org.apache.spark.sql.streaming.DataStreamReader.load(DataStreamReader.scala:195) at com.xyz.xyz.xyz.AbstractCdcMessageToSqJob.runJob(AbstractCdcMessageToSqJob.java:94) at com.xyz.xyz.xyz.CdcMessageToSqlBillerJob.main(CdcMessageToSqlBillerJob.java:30) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:951) at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1030) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1039) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)Caused by: java.lang.NoClassDefFoundError: org/apache/spark/internal/Logging$class at com.google.cloud.spark.bigquery.BigQueryUtilScala$.(BigQueryUtil.scala:34) at com.google.cloud.spark.bigquery.BigQueryUtilScala$.(BigQueryUtil.scala) at com.google.cloud.spark.bigquery.BigQueryRelationProvider.(BigQueryRelationProvider.scala:43) at com.google.cloud.spark.bigquery.BigQueryRelationProvider.(BigQueryRelationProvider.scala:50) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source) ... 33 moreCaused by: java.lang.ClassNotFoundException: org.apache.spark.internal.Logging$class at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source) at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown Source) at java.base/java.lang.ClassLoader.loadClass(Unknown Source) ... 41 more -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
