Re: Okio Vulnerability in Spark 3.4.1
[SPARK-46662][K8S][BUILD] Upgrade kubernetes-client to 6.10.0 <https://github.com/apache/spark/pull/44672> a new version of kubernets-client with okio version 1.17.6 is now merged to master and will be in the spark 4.0 version. tir. 14. nov. 2023 kl. 15:21 skrev Bjørn Jørgensen : > FYI > I have opened Update okio to version 1.17.6 > <https://github.com/fabric8io/kubernetes-client/pull/5587> for this now. > > tor. 31. aug. 2023 kl. 21:18 skrev Sean Owen : > >> It's a dependency of some other HTTP library. Use mvn dependency:tree to >> see where it comes from. It may be more straightforward to upgrade the >> library that brings it in, assuming a later version brings in a later okio. >> You can also manage up the version directly with a new entry in >> >> >> However, does this affect Spark? all else equal it doesn't hurt to >> upgrade, but wondering if there is even a theory that it needs to be >> updated. >> >> >> On Thu, Aug 31, 2023 at 7:42 AM Agrawal, Sanket < >> sankeagra...@deloitte.com> wrote: >> >>> I don’t see an entry in pom.xml while building spark. I think it is >>> being downloaded as part of some other dependency. >>> >>> >>> >>> *From:* Sean Owen >>> *Sent:* Thursday, August 31, 2023 5:10 PM >>> *To:* Agrawal, Sanket >>> *Cc:* user@spark.apache.org >>> *Subject:* [EXT] Re: Okio Vulnerability in Spark 3.4.1 >>> >>> >>> >>> Does the vulnerability affect Spark? >>> >>> In any event, have you tried updating Okio in the Spark build? I don't >>> believe you could just replace the JAR, as other libraries probably rely on >>> it and compiled against the current version. >>> >>> >>> >>> On Thu, Aug 31, 2023 at 6:02 AM Agrawal, Sanket < >>> sankeagra...@deloitte.com.invalid> wrote: >>> >>> Hi All, >>> >>> >>> >>> Amazon inspector has detected a vulnerability in okio-1.15.0.jar JAR in >>> Spark 3.4.1. It suggests to upgrade the jar version to 3.4.0. But when we >>> try this version of jar then the spark application is failing with below >>> error: >>> >>> >>> >>> py4j.protocol.Py4JJavaError: An error occurred while calling >>> None.org.apache.spark.api.java.JavaSparkContext. >>> >>> : java.lang.NoClassDefFoundError: okio/BufferedSource >>> >>> at okhttp3.internal.Util.(Util.java:62) >>> >>> at okhttp3.OkHttpClient.(OkHttpClient.java:127) >>> >>> at okhttp3.OkHttpClient$Builder.(OkHttpClient.java:475) >>> >>> at >>> io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newOkHttpClientBuilder(OkHttpClientFactory.java:41) >>> >>> at >>> io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newBuilder(OkHttpClientFactory.java:56) >>> >>> at >>> io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newBuilder(OkHttpClientFactory.java:68) >>> >>> at >>> io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newBuilder(OkHttpClientFactory.java:30) >>> >>> at >>> io.fabric8.kubernetes.client.KubernetesClientBuilder.getHttpClient(KubernetesClientBuilder.java:88) >>> >>> at >>> io.fabric8.kubernetes.client.KubernetesClientBuilder.build(KubernetesClientBuilder.java:78) >>> >>> at >>> org.apache.spark.deploy.k8s.SparkKubernetesClientFactory$.createKubernetesClient(SparkKubernetesClientFactory.scala:120) >>> >>> at >>> org.apache.spark.scheduler.cluster.k8s.KubernetesClusterManager.createSchedulerBackend(KubernetesClusterManager.scala:111) >>> >>> at >>> org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:3037) >>> >>> at org.apache.spark.SparkContext.(SparkContext.scala:568) >>> >>> at >>> org.apache.spark.api.java.JavaSparkContext.(JavaSparkContext.scala:58) >>> >>> at >>> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native >>> Method) >>> >>> at >>> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown >>> Source) >>> >>> at >>> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown >>> Source) >>
Re: Okio Vulnerability in Spark 3.4.1
FYI I have opened Update okio to version 1.17.6 <https://github.com/fabric8io/kubernetes-client/pull/5587> for this now. tor. 31. aug. 2023 kl. 21:18 skrev Sean Owen : > It's a dependency of some other HTTP library. Use mvn dependency:tree to > see where it comes from. It may be more straightforward to upgrade the > library that brings it in, assuming a later version brings in a later okio. > You can also manage up the version directly with a new entry in > > > However, does this affect Spark? all else equal it doesn't hurt to > upgrade, but wondering if there is even a theory that it needs to be > updated. > > > On Thu, Aug 31, 2023 at 7:42 AM Agrawal, Sanket > wrote: > >> I don’t see an entry in pom.xml while building spark. I think it is being >> downloaded as part of some other dependency. >> >> >> >> *From:* Sean Owen >> *Sent:* Thursday, August 31, 2023 5:10 PM >> *To:* Agrawal, Sanket >> *Cc:* user@spark.apache.org >> *Subject:* [EXT] Re: Okio Vulnerability in Spark 3.4.1 >> >> >> >> Does the vulnerability affect Spark? >> >> In any event, have you tried updating Okio in the Spark build? I don't >> believe you could just replace the JAR, as other libraries probably rely on >> it and compiled against the current version. >> >> >> >> On Thu, Aug 31, 2023 at 6:02 AM Agrawal, Sanket < >> sankeagra...@deloitte.com.invalid> wrote: >> >> Hi All, >> >> >> >> Amazon inspector has detected a vulnerability in okio-1.15.0.jar JAR in >> Spark 3.4.1. It suggests to upgrade the jar version to 3.4.0. But when we >> try this version of jar then the spark application is failing with below >> error: >> >> >> >> py4j.protocol.Py4JJavaError: An error occurred while calling >> None.org.apache.spark.api.java.JavaSparkContext. >> >> : java.lang.NoClassDefFoundError: okio/BufferedSource >> >> at okhttp3.internal.Util.(Util.java:62) >> >> at okhttp3.OkHttpClient.(OkHttpClient.java:127) >> >> at okhttp3.OkHttpClient$Builder.(OkHttpClient.java:475) >> >> at >> io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newOkHttpClientBuilder(OkHttpClientFactory.java:41) >> >> at >> io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newBuilder(OkHttpClientFactory.java:56) >> >> at >> io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newBuilder(OkHttpClientFactory.java:68) >> >> at >> io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newBuilder(OkHttpClientFactory.java:30) >> >> at >> io.fabric8.kubernetes.client.KubernetesClientBuilder.getHttpClient(KubernetesClientBuilder.java:88) >> >> at >> io.fabric8.kubernetes.client.KubernetesClientBuilder.build(KubernetesClientBuilder.java:78) >> >> at >> org.apache.spark.deploy.k8s.SparkKubernetesClientFactory$.createKubernetesClient(SparkKubernetesClientFactory.scala:120) >> >> at >> org.apache.spark.scheduler.cluster.k8s.KubernetesClusterManager.createSchedulerBackend(KubernetesClusterManager.scala:111) >> >> at >> org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:3037) >> >> at org.apache.spark.SparkContext.(SparkContext.scala:568) >> >> at >> org.apache.spark.api.java.JavaSparkContext.(JavaSparkContext.scala:58) >> >> at >> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native >> Method) >> >> at >> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown >> Source) >> >> at >> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown >> Source) >> >> at java.base/java.lang.reflect.Constructor.newInstance(Unknown >> Source) >> >> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:247) >> >> at >> py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374) >> >> at py4j.Gateway.invoke(Gateway.java:238) >> >> at >> py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80) >> >> at >> py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69) >> >> at >> py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182) >> >> at >> py4j.ClientServerConnection.run(
Re: Okio Vulnerability in Spark 3.4.1
Have tried to upgrade it. It is from kubernetes-client [SPARK-43990][BUILD] Upgrade kubernetes-client to 6.7.2 <https://github.com/apache/spark/pull/41490#issuecomment-1581275661> tor. 31. aug. 2023 kl. 14:47 skrev Agrawal, Sanket : > I don’t see an entry in pom.xml while building spark. I think it is being > downloaded as part of some other dependency. > > > > *From:* Sean Owen > *Sent:* Thursday, August 31, 2023 5:10 PM > *To:* Agrawal, Sanket > *Cc:* user@spark.apache.org > *Subject:* [EXT] Re: Okio Vulnerability in Spark 3.4.1 > > > > Does the vulnerability affect Spark? > > In any event, have you tried updating Okio in the Spark build? I don't > believe you could just replace the JAR, as other libraries probably rely on > it and compiled against the current version. > > > > On Thu, Aug 31, 2023 at 6:02 AM Agrawal, Sanket < > sankeagra...@deloitte.com.invalid> wrote: > > Hi All, > > > > Amazon inspector has detected a vulnerability in okio-1.15.0.jar JAR in > Spark 3.4.1. It suggests to upgrade the jar version to 3.4.0. But when we > try this version of jar then the spark application is failing with below > error: > > > > py4j.protocol.Py4JJavaError: An error occurred while calling > None.org.apache.spark.api.java.JavaSparkContext. > > : java.lang.NoClassDefFoundError: okio/BufferedSource > > at okhttp3.internal.Util.(Util.java:62) > > at okhttp3.OkHttpClient.(OkHttpClient.java:127) > > at okhttp3.OkHttpClient$Builder.(OkHttpClient.java:475) > > at > io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newOkHttpClientBuilder(OkHttpClientFactory.java:41) > > at > io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newBuilder(OkHttpClientFactory.java:56) > > at > io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newBuilder(OkHttpClientFactory.java:68) > > at > io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newBuilder(OkHttpClientFactory.java:30) > > at > io.fabric8.kubernetes.client.KubernetesClientBuilder.getHttpClient(KubernetesClientBuilder.java:88) > > at > io.fabric8.kubernetes.client.KubernetesClientBuilder.build(KubernetesClientBuilder.java:78) > > at > org.apache.spark.deploy.k8s.SparkKubernetesClientFactory$.createKubernetesClient(SparkKubernetesClientFactory.scala:120) > > at > org.apache.spark.scheduler.cluster.k8s.KubernetesClusterManager.createSchedulerBackend(KubernetesClusterManager.scala:111) > > at > org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:3037) > > at org.apache.spark.SparkContext.(SparkContext.scala:568) > > at > org.apache.spark.api.java.JavaSparkContext.(JavaSparkContext.scala:58) > > at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > > at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown > Source) > > at > java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown > Source) > > at java.base/java.lang.reflect.Constructor.newInstance(Unknown > Source) > > at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:247) > > at > py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374) > > at py4j.Gateway.invoke(Gateway.java:238) > > at > py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80) > > at > py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69) > > at > py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182) > > at py4j.ClientServerConnection.run(ClientServerConnection.java:106) > > at java.base/java.lang.Thread.run(Unknown Source) > > Caused by: java.lang.ClassNotFoundException: okio.BufferedSource > > at > java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source) > > at > java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown > Source) > > at java.base/java.lang.ClassLoader.loadClass(Unknown Source) > > ... 26 more > > > > Replaced the existing jar with the JAR file at > https://repo1.maven.org/maven2/com/squareup/okio/okio/3.4.0/okio-3.4.0.jar > <https://secure-web.cisco.com/1bTvNPAJgVtYdy2nfHp1eUSEqLfelqshEI8TO89yzE25dM5y8HHDCwYxrzTLlmcAFi6uIbQLO2OiJht-xgXmI3lFdV8YpP0j3re47gncrBpwO9m6xYQeLhqXUAnUVP2MoxHbdHlZcdSwDqWkjbOKudm7Go1ICzxhw_VBXuK9n8XF3y7__B86mqWNsroDGD3hbH_tTQTHpXK-4tJCeIZTKmwItL1A3zlRL8lBHG_zgTDSiX9W7ufy8rHP2JZEp_FaftGMsnPA56
Re: Okio Vulnerability in Spark 3.4.1
It's a dependency of some other HTTP library. Use mvn dependency:tree to see where it comes from. It may be more straightforward to upgrade the library that brings it in, assuming a later version brings in a later okio. You can also manage up the version directly with a new entry in However, does this affect Spark? all else equal it doesn't hurt to upgrade, but wondering if there is even a theory that it needs to be updated. On Thu, Aug 31, 2023 at 7:42 AM Agrawal, Sanket wrote: > I don’t see an entry in pom.xml while building spark. I think it is being > downloaded as part of some other dependency. > > > > *From:* Sean Owen > *Sent:* Thursday, August 31, 2023 5:10 PM > *To:* Agrawal, Sanket > *Cc:* user@spark.apache.org > *Subject:* [EXT] Re: Okio Vulnerability in Spark 3.4.1 > > > > Does the vulnerability affect Spark? > > In any event, have you tried updating Okio in the Spark build? I don't > believe you could just replace the JAR, as other libraries probably rely on > it and compiled against the current version. > > > > On Thu, Aug 31, 2023 at 6:02 AM Agrawal, Sanket < > sankeagra...@deloitte.com.invalid> wrote: > > Hi All, > > > > Amazon inspector has detected a vulnerability in okio-1.15.0.jar JAR in > Spark 3.4.1. It suggests to upgrade the jar version to 3.4.0. But when we > try this version of jar then the spark application is failing with below > error: > > > > py4j.protocol.Py4JJavaError: An error occurred while calling > None.org.apache.spark.api.java.JavaSparkContext. > > : java.lang.NoClassDefFoundError: okio/BufferedSource > > at okhttp3.internal.Util.(Util.java:62) > > at okhttp3.OkHttpClient.(OkHttpClient.java:127) > > at okhttp3.OkHttpClient$Builder.(OkHttpClient.java:475) > > at > io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newOkHttpClientBuilder(OkHttpClientFactory.java:41) > > at > io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newBuilder(OkHttpClientFactory.java:56) > > at > io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newBuilder(OkHttpClientFactory.java:68) > > at > io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newBuilder(OkHttpClientFactory.java:30) > > at > io.fabric8.kubernetes.client.KubernetesClientBuilder.getHttpClient(KubernetesClientBuilder.java:88) > > at > io.fabric8.kubernetes.client.KubernetesClientBuilder.build(KubernetesClientBuilder.java:78) > > at > org.apache.spark.deploy.k8s.SparkKubernetesClientFactory$.createKubernetesClient(SparkKubernetesClientFactory.scala:120) > > at > org.apache.spark.scheduler.cluster.k8s.KubernetesClusterManager.createSchedulerBackend(KubernetesClusterManager.scala:111) > > at > org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:3037) > > at org.apache.spark.SparkContext.(SparkContext.scala:568) > > at > org.apache.spark.api.java.JavaSparkContext.(JavaSparkContext.scala:58) > > at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > > at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown > Source) > > at > java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown > Source) > > at java.base/java.lang.reflect.Constructor.newInstance(Unknown > Source) > > at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:247) > > at > py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374) > > at py4j.Gateway.invoke(Gateway.java:238) > > at > py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80) > > at > py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69) > > at > py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182) > > at py4j.ClientServerConnection.run(ClientServerConnection.java:106) > > at java.base/java.lang.Thread.run(Unknown Source) > > Caused by: java.lang.ClassNotFoundException: okio.BufferedSource > > at > java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source) > > at > java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown > Source) > > at java.base/java.lang.ClassLoader.loadClass(Unknown Source) > >
RE: Okio Vulnerability in Spark 3.4.1
I don’t see an entry in pom.xml while building spark. I think it is being downloaded as part of some other dependency. From: Sean Owen Sent: Thursday, August 31, 2023 5:10 PM To: Agrawal, Sanket Cc: user@spark.apache.org Subject: [EXT] Re: Okio Vulnerability in Spark 3.4.1 Does the vulnerability affect Spark? In any event, have you tried updating Okio in the Spark build? I don't believe you could just replace the JAR, as other libraries probably rely on it and compiled against the current version. On Thu, Aug 31, 2023 at 6:02 AM Agrawal, Sanket mailto:sankeagra...@deloitte.com.invalid>> wrote: Hi All, Amazon inspector has detected a vulnerability in okio-1.15.0.jar JAR in Spark 3.4.1. It suggests to upgrade the jar version to 3.4.0. But when we try this version of jar then the spark application is failing with below error: py4j.protocol.Py4JJavaError: An error occurred while calling None.org.apache.spark.api.java.JavaSparkContext. : java.lang.NoClassDefFoundError: okio/BufferedSource at okhttp3.internal.Util.(Util.java:62) at okhttp3.OkHttpClient.(OkHttpClient.java:127) at okhttp3.OkHttpClient$Builder.(OkHttpClient.java:475) at io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newOkHttpClientBuilder(OkHttpClientFactory.java:41) at io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newBuilder(OkHttpClientFactory.java:56) at io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newBuilder(OkHttpClientFactory.java:68) at io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newBuilder(OkHttpClientFactory.java:30) at io.fabric8.kubernetes.client.KubernetesClientBuilder.getHttpClient(KubernetesClientBuilder.java:88) at io.fabric8.kubernetes.client.KubernetesClientBuilder.build(KubernetesClientBuilder.java:78) at org.apache.spark.deploy.k8s.SparkKubernetesClientFactory$.createKubernetesClient(SparkKubernetesClientFactory.scala:120) at org.apache.spark.scheduler.cluster.k8s.KubernetesClusterManager.createSchedulerBackend(KubernetesClusterManager.scala:111) at org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:3037) at org.apache.spark.SparkContext.(SparkContext.scala:568) at org.apache.spark.api.java.JavaSparkContext.(JavaSparkContext.scala:58) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:247) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374) at py4j.Gateway.invoke(Gateway.java:238) at py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80) at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69) at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182) at py4j.ClientServerConnection.run(ClientServerConnection.java:106) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.lang.ClassNotFoundException: okio.BufferedSource at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source) at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown Source) at java.base/java.lang.ClassLoader.loadClass(Unknown Source) ... 26 more Replaced the existing jar with the JAR file at https://repo1.maven.org/maven2/com/squareup/okio/okio/3.4.0/okio-3.4.0.jar<https://secure-web.cisco.com/1bTvNPAJgVtYdy2nfHp1eUSEqLfelqshEI8TO89yzE25dM5y8HHDCwYxrzTLlmcAFi6uIbQLO2OiJht-xgXmI3lFdV8YpP0j3re47gncrBpwO9m6xYQeLhqXUAnUVP2MoxHbdHlZcdSwDqWkjbOKudm7Go1ICzxhw_VBXuK9n8XF3y7__B86mqWNsroDGD3hbH_tTQTHpXK-4tJCeIZTKmwItL1A3zlRL8lBHG_zgTDSiX9W7ufy8rHP2JZEp_FaftGMsnPA56IGHQVQAmOIobPSQDi4MfsiyUj0HsHPH3fZaz8_8TnPu178yfi8pCurkmr7b0X0NmFTdeAuFHKhdoOYooWDPsuBIYxknd3p1wLXrQezp26QrkjEiUMjNH9S18HPLH2BfN627X6zqQD7sVUUo1hzMRvnllVZVQWPL6H7lisyk-7w2pTAX6bm9wZuWTN9U4hZzjoc1-s1YumCiexaMOfiqEbTKppNDB8jOXBPIS9HDdEVDUl8OAIKz-T480x_NePZwHGT4hHtSwUaHCw/https%3A%2F%2Frepo1.maven.org%2Fmaven2%2Fcom%2Fsquareup%2Fokio%2Fokio%2F3.4.0%2Fokio-3.4.0.jar> PFB, the vulnerability details: Link: https://nvd.nist.gov/vuln/detail/CVE-2023-3635<https://secure-web.cisco.com/1KDv1iIbxjIsZCdyvwVzp9hDXe9ClcztVaj_gKzaoEQJ0Qb1BrTG7ivs0bsKiKVJvN8BJ0KvCwQKgWJGRfrWZYTkrgVMl1RfmnIn2fTYgyXd5ATU-4FBIQstOXRlc1dQnRNW9jr8OZCqV_xqbzAuLEP--uh0URczU8BYxyefL4Ly6ntQ2Y0BtKEOq3LZflTianf1d3UH30m_mmQmt3pE_3S7qFc9R9I3NqWJmkxuYVC1gVhnWBpbelMz5P7Q8D4GXo_L7tgj_nPwQyAcwqLjaIUVf-SYPU8T-WsaxeDkW6gp5oNKuYFqDzxXghsRJxzOj7i5noa1bj3-uSj0f0tT8xZ3L42uUTN
Re: Okio Vulnerability in Spark 3.4.1
Does the vulnerability affect Spark?
In any event, have you tried updating Okio in the Spark build? I don't
believe you could just replace the JAR, as other libraries probably rely on
it and compiled against the current version.
On Thu, Aug 31, 2023 at 6:02 AM Agrawal, Sanket
wrote:
> Hi All,
>
>
>
> Amazon inspector has detected a vulnerability in okio-1.15.0.jar JAR in
> Spark 3.4.1. It suggests to upgrade the jar version to 3.4.0. But when we
> try this version of jar then the spark application is failing with below
> error:
>
>
>
> py4j.protocol.Py4JJavaError: An error occurred while calling
> None.org.apache.spark.api.java.JavaSparkContext.
>
> : java.lang.NoClassDefFoundError: okio/BufferedSource
>
> at okhttp3.internal.Util.(Util.java:62)
>
> at okhttp3.OkHttpClient.(OkHttpClient.java:127)
>
> at okhttp3.OkHttpClient$Builder.(OkHttpClient.java:475)
>
> at
> io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newOkHttpClientBuilder(OkHttpClientFactory.java:41)
>
> at
> io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newBuilder(OkHttpClientFactory.java:56)
>
> at
> io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newBuilder(OkHttpClientFactory.java:68)
>
> at
> io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newBuilder(OkHttpClientFactory.java:30)
>
> at
> io.fabric8.kubernetes.client.KubernetesClientBuilder.getHttpClient(KubernetesClientBuilder.java:88)
>
> at
> io.fabric8.kubernetes.client.KubernetesClientBuilder.build(KubernetesClientBuilder.java:78)
>
> at
> org.apache.spark.deploy.k8s.SparkKubernetesClientFactory$.createKubernetesClient(SparkKubernetesClientFactory.scala:120)
>
> at
> org.apache.spark.scheduler.cluster.k8s.KubernetesClusterManager.createSchedulerBackend(KubernetesClusterManager.scala:111)
>
> at
> org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:3037)
>
> at org.apache.spark.SparkContext.(SparkContext.scala:568)
>
> at
> org.apache.spark.api.java.JavaSparkContext.(JavaSparkContext.scala:58)
>
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
>
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown
> Source)
>
> at
> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
> Source)
>
> at java.base/java.lang.reflect.Constructor.newInstance(Unknown
> Source)
>
> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:247)
>
> at
> py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374)
>
> at py4j.Gateway.invoke(Gateway.java:238)
>
> at
> py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80)
>
> at
> py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69)
>
> at
> py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
>
> at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
>
> at java.base/java.lang.Thread.run(Unknown Source)
>
> Caused by: java.lang.ClassNotFoundException: okio.BufferedSource
>
> at
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source)
>
> at
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown
> Source)
>
> at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
>
> ... 26 more
>
>
>
> Replaced the existing jar with the JAR file at
> https://repo1.maven.org/maven2/com/squareup/okio/okio/3.4.0/okio-3.4.0.jar
>
>
>
>
>
> PFB, the vulnerability details:
>
> Link: https://nvd.nist.gov/vuln/detail/CVE-2023-3635
>
>
>
> Any guidance here would be of great help.
>
>
>
> Thanks,
>
> Sanket A.
>
> This message (including any attachments) contains confidential information
> intended for a specific individual and purpose, and is protected by law. If
> you are not the intended recipient, you should delete this message and any
> disclosure, copying, or distribution of this message, or the taking of any
> action based on it, by you is strictly prohibited.
>
> Deloitte refers to a Deloitte member firm, one of its related entities, or
> Deloitte Touche Tohmatsu Limited ("DTTL"). Each Deloitte member firm is a
> separate legal entity and a member of DTTL. DTTL does not provide services
> to clients. Please see www.deloitte.com/about to learn more.
>
> v.E.1
>
