Re: Re: maven with Spark 3.4.0 fails compilation

2023-05-29 Thread Bjørn Jørgensen
t;> <https://github.com/apache/spark/blob/88f69d6f92860823b1a90bc162ebca2b7c8132fc/pom.xml#L170>. >>> Since you are using spark-core_2.13 and spark-sql_2.13, you should stick to >>> the major(13) and the minor version(8). Not using any of these may cause >>> unex

Re: Re: maven with Spark 3.4.0 fails compilation

2023-05-29 Thread Mich Talebzadeh
......... >> Lingzhe Sun >> Hirain Technology >> >> >> *From:* Mich Talebzadeh >> *Date:* 2023-05-29 17:55 >> *To:* Bjørn Jørgensen >> *CC:* user @spark >> *Subject:* Re: maven with Spark 3.4.0 fails compila

Re: Re: maven with Spark 3.4.0 fails compilation

2023-05-29 Thread Bjørn Jørgensen
/26411339/19476830>can be a a pitfall for you. > > -- > Best Regards! > ... > Lingzhe Sun > Hirain Technology > > > *From:* Mich Talebzadeh > *Date:* 2023-05-29 17:55 > *To:* Bjørn Jørgense

Re: Re: maven with Spark 3.4.0 fails compilation

2023-05-29 Thread Lingzhe Sun
! ... Lingzhe Sun Hirain Technology From: Mich Talebzadeh Date: 2023-05-29 17:55 To: Bjørn Jørgensen CC: user @spark Subject: Re: maven with Spark 3.4.0 fails compilation Thanks for your helpful comments Bjorn. I managed to compile the code

Re: maven with Spark 3.4.0 fails compilation

2023-05-29 Thread Mich Talebzadeh
Thanks for your helpful comments Bjorn. I managed to compile the code with maven but when it run it fails with Application is ReduceByKey Exception in thread "main" java.lang.NoSuchMethodError: scala.package$.Seq()Lscala/collection/immutable/Seq$; at

Re: maven with Spark 3.4.0 fails compilation

2023-05-28 Thread Bjørn Jørgensen
>From chatgpt4 The problem appears to be that there is a mismatch between the version of Scala used by the Scala Maven plugin and the version of the Scala library defined as a dependency in your POM. You've defined your Scala version in your properties as `2.12.17` but you're pulling in