aromanenko-dev commented on issue #21092: URL: https://github.com/apache/beam/issues/21092#issuecomment-1256255125
@mosche AFAIK, there is no specific Scala dependencies for Spark portable runner and job server. They are sitting in the same package branch and have the same Spark/Scala dependencies as a native runner. I'm sure that @ibzib should know much better than me. Also, I'm wondering why Scala version was specifically set to 2.12.15 [here](https://github.com/apache/beam/blob/19f5e62c61dd9b201bc471fe79d109fa8406fff9/runners/spark/spark_runner.gradle#L164) since by default it is that way for Spark 3: ``` spark_version = '3.1.2' spark_scala_version = '2.12' ``` Indeed, the right solution would be align the spark/scala versions. The question here is do we need to support the different versions or not? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
