Re: Spark 3.0.1 and spark 3.2 compatibility

2022-04-08 Thread Darcy Shen
> Do I need to recompile my application with 3.2 dependencies or application > compiled with 3.0.1 will work fine on 3.2 ? Yes. And here is How to compile conditionally for Apache Spark 3.1.x and Apache Spark >= 3.2.x object XYZ {

Re: Spark 3.0.1 and spark 3.2 compatibility

2022-04-08 Thread Gourav Sengupta
Hi, absolutely agree with Sean, besides that please see the release notes as well for SPARK versions, they do mention about any issues around compatibility Regards, Gourav On Thu, Apr 7, 2022 at 6:32 PM Sean Owen wrote: > (Don't cross post please) > Generally you definitely want to compile and

Re: Spark 3.0.1 and spark 3.2 compatibility

2022-04-07 Thread Sean Owen
(Don't cross post please) Generally you definitely want to compile and test vs what you're running on. There shouldn't be many binary or source incompatibilities -- these are avoided in a major release where possible. So it may need no code change. But I would certainly recompile just on

Spark 3.0.1 and spark 3.2 compatibility

2022-04-07 Thread Pralabh Kumar
Hi spark community I have quick question .I am planning to migrate from spark 3.0.1 to spark 3.2. Do I need to recompile my application with 3.2 dependencies or application compiled with 3.0.1 will work fine on 3.2 ? Regards Pralabh kumar