Thanks Kent for bringing this up. I vote for O.3, and suggest declaring support for Spark 3.0 is deprecated in the 1.6.0 release notes, then do O.3 in the 1.7.0-SNAPSHOT period.
Thanks, Cheng Pan On Wed, Jun 29, 2022 at 2:59 PM Kent Yao <[email protected]> wrote: > > Hi team, > > For our master branch a.k.a v1.6.0, we now test and build against > multiple spark revisions, including spark 3.0.x/3.1.x/3.2.x/3.3.x > running with java 8 and 11, and cross-version tests with a spark 3.2 > compiled engine running on spark 3.0.x/3.1.x/3.3.x distributions. > > Considering that spark 3.0 is EOL via spark community and the heavy > workload on our CI, we'd better drop spark 3.0.x support step by step. > > Here are the options/steps I have been thinking of, > > O.1. drop the ci jobs for 3.0.x compile/build/test for java 8 and 11 only. > O.2. drop the ci job for running the 3.2 complied engine on spark 3.0 > distribution. + O.1 > O.3. drop the spark-3.0 maven profile + O.2 > > Best Regards > > Kent
