Re: Apache Spark 2.4.8 (and EOL of 2.4)

Hi Mingjia, Thanks for fixing it. I can see it is included. Liang-Chi mingjia-2 wrote > Hi, All. > > I fixed SPARK-32708 > https://issues.apache.org/jira/browse/SPARK-32708#; > a while ago after 2.4.7 release. > PR:https://github.com/apache/spark/pull/29564 > > Since it's not listed as

Re: Apache Spark 2.4.8 (and EOL of 2.4)

Hi, All. I fixed SPARK-32708 a while ago after 2.4.7 release. PR:https://github.com/apache/spark/pull/29564 Since it's not listed as one of the JIRAs in Dongjoon's initial email, i'd like to check if it will be included in 2.4.8. Can anyone

Re: Apache Spark 2.4.8 (and EOL of 2.4)

Thanks Hyukjin and Dongjoon! :) Then I will start RC. Dongjoon Hyun-2 wrote > Given that Maven passed already with that profile and you tested locally, > I'm +1 for staring RC. > > Thanks, > Dongjoon. > > On Sun, Apr 4, 2021 at 2:24 AM Hyukjin Kwon > gurwls223@ > wrote: > >> I would

Re: Apache Spark 2.4.8 (and EOL of 2.4)

Given that Maven passed already with that profile and you tested locally, I'm +1 for staring RC. Thanks, Dongjoon. On Sun, Apr 4, 2021 at 2:24 AM Hyukjin Kwon wrote: > I would +1for just going ahead. That looks flaky to me too. > > Thanks Langchi for driving this! > > > On Sun, 4 Apr 2021,

Re: Apache Spark 2.4.8 (and EOL of 2.4)

I would +1for just going ahead. That looks flaky to me too. Thanks Langchi for driving this! On Sun, 4 Apr 2021, 18:17 Liang-Chi Hsieh, wrote: > Hi devs, > > Currently no open issues or ongoing issues targeting 2.4. > > On QA test dashboard, only spark-branch-2.4-test-sbt-hadoop-2.6 is in red

Re: Apache Spark 2.4.8 (and EOL of 2.4)

Hi devs, Currently no open issues or ongoing issues targeting 2.4. On QA test dashboard, only spark-branch-2.4-test-sbt-hadoop-2.6 is in red status. The failed test is org.apache.spark.sql.streaming.StreamingQueryManagerSuite.awaitAnyTermination with timeout and resetTerminated. It looks a flaky

Re: Apache Spark 2.4.8 (and EOL of 2.4)

To update current status. The only one remaining issue for 2.4 is: [SPARK-34855][CORE]spark context - avoid using local lazy val for callSite We are waiting the author to submit a PR for 2.4 branch. Liang-Chi Hsieh wrote > Thank you so much, Takeshi! > > > Takeshi Yamamuro wrote >> Hi,

Re: Apache Spark 2.4.8 (and EOL of 2.4)

Thank you so much, Takeshi! Takeshi Yamamuro wrote > Hi, viirya > > I'm looking now into "SPARK-34607: Add `Utils.isMemberClass` to fix a > malformed class name error > on jdk8u" . > > Bests, > Takeshi > > > Takeshi Yamamuro -- Sent from:

Re: Apache Spark 2.4.8 (and EOL of 2.4)

Hi, viirya I'm looking now into "SPARK-34607: Add `Utils.isMemberClass` to fix a malformed class name error on jdk8u" . Bests, Takeshi On Tue, Mar 16, 2021 at 4:45 AM Liang-Chi Hsieh wrote: > To update with current status. > > There are three tickets targeting 2.4 that are still ongoing. > >

Re: Apache Spark 2.4.8 (and EOL of 2.4)

To update with current status. There are three tickets targeting 2.4 that are still ongoing. SPARK-34719: Correctly resolve the view query with duplicated column names SPARK-34607: Add `Utils.isMemberClass` to fix a malformed class name error on jdk8u SPARK-34726: Fix collectToPython timeouts

Re: Apache Spark 2.4.8 (and EOL of 2.4)

Thank you for the update. +1 for your plan. Bests, Dongjoon. On Tue, Mar 9, 2021 at 12:46 PM Liang-Chi Hsieh wrote: > I just contacted Shane and seems there is ongoing github fetches timing out > issue on Jenkins. > > That being said, currently the QA test is unavailable. I guess it is unsafe

Re: Apache Spark 2.4.8 (and EOL of 2.4)

I just contacted Shane and seems there is ongoing github fetches timing out issue on Jenkins. That being said, currently the QA test is unavailable. I guess it is unsafe to make a release cut due to lack of reliable QA test result. I may defer the cut until QA test comes back if no objection.

Re: Apache Spark 2.4.8 (and EOL of 2.4)

Hi devs, I'm going to cut the branch yesterday. I'd like to share current progress. I hit a problem during dry run of the release script. Fixed it in SPARK-34672. The latest dry run looks okay as build, docs, publish all success. But the last step (push the tag) has a fatal error, I'm not sure

Re: Apache Spark 2.4.8 (and EOL of 2.4)

Thank you Dongjoon. I'm going to cut the branch now. Hopefully I can make it soon (need to get familiar with the process as first time :) ) Liang-Chi Dongjoon Hyun-2 wrote > Thank you, Liang-Chi! Next Monday sounds good. > > To All. Please ping Liang-Chi if you have a missed backport. > >

Re: Apache Spark 2.4.8 (and EOL of 2.4)

Thank you, Liang-Chi! Next Monday sounds good. To All. Please ping Liang-Chi if you have a missed backport. Bests, Dongjoon. On Thu, Mar 4, 2021 at 7:00 PM Xiao Li wrote: > Thank you, Liang-Chi! > > Xiao > > On Thu, Mar 4, 2021 at 6:25 PM Hyukjin Kwon wrote: > >> Thanks @Liang-Chi Hsieh

Re: Apache Spark 2.4.8 (and EOL of 2.4)

Thank you, Liang-Chi! Xiao On Thu, Mar 4, 2021 at 6:25 PM Hyukjin Kwon wrote: > Thanks @Liang-Chi Hsieh for driving this. > > 2021년 3월 5일 (금) 오전 5:21, Liang-Chi Hsieh 님이 작성: > >> >> Thanks all for the input. >> >> If there is no objection, I am going to cut the branch next Monday. >> >>

Re: Apache Spark 2.4.8 (and EOL of 2.4)

Thanks @Liang-Chi Hsieh for driving this. 2021년 3월 5일 (금) 오전 5:21, Liang-Chi Hsieh 님이 작성: > > Thanks all for the input. > > If there is no objection, I am going to cut the branch next Monday. > > Thanks. > Liang-Chi > > > Takeshi Yamamuro wrote > > +1 for releasing 2.4.8 and thanks, Liang-chi,

Re: Apache Spark 2.4.8 (and EOL of 2.4)

Thanks all for the input. If there is no objection, I am going to cut the branch next Monday. Thanks. Liang-Chi Takeshi Yamamuro wrote > +1 for releasing 2.4.8 and thanks, Liang-chi, for volunteering. > Btw, anyone roughly know how many v2.4 users still are based on some stats > (e.g., # of

Re: Apache Spark 2.4.8 (and EOL of 2.4)

+1 for releasing 2.4.8 and thanks, Liang-chi, for volunteering. Btw, anyone roughly know how many v2.4 users still are based on some stats (e.g., # of v2.4.7 downloads from the official repos)? Most users have started using v3.x? On Thu, Mar 4, 2021 at 8:34 AM Hyukjin Kwon wrote: > Yeah, I

Re: Apache Spark 2.4.8 (and EOL of 2.4)

Yeah, I would prefer to have a 2.4.8 release as an EOL too. I don't mind having 2.4.9 as EOL too if that's preferred from more people. 2021년 3월 4일 (목) 오전 4:01, Sean Owen 님이 작성: > Sure, I'm even arguing that 2.4.8 could possibly be the final release. No > objection of course to continuing to

Re: Apache Spark 2.4.8 (and EOL of 2.4)

Sure, I'm even arguing that 2.4.8 could possibly be the final release. No objection of course to continuing to backport to 2.4.x where appropriate and cutting 2.4.9 later in the year as a final EOL release, either. On Wed, Mar 3, 2021 at 12:59 PM Dongjoon Hyun wrote: > Thank you, Sean. > > Ya,

Re: Apache Spark 2.4.8 (and EOL of 2.4)

Thank you, Sean. Ya, exactly, we can release 2.4.8 as a normal release first and use 2.4.9 as the EOL release. Since 2.4.7 was released almost 6 months ago, 2.4.8 is a little late in terms of the cadence. Bests, Dongjoon. On Wed, Mar 3, 2021 at 10:55 AM Sean Owen wrote: > For reference,

Re: Apache Spark 2.4.8 (and EOL of 2.4)

For reference, 2.3.x was maintained from February 2018 (2.3.0) to Sep 2019 (2.3.4), or about 19 months. The 2.4 branch should probably be maintained longer than that, as the final 2.x branch. 2.4.0 was released in Nov 2018. A final release in, say, April 2021 would be about 30 months. That feels

Re: Apache Spark 2.4.8 (and EOL of 2.4)

Thank you for volunteering as Apache Spark 2.4.8 release manager, Liang-chi! On Wed, Mar 3, 2021 at 10:13 AM Liang-Chi Hsieh wrote: > > Thanks Dongjoon! > > +1 and I volunteer to do the release of 2.4.8 if it passes. > > > Liang-Chi > > > > > -- > Sent from:

Re: Apache Spark 2.4.8 (and EOL of 2.4)

Thanks Dongjoon! +1 and I volunteer to do the release of 2.4.8 if it passes. Liang-Chi -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Apache Spark 2.4.8 (and EOL of 2.4)

Hi, All. We successfully completed Apache Spark 3.1.1 and 3.0.2 releases and started 3.2.0 discussion already. Let's talk about branch-2.4 because there exists some discussions on JIRA and GitHub about skipping backporting to 2.4. Since `branch-2.4` has been maintained well as LTS, I'd like to