+1 for next Monday. Dongjoon.
On Wed, May 1, 2024 at 8:46 AM Tathagata Das <tathagata.das1...@gmail.com> wrote: > Next week sounds great! Thank you Wenchen! > > On Wed, May 1, 2024 at 11:16 AM Wenchen Fan <cloud0...@gmail.com> wrote: > >> Yea I think a preview release won't hurt (without a branch cut). We don't >> need to wait for all the ongoing projects to be ready. How about we do a >> 4.0 preview release based on the current master branch next Monday? >> >> On Wed, May 1, 2024 at 11:06 PM Tathagata Das < >> tathagata.das1...@gmail.com> wrote: >> >>> Hey all, >>> >>> Reviving this thread, but Spark master has already accumulated a huge >>> amount of changes. As a downstream project maintainer, I want to really >>> start testing the new features and other breaking changes, and it's hard to >>> do that without a Preview release. So the sooner we make a Preview release, >>> the faster we can start getting feedback for fixing things for a great >>> Spark 4.0 final release. >>> >>> So I urge the community to produce a Spark 4.0 Preview soon even if >>> certain features targeting the Delta 4.0 release are still incomplete. >>> >>> Thanks! >>> >>> >>> On Wed, Apr 17, 2024 at 8:35 AM Wenchen Fan <cloud0...@gmail.com> wrote: >>> >>>> Thank you all for the replies! >>>> >>>> To @Nicholas Chammas <nicholas.cham...@gmail.com> : Thanks for >>>> cleaning up the error terminology and documentation! I've merged the first >>>> PR and let's finish others before the 4.0 release. >>>> To @Dongjoon Hyun <dongjoon.h...@gmail.com> : Thanks for driving the >>>> ANSI on by default effort! Now the vote has passed, let's flip the config >>>> and finish the DataFrame error context feature before 4.0. >>>> To @Jungtaek Lim <kabhwan.opensou...@gmail.com> : Ack. We can treat >>>> the Streaming state store data source as completed for 4.0 then. >>>> To @Cheng Pan <cheng...@apache.org> : Yea we definitely should have a >>>> preview release. Let's collect more feedback on the ongoing projects and >>>> then we can propose a date for the preview release. >>>> >>>> On Wed, Apr 17, 2024 at 1:22 PM Cheng Pan <pan3...@gmail.com> wrote: >>>> >>>>> will we have preview release for 4.0.0 like we did for 2.0.0 and 3.0.0? >>>>> >>>>> Thanks, >>>>> Cheng Pan >>>>> >>>>> >>>>> > On Apr 15, 2024, at 09:58, Jungtaek Lim < >>>>> kabhwan.opensou...@gmail.com> wrote: >>>>> > >>>>> > W.r.t. state data source - reader (SPARK-45511), there are several >>>>> follow-up tickets, but we don't plan to address them soon. The current >>>>> implementation is the final shape for Spark 4.0.0, unless there are >>>>> demands >>>>> on the follow-up tickets. >>>>> > >>>>> > We may want to check the plan for transformWithState - my >>>>> understanding is that we want to release the feature to 4.0.0, but there >>>>> are several remaining works to be done. While the tentative timeline for >>>>> releasing is June 2024, what would be the tentative timeline for the RC >>>>> cut? >>>>> > (cc. Anish to add more context on the plan for transformWithState) >>>>> > >>>>> > On Sat, Apr 13, 2024 at 3:15 AM Wenchen Fan <cloud0...@gmail.com> >>>>> wrote: >>>>> > Hi all, >>>>> > >>>>> > It's close to the previously proposed 4.0.0 release date (June >>>>> 2024), and I think it's time to prepare for it and discuss the ongoing >>>>> projects: >>>>> > • >>>>> > ANSI by default >>>>> > • Spark Connect GA >>>>> > • Structured Logging >>>>> > • Streaming state store data source >>>>> > • new data type VARIANT >>>>> > • STRING collation support >>>>> > • Spark k8s operator versioning >>>>> > Please help to add more items to this list that are missed here. I >>>>> would like to volunteer as the release manager for Apache Spark 4.0.0 if >>>>> there is no objection. Thank you all for the great work that fills Spark >>>>> 4.0! >>>>> > >>>>> > Wenchen Fan >>>>> >>>>>
