+1 发件人: Jungtaek Lim <[email protected]> 日期: 2024年5月2日 星期四 10:21 收件人: Holden Karau <[email protected]> 抄送: Chao Sun <[email protected]>, Xiao Li <[email protected]>, Tathagata Das <[email protected]>, Wenchen Fan <[email protected]>, Cheng Pan <[email protected]>, Nicholas Chammas <[email protected]>, Dongjoon Hyun <[email protected]>, Cheng Pan <[email protected]>, Spark dev list <[email protected]>, Anish Shrigondekar <[email protected]> 主题: Re: [DISCUSS] Spark 4.0.0 release
+1 love to see it! On Thu, May 2, 2024 at 10:08 AM Holden Karau <[email protected]<mailto:[email protected]>> wrote: +1 :) yay previews On Wed, May 1, 2024 at 5:36 PM Chao Sun <[email protected]<mailto:[email protected]>> wrote: +1 On Wed, May 1, 2024 at 5:23 PM Xiao Li <[email protected]<mailto:[email protected]>> wrote: +1 for next Monday. We can do more previews when the other features are ready for preview. Tathagata Das <[email protected]<mailto:[email protected]>> 于2024年5月1日周三 08:46写道： Next week sounds great! Thank you Wenchen! On Wed, May 1, 2024 at 11:16 AM Wenchen Fan <[email protected]<mailto:[email protected]>> wrote: Yea I think a preview release won't hurt (without a branch cut). We don't need to wait for all the ongoing projects to be ready. How about we do a 4.0 preview release based on the current master branch next Monday? On Wed, May 1, 2024 at 11:06 PM Tathagata Das <[email protected]<mailto:[email protected]>> wrote: Hey all, Reviving this thread, but Spark master has already accumulated a huge amount of changes. As a downstream project maintainer, I want to really start testing the new features and other breaking changes, and it's hard to do that without a Preview release. So the sooner we make a Preview release, the faster we can start getting feedback for fixing things for a great Spark 4.0 final release. So I urge the community to produce a Spark 4.0 Preview soon even if certain features targeting the Delta 4.0 release are still incomplete. Thanks! On Wed, Apr 17, 2024 at 8:35 AM Wenchen Fan <[email protected]<mailto:[email protected]>> wrote: Thank you all for the replies! To @Nicholas Chammas<mailto:[email protected]> : Thanks for cleaning up the error terminology and documentation! I've merged the first PR and let's finish others before the 4.0 release. To @Dongjoon Hyun<mailto:[email protected]> : Thanks for driving the ANSI on by default effort! Now the vote has passed, let's flip the config and finish the DataFrame error context feature before 4.0. To @Jungtaek Lim<mailto:[email protected]> : Ack. We can treat the Streaming state store data source as completed for 4.0 then. To @Cheng Pan<mailto:[email protected]> : Yea we definitely should have a preview release. Let's collect more feedback on the ongoing projects and then we can propose a date for the preview release. On Wed, Apr 17, 2024 at 1:22 PM Cheng Pan <[email protected]<mailto:[email protected]>> wrote: will we have preview release for 4.0.0 like we did for 2.0.0 and 3.0.0? Thanks, Cheng Pan > On Apr 15, 2024, at 09:58, Jungtaek Lim > <[email protected]<mailto:[email protected]>> wrote: > > W.r.t. state data source - reader (SPARK-45511), there are several follow-up > tickets, but we don't plan to address them soon. The current implementation > is the final shape for Spark 4.0.0, unless there are demands on the follow-up > tickets. > > We may want to check the plan for transformWithState - my understanding is > that we want to release the feature to 4.0.0, but there are several remaining > works to be done. While the tentative timeline for releasing is June 2024, > what would be the tentative timeline for the RC cut? > (cc. Anish to add more context on the plan for transformWithState) > > On Sat, Apr 13, 2024 at 3:15 AM Wenchen Fan > <[email protected]<mailto:[email protected]>> wrote: > Hi all, > > It's close to the previously proposed 4.0.0 release date (June 2024), and I > think it's time to prepare for it and discuss the ongoing projects: > • > ANSI by default > • Spark Connect GA > • Structured Logging > • Streaming state store data source > • new data type VARIANT > • STRING collation support > • Spark k8s operator versioning > Please help to add more items to this list that are missed here. I would like > to volunteer as the release manager for Apache Spark 4.0.0 if there is no > objection. Thank you all for the great work that fills Spark 4.0! > > Wenchen Fan -- Twitter: https://twitter.com/holdenkarau<https://mailshield.baidu.com/check?q=9DewFnOIsK%2bK64Uu60Jx4QkcL9rDgnApD6spzOBjk%2fa2KQxn> Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 <https://mailshield.baidu.com/check?q=D34Ozfkj%2bFrnkuu9ci%2b4FcMkreOvMZ3jO85bIw%3d%3d> YouTube Live Streams: https://www.youtube.com/user/holdenkarau<https://mailshield.baidu.com/check?q=nadOZCZjNeU0qOVGCJesf8dvH4OrsWdKamKIxnJncPneWoN8%2bsIqc2DWow8%3d>
