+1 发件人: Jungtaek Lim <kabhwan.opensou...@gmail.com> 日期: 2024年5月2日 星期四 10:21 收件人: Holden Karau <holden.ka...@gmail.com> 抄送: Chao Sun <sunc...@apache.org>, Xiao Li <gatorsm...@gmail.com>, Tathagata Das <tathagata.das1...@gmail.com>, Wenchen Fan <cloud0...@gmail.com>, Cheng Pan <pan3...@gmail.com>, Nicholas Chammas <nicholas.cham...@gmail.com>, Dongjoon Hyun <dongjoon.h...@gmail.com>, Cheng Pan <cheng...@apache.org>, Spark dev list <dev@spark.apache.org>, Anish Shrigondekar <anish.shrigonde...@databricks.com> 主题: Re: [DISCUSS] Spark 4.0.0 release
+1 love to see it! On Thu, May 2, 2024 at 10:08 AM Holden Karau <holden.ka...@gmail.com<mailto:holden.ka...@gmail.com>> wrote: +1 :) yay previews On Wed, May 1, 2024 at 5:36 PM Chao Sun <sunc...@apache.org<mailto:sunc...@apache.org>> wrote: +1 On Wed, May 1, 2024 at 5:23 PM Xiao Li <gatorsm...@gmail.com<mailto:gatorsm...@gmail.com>> wrote: +1 for next Monday. We can do more previews when the other features are ready for preview. Tathagata Das <tathagata.das1...@gmail.com<mailto:tathagata.das1...@gmail.com>> 于2024年5月1日周三 08:46写道： Next week sounds great! Thank you Wenchen! On Wed, May 1, 2024 at 11:16 AM Wenchen Fan <cloud0...@gmail.com<mailto:cloud0...@gmail.com>> wrote: Yea I think a preview release won't hurt (without a branch cut). We don't need to wait for all the ongoing projects to be ready. How about we do a 4.0 preview release based on the current master branch next Monday? On Wed, May 1, 2024 at 11:06 PM Tathagata Das <tathagata.das1...@gmail.com<mailto:tathagata.das1...@gmail.com>> wrote: Hey all, Reviving this thread, but Spark master has already accumulated a huge amount of changes. As a downstream project maintainer, I want to really start testing the new features and other breaking changes, and it's hard to do that without a Preview release. So the sooner we make a Preview release, the faster we can start getting feedback for fixing things for a great Spark 4.0 final release. So I urge the community to produce a Spark 4.0 Preview soon even if certain features targeting the Delta 4.0 release are still incomplete. Thanks! On Wed, Apr 17, 2024 at 8:35 AM Wenchen Fan <cloud0...@gmail.com<mailto:cloud0...@gmail.com>> wrote: Thank you all for the replies! To @Nicholas Chammas<mailto:nicholas.cham...@gmail.com> : Thanks for cleaning up the error terminology and documentation! I've merged the first PR and let's finish others before the 4.0 release. To @Dongjoon Hyun<mailto:dongjoon.h...@gmail.com> : Thanks for driving the ANSI on by default effort! Now the vote has passed, let's flip the config and finish the DataFrame error context feature before 4.0. To @Jungtaek Lim<mailto:kabhwan.opensou...@gmail.com> : Ack. We can treat the Streaming state store data source as completed for 4.0 then. To @Cheng Pan<mailto:cheng...@apache.org> : Yea we definitely should have a preview release. Let's collect more feedback on the ongoing projects and then we can propose a date for the preview release. On Wed, Apr 17, 2024 at 1:22 PM Cheng Pan <pan3...@gmail.com<mailto:pan3...@gmail.com>> wrote: will we have preview release for 4.0.0 like we did for 2.0.0 and 3.0.0? Thanks, Cheng Pan > On Apr 15, 2024, at 09:58, Jungtaek Lim > <kabhwan.opensou...@gmail.com<mailto:kabhwan.opensou...@gmail.com>> wrote: > > W.r.t. state data source - reader (SPARK-45511), there are several follow-up > tickets, but we don't plan to address them soon. The current implementation > is the final shape for Spark 4.0.0, unless there are demands on the follow-up > tickets. > > We may want to check the plan for transformWithState - my understanding is > that we want to release the feature to 4.0.0, but there are several remaining > works to be done. While the tentative timeline for releasing is June 2024, > what would be the tentative timeline for the RC cut? > (cc. Anish to add more context on the plan for transformWithState) > > On Sat, Apr 13, 2024 at 3:15 AM Wenchen Fan > <cloud0...@gmail.com<mailto:cloud0...@gmail.com>> wrote: > Hi all, > > It's close to the previously proposed 4.0.0 release date (June 2024), and I > think it's time to prepare for it and discuss the ongoing projects: > • > ANSI by default > • Spark Connect GA > • Structured Logging > • Streaming state store data source > • new data type VARIANT > • STRING collation support > • Spark k8s operator versioning > Please help to add more items to this list that are missed here. I would like > to volunteer as the release manager for Apache Spark 4.0.0 if there is no > objection. Thank you all for the great work that fills Spark 4.0! > > Wenchen Fan -- Twitter: https://twitter.com/holdenkarau<https://mailshield.baidu.com/check?q=9DewFnOIsK%2bK64Uu60Jx4QkcL9rDgnApD6spzOBjk%2fa2KQxn> Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 <https://mailshield.baidu.com/check?q=D34Ozfkj%2bFrnkuu9ci%2b4FcMkreOvMZ3jO85bIw%3d%3d> YouTube Live Streams: https://www.youtube.com/user/holdenkarau<https://mailshield.baidu.com/check?q=nadOZCZjNeU0qOVGCJesf8dvH4OrsWdKamKIxnJncPneWoN8%2bsIqc2DWow8%3d>
