+1 :) yay previews On Wed, May 1, 2024 at 5:36 PM Chao Sun <sunc...@apache.org> wrote:
> +1 > > On Wed, May 1, 2024 at 5:23 PM Xiao Li <gatorsm...@gmail.com> wrote: > >> +1 for next Monday. >> >> We can do more previews when the other features are ready for preview. >> >> Tathagata Das <tathagata.das1...@gmail.com> 于2024年5月1日周三 08:46写道： >> >>> Next week sounds great! Thank you Wenchen! >>> >>> On Wed, May 1, 2024 at 11:16 AM Wenchen Fan <cloud0...@gmail.com> wrote: >>> >>>> Yea I think a preview release won't hurt (without a branch cut). We >>>> don't need to wait for all the ongoing projects to be ready. How about we >>>> do a 4.0 preview release based on the current master branch next Monday? >>>> >>>> On Wed, May 1, 2024 at 11:06 PM Tathagata Das < >>>> tathagata.das1...@gmail.com> wrote: >>>> >>>>> Hey all, >>>>> >>>>> Reviving this thread, but Spark master has already accumulated a huge >>>>> amount of changes. As a downstream project maintainer, I want to really >>>>> start testing the new features and other breaking changes, and it's hard >>>>> to >>>>> do that without a Preview release. So the sooner we make a Preview >>>>> release, >>>>> the faster we can start getting feedback for fixing things for a great >>>>> Spark 4.0 final release. >>>>> >>>>> So I urge the community to produce a Spark 4.0 Preview soon even if >>>>> certain features targeting the Delta 4.0 release are still incomplete. >>>>> >>>>> Thanks! >>>>> >>>>> >>>>> On Wed, Apr 17, 2024 at 8:35 AM Wenchen Fan <cloud0...@gmail.com> >>>>> wrote: >>>>> >>>>>> Thank you all for the replies! >>>>>> >>>>>> To @Nicholas Chammas <nicholas.cham...@gmail.com> : Thanks for >>>>>> cleaning up the error terminology and documentation! I've merged the >>>>>> first >>>>>> PR and let's finish others before the 4.0 release. >>>>>> To @Dongjoon Hyun <dongjoon.h...@gmail.com> : Thanks for driving the >>>>>> ANSI on by default effort! Now the vote has passed, let's flip the config >>>>>> and finish the DataFrame error context feature before 4.0. >>>>>> To @Jungtaek Lim <kabhwan.opensou...@gmail.com> : Ack. We can treat >>>>>> the Streaming state store data source as completed for 4.0 then. >>>>>> To @Cheng Pan <cheng...@apache.org> : Yea we definitely should have >>>>>> a preview release. Let's collect more feedback on the ongoing projects >>>>>> and >>>>>> then we can propose a date for the preview release. >>>>>> >>>>>> On Wed, Apr 17, 2024 at 1:22 PM Cheng Pan <pan3...@gmail.com> wrote: >>>>>> >>>>>>> will we have preview release for 4.0.0 like we did for 2.0.0 and >>>>>>> 3.0.0? >>>>>>> >>>>>>> Thanks, >>>>>>> Cheng Pan >>>>>>> >>>>>>> >>>>>>> > On Apr 15, 2024, at 09:58, Jungtaek Lim < >>>>>>> kabhwan.opensou...@gmail.com> wrote: >>>>>>> > >>>>>>> > W.r.t. state data source - reader (SPARK-45511), there are several >>>>>>> follow-up tickets, but we don't plan to address them soon. The current >>>>>>> implementation is the final shape for Spark 4.0.0, unless there are >>>>>>> demands >>>>>>> on the follow-up tickets. >>>>>>> > >>>>>>> > We may want to check the plan for transformWithState - my >>>>>>> understanding is that we want to release the feature to 4.0.0, but there >>>>>>> are several remaining works to be done. While the tentative timeline for >>>>>>> releasing is June 2024, what would be the tentative timeline for the RC >>>>>>> cut? >>>>>>> > (cc. Anish to add more context on the plan for transformWithState) >>>>>>> > >>>>>>> > On Sat, Apr 13, 2024 at 3:15 AM Wenchen Fan <cloud0...@gmail.com> >>>>>>> wrote: >>>>>>> > Hi all, >>>>>>> > >>>>>>> > It's close to the previously proposed 4.0.0 release date (June >>>>>>> 2024), and I think it's time to prepare for it and discuss the ongoing >>>>>>> projects: >>>>>>> > • >>>>>>> > ANSI by default >>>>>>> > • Spark Connect GA >>>>>>> > • Structured Logging >>>>>>> > • Streaming state store data source >>>>>>> > • new data type VARIANT >>>>>>> > • STRING collation support >>>>>>> > • Spark k8s operator versioning >>>>>>> > Please help to add more items to this list that are missed here. I >>>>>>> would like to volunteer as the release manager for Apache Spark 4.0.0 if >>>>>>> there is no objection. Thank you all for the great work that fills Spark >>>>>>> 4.0! >>>>>>> > >>>>>>> > Wenchen Fan >>>>>>> >>>>>>> -- Twitter: https://twitter.com/holdenkarau Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
