Ya, it sounds like that. Could you link those items to the following JIRA?
https://issues.apache.org/jira/browse/SPARK-44111 Prepare Apache Spark 4.0.0
Dongjoon.
On Tue, Jun 20, 2023 at 12:45 PM Holden Karau wrote:
> That seems like a really good reason for a major version change given the
That seems like a really good reason for a major version change given the %
of PySpark users and the fact we are (effectively) tied to pandas APIs.
On Tue, Jun 20, 2023 at 12:24 PM Bjørn Jørgensen
wrote:
> One big thing for 4.0 will be that pandas API on spark will support pandas
> version 2.0
One big thing for 4.0 will be that pandas API on spark will support pandas
version 2.0
With the major release of pandas 2.0.0 on April 3, 2023, numerous breaking
changes have been introduced. So, we have made the decision to postpone
addressing these breaking changes until the next major release
Hi, Herman.
This is a series of discussions as I re-summarized here.
You can find some context in the previous timeline thread.
2023-05-30 Apache Spark 4.0 Timeframe?
https://lists.apache.org/thread/xhkgj60j361gdpywoxxz7qspp2w80ry6
Could you reply there to collect your timeline suggestions? We
Dongjoon, I am not sure if I am not sure if I follow the line of thought
here.
Multiple people have asked for clarification on what Spark 4.0 would mean
(Holden, Mridul, Jia & Xiao). You can - for the record - also add me to
this list. However you choose to single out Xiao because asks this
Thank you. I reviewed the threads, vote and result once more.
I found that I missed the binding vote mark on Holden in the vote result email.
The following should be "-0: Holden Karau *". Sorry for this mistake, Holden
and all.
> -0: Holden Karau
To Hyukjin, I disagree with you at the
The major concerns raised in the thread were that we should initiate the
discussion for the below first:
- Apache Spark 4.0.0 Preview (and Dates)
- Apache Spark 4.0.0 Items
- Apache Spark 4.0.0 Plan Adjustment
before setting the timeline for Spark 4.0.0 because we're unclear on the
picture of
This was a part of the following on-going discussions.
2023-05-28 Apache Spark 3.5.0 Expectations (?)
https://lists.apache.org/thread/3x6dh17bmy20n3frtt3crgxjydnxh2o0
2023-05-30 Apache Spark 4.0 Timeframe?
https://lists.apache.org/thread/xhkgj60j361gdpywoxxz7qspp2w80ry6
2023-06-05 ASF policy
The vote passes with 6 +1s (4 binding +1s), one -0, and one -1.
Thank you all for your participation and
especially your additional comments during this voting,
Mridul, Hyukjin, and Jungtaek.
(* = binding)
+1:
- Dongjoon Hyun *
- Huaxin Gao *
- Liang-Chi Hsieh *
- Kazuyuki Tanimura
- Chao Sun *
-
