Hi Holden, Thank you for your passion and for keeping this thread up to date. I truly appreciate it.
This situation is a bit tricky, as it exponentially increases the maintenance burden for the following reasons: (1) Legacy Tech Stack Support: This would require us to support Java 8, Scala 2.12, Python 3.8, and SparkR. For instance, I don’t believe we can maintain PySpark 3.5 on Python 3.8 at our previously promised service level. We would definitely need to downgrade the support level, and this must be made crystal clear to the community. (2) Impact on Subprojects: It adds a significant burden to all Apache Spark subprojects, such as the Apache K8s Operator and Spark Connect clients (Swift/Go/Rust). For example, while Spark 3.5 must support K8s 1.24+, Spark 4.0 targets K8s 1.30+. The extended period effectively locks us into supporting older K8s versions that are no longer available from major public cloud vendors. (3) Ecosystem-wide Pressure: This also places constraints on Spark-related ASF sister projects like Iceberg, Parquet, ORC, Celeborn, Gluten, and Comet. While the final decision rests with them, I believe they need to be informed of our strategic direction. To provide extended community support responsibly, we need at least three clearly defined components: (a) A predefined extension period (Confirmed) (b) A predefined pool of Release Managers (Holden has volunteered) (c) A predefined release cadence (Currently unknown as discussed so far) Regarding (c), I believe we can keep it simple, similar to the Apache Spark 4.x preview releases. Currently, Hyukjin has successfully managed all previews by leveraging automation, with the only exception being 4.1.0-preview4, which I assisted with. For 3.5.x, we should maintain the same cadence as before. Perhaps Holden could manage a release every four months if there are new patches on branch-3.5. Of course, other community members can assist as well. Sincerely, Dongjoon Hyun On 2026/02/13 19:27:48 Holden Karau wrote: > I wasn’t super sure about that, I know the initial proposal was a little > sparse for some so I figured I’d make it a bit more formal and follow the > SPIP process like we did with the overall release process change. > > Twitter: https://twitter.com/holdenkarau > Fight Health Insurance: https://www.fighthealthinsurance.com/ > <https://www.fighthealthinsurance.com/?q=hk_email> > Books (Learning Spark, High Performance Spark, etc.): > https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau > Pronouns: she/her > > > On Thu, Feb 12, 2026 at 11:58 PM Wenchen Fan <[email protected]> wrote: > > > This is a procedure change, not a code change, do we need SPIP? Anyway, +1 > > to the proposal! > > > > On Thu, Feb 12, 2026 at 6:28 AM Holden Karau <[email protected]> > > wrote: > > > >> This is a more formal (but still a discussion level thread) proposing an > >> extended 3.5 LTS window to allow for people to complete their migrations > >> given the less than one year between 4.0 release and 3.5 EOL. > >> > >> The draft SPIP is > >> https://docs.google.com/document/d/15fb-7DNztzpQMF1MQovK75UmpC6um7KgO8cg_X5oa1I/edit?usp=sharing > >> and the JIRA SPARK-55489 > >> <https://issues.apache.org/jira/browse/SPARK-55489> > >> > >> Previous discussion was: > >> https://lists.apache.org/thread/4r6q8187b30p0ppclw0pyfjp8h8xs3rq > >> > >> Cheers, > >> > >> Holden :) > >> > >> -- > >> Twitter: https://twitter.com/holdenkarau > >> Fight Health Insurance: https://www.fighthealthinsurance.com/ > >> <https://www.fighthealthinsurance.com/?q=hk_email> > >> Books (Learning Spark, High Performance Spark, etc.): > >> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> > >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau > >> Pronouns: she/her > >> > > > --------------------------------------------------------------------- To unsubscribe e-mail: [email protected]
