dongjoon-hyun commented on PR #59756: URL: https://github.com/apache/spark/pull/59756#issuecomment-6560951714
Given that new improvements are going to be available only Apache Spark 4.4 or 5. Why don't you try to use the latest ones first in your production environment? FYI, here are the list we went to discuss for a fair comparison. Since Apache Spark 4.0 is going to reach EOL in three weeks (2025-11-23), I didn't include here. - [Spark 4.3.0](https://spark.apache.org/releases/spark-release-4-3-0.html) (Oct 30 2026) - [Spark 4.2.1](https://spark.apache.org/releases/spark-release-4-2-1.html) (Oct 27 2026) - [Spark 4.1.4](https://spark.apache.org/releases/spark-release-4-1-4.html) (Oct 27 2026) Technically, Spark 3.5 (2023-09-13) sounds like too old to discuss together and the major version is also different. We don't consider it as a fair comparison target in the community discussion, @TongWei1105 . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
