[
https://issues.apache.org/jira/browse/SPARK-45390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17786401#comment-17786401
]
Nicholas Chammas commented on SPARK-45390:
------------------------------------------
{quote}We don't promise to support all future unreleased Python versions
{quote}
"all future unreleased versions" is a tall ask that no-one is making. :)
The relevant circumstances here are that a) Python 3.12 is already out and the
backwards-incompatible changes are known and [very
limited|https://docs.python.org/3/whatsnew/3.12.html], and b) Spark 4.0 may be
a disruptive change and so many people may remain on Spark 3.5 for longer than
usual.
If we expect 3.5 -> 4.0 to be an easy migration, then backporting a fix like
this to 3.5 is not as important.
{quote}we need much more validation because all Python package ecosystem should
work there without any issues
{quote}
I'm not sure what you mean here.
Anyway, I suppose we could just wait and see. Maybe I'm wrong, but I suspect
many users will find it surprising that Spark 3.5 doesn't work on Python 3.12,
especially if this is the only (or close to the only) fix required.
> Remove `distutils` usage
> ------------------------
>
> Key: SPARK-45390
> URL: https://issues.apache.org/jira/browse/SPARK-45390
> Project: Spark
> Issue Type: Sub-task
> Components: PySpark
> Affects Versions: 4.0.0
> Reporter: Dongjoon Hyun
> Assignee: Dongjoon Hyun
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
>
> [PEP-632|https://peps.python.org/pep-0632] deprecated {{distutils}} module in
> Python {{3.10}} and dropped in Python {{3.12}} in favor of {{packaging}}
> package.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
