Kiyeon Jeon created SPARK-56452:
-----------------------------------
Summary: pip install fails for prerelease versions due to strict
version regex in install.py
Key: SPARK-56452
URL: https://issues.apache.org/jira/browse/SPARK-56452
Project: Spark
Issue Type: Bug
Components: PySpark
Affects Versions: 4.2.0
Reporter: Kiyeon Jeon
When installing a PySpark prerelease via pip (e.g. version "4.2.0.dev4"), the
version validation regex in python/pyspark/install.py rejects the version
string because it doesn't account for ".dev" suffixes.
The regex at line 71 only matches the X.Y.Z format:
{code}
^[0-9]+\.[0-9]+\.[0-9]+$
{code}
This causes a RuntimeError:
{code}
Spark version should start with 'spark-' prefix; however, got 4.2.0.dev4
{code}
A similar regex at line 109 (convert_old_hadoop_version) has the same issue.
The fix is to add an optional ".devN" suffix to both regexes:
{code}
^[0-9]+\.[0-9]+\.[0-9]+(?:\.dev[0-9]+)?$
{code}
GitHub issue: https://github.com/apache/spark/issues/55289
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
