[ https://issues.apache.org/jira/browse/SPARK-42547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon updated SPARK-42547: --------------------------------- Priority: Blocker (was: Major) > Make PySpark working with Python 3.7 > ------------------------------------ > > Key: SPARK-42547 > URL: https://issues.apache.org/jira/browse/SPARK-42547 > Project: Spark > Issue Type: Bug > Components: PySpark > Affects Versions: 3.4.0 > Reporter: Hyukjin Kwon > Priority: Blocker > > {code} > + ./python/run-tests --python-executables=python3 > Running PySpark tests. Output is in /home/ec2-user/spark/python/unit-tests.log > Will test against the following Python executables: ['python3'] > Will test the following Python modules: ['pyspark-connect', 'pyspark-core', > 'pyspark-errors', 'pyspark-ml', 'pyspark-mllib', 'pyspark-pandas', > 'pyspark-pandas-slow', 'pyspark-resource', 'pyspark-sql', 'pyspark-streaming'] > python3 python_implementation is CPython > python3 version is: Python 3.7.16 > Starting test(python3): pyspark.ml.tests.test_feature (temp output: > /home/ec2-user/spark/python/target/8ca9ab1a-05cc-4845-bf89-30d9001510bc/python3__pyspark.ml.tests.test_feature__kg6sseie.log) > Starting test(python3): pyspark.ml.tests.test_base (temp output: > /home/ec2-user/spark/python/target/f2264f3b-6b26-4e61-9452-8d6ddd7eb002/python3__pyspark.ml.tests.test_base__0902zf9_.log) > Starting test(python3): pyspark.ml.tests.test_algorithms (temp output: > /home/ec2-user/spark/python/target/d1dc4e07-e58c-4c03-abe5-09d8fab22e6a/python3__pyspark.ml.tests.test_algorithms__lh3wb2u8.log) > Starting test(python3): pyspark.ml.tests.test_evaluation (temp output: > /home/ec2-user/spark/python/target/3f42dc79-c945-4cf2-a1eb-83e72b40a9ee/python3__pyspark.ml.tests.test_evaluation__89idc7fa.log) > Finished test(python3): pyspark.ml.tests.test_base (16s) > Starting test(python3): pyspark.ml.tests.test_functions (temp output: > /home/ec2-user/spark/python/target/5a3b90f0-216b-4edd-9d15-6619d3e03300/python3__pyspark.ml.tests.test_functions__g5u1290s.log) > Traceback (most recent call last): > File "/usr/lib64/python3.7/runpy.py", line 193, in _run_module_as_main > "__main__", mod_spec) > File "/usr/lib64/python3.7/runpy.py", line 85, in _run_code > exec(code, run_globals) > File "/home/ec2-user/spark/python/pyspark/ml/tests/test_functions.py", line > 21, in <module> > from pyspark.ml.functions import predict_batch_udf > File "/home/ec2-user/spark/python/pyspark/ml/functions.py", line 38, in > <module> > from typing import Any, Callable, Iterator, List, Mapping, Protocol, > TYPE_CHECKING, Tuple, Union > ImportError: cannot import name 'Protocol' from 'typing' > (/usr/lib64/python3.7/typing.py) > Had test failures in pyspark.ml.tests.test_functions with python3; see logs. > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org