[
https://issues.apache.org/jira/browse/SPARK-32797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386524#comment-17386524
]
Shane Knapp commented on SPARK-32797:
-------------------------------------
i've temporarily disabled mypy on the workers:
for x in `cat ~/txt/jenkins-workers.txt`; do ssh jenkins@${x} "mv
/home/jenkins/anaconda2/envs/py36/bin/mypy
/home/jenkins/anaconda2/envs/py36/bin/mypy.bak"; done
the files that need fixing are here:
{noformat}
starting mypy test...
mypy checks failed:
python/pyspark/pandas/data_type_ops/categorical_ops.py:128: error: Function is
missing a type annotation
python/pyspark/mllib/tree.pyi:29: error: Overloaded function signatures 1 and 2
overlap with incompatible return types
python/pyspark/mllib/tree.pyi:38: error: Overloaded function signatures 1 and 2
overlap with incompatible return types
python/pyspark/mllib/feature.pyi:34: error: Overloaded function signatures 1
and 2 overlap with incompatible return types
python/pyspark/mllib/feature.pyi:42: error: Overloaded function signatures 1
and 2 overlap with incompatible return types
python/pyspark/mllib/feature.pyi:48: error: Overloaded function signatures 1
and 2 overlap with incompatible return types
python/pyspark/mllib/feature.pyi:54: error: Overloaded function signatures 1
and 2 overlap with incompatible return types
python/pyspark/mllib/feature.pyi:76: error: Overloaded function signatures 1
and 2 overlap with incompatible return types
python/pyspark/mllib/feature.pyi:124: error: Overloaded function signatures 1
and 2 overlap with incompatible return types
python/pyspark/mllib/feature.pyi:165: error: Overloaded function signatures 1
and 2 overlap with incompatible return types
python/pyspark/mllib/clustering.pyi:45: error: Overloaded function signatures 1
and 2 overlap with incompatible return types
python/pyspark/mllib/clustering.pyi:72: error: Overloaded function signatures 1
and 2 overlap with incompatible return types
python/pyspark/mllib/classification.pyi:39: error: Overloaded function
signatures 1 and 2 overlap with incompatible return types
python/pyspark/mllib/classification.pyi:52: error: Overloaded function
signatures 1 and 2 overlap with incompatible return types
{noformat}
could someone create a PR that addresses these, and once that's done i'll
re-enable mypy on the workers and we'll coordinate a quick test.
> Install mypy on the Jenkins CI workers
> --------------------------------------
>
> Key: SPARK-32797
> URL: https://issues.apache.org/jira/browse/SPARK-32797
> Project: Spark
> Issue Type: Improvement
> Components: jenkins, PySpark
> Affects Versions: 3.1.0
> Reporter: Fokko Driesprong
> Assignee: Shane Knapp
> Priority: Major
> Fix For: 3.3.0
>
>
> We want to check the types of the PySpark code. This requires mypy to be
> installed on the CI. Can you do this [~shaneknapp]?
> Related PR: [https://github.com/apache/spark/pull/29180]
> You can install this using pip: [https://pypi.org/project/mypy/] Should be
> similar to flake8 and sphinx. The latest version is ok! Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
