[
https://issues.apache.org/jira/browse/SPARK-35176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17329971#comment-17329971
]
Yikun Jiang commented on SPARK-35176:
-------------------------------------
I write up a POC in [https://github.com/Yikun/annotation-type-checker/pull/4]
to add some simple way to do runtime type checker.
> Raise TypeError in inappropriate type case rather than ValueError
> ------------------------------------------------------------------
>
> Key: SPARK-35176
> URL: https://issues.apache.org/jira/browse/SPARK-35176
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 3.2.0
> Reporter: Yikun Jiang
> Priority: Minor
>
> There are many wrong error type usages on ValueError type.
> When an operation or function is applied to an object of inappropriate type,
> we should use TypeError rather than ValueError.
> such as:
> [https://github.com/apache/spark/blob/355c39939d9e4c87ffc9538eb822a41cb2ff93fb/python/pyspark/sql/dataframe.py#L1137]
> [https://github.com/apache/spark/blob/355c39939d9e4c87ffc9538eb822a41cb2ff93fb/python/pyspark/sql/dataframe.py#L1228]
>
> We should do some correction in some right time, note that if we do these
> corrections, it will break some catch on original ValueError.
>
> [1] https://docs.python.org/3/library/exceptions.html#TypeError
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
