[
https://issues.apache.org/jira/browse/SPARK-37683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17461881#comment-17461881
]
Maciej Szymkiewicz commented on SPARK-37683:
--------------------------------------------
cc [~hyukjin.kwon] FYI
> Nondeterministic type checker error in data tests
> -------------------------------------------------
>
> Key: SPARK-37683
> URL: https://issues.apache.org/jira/browse/SPARK-37683
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 3.3.0
> Reporter: Maciej Szymkiewicz
> Priority: Major
>
> Occasionally, I see different errors around {{_jvm}}, {{_wrapped}} and
> {{_jsparkSession}} popping out in data tests
> {code}
> E ../../spark/python/pyspark/sql/catalog:71: error: Cannot determine type
> of "_jsparkSession" [has-type] (diff)
> E ../../spark/python/pyspark/sql/catalog:72: error: Cannot determine type
> of "_jsparkSession" [has-type] (diff)
> E ../../spark/python/pyspark/sql/catalog:349: error: Cannot determine
> type of "_wrapped" [has-type] (diff)
> E ../../spark/python/pyspark/sql/catalog:360: error: Cannot determine
> type of "_wrapped" [has-type] (diff)
> E ../../spark/python/pyspark/sql/column:920: error: Cannot determine type
> of "_jsparkSession" [has-type] (diff)
> E ../../spark/python/pyspark/sql/pandas/conversion:400: error: Cannot
> determine type of "_wrapped" [has-type] (diff)
> E ../../spark/python/pyspark/sql/pandas/conversion:407: error: Cannot
> determine type of "_wrapped" [has-type] (diff)
> E ../../spark/python/pyspark/sql/pandas/conversion:413: error: Cannot
> determine type of "_wrapped" [has-type] (diff)
> E ../../spark/python/pyspark/sql/pandas/conversion:602: error: Cannot
> determine type of "_wrapped" [has-type] (diff)
> E ../../spark/python/pyspark/sql/pandas/conversion:604: error: Cannot
> determine type of "_wrapped" [has-type] (diff)
> E ../../spark/python/pyspark/sql/pandas/conversion:620: error: Cannot
> determine type of "_jvm" [has-type] (diff)
> E ../../spark/python/pyspark/sql/pandas/conversion:621: error: Cannot
> determine type of "_jvm" [has-type] (diff)
> E ../../spark/python/pyspark/sql/pandas/conversion:622: error: Cannot
> determine type of "_wrapped" [has-type] (diff)
> E ../../spark/python/pyspark/sql/readwriter:115: error: Cannot determine
> type of "_jsparkSession" [has-type] (diff)
> E ../../spark/python/pyspark/sql/readwriter:307: error: Cannot determine
> type of "_jvm" [has-type] (diff)
> E ../../spark/python/pyspark/sql/readwriter:308: error: Cannot determine
> type of "_jvm" [has-type] (diff)
> E ../../spark/python/pyspark/sql/session:283: error: Cannot determine
> type of "_jvm" [has-type] (diff)
> E ../../spark/python/pyspark/sql/session:284: error: Cannot determine
> type of "_jsparkSession" [has-type] (diff)
> E ../../spark/python/pyspark/sql/streaming:377: error: Cannot determine
> type of "_jsparkSession" [has-type] (diff)
> E ../../spark/python/pyspark/sql/udf:225: error: Cannot determine type of
> "_jsparkSession" [has-type] (diff)
> E ../../spark/python/pyspark/sql/udf:456: error: Cannot determine type of
> "_jsparkSession" [has-type] (diff)
> E ../../spark/python/pyspark/sql/udf:509: error: Cannot determine type of
> "_jsparkSession" [has-type] (diff)
> E ../../spark/python/pyspark/sql/udf:510: error: Cannot determine type of
> "_jsparkSession" [has-type] (diff)
> E ../../spark/python/pyspark/sql/udf:533: error: Cannot determine type of
> "_jsparkSession" [has-type] (diff)
> {code}
> The set of errors is not always the same, so it indicates so issues with type
> checker.
> I am going to add {{casts}} and / or annotations for these lines, so we don't
> get flaky tests.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
