Hyukjin Kwon created SPARK-61394:
------------------------------------

             Summary: Reject a bool or out-of-range thresh in dropna as PySpark 
does, and name VariantVal in the error of a pivot value
                 Key: SPARK-61394
                 URL: https://issues.apache.org/jira/browse/SPARK-61394
             Project: Spark
          Issue Type: Bug
          Components: Connect
    Affects Versions: connect-rust-4.3.0
            Reporter: Hyukjin Kwon


A follow-up of SPARK-61320, found in a post-merge review. 
df.dropna(thresh=True) is accepted as thresh=1, where PySpark raises TypeError, 
and thresh=2**40 raises OverflowError, where PySpark raises ValueError (Value 
out of range). The Rust API's pivot error names a Variant value 'Variant', not 
PySpark's 'VariantVal', and a decimal pivot value with an exponent of i32::MIN 
overflows.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to