Hyukjin Kwon created SPARK-46397:
------------------------------------
Summary: sha2(df.a, 1024) throws a different exception in Spark
Connect
Key: SPARK-46397
URL: https://issues.apache.org/jira/browse/SPARK-46397
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: 4.0.0
Reporter: Hyukjin Kwon
{code}
from pyspark.sql import functions as sf
spark.range(1).select(sf.sha2(sf.col("id"), 1024)).collect()
{code}
Non-connect:
{code}
...
pyspark.errors.exceptions.captured.IllegalArgumentException: requirement
failed: numBits 1024 is not in the permitted values (0, 224, 256, 384, 512)
{code}
Connect:
{code}
...
pyspark.errors.exceptions.connect.AnalysisException:
[DATATYPE_MISMATCH.UNEXPECTED_INPUT_TYPE] Cannot resolve "sha2(id, 1024)" due
to data type mismatch: Parameter 1 requires the "BINARY" type, however "id" has
the type "BIGINT". SQLSTATE: 42K09;
'Project [unresolvedalias(sha2(id#1L, 1024))]
+- Range (0, 1, step=1, splits=Some(1))
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
