Maciej Szymkiewicz created SPARK-33257:
------------------------------------------
Summary: Support Column inputs in PySpark ordering functions
(asc*, desc*)
Key: SPARK-33257
URL: https://issues.apache.org/jira/browse/SPARK-33257
Project: Spark
Issue Type: Bug
Components: PySpark, SQL
Affects Versions: 3.1.0
Reporter: Maciej Szymkiewicz
According to SPARK-26979, PySpark functions should support both {{Column}} and
{{str}} arguments, when possible.
However, the following ordering support only {{str}}
- {{asc}}
- {{desc}}
- {{asc_nulls_first}}
- {{asc_nulls_last}}
- {{desc_nulls_first}}
- {{desc_nulls_last}}
support only {{str}}. This is because Scala side doesn't provide {{Column =>
Column}} variants.
To fix this, we do one of the following:
- Call corresponding {{Column}} methods as
[suggested|https://github.com/apache/spark/pull/30143#discussion_r512366978] by
[~hyukjin.kwon]
- Add missing signatures on Scala side.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
