Ross Bowen created SPARK-31517:
----------------------------------
Summary: orderBy with multiple columns descending does not work
properly
Key: SPARK-31517
URL: https://issues.apache.org/jira/browse/SPARK-31517
Project: Spark
Issue Type: Bug
Components: SparkR
Affects Versions: 2.4.5
Environment: Databricks Runtime 6.5
Reporter: Ross Bowen
When specifying two columns within an `orderBy()` function, to attempt to get
an ordering by two columns in descending order, an error is returned.
{code:java}
library(magrittr) library(SparkR) cars <- cbind(model = rownames(mtcars),
mtcars) carsDF <- createDataFrame(cars) carsDF %>% mutate(rank = over(rank(),
orderBy(windowPartitionBy(column("cyl")), desc(column("mpg")),
desc(column("disp"))))) %>% head() {code}
This returns an error:
{code:java}
Error in ns[[i]] : subscript out of bounds{code}
This seems to be related to the more general issue that the following code
(excluding the use of the `desc()` function) also fails:
{code:java}
carsDF %>% mutate(rank = over(rank(),
orderBy(windowPartitionBy(column("cyl")), column("mpg"), column("disp")))) %>%
head(){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
