Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/5189#issuecomment-86685120
@cloud-fan, thanks for the further comments. Regarding not resolving sort
in in `ResolveReferences`, I agree that is a little confusing and I think you
are correct for standard SQL. However, this is not how hive does resolution
and I don't think that we can differ from their implementation. Consider the
following:
```
test("hive orderby queries") {
val df = jsonRDD(sparkContext.makeRDD(
"""{"a": {"b": 1}, "b": 2}""" ::
"""{"a": {"b": 2}, "b": 1}""" :: Nil))registerTempTable("t")
sql("CREATE TABLE t1 AS SELECT * FROM t")
runSqlHive("SELECT a.b FROM t1 ORDER BY b").foreach(println)
sql("SELECT a.b FROM t1 ORDER BY b").collect().foreach(println)
}
```
Which outputs:
```
1
2
[1]
[2]
```
So given this, I think the current way we resolve things (i.e. first
resolve based on what comes from the select clause and then fall back on its
child only when this fails) is non-standard but what we must do to avoid
breaking existing HiveQL queries.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
