[
https://issues.apache.org/jira/browse/SPARK-18455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15683798#comment-15683798
]
Nattavut Sutyanyong commented on SPARK-18455:
---------------------------------------------
Incorrect results problem
> General support for subquery processing
> ---------------------------------------
>
> Key: SPARK-18455
> URL: https://issues.apache.org/jira/browse/SPARK-18455
> Project: Spark
> Issue Type: Story
> Components: SQL
> Reporter: Nattavut Sutyanyong
>
> Subquery support has been introduced in Spark 2.0. The initial implementation
> covers the most common subquery use case: the ones used in TPC queries for
> instance.
> Spark currently supports the following subqueries:
> * Uncorrelated Scalar Subqueries. All cases are supported.
> * Correlated Scalar Subqueries. We only allow subqueries that are aggregated
> and use equality predicates.
> * Predicate Subqueries. IN or Exists type of queries. We allow most
> predicates, except when they are pulled from under an Aggregate or Window
> operator. In that case we only support equality predicates.
> However this does not cover the full range of possible subqueries. This, in
> part, has to do with the fact that we currently rewrite all correlated
> subqueries into a (LEFT/LEFT SEMI/LEFT ANTI) join.
> We currently lack supports for the following use cases:
> * The use of predicate subqueries in a projection.
> * The use of non-equality predicates below Aggregates and or Window operators.
> * The use of non-Aggregate subqueries for correlated scalar subqueries.
> This JIRA aims to lift these current limitations in subquery processing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
