[
https://issues.apache.org/jira/browse/SPARK-28228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Toth updated SPARK-28228:
-------------------------------
Description:
PostgreSQL handles nested WITHs in a different way then Spark does currently.
These queries retunes 1 in Spark while they return 2 in PostgreSQL:
{noformat}
WITH
t AS (SELECT 1),
t2 AS (
WITH t AS (SELECT 2)
SELECT * FROM t
)
SELECT * FROM t2
{noformat}
{noformat}
WITH t AS (SELECT 1)
SELECT (
WITH t AS (SELECT 2)
SELECT * FROM t
)
{noformat}
was:
Because of Spark-17590 it should be relatively easy to support WITH clause in
subqueries besides nested CTE definitions.
Here an example of a query that does not run on spark:
create table test (seqno int, k string, v int) using parquet;
insert into TABLE test values (1,'a', 99),(2, 'b', 88),(3, 'a', 77),(4, 'b',
66),(5, 'c', 55),(6, 'a', 44),(7, 'b', 33);
SELECT percentile(b, 0.5) FROM (WITH mavg AS (SELECT k, AVG(v) OVER (PARTITION
BY k ORDER BY seqno ROWS BETWEEN 3 PRECEDING AND CURRENT ROW) as b FROM test
ORDER BY seqno) SELECT k, MAX(b) as b FROM mavg GROUP BY k);
> Better support for WITH clause
> ------------------------------
>
> Key: SPARK-28228
> URL: https://issues.apache.org/jira/browse/SPARK-28228
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 3.0.0
> Reporter: Peter Toth
> Priority: Major
>
> PostgreSQL handles nested WITHs in a different way then Spark does currently.
> These queries retunes 1 in Spark while they return 2 in PostgreSQL:
> {noformat}
> WITH
> t AS (SELECT 1),
> t2 AS (
> WITH t AS (SELECT 2)
> SELECT * FROM t
> )
> SELECT * FROM t2
> {noformat}
> {noformat}
> WITH t AS (SELECT 1)
> SELECT (
> WITH t AS (SELECT 2)
> SELECT * FROM t
> )
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
