This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.2 by this push: new 817c2039f4d [SPARK-40124][SQL][TEST][3.2] Update TPCDS v1.4 q32 for Plan Stability tests 817c2039f4d is described below commit 817c2039f4d2ed888dfd3d2d557d11aa4b9138d8 Author: Kapil Kumar Singh <kapilsi...@microsoft.com> AuthorDate: Sat Aug 27 12:59:11 2022 -0700 [SPARK-40124][SQL][TEST][3.2] Update TPCDS v1.4 q32 for Plan Stability tests ### What changes were proposed in this pull request? This is port of SPARK-40124 to Spark 3.2. Fix query 32 for TPCDS v1.4 ### Why are the changes needed? Current q32.sql seems to be wrong. It is just selection `1`. Reference for query template: https://github.com/databricks/tpcds-kit/blob/eff5de2c30337b71cc0dc1976147742d2c65d378/query_templates/query32.tpl#L41 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Test change only Closes #37675 from mskapilks/change-q32-3.2. Authored-by: Kapil Kumar Singh <kapilsi...@microsoft.com> Signed-off-by: Dongjoon Hyun <dongj...@apache.org> --- .../approved-plans-v1_4/q32.sf100/explain.txt | 120 ++++++++++++--------- .../approved-plans-v1_4/q32.sf100/simplified.txt | 94 ++++++++-------- .../approved-plans-v1_4/q32/explain.txt | 120 ++++++++++++--------- .../approved-plans-v1_4/q32/simplified.txt | 92 ++++++++-------- .../resources/tpcds-query-results/v1_4/q32.sql.out | 4 +- sql/core/src/test/resources/tpcds/q32.sql | 2 +- 6 files changed, 236 insertions(+), 196 deletions(-) diff --git a/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32.sf100/explain.txt b/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32.sf100/explain.txt index d67ded6a42b..c1f2b9c8fd8 100644 --- a/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32.sf100/explain.txt +++ b/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32.sf100/explain.txt @@ -1,31 +1,33 @@ == Physical Plan == -CollectLimit (27) -+- * Project (26) - +- * BroadcastHashJoin Inner BuildRight (25) - :- * Project (23) - : +- * BroadcastHashJoin Inner BuildLeft (22) - : :- BroadcastExchange (18) - : : +- * Project (17) - : : +- * BroadcastHashJoin Inner BuildLeft (16) - : : :- BroadcastExchange (5) - : : : +- * Project (4) - : : : +- * Filter (3) - : : : +- * ColumnarToRow (2) - : : : +- Scan parquet default.item (1) - : : +- * Filter (15) - : : +- * HashAggregate (14) - : : +- Exchange (13) - : : +- * HashAggregate (12) - : : +- * Project (11) - : : +- * BroadcastHashJoin Inner BuildRight (10) - : : :- * Filter (8) - : : : +- * ColumnarToRow (7) - : : : +- Scan parquet default.catalog_sales (6) - : : +- ReusedExchange (9) - : +- * Filter (21) - : +- * ColumnarToRow (20) - : +- Scan parquet default.catalog_sales (19) - +- ReusedExchange (24) +* HashAggregate (29) ++- Exchange (28) + +- * HashAggregate (27) + +- * Project (26) + +- * BroadcastHashJoin Inner BuildRight (25) + :- * Project (23) + : +- * BroadcastHashJoin Inner BuildLeft (22) + : :- BroadcastExchange (18) + : : +- * Project (17) + : : +- * BroadcastHashJoin Inner BuildLeft (16) + : : :- BroadcastExchange (5) + : : : +- * Project (4) + : : : +- * Filter (3) + : : : +- * ColumnarToRow (2) + : : : +- Scan parquet default.item (1) + : : +- * Filter (15) + : : +- * HashAggregate (14) + : : +- Exchange (13) + : : +- * HashAggregate (12) + : : +- * Project (11) + : : +- * BroadcastHashJoin Inner BuildRight (10) + : : :- * Filter (8) + : : : +- * ColumnarToRow (7) + : : : +- Scan parquet default.catalog_sales (6) + : : +- ReusedExchange (9) + : +- * Filter (21) + : +- * ColumnarToRow (20) + : +- Scan parquet default.catalog_sales (19) + +- ReusedExchange (24) (1) Scan parquet default.item @@ -65,7 +67,7 @@ Input [3]: [cs_item_sk#3, cs_ext_discount_amt#4, cs_sold_date_sk#5] Input [3]: [cs_item_sk#3, cs_ext_discount_amt#4, cs_sold_date_sk#5] Condition : isnotnull(cs_item_sk#3) -(9) ReusedExchange [Reuses operator id: 32] +(9) ReusedExchange [Reuses operator id: 34] Output [1]: [d_date_sk#7] (10) BroadcastHashJoin [codegen id : 3] @@ -133,10 +135,10 @@ Right keys [1]: [cs_item_sk#14] Join condition: (cast(cs_ext_discount_amt#15 as decimal(14,7)) > (1.3 * avg(cs_ext_discount_amt))#13) (23) Project [codegen id : 6] -Output [1]: [cs_sold_date_sk#16] +Output [2]: [cs_ext_discount_amt#15, cs_sold_date_sk#16] Input [5]: [i_item_sk#1, (1.3 * avg(cs_ext_discount_amt))#13, cs_item_sk#14, cs_ext_discount_amt#15, cs_sold_date_sk#16] -(24) ReusedExchange [Reuses operator id: 32] +(24) ReusedExchange [Reuses operator id: 34] Output [1]: [d_date_sk#17] (25) BroadcastHashJoin [codegen id : 6] @@ -145,44 +147,58 @@ Right keys [1]: [d_date_sk#17] Join condition: None (26) Project [codegen id : 6] -Output [1]: [1 AS excess discount amount #18] -Input [2]: [cs_sold_date_sk#16, d_date_sk#17] - -(27) CollectLimit -Input [1]: [excess discount amount #18] -Arguments: 100 +Output [1]: [cs_ext_discount_amt#15] +Input [3]: [cs_ext_discount_amt#15, cs_sold_date_sk#16, d_date_sk#17] + +(27) HashAggregate [codegen id : 6] +Input [1]: [cs_ext_discount_amt#15] +Keys: [] +Functions [1]: [partial_sum(UnscaledValue(cs_ext_discount_amt#15))] +Aggregate Attributes [1]: [sum#18] +Results [1]: [sum#19] + +(28) Exchange +Input [1]: [sum#19] +Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=4] + +(29) HashAggregate [codegen id : 7] +Input [1]: [sum#19] +Keys: [] +Functions [1]: [sum(UnscaledValue(cs_ext_discount_amt#15))] +Aggregate Attributes [1]: [sum(UnscaledValue(cs_ext_discount_amt#15))#20] +Results [1]: [MakeDecimal(sum(UnscaledValue(cs_ext_discount_amt#15))#20,17,2) AS excess discount amount#21] ===== Subqueries ===== Subquery:1 Hosting operator id = 6 Hosting Expression = cs_sold_date_sk#5 IN dynamicpruning#6 -BroadcastExchange (32) -+- * Project (31) - +- * Filter (30) - +- * ColumnarToRow (29) - +- Scan parquet default.date_dim (28) +BroadcastExchange (34) ++- * Project (33) + +- * Filter (32) + +- * ColumnarToRow (31) + +- Scan parquet default.date_dim (30) -(28) Scan parquet default.date_dim -Output [2]: [d_date_sk#7, d_date#19] +(30) Scan parquet default.date_dim +Output [2]: [d_date_sk#7, d_date#22] Batched: true Location [not included in comparison]/{warehouse_dir}/date_dim] PushedFilters: [IsNotNull(d_date), GreaterThanOrEqual(d_date,2000-01-27), LessThanOrEqual(d_date,2000-04-26), IsNotNull(d_date_sk)] ReadSchema: struct<d_date_sk:int,d_date:date> -(29) ColumnarToRow [codegen id : 1] -Input [2]: [d_date_sk#7, d_date#19] +(31) ColumnarToRow [codegen id : 1] +Input [2]: [d_date_sk#7, d_date#22] -(30) Filter [codegen id : 1] -Input [2]: [d_date_sk#7, d_date#19] -Condition : (((isnotnull(d_date#19) AND (d_date#19 >= 2000-01-27)) AND (d_date#19 <= 2000-04-26)) AND isnotnull(d_date_sk#7)) +(32) Filter [codegen id : 1] +Input [2]: [d_date_sk#7, d_date#22] +Condition : (((isnotnull(d_date#22) AND (d_date#22 >= 2000-01-27)) AND (d_date#22 <= 2000-04-26)) AND isnotnull(d_date_sk#7)) -(31) Project [codegen id : 1] +(33) Project [codegen id : 1] Output [1]: [d_date_sk#7] -Input [2]: [d_date_sk#7, d_date#19] +Input [2]: [d_date_sk#7, d_date#22] -(32) BroadcastExchange +(34) BroadcastExchange Input [1]: [d_date_sk#7] -Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=4] +Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=5] Subquery:2 Hosting operator id = 19 Hosting Expression = cs_sold_date_sk#16 IN dynamicpruning#6 diff --git a/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32.sf100/simplified.txt b/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32.sf100/simplified.txt index 8ca9bf49029..27e630265a3 100644 --- a/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32.sf100/simplified.txt +++ b/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32.sf100/simplified.txt @@ -1,48 +1,52 @@ -CollectLimit - WholeStageCodegen (6) - Project - BroadcastHashJoin [cs_sold_date_sk,d_date_sk] - Project [cs_sold_date_sk] - BroadcastHashJoin [i_item_sk,cs_item_sk,cs_ext_discount_amt,(1.3 * avg(cs_ext_discount_amt))] - InputAdapter - BroadcastExchange #1 - WholeStageCodegen (4) - Project [i_item_sk,(1.3 * avg(cs_ext_discount_amt))] - BroadcastHashJoin [i_item_sk,cs_item_sk] - InputAdapter - BroadcastExchange #2 - WholeStageCodegen (1) - Project [i_item_sk] - Filter [i_manufact_id,i_item_sk] - ColumnarToRow - InputAdapter - Scan parquet default.item [i_item_sk,i_manufact_id] - Filter [(1.3 * avg(cs_ext_discount_amt))] - HashAggregate [cs_item_sk,sum,count] [avg(UnscaledValue(cs_ext_discount_amt)),(1.3 * avg(cs_ext_discount_amt)),sum,count] - InputAdapter - Exchange [cs_item_sk] #3 - WholeStageCodegen (3) - HashAggregate [cs_item_sk,cs_ext_discount_amt] [sum,count,sum,count] - Project [cs_item_sk,cs_ext_discount_amt] - BroadcastHashJoin [cs_sold_date_sk,d_date_sk] - Filter [cs_item_sk] +WholeStageCodegen (7) + HashAggregate [sum] [sum(UnscaledValue(cs_ext_discount_amt)),excess discount amount,sum] + InputAdapter + Exchange #1 + WholeStageCodegen (6) + HashAggregate [cs_ext_discount_amt] [sum,sum] + Project [cs_ext_discount_amt] + BroadcastHashJoin [cs_sold_date_sk,d_date_sk] + Project [cs_ext_discount_amt,cs_sold_date_sk] + BroadcastHashJoin [i_item_sk,cs_item_sk,cs_ext_discount_amt,(1.3 * avg(cs_ext_discount_amt))] + InputAdapter + BroadcastExchange #2 + WholeStageCodegen (4) + Project [i_item_sk,(1.3 * avg(cs_ext_discount_amt))] + BroadcastHashJoin [i_item_sk,cs_item_sk] + InputAdapter + BroadcastExchange #3 + WholeStageCodegen (1) + Project [i_item_sk] + Filter [i_manufact_id,i_item_sk] ColumnarToRow InputAdapter - Scan parquet default.catalog_sales [cs_item_sk,cs_ext_discount_amt,cs_sold_date_sk] - SubqueryBroadcast [d_date_sk] #1 - BroadcastExchange #4 - WholeStageCodegen (1) - Project [d_date_sk] - Filter [d_date,d_date_sk] - ColumnarToRow - InputAdapter - Scan parquet default.date_dim [d_date_sk,d_date] - InputAdapter - ReusedExchange [d_date_sk] #4 - Filter [cs_item_sk,cs_ext_discount_amt] - ColumnarToRow + Scan parquet default.item [i_item_sk,i_manufact_id] + Filter [(1.3 * avg(cs_ext_discount_amt))] + HashAggregate [cs_item_sk,sum,count] [avg(UnscaledValue(cs_ext_discount_amt)),(1.3 * avg(cs_ext_discount_amt)),sum,count] + InputAdapter + Exchange [cs_item_sk] #4 + WholeStageCodegen (3) + HashAggregate [cs_item_sk,cs_ext_discount_amt] [sum,count,sum,count] + Project [cs_item_sk,cs_ext_discount_amt] + BroadcastHashJoin [cs_sold_date_sk,d_date_sk] + Filter [cs_item_sk] + ColumnarToRow + InputAdapter + Scan parquet default.catalog_sales [cs_item_sk,cs_ext_discount_amt,cs_sold_date_sk] + SubqueryBroadcast [d_date_sk] #1 + BroadcastExchange #5 + WholeStageCodegen (1) + Project [d_date_sk] + Filter [d_date,d_date_sk] + ColumnarToRow + InputAdapter + Scan parquet default.date_dim [d_date_sk,d_date] + InputAdapter + ReusedExchange [d_date_sk] #5 + Filter [cs_item_sk,cs_ext_discount_amt] + ColumnarToRow + InputAdapter + Scan parquet default.catalog_sales [cs_item_sk,cs_ext_discount_amt,cs_sold_date_sk] + ReusedSubquery [d_date_sk] #1 InputAdapter - Scan parquet default.catalog_sales [cs_item_sk,cs_ext_discount_amt,cs_sold_date_sk] - ReusedSubquery [d_date_sk] #1 - InputAdapter - ReusedExchange [d_date_sk] #4 + ReusedExchange [d_date_sk] #5 diff --git a/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32/explain.txt b/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32/explain.txt index f38b3802bac..0e2e356b8cb 100644 --- a/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32/explain.txt +++ b/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32/explain.txt @@ -1,31 +1,33 @@ == Physical Plan == -CollectLimit (27) -+- * Project (26) - +- * BroadcastHashJoin Inner BuildRight (25) - :- * Project (23) - : +- * BroadcastHashJoin Inner BuildRight (22) - : :- * Project (10) - : : +- * BroadcastHashJoin Inner BuildRight (9) - : : :- * Filter (3) - : : : +- * ColumnarToRow (2) - : : : +- Scan parquet default.catalog_sales (1) - : : +- BroadcastExchange (8) - : : +- * Project (7) - : : +- * Filter (6) - : : +- * ColumnarToRow (5) - : : +- Scan parquet default.item (4) - : +- BroadcastExchange (21) - : +- * Filter (20) - : +- * HashAggregate (19) - : +- Exchange (18) - : +- * HashAggregate (17) - : +- * Project (16) - : +- * BroadcastHashJoin Inner BuildRight (15) - : :- * Filter (13) - : : +- * ColumnarToRow (12) - : : +- Scan parquet default.catalog_sales (11) - : +- ReusedExchange (14) - +- ReusedExchange (24) +* HashAggregate (29) ++- Exchange (28) + +- * HashAggregate (27) + +- * Project (26) + +- * BroadcastHashJoin Inner BuildRight (25) + :- * Project (23) + : +- * BroadcastHashJoin Inner BuildRight (22) + : :- * Project (10) + : : +- * BroadcastHashJoin Inner BuildRight (9) + : : :- * Filter (3) + : : : +- * ColumnarToRow (2) + : : : +- Scan parquet default.catalog_sales (1) + : : +- BroadcastExchange (8) + : : +- * Project (7) + : : +- * Filter (6) + : : +- * ColumnarToRow (5) + : : +- Scan parquet default.item (4) + : +- BroadcastExchange (21) + : +- * Filter (20) + : +- * HashAggregate (19) + : +- Exchange (18) + : +- * HashAggregate (17) + : +- * Project (16) + : +- * BroadcastHashJoin Inner BuildRight (15) + : :- * Filter (13) + : : +- * ColumnarToRow (12) + : : +- Scan parquet default.catalog_sales (11) + : +- ReusedExchange (14) + +- ReusedExchange (24) (1) Scan parquet default.catalog_sales @@ -89,7 +91,7 @@ Input [3]: [cs_item_sk#7, cs_ext_discount_amt#8, cs_sold_date_sk#9] Input [3]: [cs_item_sk#7, cs_ext_discount_amt#8, cs_sold_date_sk#9] Condition : isnotnull(cs_item_sk#7) -(14) ReusedExchange [Reuses operator id: 32] +(14) ReusedExchange [Reuses operator id: 34] Output [1]: [d_date_sk#10] (15) BroadcastHashJoin [codegen id : 3] @@ -133,10 +135,10 @@ Right keys [1]: [cs_item_sk#7] Join condition: (cast(cs_ext_discount_amt#2 as decimal(14,7)) > (1.3 * avg(cs_ext_discount_amt))#16) (23) Project [codegen id : 6] -Output [1]: [cs_sold_date_sk#3] +Output [2]: [cs_ext_discount_amt#2, cs_sold_date_sk#3] Input [5]: [cs_ext_discount_amt#2, cs_sold_date_sk#3, i_item_sk#5, (1.3 * avg(cs_ext_discount_amt))#16, cs_item_sk#7] -(24) ReusedExchange [Reuses operator id: 32] +(24) ReusedExchange [Reuses operator id: 34] Output [1]: [d_date_sk#17] (25) BroadcastHashJoin [codegen id : 6] @@ -145,44 +147,58 @@ Right keys [1]: [d_date_sk#17] Join condition: None (26) Project [codegen id : 6] -Output [1]: [1 AS excess discount amount #18] -Input [2]: [cs_sold_date_sk#3, d_date_sk#17] - -(27) CollectLimit -Input [1]: [excess discount amount #18] -Arguments: 100 +Output [1]: [cs_ext_discount_amt#2] +Input [3]: [cs_ext_discount_amt#2, cs_sold_date_sk#3, d_date_sk#17] + +(27) HashAggregate [codegen id : 6] +Input [1]: [cs_ext_discount_amt#2] +Keys: [] +Functions [1]: [partial_sum(UnscaledValue(cs_ext_discount_amt#2))] +Aggregate Attributes [1]: [sum#18] +Results [1]: [sum#19] + +(28) Exchange +Input [1]: [sum#19] +Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=4] + +(29) HashAggregate [codegen id : 7] +Input [1]: [sum#19] +Keys: [] +Functions [1]: [sum(UnscaledValue(cs_ext_discount_amt#2))] +Aggregate Attributes [1]: [sum(UnscaledValue(cs_ext_discount_amt#2))#20] +Results [1]: [MakeDecimal(sum(UnscaledValue(cs_ext_discount_amt#2))#20,17,2) AS excess discount amount#21] ===== Subqueries ===== Subquery:1 Hosting operator id = 1 Hosting Expression = cs_sold_date_sk#3 IN dynamicpruning#4 -BroadcastExchange (32) -+- * Project (31) - +- * Filter (30) - +- * ColumnarToRow (29) - +- Scan parquet default.date_dim (28) +BroadcastExchange (34) ++- * Project (33) + +- * Filter (32) + +- * ColumnarToRow (31) + +- Scan parquet default.date_dim (30) -(28) Scan parquet default.date_dim -Output [2]: [d_date_sk#17, d_date#19] +(30) Scan parquet default.date_dim +Output [2]: [d_date_sk#17, d_date#22] Batched: true Location [not included in comparison]/{warehouse_dir}/date_dim] PushedFilters: [IsNotNull(d_date), GreaterThanOrEqual(d_date,2000-01-27), LessThanOrEqual(d_date,2000-04-26), IsNotNull(d_date_sk)] ReadSchema: struct<d_date_sk:int,d_date:date> -(29) ColumnarToRow [codegen id : 1] -Input [2]: [d_date_sk#17, d_date#19] +(31) ColumnarToRow [codegen id : 1] +Input [2]: [d_date_sk#17, d_date#22] -(30) Filter [codegen id : 1] -Input [2]: [d_date_sk#17, d_date#19] -Condition : (((isnotnull(d_date#19) AND (d_date#19 >= 2000-01-27)) AND (d_date#19 <= 2000-04-26)) AND isnotnull(d_date_sk#17)) +(32) Filter [codegen id : 1] +Input [2]: [d_date_sk#17, d_date#22] +Condition : (((isnotnull(d_date#22) AND (d_date#22 >= 2000-01-27)) AND (d_date#22 <= 2000-04-26)) AND isnotnull(d_date_sk#17)) -(31) Project [codegen id : 1] +(33) Project [codegen id : 1] Output [1]: [d_date_sk#17] -Input [2]: [d_date_sk#17, d_date#19] +Input [2]: [d_date_sk#17, d_date#22] -(32) BroadcastExchange +(34) BroadcastExchange Input [1]: [d_date_sk#17] -Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=4] +Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=5] Subquery:2 Hosting operator id = 11 Hosting Expression = cs_sold_date_sk#9 IN dynamicpruning#4 diff --git a/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32/simplified.txt b/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32/simplified.txt index aea77be43cf..0b2410699cf 100644 --- a/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32/simplified.txt +++ b/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32/simplified.txt @@ -1,48 +1,52 @@ -CollectLimit - WholeStageCodegen (6) - Project - BroadcastHashJoin [cs_sold_date_sk,d_date_sk] - Project [cs_sold_date_sk] - BroadcastHashJoin [i_item_sk,cs_item_sk,cs_ext_discount_amt,(1.3 * avg(cs_ext_discount_amt))] - Project [cs_ext_discount_amt,cs_sold_date_sk,i_item_sk] - BroadcastHashJoin [cs_item_sk,i_item_sk] - Filter [cs_item_sk,cs_ext_discount_amt] - ColumnarToRow - InputAdapter - Scan parquet default.catalog_sales [cs_item_sk,cs_ext_discount_amt,cs_sold_date_sk] - SubqueryBroadcast [d_date_sk] #1 - BroadcastExchange #1 +WholeStageCodegen (7) + HashAggregate [sum] [sum(UnscaledValue(cs_ext_discount_amt)),excess discount amount,sum] + InputAdapter + Exchange #1 + WholeStageCodegen (6) + HashAggregate [cs_ext_discount_amt] [sum,sum] + Project [cs_ext_discount_amt] + BroadcastHashJoin [cs_sold_date_sk,d_date_sk] + Project [cs_ext_discount_amt,cs_sold_date_sk] + BroadcastHashJoin [i_item_sk,cs_item_sk,cs_ext_discount_amt,(1.3 * avg(cs_ext_discount_amt))] + Project [cs_ext_discount_amt,cs_sold_date_sk,i_item_sk] + BroadcastHashJoin [cs_item_sk,i_item_sk] + Filter [cs_item_sk,cs_ext_discount_amt] + ColumnarToRow + InputAdapter + Scan parquet default.catalog_sales [cs_item_sk,cs_ext_discount_amt,cs_sold_date_sk] + SubqueryBroadcast [d_date_sk] #1 + BroadcastExchange #2 + WholeStageCodegen (1) + Project [d_date_sk] + Filter [d_date,d_date_sk] + ColumnarToRow + InputAdapter + Scan parquet default.date_dim [d_date_sk,d_date] + InputAdapter + BroadcastExchange #3 WholeStageCodegen (1) - Project [d_date_sk] - Filter [d_date,d_date_sk] + Project [i_item_sk] + Filter [i_manufact_id,i_item_sk] ColumnarToRow InputAdapter - Scan parquet default.date_dim [d_date_sk,d_date] + Scan parquet default.item [i_item_sk,i_manufact_id] + InputAdapter + BroadcastExchange #4 + WholeStageCodegen (4) + Filter [(1.3 * avg(cs_ext_discount_amt))] + HashAggregate [cs_item_sk,sum,count] [avg(UnscaledValue(cs_ext_discount_amt)),(1.3 * avg(cs_ext_discount_amt)),sum,count] + InputAdapter + Exchange [cs_item_sk] #5 + WholeStageCodegen (3) + HashAggregate [cs_item_sk,cs_ext_discount_amt] [sum,count,sum,count] + Project [cs_item_sk,cs_ext_discount_amt] + BroadcastHashJoin [cs_sold_date_sk,d_date_sk] + Filter [cs_item_sk] + ColumnarToRow + InputAdapter + Scan parquet default.catalog_sales [cs_item_sk,cs_ext_discount_amt,cs_sold_date_sk] + ReusedSubquery [d_date_sk] #1 + InputAdapter + ReusedExchange [d_date_sk] #2 InputAdapter - BroadcastExchange #2 - WholeStageCodegen (1) - Project [i_item_sk] - Filter [i_manufact_id,i_item_sk] - ColumnarToRow - InputAdapter - Scan parquet default.item [i_item_sk,i_manufact_id] - InputAdapter - BroadcastExchange #3 - WholeStageCodegen (4) - Filter [(1.3 * avg(cs_ext_discount_amt))] - HashAggregate [cs_item_sk,sum,count] [avg(UnscaledValue(cs_ext_discount_amt)),(1.3 * avg(cs_ext_discount_amt)),sum,count] - InputAdapter - Exchange [cs_item_sk] #4 - WholeStageCodegen (3) - HashAggregate [cs_item_sk,cs_ext_discount_amt] [sum,count,sum,count] - Project [cs_item_sk,cs_ext_discount_amt] - BroadcastHashJoin [cs_sold_date_sk,d_date_sk] - Filter [cs_item_sk] - ColumnarToRow - InputAdapter - Scan parquet default.catalog_sales [cs_item_sk,cs_ext_discount_amt,cs_sold_date_sk] - ReusedSubquery [d_date_sk] #1 - InputAdapter - ReusedExchange [d_date_sk] #1 - InputAdapter - ReusedExchange [d_date_sk] #1 + ReusedExchange [d_date_sk] #2 diff --git a/sql/core/src/test/resources/tpcds-query-results/v1_4/q32.sql.out b/sql/core/src/test/resources/tpcds-query-results/v1_4/q32.sql.out index de81b93b81c..fb73f26cf39 100644 --- a/sql/core/src/test/resources/tpcds-query-results/v1_4/q32.sql.out +++ b/sql/core/src/test/resources/tpcds-query-results/v1_4/q32.sql.out @@ -1,6 +1,6 @@ -- Automatically generated by TPCDSQueryTestSuite -- !query schema -struct<excess discount amount :int> +struct<excess discount amount:decimal(17,2)> -- !query output -1 +9089.28 diff --git a/sql/core/src/test/resources/tpcds/q32.sql b/sql/core/src/test/resources/tpcds/q32.sql index a6f59ecb873..1d856ca5230 100755 --- a/sql/core/src/test/resources/tpcds/q32.sql +++ b/sql/core/src/test/resources/tpcds/q32.sql @@ -1,4 +1,4 @@ -SELECT 1 AS `excess discount amount ` +SELECT sum(cs_ext_discount_amt) AS `excess discount amount` FROM catalog_sales, item, date_dim WHERE --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org