HeartSaVioR commented on pull request #31355: URL: https://github.com/apache/spark/pull/31355#issuecomment-806332643
I took the diff on printed plans (expected, actual) in stack trace, and they seem to be identical. ``` TakeOrderedAndProject [i_item_id,i_item_desc,s_state,store_sales_quantitycount,store_sales_quantityave,store_sales_quantitystdev,store_sales_quantitycov,as_store_returns_quantitycount,as_store_returns_quantityave,as_store_returns_quantitystdev,store_returns_quantitycov,catalog_sales_quantitycount,catalog_sales_quantityave,catalog_sales_quantitystdev,catalog_sales_quantitycov] WholeStageCodegen (9) HashAggregate [i_item_id,i_item_desc,s_state,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2] [count(ss_quantity),avg(ss_quantity),stddev_samp(cast(ss_quantity as double)),count(sr_return_quantity),avg(sr_return_quantity),stddev_samp(cast(sr_return_quantity as double)),count(cs_quantity),avg(cs_quantity),stddev_samp(cast(cs_quantity as double)),store_sales_quantitycount,store_sales_quantityave,store_sales_quantitystdev,store_sales_quantitycov,as_store_returns_quantitycount,as_store_returns_quantityave,as_store_returns_quantitystdev,store_returns_quantitycov,catalog_sales_quantitycount,catalog_sales_quantityave,catalog_sales_quantitystdev,catalog_sales_quantitycov,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2] InputAdapter Exchange [i_item_id,i_item_desc,s_state] #1 WholeStageCodegen (8) HashAggregate [i_item_id,i_item_desc,s_state,ss_quantity,sr_return_quantity,cs_quantity] [count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2] Project [ss_quantity,sr_return_quantity,cs_quantity,s_state,i_item_id,i_item_desc] BroadcastHashJoin [ss_item_sk,i_item_sk] Project [ss_item_sk,ss_quantity,sr_return_quantity,cs_quantity,s_state] BroadcastHashJoin [ss_store_sk,s_store_sk] Project [ss_item_sk,ss_store_sk,ss_quantity,sr_return_quantity,cs_quantity] BroadcastHashJoin [cs_sold_date_sk,d_date_sk] Project [ss_item_sk,ss_store_sk,ss_quantity,sr_return_quantity,cs_quantity,cs_sold_date_sk] BroadcastHashJoin [sr_returned_date_sk,d_date_sk] Project [ss_item_sk,ss_store_sk,ss_quantity,sr_return_quantity,sr_returned_date_sk,cs_quantity,cs_sold_date_sk] BroadcastHashJoin [ss_sold_date_sk,d_date_sk] Project [ss_item_sk,ss_store_sk,ss_quantity,ss_sold_date_sk,sr_return_quantity,sr_returned_date_sk,cs_quantity,cs_sold_date_sk] BroadcastHashJoin [sr_customer_sk,sr_item_sk,cs_bill_customer_sk,cs_item_sk] Project [ss_item_sk,ss_store_sk,ss_quantity,ss_sold_date_sk,sr_item_sk,sr_customer_sk,sr_return_quantity,sr_returned_date_sk] BroadcastHashJoin [ss_customer_sk,ss_item_sk,ss_ticket_number,sr_customer_sk,sr_item_sk,sr_ticket_number] Filter [ss_customer_sk,ss_item_sk,ss_ticket_number,ss_store_sk] ColumnarToRow InputAdapter Scan parquet default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity,ss_sold_date_sk] SubqueryBroadcast [d_date_sk] #1 ReusedExchange [d_date_sk] #2 InputAdapter BroadcastExchange #3 WholeStageCodegen (1) Filter [sr_customer_sk,sr_item_sk,sr_ticket_number] ColumnarToRow InputAdapter Scan parquet default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity,sr_returned_date_sk] SubqueryBroadcast [d_date_sk] #2 ReusedExchange [d_date_sk] #4 InputAdapter BroadcastExchange #5 WholeStageCodegen (2) Filter [cs_bill_customer_sk,cs_item_sk] ColumnarToRow InputAdapter Scan parquet default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_sold_date_sk] SubqueryBroadcast [d_date_sk] #3 ReusedExchange [d_date_sk] #4 InputAdapter BroadcastExchange #2 WholeStageCodegen (3) Project [d_date_sk] Filter [d_quarter_name,d_date_sk] ColumnarToRow InputAdapter Scan parquet default.date_dim [d_date_sk,d_quarter_name] InputAdapter BroadcastExchange #4 WholeStageCodegen (4) Project [d_date_sk] Filter [d_quarter_name,d_date_sk] ColumnarToRow InputAdapter Scan parquet default.date_dim [d_date_sk,d_quarter_name] InputAdapter ReusedExchange [d_date_sk] #4 InputAdapter BroadcastExchange #6 WholeStageCodegen (6) Filter [s_store_sk] ColumnarToRow InputAdapter Scan parquet default.store [s_store_sk,s_state] InputAdapter BroadcastExchange #7 WholeStageCodegen (7) Filter [i_item_sk] ColumnarToRow InputAdapter Scan parquet default.item [i_item_sk,i_item_id,i_item_desc] ``` ``` TakeOrderedAndProject [i_item_id,i_item_desc,s_state,store_sales_quantitycount,store_sales_quantityave,store_sales_quantitystdev,store_sales_quantitycov,as_store_returns_quantitycount,as_store_returns_quantityave,as_store_returns_quantitystdev,store_returns_quantitycov,catalog_sales_quantitycount,catalog_sales_quantityave,catalog_sales_quantitystdev,catalog_sales_quantitycov] WholeStageCodegen (9) HashAggregate [i_item_id,i_item_desc,s_state,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2] [count(ss_quantity),avg(ss_quantity),stddev_samp(cast(ss_quantity as double)),count(sr_return_quantity),avg(sr_return_quantity),stddev_samp(cast(sr_return_quantity as double)),count(cs_quantity),avg(cs_quantity),stddev_samp(cast(cs_quantity as double)),store_sales_quantitycount,store_sales_quantityave,store_sales_quantitystdev,store_sales_quantitycov,as_store_returns_quantitycount,as_store_returns_quantityave,as_store_returns_quantitystdev,store_returns_quantitycov,catalog_sales_quantitycount,catalog_sales_quantityave,catalog_sales_quantitystdev,catalog_sales_quantitycov,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2] InputAdapter Exchange [i_item_id,i_item_desc,s_state] #1 WholeStageCodegen (8) HashAggregate [i_item_id,i_item_desc,s_state,ss_quantity,sr_return_quantity,cs_quantity] [count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2] Project [ss_quantity,sr_return_quantity,cs_quantity,s_state,i_item_id,i_item_desc] BroadcastHashJoin [ss_item_sk,i_item_sk] Project [ss_item_sk,ss_quantity,sr_return_quantity,cs_quantity,s_state] BroadcastHashJoin [ss_store_sk,s_store_sk] Project [ss_item_sk,ss_store_sk,ss_quantity,sr_return_quantity,cs_quantity] BroadcastHashJoin [cs_sold_date_sk,d_date_sk] Project [ss_item_sk,ss_store_sk,ss_quantity,sr_return_quantity,cs_quantity,cs_sold_date_sk] BroadcastHashJoin [sr_returned_date_sk,d_date_sk] Project [ss_item_sk,ss_store_sk,ss_quantity,sr_return_quantity,sr_returned_date_sk,cs_quantity,cs_sold_date_sk] BroadcastHashJoin [ss_sold_date_sk,d_date_sk] Project [ss_item_sk,ss_store_sk,ss_quantity,ss_sold_date_sk,sr_return_quantity,sr_returned_date_sk,cs_quantity,cs_sold_date_sk] BroadcastHashJoin [sr_customer_sk,sr_item_sk,cs_bill_customer_sk,cs_item_sk] Project [ss_item_sk,ss_store_sk,ss_quantity,ss_sold_date_sk,sr_item_sk,sr_customer_sk,sr_return_quantity,sr_returned_date_sk] BroadcastHashJoin [ss_customer_sk,ss_item_sk,ss_ticket_number,sr_customer_sk,sr_item_sk,sr_ticket_number] Filter [ss_customer_sk,ss_item_sk,ss_ticket_number,ss_store_sk] ColumnarToRow InputAdapter Scan parquet default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity,ss_sold_date_sk] SubqueryBroadcast [d_date_sk] #1 ReusedExchange [d_date_sk] #2 InputAdapter BroadcastExchange #3 WholeStageCodegen (1) Filter [sr_customer_sk,sr_item_sk,sr_ticket_number] ColumnarToRow InputAdapter Scan parquet default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity,sr_returned_date_sk] SubqueryBroadcast [d_date_sk] #2 ReusedExchange [d_date_sk] #4 InputAdapter BroadcastExchange #5 WholeStageCodegen (2) Filter [cs_bill_customer_sk,cs_item_sk] ColumnarToRow InputAdapter Scan parquet default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_sold_date_sk] SubqueryBroadcast [d_date_sk] #3 ReusedExchange [d_date_sk] #4 InputAdapter BroadcastExchange #2 WholeStageCodegen (3) Project [d_date_sk] Filter [d_quarter_name,d_date_sk] ColumnarToRow InputAdapter Scan parquet default.date_dim [d_date_sk,d_quarter_name] InputAdapter BroadcastExchange #4 WholeStageCodegen (4) Project [d_date_sk] Filter [d_quarter_name,d_date_sk] ColumnarToRow InputAdapter Scan parquet default.date_dim [d_date_sk,d_quarter_name] InputAdapter ReusedExchange [d_date_sk] #4 InputAdapter BroadcastExchange #6 WholeStageCodegen (6) Filter [s_store_sk] ColumnarToRow InputAdapter Scan parquet default.store [s_store_sk,s_state] InputAdapter BroadcastExchange #7 WholeStageCodegen (7) Filter [i_item_sk] ColumnarToRow InputAdapter Scan parquet default.item [i_item_sk,i_item_id,i_item_desc] ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org