[ https://issues.apache.org/jira/browse/IMPALA-10758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Qifan Chen closed IMPALA-10758. ------------------------------- Resolution: Not A Bug Verified that the plan difference does not show up in recent core s3 tests. The test passes. > S3PlannerTest.testNestedCollections fails because of mismatch plan > ------------------------------------------------------------------ > > Key: IMPALA-10758 > URL: https://issues.apache.org/jira/browse/IMPALA-10758 > Project: IMPALA > Issue Type: Bug > Components: Frontend > Reporter: Yongzhi Chen > Assignee: Qifan Chen > Priority: Critical > > S3PlannerTest.testNestedCollections fails in impala-asf-master-core-s3 with > following error: > {noformat} > Error Message > Section PLAN of query: > select 1 > from tpch_nested_parquet.region.r_nations t1 > inner join tpch_nested_parquet.customer t2 on t2.c_nationkey = t1.pos > inner join tpch_nested_parquet.region t3 on t3.r_comment = t2.c_address > left join t2.c_orders t4 > inner join tpch_nested_parquet.region t5 on t5.r_regionkey = t2.c_custkey > left join t4.item.o_lineitems t6 on t6.item.l_returnflag = > t4.item.o_orderpriority > Actual does not match expected result: > PLAN-ROOT SINK > | > 14:SUBPLAN > | row-size=183B cardinality=1 > | > |--12:SUBPLAN > | | row-size=183B cardinality=1 > | | > | |--10:NESTED LOOP JOIN [RIGHT OUTER JOIN] > | | | join predicates: t6.item.l_returnflag = t4.item.o_orderpriority > | | | row-size=183B cardinality=10 > | | | > | | |--08:SINGULAR ROW SRC > | | | row-size=171B cardinality=1 > | | | > | | 09:UNNEST [t4.item.o_lineitems t6] > | | row-size=0B cardinality=10 > | | > | 11:NESTED LOOP JOIN [RIGHT OUTER JOIN] > | | row-size=171B cardinality=1 > | | > | |--06:SINGULAR ROW SRC > | | row-size=147B cardinality=1 > | | > | 07:UNNEST [t2.c_orders t4] > | row-size=0B cardinality=10 > | > 13:HASH JOIN [INNER JOIN] > | hash predicates: t1.pos = t2.c_nationkey > | runtime filters: RF000 <- t2.c_nationkey, RF001 <- t2.c_nationkey > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > | row-size=147B cardinality=1 > | > |--05:HASH JOIN [INNER JOIN] > | | hash predicates: t3.r_comment = t2.c_address > | | runtime filters: RF002 <- t2.c_address > | | row-size=139B cardinality=1 > | | > | |--04:HASH JOIN [INNER JOIN] > | | | hash predicates: t2.c_custkey = t5.r_regionkey > | | | runtime filters: RF004 <- t5.r_regionkey > | | | row-size=61B cardinality=5 > | | | > | | |--03:SCAN S3 [tpch_nested_parquet.region t5] > | | | S3 partitions=1/1 files=1 size=3.59KB > | | | row-size=2B cardinality=5 > | | | > | | 01:SCAN S3 [tpch_nested_parquet.customer t2] > | | S3 partitions=1/1 files=4 size=289.06MB > | | runtime filters: RF004 -> t2.c_custkey > | | row-size=59B cardinality=150.00K > | | > | 02:SCAN S3 [tpch_nested_parquet.region t3] > | S3 partitions=1/1 files=1 size=3.59KB > | runtime filters: RF002 -> t3.r_comment > | row-size=78B cardinality=5 > | > 00:SCAN S3 [tpch_nested_parquet.region.r_nations t1] > S3 partitions=1/1 files=1 size=3.59KB > runtime filters: RF001 -> t1.pos, RF000 -> t1.pos > row-size=8B cardinality=50 > Expected: > PLAN-ROOT SINK > | > 14:SUBPLAN > | row-size=183B cardinality=1 > | > |--12:SUBPLAN > | | row-size=183B cardinality=1 > | | > | |--10:NESTED LOOP JOIN [RIGHT OUTER JOIN] > | | | join predicates: t6.item.l_returnflag = t4.item.o_orderpriority > | | | row-size=183B cardinality=10 > | | | > | | |--08:SINGULAR ROW SRC > | | | row-size=171B cardinality=1 > | | | > | | 09:UNNEST [t4.item.o_lineitems t6] > | | row-size=0B cardinality=10 > | | > | 11:NESTED LOOP JOIN [RIGHT OUTER JOIN] > | | row-size=171B cardinality=1 > | | > | |--06:SINGULAR ROW SRC > | | row-size=147B cardinality=1 > | | > | 07:UNNEST [t2.c_orders t4] > | row-size=0B cardinality=10 > | > 13:HASH JOIN [INNER JOIN] > | hash predicates: t1.pos = t2.c_nationkey > | runtime filters: RF000 <- t2.c_nationkey > | row-size=147B cardinality=1 > | > |--05:HASH JOIN [INNER JOIN] > | | hash predicates: t3.r_comment = t2.c_address > | | runtime filters: RF002 <- t2.c_address > | | row-size=139B cardinality=1 > | | > | |--04:HASH JOIN [INNER JOIN] > | | | hash predicates: t2.c_custkey = t5.r_regionkey > | | | runtime filters: RF004 <- t5.r_regionkey > | | | row-size=61B cardinality=5 > | | | > | | |--03:SCAN HDFS [tpch_nested_parquet.region t5] > | | | HDFS partitions=1/1 files=1 size=3.59KB > | | | row-size=2B cardinality=5 > | | | > | | 01:SCAN HDFS [tpch_nested_parquet.customer t2] > | | HDFS partitions=1/1 files=4 size=289.02MB > | | runtime filters: RF004 -> t2.c_custkey > | | row-size=59B cardinality=150.00K > | | > | 02:SCAN HDFS [tpch_nested_parquet.region t3] > | HDFS partitions=1/1 files=1 size=3.59KB > | runtime filters: RF002 -> t3.r_comment > | row-size=78B cardinality=5 > | > 00:SCAN HDFS [tpch_nested_parquet.region.r_nations t1] > HDFS partitions=1/1 files=1 size=3.59KB > runtime filters: RF000 -> t1.pos > row-size=8B cardinality=50 > Verbose plan: > F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 > Per-Host Resources: mem-estimate=500.81MB mem-reservation=28.84MB > thread-reservation=5 runtime-filters-memory=3.00MB > PLAN-ROOT SINK > | output exprs: CAST(1 AS TINYINT) > | mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB > thread-reservation=0 > | > 14:SUBPLAN > | mem-estimate=0B mem-reservation=0B thread-reservation=0 > | tuple-ids=5N,3N,0,2,1,4 row-size=183B cardinality=1 > | in pipelines: 00(GETNEXT) > | > |--12:SUBPLAN > | | mem-estimate=0B mem-reservation=0B thread-reservation=0 > | | tuple-ids=5N,3N,0,2,1,4 row-size=183B cardinality=1 > | | in pipelines: 00(GETNEXT) > | | > | |--10:NESTED LOOP JOIN [RIGHT OUTER JOIN] > | | | join predicates: t6.item.l_returnflag = t4.item.o_orderpriority > | | | mem-estimate=172B mem-reservation=0B thread-reservation=0 > | | | tuple-ids=5N,3N,0,2,1,4 row-size=183B cardinality=10 > | | | in pipelines: 00(GETNEXT) > | | | > | | |--08:SINGULAR ROW SRC > | | | parent-subplan=12 > | | | mem-estimate=0B mem-reservation=0B thread-reservation=0 > | | | tuple-ids=3N,0,2,1,4 row-size=171B cardinality=1 > | | | in pipelines: 00(GETNEXT) > | | | > | | 09:UNNEST [t4.item.o_lineitems t6] > | | parent-subplan=12 > | | mem-estimate=0B mem-reservation=0B thread-reservation=0 > | | tuple-ids=5 row-size=0B cardinality=10 > | | in pipelines: 00(GETNEXT) > | | > | 11:NESTED LOOP JOIN [RIGHT OUTER JOIN] > | | mem-estimate=148B mem-reservation=0B thread-reservation=0 > | | tuple-ids=3N,0,2,1,4 row-size=171B cardinality=1 > | | in pipelines: 00(GETNEXT) > | | > | |--06:SINGULAR ROW SRC > | | parent-subplan=14 > | | mem-estimate=0B mem-reservation=0B thread-reservation=0 > | | tuple-ids=0,2,1,4 row-size=147B cardinality=1 > | | in pipelines: 00(GETNEXT) > | | > | 07:UNNEST [t2.c_orders t4] > | parent-subplan=14 > | mem-estimate=0B mem-reservation=0B thread-reservation=0 > | tuple-ids=3 row-size=0B cardinality=10 > | in pipelines: 00(GETNEXT) > | > 13:HASH JOIN [INNER JOIN] > | hash predicates: t1.pos = t2.c_nationkey > | fk/pk conjuncts: assumed fk/pk > | runtime filters: RF000[bloom] <- t2.c_nationkey, RF001[min_max] <- > t2.c_nationkey > | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB > thread-reservation=0 > | tuple-ids=0,2,1,4 row-size=147B cardinality=1 > | in pipelines: 00(GETNEXT), 02(OPEN) > | > |--05:HASH JOIN [INNER JOIN] > | | hash predicates: t3.r_comment = t2.c_address > | | fk/pk conjuncts: t3.r_comment = t2.c_address > | | runtime filters: RF002[bloom] <- t2.c_address > | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB > thread-reservation=0 > | | tuple-ids=2,1,4 row-size=139B cardinality=1 > | | in pipelines: 02(GETNEXT), 01(OPEN) > | | > | |--04:HASH JOIN [INNER JOIN] > | | | hash predicates: t2.c_custkey = t5.r_regionkey > | | | fk/pk conjuncts: t2.c_custkey = t5.r_regionkey > | | | runtime filters: RF004[bloom] <- t5.r_regionkey > | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB > thread-reservation=0 > | | | tuple-ids=1,4 row-size=61B cardinality=5 > | | | in pipelines: 01(GETNEXT), 03(OPEN) > | | | > | | |--03:SCAN S3 [tpch_nested_parquet.region t5] > | | | S3 partitions=1/1 files=1 size=3.59KB > | | | stored statistics: > | | | table: rows=5 size=3.59KB > | | | columns: all > | | | extrapolated-rows=disabled max-scan-range-rows=5 > | | | file formats: [PARQUET] > | | | mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=1 > | | | tuple-ids=4 row-size=2B cardinality=5 > | | | in pipelines: 03(GETNEXT) > | | | > | | 01:SCAN S3 [tpch_nested_parquet.customer t2] > | | S3 partitions=1/1 files=4 size=289.06MB > | | runtime filters: RF004[bloom] -> t2.c_custkey > | | stored statistics: > | | table: rows=150.00K size=289.06MB > | | columns missing stats: c_orders > | | extrapolated-rows=disabled max-scan-range-rows=50.28K > | | file formats: [PARQUET] > | | mem-estimate=440.00MB mem-reservation=16.00MB thread-reservation=1 > | | tuple-ids=1 row-size=59B cardinality=150.00K > | | in pipelines: 01(GETNEXT) > | | > | 02:SCAN S3 [tpch_nested_parquet.region t3] > | S3 partitions=1/1 files=1 size=3.59KB > | runtime filters: RF002[bloom] -> t3.r_comment > | stored statistics: > | table: rows=5 size=3.59KB > | columns: all > | extrapolated-rows=disabled max-scan-range-rows=5 > | file formats: [PARQUET] > | mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=1 > | tuple-ids=2 row-size=78B cardinality=5 > | in pipelines: 02(GETNEXT) > | > 00:SCAN S3 [tpch_nested_parquet.region.r_nations t1] > S3 partitions=1/1 files=1 size=3.59KB > runtime filters: RF001[min_max] -> t1.pos, RF000[bloom] -> t1.pos > stored statistics: > table: rows=5 size=3.59KB > columns: all > extrapolated-rows=disabled max-scan-range-rows=5 > file formats: [PARQUET] > mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=1 > tuple-ids=0 row-size=8B cardinality=50 > in pipelines: 00(GETNEXT) > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010)