[ https://issues.apache.org/jira/browse/DRILL-7227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830888#comment-16830888 ]
Robert Hou commented on DRILL-7227: ----------------------------------- Query 59 is fixed. Queries 47 and 57 are still failing. Here is the stack trace for query 47: {noformat} 2019-04-30 22:42:31,964 [2336ce39-d449-3a2b-2ab4-c890128bde02:foreman] INFO o.a.d.e.store.mock.MockStorageEngine - Took 21 ms to read statistics from parquet format plugin 2019-04-30 22:42:32,041 [2336ce39-d449-3a2b-2ab4-c890128bde02:foreman] INFO o.a.d.e.store.mock.MockStorageEngine - Took 12 ms to read statistics from parquet format plugin 2019-04-30 22:42:32,154 [2336ce39-d449-3a2b-2ab4-c890128bde02:foreman] INFO o.a.d.e.store.mock.MockStorageEngine - Took 9 ms to read statistics from parquet format plugin 2019-04-30 22:42:32,183 [2336ce39-d449-3a2b-2ab4-c890128bde02:foreman] INFO o.a.d.e.store.mock.MockStorageEngine - Took 9 ms to read statistics from parquet format plugin 2019-04-30 22:42:33,705 [2336ce39-d449-3a2b-2ab4-c890128bde02:foreman] ERROR o.a.drill.exec.work.foreman.Foreman - SYSTEM ERROR: NullPointerException Please, refer to logs for more information. [Error Id: 6d98ee46-91d5-4ad9-84a2-2a0903e8d977 on ucs-node3.perf.lab:31010] org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: NullPointerException Please, refer to logs for more information. [Error Id: 6d98ee46-91d5-4ad9-84a2-2a0903e8d977 on ucs-node3.perf.lab:31010] at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:630) ~[drill-common-1.16.0.0-mapr.jar:1.16.0.0-mapr] at org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:789) [drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at org.apache.drill.exec.work.foreman.QueryStateProcessor.checkCommonStates(QueryStateProcessor.java:325) [drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at org.apache.drill.exec.work.foreman.QueryStateProcessor.planning(QueryStateProcessor.java:221) [drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at org.apache.drill.exec.work.foreman.QueryStateProcessor.moveToState(QueryStateProcessor.java:83) [drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:304) [drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_112] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_112] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_112] Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception during fragment initialization: Error while applying rule DrillPushProjectIntoScanRule:enumerable, args [rel#12063:LogicalProject.NONE.ANY([]).[](input=rel#12062:Subset#2.ENUMERABLE.ANY([]).[],ss_sold_date_sk=$1,ss_sold_time_sk=$2,ss_item_sk=$3,ss_customer_sk=$4,ss_cdemo_sk=$5,ss_hdemo_sk=$6,ss_addr_sk=$7,ss_store_sk=$8,ss_promo_sk=$9,ss_ticket_number=$10,ss_quantity=$11,ss_wholesale_cost=$12,ss_list_price=$13,ss_sales_price=$14,ss_ext_discount_amt=$15,ss_ext_sales_price=$16,ss_ext_wholesale_cost=$17,ss_ext_list_price=$18,ss_ext_tax=$19,ss_coupon_amt=$20,ss_net_paid=$21,ss_net_paid_inc_tax=$22,ss_net_profit=$23), rel#11499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[dfs, /tpcdsParquet10/SF100/store_sales])] at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:305) [drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] ... 3 common frames omitted Caused by: java.lang.RuntimeException: Error while applying rule DrillPushProjectIntoScanRule:enumerable, args [rel#12063:LogicalProject.NONE.ANY([]).[](input=rel#12062:Subset#2.ENUMERABLE.ANY([]).[],ss_sold_date_sk=$1,ss_sold_time_sk=$2,ss_item_sk=$3,ss_customer_sk=$4,ss_cdemo_sk=$5,ss_hdemo_sk=$6,ss_addr_sk=$7,ss_store_sk=$8,ss_promo_sk=$9,ss_ticket_number=$10,ss_quantity=$11,ss_wholesale_cost=$12,ss_list_price=$13,ss_sales_price=$14,ss_ext_discount_amt=$15,ss_ext_sales_price=$16,ss_ext_wholesale_cost=$17,ss_ext_list_price=$18,ss_ext_tax=$19,ss_coupon_amt=$20,ss_net_paid=$21,ss_net_paid_inc_tax=$22,ss_net_profit=$23), rel#11499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[dfs, /tpcdsParquet10/SF100/store_sales])] at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:236) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:643) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:339) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:430) ~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:370) ~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel(DefaultSqlHandler.java:250) ~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:319) ~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:177) ~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan(DrillSqlWorker.java:216) ~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at org.apache.drill.exec.planner.sql.DrillSqlWorker.convertPlan(DrillSqlWorker.java:121) ~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:87) ~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:593) [drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:276) [drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] ... 3 common frames omitted Caused by: java.lang.RuntimeException: Error occurred while applying rule DrillPushProjectIntoScanRule:enumerable at org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo(VolcanoRuleCall.java:149) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:234) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.drill.exec.planner.logical.DrillPushProjectIntoScanRule.onMatch(DrillPushProjectIntoScanRule.java:124) ~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:212) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] ... 15 common frames omitted Caused by: java.lang.NullPointerException: null at org.apache.drill.exec.planner.cost.DrillRelMdDistinctRowCount.getDistinctRowCountInternal(DrillRelMdDistinctRowCount.java:263) ~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at org.apache.drill.exec.planner.cost.DrillRelMdDistinctRowCount.getDistinctRowCount(DrillRelMdDistinctRowCount.java:117) ~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at org.apache.drill.exec.planner.cost.DrillRelMdDistinctRowCount.getDistinctRowCount(DrillRelMdDistinctRowCount.java:84) ~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount_$(Unknown Source) ~[na:na] at GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount(Unknown Source) ~[na:na] at org.apache.calcite.rel.metadata.RelMetadataQuery.getDistinctRowCount(RelMetadataQuery.java:774) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.drill.exec.planner.cost.DrillRelMdDistinctRowCount.getDistinctRowCount(DrillRelMdDistinctRowCount.java:332) ~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount_$(Unknown Source) ~[na:na] at GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount(Unknown Source) ~[na:na] at org.apache.calcite.rel.metadata.RelMetadataQuery.getDistinctRowCount(RelMetadataQuery.java:774) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.rel.metadata.RelMdDistinctRowCount.getDistinctRowCount(RelMdDistinctRowCount.java:138) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount_$(Unknown Source) ~[na:na] at GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount(Unknown Source) ~[na:na] at org.apache.calcite.rel.metadata.RelMetadataQuery.getDistinctRowCount(RelMetadataQuery.java:774) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.drill.exec.planner.cost.DrillRelMdDistinctRowCount.getDistinctRowCount(DrillRelMdDistinctRowCount.java:332) ~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount_$(Unknown Source) ~[na:na] at GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount(Unknown Source) ~[na:na] at org.apache.calcite.rel.metadata.RelMetadataQuery.getDistinctRowCount(RelMetadataQuery.java:774) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.rel.metadata.RelMdDistinctRowCount.getDistinctRowCount(RelMdDistinctRowCount.java:138) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount_$(Unknown Source) ~[na:na] at GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount(Unknown Source) ~[na:na] at org.apache.calcite.rel.metadata.RelMetadataQuery.getDistinctRowCount(RelMetadataQuery.java:774) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.drill.exec.planner.cost.DrillRelMdDistinctRowCount.getDistinctRowCount(DrillRelMdDistinctRowCount.java:332) ~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount_$(Unknown Source) ~[na:na] at GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount(Unknown Source) ~[na:na] at org.apache.calcite.rel.metadata.RelMetadataQuery.getDistinctRowCount(RelMetadataQuery.java:774) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.drill.exec.planner.common.DrillJoinRelBase.estimateRowCount(DrillJoinRelBase.java:123) ~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at org.apache.drill.exec.planner.cost.DrillRelMdRowCount.getRowCount(DrillRelMdRowCount.java:102) ~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source) ~[na:na] at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source) ~[na:na] at org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:225) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:72) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source) ~[na:na] at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source) ~[na:na] at org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:225) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.rel.SingleRel.estimateRowCount(SingleRel.java:68) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.drill.exec.planner.cost.DrillRelMdRowCount.getRowCount(DrillRelMdRowCount.java:87) ~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source) ~[na:na] at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source) ~[na:na] at org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:225) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.drill.exec.planner.common.DrillProjectRelBase.computeSelfCost(DrillProjectRelBase.java:77) ~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at org.apache.calcite.rel.metadata.RelMdPercentageOriginalRows.getNonCumulativeCost(RelMdPercentageOriginalRows.java:162) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at GeneratedMetadataHandler_NonCumulativeCost.getNonCumulativeCost_$(Unknown Source) ~[na:na] at GeneratedMetadataHandler_NonCumulativeCost.getNonCumulativeCost(Unknown Source) ~[na:na] at org.apache.calcite.rel.metadata.RelMetadataQuery.getNonCumulativeCost(RelMetadataQuery.java:301) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.VolcanoPlanner.getCost(VolcanoPlanner.java:947) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:340) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.VolcanoPlanner.addRelToSet(VolcanoPlanner.java:1638) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1574) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:856) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:878) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:1750) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] at org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo(VolcanoRuleCall.java:135) ~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0] ... 18 common frames omitted 2019-04-30 22:42:38,857 [qtp713423799-8012] ERROR o.a.d.e.s.r.profile.ProfileResources - Exception was thrown when fetching profile .json : {} java.lang.IllegalArgumentException: Invalid UUID string: .json at java.util.UUID.fromString(UUID.java:194) ~[na:1.8.0_112] at org.apache.drill.exec.proto.helper.QueryIdHelper.getQueryIdFromString(QueryIdHelper.java:35) ~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at org.apache.drill.exec.server.rest.profile.ProfileResources.getQueryProfile(ProfileResources.java:316) ~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at org.apache.drill.exec.server.rest.profile.ProfileResources.getProfile(ProfileResources.java:378) ~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_112] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_112] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_112] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_112] at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) [jersey-server-2.25.1.jar:na] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) [jersey-server-2.25.1.jar:na] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) [jersey-server-2.25.1.jar:na] at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) [jersey-server-2.25.1.jar:na] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) [jersey-server-2.25.1.jar:na] at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) [jersey-server-2.25.1.jar:na] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) [jersey-server-2.25.1.jar:na] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) [jersey-server-2.25.1.jar:na] at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [jersey-server-2.25.1.jar:na] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [jersey-common-2.25.1.jar:na] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [jersey-common-2.25.1.jar:na] at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [jersey-common-2.25.1.jar:na] at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [jersey-common-2.25.1.jar:na] at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [jersey-common-2.25.1.jar:na] at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [jersey-common-2.25.1.jar:na] at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [jersey-server-2.25.1.jar:na] at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [jersey-server-2.25.1.jar:na] at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) [jersey-container-servlet-core-2.25.1.jar:na] at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) [jersey-container-servlet-core-2.25.1.jar:na] at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) [jersey-container-servlet-core-2.25.1.jar:na] at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) [jersey-container-servlet-core-2.25.1.jar:na] at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) [jersey-container-servlet-core-2.25.1.jar:na] at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848) [jetty-servlet-9.3.25.v20180904.jar:9.3.25.v20180904] at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) [jetty-servlet-9.3.25.v20180904.jar:9.3.25.v20180904] at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) [jetty-server-9.3.25.v20180904.jar:9.3.25.v20180904] at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) [jetty-server-9.3.25.v20180904.jar:9.3.25.v20180904] at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:513) [jetty-servlet-9.3.25.v20180904.jar:9.3.25.v20180904] at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) [jetty-server-9.3.25.v20180904.jar:9.3.25.v20180904] at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) [jetty-server-9.3.25.v20180904.jar:9.3.25.v20180904] at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [jetty-server-9.3.25.v20180904.jar:9.3.25.v20180904] at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) [jetty-server-9.3.25.v20180904.jar:9.3.25.v20180904] at org.eclipse.jetty.server.Server.handle(Server.java:539) [jetty-server-9.3.25.v20180904.jar:9.3.25.v20180904] at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333) [jetty-server-9.3.25.v20180904.jar:9.3.25.v20180904] at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) [jetty-server-9.3.25.v20180904.jar:9.3.25.v20180904] at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) [jetty-io-9.3.25.v20180904.jar:9.3.25.v20180904] at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) [jetty-io-9.3.25.v20180904.jar:9.3.25.v20180904] at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) [jetty-io-9.3.25.v20180904.jar:9.3.25.v20180904] at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) [jetty-util-9.3.25.v20180904.jar:9.3.25.v20180904] at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) [jetty-util-9.3.25.v20180904.jar:9.3.25.v20180904] at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) [jetty-util-9.3.25.v20180904.jar:9.3.25.v20180904] at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) [jetty-util-9.3.25.v20180904.jar:9.3.25.v20180904] at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) [jetty-util-9.3.25.v20180904.jar:9.3.25.v20180904] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_112] {noformat} > TPCDS queries 47, 57, 59 fail to run with Statistics enabled at sf100 > --------------------------------------------------------------------- > > Key: DRILL-7227 > URL: https://issues.apache.org/jira/browse/DRILL-7227 > Project: Apache Drill > Issue Type: Bug > Components: Metadata > Affects Versions: 1.16.0 > Reporter: Robert Hou > Assignee: Gautam Parai > Priority: Major > Fix For: 1.17.0 > > Attachments: 23387ab0-cb1c-cd5e-449a-c9bcefc901c1.sys.drill, > 2338ae93-155b-356d-382e-0da949c6f439.sys.drill > > > Here is query 78: > {noformat} > WITH ws > AS (SELECT d_year AS ws_sold_year, > ws_item_sk, > ws_bill_customer_sk ws_customer_sk, > Sum(ws_quantity) ws_qty, > Sum(ws_wholesale_cost) ws_wc, > Sum(ws_sales_price) ws_sp > FROM web_sales > LEFT JOIN web_returns > ON wr_order_number = ws_order_number > AND ws_item_sk = wr_item_sk > JOIN date_dim > ON ws_sold_date_sk = d_date_sk > WHERE wr_order_number IS NULL > GROUP BY d_year, > ws_item_sk, > ws_bill_customer_sk), > cs > AS (SELECT d_year AS cs_sold_year, > cs_item_sk, > cs_bill_customer_sk cs_customer_sk, > Sum(cs_quantity) cs_qty, > Sum(cs_wholesale_cost) cs_wc, > Sum(cs_sales_price) cs_sp > FROM catalog_sales > LEFT JOIN catalog_returns > ON cr_order_number = cs_order_number > AND cs_item_sk = cr_item_sk > JOIN date_dim > ON cs_sold_date_sk = d_date_sk > WHERE cr_order_number IS NULL > GROUP BY d_year, > cs_item_sk, > cs_bill_customer_sk), > ss > AS (SELECT d_year AS ss_sold_year, > ss_item_sk, > ss_customer_sk, > Sum(ss_quantity) ss_qty, > Sum(ss_wholesale_cost) ss_wc, > Sum(ss_sales_price) ss_sp > FROM store_sales > LEFT JOIN store_returns > ON sr_ticket_number = ss_ticket_number > AND ss_item_sk = sr_item_sk > JOIN date_dim > ON ss_sold_date_sk = d_date_sk > WHERE sr_ticket_number IS NULL > GROUP BY d_year, > ss_item_sk, > ss_customer_sk) > SELECT ss_item_sk, > Round(ss_qty / ( COALESCE(ws_qty + cs_qty, 1) ), 2) ratio, > ss_qty store_qty, > ss_wc > store_wholesale_cost, > ss_sp > store_sales_price, > COALESCE(ws_qty, 0) + COALESCE(cs_qty, 0) > other_chan_qty, > COALESCE(ws_wc, 0) + COALESCE(cs_wc, 0) > other_chan_wholesale_cost, > COALESCE(ws_sp, 0) + COALESCE(cs_sp, 0) > other_chan_sales_price > FROM ss > LEFT JOIN ws > ON ( ws_sold_year = ss_sold_year > AND ws_item_sk = ss_item_sk > AND ws_customer_sk = ss_customer_sk ) > LEFT JOIN cs > ON ( cs_sold_year = ss_sold_year > AND cs_item_sk = cs_item_sk > AND cs_customer_sk = ss_customer_sk ) > WHERE COALESCE(ws_qty, 0) > 0 > AND COALESCE(cs_qty, 0) > 0 > AND ss_sold_year = 1999 > ORDER BY ss_item_sk, > ss_qty DESC, > ss_wc DESC, > ss_sp DESC, > other_chan_qty, > other_chan_wholesale_cost, > other_chan_sales_price, > Round(ss_qty / ( COALESCE(ws_qty + cs_qty, 1) ), 2) > LIMIT 100; > {noformat} > The profile for the new plan is 2338ae93-155b-356d-382e-0da949c6f439. Hash > partition sender operator (10-00) takes 10-15 minutes. I am not sure why it > takes so long. It has 10 minor fragments sending to receiver (06-05), which > has 62 minor fragments. But hash partition sender (16-00) has 10 minor > fragments sending to receiver (12-06), which has 220 minor fragments, and > there is no performance issue. > The profile for the old plan is 23387ab0-cb1c-cd5e-449a-c9bcefc901c1. Both > plans use the same commit. The old plan is created by disabling statistics. > I have not included the plans in the Jira because Jira has a max of 32K. -- This message was sent by Atlassian JIRA (v7.6.3#76005)