[
https://issues.apache.org/jira/browse/DRILL-7227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830888#comment-16830888
]
Robert Hou commented on DRILL-7227:
-----------------------------------
Query 59 is fixed. Queries 47 and 57 are still failing. Here is the stack
trace for query 47:
{noformat}
2019-04-30 22:42:31,964 [2336ce39-d449-3a2b-2ab4-c890128bde02:foreman] INFO
o.a.d.e.store.mock.MockStorageEngine - Took 21 ms to read statistics from
parquet format plugin
2019-04-30 22:42:32,041 [2336ce39-d449-3a2b-2ab4-c890128bde02:foreman] INFO
o.a.d.e.store.mock.MockStorageEngine - Took 12 ms to read statistics from
parquet format plugin
2019-04-30 22:42:32,154 [2336ce39-d449-3a2b-2ab4-c890128bde02:foreman] INFO
o.a.d.e.store.mock.MockStorageEngine - Took 9 ms to read statistics from
parquet format plugin
2019-04-30 22:42:32,183 [2336ce39-d449-3a2b-2ab4-c890128bde02:foreman] INFO
o.a.d.e.store.mock.MockStorageEngine - Took 9 ms to read statistics from
parquet format plugin
2019-04-30 22:42:33,705 [2336ce39-d449-3a2b-2ab4-c890128bde02:foreman] ERROR
o.a.drill.exec.work.foreman.Foreman - SYSTEM ERROR: NullPointerException
Please, refer to logs for more information.
[Error Id: 6d98ee46-91d5-4ad9-84a2-2a0903e8d977 on ucs-node3.perf.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
NullPointerException
Please, refer to logs for more information.
[Error Id: 6d98ee46-91d5-4ad9-84a2-2a0903e8d977 on ucs-node3.perf.lab:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:630)
~[drill-common-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:789)
[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
org.apache.drill.exec.work.foreman.QueryStateProcessor.checkCommonStates(QueryStateProcessor.java:325)
[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
org.apache.drill.exec.work.foreman.QueryStateProcessor.planning(QueryStateProcessor.java:221)
[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
org.apache.drill.exec.work.foreman.QueryStateProcessor.moveToState(QueryStateProcessor.java:83)
[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:304)
[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_112]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_112]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_112]
Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected
exception during fragment initialization: Error while applying rule
DrillPushProjectIntoScanRule:enumerable, args
[rel#12063:LogicalProject.NONE.ANY([]).[](input=rel#12062:Subset#2.ENUMERABLE.ANY([]).[],ss_sold_date_sk=$1,ss_sold_time_sk=$2,ss_item_sk=$3,ss_customer_sk=$4,ss_cdemo_sk=$5,ss_hdemo_sk=$6,ss_addr_sk=$7,ss_store_sk=$8,ss_promo_sk=$9,ss_ticket_number=$10,ss_quantity=$11,ss_wholesale_cost=$12,ss_list_price=$13,ss_sales_price=$14,ss_ext_discount_amt=$15,ss_ext_sales_price=$16,ss_ext_wholesale_cost=$17,ss_ext_list_price=$18,ss_ext_tax=$19,ss_coupon_amt=$20,ss_net_paid=$21,ss_net_paid_inc_tax=$22,ss_net_profit=$23),
rel#11499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[dfs,
/tpcdsParquet10/SF100/store_sales])]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:305)
[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
... 3 common frames omitted
Caused by: java.lang.RuntimeException: Error while applying rule
DrillPushProjectIntoScanRule:enumerable, args
[rel#12063:LogicalProject.NONE.ANY([]).[](input=rel#12062:Subset#2.ENUMERABLE.ANY([]).[],ss_sold_date_sk=$1,ss_sold_time_sk=$2,ss_item_sk=$3,ss_customer_sk=$4,ss_cdemo_sk=$5,ss_hdemo_sk=$6,ss_addr_sk=$7,ss_store_sk=$8,ss_promo_sk=$9,ss_ticket_number=$10,ss_quantity=$11,ss_wholesale_cost=$12,ss_list_price=$13,ss_sales_price=$14,ss_ext_discount_amt=$15,ss_ext_sales_price=$16,ss_ext_wholesale_cost=$17,ss_ext_list_price=$18,ss_ext_tax=$19,ss_coupon_amt=$20,ss_net_paid=$21,ss_net_paid_inc_tax=$22,ss_net_profit=$23),
rel#11499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[dfs,
/tpcdsParquet10/SF100/store_sales])]
at
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:236)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:643)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:339)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:430)
~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:370)
~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel(DefaultSqlHandler.java:250)
~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:319)
~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:177)
~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan(DrillSqlWorker.java:216)
~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.convertPlan(DrillSqlWorker.java:121)
~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:87)
~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:593)
[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:276)
[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
... 3 common frames omitted
Caused by: java.lang.RuntimeException: Error occurred while applying rule
DrillPushProjectIntoScanRule:enumerable
at
org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo(VolcanoRuleCall.java:149)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:234)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.drill.exec.planner.logical.DrillPushProjectIntoScanRule.onMatch(DrillPushProjectIntoScanRule.java:124)
~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:212)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
... 15 common frames omitted
Caused by: java.lang.NullPointerException: null
at
org.apache.drill.exec.planner.cost.DrillRelMdDistinctRowCount.getDistinctRowCountInternal(DrillRelMdDistinctRowCount.java:263)
~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
org.apache.drill.exec.planner.cost.DrillRelMdDistinctRowCount.getDistinctRowCount(DrillRelMdDistinctRowCount.java:117)
~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
org.apache.drill.exec.planner.cost.DrillRelMdDistinctRowCount.getDistinctRowCount(DrillRelMdDistinctRowCount.java:84)
~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount_$(Unknown Source)
~[na:na]
at
GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount(Unknown Source)
~[na:na]
at
org.apache.calcite.rel.metadata.RelMetadataQuery.getDistinctRowCount(RelMetadataQuery.java:774)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.drill.exec.planner.cost.DrillRelMdDistinctRowCount.getDistinctRowCount(DrillRelMdDistinctRowCount.java:332)
~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount_$(Unknown Source)
~[na:na]
at
GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount(Unknown Source)
~[na:na]
at
org.apache.calcite.rel.metadata.RelMetadataQuery.getDistinctRowCount(RelMetadataQuery.java:774)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.rel.metadata.RelMdDistinctRowCount.getDistinctRowCount(RelMdDistinctRowCount.java:138)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount_$(Unknown Source)
~[na:na]
at
GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount(Unknown Source)
~[na:na]
at
org.apache.calcite.rel.metadata.RelMetadataQuery.getDistinctRowCount(RelMetadataQuery.java:774)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.drill.exec.planner.cost.DrillRelMdDistinctRowCount.getDistinctRowCount(DrillRelMdDistinctRowCount.java:332)
~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount_$(Unknown Source)
~[na:na]
at
GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount(Unknown Source)
~[na:na]
at
org.apache.calcite.rel.metadata.RelMetadataQuery.getDistinctRowCount(RelMetadataQuery.java:774)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.rel.metadata.RelMdDistinctRowCount.getDistinctRowCount(RelMdDistinctRowCount.java:138)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount_$(Unknown Source)
~[na:na]
at
GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount(Unknown Source)
~[na:na]
at
org.apache.calcite.rel.metadata.RelMetadataQuery.getDistinctRowCount(RelMetadataQuery.java:774)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.drill.exec.planner.cost.DrillRelMdDistinctRowCount.getDistinctRowCount(DrillRelMdDistinctRowCount.java:332)
~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount_$(Unknown Source)
~[na:na]
at
GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount(Unknown Source)
~[na:na]
at
org.apache.calcite.rel.metadata.RelMetadataQuery.getDistinctRowCount(RelMetadataQuery.java:774)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.drill.exec.planner.common.DrillJoinRelBase.estimateRowCount(DrillJoinRelBase.java:123)
~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
org.apache.drill.exec.planner.cost.DrillRelMdRowCount.getRowCount(DrillRelMdRowCount.java:102)
~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source)
~[na:na]
at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source)
~[na:na]
at
org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:225)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:72)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source)
~[na:na]
at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source)
~[na:na]
at
org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:225)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at org.apache.calcite.rel.SingleRel.estimateRowCount(SingleRel.java:68)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.drill.exec.planner.cost.DrillRelMdRowCount.getRowCount(DrillRelMdRowCount.java:87)
~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source)
~[na:na]
at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source)
~[na:na]
at
org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:225)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.drill.exec.planner.common.DrillProjectRelBase.computeSelfCost(DrillProjectRelBase.java:77)
~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
org.apache.calcite.rel.metadata.RelMdPercentageOriginalRows.getNonCumulativeCost(RelMdPercentageOriginalRows.java:162)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
GeneratedMetadataHandler_NonCumulativeCost.getNonCumulativeCost_$(Unknown
Source) ~[na:na]
at
GeneratedMetadataHandler_NonCumulativeCost.getNonCumulativeCost(Unknown Source)
~[na:na]
at
org.apache.calcite.rel.metadata.RelMetadataQuery.getNonCumulativeCost(RelMetadataQuery.java:301)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.VolcanoPlanner.getCost(VolcanoPlanner.java:947)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:340)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:357)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:323)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.VolcanoPlanner.addRelToSet(VolcanoPlanner.java:1638)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1574)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:856)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:878)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:1750)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
at
org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo(VolcanoRuleCall.java:135)
~[calcite-core-1.18.0-drill-r0.jar:1.18.0-drill-r0]
... 18 common frames omitted
2019-04-30 22:42:38,857 [qtp713423799-8012] ERROR
o.a.d.e.s.r.profile.ProfileResources - Exception was thrown when fetching
profile .json :
{}
java.lang.IllegalArgumentException: Invalid UUID string: .json
at java.util.UUID.fromString(UUID.java:194) ~[na:1.8.0_112]
at
org.apache.drill.exec.proto.helper.QueryIdHelper.getQueryIdFromString(QueryIdHelper.java:35)
~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
org.apache.drill.exec.server.rest.profile.ProfileResources.getQueryProfile(ProfileResources.java:316)
~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at
org.apache.drill.exec.server.rest.profile.ProfileResources.getProfile(ProfileResources.java:378)
~[drill-java-exec-1.16.0.0-mapr.jar:1.16.0.0-mapr]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[na:1.8.0_112]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[na:1.8.0_112]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.8.0_112]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_112]
at
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
[jersey-server-2.25.1.jar:na]
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144)
[jersey-server-2.25.1.jar:na]
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)
[jersey-server-2.25.1.jar:na]
at
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205)
[jersey-server-2.25.1.jar:na]
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99)
[jersey-server-2.25.1.jar:na]
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389)
[jersey-server-2.25.1.jar:na]
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347)
[jersey-server-2.25.1.jar:na]
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102)
[jersey-server-2.25.1.jar:na]
at
org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326)
[jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
[jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
[jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
[jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
[jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
[jersey-common-2.25.1.jar:na]
at
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
[jersey-common-2.25.1.jar:na]
at
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
[jersey-server-2.25.1.jar:na]
at
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
[jersey-server-2.25.1.jar:na]
at
org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473)
[jersey-container-servlet-core-2.25.1.jar:na]
at
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
[jersey-container-servlet-core-2.25.1.jar:na]
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
[jersey-container-servlet-core-2.25.1.jar:na]
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
[jersey-container-servlet-core-2.25.1.jar:na]
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
[jersey-container-servlet-core-2.25.1.jar:na]
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
[jetty-servlet-9.3.25.v20180904.jar:9.3.25.v20180904]
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
[jetty-servlet-9.3.25.v20180904.jar:9.3.25.v20180904]
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[jetty-server-9.3.25.v20180904.jar:9.3.25.v20180904]
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[jetty-server-9.3.25.v20180904.jar:9.3.25.v20180904]
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:513)
[jetty-servlet-9.3.25.v20180904.jar:9.3.25.v20180904]
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[jetty-server-9.3.25.v20180904.jar:9.3.25.v20180904]
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[jetty-server-9.3.25.v20180904.jar:9.3.25.v20180904]
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[jetty-server-9.3.25.v20180904.jar:9.3.25.v20180904]
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[jetty-server-9.3.25.v20180904.jar:9.3.25.v20180904]
at org.eclipse.jetty.server.Server.handle(Server.java:539)
[jetty-server-9.3.25.v20180904.jar:9.3.25.v20180904]
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
[jetty-server-9.3.25.v20180904.jar:9.3.25.v20180904]
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[jetty-server-9.3.25.v20180904.jar:9.3.25.v20180904]
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[jetty-io-9.3.25.v20180904.jar:9.3.25.v20180904]
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[jetty-io-9.3.25.v20180904.jar:9.3.25.v20180904]
at
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[jetty-io-9.3.25.v20180904.jar:9.3.25.v20180904]
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[jetty-util-9.3.25.v20180904.jar:9.3.25.v20180904]
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[jetty-util-9.3.25.v20180904.jar:9.3.25.v20180904]
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[jetty-util-9.3.25.v20180904.jar:9.3.25.v20180904]
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[jetty-util-9.3.25.v20180904.jar:9.3.25.v20180904]
at
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[jetty-util-9.3.25.v20180904.jar:9.3.25.v20180904]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_112]
{noformat}
> TPCDS queries 47, 57, 59 fail to run with Statistics enabled at sf100
> ---------------------------------------------------------------------
>
> Key: DRILL-7227
> URL: https://issues.apache.org/jira/browse/DRILL-7227
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
> Affects Versions: 1.16.0
> Reporter: Robert Hou
> Assignee: Gautam Parai
> Priority: Major
> Fix For: 1.17.0
>
> Attachments: 23387ab0-cb1c-cd5e-449a-c9bcefc901c1.sys.drill,
> 2338ae93-155b-356d-382e-0da949c6f439.sys.drill
>
>
> Here is query 78:
> {noformat}
> WITH ws
> AS (SELECT d_year AS ws_sold_year,
> ws_item_sk,
> ws_bill_customer_sk ws_customer_sk,
> Sum(ws_quantity) ws_qty,
> Sum(ws_wholesale_cost) ws_wc,
> Sum(ws_sales_price) ws_sp
> FROM web_sales
> LEFT JOIN web_returns
> ON wr_order_number = ws_order_number
> AND ws_item_sk = wr_item_sk
> JOIN date_dim
> ON ws_sold_date_sk = d_date_sk
> WHERE wr_order_number IS NULL
> GROUP BY d_year,
> ws_item_sk,
> ws_bill_customer_sk),
> cs
> AS (SELECT d_year AS cs_sold_year,
> cs_item_sk,
> cs_bill_customer_sk cs_customer_sk,
> Sum(cs_quantity) cs_qty,
> Sum(cs_wholesale_cost) cs_wc,
> Sum(cs_sales_price) cs_sp
> FROM catalog_sales
> LEFT JOIN catalog_returns
> ON cr_order_number = cs_order_number
> AND cs_item_sk = cr_item_sk
> JOIN date_dim
> ON cs_sold_date_sk = d_date_sk
> WHERE cr_order_number IS NULL
> GROUP BY d_year,
> cs_item_sk,
> cs_bill_customer_sk),
> ss
> AS (SELECT d_year AS ss_sold_year,
> ss_item_sk,
> ss_customer_sk,
> Sum(ss_quantity) ss_qty,
> Sum(ss_wholesale_cost) ss_wc,
> Sum(ss_sales_price) ss_sp
> FROM store_sales
> LEFT JOIN store_returns
> ON sr_ticket_number = ss_ticket_number
> AND ss_item_sk = sr_item_sk
> JOIN date_dim
> ON ss_sold_date_sk = d_date_sk
> WHERE sr_ticket_number IS NULL
> GROUP BY d_year,
> ss_item_sk,
> ss_customer_sk)
> SELECT ss_item_sk,
> Round(ss_qty / ( COALESCE(ws_qty + cs_qty, 1) ), 2) ratio,
> ss_qty store_qty,
> ss_wc
> store_wholesale_cost,
> ss_sp
> store_sales_price,
> COALESCE(ws_qty, 0) + COALESCE(cs_qty, 0)
> other_chan_qty,
> COALESCE(ws_wc, 0) + COALESCE(cs_wc, 0)
> other_chan_wholesale_cost,
> COALESCE(ws_sp, 0) + COALESCE(cs_sp, 0)
> other_chan_sales_price
> FROM ss
> LEFT JOIN ws
> ON ( ws_sold_year = ss_sold_year
> AND ws_item_sk = ss_item_sk
> AND ws_customer_sk = ss_customer_sk )
> LEFT JOIN cs
> ON ( cs_sold_year = ss_sold_year
> AND cs_item_sk = cs_item_sk
> AND cs_customer_sk = ss_customer_sk )
> WHERE COALESCE(ws_qty, 0) > 0
> AND COALESCE(cs_qty, 0) > 0
> AND ss_sold_year = 1999
> ORDER BY ss_item_sk,
> ss_qty DESC,
> ss_wc DESC,
> ss_sp DESC,
> other_chan_qty,
> other_chan_wholesale_cost,
> other_chan_sales_price,
> Round(ss_qty / ( COALESCE(ws_qty + cs_qty, 1) ), 2)
> LIMIT 100;
> {noformat}
> The profile for the new plan is 2338ae93-155b-356d-382e-0da949c6f439. Hash
> partition sender operator (10-00) takes 10-15 minutes. I am not sure why it
> takes so long. It has 10 minor fragments sending to receiver (06-05), which
> has 62 minor fragments. But hash partition sender (16-00) has 10 minor
> fragments sending to receiver (12-06), which has 220 minor fragments, and
> there is no performance issue.
> The profile for the old plan is 23387ab0-cb1c-cd5e-449a-c9bcefc901c1. Both
> plans use the same commit. The old plan is created by disabling statistics.
> I have not included the plans in the Jira because Jira has a max of 32K.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
