Chun Chang created DRILL-1200:
---------------------------------
Summary: mondrian2580.q - cause schema change exception
Key: DRILL-1200
URL: https://issues.apache.org/jira/browse/DRILL-1200
Project: Apache Drill
Issue Type: Bug
Components: Execution - Flow
Reporter: Chun Chang
#Mon Jul 21 10:24:21 PDT 2014
git.commit.id.abbrev=e5c2da0
The following mondrian query (query2580.q) caused schema change exception:
0: jdbc:drill:schema=dfs> select time_by_day.the_year as c0,
sum(sales_fact_1997.unit_sales) as m0, sum(sales_fact_1997.store_cost) as m1,
sum(sales_fact_1997.store_sales) as m2, count(sales_fact_1997.product_id) as
m3, count(distinct sales_fact_1997.customer_id) as m4, sum((case when
sales_fact_1997.promotion_id = 0 then 0 else sales_fact_1997.store_sales end))
as m5 from time_by_day as time_by_day, sales_fact_1997 as sales_fact_1997 where
sales_fact_1997.time_id = time_by_day.time_id and time_by_day.the_year = 1997
group by time_by_day.the_year;
Query failed: org.apache.drill.exec.rpc.RpcException: Remote failure while
running query.[error_id: "b9fa1177-505a-4ae1-9d44-52b1d40f9a92"
endpoint {
address: "qa-node120.qa.lab"
user_port: 31010
control_port: 31011
data_port: 31012
}
error_type: 0
message: "Failure while running fragment. < IllegalStateException:[ You tried
to do a batch data read operation when you were in a state of STOP. You can
only do this type of operation when you are in a state of OK or OK_NEW_SCHEMA.
]"
]
Error: exception while executing query (state=,code=0)
physical plan:
14:31:16.198 [19aa0bca-d59e-4061-962d-b3fabb0f24b7:foreman] DEBUG
o.a.d.e.p.s.h.DefaultSqlHandler - Drill Physical :
00-00 Screen: rowcount = 1.0, cumulative cost = {874658.1 rows, 3052726.1
cpu, 0.0 io, 0.0 network}, id = 3109
00-01 Project(c0=[$0], m0=[$1], m1=[$2], m2=[$3], m3=[$4], m4=[$5],
m5=[$6]): rowcount = 1.0, cumulative cost = {874658.0 rows, 3052726.0 cpu, 0.0
io, 0.0 network}, id = 3108
00-02 Project(c0=[$0], m0=[CASE(=($2, 0), null, $1)], m1=[CASE(=($4, 0),
null, $3)], m2=[CASE(=($6, 0), null, $5)], m3=[$7], m4=[$11], m5=[CASE(=($9,
0), null, $8)]): rowcount = 1.0, cumulative cost = {874657.0 rows, 3052698.0
cpu, 0.0 io, 0.0 network}, id = 3107
00-03 HashJoin(condition=[IS NOT DISTINCT FROM($0, $10)],
joinType=[inner]): rowcount = 1.0, cumulative cost = {874656.0 rows, 3052670.0
cpu, 0.0 io, 0.0 network}, id = 3106
00-05 HashAgg(group=[{0}], agg#0=[$SUM0($1)], agg#1=[COUNT($1)],
agg#2=[$SUM0($2)], agg#3=[COUNT($2)], agg#4=[$SUM0($3)], agg#5=[COUNT($3)],
m3=[COUNT($4)], agg#7=[$SUM0($6)], agg#8=[COUNT($6)]): rowcount = 1.0,
cumulative cost = {437326.0 rows, 1743481.5 cpu, 0.0 io, 0.0 network}, id = 3093
00-07 Project(c0=[$0], unit_sales=[$3], store_cost=[$4],
store_sales=[$5], product_id=[$6], customer_id=[$7],
$f6=[CASE(=(CAST($8):INTEGER, 0), CAST(0):ANY, $5)]): rowcount = 1.0,
cumulative cost = {437325.0 rows, 1743365.5 cpu, 0.0 io, 0.0 network}, id = 3092
00-09 HashJoin(condition=[=($2, $1)], joinType=[inner]):
rowcount = 1.0, cumulative cost = {437324.0 rows, 1743337.5 cpu, 0.0 io, 0.0
network}, id = 3091
00-12 SelectionVectorRemover: rowcount = 109.5, cumulative
cost = {3029.5 rows, 5227.5 cpu, 0.0 io, 0.0 network}, id = 3087
00-15 Filter(condition=[=(CAST($0):INTEGER, 1997)]):
rowcount = 109.5, cumulative cost = {2920.0 rows, 5118.0 cpu, 0.0 io, 0.0
network}, id = 3086
00-19 Project(the_year=[$1], time_id=[$0]): rowcount =
730.0, cumulative cost = {2190.0 rows, 2198.0 cpu, 0.0 io, 0.0 network}, id =
3085
00-23 ProducerConsumer: rowcount = 730.0, cumulative
cost = {1460.0 rows, 2190.0 cpu, 0.0 io, 0.0 network}, id = 3084
00-26 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=maprfs:/user/root/mondrian/time_by_day]],
selectionRoot=/user/root/mondrian/time_by_day, columns=[SchemaPath
[`the_year`], SchemaPath [`time_id`]]]]): rowcount = 730.0, cumulative cost =
{730.0 rows, 1460.0 cpu, 0.0 io, 0.0 network}, id = 3058
00-11 Project(time_id0=[$0], unit_sales=[$1], store_cost=[$2],
store_sales=[$3], product_id=[$4], customer_id=[$5], promotion_id=[$6]):
rowcount = 86837.0, cumulative cost = {347348.0 rows, 694752.0 cpu, 0.0 io, 0.0
network}, id = 3090
00-14 Project(time_id=[$4], unit_sales=[$1],
store_cost=[$5], store_sales=[$6], product_id=[$2], customer_id=[$0],
promotion_id=[$3]): rowcount = 86837.0, cumulative cost = {260511.0 rows,
694724.0 cpu, 0.0 io, 0.0 network}, id = 3089
00-18 ProducerConsumer: rowcount = 86837.0, cumulative
cost = {173674.0 rows, 694696.0 cpu, 0.0 io, 0.0 network}, id = 3088
00-22 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=maprfs:/user/root/mondrian/sales_fact_1997]],
selectionRoot=/user/root/mondrian/sales_fact_1997, columns=[SchemaPath
[`time_id`], SchemaPath [`unit_sales`], SchemaPath [`store_cost`], SchemaPath
[`store_sales`], SchemaPath [`product_id`], SchemaPath [`customer_id`],
SchemaPath [`promotion_id`]]]]): rowcount = 86837.0, cumulative cost = {86837.0
rows, 607859.0 cpu, 0.0 io, 0.0 network}, id = 3062
00-04 Project(c00=[$0], m4=[$1]): rowcount = 1.0, cumulative cost =
{437328.0 rows, 1309164.5 cpu, 0.0 io, 0.0 network}, id = 3105
00-06 HashAgg(group=[{0}], m4=[COUNT($1)]): rowcount = 1.0,
cumulative cost = {437327.0 rows, 1309156.5 cpu, 0.0 io, 0.0 network}, id = 3104
00-08 HashAgg(group=[{0, 1}]): rowcount = 1.0, cumulative cost =
{437326.0 rows, 1309136.5 cpu, 0.0 io, 0.0 network}, id = 3103
00-10 Project(c0=[$0], customer_id=[$3]): rowcount = 1.0,
cumulative cost = {437325.0 rows, 1309120.5 cpu, 0.0 io, 0.0 network}, id = 3102
00-13 HashJoin(condition=[=($2, $1)], joinType=[inner]):
rowcount = 1.0, cumulative cost = {437324.0 rows, 1309112.5 cpu, 0.0 io, 0.0
network}, id = 3101
00-17 SelectionVectorRemover: rowcount = 109.5, cumulative
cost = {3029.5 rows, 5227.5 cpu, 0.0 io, 0.0 network}, id = 3097
00-21 Filter(condition=[=(CAST($0):INTEGER, 1997)]):
rowcount = 109.5, cumulative cost = {2920.0 rows, 5118.0 cpu, 0.0 io, 0.0
network}, id = 3096
00-25 Project(the_year=[$1], time_id=[$0]): rowcount =
730.0, cumulative cost = {2190.0 rows, 2198.0 cpu, 0.0 io, 0.0 network}, id =
3095
00-28 ProducerConsumer: rowcount = 730.0, cumulative
cost = {1460.0 rows, 2190.0 cpu, 0.0 io, 0.0 network}, id = 3094
00-29 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=maprfs:/user/root/mondrian/time_by_day]],
selectionRoot=/user/root/mondrian/time_by_day, columns=[SchemaPath
[`the_year`], SchemaPath [`time_id`]]]]): rowcount = 730.0, cumulative cost =
{730.0 rows, 1460.0 cpu, 0.0 io, 0.0 network}, id = 3068
00-16 Project(time_id0=[$0], customer_id=[$1]): rowcount =
86837.0, cumulative cost = {347348.0 rows, 260527.0 cpu, 0.0 io, 0.0 network},
id = 3100
00-20 Project(time_id=[$1], customer_id=[$0]): rowcount
= 86837.0, cumulative cost = {260511.0 rows, 260519.0 cpu, 0.0 io, 0.0
network}, id = 3099
00-24 ProducerConsumer: rowcount = 86837.0, cumulative
cost = {173674.0 rows, 260511.0 cpu, 0.0 io, 0.0 network}, id = 3098
00-27 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=maprfs:/user/root/mondrian/sales_fact_1997]],
selectionRoot=/user/root/mondrian/sales_fact_1997, columns=[SchemaPath
[`time_id`], SchemaPath [`customer_id`]]]]): rowcount = 86837.0, cumulative
cost = {86837.0 rows, 173674.0 cpu, 0.0 io, 0.0 network}, id = 3072
14:31:16.352 [19aa0bca-d59e-4061-962d-b3fabb0f24b7:foreman] DEBUG
o.a.d.e.p.s.h.DefaultSqlHandler - Drill Plan :
{
"head" : {
"version" : 1,
"generator" : {
"type" : "DefaultSqlHandler",
"info" : ""
},
"type" : "APACHE_DRILL_PHYSICAL",
"options" : [ ],
"queue" : 0,
"resultMode" : "EXEC"
},
"graph" : [ {
"pop" : "parquet-scan",
"@id" : 27,
"entries" : [ {
"path" : "maprfs:/user/root/mondrian/sales_fact_1997"
} ],
"storage" : {
"type" : "file",
"enabled" : true,
"connection" : "maprfs:///",
"workspaces" : {
"default" : {
"location" : "/user/root/mondrian/",
"writable" : false,
"storageformat" : null
},
"home" : {
"location" : "/",
"writable" : false,
"storageformat" : null
},
"root" : {
"location" : "/",
"writable" : false,
"storageformat" : null
},
"tmp" : {
"location" : "/tmp",
"writable" : true,
"storageformat" : "csv"
}
},
"formats" : {
"psv" : {
"type" : "text",
"extensions" : [ "tbl" ],
"delimiter" : "|"
},
"csv" : {
"type" : "text",
"extensions" : [ "csv" ],
"delimiter" : ","
},
"tsv" : {
"type" : "text",
"extensions" : [ "tsv" ],
"delimiter" : "\t"
},
"parquet" : {
"type" : "parquet"
},
"json" : {
"type" : "json"
}
}
},
"format" : {
"type" : "parquet"
},
"columns" : [ "`time_id`", "`customer_id`" ],
"selectionRoot" : "/user/root/mondrian/sales_fact_1997",
"cost" : 86837.0
}, {
"pop" : "producer-consumer",
"@id" : 24,
"child" : 27,
"size" : 10,
"initialAllocation" : 1000000,
"maxAllocation" : 10000000000,
"cost" : 86837.0
}, {
"pop" : "project",
"@id" : 20,
"exprs" : [ {
"ref" : "`time_id`",
"expr" : "`time_id`"
}, {
"ref" : "`customer_id`",
"expr" : "`customer_id`"
} ],
"child" : 24,
"initialAllocation" : 1000000,
"maxAllocation" : 10000000000,
"cost" : 86837.0
}, {
"pop" : "project",
"@id" : 16,
"exprs" : [ {
"ref" : "`time_id0`",
"expr" : "`time_id`"
}, {
"ref" : "`customer_id`",
"expr" : "`customer_id`"
} ],
"child" : 20,
"initialAllocation" : 1000000,
"maxAllocation" : 10000000000,
"cost" : 86837.0
}, {
"pop" : "parquet-scan",
"@id" : 22,
"entries" : [ {
"path" : "maprfs:/user/root/mondrian/sales_fact_1997"
} ],
"storage" : {
"type" : "file",
"enabled" : true,
"connection" : "maprfs:///",
"workspaces" : {
"default" : {
"location" : "/user/root/mondrian/",
"writable" : false,
"storageformat" : null
},
"home" : {
"location" : "/",
"writable" : false,
"storageformat" : null
},
"root" : {
"location" : "/",
"writable" : false,
"storageformat" : null
},
"tmp" : {
"location" : "/tmp",
"writable" : true,
"storageformat" : "csv"
}
},
"formats" : {
"psv" : {
"type" : "text",
"extensions" : [ "tbl" ],
"delimiter" : "|"
},
"csv" : {
"type" : "text",
"extensions" : [ "csv" ],
"delimiter" : ","
},
"tsv" : {
"type" : "text",
"extensions" : [ "tsv" ],
"delimiter" : "\t"
},
"parquet" : {
"type" : "parquet"
},
"json" : {
"type" : "json"
}
}
},
"format" : {
"type" : "parquet"
},
"columns" : [ "`time_id`", "`unit_sales`", "`store_cost`", "`store_sales`",
"`product_id`", "`customer_id`", "`promotion_id`" ],
"selectionRoot" : "/user/root/mondrian/sales_fact_1997",
"cost" : 86837.0
}, {
"pop" : "producer-consumer",
"@id" : 18,
"child" : 22,
"size" : 10,
"initialAllocation" : 1000000,
"maxAllocation" : 10000000000,
"cost" : 86837.0
}, {
"pop" : "project",
"@id" : 14,
"exprs" : [ {
"ref" : "`time_id`",
"expr" : "`time_id`"
}, {
"ref" : "`unit_sales`",
"expr" : "`unit_sales`"
}, {
"ref" : "`store_cost`",
"expr" : "`store_cost`"
}, {
"ref" : "`store_sales`",
"expr" : "`store_sales`"
}, {
"ref" : "`product_id`",
"expr" : "`product_id`"
}, {
"ref" : "`customer_id`",
"expr" : "`customer_id`"
}, {
"ref" : "`promotion_id`",
"expr" : "`promotion_id`"
} ],
"child" : 18,
"initialAllocation" : 1000000,
"maxAllocation" : 10000000000,
"cost" : 86837.0
}, {
"pop" : "project",
"@id" : 11,
"exprs" : [ {
"ref" : "`time_id0`",
"expr" : "`time_id`"
}, {
"ref" : "`unit_sales`",
"expr" : "`unit_sales`"
}, {
"ref" : "`store_cost`",
"expr" : "`store_cost`"
}, {
"ref" : "`store_sales`",
"expr" : "`store_sales`"
}, {
"ref" : "`product_id`",
"expr" : "`product_id`"
}, {
"ref" : "`customer_id`",
"expr" : "`customer_id`"
}, {
"ref" : "`promotion_id`",
"expr" : "`promotion_id`"
} ],
"child" : 14,
"initialAllocation" : 1000000,
"maxAllocation" : 10000000000,
"cost" : 86837.0
}, {
"pop" : "parquet-scan",
"@id" : 26,
"entries" : [ {
"path" : "maprfs:/user/root/mondrian/time_by_day"
} ],
"storage" : {
"type" : "file",
"enabled" : true,
"connection" : "maprfs:///",
"workspaces" : {
"default" : {
"location" : "/user/root/mondrian/",
"writable" : false,
"storageformat" : null
},
"home" : {
"location" : "/",
"writable" : false,
"storageformat" : null
},
"root" : {
"location" : "/",
"writable" : false,
"storageformat" : null
},
"tmp" : {
"location" : "/tmp",
"writable" : true,
"storageformat" : "csv"
}
},
"formats" : {
"psv" : {
"type" : "text",
"extensions" : [ "tbl" ],
"delimiter" : "|"
},
"csv" : {
"type" : "text",
"extensions" : [ "csv" ],
"delimiter" : ","
},
"tsv" : {
"type" : "text",
"extensions" : [ "tsv" ],
"delimiter" : "\t"
},
"parquet" : {
"type" : "parquet"
},
"json" : {
"type" : "json"
}
}
},
"format" : {
"type" : "parquet"
},
"columns" : [ "`the_year`", "`time_id`" ],
"selectionRoot" : "/user/root/mondrian/time_by_day",
"cost" : 730.0
}, {
"pop" : "producer-consumer",
"@id" : 23,
"child" : 26,
"size" : 10,
"initialAllocation" : 1000000,
"maxAllocation" : 10000000000,
"cost" : 730.0
}, {
"pop" : "project",
"@id" : 19,
"exprs" : [ {
"ref" : "`the_year`",
"expr" : "`the_year`"
}, {
"ref" : "`time_id`",
"expr" : "`time_id`"
} ],
"child" : 23,
"initialAllocation" : 1000000,
"maxAllocation" : 10000000000,
"cost" : 730.0
}, {
"pop" : "filter",
"@id" : 15,
"child" : 19,
"expr" : "equal(cast( (`the_year` ) as INT ), 1997) ",
"initialAllocation" : 1000000,
"maxAllocation" : 10000000000,
"cost" : 109.5
}, {
"pop" : "selection-vector-remover",
"@id" : 12,
"child" : 15,
"initialAllocation" : 1000000,
"maxAllocation" : 10000000000,
"cost" : 109.5
}, {
"pop" : "hash-join",
"@id" : 9,
"left" : 12,
"right" : 11,
"conditions" : [ {
"relationship" : "==",
"left" : "`time_id`",
"right" : "`time_id0`"
} ],
"joinType" : "INNER",
"initialAllocation" : 1000000,
"maxAllocation" : 10000000000,
"cost" : 0.0
}, {
"pop" : "project",
"@id" : 7,
"exprs" : [ {
"ref" : "`c0`",
"expr" : "`the_year`"
}, {
"ref" : "`unit_sales`",
"expr" : "`unit_sales`"
}, {
"ref" : "`store_cost`",
"expr" : "`store_cost`"
}, {
"ref" : "`store_sales`",
"expr" : "`store_sales`"
}, {
"ref" : "`product_id`",
"expr" : "`product_id`"
}, {
"ref" : "`customer_id`",
"expr" : "`customer_id`"
}, {
"ref" : "`$f6`",
"expr" : " ( if (equal(cast( (`promotion_id` ) as INT ), 0) ) then (0 )
else (`store_sales` ) end ) "
} ],
"child" : 9,
"initialAllocation" : 1000000,
"maxAllocation" : 10000000000,
"cost" : 1.0
}, {
"pop" : "hash-aggregate",
"@id" : 5,
"child" : 7,
"cardinality" : 1.0,
"initialAllocation" : 1000000,
"maxAllocation" : 10000000000,
"cost" : 0.5,
"groupByExprs" : [ {
"ref" : "`c0`",
"expr" : "`c0`"
} ],
"aggrExprs" : [ {
"ref" : "`$f1`",
"expr" : "$sum0(`unit_sales`) "
}, {
"ref" : "`$f2`",
"expr" : "count(`unit_sales`) "
}, {
"ref" : "`$f3`",
"expr" : "$sum0(`store_cost`) "
}, {
"ref" : "`$f4`",
"expr" : "count(`store_cost`) "
}, {
"ref" : "`$f5`",
"expr" : "$sum0(`store_sales`) "
}, {
"ref" : "`$f6`",
"expr" : "count(`store_sales`) "
}, {
"ref" : "`m3`",
"expr" : "count(`product_id`) "
}, {
"ref" : "`$f8`",
"expr" : "$sum0(`$f6`) "
}, {
"ref" : "`$f9`",
"expr" : "count(`$f6`) "
} ]
}, {
"pop" : "parquet-scan",
"@id" : 29,
"entries" : [ {
"path" : "maprfs:/user/root/mondrian/time_by_day"
} ],
"storage" : {
"type" : "file",
"enabled" : true,
"connection" : "maprfs:///",
"workspaces" : {
"default" : {
"location" : "/user/root/mondrian/",
"writable" : false,
....
This is too long. Will regenerate the plan if needed.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
