[ https://issues.apache.org/jira/browse/IMPALA-5474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703872#comment-16703872 ]
Yongjun Zhang commented on IMPALA-5474: --------------------------------------- Thanks [Lars Volker|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=lv] for reporting the issue and [Tim Armstrong|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=tarmstrong] for the comments. I finally got to spend time on this issue. Had the following observation and understanding: Let's call "select id from functional_parquet.bad_column_metadata t" query1 and "select id, cnt from functional_parquet.bad_column_metadata t, (select 1 cnt) u;" query2. # query1 always reports ERROR, however, query2 reports WARN most of the time but it intermittently reports ERROR. That seems to indicate some race condition internally. # query1 reports ERROR because it finds out that the query is in FINISHED state instead of EXCEPTION state at [line 1037 |#L1037], thus it can reach [line 1058 |#L1058] and report ERROR there. # query2 always detects EXCEPTION state at [line 1037 |#L1037] which reports a WARN. We can see that there are three issues: # Instead of reporting WARN at 3 above, it should report an ERROR for the EXCEPTION state. # Why the two queries detect different states at [line 1037 |#L1037]? # Why query1 has intermittent behavior? We can fix this issue with this jira here. However, Issue #2 and #3 deserves some more investigation (they may just have the same root cause), for which I will create a new jira with more details. Thanks [~kwho] and [~joemcdonnell] for some good discussion. > Adding a trivial subquery turns error into warning > -------------------------------------------------- > > Key: IMPALA-5474 > URL: https://issues.apache.org/jira/browse/IMPALA-5474 > Project: IMPALA > Issue Type: Bug > Components: Backend > Affects Versions: Impala 2.9.0 > Reporter: Lars Volker > Assignee: Yongjun Zhang > Priority: Major > > This query without a subquery results in an error, which is expected. > {noformat} > [localhost:21000] > set abort_on_error=1; > ABORT_ON_ERROR set to 1 > [localhost:21000] > select id from functional_parquet.bad_column_metadata t; > Query: select id from functional_parquet.bad_column_metadata t > Query submitted at: 2017-06-08 15:30:44 (Coordinator: http://lv-desktop:25000) > Query progress can be monitored at: > http://lv-desktop:25000/query_plan?query_id=44e98719ba4c073:dec9e4d00000000 > ERROR: Column metadata states there are 11 values, but read 10 values from > column id. > file=hdfs://localhost:20500/test-warehouse/bad_column_metadata_parquet/bad_column_metadata.parquet > {noformat} > Adding a trivial subquery turns the error into a warning: > {noformat} > [localhost:21000] > set abort_on_error=1; > ABORT_ON_ERROR set to 1 > [localhost:21000] > select id, cnt from > functional_parquet.bad_column_metadata t, (select 1 cnt) u; > Query: select id, cnt from functional_parquet.bad_column_metadata t, (select > 1 cnt) u > Query submitted at: 2017-06-08 15:34:57 (Coordinator: http://lv-desktop:25000) > Query progress can be monitored at: > http://lv-desktop:25000/query_plan?query_id=7a4eb13bf147d0f1:9156b2700000000 > WARNINGS: Column metadata states there are 11 values, but read 10 values from > column id. > file=hdfs://localhost:20500/test-warehouse/bad_column_metadata_parquet/bad_column_metadata.parquet > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org