[jira] [Commented] (IMPALA-8007) test_slow_subscriber is flaky
[
https://issues.apache.org/jira/browse/IMPALA-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736730#comment-16736730
]
Pooja Nilangekar commented on IMPALA-8007:
--
I was looking for reasons for the secs_since_heartbeat to be lower than the
sleep time and I found the following in the python docs:
{code:java}
The actual suspension time may be less than that requested because any caught
signal will terminate the sleep() following execution of that signal’s catching
routine. Also, the suspension time may be longer than requested by an arbitrary
amount because of the scheduling of other activity in the system.
Changed in version 3.5: The function now sleeps at least secs even if the sleep
is interrupted by a signal, except if the signal handler raises an exception
(see PEP 475 for the rationale).
{code}
As per [~tarmstrong]'s suggestion I modified the test to check that the
secs_since_heartbeats is always greater than the previous time. This test also
fails because the statestore's web UI provides the duration in millisecond
precision while the sleep function can wake up in less than one millisecond. So
I there are two options here
# Check for a monotonically increasing duration instead of a strictly
increasing duration. This would help solve the issue of the test but I am not
sure that it'd actually validate the correctness of the monitoring thread. (A
thread could return the exact same value for several seconds/minutes and that
would still be accepted by the thread).
# Update the version of the time library used to 3.5 or higher and then check
for strictly increasing duration since heartbeat. This might affect other
instances where "time" is imported but would actually validate the heartbeat
monitoring thread.
Tim, what do you suggest?
> test_slow_subscriber is flaky
> -
>
> Key: IMPALA-8007
> URL: https://issues.apache.org/jira/browse/IMPALA-8007
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: bharath v
>Assignee: Pooja Nilangekar
>Priority: Major
> Labels: broken-build, flaky
> Fix For: Impala 3.2.0
>
>
> We have hit both the asserts in the test.
> *Exhaustive:*
> {noformat}
> statestore/test_statestore.py:574: in test_slow_subscriber assert
> (secs_since_heartbeat < float(sleep_time + 1.0)) E assert
> 8.8043 < 6.0 E+ where 6.0 = float((5 + 1.0))
> Stacktrace
> statestore/test_statestore.py:574: in test_slow_subscriber
> assert (secs_since_heartbeat < float(sleep_time + 1.0))
> E assert 8.8043 < 6.0
> E+ where 6.0 = float((5 + 1.0))
> {noformat}
> *ASAN*
> {noformat}
> Error Message
> statestore/test_statestore.py:573: in t assert (secs_since_heartbeat >
> float(sleep_time - 1.0)) E assert 4.995 > 5.0 E+ where 5.0 = float((6
> - 1.0))
> Stacktrace
> statestore/test_statestore.py:573: in test_slow_subscriber
> assert (secs_since_heartbeat > float(sleep_time - 1.0))
> E assert 4.995 > 5.0
> E+ where 5.0 = float((6 - 1.0))
> {noformat}
> I only noticed this happen twice (the above two instances) since the patch is
> committed. So, looks like a racy bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8055) run-tests.py reports tests as passed even if the did not
[
https://issues.apache.org/jira/browse/IMPALA-8055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736703#comment-16736703
]
Philip Zeyliger commented on IMPALA-8055:
-
Are you just running into {{$MAX_PYTEST_FAILURES}} or something else? That's an
option that configures when to give up. The thorny issue is that sometimes you
want to run all of them, but, sometimes, if a test crashes impala, the
gazillion test failures after the crash aren't particularly useful.
It seems like a test shouldn't have said "passed" if it "failed", and, if you
know how to reproduce that, let's figure that out. Do you have a repro handy?
> run-tests.py reports tests as passed even if the did not
>
>
> Key: IMPALA-8055
> URL: https://issues.apache.org/jira/browse/IMPALA-8055
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.1.0
>Reporter: Paul Rogers
>Priority: Minor
>
> Been mucking about with the EXPLAIN output format which required rebasing a
> bunch of tests on the new format. PlannerTest is fine: it clearly fails when
> the expected ".test" files don't match the new "actual" files.
> When run on Jenkins in "pre-review" mode, the build does fail if a Python
> end-to-end test fails. But, the job seems to give up at that point, not
> running other tests and finding more problems. (There were three separate
> test cases that needed fixing; took multiple runs to find them.)
> When run on my dev box, I get the following (highly abbreviated) output:
> {noformat}
> '| in pipelines: 00(GETNEXT)' != '| row-size=402B cardinality=5.76M'
> ...
> [gw3] PASSED
> metadata/test_explain.py::TestExplain::test_explain_level0[protocol: beeswax
> | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold':
> 0} | table_format: text/none]
> ...
> 6 passed in 68.63 seconds =
> {noformat}
> I've learned that "passed" means "maybe failed" and to go back and inspect
> the actual output to figure out if the test did, indeed, fail. I suspect
> "passed" means "didn't crash" rather than "tests worked."
> Would be very helpful to plumb the failure through to the summary line so it
> said "3 passed, 3 failed" or whatever. Would be a huge time-saver.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8055) run-tests.py reports tests as passed even if the did not
Paul Rogers created IMPALA-8055:
---
Summary: run-tests.py reports tests as passed even if the did not
Key: IMPALA-8055
URL: https://issues.apache.org/jira/browse/IMPALA-8055
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 3.1.0
Reporter: Paul Rogers
Been mucking about with the EXPLAIN output format which required rebasing a
bunch of tests on the new format. PlannerTest is fine: it clearly fails when
the expected ".test" files don't match the new "actual" files.
When run on Jenkins in "pre-review" mode, the build does fail if a Python
end-to-end test fails. But, the job seems to give up at that point, not running
other tests and finding more problems. (There were three separate test cases
that needed fixing; took multiple runs to find them.)
When run on my dev box, I get the following (highly abbreviated) output:
{noformat}
'| in pipelines: 00(GETNEXT)' != '| row-size=402B cardinality=5.76M'
...
[gw3] PASSED
metadata/test_explain.py::TestExplain::test_explain_level0[protocol: beeswax |
exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold':
0} | table_format: text/none]
...
6 passed in 68.63 seconds =
{noformat}
I've learned that "passed" means "maybe failed" and to go back and inspect the
actual output to figure out if the test did, indeed, fail. I suspect "passed"
means "didn't crash" rather than "tests worked."
Would be very helpful to plumb the failure through to the summary line so it
said "3 passed, 3 failed" or whatever. Would be a huge time-saver.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IMPALA-8055) run-tests.py reports tests as passed even if the did not
Paul Rogers created IMPALA-8055:
---
Summary: run-tests.py reports tests as passed even if the did not
Key: IMPALA-8055
URL: https://issues.apache.org/jira/browse/IMPALA-8055
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 3.1.0
Reporter: Paul Rogers
Been mucking about with the EXPLAIN output format which required rebasing a
bunch of tests on the new format. PlannerTest is fine: it clearly fails when
the expected ".test" files don't match the new "actual" files.
When run on Jenkins in "pre-review" mode, the build does fail if a Python
end-to-end test fails. But, the job seems to give up at that point, not running
other tests and finding more problems. (There were three separate test cases
that needed fixing; took multiple runs to find them.)
When run on my dev box, I get the following (highly abbreviated) output:
{noformat}
'| in pipelines: 00(GETNEXT)' != '| row-size=402B cardinality=5.76M'
...
[gw3] PASSED
metadata/test_explain.py::TestExplain::test_explain_level0[protocol: beeswax |
exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold':
0} | table_format: text/none]
...
6 passed in 68.63 seconds =
{noformat}
I've learned that "passed" means "maybe failed" and to go back and inspect the
actual output to figure out if the test did, indeed, fail. I suspect "passed"
means "didn't crash" rather than "tests worked."
Would be very helpful to plumb the failure through to the summary line so it
said "3 passed, 3 failed" or whatever. Would be a huge time-saver.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-7625) test_web_pages.py backend tests are failing
[
https://issues.apache.org/jira/browse/IMPALA-7625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar resolved IMPALA-7625.
--
Resolution: Fixed
Fix Version/s: Impala 3.2.0
> test_web_pages.py backend tests are failing
> ---
>
> Key: IMPALA-7625
> URL: https://issues.apache.org/jira/browse/IMPALA-7625
> Project: IMPALA
> Issue Type: Test
> Components: Infrastructure
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Fix For: Impala 3.2.0
>
>
> While working on IMPALA-6249, we found that the tests under
> {{webserver/test_web_pages.py}} are not being run by Jenkins. We re-enabled
> the tests, however, a few of the backend specific tests are failing.
> IMPALA-6249 disabled these tests. This JIRA is to follow up on these tests
> and fix them.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-7625) test_web_pages.py backend tests are failing
[
https://issues.apache.org/jira/browse/IMPALA-7625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar resolved IMPALA-7625.
--
Resolution: Fixed
Fix Version/s: Impala 3.2.0
> test_web_pages.py backend tests are failing
> ---
>
> Key: IMPALA-7625
> URL: https://issues.apache.org/jira/browse/IMPALA-7625
> Project: IMPALA
> Issue Type: Test
> Components: Infrastructure
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Fix For: Impala 3.2.0
>
>
> While working on IMPALA-6249, we found that the tests under
> {{webserver/test_web_pages.py}} are not being run by Jenkins. We re-enabled
> the tests, however, a few of the backend specific tests are failing.
> IMPALA-6249 disabled these tests. This JIRA is to follow up on these tests
> and fix them.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IMPALA-8026) Actual row counts for nested loop join are way too high while the query is executing
[
https://issues.apache.org/jira/browse/IMPALA-8026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-8026.
---
Resolution: Fixed
Fix Version/s: Impala 3.2.0
> Actual row counts for nested loop join are way too high while the query is
> executing
>
>
> Key: IMPALA-8026
> URL: https://issues.apache.org/jira/browse/IMPALA-8026
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: Paul Rogers
>Assignee: Tim Armstrong
>Priority: Major
> Fix For: Impala 3.2.0
>
>
> Consider this extract from a query plan:
> {noformat}
> Operator #Rows Est. #Rows
> --
> …
> | 10:HASH JOIN 9.53M 18.14K
> | |--19:EXCHANGE 1 1
> | | 00:SCAN HDFS1 1
> | 06:NESTED LOOP JOIN4.88B 863.84K
> | |--18:EXCHANGE 1 1
> | | 04:SCAN HDFS1 1
> | 05:HASH JOIN 9.53M 863.84K
> {noformat}
> If the above is to be believed, the 06 nested loop join produced 5 billion
> rows. But, the actual number is far too huge for that: joining 1 row with 10
> million rows cannot produce 500 times that number of rows.
> It appears that the nested loop join actually processed and returned the 9.5
> million rows, since that is the same number produced by the 10 hash join
> which joins a single row with the output of the nested loop join.
> Because this same bogus result appears across multiple plans, it is likely
> that the actual number is completely wrong and bears no relation to the
> number of rows actually returned.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8026) Actual row counts for nested loop join are way too high while the query is executing
[
https://issues.apache.org/jira/browse/IMPALA-8026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-8026.
---
Resolution: Fixed
Fix Version/s: Impala 3.2.0
> Actual row counts for nested loop join are way too high while the query is
> executing
>
>
> Key: IMPALA-8026
> URL: https://issues.apache.org/jira/browse/IMPALA-8026
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: Paul Rogers
>Assignee: Tim Armstrong
>Priority: Major
> Fix For: Impala 3.2.0
>
>
> Consider this extract from a query plan:
> {noformat}
> Operator #Rows Est. #Rows
> --
> …
> | 10:HASH JOIN 9.53M 18.14K
> | |--19:EXCHANGE 1 1
> | | 00:SCAN HDFS1 1
> | 06:NESTED LOOP JOIN4.88B 863.84K
> | |--18:EXCHANGE 1 1
> | | 04:SCAN HDFS1 1
> | 05:HASH JOIN 9.53M 863.84K
> {noformat}
> If the above is to be believed, the 06 nested loop join produced 5 billion
> rows. But, the actual number is far too huge for that: joining 1 row with 10
> million rows cannot produce 500 times that number of rows.
> It appears that the nested loop join actually processed and returned the 9.5
> million rows, since that is the same number produced by the 10 hash join
> which joins a single row with the output of the nested loop join.
> Because this same bogus result appears across multiple plans, it is likely
> that the actual number is completely wrong and bears no relation to the
> number of rows actually returned.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IMPALA-7468) Port CancelQueryFInstances() to KRPC
[ https://issues.apache.org/jira/browse/IMPALA-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736563#comment-16736563 ] ASF subversion and git services commented on IMPALA-7468: - Commit e4cff7d0d60e60ddf6c77b48ca0da7b878672c1a in impala's branch refs/heads/master from Andrew Sherman [ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=e4cff7d ] IMPALA-7468: Port CancelQueryFInstances() to KRPC. When the Coordinator needs to cancel a query (for example because a user has hit Control-C), it does this by sending a CancelQueryFInstances message to each fragment instance. This change switches this code to use KRPC. Add new protobuf definitions for the messages, and remove the old thrift definitions. Move the server-side implementation of Cancel() from ImpalaInternalService to ControlService. Rework the scheduler so that the FInstanceExecParams always contains the KRPC address of the fragment executors, this address can then be used if a query is to be cancelled. For now keep the KRPC calls to CancelQueryFInstances() as synchronous. While moving the client-side code, remove the fault injection code that was inserted with FAULT_INJECTION_SEND_RPC_EXCEPTION and FAULT_INJECTION_RECV_RPC_EXCEPTION (triggered by running impalad with --fault_injection_rpc_exception_type=1) as this tickles code in client-cache.h which is now not used. TESTING: Ran all end-to-end tests. No new tests as test_cancellation.py provides good coverage. Checked in debugger that DebugAction style fault injection (triggered from test_cancellation.py) was working correctly. Change-Id: I625030c3f1068061aa029e6e242f016cadd84969 Reviewed-on: http://gerrit.cloudera.org:8080/12142 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Port CancelQueryFInstances() to KRPC > > > Key: IMPALA-7468 > URL: https://issues.apache.org/jira/browse/IMPALA-7468 > Project: IMPALA > Issue Type: Sub-task > Components: Distributed Exec >Affects Versions: Impala 3.1.0 >Reporter: Michael Ho >Assignee: Andrew Sherman >Priority: Major > Labels: ramp-up > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8026) Actual row counts for nested loop join are way too high while the query is executing
[
https://issues.apache.org/jira/browse/IMPALA-8026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736564#comment-16736564
]
ASF subversion and git services commented on IMPALA-8026:
-
Commit 78da2b135143ac1faecd9ed1a62e1ce926555de3 in impala's branch
refs/heads/master from Tim Armstrong
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=78da2b1 ]
IMPALA-8026: Fix #rows accounting for NLJ
Use the same, much simpler, approach used by all other ExecNodes.
Testing:
Manually tested by adding logging to print values of the counter.
Confirmed that the fix led to the counter having the correct
values (instead of values exceeding num_rows_returned_).
Change-Id: I43e2e1d8f85478ff36c8cddc69bac3bdccd2c824
Reviewed-on: http://gerrit.cloudera.org:8080/12164
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> Actual row counts for nested loop join are way too high while the query is
> executing
>
>
> Key: IMPALA-8026
> URL: https://issues.apache.org/jira/browse/IMPALA-8026
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: Paul Rogers
>Assignee: Tim Armstrong
>Priority: Major
>
> Consider this extract from a query plan:
> {noformat}
> Operator #Rows Est. #Rows
> --
> …
> | 10:HASH JOIN 9.53M 18.14K
> | |--19:EXCHANGE 1 1
> | | 00:SCAN HDFS1 1
> | 06:NESTED LOOP JOIN4.88B 863.84K
> | |--18:EXCHANGE 1 1
> | | 04:SCAN HDFS1 1
> | 05:HASH JOIN 9.53M 863.84K
> {noformat}
> If the above is to be believed, the 06 nested loop join produced 5 billion
> rows. But, the actual number is far too huge for that: joining 1 row with 10
> million rows cannot produce 500 times that number of rows.
> It appears that the nested loop join actually processed and returned the 9.5
> million rows, since that is the same number produced by the 10 hash join
> which joins a single row with the output of the nested loop join.
> Because this same bogus result appears across multiple plans, it is likely
> that the actual number is completely wrong and bears no relation to the
> number of rows actually returned.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7625) test_web_pages.py backend tests are failing
[
https://issues.apache.org/jira/browse/IMPALA-7625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736562#comment-16736562
]
ASF subversion and git services commented on IMPALA-7625:
-
Commit ec2dfd9482d48470176ee3c7925875855879a0b2 in impala's branch
refs/heads/master from Sahil Takiar
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=ec2dfd9 ]
IMPALA-7625: test_web_pages.py backend tests are failing
The backend tests in test_web_page.py do some basic validation of
the query_backends and query_finstances endpoints. The tests were
failing due to a race condition in the test itself. The issue is that
the backend endpoints are only usable while a query is running. When
queried against completed queries, the backend endpoints return nothing.
The original test worked by running a "complex" query asynchronously and
then querying the backend endpoints before the query completed. The
assumption was that the query would run long enough for the request to
the backend endpoint to complete. However, that is not always the case
and the query easily completes before the backend endpoints can be
reached.
This patch updates the test so that instead of running a "complex"
query, it runs a query that calls the sleep UDF, guaranteeing the query
will take enough time for the backend endpoints to be reached.
Furthermore, the patch adds additional validation and cleanup of all the
backend endpoint tests.
Testing:
Ran core tests.
Change-Id: I2092afe1bfaec30cd3e7b0040f06865e43fe62fb
Reviewed-on: http://gerrit.cloudera.org:8080/12024
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> test_web_pages.py backend tests are failing
> ---
>
> Key: IMPALA-7625
> URL: https://issues.apache.org/jira/browse/IMPALA-7625
> Project: IMPALA
> Issue Type: Test
> Components: Infrastructure
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
>
> While working on IMPALA-6249, we found that the tests under
> {{webserver/test_web_pages.py}} are not being run by Jenkins. We re-enabled
> the tests, however, a few of the backend specific tests are failing.
> IMPALA-6249 disabled these tests. This JIRA is to follow up on these tests
> and fix them.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8054) Implicit cast fails with {const INT} BETWEEN FLOAT and INT
[
https://issues.apache.org/jira/browse/IMPALA-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Rahn updated IMPALA-8054:
--
Description:
In the following query the literal number 10 needs to be compared to two
different types: INT and FLOAT, but Impala fails to make the implicit cast for
the FLOAT.
The predicates should be
{noformat}
predicates: (cast(10 as float) >= col4) AND (10 <= col3)
{noformat}
*Test cases:*
{noformat}
sql> describe tab4
+--++-+
| name | type | comment |
+--++-+
| pk | int| |
| col0 | int| |
| col1 | float | |
| col2 | string | |
| col3 | int| |
| col4 | float | |
| col5 | string | |
+--++-+
sql> SELECT col0 FROM tab4 WHERE 10 BETWEEN col4 AND col3;
ERROR: IllegalStateException: child 0 type: FLOAT child 1 type: DOUBLE
sql> SELECT * FROM tab4 WHERE NULL NOT BETWEEN col3 AND col1;
ERROR: IllegalStateException: child 0 type: INT child 1 type: DOUBLE
sql> SELECT * FROM tab4 WHERE NULL BETWEEN CAST ( NULL AS INTEGER ) AND col4
ERROR: IllegalStateException: child 0 type: DOUBLE child 1 type: INT
{noformat}
was:
In the following query the literal number 10 needs to be compared to two
different types: INT and FLOAT, but Impala fails to make the implicit cast for
the FLOAT.
The predicates should be
{noformat}
predicates: (cast(10 as float) >= col4) AND (10 <= col3)
{noformat}
*Test cases:*
{noformat}
sql> describe tab4
+--++-+
| name | type | comment |
+--++-+
| pk | int| |
| col0 | int| |
| col1 | float | |
| col2 | string | |
| col3 | int| |
| col4 | float | |
| col5 | string | |
+--++-+
sql> SELECT col0 FROM tab4 WHERE 10 BETWEEN col4 AND col3;
ERROR: IllegalStateException: child 0 type: FLOAT child 1 type: DOUBLE
sql> SELECT * FROM tab4 WHERE NULL NOT BETWEEN col3 AND col1;
ERROR: IllegalStateException: child 0 type: INT child 1 type: DOUBLE
{noformat}
> Implicit cast fails with {const INT} BETWEEN FLOAT and INT
> --
>
> Key: IMPALA-8054
> URL: https://issues.apache.org/jira/browse/IMPALA-8054
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.2.0
>Reporter: Greg Rahn
>Assignee: Paul Rogers
>Priority: Major
>
> In the following query the literal number 10 needs to be compared to two
> different types: INT and FLOAT, but Impala fails to make the implicit cast
> for the FLOAT.
> The predicates should be
> {noformat}
> predicates: (cast(10 as float) >= col4) AND (10 <= col3)
> {noformat}
> *Test cases:*
> {noformat}
> sql> describe tab4
> +--++-+
> | name | type | comment |
> +--++-+
> | pk | int| |
> | col0 | int| |
> | col1 | float | |
> | col2 | string | |
> | col3 | int| |
> | col4 | float | |
> | col5 | string | |
> +--++-+
> sql> SELECT col0 FROM tab4 WHERE 10 BETWEEN col4 AND col3;
> ERROR: IllegalStateException: child 0 type: FLOAT child 1 type: DOUBLE
> sql> SELECT * FROM tab4 WHERE NULL NOT BETWEEN col3 AND col1;
> ERROR: IllegalStateException: child 0 type: INT child 1 type: DOUBLE
> sql> SELECT * FROM tab4 WHERE NULL BETWEEN CAST ( NULL AS INTEGER ) AND col4
> ERROR: IllegalStateException: child 0 type: DOUBLE child 1 type: INT
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8054) Implicit cast fails with {const INT} BETWEEN FLOAT and INT
[
https://issues.apache.org/jira/browse/IMPALA-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Rahn updated IMPALA-8054:
--
Description:
In the following query the literal number 10 needs to be compared to two
different types: INT and FLOAT, but Impala fails to make the implicit cast for
the FLOAT.
The predicates should be
{noformat}
predicates: (cast(10 as float) >= col4) AND (10 <= col3)
{noformat}
*Test cases:*
{noformat}
sql> describe tab4
+--++-+
| name | type | comment |
+--++-+
| pk | int| |
| col0 | int| |
| col1 | float | |
| col2 | string | |
| col3 | int| |
| col4 | float | |
| col5 | string | |
+--++-+
sql> SELECT col0 FROM tab4 WHERE 10 BETWEEN col4 AND col3;
ERROR: IllegalStateException: child 0 type: FLOAT child 1 type: DOUBLE
sql> SELECT * FROM tab4 WHERE NULL NOT BETWEEN col3 AND col1;
ERROR: IllegalStateException: child 0 type: INT child 1 type: DOUBLE
{noformat}
was:
In the following query the literal number 10 needs to be compared to two
different types: INT and FLOAT, but Impala fails to make the implicit cast for
the FLOAT.
The predicates should be
{noformat}
predicates: (cast(10 as float) >= col4) AND (10 <= col3)
{noformat}
*Test case:*
{noformat}
sql> describe tab4
+--++-+
| name | type | comment |
+--++-+
| pk | int| |
| col0 | int| |
| col1 | float | |
| col2 | string | |
| col3 | int| |
| col4 | float | |
| col5 | string | |
+--++-+
sql> SELECT col0 FROM tab4 WHERE 10 BETWEEN col4 AND col3;
ERROR: IllegalStateException: child 0 type: FLOAT child 1 type: DOUBLE
{noformat}
> Implicit cast fails with {const INT} BETWEEN FLOAT and INT
> --
>
> Key: IMPALA-8054
> URL: https://issues.apache.org/jira/browse/IMPALA-8054
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.2.0
>Reporter: Greg Rahn
>Assignee: Paul Rogers
>Priority: Major
>
> In the following query the literal number 10 needs to be compared to two
> different types: INT and FLOAT, but Impala fails to make the implicit cast
> for the FLOAT.
> The predicates should be
> {noformat}
> predicates: (cast(10 as float) >= col4) AND (10 <= col3)
> {noformat}
> *Test cases:*
> {noformat}
> sql> describe tab4
> +--++-+
> | name | type | comment |
> +--++-+
> | pk | int| |
> | col0 | int| |
> | col1 | float | |
> | col2 | string | |
> | col3 | int| |
> | col4 | float | |
> | col5 | string | |
> +--++-+
> sql> SELECT col0 FROM tab4 WHERE 10 BETWEEN col4 AND col3;
> ERROR: IllegalStateException: child 0 type: FLOAT child 1 type: DOUBLE
> sql> SELECT * FROM tab4 WHERE NULL NOT BETWEEN col3 AND col1;
> ERROR: IllegalStateException: child 0 type: INT child 1 type: DOUBLE
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8054) Implicit cast fails with {const INT} BETWEEN FLOAT and INT
[
https://issues.apache.org/jira/browse/IMPALA-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Rahn updated IMPALA-8054:
--
Description:
In the following query the literal number 10 needs to be compared to two
different types: INT and FLOAT, but Impala fails to make the implicit cast for
the FLOAT.
The predicates should be
{noformat}
predicates: (cast(10 as float) >= col4) AND (10 <= col3)
{noformat}
*Test case:*
{noformat}
sql> describe tab4
+--++-+
| name | type | comment |
+--++-+
| pk | int| |
| col0 | int| |
| col1 | float | |
| col2 | string | |
| col3 | int| |
| col4 | float | |
| col5 | string | |
+--++-+
sql> SELECT col0 FROM tab4 WHERE 10 BETWEEN col4 AND col3;
ERROR: IllegalStateException: child 0 type: FLOAT child 1 type: DOUBLE
{noformat}
was:
In the following query the literal number 10 needs to be compared to two
different types: INT and FLOAT, but Impala fails to make the implicit cast for
the FLOAD.
The predicates should be
{noformat}
predicates: (cast(10 as float) >= col4) AND (10 <= col3)
{noformat}
*Test case:*
{noformat}
sql> describe tab4
+--++-+
| name | type | comment |
+--++-+
| pk | int| |
| col0 | int| |
| col1 | float | |
| col2 | string | |
| col3 | int| |
| col4 | float | |
| col5 | string | |
+--++-+
sql> SELECT col0 FROM tab4 WHERE 10 BETWEEN col4 AND col3;
ERROR: IllegalStateException: child 0 type: FLOAT child 1 type: DOUBLE
{noformat}
> Implicit cast fails with {const INT} BETWEEN FLOAT and INT
> --
>
> Key: IMPALA-8054
> URL: https://issues.apache.org/jira/browse/IMPALA-8054
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.2.0
>Reporter: Greg Rahn
>Assignee: Paul Rogers
>Priority: Major
>
> In the following query the literal number 10 needs to be compared to two
> different types: INT and FLOAT, but Impala fails to make the implicit cast
> for the FLOAT.
> The predicates should be
> {noformat}
> predicates: (cast(10 as float) >= col4) AND (10 <= col3)
> {noformat}
> *Test case:*
> {noformat}
> sql> describe tab4
> +--++-+
> | name | type | comment |
> +--++-+
> | pk | int| |
> | col0 | int| |
> | col1 | float | |
> | col2 | string | |
> | col3 | int| |
> | col4 | float | |
> | col5 | string | |
> +--++-+
> sql> SELECT col0 FROM tab4 WHERE 10 BETWEEN col4 AND col3;
> ERROR: IllegalStateException: child 0 type: FLOAT child 1 type: DOUBLE
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8054) Implicit cast fails with {const INT} BETWEEN FLOAT and INT
Greg Rahn created IMPALA-8054:
-
Summary: Implicit cast fails with {const INT} BETWEEN FLOAT and INT
Key: IMPALA-8054
URL: https://issues.apache.org/jira/browse/IMPALA-8054
Project: IMPALA
Issue Type: Bug
Components: Frontend
Affects Versions: Impala 3.2.0
Reporter: Greg Rahn
Assignee: Paul Rogers
In the following query the literal number 10 needs to be compared to two
different types: INT and FLOAT, but Impala fails to make the implicit cast for
the FLOAD.
The predicates should be
{noformat}
predicates: (cast(10 as float) >= col4) AND (10 <= col3)
{noformat}
*Test case:*
{noformat}
sql> describe tab4
+--++-+
| name | type | comment |
+--++-+
| pk | int| |
| col0 | int| |
| col1 | float | |
| col2 | string | |
| col3 | int| |
| col4 | float | |
| col5 | string | |
+--++-+
sql> SELECT col0 FROM tab4 WHERE 10 BETWEEN col4 AND col3;
ERROR: IllegalStateException: child 0 type: FLOAT child 1 type: DOUBLE
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IMPALA-8054) Implicit cast fails with {const INT} BETWEEN FLOAT and INT
Greg Rahn created IMPALA-8054:
-
Summary: Implicit cast fails with {const INT} BETWEEN FLOAT and INT
Key: IMPALA-8054
URL: https://issues.apache.org/jira/browse/IMPALA-8054
Project: IMPALA
Issue Type: Bug
Components: Frontend
Affects Versions: Impala 3.2.0
Reporter: Greg Rahn
Assignee: Paul Rogers
In the following query the literal number 10 needs to be compared to two
different types: INT and FLOAT, but Impala fails to make the implicit cast for
the FLOAD.
The predicates should be
{noformat}
predicates: (cast(10 as float) >= col4) AND (10 <= col3)
{noformat}
*Test case:*
{noformat}
sql> describe tab4
+--++-+
| name | type | comment |
+--++-+
| pk | int| |
| col0 | int| |
| col1 | float | |
| col2 | string | |
| col3 | int| |
| col4 | float | |
| col5 | string | |
+--++-+
sql> SELECT col0 FROM tab4 WHERE 10 BETWEEN col4 AND col3;
ERROR: IllegalStateException: child 0 type: FLOAT child 1 type: DOUBLE
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7992) test_decimal_fuzz.py/test_decimal_ops failing in exhaustive runs
[
https://issues.apache.org/jira/browse/IMPALA-7992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736521#comment-16736521
]
Philip Zeyliger commented on IMPALA-7992:
-
I think that means our randomness isn't seeded? That means the change to reduce
iterations lost coverage (as opposed to getting coverage over time), which
wasn't quite the intent.
> test_decimal_fuzz.py/test_decimal_ops failing in exhaustive runs
>
>
> Key: IMPALA-7992
> URL: https://issues.apache.org/jira/browse/IMPALA-7992
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: bharath v
>Assignee: Csaba Ringhofer
>Priority: Blocker
> Labels: broken-build
>
> Error Message
> {noformat}
> query_test/test_decimal_fuzz.py:251: in test_decimal_ops
> self.execute_one_decimal_op() query_test/test_decimal_fuzz.py:247: in
> execute_one_decimal_op assert self.result_equals(expected_result, result) E
> assert >(Decimal('-0.80'),
> None) E + where > =
> .result_equals
> {noformat}
> Stacktrace
> {noformat}
> query_test/test_decimal_fuzz.py:251: in test_decimal_ops
> self.execute_one_decimal_op() query_test/test_decimal_fuzz.py:247: in
> execute_one_decimal_op assert self.result_equals(expected_result, result) E
> assert >(Decimal('-0.80'),
> None) E + where > =
> .result_equals
> {noformat}
> stderr
> {noformat}
> -- 2018-12-16 00:10:48,905 INFO MainThread: Started query
> aa4b44ad5b34c3fb:24d18385
> SET decimal_v2=true;
> -- executing against localhost:21000
> select cast(-879550566.24 as decimal(11,2)) %
> cast(-100.000 as decimal(28,5));
> -- 2018-12-16 00:10:48,979 INFO MainThread: Started query
> b24acf22b1607dc6:4f287530
> SET decimal_v2=true;
> -- executing against localhost:21000
> select cast(17179869.184 as decimal(19,7)) /
> cast(-87808593158000679814.7939232649738916 as decimal(38,17));
> -- 2018-12-16 00:10:49,054 INFO MainThread: Started query
> 38435f02022e590a:18f7e97
> SET decimal_v2=true;
> -- executing against localhost:21000
> select cast(99 as decimal(32,2)) -
> cast(-519203.671959101313 as decimal(18,12));
> -- 2018-12-16 00:10:49,132 INFO MainThread: Started query
> 504edbac7ecb32ce:bfbbbe93
> ~ Stack of (140061483271936)
> ~
> File
> "/data/jenkins/workspace/impala-asf-master-exhaustive-centos6/repos/Impala/infra/python/env/lib/python2.6/site-packages/execnet/gateway_base.py",
> line 277, in _perform_spawn
> reply.run()
> File
> "/data/jenkins/workspace/impala-asf-master-exhaustive-centos6/repos/Impala/infra/python/env/lib/python2.6/site-packages/execnet/gateway_base.py",
> line 213, in run
> self._result = func(*args, **kwargs)
> File
> "/data/jenkins/workspace/impala-asf-master-exhaustive-centos6/repos/Impala/infra/python/env/lib/python2.6/site-packages/execnet/gateway_base.py",
> line 954, in _thread_receiver
> msg = Message.from_io(io)
> File
> "/data/jenkins/workspace/impala-asf-master-exhaustive-centos6/repos/Impala/infra/python/env/lib/python2.6/site-packages/execnet/gateway_base.py",
> line 418, in from_io
> header = io.read(9) # type 1, channel 4, payload 4
> File
> "/data/jenkins/workspace/impala-asf-master-exhaustive-centos6/repos/Impala/infra/python/env/lib/python2.6/site-packages/execnet/gateway_base.py",
> line 386, in read
> data = self._read(numbytes-len(buf))
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8045) Rollup of Smaller Join Cardinality Issues
[
https://issues.apache.org/jira/browse/IMPALA-8045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Rogers updated IMPALA-8045:
Description:
The work to review join cardinality has found some major issues recorded as
JIRA tickets. This ticket records a number of smaller issues. Some of these
issues are a bit tricky because they appear only when some of the other issues
are resolved. Reporting them directly could be misleading.
h4. ScanNode confusion between table and scan input cardinality
The {{ScanNode}} class in the scanner contains an {{inputCardinality_}} field
used by join calculations as a proxy for the table size. However, the actual
scan node implementations set the {{inputCardinality_}} to the estimated number
of rows *read* by the scan, which is useful when understanding the physical
scan structure. But, for joins, we need the base table cardinality.
For example, the join may use the input cardinality to understand the reduction
in rows due to filters in order to adjust the NDV of key columns. But, since
the input cardinality is the scan count, not the table row count, the math does
not work out.
The solution is to clarify the code to separate the idea of scan count vs. base
table row count.
h4. Selectivity Confusion
Similarly, each node computes its selectivity. However, the selectivity is only
for those predicates that will be applied via a projection. Predicates that can
be applied because of partition pruning (HDFS), key range pruning (HBase) and
so on do not "count". While this produces accurate execution estimates, it is
not helpful for join planning.
In join planning, we need to know the number of filtered rows relative to the
total table cardinality. This allows us to adjust HDV key cardinality in order
to estimate the number of rows produced by the join.
Using the partial selectivity, or partial input cardinality (above issue)
causes inaccurate key cardinality adjustments and incorrect join cardinality
estimates.
h4. Join Node Does not Apply Selectivity from Its Predicates
A join node can have "additional predicates" applied after creating a join row.
Accurate estimation of join cardinality must include the selectivity from those
predicates, but is not currently done. Perhaps because such predicates, in the
current estimation scheme, always produce an estimated selectivity of .1. This
will be more important as we add more realistic estimates.
h4. Use Double, not Long for Cardinality Values
In scan nodes, row counts can be reasonable numbers and a Java {{long}} is
fine. But, once one starts computing join cardinalities, values can grow fast,
especially for cross joins. The code currently has special checks to limit
products to {{Long.MAX_VALUE}}. While this solves the overflow issue, it has
undesirable downstream affects. First, it throws of selectivity calculations
since the reported cardinality is not the real cardinality. Second, it requires
special math calls whenever we multiply cardinalities.
Much simper to work with a {{double}}. When values get large, the extra
precision from a integer value is completely lost in the noise of assumptions
and estimations.
h4. Revisit Cardinality Calcs for Join Nodes
The method {{JoinNode.computeStats()}} is a bit muddled. It starts by computing
cardinality depending on the major type family (semi-join, inner/outer join,
cross join). It then revises those calcs based on the specific join type. This
makes it very hard to follow the logic case we have to follow two distinct
blocks of code. There is also redundancy. The cross join cardinality is
calculated twice, for example.
Refactor to have a cardinality/selectivity calculation per join type.
h4. Disallow Unknown Cardinality
Multiple nodes can produce a cardinality of -1 (unknown). Since it is
impossible to plan based on an unknown cardinality, we must have an estimate,
however good or bad. For cases where we have no stats, estimate cardinality
based on other factors. If we have no column NDV, perhaps guesstimate
something, or use an alternative join calculation that avoids the need for NDV
(while producing much cruder estimates.) However, refusing to play the game at
all is not helpful unless we choose to fail the query for lack of stats.
h4. Revisit Join Cardinality Limit
The JoinNode has several methods that limit cardinality:
{code:java}
public void computeStats(Analyzer analyzer) {
...
cardinality_ = capCardinalityAtLimit(cardinality_);
...
}
public boolean hasLimit() { return limit_ > -1; }
protected long capCardinalityAtLimit(long cardinality) {
if (hasLimit()) {
return capCardinalityAtLimit(cardinality, limit_);
}
return cardinality;
}
{code}
It is not clear when or why we apply a limit. Perhaps as part of {{LIMIT x}}
processing? Revisit if the limit is helpful, and remove it if not. Imposing a
limit throws
[jira] [Comment Edited] (IMPALA-8051) Compute stats fails on a column with comment character in name
[
https://issues.apache.org/jira/browse/IMPALA-8051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736503#comment-16736503
]
Paul Rogers edited comment on IMPALA-8051 at 1/8/19 12:09 AM:
--
Turns out that the "#" character has unique problems. It is a comment
character, which seems to fall through the cracks in the techniques we use to
detect that the identifier must be quoted. To check for quotes, we let Hive
parse the token. If the name is, say, "foo+", we get two tokens, and so quote
the name. But, for "foo#", we get just one token because the "#" starts a
comment.
Added this trivial fix to the patch for IMPALA-7905.
was (Author: paul.rogers):
Turns out that the "#" character has unique problems. It is a comment
character, which seems to fall through the cracks in the techniques we use to
detect that the identifier must be quoted. To check for quotes, we let Hive
parse the token. If the name is, say, "foo+", we get two tokens, and so quote
the name. But, for "foo#", we get just one token because the "#" starts a
comment.
> Compute stats fails on a column with comment character in name
> --
>
> Key: IMPALA-8051
> URL: https://issues.apache.org/jira/browse/IMPALA-8051
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.1.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
>Priority: Minor
>
> Problem - "compute stats" query executed on a table containing a special
> character "#" in one of its columns is failing with below error:
> WARNINGS: AnalysisException: Syntax error in line 1:
> ...length(cola)), NDV(colb#) AS colb#, CAST(-1 as BIG...
> ^
> Encountered: Unexpected character
> Expected: ADD, ALTER, AND, ARRAY, AS, ASC, BETWEEN, BIGINT, BINARY,
> BLOCK_SIZE, BOOLEAN, CACHED, CASCADE, CHANGE, CHAR, COMMENT, COMPRESSION,
> CROSS, DATE, DATETIME, DECIMAL, DEFAULT, DESC, DIV, REAL, DROP, ELSE,
> ENCODING, END, FLOAT, FOLLOWING, FROM, FULL, GROUP, IGNORE, HAVING, ILIKE,
> IN, INNER, INTEGER, IREGEXP, IS, JOIN, LEFT, LIKE, LIMIT, LOCATION, MAP, NOT,
> NULL, NULLS, OFFSET, ON, OR, ORDER, PARTITION, PARTITIONED, PRECEDING,
> PRIMARY, PURGE, RANGE, RECOVER, REGEXP, RENAME, REPLACE, RESTRICT, RIGHT,
> RLIKE, ROW, ROWS, SELECT, SET, SMALLINT, SORT, STORED, STRAIGHT_JOIN, STRING,
> STRUCT, TABLESAMPLE, TBLPROPERTIES, THEN, TIMESTAMP, TINYINT, TO, UNCACHED,
> UNION, USING, VALUES, VARCHAR, WHEN, WHERE, WITH, COMMA, IDENTIFIER
> Steps to reproduce the issue -
> # Create a table containing special character in one of it columns from Hive.
> For example:
> {code:sql}
> CREATE TABLE test_special_character (`id#` int);
> {code}
> # Execute "INVALIDATE METADATA test_special_character" from Impala.
> # Execute "COMPUTE STATS test_special_character" from Impala and it'll lead
> to above mentioned error.
> Impala does not allow to create tables with columns containing special
> characters but Hive allows it by using back ticks (``) to escape it. However,
> Impala still can load the metadata of table and can read from column
> containing special character as well by escaping the special character using
> back ticks (``). For example, below query can be executed from Impala -
> {code:sql}
> select `id#` from test_special_character;
> {code}
> However, when "compute stats" command is executed, the query triggered by
> this command to compute column-level stats does not use back ticks to escape
> the special character present in one of the columns as it does not know that
> column contains a special character and this is the cause of this issue.
> (Reported by a user.)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8051) Compute stats fails on a column with comment character in name
[
https://issues.apache.org/jira/browse/IMPALA-8051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Rogers updated IMPALA-8051:
Summary: Compute stats fails on a column with comment character in name
(was: Compute stats fails on a column with special character in name)
> Compute stats fails on a column with comment character in name
> --
>
> Key: IMPALA-8051
> URL: https://issues.apache.org/jira/browse/IMPALA-8051
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.1.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
>Priority: Minor
>
> Problem - "compute stats" query executed on a table containing a special
> character "#" in one of its columns is failing with below error:
> WARNINGS: AnalysisException: Syntax error in line 1:
> ...length(cola)), NDV(colb#) AS colb#, CAST(-1 as BIG...
> ^
> Encountered: Unexpected character
> Expected: ADD, ALTER, AND, ARRAY, AS, ASC, BETWEEN, BIGINT, BINARY,
> BLOCK_SIZE, BOOLEAN, CACHED, CASCADE, CHANGE, CHAR, COMMENT, COMPRESSION,
> CROSS, DATE, DATETIME, DECIMAL, DEFAULT, DESC, DIV, REAL, DROP, ELSE,
> ENCODING, END, FLOAT, FOLLOWING, FROM, FULL, GROUP, IGNORE, HAVING, ILIKE,
> IN, INNER, INTEGER, IREGEXP, IS, JOIN, LEFT, LIKE, LIMIT, LOCATION, MAP, NOT,
> NULL, NULLS, OFFSET, ON, OR, ORDER, PARTITION, PARTITIONED, PRECEDING,
> PRIMARY, PURGE, RANGE, RECOVER, REGEXP, RENAME, REPLACE, RESTRICT, RIGHT,
> RLIKE, ROW, ROWS, SELECT, SET, SMALLINT, SORT, STORED, STRAIGHT_JOIN, STRING,
> STRUCT, TABLESAMPLE, TBLPROPERTIES, THEN, TIMESTAMP, TINYINT, TO, UNCACHED,
> UNION, USING, VALUES, VARCHAR, WHEN, WHERE, WITH, COMMA, IDENTIFIER
> Steps to reproduce the issue -
> # Create a table containing special character in one of it columns from Hive.
> For example:
> {code:sql}
> CREATE TABLE test_special_character (`id#` int);
> {code}
> # Execute "INVALIDATE METADATA test_special_character" from Impala.
> # Execute "COMPUTE STATS test_special_character" from Impala and it'll lead
> to above mentioned error.
> Impala does not allow to create tables with columns containing special
> characters but Hive allows it by using back ticks (``) to escape it. However,
> Impala still can load the metadata of table and can read from column
> containing special character as well by escaping the special character using
> back ticks (``). For example, below query can be executed from Impala -
> {code:sql}
> select `id#` from test_special_character;
> {code}
> However, when "compute stats" command is executed, the query triggered by
> this command to compute column-level stats does not use back ticks to escape
> the special character present in one of the columns as it does not know that
> column contains a special character and this is the cause of this issue.
> (Reported by a user.)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8051) Compute stats fails on a column with special character in name
[
https://issues.apache.org/jira/browse/IMPALA-8051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736503#comment-16736503
]
Paul Rogers commented on IMPALA-8051:
-
Turns out that the "#" character has unique problems. It is a comment
character, which seems to fall through the cracks in the techniques we use to
detect that the identifier must be quoted. To check for quotes, we let Hive
parse the token. If the name is, say, "foo+", we get two tokens, and so quote
the name. But, for "foo#", we get just one token because the "#" starts a
comment.
> Compute stats fails on a column with special character in name
> --
>
> Key: IMPALA-8051
> URL: https://issues.apache.org/jira/browse/IMPALA-8051
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.1.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
>Priority: Minor
>
> Problem - "compute stats" query executed on a table containing a special
> character "#" in one of its columns is failing with below error:
> WARNINGS: AnalysisException: Syntax error in line 1:
> ...length(cola)), NDV(colb#) AS colb#, CAST(-1 as BIG...
> ^
> Encountered: Unexpected character
> Expected: ADD, ALTER, AND, ARRAY, AS, ASC, BETWEEN, BIGINT, BINARY,
> BLOCK_SIZE, BOOLEAN, CACHED, CASCADE, CHANGE, CHAR, COMMENT, COMPRESSION,
> CROSS, DATE, DATETIME, DECIMAL, DEFAULT, DESC, DIV, REAL, DROP, ELSE,
> ENCODING, END, FLOAT, FOLLOWING, FROM, FULL, GROUP, IGNORE, HAVING, ILIKE,
> IN, INNER, INTEGER, IREGEXP, IS, JOIN, LEFT, LIKE, LIMIT, LOCATION, MAP, NOT,
> NULL, NULLS, OFFSET, ON, OR, ORDER, PARTITION, PARTITIONED, PRECEDING,
> PRIMARY, PURGE, RANGE, RECOVER, REGEXP, RENAME, REPLACE, RESTRICT, RIGHT,
> RLIKE, ROW, ROWS, SELECT, SET, SMALLINT, SORT, STORED, STRAIGHT_JOIN, STRING,
> STRUCT, TABLESAMPLE, TBLPROPERTIES, THEN, TIMESTAMP, TINYINT, TO, UNCACHED,
> UNION, USING, VALUES, VARCHAR, WHEN, WHERE, WITH, COMMA, IDENTIFIER
> Steps to reproduce the issue -
> # Create a table containing special character in one of it columns from Hive.
> For example:
> {code:sql}
> CREATE TABLE test_special_character (`id#` int);
> {code}
> # Execute "INVALIDATE METADATA test_special_character" from Impala.
> # Execute "COMPUTE STATS test_special_character" from Impala and it'll lead
> to above mentioned error.
> Impala does not allow to create tables with columns containing special
> characters but Hive allows it by using back ticks (``) to escape it. However,
> Impala still can load the metadata of table and can read from column
> containing special character as well by escaping the special character using
> back ticks (``). For example, below query can be executed from Impala -
> {code:sql}
> select `id#` from test_special_character;
> {code}
> However, when "compute stats" command is executed, the query triggered by
> this command to compute column-level stats does not use back ticks to escape
> the special character present in one of the columns as it does not know that
> column contains a special character and this is the cause of this issue.
> (Reported by a user.)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8047) Add support for the .proto file extension to .clang-format
[
https://issues.apache.org/jira/browse/IMPALA-8047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736414#comment-16736414
]
ASF subversion and git services commented on IMPALA-8047:
-
Commit c7ff26a0435ec0bfcd228725c834da406baeb34c in impala's branch
refs/heads/master from Andrew Sherman
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=c7ff26a ]
IMPALA-8047 Support .proto files in .clang-format
The .proto file extension is used for the Google Protocol Buffers
language. Impala uses this language to specify the format of messages
used by KRPC. Add support for this language to .clang-format so that we
can have consistent formatting.
The proposed support is:
Language: Proto
BasedOnStyle: Google
ColumnLimit: 90
This produces only a few diffs when run against the existing Impala
code. I’m not proposing to make any changes to .proto files, this is
just to show what clang-format will do. Apart from wrapping comments and
code at 90 chars, the diffs are mostly of the form
-syntax="proto2";
+syntax = "proto2";
- message Certificate {};
+ message Certificate {
+ };
- optional bool client_timeout_defined = 4 [ default = false ];
+ optional bool client_timeout_defined = 4 [default = false];
-UNKNOWN= 999;
-NEGOTIATE = 1;
-SASL_SUCCESS = 0;
-SASL_INITIATE = 2;
+UNKNOWN = 999;
+NEGOTIATE = 1;
+SASL_SUCCESS = 0;
+SASL_INITIATE = 2;
This last change can be configured using “AlignConsecutiveAssignments:
true” but that creates a different set of diffs.
Change-Id: I0c2dcfc21fc8f9206adb64166fbd05580516791f
Reviewed-on: http://gerrit.cloudera.org:8080/12165
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> Add support for the .proto file extension to .clang-format
> ---
>
> Key: IMPALA-8047
> URL: https://issues.apache.org/jira/browse/IMPALA-8047
> Project: IMPALA
> Issue Type: Improvement
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
>Priority: Major
>
> The .proto file extension is used for the Google Protocol Buffers language.
> Impala uses this language to specify the format of messages used by KRPC. Add
> support for this language to .clang-format so that we can have consistent
> formatting.
> The proposed support is:
> {{Language: Proto
> BasedOnStyle: Google
> ColumnLimit: 90}}
> This produces only a few diffs when run against the existing Impala code. I’m
> not proposing to make any changes to .proto files, this is just to show what
> clang-format will do. Apart from wrapping comments and code at 90 chars, the
> diffs are mostly of the form
> {{-syntax="proto2";
> +syntax = "proto2";}}
> {{- message Certificate {};
> + message Certificate {
> + };}}
> {{- optional bool client_timeout_defined = 4 [ default = false ];
> + optional bool client_timeout_defined = 4 [default = false];}}
> {{- UNKNOWN = 999;
> - NEGOTIATE = 1;
> - SASL_SUCCESS = 0;
> - SASL_INITIATE = 2;
> + UNKNOWN = 999;
> + NEGOTIATE = 1;
> + SASL_SUCCESS = 0;
> + SASL_INITIATE = 2;}}
> This last change can be configured using “AlignConsecutiveAssignments: true”
> but that creates a different set of diffs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-7550) Create documentation for all profile fields
[ https://issues.apache.org/jira/browse/IMPALA-7550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Volker reassigned IMPALA-7550: --- Assignee: Lars Volker > Create documentation for all profile fields > --- > > Key: IMPALA-7550 > URL: https://issues.apache.org/jira/browse/IMPALA-7550 > Project: IMPALA > Issue Type: Improvement > Components: Docs >Affects Versions: Impala 3.1.0 >Reporter: Lars Volker >Assignee: Lars Volker >Priority: Major > Labels: supportability > > It would be great to have some documentation for all profile fields. > Currently users often have to look at the source code to figure out the exact > meaning of a field. Instead we should investigate ways to generate > documentation on the fields, e.g. by cleaning and automatically extracting > relevant comments from the source code. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8043) ExprTest fails on Ubuntu 16 when the timezone is America/Los_Angeles
[
https://issues.apache.org/jira/browse/IMPALA-8043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736173#comment-16736173
]
Attila Jeges commented on IMPALA-8043:
--
1. Some BE tests (like ExprTest.TimestampFunctions) use the system's default
timezone but run against a test timezone db (instead of the system's default
timezone db).
2. On some Ubuntu installations /usr/share/zoneinfo/America/Los_Angeles is a
symlink to ../SystemV/PST8PDT.
3. The SystemV directory is not part of the test timezone db.
If the system's default timezone is set to America/Los_Angeles, Impala BE tests
will default to UTC instead and consequently some tests will fail.
I'll look into fixing this shortly.
> ExprTest fails on Ubuntu 16 when the timezone is America/Los_Angeles
>
>
> Key: IMPALA-8043
> URL: https://issues.apache.org/jira/browse/IMPALA-8043
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: Joe McDonnell
>Assignee: Attila Jeges
>Priority: Major
>
> When setting up an Ubuntu development environment, I installed Ubuntu 16.04
> using America/Los_Angeles as the timezone. When running on this
> configuration, ExprTest.TimestampFunctions() fails with several timezone
> issues:
> {noformat}
> [ RUN ] ExprTest.TimestampFunctions
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: -28800
> Expected: expected_result
> Which is: 0
> unix_timestamp(cast('1969-12-31 16:00:00' as timestamp))
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: -28800
> Expected: expected_result
> Which is: 0
> unix_timestamp('1969-12-31 16:00:00')
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: -28800
> Expected: expected_result
> Which is: 0
> unix_timestamp('1969-12-31 16:00:00', '-MM-dd HH:mm:ss')
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: 0
> Expected: expected_result
> Which is: 28800
> unix_timestamp('1970-01-01', '-MM-dd')
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: 0
> Expected: expected_result
> Which is: 28800
> unix_timestamp('1970-01-01 10:10:10', '-MM-dd')
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: -28800
> Expected: expected_result
> Which is: 0
> unix_timestamp('1969-12-31 16:00:00 extra text', '-MM-dd HH:mm:ss')
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> cast(cast(0 as timestamp) as string)
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> cast(cast(0 as timestamp) as string)
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> from_unixtime(0)
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> from_unixtime(cast(0 as bigint))
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> from_unixtime(0, '-MM-dd HH:mm:ss')
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> from_unixtime(cast(0 as bigint), '-MM-dd HH:mm:ss')
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 08:00:00"
> Expected: expected_result
> Which is: "1970-01-01 00:00:00"
> cast(to_timestamp(28800) as string)
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:445: Failure
> Expected: (value) <= (end), actual: 1546465102 vs 1546436302
> Google Test trace:
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:6572:
> [ FAILED ] ExprTest.TimestampFunctions (3577 ms)
> {noformat}
> There are a couple workarounds to this:
> {noformat}
> # Set UTC as timezone
> sudo timedatectl set-timezone UTC
> # Use actual timezone name (e.g. PST8PDT rather than America/Los_Angeles
> # or CST6CDT rather than America/Chicago)
> sudo timedatectl set-timezone
[jira] [Assigned] (IMPALA-7985) Port RemoteShutdown() to KRPC
[ https://issues.apache.org/jira/browse/IMPALA-7985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman reassigned IMPALA-7985: -- Assignee: Andrew Sherman > Port RemoteShutdown() to KRPC > - > > Key: IMPALA-7985 > URL: https://issues.apache.org/jira/browse/IMPALA-7985 > Project: IMPALA > Issue Type: Sub-task > Components: Distributed Exec >Affects Versions: Impala 3.1.0 >Reporter: Michael Ho >Assignee: Andrew Sherman >Priority: Major > Labels: ramp-up > > Port RemoteShutdown() to KRPC. It's currently implemented as Thrift RPC. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7778) RCFile parser ignores escape characters
[
https://issues.apache.org/jira/browse/IMPALA-7778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong updated IMPALA-7778:
--
Priority: Minor (was: Major)
> RCFile parser ignores escape characters
> ---
>
> Key: IMPALA-7778
> URL: https://issues.apache.org/jira/browse/IMPALA-7778
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 2.0, Impala 2.1, Impala 2.2, Impala 2.3.0, Impala
> 2.5.0, Impala 2.4.0, Impala 2.6.0, Impala 2.7.0, Impala 2.8.0, Impala 2.9.0,
> Impala 2.10.0, Impala 2.11.0, Impala 3.0, Impala 3.1.0
>Reporter: Tim Armstrong
>Priority: Minor
> Labels: correctness
>
> If an RCFile table has an escape character specified then it is ignored by
> Impala.
> {code}
> -- HIVE
> CREATE TABLE rc_escape ( s string)
> ROW FORMAT delimited fields terminated by ',' escaped by '\\'
> STORED AS RCFILE;
> insert into rc_escape select '\\"';
> select length(s), s from rc_escape;
> -- +-+-+
> -- | c0 | s |
> -- +-+-+
> -- | 2 | \" |
> -- +-+-+
> -- IMPALA
> invalidate metadata rc_escape;
> select length(s), s from rc_escape;
> -- +---+-+
> -- | length(s) | s |
> -- +---+-+
> -- | 3 | \\" |
> -- +---+-+
> {code}
> I reproduced on my dev env with "beeline -n $USER -u
> jdbc:hive2://localhost:11050/default" for Hive and "impala-shell" for Impala.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7931) test_shutdown_executor fails with timeout waiting for query target state
[
https://issues.apache.org/jira/browse/IMPALA-7931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736116#comment-16736116
]
ASF subversion and git services commented on IMPALA-7931:
-
Commit a91b24cb7962200f330c4887f38f4704a52f7c7e in impala's branch
refs/heads/master from Tim Armstrong
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=a91b24c ]
IMPALA-7931: fix executor shutdown races
There were two races:
* queries were terminated because of an impalad being detected
as failed by the statestore even if the query had finished
executing on that impalad.
* NUM_FRAGMENTS_IN_FLIGHT was used to detect the backend being
idle, but it was decremented before the final status report
was sent.
The fixes are:
* keep track of the backends that triggered the potential cancellation,
and only proceed with the cancellation if the coordinator has fragments
still executing on the backend.
* add a new metric that keeps track of the number of executing queries,
which isn't decremented until the final status report is sent.
Also do some cleanup/improvements in this code:
* use proper error codes for some errors
* more overloads for Status::Expected()
* also add a metric for the total number of queries executed on the
backend
Testing:
Add a new version of test_shutdown_executor with delays that
trigger both races. This test only runs in exhaustive to avoid
adding ~20s to core build time.
Ran exhaustive tests.
Looped test_restart_services overnight.
Change-Id: I7c1a80304cb6695d228aca8314e2231727ab1998
Reviewed-on: http://gerrit.cloudera.org:8080/12082
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> test_shutdown_executor fails with timeout waiting for query target state
>
>
> Key: IMPALA-7931
> URL: https://issues.apache.org/jira/browse/IMPALA-7931
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: Lars Volker
>Assignee: Tim Armstrong
>Priority: Critical
> Labels: broken-build
> Attachments: impala-7931-impalad-logs.tar.gz
>
>
> On a recent S3 test run test_shutdown_executor hit a timeout waiting for a
> query to reach state FINISHED. Instead the query stays at state 5 (EXCEPTION).
> {noformat}
> 12:51:11 __ TestShutdownCommand.test_shutdown_executor
> __
> 12:51:11 custom_cluster/test_restart_services.py:209: in
> test_shutdown_executor
> 12:51:11 assert self.__fetch_and_get_num_backends(QUERY,
> before_shutdown_handle) == 3
> 12:51:11 custom_cluster/test_restart_services.py:356: in
> __fetch_and_get_num_backends
> 12:51:11 self.client.QUERY_STATES['FINISHED'], timeout=20)
> 12:51:11 common/impala_service.py:267: in wait_for_query_state
> 12:51:11 target_state, query_state)
> 12:51:11 E AssertionError: Did not reach query state in time target=4
> actual=5
> {noformat}
> From the logs I can see that the query fails because one of the executors
> becomes unreachable:
> {noformat}
> I1204 12:31:39.954125 5609 impala-server.cc:1792] Query
> a34c3a84775e5599:b2b25eb9: Failed due to unreachable impalad(s):
> jenkins-worker:22001
> {noformat}
> The query was {{select count\(*) from functional_parquet.alltypes where
> sleep(1) = bool_col}}.
> It seems that the query took longer than expected and was still running when
> the executor shut down.
> I can reproduce by adding a sleep to the test:
> {noformat}
> diff --git a/tests/custom_cluster/test_restart_services.py
> b/tests/custom_cluster/test_restart_services.py
> index e441cbc..32bc8a1 100644
> --- a/tests/custom_cluster/test_restart_services.py
> +++ b/tests/custom_cluster/test_restart_services.py
> @@ -206,7 +206,7 @@ class TestShutdownCommand(CustomClusterTestSuite,
> HS2TestSuite):
> after_shutdown_handle = self.__exec_and_wait_until_running(QUERY)
>
> # Finish executing the first query before the backend exits.
> -assert self.__fetch_and_get_num_backends(QUERY, before_shutdown_handle)
> == 3
> +assert self.__fetch_and_get_num_backends(QUERY, before_shutdown_handle,
> delay=5) == 3
>
> # Wait for the impalad to exit, then start it back up and run another
> query, which
> # should be scheduled on it again.
> @@ -349,11 +349,14 @@ class TestShutdownCommand(CustomClusterTestSuite,
> HS2TestSuite):
> self.client.QUERY_STATES['RUNNING'], timeout=20)
> return handle
>
> - def __fetch_and_get_num_backends(self, query, handle):
> + def __fetch_and_get_num_backends(self, query, handle, delay=0):
> """Fetch the results of 'query' from the beeswax handle 'handle', close
> the
> query and return the number of backends obtained from the profile."""
>
[jira] [Commented] (IMPALA-6742) Profiles of running queries should include execution summary
[ https://issues.apache.org/jira/browse/IMPALA-6742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736117#comment-16736117 ] ASF subversion and git services commented on IMPALA-6742: - Commit e9652a48dd00c3c076ddccaa940d074b6970b7fc in impala's branch refs/heads/master from Yongjun Zhang [ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=e9652a4 ] IMPALA-6742: Profiles of running queries should include execution summary. Currently execution summary is not included in the profiles of running queries, and it's only reported when the query is finished. This jira makes the execution summary to the profile reported when queries are still running. Testing: Added unit test. Done with real cluster. Change-Id: Idc7f714c9427d4b26d4e78cf27ceca2b0b336699 Reviewed-on: http://gerrit.cloudera.org:8080/11591 Reviewed-by: Joe McDonnell Tested-by: Impala Public Jenkins > Profiles of running queries should include execution summary > > > Key: IMPALA-6742 > URL: https://issues.apache.org/jira/browse/IMPALA-6742 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Balazs Jeszenszky >Assignee: Yongjun Zhang >Priority: Minor > > The profile omits the execution summary when taken off of a running query. It > would be helpful to have it there as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8052) Fail to run ExprTest.TimestampFunctions
[
https://issues.apache.org/jira/browse/IMPALA-8052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736062#comment-16736062
]
Tim Armstrong commented on IMPALA-8052:
---
[~stiga-huang] looks like others have seen this too.
> Fail to run ExprTest.TimestampFunctions
> ---
>
> Key: IMPALA-8052
> URL: https://issues.apache.org/jira/browse/IMPALA-8052
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.1.0
>Reporter: Quanlong Huang
>Priority: Major
>
> Encountered the following test errors from ExprTest.TimestampFunctions:
> {code:java}
> [ RUN ] ExprTest.TimestampFunctions
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: -28800
> Expected: expected_result
> Which is: 0
> unix_timestamp(cast('1969-12-31 16:00:00' as timestamp))
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: -28800
> Expected: expected_result
> Which is: 0
> unix_timestamp('1969-12-31 16:00:00')
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: -28800
> Expected: expected_result
> Which is: 0
> unix_timestamp('1969-12-31 16:00:00', '-MM-dd HH:mm:ss')
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: 0
> Expected: expected_result
> Which is: 28800
> unix_timestamp('1970-01-01', '-MM-dd')
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: 0
> Expected: expected_result
> Which is: 28800
> unix_timestamp('1970-01-01 10:10:10', '-MM-dd')
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: -28800
> Expected: expected_result
> Which is: 0
> unix_timestamp('1969-12-31 16:00:00 extra text', '-MM-dd HH:mm:ss')
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> cast(cast(0 as timestamp) as string)
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> cast(cast(0 as timestamp) as string)
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> from_unixtime(0)
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> from_unixtime(cast(0 as bigint))
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> from_unixtime(0, '-MM-dd HH:mm:ss')
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> from_unixtime(cast(0 as bigint), '-MM-dd HH:mm:ss')
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 08:00:00"
> Expected: expected_result
> Which is: "1970-01-01 00:00:00"
> cast(to_timestamp(28800) as string)
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:445: Failure
> Expected: (value) <= (end), actual: 1546742379 vs 1546713579
> Google Test trace:
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:6572:
> [ FAILED ] ExprTest.TimestampFunctions (6409 ms)
> [ RUN ] ExprTest.ConditionalFunctions
> [ OK ] ExprTest.ConditionalFunctions (1623 ms)
> [ RUN ] ExprTest.ConditionalFunctionIsTrue
> [ OK ] ExprTest.ConditionalFunctionIsTrue (105 ms)
> [ RUN ] ExprTest.ConditionalFunctionIsFalse
> [ OK ] ExprTest.ConditionalFunctionIsFalse (105 ms)
> [ RUN ] ExprTest.ConditionalFunctionIsNotTrue
> [ OK ] ExprTest.ConditionalFunctionIsNotTrue (105 ms)
> [ RUN ] ExprTest.ConditionalFunctionIsNotFalse
> [ OK ] ExprTest.ConditionalFunctionIsNotFalse (102 ms)
> [ RUN ] ExprTest.ResultsLayoutTest
> [ OK ] ExprTest.ResultsLayoutTest (0 ms)
> [ RUN ] ExprTest.DecimalFunctions
> [ OK ] ExprTest.DecimalFunctions (2990 ms)
> [ RUN ] ExprTest.DecimalOverflowCastsDecimalV1
> [ OK ]
[jira] [Assigned] (IMPALA-8043) ExprTest fails on Ubuntu 16 when the timezone is America/Los_Angeles
[
https://issues.apache.org/jira/browse/IMPALA-8043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong reassigned IMPALA-8043:
-
Assignee: Attila Jeges
> ExprTest fails on Ubuntu 16 when the timezone is America/Los_Angeles
>
>
> Key: IMPALA-8043
> URL: https://issues.apache.org/jira/browse/IMPALA-8043
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: Joe McDonnell
>Assignee: Attila Jeges
>Priority: Major
>
> When setting up an Ubuntu development environment, I installed Ubuntu 16.04
> using America/Los_Angeles as the timezone. When running on this
> configuration, ExprTest.TimestampFunctions() fails with several timezone
> issues:
> {noformat}
> [ RUN ] ExprTest.TimestampFunctions
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: -28800
> Expected: expected_result
> Which is: 0
> unix_timestamp(cast('1969-12-31 16:00:00' as timestamp))
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: -28800
> Expected: expected_result
> Which is: 0
> unix_timestamp('1969-12-31 16:00:00')
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: -28800
> Expected: expected_result
> Which is: 0
> unix_timestamp('1969-12-31 16:00:00', '-MM-dd HH:mm:ss')
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: 0
> Expected: expected_result
> Which is: 28800
> unix_timestamp('1970-01-01', '-MM-dd')
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: 0
> Expected: expected_result
> Which is: 28800
> unix_timestamp('1970-01-01 10:10:10', '-MM-dd')
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: -28800
> Expected: expected_result
> Which is: 0
> unix_timestamp('1969-12-31 16:00:00 extra text', '-MM-dd HH:mm:ss')
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> cast(cast(0 as timestamp) as string)
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> cast(cast(0 as timestamp) as string)
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> from_unixtime(0)
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> from_unixtime(cast(0 as bigint))
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> from_unixtime(0, '-MM-dd HH:mm:ss')
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> from_unixtime(cast(0 as bigint), '-MM-dd HH:mm:ss')
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 08:00:00"
> Expected: expected_result
> Which is: "1970-01-01 00:00:00"
> cast(to_timestamp(28800) as string)
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:445: Failure
> Expected: (value) <= (end), actual: 1546465102 vs 1546436302
> Google Test trace:
> /home/joe/view2/Impala/be/src/exprs/expr-test.cc:6572:
> [ FAILED ] ExprTest.TimestampFunctions (3577 ms)
> {noformat}
> There are a couple workarounds to this:
> {noformat}
> # Set UTC as timezone
> sudo timedatectl set-timezone UTC
> # Use actual timezone name (e.g. PST8PDT rather than America/Los_Angeles
> # or CST6CDT rather than America/Chicago)
> sudo timedatectl set-timezone PST8PDT{noformat}
> This looks like it is coming from the tests using
> TestTimestampUnixEpochConversions(). The tests work on other timezones
> (Americas/La_Paz, Europe/Paris, Asia/Tokyo all work). It is likely that this
> is a duplicate of some other known issue.
> Filing this Jira so that others who run into this might find this workaround.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail:
[jira] [Resolved] (IMPALA-8052) Fail to run ExprTest.TimestampFunctions
[
https://issues.apache.org/jira/browse/IMPALA-8052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-8052.
---
Resolution: Duplicate
> Fail to run ExprTest.TimestampFunctions
> ---
>
> Key: IMPALA-8052
> URL: https://issues.apache.org/jira/browse/IMPALA-8052
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.1.0
>Reporter: Quanlong Huang
>Priority: Major
>
> Encountered the following test errors from ExprTest.TimestampFunctions:
> {code:java}
> [ RUN ] ExprTest.TimestampFunctions
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: -28800
> Expected: expected_result
> Which is: 0
> unix_timestamp(cast('1969-12-31 16:00:00' as timestamp))
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: -28800
> Expected: expected_result
> Which is: 0
> unix_timestamp('1969-12-31 16:00:00')
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: -28800
> Expected: expected_result
> Which is: 0
> unix_timestamp('1969-12-31 16:00:00', '-MM-dd HH:mm:ss')
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: 0
> Expected: expected_result
> Which is: 28800
> unix_timestamp('1970-01-01', '-MM-dd')
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: 0
> Expected: expected_result
> Which is: 28800
> unix_timestamp('1970-01-01 10:10:10', '-MM-dd')
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: -28800
> Expected: expected_result
> Which is: 0
> unix_timestamp('1969-12-31 16:00:00 extra text', '-MM-dd HH:mm:ss')
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> cast(cast(0 as timestamp) as string)
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> cast(cast(0 as timestamp) as string)
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> from_unixtime(0)
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> from_unixtime(cast(0 as bigint))
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> from_unixtime(0, '-MM-dd HH:mm:ss')
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> from_unixtime(cast(0 as bigint), '-MM-dd HH:mm:ss')
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 08:00:00"
> Expected: expected_result
> Which is: "1970-01-01 00:00:00"
> cast(to_timestamp(28800) as string)
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:445: Failure
> Expected: (value) <= (end), actual: 1546742379 vs 1546713579
> Google Test trace:
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:6572:
> [ FAILED ] ExprTest.TimestampFunctions (6409 ms)
> [ RUN ] ExprTest.ConditionalFunctions
> [ OK ] ExprTest.ConditionalFunctions (1623 ms)
> [ RUN ] ExprTest.ConditionalFunctionIsTrue
> [ OK ] ExprTest.ConditionalFunctionIsTrue (105 ms)
> [ RUN ] ExprTest.ConditionalFunctionIsFalse
> [ OK ] ExprTest.ConditionalFunctionIsFalse (105 ms)
> [ RUN ] ExprTest.ConditionalFunctionIsNotTrue
> [ OK ] ExprTest.ConditionalFunctionIsNotTrue (105 ms)
> [ RUN ] ExprTest.ConditionalFunctionIsNotFalse
> [ OK ] ExprTest.ConditionalFunctionIsNotFalse (102 ms)
> [ RUN ] ExprTest.ResultsLayoutTest
> [ OK ] ExprTest.ResultsLayoutTest (0 ms)
> [ RUN ] ExprTest.DecimalFunctions
> [ OK ] ExprTest.DecimalFunctions (2990 ms)
> [ RUN ] ExprTest.DecimalOverflowCastsDecimalV1
> [ OK ] ExprTest.DecimalOverflowCastsDecimalV1 (357 ms)
> [ RUN ]
[jira] [Resolved] (IMPALA-8052) Fail to run ExprTest.TimestampFunctions
[
https://issues.apache.org/jira/browse/IMPALA-8052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-8052.
---
Resolution: Duplicate
> Fail to run ExprTest.TimestampFunctions
> ---
>
> Key: IMPALA-8052
> URL: https://issues.apache.org/jira/browse/IMPALA-8052
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.1.0
>Reporter: Quanlong Huang
>Priority: Major
>
> Encountered the following test errors from ExprTest.TimestampFunctions:
> {code:java}
> [ RUN ] ExprTest.TimestampFunctions
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: -28800
> Expected: expected_result
> Which is: 0
> unix_timestamp(cast('1969-12-31 16:00:00' as timestamp))
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: -28800
> Expected: expected_result
> Which is: 0
> unix_timestamp('1969-12-31 16:00:00')
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: -28800
> Expected: expected_result
> Which is: 0
> unix_timestamp('1969-12-31 16:00:00', '-MM-dd HH:mm:ss')
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: 0
> Expected: expected_result
> Which is: 28800
> unix_timestamp('1970-01-01', '-MM-dd')
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: 0
> Expected: expected_result
> Which is: 28800
> unix_timestamp('1970-01-01 10:10:10', '-MM-dd')
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:559: Failure
> Value of: ConvertValue(result)
> Actual: -28800
> Expected: expected_result
> Which is: 0
> unix_timestamp('1969-12-31 16:00:00 extra text', '-MM-dd HH:mm:ss')
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> cast(cast(0 as timestamp) as string)
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> cast(cast(0 as timestamp) as string)
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> from_unixtime(0)
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> from_unixtime(cast(0 as bigint))
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> from_unixtime(0, '-MM-dd HH:mm:ss')
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 00:00:00"
> Expected: expected_result
> Which is: "1969-12-31 16:00:00"
> from_unixtime(cast(0 as bigint), '-MM-dd HH:mm:ss')
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:289: Failure
> Value of: GetValue(expr, TYPE_STRING)
> Actual: "1970-01-01 08:00:00"
> Expected: expected_result
> Which is: "1970-01-01 00:00:00"
> cast(to_timestamp(28800) as string)
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:445: Failure
> Expected: (value) <= (end), actual: 1546742379 vs 1546713579
> Google Test trace:
> /tmp/jenkins/workspace/impala-hulu/be/src/exprs/expr-test.cc:6572:
> [ FAILED ] ExprTest.TimestampFunctions (6409 ms)
> [ RUN ] ExprTest.ConditionalFunctions
> [ OK ] ExprTest.ConditionalFunctions (1623 ms)
> [ RUN ] ExprTest.ConditionalFunctionIsTrue
> [ OK ] ExprTest.ConditionalFunctionIsTrue (105 ms)
> [ RUN ] ExprTest.ConditionalFunctionIsFalse
> [ OK ] ExprTest.ConditionalFunctionIsFalse (105 ms)
> [ RUN ] ExprTest.ConditionalFunctionIsNotTrue
> [ OK ] ExprTest.ConditionalFunctionIsNotTrue (105 ms)
> [ RUN ] ExprTest.ConditionalFunctionIsNotFalse
> [ OK ] ExprTest.ConditionalFunctionIsNotFalse (102 ms)
> [ RUN ] ExprTest.ResultsLayoutTest
> [ OK ] ExprTest.ResultsLayoutTest (0 ms)
> [ RUN ] ExprTest.DecimalFunctions
> [ OK ] ExprTest.DecimalFunctions (2990 ms)
> [ RUN ] ExprTest.DecimalOverflowCastsDecimalV1
> [ OK ] ExprTest.DecimalOverflowCastsDecimalV1 (357 ms)
> [ RUN ]
[jira] [Commented] (IMPALA-6503) Support reading complex types from ORC format files
[ https://issues.apache.org/jira/browse/IMPALA-6503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16735923#comment-16735923 ] Quanlong Huang commented on IMPALA-6503: I spent a lot of time fixing my implementation due to some wrong assumptions about the descriptors. However, I still make one assumption: 'col_path' of the SlotDescriptor won't map to a STRUCT column. It will point directly to a primitive/collection column. Please give me cases if I'm wrong. Patch ready for review: [https://gerrit.cloudera.org/#/c/12168/] Doc about the patch: [https://docs.google.com/document/d/10Gzcge57VUOQ0ZQWfWiIW9cUix8_FumMewl0Gja3ZA4] Have passed the pre-review-test: [https://jenkins.impala.io/job/pre-review-test/269/] > Support reading complex types from ORC format files > --- > > Key: IMPALA-6503 > URL: https://issues.apache.org/jira/browse/IMPALA-6503 > Project: IMPALA > Issue Type: New Feature > Components: Backend, Frontend >Reporter: Quanlong Huang >Assignee: Quanlong Huang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
