[jira] [Commented] (IMPALA-9908) SHOW PARTITIONS on a kudu backed impala table return NULL on Replicas
[
https://issues.apache.org/jira/browse/IMPALA-9908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148154#comment-17148154
]
Xiaomeng Zhang commented on IMPALA-9908:
>From debugging, I can see schema is transferred from
impala::TYPE_INT to
{apache::hive::service::cli::thrift::TTypeEntry & }
apache::hive::service::cli::thrift::TTypeId::INT_TYPE,
while in beeswex it's transferred from
Impala::TPrimitiveType::INT to
{Apache::Hadoop::Hive::FieldSchema & }
Type = “int”
> SHOW PARTITIONS on a kudu backed impala table return NULL on Replicas
> -
>
> Key: IMPALA-9908
> URL: https://issues.apache.org/jira/browse/IMPALA-9908
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 4.0
>Reporter: Xiaomeng Zhang
>Priority: Major
>
> When using hs2 protocol, we get NULL Replicas for SHOW PARTITIONS on a kudu
> backed impala table.
> To reproduce:
> impala-shell --protocol=hs2
> create table xm(id int, value int, PRIMARY KEY(id)) PARTITION BY HASH
> PARTITIONS 16 STORED AS KUDU;
> [localhost:21050] default> show partitions xm;
> Query: show partitions xm
> +---+--+-+---+
> | Start Key | Stop Key | Leader Replica | #Replicas |
> +---+--+-+---+
> | | 0001 | 127.0.0.1:31201 | NULL |
> | 0001 | 0002 | 127.0.0.1:31202 | NULL |
> | 0002 | 0003 | 127.0.0.1:31202 | NULL |
> | 0003 | 0004 | 127.0.0.1:31202 | NULL |
> | 0004 | 0005 | 127.0.0.1:31202 | NULL |
> | 0005 | 0006 | 127.0.0.1:31202 | NULL |
> | 0006 | 0007 | 127.0.0.1:31202 | NULL |
> | 0007 | 0008 | 127.0.0.1:31200 | NULL |
> | 0008 | 0009 | 127.0.0.1:31201 | NULL |
> | 0009 | 000A | 127.0.0.1:31202 | NULL |
> | 000A | 000B | 127.0.0.1:31200 | NULL |
> | 000B | 000C | 127.0.0.1:31202 | NULL |
> | 000C | 000D | 127.0.0.1:31201 | NULL |
> | 000D | 000E | 127.0.0.1:31202 | NULL |
> | 000E | 000F | 127.0.0.1:31201 | NULL |
> | 000F | | 127.0.0.1:31202 | NULL |
> +---+--+-+---+
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-9908) SHOW PARTITIONS on a kudu backed impala table return NULL on Replicas
[
https://issues.apache.org/jira/browse/IMPALA-9908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148154#comment-17148154
]
Xiaomeng Zhang edited comment on IMPALA-9908 at 6/29/20, 9:38 PM:
--
>From debugging, I can see schema in hs2 protocol is transferred from
impala::TYPE_INT to
{apache::hive::service::cli::thrift::TTypeEntry & }
apache::hive::service::cli::thrift::TTypeId::INT_TYPE,
while in beeswex protocol it's transferred from
Impala::TPrimitiveType::INT to
{Apache::Hadoop::Hive::FieldSchema & }
Type = “int”
was (Author: xiaomeng zhang):
>From debugging, I can see schema is transferred from
impala::TYPE_INT to
{apache::hive::service::cli::thrift::TTypeEntry & }
apache::hive::service::cli::thrift::TTypeId::INT_TYPE,
while in beeswex it's transferred from
Impala::TPrimitiveType::INT to
{Apache::Hadoop::Hive::FieldSchema & }
Type = “int”
> SHOW PARTITIONS on a kudu backed impala table return NULL on Replicas
> -
>
> Key: IMPALA-9908
> URL: https://issues.apache.org/jira/browse/IMPALA-9908
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 4.0
>Reporter: Xiaomeng Zhang
>Priority: Major
>
> When using hs2 protocol, we get NULL Replicas for SHOW PARTITIONS on a kudu
> backed impala table.
> To reproduce:
> impala-shell --protocol=hs2
> create table xm(id int, value int, PRIMARY KEY(id)) PARTITION BY HASH
> PARTITIONS 16 STORED AS KUDU;
> [localhost:21050] default> show partitions xm;
> Query: show partitions xm
> +---+--+-+---+
> | Start Key | Stop Key | Leader Replica | #Replicas |
> +---+--+-+---+
> | | 0001 | 127.0.0.1:31201 | NULL |
> | 0001 | 0002 | 127.0.0.1:31202 | NULL |
> | 0002 | 0003 | 127.0.0.1:31202 | NULL |
> | 0003 | 0004 | 127.0.0.1:31202 | NULL |
> | 0004 | 0005 | 127.0.0.1:31202 | NULL |
> | 0005 | 0006 | 127.0.0.1:31202 | NULL |
> | 0006 | 0007 | 127.0.0.1:31202 | NULL |
> | 0007 | 0008 | 127.0.0.1:31200 | NULL |
> | 0008 | 0009 | 127.0.0.1:31201 | NULL |
> | 0009 | 000A | 127.0.0.1:31202 | NULL |
> | 000A | 000B | 127.0.0.1:31200 | NULL |
> | 000B | 000C | 127.0.0.1:31202 | NULL |
> | 000C | 000D | 127.0.0.1:31201 | NULL |
> | 000D | 000E | 127.0.0.1:31202 | NULL |
> | 000E | 000F | 127.0.0.1:31201 | NULL |
> | 000F | | 127.0.0.1:31202 | NULL |
> +---+--+-+---+
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-9908) SHOW PARTITIONS on a kudu backed impala table return NULL on Replicas
Xiaomeng Zhang created IMPALA-9908: -- Summary: SHOW PARTITIONS on a kudu backed impala table return NULL on Replicas Key: IMPALA-9908 URL: https://issues.apache.org/jira/browse/IMPALA-9908 Project: IMPALA Issue Type: Bug Affects Versions: Impala 4.0 Reporter: Xiaomeng Zhang When using hs2 protocol, we get NULL Replicas for SHOW PARTITIONS on a kudu backed impala table. To reproduce: impala-shell --protocol=hs2 create table xm(id int, value int, PRIMARY KEY(id)) PARTITION BY HASH PARTITIONS 16 STORED AS KUDU; [localhost:21050] default> show partitions xm; Query: show partitions xm +---+--+-+---+ | Start Key | Stop Key | Leader Replica | #Replicas | +---+--+-+---+ | | 0001 | 127.0.0.1:31201 | NULL | | 0001 | 0002 | 127.0.0.1:31202 | NULL | | 0002 | 0003 | 127.0.0.1:31202 | NULL | | 0003 | 0004 | 127.0.0.1:31202 | NULL | | 0004 | 0005 | 127.0.0.1:31202 | NULL | | 0005 | 0006 | 127.0.0.1:31202 | NULL | | 0006 | 0007 | 127.0.0.1:31202 | NULL | | 0007 | 0008 | 127.0.0.1:31200 | NULL | | 0008 | 0009 | 127.0.0.1:31201 | NULL | | 0009 | 000A | 127.0.0.1:31202 | NULL | | 000A | 000B | 127.0.0.1:31200 | NULL | | 000B | 000C | 127.0.0.1:31202 | NULL | | 000C | 000D | 127.0.0.1:31201 | NULL | | 000D | 000E | 127.0.0.1:31202 | NULL | | 000E | 000F | 127.0.0.1:31201 | NULL | | 000F | | 127.0.0.1:31202 | NULL | +---+--+-+---+ -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-9802) TestCompressedFormats.test_compressed_formats fails in HDFS copy
[
https://issues.apache.org/jira/browse/IMPALA-9802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang resolved IMPALA-9802.
Fix Version/s: Impala 4.0
Resolution: Fixed
> TestCompressedFormats.test_compressed_formats fails in HDFS copy
>
>
> Key: IMPALA-9802
> URL: https://issues.apache.org/jira/browse/IMPALA-9802
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 4.0
>Reporter: Quanlong Huang
>Assignee: Xiaomeng Zhang
>Priority: Critical
> Labels: broken-build
> Fix For: Impala 4.0
>
>
> See this fails in rc and seq file formats:
> query_test.test_compressed_formats.TestCompressedFormats.test_compressed_formats[compression_format:
> ('.bz2', 'bzip') | file_format: rc]
> query_test.test_compressed_formats.TestCompressedFormats.test_compressed_formats[compression_format:
> ('.deflate', 'def') | file_format: rc]
> query_test.test_compressed_formats.TestCompressedFormats.test_compressed_formats[compression_format:
> ('.gz', 'gzip') | file_format: rc]
> query_test.test_compressed_formats.TestCompressedFormats.test_compressed_formats[compression_format:
> ('.snappy', 'snap') | file_format: rc]
> query_test.test_compressed_formats.TestCompressedFormats.test_compressed_formats[compression_format:
> ('.snappy', 'snap') | file_format: seq]
> query_test.test_compressed_formats.TestCompressedFormats.test_compressed_formats[compression_format:
> ('.gz', 'gzip') | file_format: seq]
> query_test.test_compressed_formats.TestCompressedFormats.test_compressed_formats[compression_format:
> ('.deflate', 'def') | file_format: seq]
> query_test.test_compressed_formats.TestCompressedFormats.test_compressed_formats[compression_format:
> ('.bz2', 'bzip') | file_format: seq]
> One of the stacktrace:
> {code:java}
> query_test/test_compressed_formats.py:83: in test_compressed_formats
> 'tinytable', db_suffix, suffix, '00_0', extension)
> query_test/test_compressed_formats.py:114: in _copy_and_query_compressed_file
> self.filesystem_client.copy(src_file, dest_file, overwrite=True)
> /data/jenkins/workspace/impala-cdpd-master-exhaustive/repos/Impala/tests/util/hdfs_util.py:79:
> in copy
> self.hdfs_filesystem_client.copy(src, dst, overwrite)
> /data/jenkins/workspace/impala-cdpd-master-exhaustive/repos/Impala/tests/util/hdfs_util.py:241:
> in copy
> '{0} copy failed: '.format(self.filesystem_type) + stderr + "; " + stdout
> E AssertionError: HDFS copy failed: cp:
> `/test-warehouse/managed/tinytable_snap_copy/00_0.snappy': No such file
> or directory:
> `hdfs://localhost:20500/test-warehouse/managed/tinytable_snap_copy/00_0.snappy'
> E ; {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-9673) Tests expecting results to be in test-warehouse/managed but find test-warehouse
[
https://issues.apache.org/jira/browse/IMPALA-9673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang resolved IMPALA-9673.
Fix Version/s: Impala 4.0
Resolution: Fixed
> Tests expecting results to be in test-warehouse/managed but find
> test-warehouse
>
>
> Key: IMPALA-9673
> URL: https://issues.apache.org/jira/browse/IMPALA-9673
> Project: IMPALA
> Issue Type: Bug
>Reporter: Andrew Sherman
>Assignee: Xiaomeng Zhang
>Priority: Critical
> Fix For: Impala 4.0
>
>
> TestDdlStatements.test_create_database
> {code}
> ERROR:test_configuration:Comparing QueryTestResults (expected vs actual):
> 'test_create_database_b10825a1_2','hdfs://localhost:20500/test-warehouse/managed/test_create_database_b10825a1_2.db','For
> testing' !=
> 'test_create_database_b10825a1_2','hdfs://localhost:20500/test-warehouse/test_create_database_b10825a1_2.db','For
> testing'
> {code}
> TestMetadataQueryStatements.test_describe_db
> {code}
> ERROR:test_configuration:Comparing QueryTestResults (expected vs actual):
> 'default','hdfs://localhost:20500/test-warehouse/managed','Default Hive
> database' != 'default','hdfs://localhost:20500/test-warehouse','Default Hive
> database'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-9833) query_test.test_observability.TestQueryStates.test_error_query_state is flaky
Xiaomeng Zhang created IMPALA-9833:
--
Summary:
query_test.test_observability.TestQueryStates.test_error_query_state is flaky
Key: IMPALA-9833
URL: https://issues.apache.org/jira/browse/IMPALA-9833
Project: IMPALA
Issue Type: Bug
Affects Versions: Impala 4.0
Reporter: Xiaomeng Zhang
[https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/2521/testReport/junit/query_test.test_observability/TestQueryStates/test_error_query_state/]
{code:java}
Stacktracequery_test/test_observability.py:777: in test_error_query_state
lambda: self.client.get_runtime_profile(handle))
common/impala_test_suite.py:1120: in assert_eventually
count, timeout_s, error_msg_str))
E Timeout: Check failed to return True after 30 tries and 30 seconds error
message: Query (id=fe45e8bfd138acd3:c67a3796)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-9833) query_test.test_observability.TestQueryStates.test_error_query_state is flaky
[
https://issues.apache.org/jira/browse/IMPALA-9833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang updated IMPALA-9833:
---
Description:
[https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/2521/testReport/junit/query_test.test_observability/TestQueryStates/test_error_query_state/]
It seems the test could not get query profile after retries in 30s.
{code:java}
Stacktracequery_test/test_observability.py:777: in test_error_query_state
lambda: self.client.get_runtime_profile(handle))
common/impala_test_suite.py:1120: in assert_eventually
count, timeout_s, error_msg_str))
E Timeout: Check failed to return True after 30 tries and 30 seconds error
message: Query (id=fe45e8bfd138acd3:c67a3796)
{code}
was:
[https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/2521/testReport/junit/query_test.test_observability/TestQueryStates/test_error_query_state/]
{code:java}
Stacktracequery_test/test_observability.py:777: in test_error_query_state
lambda: self.client.get_runtime_profile(handle))
common/impala_test_suite.py:1120: in assert_eventually
count, timeout_s, error_msg_str))
E Timeout: Check failed to return True after 30 tries and 30 seconds error
message: Query (id=fe45e8bfd138acd3:c67a3796)
{code}
> query_test.test_observability.TestQueryStates.test_error_query_state is flaky
> -
>
> Key: IMPALA-9833
> URL: https://issues.apache.org/jira/browse/IMPALA-9833
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 4.0
>Reporter: Xiaomeng Zhang
>Priority: Major
>
> [https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/2521/testReport/junit/query_test.test_observability/TestQueryStates/test_error_query_state/]
> It seems the test could not get query profile after retries in 30s.
> {code:java}
> Stacktracequery_test/test_observability.py:777: in test_error_query_state
> lambda: self.client.get_runtime_profile(handle))
> common/impala_test_suite.py:1120: in assert_eventually
> count, timeout_s, error_msg_str))
> E Timeout: Check failed to return True after 30 tries and 30 seconds error
> message: Query (id=fe45e8bfd138acd3:c67a3796)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-9673) Tests expecting results to be in test-warehouse/managed but find test-warehouse
[
https://issues.apache.org/jira/browse/IMPALA-9673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106538#comment-17106538
]
Xiaomeng Zhang commented on IMPALA-9673:
The downstream Jira has more details of discussion. As the behavior confirmed
by Hive, I can work on the fix now.
> Tests expecting results to be in test-warehouse/managed but find
> test-warehouse
>
>
> Key: IMPALA-9673
> URL: https://issues.apache.org/jira/browse/IMPALA-9673
> Project: IMPALA
> Issue Type: Bug
>Reporter: Andrew Sherman
>Assignee: Xiaomeng Zhang
>Priority: Critical
>
> TestDdlStatements.test_create_database
> {code}
> ERROR:test_configuration:Comparing QueryTestResults (expected vs actual):
> 'test_create_database_b10825a1_2','hdfs://localhost:20500/test-warehouse/managed/test_create_database_b10825a1_2.db','For
> testing' !=
> 'test_create_database_b10825a1_2','hdfs://localhost:20500/test-warehouse/test_create_database_b10825a1_2.db','For
> testing'
> {code}
> TestMetadataQueryStatements.test_describe_db
> {code}
> ERROR:test_configuration:Comparing QueryTestResults (expected vs actual):
> 'default','hdfs://localhost:20500/test-warehouse/managed','Default Hive
> database' != 'default','hdfs://localhost:20500/test-warehouse','Default Hive
> database'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-9664) Insert events on transactional tables need to call addWriteNotificationLog API
[
https://issues.apache.org/jira/browse/IMPALA-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085962#comment-17085962
]
Xiaomeng Zhang commented on IMPALA-9664:
[~vihangk1] consider what we do for bulk insert, do we have to do same thing
for addWriteNotificationLog API?
> Insert events on transactional tables need to call addWriteNotificationLog API
> --
>
> Key: IMPALA-9664
> URL: https://issues.apache.org/jira/browse/IMPALA-9664
> Project: IMPALA
> Issue Type: Bug
>Reporter: Vihang Karajgaonkar
>Assignee: Xiaomeng Zhang
>Priority: Major
>
> According to what we see in Hive source code, for transactional tables, the
> insert events are fired with a different API {{addWriteNotificationLog}}.
> Currently Impala fires {{firelistenerEvent}} for both transactional and
> non-transactional tables. We should look at what is the difference between
> the two APIs and see if we need to handle transactional tables differently.
> References:
> https://github.com/apache/hive/blob/c3afb57bdb1041f566fbbd896f625328fc9656a0/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2402
> https://github.com/apache/hive/blob/c3afb57bdb1041f566fbbd896f625328fc9656a0/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2236
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-9664) Insert events on transactional tables need to call addWriteNotificationLog API
[
https://issues.apache.org/jira/browse/IMPALA-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085302#comment-17085302
]
Xiaomeng Zhang commented on IMPALA-9664:
Comparing InsertEvent and AcidWriteEvent, AcidWriteEvent has a variable of
writeId,
[https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/events/AcidWriteEvent.java#L76,|https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/events/AcidWriteEvent.java#L76]
which I think will be used by Hive while processing transactional table.
> Insert events on transactional tables need to call addWriteNotificationLog API
> --
>
> Key: IMPALA-9664
> URL: https://issues.apache.org/jira/browse/IMPALA-9664
> Project: IMPALA
> Issue Type: Bug
>Reporter: Vihang Karajgaonkar
>Assignee: Xiaomeng Zhang
>Priority: Major
>
> According to what we see in Hive source code, for transactional tables, the
> insert events are fired with a different API {{addWriteNotificationLog}}.
> Currently Impala fires {{firelistenerEvent}} for both transactional and
> non-transactional tables. We should look at what is the difference between
> the two APIs and see if we need to handle transactional tables differently.
> References:
> https://github.com/apache/hive/blob/c3afb57bdb1041f566fbbd896f625328fc9656a0/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2402
> https://github.com/apache/hive/blob/c3afb57bdb1041f566fbbd896f625328fc9656a0/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2236
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-9664) Insert events on transactional tables need to call addWriteNotificationLog API
[
https://issues.apache.org/jira/browse/IMPALA-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang reassigned IMPALA-9664:
--
Assignee: Xiaomeng Zhang
> Insert events on transactional tables need to call addWriteNotificationLog API
> --
>
> Key: IMPALA-9664
> URL: https://issues.apache.org/jira/browse/IMPALA-9664
> Project: IMPALA
> Issue Type: Bug
>Reporter: Vihang Karajgaonkar
>Assignee: Xiaomeng Zhang
>Priority: Major
>
> According to what we see in Hive source code, for transactional tables, the
> insert events are fired with a different API {{addWriteNotificationLog}}.
> Currently Impala fires {{firelistenerEvent}} for both transactional and
> non-transactional tables. We should look at what is the difference between
> the two APIs and see if we need to handle transactional tables differently.
> References:
> https://github.com/apache/hive/blob/c3afb57bdb1041f566fbbd896f625328fc9656a0/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2402
> https://github.com/apache/hive/blob/c3afb57bdb1041f566fbbd896f625328fc9656a0/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2236
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8632) Add support for self-event detection for insert events
[
https://issues.apache.org/jira/browse/IMPALA-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang resolved IMPALA-8632.
Fix Version/s: Impala 4.0
Resolution: Fixed
> Add support for self-event detection for insert events
> --
>
> Key: IMPALA-8632
> URL: https://issues.apache.org/jira/browse/IMPALA-8632
> Project: IMPALA
> Issue Type: Sub-task
>Reporter: Vihang Karajgaonkar
>Assignee: Xiaomeng Zhang
>Priority: Critical
> Fix For: Impala 4.0
>
>
> In case of {{INSERT_EVENTS}} if Impala inserts into a table it causes a
> refresh to the underlying table/partition. This could be unnecessary when
> there is only one Impala cluster in the system. The existing self-event
> detection framework cannot identify such events because they are not sending
> HMS objects like tables and partitions to the HMS. Instead in case of
> {{INSERT_EVENT}} HMS API only asks for a table name or partition value to
> fire a insert event on it.
> We can detect a self-event in such cases if the HMS API to fire a listener
> event is improved to return the event id. This would be used by
> EventProcessor to ignore the event when it is fetched later in the next
> polling cycle. In order to support this, we will need to make a change to
> Hive as well so that the enhanced API can be used.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-9483) Add logs for debugging builtin functions throw unknown exception randomly
[
https://issues.apache.org/jira/browse/IMPALA-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang resolved IMPALA-9483.
Fix Version/s: Impala 4.0
Resolution: Fixed
> Add logs for debugging builtin functions throw unknown exception randomly
> -
>
> Key: IMPALA-9483
> URL: https://issues.apache.org/jira/browse/IMPALA-9483
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.4.0
>Reporter: Xiaomeng Zhang
>Assignee: Xiaomeng Zhang
>Priority: Major
> Fix For: Impala 4.0
>
>
> This is a random issue happens in Impala 3.4.0 env:
> # has sentry installed
> # high concurrency on impalad
> # happens on view that calls buildin functions
> It looks that restart impalad solves this problem. We should add more info in
> exception string to help identify the issue when it happens again.
> Error message as below:
> {code:java}
> org.apache.impala.common.AnalysisException: trim() unknown
> at
> org.apache.impala.analysis.FunctionCallExpr.analyzeImpl(FunctionCallExpr.java:496)
> at org.apache.impala.analysis.Expr.analyze(Expr.java:357)
> at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:193)
> at org.apache.impala.analysis.InlineViewRef.analyze(InlineViewRef.java:166)
> at org.apache.impala.analysis.FromClause.analyze(FromClause.java:62)
> at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:169)
> at org.apache.impala.analysis.InlineViewRef.analyze(InlineViewRef.java:166)
> at org.apache.impala.analysis.FromClause.analyze(FromClause.java:62)
> at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:169)
> at
> org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:446)
> at
> org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:416)
> at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1244)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1212)
> at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1184)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:168)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-9075) Add support for reading zstd text files
[ https://issues.apache.org/jira/browse/IMPALA-9075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaomeng Zhang resolved IMPALA-9075. Fix Version/s: Impala 4.0 Resolution: Fixed > Add support for reading zstd text files > --- > > Key: IMPALA-9075 > URL: https://issues.apache.org/jira/browse/IMPALA-9075 > Project: IMPALA > Issue Type: Bug >Affects Versions: Impala 3.3.0 >Reporter: Andrew Sherman >Assignee: Xiaomeng Zhang >Priority: Critical > Fix For: Impala 4.0 > > > IMPALA-8450 added support for zstd in parquet. > We should also support support for reading zstd encoded text files. > Another useful jira to look at is IMPALA-8549 (Add support for scanning > DEFLATE text files) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-9451) custom_cluster.test_hive_text_codec_interop.TestTextInterop.test_hive_impala_interop fails on CDP build
[
https://issues.apache.org/jira/browse/IMPALA-9451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang resolved IMPALA-9451.
Resolution: Fixed
> custom_cluster.test_hive_text_codec_interop.TestTextInterop.test_hive_impala_interop
> fails on CDP build
> ---
>
> Key: IMPALA-9451
> URL: https://issues.apache.org/jira/browse/IMPALA-9451
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Reporter: Tim Armstrong
>Assignee: Xiaomeng Zhang
>Priority: Blocker
> Labels: broken-build
>
> {noformat}
> custom_cluster.test_hive_text_codec_interop.TestTextInterop.test_hive_impala_interop
> assert [] == ['0\ttrue\t0\t0\t0\t0\t0\t0\...t01/01/09\t5\t2009\t1', ...]
> Right contains more items, first extra item:
> '0\ttrue\t0\t0\t0\t0\t0\t0\t01/01/09\t0\t2009\t1' Full diff: - [] +
> ['0\ttrue\t0\t0\t0\t0\t0\t0\t01/01/09\t0\t2009\t1', +
> '1\tfalse\t1\t1\t1\t10\t1.10023841858\t10.1\t01/01/09\t1\t2009\t1', +
> '2\ttrue\t2\t2\t2\t20\t2.20047683716\t20.2\t01/01/09\t2\t2009\t1', +
> '3\tfalse\t3\t3\t3\t30\t3.29952316284\t30.3\t01/01/09\t3\t2009\t1', +
> '4\ttrue\t4\t4\t4\t40\t4.40095367432\t40.4\t01/01/09\t4\t2009\t1', +
> '5\tfalse\t5\t5\t5\t50\t5.5\t50.5\t01/01/09\t5\t2009\t1', +
> '6\ttrue\t6\t6\t6\t60\t6.59904632568\t60.59\t01/01/09\t6\t2009\t1',
>+ '7\tfalse\t7\t7\t7\t70\t7.69809265137\t70.7\t01/01/09\t7\t2009\t1',
> + '8\ttrue\t8\t8\t8\t80\t8.80190734863\t80.8\t01/01/09\t8\t2009\t1',
> +
> '9\tfalse\t9\t9\t9\t90\t9.89618530273\t90.89\t01/01/09\t9\t2009\t1',
>+ '10\ttrue\t0\t0\t0\t0\t0\t0\t01/02/09\t0\t2009\t1', +
> '11\tfalse\t1\t1\t1\t10\t1.10023841858\t10.1\t01/02/09\t1\t2009\t1', +
> '12\ttrue\t2\t2\t2\t20\t2.20047683716\t20.2\t01/02/09\t2\t2009\t1', +
> '13\tfalse\t3\t3\t3\t30\t3.29952316284\t30.3\t01/02/09\t3\t2009\t1', +
> '14\ttrue\t4\t4\t4\t40\t4.40095367432\t40.4\t01/02/09\t4\t2009\t1', +
> '15\tfalse\t5\t5\t5\t50\t5.5\t50.5\t01/02/09\t5\t2009\t1', +
> '16\ttrue\t6\t6\t6\t60\t6.59904632568\t60.59\t01/02/09\t6\t2009\t1',
>+ '17\tfalse\t7\t7\t7\t70\t7.69809265137\t70.7\t01/02/09\t7\t2009\t1',
> ...
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-9446) query_test.test_compressed_formats.TestReadZtsdLibCompressedFile fails on S3
[
https://issues.apache.org/jira/browse/IMPALA-9446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang resolved IMPALA-9446.
Resolution: Fixed
> query_test.test_compressed_formats.TestReadZtsdLibCompressedFile fails on S3
>
>
> Key: IMPALA-9446
> URL: https://issues.apache.org/jira/browse/IMPALA-9446
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.4.0
>Reporter: Joe McDonnell
>Assignee: Xiaomeng Zhang
>Priority: Blocker
> Labels: broken-build
>
> On the S3 configuration,
> query_test.test_compressed_formats.TestReadZtsdLibCompressedFile is
> encountering the following error:
>
> {noformat}
> query_test/test_compressed_formats.py:325: in test_query_large_file
> result = self.client.execute("select count(*) from %s" %
> self.COMPRESSED_TABLE_NAME)
> common/impala_connection.py:205: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:187: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:365: in __execute_query
> self.wait_for_finished(handle)
> beeswax/impala_beeswax.py:386: in wait_for_finished
> raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> EQuery aborted:Rejected query from pool default-pool: Error during
> scheduling: Invalid file descriptor compression code: 10{noformat}
> The error comes from here:
>
> [https://github.com/apache/impala/blob/master/be/src/util/flat_buffer.cc#L63]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-9451) custom_cluster.test_hive_text_codec_interop.TestTextInterop.test_hive_impala_interop fails on CDP build
[
https://issues.apache.org/jira/browse/IMPALA-9451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17064387#comment-17064387
]
Xiaomeng Zhang commented on IMPALA-9451:
Thanks [~boroknagyz], I have a code review out for the fix
[https://gerrit.cloudera.org/#/c/15520/]
> custom_cluster.test_hive_text_codec_interop.TestTextInterop.test_hive_impala_interop
> fails on CDP build
> ---
>
> Key: IMPALA-9451
> URL: https://issues.apache.org/jira/browse/IMPALA-9451
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Reporter: Tim Armstrong
>Assignee: Xiaomeng Zhang
>Priority: Blocker
> Labels: broken-build
>
> {noformat}
> custom_cluster.test_hive_text_codec_interop.TestTextInterop.test_hive_impala_interop
> assert [] == ['0\ttrue\t0\t0\t0\t0\t0\t0\...t01/01/09\t5\t2009\t1', ...]
> Right contains more items, first extra item:
> '0\ttrue\t0\t0\t0\t0\t0\t0\t01/01/09\t0\t2009\t1' Full diff: - [] +
> ['0\ttrue\t0\t0\t0\t0\t0\t0\t01/01/09\t0\t2009\t1', +
> '1\tfalse\t1\t1\t1\t10\t1.10023841858\t10.1\t01/01/09\t1\t2009\t1', +
> '2\ttrue\t2\t2\t2\t20\t2.20047683716\t20.2\t01/01/09\t2\t2009\t1', +
> '3\tfalse\t3\t3\t3\t30\t3.29952316284\t30.3\t01/01/09\t3\t2009\t1', +
> '4\ttrue\t4\t4\t4\t40\t4.40095367432\t40.4\t01/01/09\t4\t2009\t1', +
> '5\tfalse\t5\t5\t5\t50\t5.5\t50.5\t01/01/09\t5\t2009\t1', +
> '6\ttrue\t6\t6\t6\t60\t6.59904632568\t60.59\t01/01/09\t6\t2009\t1',
>+ '7\tfalse\t7\t7\t7\t70\t7.69809265137\t70.7\t01/01/09\t7\t2009\t1',
> + '8\ttrue\t8\t8\t8\t80\t8.80190734863\t80.8\t01/01/09\t8\t2009\t1',
> +
> '9\tfalse\t9\t9\t9\t90\t9.89618530273\t90.89\t01/01/09\t9\t2009\t1',
>+ '10\ttrue\t0\t0\t0\t0\t0\t0\t01/02/09\t0\t2009\t1', +
> '11\tfalse\t1\t1\t1\t10\t1.10023841858\t10.1\t01/02/09\t1\t2009\t1', +
> '12\ttrue\t2\t2\t2\t20\t2.20047683716\t20.2\t01/02/09\t2\t2009\t1', +
> '13\tfalse\t3\t3\t3\t30\t3.29952316284\t30.3\t01/02/09\t3\t2009\t1', +
> '14\ttrue\t4\t4\t4\t40\t4.40095367432\t40.4\t01/02/09\t4\t2009\t1', +
> '15\tfalse\t5\t5\t5\t50\t5.5\t50.5\t01/02/09\t5\t2009\t1', +
> '16\ttrue\t6\t6\t6\t60\t6.59904632568\t60.59\t01/02/09\t6\t2009\t1',
>+ '17\tfalse\t7\t7\t7\t70\t7.69809265137\t70.7\t01/02/09\t7\t2009\t1',
> ...
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-9532) Functions can disappear when a concurrent invalidate metadata is running
[
https://issues.apache.org/jira/browse/IMPALA-9532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17063554#comment-17063554
]
Xiaomeng Zhang commented on IMPALA-9532:
Not sure if same reason as this one
https://issues.apache.org/jira/browse/IMPALA-9483. This is buildin function
missing randomly. But buildins database can't be altered by user.
> Functions can disappear when a concurrent invalidate metadata is running
>
>
> Key: IMPALA-9532
> URL: https://issues.apache.org/jira/browse/IMPALA-9532
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Reporter: Vihang Karajgaonkar
>Priority: Major
> Labels: concurrency
>
> The global invalidate metadata takes a write lock on the {{versionLock_}}.
> However, the locking protocol for ddls release the {{versionLock_}} as soon
> as the table level lock is acquired. This allows for a concurrent
> {{invalidate metadata}} to run while the DDL operation is in progress. This
> can lead to weird race conditions. One such example is below can lead to
> functions disappearing from the catalog until a invalidate metadata is issued
> again.
> Following sequence of events can reproduce this race condition:
> {noformat}
> [localhost:21000] default> create function default.f() returns int location
> '/test-warehouse/libTestUdfs.so' symbol='NoArgs';
> Query: create function default.f() returns int location
> '/test-warehouse/libTestUdfs.so' symbol='NoArgs'
> ++
> | summary|
> ++
> | Function has been created. |
> ++
> Fetched 1 row(s) in 10.26s
> --> Session 2 invokes invalidate metadata concurrently
> [localhost:21001] default> invalidate metadata; Query: invalidate metadata
> Query submitted at: 2020-03-18 15:04:25 (Coordinator:
> http://vihang-Precision-21575:25001) Query progress can be monitored at:
> http:///query_plan?query_id=d3463484ff635684:620fbfef
> Fetched 0 row(s) in 4.30s
> --> drop function from session1 says function does not exist but show
> functions shows it.
> [localhost:21000] default> drop function f();
> Query: drop function f()
> ERROR: CatalogException: Function: f() does not exist.
> [localhost:21000] default> show functions;
> Query: show functions
> +-+---+-+---+
> | return type | signature | binary type | is persistent |
> +-+---+-+---+
> | INT | f() | NATIVE | true |
> +-+---+-+---+
> Fetched 1 row(s) in 0.01s
> [localhost:21000] default>
> -- Session 2 never sees the function f:
> [localhost:21001] default> show functions;
> Query: show functions
> Fetched 0 row(s) in 0.00s
> {noformat}
> When the create function statement is executing in {{CatalogOpExecutor}} we
> apply the alterDatabase in HMS to persist the new db parameters here:
> [https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java#L1409]
> Note the we have released the {{versionLock_}} by line 1409. Meanwhile a
> concurrent {{invalidate metadata}} fetches the db params from HMS here
> [https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java#L1326]
> which will override the parameters of the newly created Db object. Hence
> effectively we are removing the function from the parameters since the
> operation 1 to alterDatabase is not yet committed in HMS.
> All subsequent commands of {{show functions}}, {{drop function}} will show
> inconsistent results. I was able to reproduce this race condition by added a
> sleep statement just before the alterDatabase call in the createFunction
> method.
> Note: Above code links are based of commit hash
> {{7dd13f72784514a59f82c9a7a5e2250503dbfaf0}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-9483) Add logs for debugging builtin functions throw unknown exception randomly
[
https://issues.apache.org/jira/browse/IMPALA-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang updated IMPALA-9483:
---
Description:
This is a random issue happens in Impala 3.4.0 env:
# has sentry installed
# high concurrency on impalad
# happens on view that calls buildin functions
It looks that restart impalad solves this problem. We should add more info in
exception string to help identify the issue when it happens again.
Error message as below:
{code:java}
org.apache.impala.common.AnalysisException: trim() unknown
at
org.apache.impala.analysis.FunctionCallExpr.analyzeImpl(FunctionCallExpr.java:496)
at org.apache.impala.analysis.Expr.analyze(Expr.java:357)
at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:193)
at org.apache.impala.analysis.InlineViewRef.analyze(InlineViewRef.java:166)
at org.apache.impala.analysis.FromClause.analyze(FromClause.java:62)
at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:169)
at org.apache.impala.analysis.InlineViewRef.analyze(InlineViewRef.java:166)
at org.apache.impala.analysis.FromClause.analyze(FromClause.java:62)
at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:169)
at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:446)
at
org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:416)
at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1244)
at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1212)
at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1184)
at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:168)
{code}
was:
This is a random issue happens in Impala 3.4.0 env:
# has sentry installed
# high concurrency on impalad
# happens on view that calls buildin functions
It looks that restart impalad solves this problem. We should add more info in
exception string to help identify the issue when it happens again.
> Add logs for debugging builtin functions throw unknown exception randomly
> -
>
> Key: IMPALA-9483
> URL: https://issues.apache.org/jira/browse/IMPALA-9483
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.4.0
>Reporter: Xiaomeng Zhang
>Assignee: Xiaomeng Zhang
>Priority: Major
>
> This is a random issue happens in Impala 3.4.0 env:
> # has sentry installed
> # high concurrency on impalad
> # happens on view that calls buildin functions
> It looks that restart impalad solves this problem. We should add more info in
> exception string to help identify the issue when it happens again.
> Error message as below:
> {code:java}
> org.apache.impala.common.AnalysisException: trim() unknown
> at
> org.apache.impala.analysis.FunctionCallExpr.analyzeImpl(FunctionCallExpr.java:496)
> at org.apache.impala.analysis.Expr.analyze(Expr.java:357)
> at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:193)
> at org.apache.impala.analysis.InlineViewRef.analyze(InlineViewRef.java:166)
> at org.apache.impala.analysis.FromClause.analyze(FromClause.java:62)
> at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:169)
> at org.apache.impala.analysis.InlineViewRef.analyze(InlineViewRef.java:166)
> at org.apache.impala.analysis.FromClause.analyze(FromClause.java:62)
> at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:169)
> at
> org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:446)
> at
> org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:416)
> at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1244)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1212)
> at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1184)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:168)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-9483) Add logs for debugging builtin functions throw unknown exception randomly
Xiaomeng Zhang created IMPALA-9483: -- Summary: Add logs for debugging builtin functions throw unknown exception randomly Key: IMPALA-9483 URL: https://issues.apache.org/jira/browse/IMPALA-9483 Project: IMPALA Issue Type: Bug Components: Frontend Affects Versions: Impala 3.4.0 Reporter: Xiaomeng Zhang Assignee: Xiaomeng Zhang This is a random issue happens in Impala 3.4.0 env: # has sentry installed # high concurrency on impalad # happens on view that calls buildin functions It looks that restart impalad solves this problem. We should add more info in exception string to help identify the issue when it happens again. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-9389) Impala Doc: support reading zstd text files
[ https://issues.apache.org/jira/browse/IMPALA-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045766#comment-17045766 ] Xiaomeng Zhang commented on IMPALA-9389: I am not sure for parquet zstd, as that was added by [~arawat] in https://issues.apache.org/jira/browse/IMPALA-8450 last year. This one I newly added is for reading text zstd. I think this file should be updated [https://impala.apache.org/docs/build/html/topics/impala_txtfile.html] > Impala Doc: support reading zstd text files > --- > > Key: IMPALA-9389 > URL: https://issues.apache.org/jira/browse/IMPALA-9389 > Project: IMPALA > Issue Type: Documentation > Components: Backend >Affects Versions: Impala 3.3.0 >Reporter: Xiaomeng Zhang >Assignee: Kris Hahn >Priority: Major > > [https://gerrit.cloudera.org/#/c/15023/] > We add support for reading zstd encoded text files. > This includes: > # support reading zstd file written by Hive which uses streaming. > # 2. support reading zstd file compressed by standard zstd library which > uses block. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-9389) Impala Doc: support reading zstd text files
Xiaomeng Zhang created IMPALA-9389: -- Summary: Impala Doc: support reading zstd text files Key: IMPALA-9389 URL: https://issues.apache.org/jira/browse/IMPALA-9389 Project: IMPALA Issue Type: Documentation Components: Backend Affects Versions: Impala 3.3.0 Reporter: Xiaomeng Zhang Assignee: Xiaomeng Zhang [https://gerrit.cloudera.org/#/c/15023/] We add support for reading zstd encoded text files. This includes: # support reading zstd file written by Hive which uses streaming. # 2. support reading zstd file compressed by standard zstd library which uses block. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-9325) Describe statement fails if its a Hive Materialized view
[ https://issues.apache.org/jira/browse/IMPALA-9325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaomeng Zhang reassigned IMPALA-9325: -- Assignee: Xiaomeng Zhang (was: Vihang Karajgaonkar) > Describe statement fails if its a Hive Materialized view > - > > Key: IMPALA-9325 > URL: https://issues.apache.org/jira/browse/IMPALA-9325 > Project: IMPALA > Issue Type: Bug >Affects Versions: Impala 3.4.0 >Reporter: Vihang Karajgaonkar >Assignee: Xiaomeng Zhang >Priority: Minor > > Based on what I see here > [https://github.com/apache/impala/blob/master/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java#L349] > Impala treats Hive's materialized views as views. > > However, when we issue describe statement on a materialized in catalog-v2 > mode (--use-local-catalog=true) we see the following exception: > > Error: LocalCatalogException: Could not load partition names for table > default.mv3 CAUSED BY: TException: Invalid response from catalogd for request > TGetPartialCatalogObjectRequest(protocol_version:V1, > object_desc:TCatalogObject(type:TABLE, catalog_version:16, > table:TTable(db_name:default, tbl_name:mv3)), > table_info_selector:TTableInfoSelector(want_hms_table:false, > want_partition_names:true, want_partition_metadata:false, > want_partition_files:false, want_partition_stats:false, > want_table_constraints:false)): missing partition list result > > We should either have a more user-friendly error message are treat it as a > View until we add a full support to materialized views. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-9321) TestParquetMaxPageHeader.test_large_page_header_config failed with error "Invalid configuration of tez jars"
[
https://issues.apache.org/jira/browse/IMPALA-9321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang updated IMPALA-9321:
---
Fix Version/s: Impala 3.4.0
> TestParquetMaxPageHeader.test_large_page_header_config failed with error
> "Invalid configuration of tez jars"
>
>
> Key: IMPALA-9321
> URL: https://issues.apache.org/jira/browse/IMPALA-9321
> Project: IMPALA
> Issue Type: Test
> Components: Infrastructure
>Affects Versions: Impala 3.4.0
>Reporter: Xiaomeng Zhang
>Assignee: Xiaomeng Zhang
>Priority: Major
> Labels: broken-build
> Fix For: Impala 3.4.0
>
>
> This test has been failed in last 10 builds of impala-cdpd-master-exhuastive
> job.
> {code:java}
> E INFO : Compiling
> command(queryId=jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a):
> INSERT OVERWRITE TABLE large_page_header SELECT col FROM
> parquet_test_data_text
> E INFO : Concurrency mode is disabled, not creating a lock manager
> E INFO : No Stats for default@parquet_test_data_text, Columns: col
> E INFO : Semantic Analysis Completed (retrial = false)
> E INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:col,
> type:string, comment:null)], properties:null)
> E INFO : Completed compiling
> command(queryId=jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a);
> Time taken: 0.671 seconds
> E INFO : Concurrency mode is disabled, not creating a lock manager
> E INFO : Executing
> command(queryId=jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a):
> INSERT OVERWRITE TABLE large_page_header SELECT col FROM
> parquet_test_data_text
> E INFO : Query ID =
> jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a
> E INFO : Total jobs = 1
> E INFO : Launching Job 1 out of 1
> E INFO : Starting task [Stage-1:MAPRED] in serial mode
> E INFO : Subscribed to counters: [] for queryId:
> jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a
> E INFO : Tez session hasn't been created yet. Opening session
> E ERROR : Failed to execute tez graph.
> E org.apache.tez.dag.api.TezUncheckedException: Invalid configuration of
> tez jars, tez.lib.uris is not defined in the configuration
> E at
> org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:179)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at
> org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:1163)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at
> org.apache.tez.client.TezClient.setupApplicationContext(TezClient.java:477)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at org.apache.tez.client.TezClient.start(TezClient.java:403)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:522)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:373)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:298)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolSession.open(TezSessionPoolSession.java:106)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezTask.ensureSessionHasResources(TezTask.java:380)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:203)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:212)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:103)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2302)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1954)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1627)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1387)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1381)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
[jira] [Resolved] (IMPALA-9321) TestParquetMaxPageHeader.test_large_page_header_config failed with error "Invalid configuration of tez jars"
[
https://issues.apache.org/jira/browse/IMPALA-9321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang resolved IMPALA-9321.
Resolution: Fixed
> TestParquetMaxPageHeader.test_large_page_header_config failed with error
> "Invalid configuration of tez jars"
>
>
> Key: IMPALA-9321
> URL: https://issues.apache.org/jira/browse/IMPALA-9321
> Project: IMPALA
> Issue Type: Test
> Components: Infrastructure
>Affects Versions: Impala 3.4.0
>Reporter: Xiaomeng Zhang
>Assignee: Xiaomeng Zhang
>Priority: Major
> Labels: broken-build
>
> This test has been failed in last 10 builds of impala-cdpd-master-exhuastive
> job.
> {code:java}
> E INFO : Compiling
> command(queryId=jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a):
> INSERT OVERWRITE TABLE large_page_header SELECT col FROM
> parquet_test_data_text
> E INFO : Concurrency mode is disabled, not creating a lock manager
> E INFO : No Stats for default@parquet_test_data_text, Columns: col
> E INFO : Semantic Analysis Completed (retrial = false)
> E INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:col,
> type:string, comment:null)], properties:null)
> E INFO : Completed compiling
> command(queryId=jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a);
> Time taken: 0.671 seconds
> E INFO : Concurrency mode is disabled, not creating a lock manager
> E INFO : Executing
> command(queryId=jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a):
> INSERT OVERWRITE TABLE large_page_header SELECT col FROM
> parquet_test_data_text
> E INFO : Query ID =
> jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a
> E INFO : Total jobs = 1
> E INFO : Launching Job 1 out of 1
> E INFO : Starting task [Stage-1:MAPRED] in serial mode
> E INFO : Subscribed to counters: [] for queryId:
> jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a
> E INFO : Tez session hasn't been created yet. Opening session
> E ERROR : Failed to execute tez graph.
> E org.apache.tez.dag.api.TezUncheckedException: Invalid configuration of
> tez jars, tez.lib.uris is not defined in the configuration
> E at
> org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:179)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at
> org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:1163)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at
> org.apache.tez.client.TezClient.setupApplicationContext(TezClient.java:477)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at org.apache.tez.client.TezClient.start(TezClient.java:403)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:522)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:373)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:298)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolSession.open(TezSessionPoolSession.java:106)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezTask.ensureSessionHasResources(TezTask.java:380)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:203)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:212)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:103)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2302)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1954)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1627)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1387)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1381)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
>
[jira] [Resolved] (IMPALA-9322) authorization.test_grant_revoke.TestGrantRevoke.test_grant_revoke failed with "Unable to start Sentry"
[
https://issues.apache.org/jira/browse/IMPALA-9322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang resolved IMPALA-9322.
Fix Version/s: Impala 3.4.0
Resolution: Fixed
> authorization.test_grant_revoke.TestGrantRevoke.test_grant_revoke failed with
> "Unable to start Sentry"
> --
>
> Key: IMPALA-9322
> URL: https://issues.apache.org/jira/browse/IMPALA-9322
> Project: IMPALA
> Issue Type: Test
> Components: Infrastructure
>Affects Versions: Impala 3.4.0
>Reporter: Xiaomeng Zhang
>Assignee: Xiaomeng Zhang
>Priority: Major
> Labels: broken-build
> Fix For: Impala 3.4.0
>
>
> In last few builds of impala-cdpd-master-exhuastive tests, authorization test
> failed with error "Unable to start Sentry"
> {code:java}
> Error Messagetest setup
> failureStacktraceauthorization/test_grant_revoke.py:51: in setup_method
> super(TestGrantRevoke, self).setup_method(method)
> common/custom_cluster_test_suite.py:164: in setup_method
> method.func_dict.get(SENTRY_LOG_DIR))
> common/custom_cluster_test_suite.py:231: in _start_sentry_service
> raise RuntimeError("Unable to start Sentry")
> E RuntimeError: Unable to start SentryStandard OutputStopping Sentry
> ERROR in
> /data/jenkins/workspace/impala-cdpd-master-exhaustive/repos/Impala/testdata/bin/run-sentry-service.sh
> at line 54: "$JAVA" -cp $CLASSPATH
> org.apache.impala.testutil.SentryServicePinger \
> Generated:
> /data/jenkins/workspace/impala-cdpd-master-exhaustive/repos/Impala/logs/extra_junit_xml_logs/generate_junitxml.buildall.run-sentry-service.20200122_02_11_24.xml
> {code}
> {code:java}
> org.apache.impala.common.InternalException: Error making 'listRoles' RPC to
> Sentry Service:
> at
> org.apache.impala.authorization.sentry.SentryPolicyService.listAllRoles(SentryPolicyService.java:422)
> at
> org.apache.impala.testutil.SentryServicePinger.main(SentryServicePinger.java:83)
> Caused by: org.apache.sentry.core.common.exception.SentryUserException:
> java.net.ConnectException: Connection refused (Connection refused)
> at
> org.apache.sentry.core.common.transport.RetryClientInvocationHandler.connect(RetryClientInvocationHandler.java:166)
> at
> org.apache.sentry.core.common.transport.RetryClientInvocationHandler.invokeImpl(RetryClientInvocationHandler.java:90)
> at
> org.apache.sentry.core.common.transport.SentryClientInvocationHandler.invoke(SentryClientInvocationHandler.java:41)
> at com.sun.proxy.$Proxy5.listAllRoles(Unknown Source)
> at
> org.apache.impala.authorization.sentry.SentryPolicyService.listAllRoles(SentryPolicyService.java:416)
> ... 1 more
> Caused by: org.apache.thrift.transport.TTransportException:
> java.net.ConnectException: Connection refused (Connection refused)
> at org.apache.thrift.transport.TSocket.open(TSocket.java:226)
> at
> org.apache.sentry.core.common.transport.SentryTransportFactory.connectToServer(SentryTransportFactory.java:99)
> at
> org.apache.sentry.core.common.transport.SentryTransportFactory.getTransport(SentryTransportFactory.java:86)
> at
> org.apache.sentry.core.common.transport.SentryTransportPool$PoolFactory.create(SentryTransportPool.java:302)
> at
> org.apache.sentry.core.common.transport.SentryTransportPool$PoolFactory.create(SentryTransportPool.java:271)
> at
> org.apache.commons.pool2.BaseKeyedPooledObjectFactory.makeObject(BaseKeyedPooledObjectFactory.java:62)
> at
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.create(GenericKeyedObjectPool.java:1041)
> at
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:380)
> at
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:279)
> at
> org.apache.sentry.core.common.transport.SentryTransportPool.getTransport(SentryTransportPool.java:183)
> at
> org.apache.sentry.api.service.thrift.SentryPolicyServiceClientDefaultImpl.connect(SentryPolicyServiceClientDefaultImpl.java:100)
> at
> org.apache.sentry.core.common.transport.RetryClientInvocationHandler.connect(RetryClientInvocationHandler.java:141)
> ... 5 more
> Caused by: java.net.ConnectException: Connection refused (Connection refused)
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at
[jira] [Assigned] (IMPALA-9322) authorization.test_grant_revoke.TestGrantRevoke.test_grant_revoke failed with "Unable to start Sentry"
[
https://issues.apache.org/jira/browse/IMPALA-9322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang reassigned IMPALA-9322:
--
Assignee: Xiaomeng Zhang
> authorization.test_grant_revoke.TestGrantRevoke.test_grant_revoke failed with
> "Unable to start Sentry"
> --
>
> Key: IMPALA-9322
> URL: https://issues.apache.org/jira/browse/IMPALA-9322
> Project: IMPALA
> Issue Type: Test
> Components: Infrastructure
>Affects Versions: Impala 3.4.0
>Reporter: Xiaomeng Zhang
>Assignee: Xiaomeng Zhang
>Priority: Major
> Labels: broken-build
>
> In last few builds of impala-cdpd-master-exhuastive tests, authorization test
> failed with error "Unable to start Sentry"
> {code:java}
> Error Messagetest setup
> failureStacktraceauthorization/test_grant_revoke.py:51: in setup_method
> super(TestGrantRevoke, self).setup_method(method)
> common/custom_cluster_test_suite.py:164: in setup_method
> method.func_dict.get(SENTRY_LOG_DIR))
> common/custom_cluster_test_suite.py:231: in _start_sentry_service
> raise RuntimeError("Unable to start Sentry")
> E RuntimeError: Unable to start SentryStandard OutputStopping Sentry
> ERROR in
> /data/jenkins/workspace/impala-cdpd-master-exhaustive/repos/Impala/testdata/bin/run-sentry-service.sh
> at line 54: "$JAVA" -cp $CLASSPATH
> org.apache.impala.testutil.SentryServicePinger \
> Generated:
> /data/jenkins/workspace/impala-cdpd-master-exhaustive/repos/Impala/logs/extra_junit_xml_logs/generate_junitxml.buildall.run-sentry-service.20200122_02_11_24.xml
> {code}
> {code:java}
> org.apache.impala.common.InternalException: Error making 'listRoles' RPC to
> Sentry Service:
> at
> org.apache.impala.authorization.sentry.SentryPolicyService.listAllRoles(SentryPolicyService.java:422)
> at
> org.apache.impala.testutil.SentryServicePinger.main(SentryServicePinger.java:83)
> Caused by: org.apache.sentry.core.common.exception.SentryUserException:
> java.net.ConnectException: Connection refused (Connection refused)
> at
> org.apache.sentry.core.common.transport.RetryClientInvocationHandler.connect(RetryClientInvocationHandler.java:166)
> at
> org.apache.sentry.core.common.transport.RetryClientInvocationHandler.invokeImpl(RetryClientInvocationHandler.java:90)
> at
> org.apache.sentry.core.common.transport.SentryClientInvocationHandler.invoke(SentryClientInvocationHandler.java:41)
> at com.sun.proxy.$Proxy5.listAllRoles(Unknown Source)
> at
> org.apache.impala.authorization.sentry.SentryPolicyService.listAllRoles(SentryPolicyService.java:416)
> ... 1 more
> Caused by: org.apache.thrift.transport.TTransportException:
> java.net.ConnectException: Connection refused (Connection refused)
> at org.apache.thrift.transport.TSocket.open(TSocket.java:226)
> at
> org.apache.sentry.core.common.transport.SentryTransportFactory.connectToServer(SentryTransportFactory.java:99)
> at
> org.apache.sentry.core.common.transport.SentryTransportFactory.getTransport(SentryTransportFactory.java:86)
> at
> org.apache.sentry.core.common.transport.SentryTransportPool$PoolFactory.create(SentryTransportPool.java:302)
> at
> org.apache.sentry.core.common.transport.SentryTransportPool$PoolFactory.create(SentryTransportPool.java:271)
> at
> org.apache.commons.pool2.BaseKeyedPooledObjectFactory.makeObject(BaseKeyedPooledObjectFactory.java:62)
> at
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.create(GenericKeyedObjectPool.java:1041)
> at
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:380)
> at
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:279)
> at
> org.apache.sentry.core.common.transport.SentryTransportPool.getTransport(SentryTransportPool.java:183)
> at
> org.apache.sentry.api.service.thrift.SentryPolicyServiceClientDefaultImpl.connect(SentryPolicyServiceClientDefaultImpl.java:100)
> at
> org.apache.sentry.core.common.transport.RetryClientInvocationHandler.connect(RetryClientInvocationHandler.java:141)
> ... 5 more
> Caused by: java.net.ConnectException: Connection refused (Connection refused)
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:589)
> at
[jira] [Assigned] (IMPALA-9321) TestParquetMaxPageHeader.test_large_page_header_config failed with error "Invalid configuration of tez jars"
[
https://issues.apache.org/jira/browse/IMPALA-9321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang reassigned IMPALA-9321:
--
Assignee: Xiaomeng Zhang
> TestParquetMaxPageHeader.test_large_page_header_config failed with error
> "Invalid configuration of tez jars"
>
>
> Key: IMPALA-9321
> URL: https://issues.apache.org/jira/browse/IMPALA-9321
> Project: IMPALA
> Issue Type: Test
> Components: Infrastructure
>Affects Versions: Impala 3.4.0
>Reporter: Xiaomeng Zhang
>Assignee: Xiaomeng Zhang
>Priority: Major
> Labels: broken-build
>
> This test has been failed in last 10 builds of impala-cdpd-master-exhuastive
> job.
> {code:java}
> E INFO : Compiling
> command(queryId=jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a):
> INSERT OVERWRITE TABLE large_page_header SELECT col FROM
> parquet_test_data_text
> E INFO : Concurrency mode is disabled, not creating a lock manager
> E INFO : No Stats for default@parquet_test_data_text, Columns: col
> E INFO : Semantic Analysis Completed (retrial = false)
> E INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:col,
> type:string, comment:null)], properties:null)
> E INFO : Completed compiling
> command(queryId=jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a);
> Time taken: 0.671 seconds
> E INFO : Concurrency mode is disabled, not creating a lock manager
> E INFO : Executing
> command(queryId=jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a):
> INSERT OVERWRITE TABLE large_page_header SELECT col FROM
> parquet_test_data_text
> E INFO : Query ID =
> jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a
> E INFO : Total jobs = 1
> E INFO : Launching Job 1 out of 1
> E INFO : Starting task [Stage-1:MAPRED] in serial mode
> E INFO : Subscribed to counters: [] for queryId:
> jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a
> E INFO : Tez session hasn't been created yet. Opening session
> E ERROR : Failed to execute tez graph.
> E org.apache.tez.dag.api.TezUncheckedException: Invalid configuration of
> tez jars, tez.lib.uris is not defined in the configuration
> E at
> org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:179)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at
> org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:1163)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at
> org.apache.tez.client.TezClient.setupApplicationContext(TezClient.java:477)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at org.apache.tez.client.TezClient.start(TezClient.java:403)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:522)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:373)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:298)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolSession.open(TezSessionPoolSession.java:106)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezTask.ensureSessionHasResources(TezTask.java:380)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:203)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:212)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:103)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2302)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1954)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1627)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1387)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1381)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
>
[jira] [Commented] (IMPALA-9322) authorization.test_grant_revoke.TestGrantRevoke.test_grant_revoke failed with "Unable to start Sentry"
[
https://issues.apache.org/jira/browse/IMPALA-9322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022507#comment-17022507
]
Xiaomeng Zhang commented on IMPALA-9322:
This may due to https://issues.apache.org/jira/browse/IMPALA-9287, let's see if
the failure would be fixed after disable that test.
> authorization.test_grant_revoke.TestGrantRevoke.test_grant_revoke failed with
> "Unable to start Sentry"
> --
>
> Key: IMPALA-9322
> URL: https://issues.apache.org/jira/browse/IMPALA-9322
> Project: IMPALA
> Issue Type: Test
> Components: Infrastructure
>Affects Versions: Impala 3.4.0
>Reporter: Xiaomeng Zhang
>Priority: Major
> Labels: broken-build
>
> In last few builds of impala-cdpd-master-exhuastive tests, authorization test
> failed with error "Unable to start Sentry"
> {code:java}
> Error Messagetest setup
> failureStacktraceauthorization/test_grant_revoke.py:51: in setup_method
> super(TestGrantRevoke, self).setup_method(method)
> common/custom_cluster_test_suite.py:164: in setup_method
> method.func_dict.get(SENTRY_LOG_DIR))
> common/custom_cluster_test_suite.py:231: in _start_sentry_service
> raise RuntimeError("Unable to start Sentry")
> E RuntimeError: Unable to start SentryStandard OutputStopping Sentry
> ERROR in
> /data/jenkins/workspace/impala-cdpd-master-exhaustive/repos/Impala/testdata/bin/run-sentry-service.sh
> at line 54: "$JAVA" -cp $CLASSPATH
> org.apache.impala.testutil.SentryServicePinger \
> Generated:
> /data/jenkins/workspace/impala-cdpd-master-exhaustive/repos/Impala/logs/extra_junit_xml_logs/generate_junitxml.buildall.run-sentry-service.20200122_02_11_24.xml
> {code}
> {code:java}
> org.apache.impala.common.InternalException: Error making 'listRoles' RPC to
> Sentry Service:
> at
> org.apache.impala.authorization.sentry.SentryPolicyService.listAllRoles(SentryPolicyService.java:422)
> at
> org.apache.impala.testutil.SentryServicePinger.main(SentryServicePinger.java:83)
> Caused by: org.apache.sentry.core.common.exception.SentryUserException:
> java.net.ConnectException: Connection refused (Connection refused)
> at
> org.apache.sentry.core.common.transport.RetryClientInvocationHandler.connect(RetryClientInvocationHandler.java:166)
> at
> org.apache.sentry.core.common.transport.RetryClientInvocationHandler.invokeImpl(RetryClientInvocationHandler.java:90)
> at
> org.apache.sentry.core.common.transport.SentryClientInvocationHandler.invoke(SentryClientInvocationHandler.java:41)
> at com.sun.proxy.$Proxy5.listAllRoles(Unknown Source)
> at
> org.apache.impala.authorization.sentry.SentryPolicyService.listAllRoles(SentryPolicyService.java:416)
> ... 1 more
> Caused by: org.apache.thrift.transport.TTransportException:
> java.net.ConnectException: Connection refused (Connection refused)
> at org.apache.thrift.transport.TSocket.open(TSocket.java:226)
> at
> org.apache.sentry.core.common.transport.SentryTransportFactory.connectToServer(SentryTransportFactory.java:99)
> at
> org.apache.sentry.core.common.transport.SentryTransportFactory.getTransport(SentryTransportFactory.java:86)
> at
> org.apache.sentry.core.common.transport.SentryTransportPool$PoolFactory.create(SentryTransportPool.java:302)
> at
> org.apache.sentry.core.common.transport.SentryTransportPool$PoolFactory.create(SentryTransportPool.java:271)
> at
> org.apache.commons.pool2.BaseKeyedPooledObjectFactory.makeObject(BaseKeyedPooledObjectFactory.java:62)
> at
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.create(GenericKeyedObjectPool.java:1041)
> at
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:380)
> at
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:279)
> at
> org.apache.sentry.core.common.transport.SentryTransportPool.getTransport(SentryTransportPool.java:183)
> at
> org.apache.sentry.api.service.thrift.SentryPolicyServiceClientDefaultImpl.connect(SentryPolicyServiceClientDefaultImpl.java:100)
> at
> org.apache.sentry.core.common.transport.RetryClientInvocationHandler.connect(RetryClientInvocationHandler.java:141)
> ... 5 more
> Caused by: java.net.ConnectException: Connection refused (Connection refused)
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> at
[jira] [Commented] (IMPALA-9321) TestParquetMaxPageHeader.test_large_page_header_config failed with error "Invalid configuration of tez jars"
[
https://issues.apache.org/jira/browse/IMPALA-9321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022506#comment-17022506
]
Xiaomeng Zhang commented on IMPALA-9321:
This may due to https://issues.apache.org/jira/browse/IMPALA-9287, let's see if
the failure would be fixed after disable that test.
> TestParquetMaxPageHeader.test_large_page_header_config failed with error
> "Invalid configuration of tez jars"
>
>
> Key: IMPALA-9321
> URL: https://issues.apache.org/jira/browse/IMPALA-9321
> Project: IMPALA
> Issue Type: Test
> Components: Infrastructure
>Affects Versions: Impala 3.4.0
>Reporter: Xiaomeng Zhang
>Priority: Major
> Labels: broken-build
>
> This test has been failed in last 10 builds of impala-cdpd-master-exhuastive
> job.
> {code:java}
> E INFO : Compiling
> command(queryId=jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a):
> INSERT OVERWRITE TABLE large_page_header SELECT col FROM
> parquet_test_data_text
> E INFO : Concurrency mode is disabled, not creating a lock manager
> E INFO : No Stats for default@parquet_test_data_text, Columns: col
> E INFO : Semantic Analysis Completed (retrial = false)
> E INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:col,
> type:string, comment:null)], properties:null)
> E INFO : Completed compiling
> command(queryId=jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a);
> Time taken: 0.671 seconds
> E INFO : Concurrency mode is disabled, not creating a lock manager
> E INFO : Executing
> command(queryId=jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a):
> INSERT OVERWRITE TABLE large_page_header SELECT col FROM
> parquet_test_data_text
> E INFO : Query ID =
> jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a
> E INFO : Total jobs = 1
> E INFO : Launching Job 1 out of 1
> E INFO : Starting task [Stage-1:MAPRED] in serial mode
> E INFO : Subscribed to counters: [] for queryId:
> jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a
> E INFO : Tez session hasn't been created yet. Opening session
> E ERROR : Failed to execute tez graph.
> E org.apache.tez.dag.api.TezUncheckedException: Invalid configuration of
> tez jars, tez.lib.uris is not defined in the configuration
> E at
> org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:179)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at
> org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:1163)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at
> org.apache.tez.client.TezClient.setupApplicationContext(TezClient.java:477)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at org.apache.tez.client.TezClient.start(TezClient.java:403)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:522)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:373)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:298)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolSession.open(TezSessionPoolSession.java:106)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezTask.ensureSessionHasResources(TezTask.java:380)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:203)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:212)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:103)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2302)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1954)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1627)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1387)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1381)
>
[jira] [Commented] (IMPALA-9321) TestParquetMaxPageHeader.test_large_page_header_config failed with error "Invalid configuration of tez jars"
[
https://issues.apache.org/jira/browse/IMPALA-9321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021640#comment-17021640
]
Xiaomeng Zhang commented on IMPALA-9321:
>From here I can see we set 'tez.ignore.lib.uris': 'true'
[https://github.com/apache/impala/blame/497a17dbdc0669abd47c2360b8ca94de8b54d413/fe/src/test/resources/hive-site.xml.py#L97]
> TestParquetMaxPageHeader.test_large_page_header_config failed with error
> "Invalid configuration of tez jars"
>
>
> Key: IMPALA-9321
> URL: https://issues.apache.org/jira/browse/IMPALA-9321
> Project: IMPALA
> Issue Type: Test
> Components: Infrastructure
>Affects Versions: Impala 3.4.0
>Reporter: Xiaomeng Zhang
>Priority: Major
> Labels: broken-build
>
> This test has been failed in last 10 builds of impala-cdpd-master-exhuastive
> job.
> {code:java}
> E INFO : Compiling
> command(queryId=jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a):
> INSERT OVERWRITE TABLE large_page_header SELECT col FROM
> parquet_test_data_text
> E INFO : Concurrency mode is disabled, not creating a lock manager
> E INFO : No Stats for default@parquet_test_data_text, Columns: col
> E INFO : Semantic Analysis Completed (retrial = false)
> E INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:col,
> type:string, comment:null)], properties:null)
> E INFO : Completed compiling
> command(queryId=jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a);
> Time taken: 0.671 seconds
> E INFO : Concurrency mode is disabled, not creating a lock manager
> E INFO : Executing
> command(queryId=jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a):
> INSERT OVERWRITE TABLE large_page_header SELECT col FROM
> parquet_test_data_text
> E INFO : Query ID =
> jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a
> E INFO : Total jobs = 1
> E INFO : Launching Job 1 out of 1
> E INFO : Starting task [Stage-1:MAPRED] in serial mode
> E INFO : Subscribed to counters: [] for queryId:
> jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a
> E INFO : Tez session hasn't been created yet. Opening session
> E ERROR : Failed to execute tez graph.
> E org.apache.tez.dag.api.TezUncheckedException: Invalid configuration of
> tez jars, tez.lib.uris is not defined in the configuration
> E at
> org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:179)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at
> org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:1163)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at
> org.apache.tez.client.TezClient.setupApplicationContext(TezClient.java:477)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at org.apache.tez.client.TezClient.start(TezClient.java:403)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:522)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:373)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:298)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolSession.open(TezSessionPoolSession.java:106)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezTask.ensureSessionHasResources(TezTask.java:380)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:203)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:212)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:103)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2302)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1954)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1627)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1387)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
[jira] [Updated] (IMPALA-9323) test_blacklist.TestBlacklist.test_kill_impalad_with_running_queries failed
[
https://issues.apache.org/jira/browse/IMPALA-9323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang updated IMPALA-9323:
---
Description:
[https://master-02.jenkins.cloudera.com/job/impala-cdpd-master-exhaustive-release/246/testReport/junit/custom_cluster.test_blacklist/TestBlacklist/test_kill_impalad_with_running_queries/]
{code:java}
ERROR:impala.hiveserver2:Failed to open transport (tries_left=3)
Traceback (most recent call last):
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/repos/Impala/infra/python/env/lib/python2.7/site-packages/impala/hiveserver2.py",
line 1009, in _execute
return func(request)
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/repos/Impala/infra/python/env/lib/python2.7/site-packages/impala/_thrift_gen/TCLIService/TCLIService.py",
line 176, in OpenSession
return self.recv_OpenSession()
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/repos/Impala/infra/python/env/lib/python2.7/site-packages/impala/_thrift_gen/TCLIService/TCLIService.py",
line 188, in recv_OpenSession
(fname, mtype, rseqid) = iprot.readMessageBegin()
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/Impala-Toolchain/thrift-0.9.3-p7/python/lib64/python2.7/site-packages/thrift/protocol/TBinaryProtocol.py",
line 126, in readMessageBegin
sz = self.readI32()
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/Impala-Toolchain/thrift-0.9.3-p7/python/lib64/python2.7/site-packages/thrift/protocol/TBinaryProtocol.py",
line 206, in readI32
buff = self.trans.readAll(4)
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/Impala-Toolchain/thrift-0.9.3-p7/python/lib64/python2.7/site-packages/thrift/transport/TTransport.py",
line 58, in readAll
chunk = self.read(sz - have)
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/Impala-Toolchain/thrift-0.9.3-p7/python/lib64/python2.7/site-packages/thrift/transport/TTransport.py",
line 159, in read
self.__rbuf = StringIO(self.__trans.read(max(sz, self.__rbuf_size)))
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/Impala-Toolchain/thrift-0.9.3-p7/python/lib64/python2.7/site-packages/thrift/transport/TSocket.py",
line 120, in read
message='TSocket read 0 bytes')
TTransportException: TSocket read 0 bytes
INFO:impala_cluster:Killing with signal 9
{code}
I can see this test was newly added in
[https://github.com/apache/impala/commit/8a4fececcf8e9599978cc1a532386b8e924838ed]
It has been failed in latest 10 builds
was:
[https://master-02.jenkins.cloudera.com/job/impala-cdpd-master-exhaustive-release/246/testReport/junit/custom_cluster.test_blacklist/TestBlacklist/test_kill_impalad_with_running_queries/]
{code:java}
ERROR:impala.hiveserver2:Failed to open transport (tries_left=3)
Traceback (most recent call last):
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/repos/Impala/infra/python/env/lib/python2.7/site-packages/impala/hiveserver2.py",
line 1009, in _execute
return func(request)
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/repos/Impala/infra/python/env/lib/python2.7/site-packages/impala/_thrift_gen/TCLIService/TCLIService.py",
line 176, in OpenSession
return self.recv_OpenSession()
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/repos/Impala/infra/python/env/lib/python2.7/site-packages/impala/_thrift_gen/TCLIService/TCLIService.py",
line 188, in recv_OpenSession
(fname, mtype, rseqid) = iprot.readMessageBegin()
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/Impala-Toolchain/thrift-0.9.3-p7/python/lib64/python2.7/site-packages/thrift/protocol/TBinaryProtocol.py",
line 126, in readMessageBegin
sz = self.readI32()
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/Impala-Toolchain/thrift-0.9.3-p7/python/lib64/python2.7/site-packages/thrift/protocol/TBinaryProtocol.py",
line 206, in readI32
buff = self.trans.readAll(4)
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/Impala-Toolchain/thrift-0.9.3-p7/python/lib64/python2.7/site-packages/thrift/transport/TTransport.py",
line 58, in readAll
chunk = self.read(sz - have)
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/Impala-Toolchain/thrift-0.9.3-p7/python/lib64/python2.7/site-packages/thrift/transport/TTransport.py",
line 159, in read
self.__rbuf = StringIO(self.__trans.read(max(sz, self.__rbuf_size)))
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/Impala-Toolchain/thrift-0.9.3-p7/python/lib64/python2.7/site-packages/thrift/transport/TSocket.py",
line 120, in read
message='TSocket read 0 bytes')
TTransportException: TSocket read 0 bytes
INFO:impala_cluster:Killing with signal 9
{code}
I can see this test was newly added in
[jira] [Created] (IMPALA-9323) test_blacklist.TestBlacklist.test_kill_impalad_with_running_queries failed
Xiaomeng Zhang created IMPALA-9323:
--
Summary:
test_blacklist.TestBlacklist.test_kill_impalad_with_running_queries failed
Key: IMPALA-9323
URL: https://issues.apache.org/jira/browse/IMPALA-9323
Project: IMPALA
Issue Type: Test
Components: Infrastructure
Affects Versions: Impala 3.4.0
Reporter: Xiaomeng Zhang
Assignee: Sahil Takiar
[https://master-02.jenkins.cloudera.com/job/impala-cdpd-master-exhaustive-release/246/testReport/junit/custom_cluster.test_blacklist/TestBlacklist/test_kill_impalad_with_running_queries/]
{code:java}
ERROR:impala.hiveserver2:Failed to open transport (tries_left=3)
Traceback (most recent call last):
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/repos/Impala/infra/python/env/lib/python2.7/site-packages/impala/hiveserver2.py",
line 1009, in _execute
return func(request)
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/repos/Impala/infra/python/env/lib/python2.7/site-packages/impala/_thrift_gen/TCLIService/TCLIService.py",
line 176, in OpenSession
return self.recv_OpenSession()
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/repos/Impala/infra/python/env/lib/python2.7/site-packages/impala/_thrift_gen/TCLIService/TCLIService.py",
line 188, in recv_OpenSession
(fname, mtype, rseqid) = iprot.readMessageBegin()
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/Impala-Toolchain/thrift-0.9.3-p7/python/lib64/python2.7/site-packages/thrift/protocol/TBinaryProtocol.py",
line 126, in readMessageBegin
sz = self.readI32()
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/Impala-Toolchain/thrift-0.9.3-p7/python/lib64/python2.7/site-packages/thrift/protocol/TBinaryProtocol.py",
line 206, in readI32
buff = self.trans.readAll(4)
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/Impala-Toolchain/thrift-0.9.3-p7/python/lib64/python2.7/site-packages/thrift/transport/TTransport.py",
line 58, in readAll
chunk = self.read(sz - have)
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/Impala-Toolchain/thrift-0.9.3-p7/python/lib64/python2.7/site-packages/thrift/transport/TTransport.py",
line 159, in read
self.__rbuf = StringIO(self.__trans.read(max(sz, self.__rbuf_size)))
File
"/data/jenkins/workspace/impala-cdpd-master-exhaustive-release/Impala-Toolchain/thrift-0.9.3-p7/python/lib64/python2.7/site-packages/thrift/transport/TSocket.py",
line 120, in read
message='TSocket read 0 bytes')
TTransportException: TSocket read 0 bytes
INFO:impala_cluster:Killing with signal 9
{code}
I can see this test was newly added in
[https://github.com/apache/impala/commit/8a4fececcf8e9599978cc1a532386b8e924838ed]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-9322) authorization.test_grant_revoke.TestGrantRevoke.test_grant_revoke failed with "Unable to start Sentry"
[
https://issues.apache.org/jira/browse/IMPALA-9322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang updated IMPALA-9322:
---
Description:
In last few builds of impala-cdpd-master-exhuastive tests, authorization test
failed with error "Unable to start Sentry"
{code:java}
Error Messagetest setup failureStacktraceauthorization/test_grant_revoke.py:51:
in setup_method
super(TestGrantRevoke, self).setup_method(method)
common/custom_cluster_test_suite.py:164: in setup_method
method.func_dict.get(SENTRY_LOG_DIR))
common/custom_cluster_test_suite.py:231: in _start_sentry_service
raise RuntimeError("Unable to start Sentry")
E RuntimeError: Unable to start SentryStandard OutputStopping Sentry
ERROR in
/data/jenkins/workspace/impala-cdpd-master-exhaustive/repos/Impala/testdata/bin/run-sentry-service.sh
at line 54: "$JAVA" -cp $CLASSPATH
org.apache.impala.testutil.SentryServicePinger \
Generated:
/data/jenkins/workspace/impala-cdpd-master-exhaustive/repos/Impala/logs/extra_junit_xml_logs/generate_junitxml.buildall.run-sentry-service.20200122_02_11_24.xml
{code}
{code:java}
org.apache.impala.common.InternalException: Error making 'listRoles' RPC to
Sentry Service:
at
org.apache.impala.authorization.sentry.SentryPolicyService.listAllRoles(SentryPolicyService.java:422)
at
org.apache.impala.testutil.SentryServicePinger.main(SentryServicePinger.java:83)
Caused by: org.apache.sentry.core.common.exception.SentryUserException:
java.net.ConnectException: Connection refused (Connection refused)
at
org.apache.sentry.core.common.transport.RetryClientInvocationHandler.connect(RetryClientInvocationHandler.java:166)
at
org.apache.sentry.core.common.transport.RetryClientInvocationHandler.invokeImpl(RetryClientInvocationHandler.java:90)
at
org.apache.sentry.core.common.transport.SentryClientInvocationHandler.invoke(SentryClientInvocationHandler.java:41)
at com.sun.proxy.$Proxy5.listAllRoles(Unknown Source)
at
org.apache.impala.authorization.sentry.SentryPolicyService.listAllRoles(SentryPolicyService.java:416)
... 1 more
Caused by: org.apache.thrift.transport.TTransportException:
java.net.ConnectException: Connection refused (Connection refused)
at org.apache.thrift.transport.TSocket.open(TSocket.java:226)
at
org.apache.sentry.core.common.transport.SentryTransportFactory.connectToServer(SentryTransportFactory.java:99)
at
org.apache.sentry.core.common.transport.SentryTransportFactory.getTransport(SentryTransportFactory.java:86)
at
org.apache.sentry.core.common.transport.SentryTransportPool$PoolFactory.create(SentryTransportPool.java:302)
at
org.apache.sentry.core.common.transport.SentryTransportPool$PoolFactory.create(SentryTransportPool.java:271)
at
org.apache.commons.pool2.BaseKeyedPooledObjectFactory.makeObject(BaseKeyedPooledObjectFactory.java:62)
at
org.apache.commons.pool2.impl.GenericKeyedObjectPool.create(GenericKeyedObjectPool.java:1041)
at
org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:380)
at
org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:279)
at
org.apache.sentry.core.common.transport.SentryTransportPool.getTransport(SentryTransportPool.java:183)
at
org.apache.sentry.api.service.thrift.SentryPolicyServiceClientDefaultImpl.connect(SentryPolicyServiceClientDefaultImpl.java:100)
at
org.apache.sentry.core.common.transport.RetryClientInvocationHandler.connect(RetryClientInvocationHandler.java:141)
... 5 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.thrift.transport.TSocket.open(TSocket.java:221)
... 16 more
{code}
was:
In last few builds of impala-cdpd-master-exhuastive tests, authorization test
failed with error "Unable to start Sentry"
{code:java}
org.apache.impala.common.InternalException: Error making 'listRoles' RPC to
Sentry Service:
at
org.apache.impala.authorization.sentry.SentryPolicyService.listAllRoles(SentryPolicyService.java:422)
at
org.apache.impala.testutil.SentryServicePinger.main(SentryServicePinger.java:83)
Caused by: org.apache.sentry.core.common.exception.SentryUserException:
java.net.ConnectException: Connection refused (Connection refused)
at
[jira] [Created] (IMPALA-9322) authorization.test_grant_revoke.TestGrantRevoke.test_grant_revoke failed with "Unable to start Sentry"
Xiaomeng Zhang created IMPALA-9322:
--
Summary:
authorization.test_grant_revoke.TestGrantRevoke.test_grant_revoke failed with
"Unable to start Sentry"
Key: IMPALA-9322
URL: https://issues.apache.org/jira/browse/IMPALA-9322
Project: IMPALA
Issue Type: Test
Components: Infrastructure
Affects Versions: Impala 3.4.0
Reporter: Xiaomeng Zhang
In last few builds of impala-cdpd-master-exhuastive tests, authorization test
failed with error "Unable to start Sentry"
{code:java}
org.apache.impala.common.InternalException: Error making 'listRoles' RPC to
Sentry Service:
at
org.apache.impala.authorization.sentry.SentryPolicyService.listAllRoles(SentryPolicyService.java:422)
at
org.apache.impala.testutil.SentryServicePinger.main(SentryServicePinger.java:83)
Caused by: org.apache.sentry.core.common.exception.SentryUserException:
java.net.ConnectException: Connection refused (Connection refused)
at
org.apache.sentry.core.common.transport.RetryClientInvocationHandler.connect(RetryClientInvocationHandler.java:166)
at
org.apache.sentry.core.common.transport.RetryClientInvocationHandler.invokeImpl(RetryClientInvocationHandler.java:90)
at
org.apache.sentry.core.common.transport.SentryClientInvocationHandler.invoke(SentryClientInvocationHandler.java:41)
at com.sun.proxy.$Proxy5.listAllRoles(Unknown Source)
at
org.apache.impala.authorization.sentry.SentryPolicyService.listAllRoles(SentryPolicyService.java:416)
... 1 more
Caused by: org.apache.thrift.transport.TTransportException:
java.net.ConnectException: Connection refused (Connection refused)
at org.apache.thrift.transport.TSocket.open(TSocket.java:226)
at
org.apache.sentry.core.common.transport.SentryTransportFactory.connectToServer(SentryTransportFactory.java:99)
at
org.apache.sentry.core.common.transport.SentryTransportFactory.getTransport(SentryTransportFactory.java:86)
at
org.apache.sentry.core.common.transport.SentryTransportPool$PoolFactory.create(SentryTransportPool.java:302)
at
org.apache.sentry.core.common.transport.SentryTransportPool$PoolFactory.create(SentryTransportPool.java:271)
at
org.apache.commons.pool2.BaseKeyedPooledObjectFactory.makeObject(BaseKeyedPooledObjectFactory.java:62)
at
org.apache.commons.pool2.impl.GenericKeyedObjectPool.create(GenericKeyedObjectPool.java:1041)
at
org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:380)
at
org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:279)
at
org.apache.sentry.core.common.transport.SentryTransportPool.getTransport(SentryTransportPool.java:183)
at
org.apache.sentry.api.service.thrift.SentryPolicyServiceClientDefaultImpl.connect(SentryPolicyServiceClientDefaultImpl.java:100)
at
org.apache.sentry.core.common.transport.RetryClientInvocationHandler.connect(RetryClientInvocationHandler.java:141)
... 5 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.thrift.transport.TSocket.open(TSocket.java:221)
... 16 more
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-9321) TestParquetMaxPageHeader.test_large_page_header_config failed with error "Invalid configuration of tez jars"
[
https://issues.apache.org/jira/browse/IMPALA-9321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang updated IMPALA-9321:
---
Labels: broken-build (was: )
> TestParquetMaxPageHeader.test_large_page_header_config failed with error
> "Invalid configuration of tez jars"
>
>
> Key: IMPALA-9321
> URL: https://issues.apache.org/jira/browse/IMPALA-9321
> Project: IMPALA
> Issue Type: Test
> Components: Infrastructure
>Affects Versions: Impala 3.4.0
>Reporter: Xiaomeng Zhang
>Priority: Major
> Labels: broken-build
>
> This test has been failed in last 10 builds of impala-cdpd-master-exhuastive
> job.
> {code:java}
> E INFO : Compiling
> command(queryId=jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a):
> INSERT OVERWRITE TABLE large_page_header SELECT col FROM
> parquet_test_data_text
> E INFO : Concurrency mode is disabled, not creating a lock manager
> E INFO : No Stats for default@parquet_test_data_text, Columns: col
> E INFO : Semantic Analysis Completed (retrial = false)
> E INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:col,
> type:string, comment:null)], properties:null)
> E INFO : Completed compiling
> command(queryId=jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a);
> Time taken: 0.671 seconds
> E INFO : Concurrency mode is disabled, not creating a lock manager
> E INFO : Executing
> command(queryId=jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a):
> INSERT OVERWRITE TABLE large_page_header SELECT col FROM
> parquet_test_data_text
> E INFO : Query ID =
> jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a
> E INFO : Total jobs = 1
> E INFO : Launching Job 1 out of 1
> E INFO : Starting task [Stage-1:MAPRED] in serial mode
> E INFO : Subscribed to counters: [] for queryId:
> jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a
> E INFO : Tez session hasn't been created yet. Opening session
> E ERROR : Failed to execute tez graph.
> E org.apache.tez.dag.api.TezUncheckedException: Invalid configuration of
> tez jars, tez.lib.uris is not defined in the configuration
> E at
> org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:179)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at
> org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:1163)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at
> org.apache.tez.client.TezClient.setupApplicationContext(TezClient.java:477)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at org.apache.tez.client.TezClient.start(TezClient.java:403)
> ~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:522)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:373)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:298)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolSession.open(TezSessionPoolSession.java:106)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.tez.TezTask.ensureSessionHasResources(TezTask.java:380)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:203)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:212)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:103)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2302)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1954)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1627)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1387)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1381)
> ~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
> E at
>
[jira] [Created] (IMPALA-9321) TestParquetMaxPageHeader.test_large_page_header_config failed with error "Invalid configuration of tez jars"
Xiaomeng Zhang created IMPALA-9321:
--
Summary: TestParquetMaxPageHeader.test_large_page_header_config
failed with error "Invalid configuration of tez jars"
Key: IMPALA-9321
URL: https://issues.apache.org/jira/browse/IMPALA-9321
Project: IMPALA
Issue Type: Test
Components: Infrastructure
Affects Versions: Impala 3.4.0
Reporter: Xiaomeng Zhang
This test has been failed in last 10 builds of impala-cdpd-master-exhuastive
job.
{code:java}
E INFO : Compiling
command(queryId=jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a):
INSERT OVERWRITE TABLE large_page_header SELECT col FROM parquet_test_data_text
E INFO : Concurrency mode is disabled, not creating a lock manager
E INFO : No Stats for default@parquet_test_data_text, Columns: col
E INFO : Semantic Analysis Completed (retrial = false)
E INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:col,
type:string, comment:null)], properties:null)
E INFO : Completed compiling
command(queryId=jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a);
Time taken: 0.671 seconds
E INFO : Concurrency mode is disabled, not creating a lock manager
E INFO : Executing
command(queryId=jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a):
INSERT OVERWRITE TABLE large_page_header SELECT col FROM parquet_test_data_text
E INFO : Query ID =
jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a
E INFO : Total jobs = 1
E INFO : Launching Job 1 out of 1
E INFO : Starting task [Stage-1:MAPRED] in serial mode
E INFO : Subscribed to counters: [] for queryId:
jenkins_20200121171151_3f3f42dc-08f8-4f6a-a635-c29fdc34695a
E INFO : Tez session hasn't been created yet. Opening session
E ERROR : Failed to execute tez graph.
E org.apache.tez.dag.api.TezUncheckedException: Invalid configuration of tez
jars, tez.lib.uris is not defined in the configuration
E at
org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:179)
~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
E at
org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:1163)
~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
E at
org.apache.tez.client.TezClient.setupApplicationContext(TezClient.java:477)
~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
E at org.apache.tez.client.TezClient.start(TezClient.java:403)
~[tez-api-0.9.1.7.0.2.0-212.jar:0.9.1.7.0.2.0-212]
E at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:522)
~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
E at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:373)
~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
E at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:298)
~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
E at
org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolSession.open(TezSessionPoolSession.java:106)
~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
E at
org.apache.hadoop.hive.ql.exec.tez.TezTask.ensureSessionHasResources(TezTask.java:380)
~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
E at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:203)
~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
E at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:212)
~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
E at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:103)
~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
E at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2302)
~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
E at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1954)
~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
E at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1627)
~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
E at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1387)
~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
E at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1381)
~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
E at
org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157)
~[hive-exec-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
E at
org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:229)
~[hive-service-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
E at
org.apache.hive.service.cli.operation.SQLOperation.access$600(SQLOperation.java:87)
~[hive-service-3.1.2000.7.0.2.0-212.jar:3.1.2000.7.0.2.0-212]
E at
[jira] [Updated] (IMPALA-9320) test_udf_concurrency.TestUdfConcurrency.test_concurrent_jar_drop_use failed with error hdfs path doesn't exist
[
https://issues.apache.org/jira/browse/IMPALA-9320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang updated IMPALA-9320:
---
Description:
{code:java}
custom_cluster/test_udf_concurrency.py:162: in test_concurrent_jar_drop_use
self.filesystem_client.copy_from_local(udf_src_path, udf_tgt_path)
util/hdfs_util.py:82: in copy_from_local
self.hdfs_filesystem_client.copy_from_local(src, dst)
util/hdfs_util.py:256: in copy_from_local
src, dst) + stderr + '; ' + stdout
E AssertionError: HDFS copy from
/data/jenkins/workspace/impala-cdpd-master-exhaustive/repos/Impala/testdata/udfs/impala-hive-udfs.jar
to
/test-warehouse/test_concurrent_jar_drop_use_91093fa5.db/impala-hive-udfs.jar
failed: copyFromLocal:
`/test-warehouse/test_concurrent_jar_drop_use_91093fa5.db/impala-hive-udfs.jar':
No such file or directory:
`hdfs://localhost:20500/test-warehouse/test_concurrent_jar_drop_use_91093fa5.db/impala-hive-udfs.jar'
E ;
{code}
[https://master-02.jenkins.cloudera.com/job/impala-cdpd-master-exhaustive/244/testReport/junit/custom_cluster.test_udf_concurrency/TestUdfConcurrency/test_concurrent_jar_drop_use_protocol__beeswax___exec_optionbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/]
This test has been continuously failing in last 10 builds.
was:
{code:java}
custom_cluster/test_udf_concurrency.py:162: in test_concurrent_jar_drop_use
self.filesystem_client.copy_from_local(udf_src_path, udf_tgt_path)
util/hdfs_util.py:82: in copy_from_local
self.hdfs_filesystem_client.copy_from_local(src, dst)
util/hdfs_util.py:256: in copy_from_local
src, dst) + stderr + '; ' + stdout
E AssertionError: HDFS copy from
/data/jenkins/workspace/impala-cdpd-master-exhaustive/repos/Impala/testdata/udfs/impala-hive-udfs.jar
to
/test-warehouse/test_concurrent_jar_drop_use_91093fa5.db/impala-hive-udfs.jar
failed: copyFromLocal:
`/test-warehouse/test_concurrent_jar_drop_use_91093fa5.db/impala-hive-udfs.jar':
No such file or directory:
`hdfs://localhost:20500/test-warehouse/test_concurrent_jar_drop_use_91093fa5.db/impala-hive-udfs.jar'
E ;
{code}
https://master-02.jenkins.cloudera.com/job/impala-cdpd-master-exhaustive/244/testReport/junit/custom_cluster.test_udf_concurrency/TestUdfConcurrency/test_concurrent_jar_drop_use_protocol__beeswax___exec_optionbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/
> test_udf_concurrency.TestUdfConcurrency.test_concurrent_jar_drop_use failed
> with error hdfs path doesn't exist
> --
>
> Key: IMPALA-9320
> URL: https://issues.apache.org/jira/browse/IMPALA-9320
> Project: IMPALA
> Issue Type: Test
> Components: Infrastructure
>Affects Versions: Impala 3.4.0
>Reporter: Xiaomeng Zhang
>Assignee: Joe McDonnell
>Priority: Major
> Labels: broken-build
>
> {code:java}
> custom_cluster/test_udf_concurrency.py:162: in test_concurrent_jar_drop_use
> self.filesystem_client.copy_from_local(udf_src_path, udf_tgt_path)
> util/hdfs_util.py:82: in copy_from_local
> self.hdfs_filesystem_client.copy_from_local(src, dst)
> util/hdfs_util.py:256: in copy_from_local
> src, dst) + stderr + '; ' + stdout
> E AssertionError: HDFS copy from
> /data/jenkins/workspace/impala-cdpd-master-exhaustive/repos/Impala/testdata/udfs/impala-hive-udfs.jar
> to
> /test-warehouse/test_concurrent_jar_drop_use_91093fa5.db/impala-hive-udfs.jar
> failed: copyFromLocal:
> `/test-warehouse/test_concurrent_jar_drop_use_91093fa5.db/impala-hive-udfs.jar':
> No such file or directory:
> `hdfs://localhost:20500/test-warehouse/test_concurrent_jar_drop_use_91093fa5.db/impala-hive-udfs.jar'
> E ;
> {code}
> [https://master-02.jenkins.cloudera.com/job/impala-cdpd-master-exhaustive/244/testReport/junit/custom_cluster.test_udf_concurrency/TestUdfConcurrency/test_concurrent_jar_drop_use_protocol__beeswax___exec_optionbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/]
> This test has been continuously failing in last 10 builds.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-9320) test_udf_concurrency.TestUdfConcurrency.test_concurrent_jar_drop_use failed with error hdfs path doesn't exist
[
https://issues.apache.org/jira/browse/IMPALA-9320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang reassigned IMPALA-9320:
--
Assignee: Joe McDonnell
> test_udf_concurrency.TestUdfConcurrency.test_concurrent_jar_drop_use failed
> with error hdfs path doesn't exist
> --
>
> Key: IMPALA-9320
> URL: https://issues.apache.org/jira/browse/IMPALA-9320
> Project: IMPALA
> Issue Type: Test
> Components: Infrastructure
>Affects Versions: Impala 3.4.0
>Reporter: Xiaomeng Zhang
>Assignee: Joe McDonnell
>Priority: Major
> Labels: broken-build
>
> {code:java}
> custom_cluster/test_udf_concurrency.py:162: in test_concurrent_jar_drop_use
> self.filesystem_client.copy_from_local(udf_src_path, udf_tgt_path)
> util/hdfs_util.py:82: in copy_from_local
> self.hdfs_filesystem_client.copy_from_local(src, dst)
> util/hdfs_util.py:256: in copy_from_local
> src, dst) + stderr + '; ' + stdout
> E AssertionError: HDFS copy from
> /data/jenkins/workspace/impala-cdpd-master-exhaustive/repos/Impala/testdata/udfs/impala-hive-udfs.jar
> to
> /test-warehouse/test_concurrent_jar_drop_use_91093fa5.db/impala-hive-udfs.jar
> failed: copyFromLocal:
> `/test-warehouse/test_concurrent_jar_drop_use_91093fa5.db/impala-hive-udfs.jar':
> No such file or directory:
> `hdfs://localhost:20500/test-warehouse/test_concurrent_jar_drop_use_91093fa5.db/impala-hive-udfs.jar'
> E ;
> {code}
> https://master-02.jenkins.cloudera.com/job/impala-cdpd-master-exhaustive/244/testReport/junit/custom_cluster.test_udf_concurrency/TestUdfConcurrency/test_concurrent_jar_drop_use_protocol__beeswax___exec_optionbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-9320) test_udf_concurrency.TestUdfConcurrency.test_concurrent_jar_drop_use failed with error hdfs path doesn't exist
Xiaomeng Zhang created IMPALA-9320:
--
Summary:
test_udf_concurrency.TestUdfConcurrency.test_concurrent_jar_drop_use failed
with error hdfs path doesn't exist
Key: IMPALA-9320
URL: https://issues.apache.org/jira/browse/IMPALA-9320
Project: IMPALA
Issue Type: Test
Components: Infrastructure
Affects Versions: Impala 3.4.0
Reporter: Xiaomeng Zhang
{code:java}
custom_cluster/test_udf_concurrency.py:162: in test_concurrent_jar_drop_use
self.filesystem_client.copy_from_local(udf_src_path, udf_tgt_path)
util/hdfs_util.py:82: in copy_from_local
self.hdfs_filesystem_client.copy_from_local(src, dst)
util/hdfs_util.py:256: in copy_from_local
src, dst) + stderr + '; ' + stdout
E AssertionError: HDFS copy from
/data/jenkins/workspace/impala-cdpd-master-exhaustive/repos/Impala/testdata/udfs/impala-hive-udfs.jar
to
/test-warehouse/test_concurrent_jar_drop_use_91093fa5.db/impala-hive-udfs.jar
failed: copyFromLocal:
`/test-warehouse/test_concurrent_jar_drop_use_91093fa5.db/impala-hive-udfs.jar':
No such file or directory:
`hdfs://localhost:20500/test-warehouse/test_concurrent_jar_drop_use_91093fa5.db/impala-hive-udfs.jar'
E ;
{code}
https://master-02.jenkins.cloudera.com/job/impala-cdpd-master-exhaustive/244/testReport/junit/custom_cluster.test_udf_concurrency/TestUdfConcurrency/test_concurrent_jar_drop_use_protocol__beeswax___exec_optionbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-9311) test_show_create_table failed with primary key mismatch
[
https://issues.apache.org/jira/browse/IMPALA-9311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang updated IMPALA-9311:
---
Issue Type: Test (was: Bug)
> test_show_create_table failed with primary key mismatch
> ---
>
> Key: IMPALA-9311
> URL: https://issues.apache.org/jira/browse/IMPALA-9311
> Project: IMPALA
> Issue Type: Test
>Affects Versions: Impala 3.4.0
>Reporter: Xiaomeng Zhang
>Assignee: Anurag Mantripragada
>Priority: Major
> Labels: broken-build
>
> {code:java}
> Error Messagemetadata/test_show_create_table.py:62: in test_show_create_table
> unique_database) metadata/test_show_create_table.py:110: in
> __run_show_create_table_test_case self.__compare_result(expected_result,
> create_table_result) metadata/test_show_create_table.py:146: in
> __compare_result assert expected_sql_filtered == actual_sql_filtered E
> assert "CREATE EXTER...parent_table'" == "CREATE EXTERN...parent_table'" E
> Skipping 71 identical leading characters in diff, use -v to show E
> Skipping 126 identical trailing characters in diff, use -v to show E -
> MARY KEY (year, id)) ROW FO E ? E + MARY KEY (id,
> year)) ROW FO E ?
> Stacktracemetadata/test_show_create_table.py:62: in test_show_create_table
> unique_database)
> metadata/test_show_create_table.py:110: in __run_show_create_table_test_case
> self.__compare_result(expected_result, create_table_result)
> metadata/test_show_create_table.py:146: in __compare_result
> assert expected_sql_filtered == actual_sql_filtered
> E assert "CREATE EXTER...parent_table'" == "CREATE EXTERN...parent_table'"
> E Skipping 71 identical leading characters in diff, use -v to show
> E Skipping 126 identical trailing characters in diff, use -v to show
> E - MARY KEY (year, id)) ROW FO
> E ?
> E + MARY KEY (id, year)) ROW FO
> E ? {code}
> I think this is due to commit
> [https://github.com/apache/impala/commit/cfe60858da110cf1256bd3aa5d4f8d374578a33d]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-9311) test_show_create_table failed with primary key mismatch
[
https://issues.apache.org/jira/browse/IMPALA-9311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang updated IMPALA-9311:
---
Labels: broken-build (was: )
> test_show_create_table failed with primary key mismatch
> ---
>
> Key: IMPALA-9311
> URL: https://issues.apache.org/jira/browse/IMPALA-9311
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.4.0
>Reporter: Xiaomeng Zhang
>Assignee: Anurag Mantripragada
>Priority: Major
> Labels: broken-build
>
> {code:java}
> Error Messagemetadata/test_show_create_table.py:62: in test_show_create_table
> unique_database) metadata/test_show_create_table.py:110: in
> __run_show_create_table_test_case self.__compare_result(expected_result,
> create_table_result) metadata/test_show_create_table.py:146: in
> __compare_result assert expected_sql_filtered == actual_sql_filtered E
> assert "CREATE EXTER...parent_table'" == "CREATE EXTERN...parent_table'" E
> Skipping 71 identical leading characters in diff, use -v to show E
> Skipping 126 identical trailing characters in diff, use -v to show E -
> MARY KEY (year, id)) ROW FO E ? E + MARY KEY (id,
> year)) ROW FO E ?
> Stacktracemetadata/test_show_create_table.py:62: in test_show_create_table
> unique_database)
> metadata/test_show_create_table.py:110: in __run_show_create_table_test_case
> self.__compare_result(expected_result, create_table_result)
> metadata/test_show_create_table.py:146: in __compare_result
> assert expected_sql_filtered == actual_sql_filtered
> E assert "CREATE EXTER...parent_table'" == "CREATE EXTERN...parent_table'"
> E Skipping 71 identical leading characters in diff, use -v to show
> E Skipping 126 identical trailing characters in diff, use -v to show
> E - MARY KEY (year, id)) ROW FO
> E ?
> E + MARY KEY (id, year)) ROW FO
> E ? {code}
> I think this is due to commit
> [https://github.com/apache/impala/commit/cfe60858da110cf1256bd3aa5d4f8d374578a33d]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-9311) test_show_create_table failed with primary key mismatch
[
https://issues.apache.org/jira/browse/IMPALA-9311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang updated IMPALA-9311:
---
Affects Version/s: Impala 3.4.0
> test_show_create_table failed with primary key mismatch
> ---
>
> Key: IMPALA-9311
> URL: https://issues.apache.org/jira/browse/IMPALA-9311
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.4.0
>Reporter: Xiaomeng Zhang
>Assignee: Anurag Mantripragada
>Priority: Major
>
> {code:java}
> Error Messagemetadata/test_show_create_table.py:62: in test_show_create_table
> unique_database) metadata/test_show_create_table.py:110: in
> __run_show_create_table_test_case self.__compare_result(expected_result,
> create_table_result) metadata/test_show_create_table.py:146: in
> __compare_result assert expected_sql_filtered == actual_sql_filtered E
> assert "CREATE EXTER...parent_table'" == "CREATE EXTERN...parent_table'" E
> Skipping 71 identical leading characters in diff, use -v to show E
> Skipping 126 identical trailing characters in diff, use -v to show E -
> MARY KEY (year, id)) ROW FO E ? E + MARY KEY (id,
> year)) ROW FO E ?
> Stacktracemetadata/test_show_create_table.py:62: in test_show_create_table
> unique_database)
> metadata/test_show_create_table.py:110: in __run_show_create_table_test_case
> self.__compare_result(expected_result, create_table_result)
> metadata/test_show_create_table.py:146: in __compare_result
> assert expected_sql_filtered == actual_sql_filtered
> E assert "CREATE EXTER...parent_table'" == "CREATE EXTERN...parent_table'"
> E Skipping 71 identical leading characters in diff, use -v to show
> E Skipping 126 identical trailing characters in diff, use -v to show
> E - MARY KEY (year, id)) ROW FO
> E ?
> E + MARY KEY (id, year)) ROW FO
> E ? {code}
> I think this is due to commit
> [https://github.com/apache/impala/commit/cfe60858da110cf1256bd3aa5d4f8d374578a33d]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-9311) test_show_create_table failed with primary key mismatch
Xiaomeng Zhang created IMPALA-9311:
--
Summary: test_show_create_table failed with primary key mismatch
Key: IMPALA-9311
URL: https://issues.apache.org/jira/browse/IMPALA-9311
Project: IMPALA
Issue Type: Bug
Reporter: Xiaomeng Zhang
Assignee: Anurag Mantripragada
{code:java}
Error Messagemetadata/test_show_create_table.py:62: in test_show_create_table
unique_database) metadata/test_show_create_table.py:110: in
__run_show_create_table_test_case self.__compare_result(expected_result,
create_table_result) metadata/test_show_create_table.py:146: in
__compare_result assert expected_sql_filtered == actual_sql_filtered E
assert "CREATE EXTER...parent_table'" == "CREATE EXTERN...parent_table'" E
Skipping 71 identical leading characters in diff, use -v to show E Skipping
126 identical trailing characters in diff, use -v to show E - MARY KEY
(year, id)) ROW FO E ? E + MARY KEY (id, year)) ROW
FO E ? Stacktracemetadata/test_show_create_table.py:62: in
test_show_create_table
unique_database)
metadata/test_show_create_table.py:110: in __run_show_create_table_test_case
self.__compare_result(expected_result, create_table_result)
metadata/test_show_create_table.py:146: in __compare_result
assert expected_sql_filtered == actual_sql_filtered
E assert "CREATE EXTER...parent_table'" == "CREATE EXTERN...parent_table'"
E Skipping 71 identical leading characters in diff, use -v to show
E Skipping 126 identical trailing characters in diff, use -v to show
E - MARY KEY (year, id)) ROW FO
E ?
E + MARY KEY (id, year)) ROW FO
E ? {code}
I think this is due to commit
[https://github.com/apache/impala/commit/cfe60858da110cf1256bd3aa5d4f8d374578a33d]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8065) OSInfo produces somewhat misleading output when running in container
[ https://issues.apache.org/jira/browse/IMPALA-8065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaomeng Zhang resolved IMPALA-8065. Fix Version/s: Impala 3.4.0 Resolution: Fixed > OSInfo produces somewhat misleading output when running in container > > > Key: IMPALA-8065 > URL: https://issues.apache.org/jira/browse/IMPALA-8065 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 3.1.0 >Reporter: Tim Armstrong >Assignee: Xiaomeng Zhang >Priority: Critical > Fix For: Impala 3.4.0 > > > It uses /proc/version, which returns the host version. It would be good to > also get the version from lsb-release from the Ubuntu container we're running > in and disambiguate on the debug page. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-9201) Impala can't read parquet file compressed by zstd bash command
[
https://issues.apache.org/jira/browse/IMPALA-9201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang updated IMPALA-9201:
---
Description:
To reproduce:
# get a parquet file written by impala
# use "hadoop fs -get" to download locally
# use command "zstd -i parquetfile -o zstdfile" to get a zstd compressed file
parquet.zst.
# use "hadoop fs -put" to put zstd file in directory "/test-warehouse/par_zstd"
# in impala, create table with location on -"/test-warehouse/par_zstd"
# run select * from that table, get error :
{code:java}
[localhost:21000] default> select * from par_zstd;
Query: select * from par_zstd
Query submitted at: 2019-11-25 14:59:07 (Coordinator:
http://xiaomeng-OptiPlex-9020:25000)
Query progress can be monitored at:
http://xiaomeng-OptiPlex-9020:25000/query_plan?query_id=b0411d5136965e30:549208ad
ERROR: File 'hdfs://localhost:20500/test-warehouse/par_zstd/parquet.zst' has an
invalid Parquet version number:
. Please check that it is a valid Parquet file. This error can also occur due
to stale metadata. If you believe this is a valid Parquet file, try running
"refresh default.par_zstd".
{code}
In hive run select * from table, get error:
{code:java}
Error: java.io.IOException: java.lang.RuntimeException:
hdfs://localhost:20500/test-warehouse/par_zstd/parquet.zstd is not a Parquet
file. expected magic number at tail [80, 65, 82, 49] but found [-2, -72, -113,
-90] (state=,code=0)
{code}
was:
To reproduce:
# get a parquet file written by impala
# use "hadoop fs -get" to download locally
# use command "zstd -i parquetfile -o zstdfile" to get a zstd compressed file
parquet.zst.
# use "hadoop fs -put" to put zstd file in directory "/test-warehouse/par_zstd"
# in impala, create table with location on -"/test-warehouse/par_zstd"
# run select * from that table, get error :
{code:java}
[localhost:21000] default> select * from par_zstd;
Query: select * from par_zstd
Query submitted at: 2019-11-25 14:59:07 (Coordinator:
http://xiaomeng-OptiPlex-9020:25000)
Query progress can be monitored at:
http://xiaomeng-OptiPlex-9020:25000/query_plan?query_id=b0411d5136965e30:549208ad
ERROR: File 'hdfs://localhost:20500/test-warehouse/par_zstd/parquet.zst' has an
invalid Parquet version number:
. Please check that it is a valid Parquet file. This error can also occur due
to stale metadata. If you believe this is a valid Parquet file, try running
"refresh default.par_zstd".
{code}
# In hive run select * from table, get error:
{code:java}
Error: java.io.IOException: java.lang.RuntimeException:
hdfs://localhost:20500/test-warehouse/par_zstd/parquet.zstd is not a Parquet
file. expected magic number at tail [80, 65, 82, 49] but found [-2, -72, -113,
-90] (state=,code=0)
{code}
> Impala can't read parquet file compressed by zstd bash command
> --
>
> Key: IMPALA-9201
> URL: https://issues.apache.org/jira/browse/IMPALA-9201
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.4.0
>Reporter: Xiaomeng Zhang
>Assignee: Abhishek Rawat
>Priority: Major
>
> To reproduce:
> # get a parquet file written by impala
> # use "hadoop fs -get" to download locally
> # use command "zstd -i parquetfile -o zstdfile" to get a zstd compressed
> file parquet.zst.
> # use "hadoop fs -put" to put zstd file in directory
> "/test-warehouse/par_zstd"
> # in impala, create table with location on -"/test-warehouse/par_zstd"
> # run select * from that table, get error :
> {code:java}
> [localhost:21000] default> select * from par_zstd;
> Query: select * from par_zstd
> Query submitted at: 2019-11-25 14:59:07 (Coordinator:
> http://xiaomeng-OptiPlex-9020:25000)
> Query progress can be monitored at:
> http://xiaomeng-OptiPlex-9020:25000/query_plan?query_id=b0411d5136965e30:549208ad
> ERROR: File 'hdfs://localhost:20500/test-warehouse/par_zstd/parquet.zst' has
> an invalid Parquet version number:
> . Please check that it is a valid Parquet file. This error can also occur due
> to stale metadata. If you believe this is a valid Parquet file, try running
> "refresh default.par_zstd".
> {code}
> In hive run select * from table, get error:
> {code:java}
> Error: java.io.IOException: java.lang.RuntimeException:
> hdfs://localhost:20500/test-warehouse/par_zstd/parquet.zstd is not a Parquet
> file. expected magic number at tail [80, 65, 82, 49] but found [-2, -72,
> -113, -90] (state=,code=0)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-9201) Impala can't read zstd file compressed by zstd command
Xiaomeng Zhang created IMPALA-9201:
--
Summary: Impala can't read zstd file compressed by zstd command
Key: IMPALA-9201
URL: https://issues.apache.org/jira/browse/IMPALA-9201
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 3.4.0
Reporter: Xiaomeng Zhang
Assignee: Abhishek Rawat
To reproduce:
# get a parquet file written by impala
# use "hadoop fs -get" to download locally
# use command "zstd -i parquetfile -o zstdfile" to get a zstd compressed file
parquet.zst.
# use "hadoop fs -put" to put zstd file in directory "/test-warehouse/par_zstd"
# in impala, create table with location on -"/test-warehouse/par_zstd"
# run select * from that table, get error :
{code:java}
[localhost:21000] default> select * from par_zstd;
Query: select * from par_zstd
Query submitted at: 2019-11-25 14:59:07 (Coordinator:
http://xiaomeng-OptiPlex-9020:25000)
Query progress can be monitored at:
http://xiaomeng-OptiPlex-9020:25000/query_plan?query_id=b0411d5136965e30:549208ad
ERROR: File 'hdfs://localhost:20500/test-warehouse/par_zstd/parquet.zst' has an
invalid Parquet version number:
. Please check that it is a valid Parquet file. This error can also occur due
to stale metadata. If you believe this is a valid Parquet file, try running
"refresh default.par_zstd".
{code}
# In hive run select * from table, get error:
{code:java}
Error: java.io.IOException: java.lang.RuntimeException:
hdfs://localhost:20500/test-warehouse/par_zstd/parquet.zstd is not a Parquet
file. expected magic number at tail [80, 65, 82, 49] but found [-2, -72, -113,
-90] (state=,code=0)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-9090) Scan node in HDFS profile should include name of table being scanned
[ https://issues.apache.org/jira/browse/IMPALA-9090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaomeng Zhang resolved IMPALA-9090. Fix Version/s: Impala 3.4.0 Resolution: Fixed > Scan node in HDFS profile should include name of table being scanned > > > Key: IMPALA-9090 > URL: https://issues.apache.org/jira/browse/IMPALA-9090 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Tim Armstrong >Assignee: Xiaomeng Zhang >Priority: Critical > Fix For: Impala 3.4.0 > > > The only way to figure out the table being scanned by a scan node in the > profile is to pull the string out of the explain plan or execsummary. This is > awkward, both for manual and automated analysis of the profiles. We should > include the table name as a string in the SCAN_NODE implementation. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-9090) Scan node in HDFS profile should include name of table being scanned
[ https://issues.apache.org/jira/browse/IMPALA-9090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-9090 started by Xiaomeng Zhang. -- > Scan node in HDFS profile should include name of table being scanned > > > Key: IMPALA-9090 > URL: https://issues.apache.org/jira/browse/IMPALA-9090 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Tim Armstrong >Assignee: Xiaomeng Zhang >Priority: Critical > > The only way to figure out the table being scanned by a scan node in the > profile is to pull the string out of the explain plan or execsummary. This is > awkward, both for manual and automated analysis of the profiles. We should > include the table name as a string in the SCAN_NODE implementation. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-8065) OSInfo produces somewhat misleading output when running in container
[ https://issues.apache.org/jira/browse/IMPALA-8065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-8065 started by Xiaomeng Zhang. -- > OSInfo produces somewhat misleading output when running in container > > > Key: IMPALA-8065 > URL: https://issues.apache.org/jira/browse/IMPALA-8065 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 3.1.0 >Reporter: Tim Armstrong >Assignee: Xiaomeng Zhang >Priority: Critical > > It uses /proc/version, which returns the host version. It would be good to > also get the version from lsb-release from the Ubuntu container we're running > in and disambiguate on the debug page. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8065) OSInfo produces somewhat misleading output when running in container
[ https://issues.apache.org/jira/browse/IMPALA-8065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaomeng Zhang updated IMPALA-8065: --- Priority: Major (was: Minor) > OSInfo produces somewhat misleading output when running in container > > > Key: IMPALA-8065 > URL: https://issues.apache.org/jira/browse/IMPALA-8065 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 3.1.0 >Reporter: Tim Armstrong >Assignee: Xiaomeng Zhang >Priority: Major > > It uses /proc/version, which returns the host version. It would be good to > also get the version from lsb-release from the Ubuntu container we're running > in and disambiguate on the debug page. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7504) ParseKerberosPrincipal() should use krb5_parse_name() instead
[
https://issues.apache.org/jira/browse/IMPALA-7504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang updated IMPALA-7504:
---
Fix Version/s: Impala 3.4.0
> ParseKerberosPrincipal() should use krb5_parse_name() instead
> -
>
> Key: IMPALA-7504
> URL: https://issues.apache.org/jira/browse/IMPALA-7504
> Project: IMPALA
> Issue Type: Improvement
> Components: Security
>Affects Versions: Impala 3.0, Impala 2.12.0
>Reporter: Michael Ho
>Assignee: Xiaomeng Zhang
>Priority: Minor
> Labels: ramp-up
> Fix For: Impala 3.4.0
>
>
> [~tlipcon] pointed out during code review that we should be using
> krb5_parse_name() to parse the principal instead of creating our own
> bq. I wonder whether we should just be using krb5_parse_name here instead of
> implementing our own parsing? According to
> [http://web.mit.edu/kerberos/krb5-1.15/doc/appdev/refs/api/krb5_parse_name.html]
> there are various escapings, etc, that this function isn't currently
> supporting.
> We currently do the following to parse the principal:
> {noformat}
> vector names;
> split(names, principal, is_any_of("/"));
> if (names.size() != 2) return Status(TErrorCode::BAD_PRINCIPAL_FORMAT,
> principal);
> *service_name = names[0];
> string remaining_principal = names[1];
> split(names, remaining_principal, is_any_of("@"));
> if (names.size() != 2) return Status(TErrorCode::BAD_PRINCIPAL_FORMAT,
> principal);
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-7504) ParseKerberosPrincipal() should use krb5_parse_name() instead
[
https://issues.apache.org/jira/browse/IMPALA-7504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaomeng Zhang resolved IMPALA-7504.
Resolution: Fixed
> ParseKerberosPrincipal() should use krb5_parse_name() instead
> -
>
> Key: IMPALA-7504
> URL: https://issues.apache.org/jira/browse/IMPALA-7504
> Project: IMPALA
> Issue Type: Improvement
> Components: Security
>Affects Versions: Impala 3.0, Impala 2.12.0
>Reporter: Michael Ho
>Assignee: Xiaomeng Zhang
>Priority: Minor
> Labels: ramp-up
>
> [~tlipcon] pointed out during code review that we should be using
> krb5_parse_name() to parse the principal instead of creating our own
> bq. I wonder whether we should just be using krb5_parse_name here instead of
> implementing our own parsing? According to
> [http://web.mit.edu/kerberos/krb5-1.15/doc/appdev/refs/api/krb5_parse_name.html]
> there are various escapings, etc, that this function isn't currently
> supporting.
> We currently do the following to parse the principal:
> {noformat}
> vector names;
> split(names, principal, is_any_of("/"));
> if (names.size() != 2) return Status(TErrorCode::BAD_PRINCIPAL_FORMAT,
> principal);
> *service_name = names[0];
> string remaining_principal = names[1];
> split(names, remaining_principal, is_any_of("@"));
> if (names.size() != 2) return Status(TErrorCode::BAD_PRINCIPAL_FORMAT,
> principal);
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-9090) Scan node in HDFS profile should include name of table being scanned
[
https://issues.apache.org/jira/browse/IMPALA-9090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962526#comment-16962526
]
Xiaomeng Zhang commented on IMPALA-9090:
Hi [~tarmstrong], from query profile I can see table name after "SCAN HDFS"
{code:java}
00:SCAN HDFS [default.customers, RANDOM]
HDFS partitions=1/1 files=1 size=15.44KB
stored statistics:
table: rows=0 size=15.44KB
columns: all
extrapolated-rows=disabled max-scan-range-rows=0
mem-estimate=1.00MB mem-reservation=16.00KB thread-reservation=1
tuple-ids=0 row-size=8B cardinality=0
in pipelines: 00(GETNEXT)
{code}
Is there more info needed?
> Scan node in HDFS profile should include name of table being scanned
>
>
> Key: IMPALA-9090
> URL: https://issues.apache.org/jira/browse/IMPALA-9090
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Reporter: Tim Armstrong
>Assignee: Xiaomeng Zhang
>Priority: Critical
>
> The only way to figure out the table being scanned by a scan node in the
> profile is to pull the string out of the explain plan or execsummary. This is
> awkward, both for manual and automated analysis of the profiles. We should
> include the table name as a string in the SCAN_NODE implementation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8065) OSInfo produces somewhat misleading output when running in container
[
https://issues.apache.org/jira/browse/IMPALA-8065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16956445#comment-16956445
]
Xiaomeng Zhang commented on IMPALA-8065:
Here is what I found
{code:java}
The /etc/lsb-release file is a file that some, but not all, Linux distributions
put there for older programs to use. The "lsb" refers to the Linux Standard
Base, a project working to define a common set of standards for any Linux
distribution to follow, including things like filesystem layout. However, that
file, /etc/lsb-release, isn't part of the standard. It's an extra thing that
some distributions use, but not all.
The /etc/os-release file is the standard, however. Any distribution based on
systemd, including Red Hat Enterprise Linux, CentOS, Fedora, Gentoo, Debian,
Mint, Ubuntu, and many others, is required to have that file. Distributions
that don't use systemd may also have the file.
{code}
I use os-release because in docker, lsb_release command doesn't work.
> OSInfo produces somewhat misleading output when running in container
>
>
> Key: IMPALA-8065
> URL: https://issues.apache.org/jira/browse/IMPALA-8065
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: Tim Armstrong
>Assignee: Xiaomeng Zhang
>Priority: Minor
>
> It uses /proc/version, which returns the host version. It would be good to
> also get the version from lsb-release from the Ubuntu container we're running
> in and disambiguate on the debug page.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7504) ParseKerberosPrincipal() should use krb5_parse_name() instead
[
https://issues.apache.org/jira/browse/IMPALA-7504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16956353#comment-16956353
]
Xiaomeng Zhang commented on IMPALA-7504:
Init krb5_context in auth-utill.cc got jenkins test failure ""Bad SAM flags in
obtain_sam_padata", and it could not reproduced in my local machine.
Discussed with [~kwho], I put that method under kudu/security/init.cc, as it
has initialized krb5_context.
> ParseKerberosPrincipal() should use krb5_parse_name() instead
> -
>
> Key: IMPALA-7504
> URL: https://issues.apache.org/jira/browse/IMPALA-7504
> Project: IMPALA
> Issue Type: Improvement
> Components: Security
>Affects Versions: Impala 3.0, Impala 2.12.0
>Reporter: Michael Ho
>Assignee: Xiaomeng Zhang
>Priority: Minor
> Labels: ramp-up
>
> [~tlipcon] pointed out during code review that we should be using
> krb5_parse_name() to parse the principal instead of creating our own
> bq. I wonder whether we should just be using krb5_parse_name here instead of
> implementing our own parsing? According to
> [http://web.mit.edu/kerberos/krb5-1.15/doc/appdev/refs/api/krb5_parse_name.html]
> there are various escapings, etc, that this function isn't currently
> supporting.
> We currently do the following to parse the principal:
> {noformat}
> vector names;
> split(names, principal, is_any_of("/"));
> if (names.size() != 2) return Status(TErrorCode::BAD_PRINCIPAL_FORMAT,
> principal);
> *service_name = names[0];
> string remaining_principal = names[1];
> split(names, remaining_principal, is_any_of("@"));
> if (names.size() != 2) return Status(TErrorCode::BAD_PRINCIPAL_FORMAT,
> principal);
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-8065) OSInfo produces somewhat misleading output when running in container
[
https://issues.apache.org/jira/browse/IMPALA-8065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16956340#comment-16956340
]
Xiaomeng Zhang edited comment on IMPALA-8065 at 10/21/19 6:02 PM:
--
Hi [~tarmstrong], currently the OS info is like
{code:java}
OS version: Linux version 3.10.0-514.26.2.el7.x86_64
(buil...@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat
4.8.5-11) (GCC) ) #1 SMP Tue Jul 4 15:04:05 UTC 2017
{code}
Do we want to simplify it by using /etc/os-release?
{code:java}
CentOS Linux 7 (Core)
{code}
was (Author: xiaomeng zhang):
Hi [~tarmstrong], currently the OS info is like
{code:java}
OS version: Linux version 3.10.0-514.26.2.el7.x86_64
(buil...@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat
4.8.5-11) (GCC) ) #1 SMP Tue Jul 4 15:04:05 UTC 2017
{code}
Do we want to simplify to by using /etc/os-release?
{code:java}
CentOS Linux 7 (Core)
{code}
> OSInfo produces somewhat misleading output when running in container
>
>
> Key: IMPALA-8065
> URL: https://issues.apache.org/jira/browse/IMPALA-8065
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: Tim Armstrong
>Assignee: Xiaomeng Zhang
>Priority: Minor
>
> It uses /proc/version, which returns the host version. It would be good to
> also get the version from lsb-release from the Ubuntu container we're running
> in and disambiguate on the debug page.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8065) OSInfo produces somewhat misleading output when running in container
[
https://issues.apache.org/jira/browse/IMPALA-8065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16956340#comment-16956340
]
Xiaomeng Zhang commented on IMPALA-8065:
Hi [~tarmstrong], currently the OS info is like
{code:java}
OS version: Linux version 3.10.0-514.26.2.el7.x86_64
(buil...@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat
4.8.5-11) (GCC) ) #1 SMP Tue Jul 4 15:04:05 UTC 2017
{code}
Do we want to simplify to by using /etc/os-release?
{code:java}
CentOS Linux 7 (Core)
{code}
> OSInfo produces somewhat misleading output when running in container
>
>
> Key: IMPALA-8065
> URL: https://issues.apache.org/jira/browse/IMPALA-8065
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: Tim Armstrong
>Assignee: Xiaomeng Zhang
>Priority: Minor
>
> It uses /proc/version, which returns the host version. It would be good to
> also get the version from lsb-release from the Ubuntu container we're running
> in and disambiguate on the debug page.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8124) Build failure: webserver/test_web_pages.py
[
https://issues.apache.org/jira/browse/IMPALA-8124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903360#comment-16903360
]
Xiaomeng Zhang commented on IMPALA-8124:
Hit this issue again in
[webserver.test_web_pages.TestWebPage.test_catalog|https://master-02.jenkins.cloudera.com/job/impala-asf-master-core-data-load/712/testReport/junit/webserver.test_web_pages/TestWebPage/test_catalog/]
> Build failure: webserver/test_web_pages.py
> --
>
> Key: IMPALA-8124
> URL: https://issues.apache.org/jira/browse/IMPALA-8124
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 3.1.0, Impala 3.2.0
>Reporter: Paul Rogers
>Assignee: Anurag Mantripragada
>Priority: Major
> Labels: build-failure, flaky
> Fix For: Impala 3.3.0
>
>
> Build log output:
> {noformat}
> 04:20:18 === FAILURES
> ===
> 04:20:18 ___ TestWebPage.test_catalog
> ___
> 04:20:18 [gw4] linux2 -- Python 2.7.5
> /data/jenkins/workspace/impala-cdh6.x-core/repos/Impala/bin/../infra/python/env/bin/python
> 04:20:18 webserver/test_web_pages.py:265: in test_catalog
> 04:20:18 self.__test_table_metrics("functional", "alltypes",
> "total-file-size-bytes")
> 04:20:18 webserver/test_web_pages.py:283: in __test_table_metrics
> 04:20:18 "?name=%s.%s" % (db_name, tbl_name), metric,
> ports_to_test=self.CATALOG_TEST_PORT)
> 04:20:18 webserver/test_web_pages.py:144: in get_and_check_status
> 04:20:18 assert response.status_code == requests.codes.ok\
> 04:20:18 E AssertionError: Offending url:
> http://localhost:25020/table_metrics?name=functional.alltypes
> 04:20:18 E assert (200 == 200 and 'total-file-size-bytes' in
>
[jira] [Created] (IMPALA-8840) Check failed: num_bytes <= sizeof(T) (5 vs. 4)
Xiaomeng Zhang created IMPALA-8840:
--
Summary: Check failed: num_bytes <= sizeof(T) (5 vs. 4)
Key: IMPALA-8840
URL: https://issues.apache.org/jira/browse/IMPALA-8840
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 3.3.0
Reporter: Xiaomeng Zhang
Assignee: Daniel Becker
Not sure if this is due to same issue as
https://issues.apache.org/jira/browse/IMPALA-8833#, the error message is a
little different.
{code:java}
F0805 18:48:08.737411 5488 bit-stream-utils.inline.h:173]
284731e5d1aad693:05c883020001] Check failed: num_bytes <= sizeof(T) (8 vs.
4)
*** Check failure stack trace: ***
@ 0x52fb9bc google::LogMessage::Fail()
@ 0x52fd261 google::LogMessage::SendToLog()
@ 0x52fb396 google::LogMessage::Flush()
@ 0x52fe95d google::LogMessageFatal::~LogMessageFatal()
@ 0x2b2b867 impala::BatchedBitReader::GetBytes<>()
@ 0x2aeda65 impala::RleBatchDecoder<>::NextCounts()
@ 0x2a82896 impala::RleBatchDecoder<>::NextNumRepeats()
@ 0x2b1927f impala::ScalarColumnReader<>::ReadSlotsNoConversion()
@ 0x2ac7c2c impala::ScalarColumnReader<>::ReadSlots()
@ 0x2a7b861
impala::ScalarColumnReader<>::MaterializeValueBatchRepeatedDefLevel()
@ 0x2a5b3b0 impala::ScalarColumnReader<>::ReadValueBatch<>()
@ 0x2a256a4 impala::ScalarColumnReader<>::ReadNonRepeatedValueBatch()
@ 0x29b6eb6 impala::HdfsParquetScanner::AssembleRows()
@ 0x29b1cf8 impala::HdfsParquetScanner::GetNextInternal()
@ 0x29afc70 impala::HdfsParquetScanner::ProcessSplit()
@ 0x2494bc3 impala::HdfsScanNode::ProcessSplit()
@ 0x2493d98 impala::HdfsScanNode::ScannerThread()
@ 0x2493121
_ZZN6impala12HdfsScanNode22ThreadTokenAvailableCbEPNS_18ThreadResourcePoolEENKUlvE_clEv
@ 0x24956e9
_ZN5boost6detail8function26void_function_obj_invoker0IZN6impala12HdfsScanNode22ThreadTokenAvailableCbEPNS3_18ThreadResourcePoolEEUlvE_vE6invokeERNS1_15function_bufferE
@ 0x1ea0241 boost::function0<>::operator()()
@ 0x23de77a impala::Thread::SuperviseThread()
@ 0x23e6afe boost::_bi::list5<>::operator()<>()
@ 0x23e6a22 boost::_bi::bind_t<>::operator()()
@ 0x23e69e5 boost::detail::thread_data<>::run()
@ 0x4224819 thread_proxy
@ 0x7fc1818c5e24 start_thread
@ 0x7fc17e01f34c __clone
{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8434) Alter Db leads to functions missing unless run "refresh functions"
[
https://issues.apache.org/jira/browse/IMPALA-8434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16842424#comment-16842424
]
Xiaomeng Zhang commented on IMPALA-8434:
The root cause is in
[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java#L395]
{code:java}
private void addDb(TDatabase thriftDb, long catalogVersion) {
Db existingDb = getDb(thriftDb.getDb_name());
if (existingDb == null ||
existingDb.getCatalogVersion() < catalogVersion) {
Db newDb = Db.fromTDatabase(thriftDb);
newDb.setCatalogVersion(catalogVersion);
addDb(newDb);
if (existingDb != null) {
CatalogObjectVersionSet.INSTANCE.updateVersions(
existingDb.getCatalogVersion(), catalogVersion);
CatalogObjectVersionSet.INSTANCE.removeAll(existingDb.getTables());
CatalogObjectVersionSet.INSTANCE.removeAll(
existingDb.getFunctions(null, new PatternMatcher()));
} else {
CatalogObjectVersionSet.INSTANCE.addVersion(catalogVersion);
}
}
}
{code}
AlterDb will trigger method addDb to replace existing Db with thriftDb. But
thriftDb doesn't have table and function inside. So after alterDb, impalad lose
table and function info, require "invalidate metadata".
I tried to fix it by not removing table & function version, adding table &
function back into new created Db. But this change cause "invalidate metadata"
hanging, as "invalidate metadata" also triggers method addDb, to replace
current Db to incomplete one. And it has to remove old table & function version
to let min catalog version increase to catalog's latest update version
[https://github.com/apache/impala/blob/master/be/src/service/impala-server.cc#L1691]
So the best way maybe add table& function only for alterDb command, but from
the passing info, I can't find any parameter can be used as alterDb type
identification. Any help?
> Alter Db leads to functions missing unless run "refresh functions"
> ---
>
> Key: IMPALA-8434
> URL: https://issues.apache.org/jira/browse/IMPALA-8434
> Project: IMPALA
> Issue Type: Bug
>Reporter: Xiaomeng Zhang
>Assignee: Xiaomeng Zhang
>Priority: Critical
>
> I was testing on master branch. In a database with java and native functions.
> Run queries below, all functions are missing after alter db until run
> "refresh functions" in db.
> {code:java}
> [localhost:21000] xm> show functions;
> Query: show functions
> +-++-+---+
> | return type | signature | binary type | is persistent |
> +-++-+---+
> | STRING | add10impala(STRING) | JAVA | true |
> | STRING | add10udf(STRING) | JAVA | true |
> | INT | add2(INT, INT) | NATIVE | true |
> | INT | add2xm(INT, INT) | NATIVE | true |
> | INT | addtwomaster(INT, INT) | NATIVE | true |
> +-++-+---+
> Fetched 5 row(s) in 0.01s
> [localhost:21000] xm> alter database xm set owner user impala218;
> Query: alter database xm set owner user impala218
> +---+
> | summary |
> +---+
> | Updated database. |
> +---+
> Fetched 1 row(s) in 0.59s
> [localhost:21000] xm> show functions;
> Query: show functions
> Fetched 0 row(s) in 0.01s
> [localhost:21000] xm> refresh functions xm;
> Query: refresh functions xm
> Query submitted at: 2019-04-18 14:19:00 (Coordinator:
> http://xiaomeng-OptiPlex-9020:25000)
> Query progress can be monitored at:
> http://xiaomeng-OptiPlex-9020:25000/query_plan?query_id=fa40cdffde223550:df2a6cc0
> Fetched 0 row(s) in 0.08s
> [localhost:21000] xm> show functions;
> Query: show functions
> +-++-+---+
> | return type | signature | binary type | is persistent |
> +-++-+---+
> | STRING | add10impala(STRING) | JAVA | true |
> | STRING | add10udf(STRING) | JAVA | true |
> | INT | add2(INT, INT) | NATIVE | true |
> | INT | add2xm(INT, INT) | NATIVE | true |
> | INT | addtwomaster(INT, INT) | NATIVE | true |
> +-++-+---+
> Fetched 5 row(s) in 0.00s
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8149) Add support for alter_database events
[ https://issues.apache.org/jira/browse/IMPALA-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaomeng Zhang updated IMPALA-8149: --- Affects Version/s: Impala 3.3.0 > Add support for alter_database events > - > > Key: IMPALA-8149 > URL: https://issues.apache.org/jira/browse/IMPALA-8149 > Project: IMPALA > Issue Type: Sub-task >Affects Versions: Impala 3.3.0 >Reporter: Vihang Karajgaonkar >Assignee: Xiaomeng Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8149) Add support for alter_database events
[ https://issues.apache.org/jira/browse/IMPALA-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaomeng Zhang updated IMPALA-8149: --- Fix Version/s: Impala 3.3.0 > Add support for alter_database events > - > > Key: IMPALA-8149 > URL: https://issues.apache.org/jira/browse/IMPALA-8149 > Project: IMPALA > Issue Type: Sub-task >Affects Versions: Impala 3.3.0 >Reporter: Vihang Karajgaonkar >Assignee: Xiaomeng Zhang >Priority: Major > Fix For: Impala 3.3.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8149) Add support for alter_database events
[ https://issues.apache.org/jira/browse/IMPALA-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830565#comment-16830565 ] Xiaomeng Zhang commented on IMPALA-8149: Hi Tim, what version is master branch? How can I find version number in git branch? > Add support for alter_database events > - > > Key: IMPALA-8149 > URL: https://issues.apache.org/jira/browse/IMPALA-8149 > Project: IMPALA > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Xiaomeng Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8149) Add support for alter_database events
[ https://issues.apache.org/jira/browse/IMPALA-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaomeng Zhang resolved IMPALA-8149. Resolution: Fixed > Add support for alter_database events > - > > Key: IMPALA-8149 > URL: https://issues.apache.org/jira/browse/IMPALA-8149 > Project: IMPALA > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Xiaomeng Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-8434) Alter Db leads to functions missing unless run "refresh functions"
[
https://issues.apache.org/jira/browse/IMPALA-8434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821643#comment-16821643
]
Xiaomeng Zhang edited comment on IMPALA-8434 at 4/19/19 3:55 AM:
-
I just run with default configs: $IMPALA_HOME/bin/start-impala-cluster.py.
The hash of my git master is
{code:java}
commit 5c2d211a88c1190a02ce7720fda7b0871f1e5cf5
Author: Bharath Krishna
Date: Wed Apr 17 09:54:13 2019 -0700
IMPALA-8430: Fix flakiness in testCreateDropCreateDatabase
The test fails because of two Databases getting created with
same CREATION_TIME. Hence, adding a sleep of 2 seconds to
avoid this case. Also fixing other tests with similar use-case.
Testing
- Fixed the unit tests
Change-Id: I30bf4535d54c9cd8d257b528dc7a1b42f106800d
Reviewed-on: http://gerrit.cloudera.org:8080/13058
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
{code}
was (Author: xiaomeng zhang):
I just run with default configs: $IMPALA_HOME/bin/start-impala-cluster.py.
The hash of my git master is
commit 5c2d211a88c1190a02ce7720fda7b0871f1e5cf5
> Alter Db leads to functions missing unless run "refresh functions"
> ---
>
> Key: IMPALA-8434
> URL: https://issues.apache.org/jira/browse/IMPALA-8434
> Project: IMPALA
> Issue Type: Bug
>Reporter: Xiaomeng Zhang
>Assignee: Xiaomeng Zhang
>Priority: Critical
>
> I was testing on master branch. In a database with java and native functions.
> Run queries below, all functions are missing after alter db until run
> "refresh functions" in db.
> {code:java}
> [localhost:21000] xm> show functions;
> Query: show functions
> +-++-+---+
> | return type | signature | binary type | is persistent |
> +-++-+---+
> | STRING | add10impala(STRING) | JAVA | true |
> | STRING | add10udf(STRING) | JAVA | true |
> | INT | add2(INT, INT) | NATIVE | true |
> | INT | add2xm(INT, INT) | NATIVE | true |
> | INT | addtwomaster(INT, INT) | NATIVE | true |
> +-++-+---+
> Fetched 5 row(s) in 0.01s
> [localhost:21000] xm> alter database xm set owner user impala218;
> Query: alter database xm set owner user impala218
> +---+
> | summary |
> +---+
> | Updated database. |
> +---+
> Fetched 1 row(s) in 0.59s
> [localhost:21000] xm> show functions;
> Query: show functions
> Fetched 0 row(s) in 0.01s
> [localhost:21000] xm> refresh functions xm;
> Query: refresh functions xm
> Query submitted at: 2019-04-18 14:19:00 (Coordinator:
> http://xiaomeng-OptiPlex-9020:25000)
> Query progress can be monitored at:
> http://xiaomeng-OptiPlex-9020:25000/query_plan?query_id=fa40cdffde223550:df2a6cc0
> Fetched 0 row(s) in 0.08s
> [localhost:21000] xm> show functions;
> Query: show functions
> +-++-+---+
> | return type | signature | binary type | is persistent |
> +-++-+---+
> | STRING | add10impala(STRING) | JAVA | true |
> | STRING | add10udf(STRING) | JAVA | true |
> | INT | add2(INT, INT) | NATIVE | true |
> | INT | add2xm(INT, INT) | NATIVE | true |
> | INT | addtwomaster(INT, INT) | NATIVE | true |
> +-++-+---+
> Fetched 5 row(s) in 0.00s
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-8434) Alter Db leads to functions missing unless run "refresh functions"
[
https://issues.apache.org/jira/browse/IMPALA-8434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821643#comment-16821643
]
Xiaomeng Zhang edited comment on IMPALA-8434 at 4/19/19 3:54 AM:
-
I just run with default configs: $IMPALA_HOME/bin/start-impala-cluster.py.
The hash of my git master is
commit 5c2d211a88c1190a02ce7720fda7b0871f1e5cf5
was (Author: xiaomeng zhang):
I just run with default configs: $IMPALA_HOME/bin/start-impala-cluster.py
> Alter Db leads to functions missing unless run "refresh functions"
> ---
>
> Key: IMPALA-8434
> URL: https://issues.apache.org/jira/browse/IMPALA-8434
> Project: IMPALA
> Issue Type: Bug
>Reporter: Xiaomeng Zhang
>Assignee: Xiaomeng Zhang
>Priority: Critical
>
> I was testing on master branch. In a database with java and native functions.
> Run queries below, all functions are missing after alter db until run
> "refresh functions" in db.
> {code:java}
> [localhost:21000] xm> show functions;
> Query: show functions
> +-++-+---+
> | return type | signature | binary type | is persistent |
> +-++-+---+
> | STRING | add10impala(STRING) | JAVA | true |
> | STRING | add10udf(STRING) | JAVA | true |
> | INT | add2(INT, INT) | NATIVE | true |
> | INT | add2xm(INT, INT) | NATIVE | true |
> | INT | addtwomaster(INT, INT) | NATIVE | true |
> +-++-+---+
> Fetched 5 row(s) in 0.01s
> [localhost:21000] xm> alter database xm set owner user impala218;
> Query: alter database xm set owner user impala218
> +---+
> | summary |
> +---+
> | Updated database. |
> +---+
> Fetched 1 row(s) in 0.59s
> [localhost:21000] xm> show functions;
> Query: show functions
> Fetched 0 row(s) in 0.01s
> [localhost:21000] xm> refresh functions xm;
> Query: refresh functions xm
> Query submitted at: 2019-04-18 14:19:00 (Coordinator:
> http://xiaomeng-OptiPlex-9020:25000)
> Query progress can be monitored at:
> http://xiaomeng-OptiPlex-9020:25000/query_plan?query_id=fa40cdffde223550:df2a6cc0
> Fetched 0 row(s) in 0.08s
> [localhost:21000] xm> show functions;
> Query: show functions
> +-++-+---+
> | return type | signature | binary type | is persistent |
> +-++-+---+
> | STRING | add10impala(STRING) | JAVA | true |
> | STRING | add10udf(STRING) | JAVA | true |
> | INT | add2(INT, INT) | NATIVE | true |
> | INT | add2xm(INT, INT) | NATIVE | true |
> | INT | addtwomaster(INT, INT) | NATIVE | true |
> +-++-+---+
> Fetched 5 row(s) in 0.00s
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8434) Alter Db leads to functions missing unless run "refresh functions"
[
https://issues.apache.org/jira/browse/IMPALA-8434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821643#comment-16821643
]
Xiaomeng Zhang commented on IMPALA-8434:
I just run with default configs: $IMPALA_HOME/bin/start-impala-cluster.py
> Alter Db leads to functions missing unless run "refresh functions"
> ---
>
> Key: IMPALA-8434
> URL: https://issues.apache.org/jira/browse/IMPALA-8434
> Project: IMPALA
> Issue Type: Bug
>Reporter: Xiaomeng Zhang
>Assignee: Xiaomeng Zhang
>Priority: Critical
>
> I was testing on master branch. In a database with java and native functions.
> Run queries below, all functions are missing after alter db until run
> "refresh functions" in db.
> {code:java}
> [localhost:21000] xm> show functions;
> Query: show functions
> +-++-+---+
> | return type | signature | binary type | is persistent |
> +-++-+---+
> | STRING | add10impala(STRING) | JAVA | true |
> | STRING | add10udf(STRING) | JAVA | true |
> | INT | add2(INT, INT) | NATIVE | true |
> | INT | add2xm(INT, INT) | NATIVE | true |
> | INT | addtwomaster(INT, INT) | NATIVE | true |
> +-++-+---+
> Fetched 5 row(s) in 0.01s
> [localhost:21000] xm> alter database xm set owner user impala218;
> Query: alter database xm set owner user impala218
> +---+
> | summary |
> +---+
> | Updated database. |
> +---+
> Fetched 1 row(s) in 0.59s
> [localhost:21000] xm> show functions;
> Query: show functions
> Fetched 0 row(s) in 0.01s
> [localhost:21000] xm> refresh functions xm;
> Query: refresh functions xm
> Query submitted at: 2019-04-18 14:19:00 (Coordinator:
> http://xiaomeng-OptiPlex-9020:25000)
> Query progress can be monitored at:
> http://xiaomeng-OptiPlex-9020:25000/query_plan?query_id=fa40cdffde223550:df2a6cc0
> Fetched 0 row(s) in 0.08s
> [localhost:21000] xm> show functions;
> Query: show functions
> +-++-+---+
> | return type | signature | binary type | is persistent |
> +-++-+---+
> | STRING | add10impala(STRING) | JAVA | true |
> | STRING | add10udf(STRING) | JAVA | true |
> | INT | add2(INT, INT) | NATIVE | true |
> | INT | add2xm(INT, INT) | NATIVE | true |
> | INT | addtwomaster(INT, INT) | NATIVE | true |
> +-++-+---+
> Fetched 5 row(s) in 0.00s
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8434) Alter Db leads to functions missing unless run "refresh functions"
[
https://issues.apache.org/jira/browse/IMPALA-8434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821522#comment-16821522
]
Xiaomeng Zhang commented on IMPALA-8434:
More findings. It looks like after alter db, not only lose functions, tables
under that db also get lost.
{code:java}
[localhost:21000] xm> show tables;
Query: show tables
+--+
| name |
+--+
| xm |
| xms |
| xmt |
+--+
Fetched 3 row(s) in 0.01s
[localhost:21000] xm> alter database xm set owner user impala258;
Query: alter database xm set owner user impala258
+---+
| summary |
+---+
| Updated database. |
+---+
Fetched 1 row(s) in 0.51s
[localhost:21000] xm> show tables;
Query: show tables
Fetched 0 row(s) in 0.01s
[localhost:21000] xm> invalidate metadata;
Query: invalidate metadata
Query submitted at: 2019-04-18 15:00:40 (Coordinator:
http://xiaomeng-OptiPlex-9020:25000)
Query progress can be monitored at:
http://xiaomeng-OptiPlex-9020:25000/query_plan?query_id=404c09dd94296d20:b994e9fa
Fetched 0 row(s) in 3.71s
[localhost:21000] xm> show tables;
Query: show tables
+--+
| name |
+--+
| xm |
| xms |
| xmt |
+--+
Fetched 3 row(s) in 0.01s
{code}
> Alter Db leads to functions missing unless run "refresh functions"
> ---
>
> Key: IMPALA-8434
> URL: https://issues.apache.org/jira/browse/IMPALA-8434
> Project: IMPALA
> Issue Type: Bug
>Reporter: Xiaomeng Zhang
>Assignee: Xiaomeng Zhang
>Priority: Critical
>
> I was testing on master branch. In a database with java and native functions.
> Run queries below, all functions are missing after alter db until run
> "refresh functions" in db.
> {code:java}
> [localhost:21000] xm> show functions;
> Query: show functions
> +-++-+---+
> | return type | signature | binary type | is persistent |
> +-++-+---+
> | STRING | add10impala(STRING) | JAVA | true |
> | STRING | add10udf(STRING) | JAVA | true |
> | INT | add2(INT, INT) | NATIVE | true |
> | INT | add2xm(INT, INT) | NATIVE | true |
> | INT | addtwomaster(INT, INT) | NATIVE | true |
> +-++-+---+
> Fetched 5 row(s) in 0.01s
> [localhost:21000] xm> alter database xm set owner user impala218;
> Query: alter database xm set owner user impala218
> +---+
> | summary |
> +---+
> | Updated database. |
> +---+
> Fetched 1 row(s) in 0.59s
> [localhost:21000] xm> show functions;
> Query: show functions
> Fetched 0 row(s) in 0.01s
> [localhost:21000] xm> refresh functions xm;
> Query: refresh functions xm
> Query submitted at: 2019-04-18 14:19:00 (Coordinator:
> http://xiaomeng-OptiPlex-9020:25000)
> Query progress can be monitored at:
> http://xiaomeng-OptiPlex-9020:25000/query_plan?query_id=fa40cdffde223550:df2a6cc0
> Fetched 0 row(s) in 0.08s
> [localhost:21000] xm> show functions;
> Query: show functions
> +-++-+---+
> | return type | signature | binary type | is persistent |
> +-++-+---+
> | STRING | add10impala(STRING) | JAVA | true |
> | STRING | add10udf(STRING) | JAVA | true |
> | INT | add2(INT, INT) | NATIVE | true |
> | INT | add2xm(INT, INT) | NATIVE | true |
> | INT | addtwomaster(INT, INT) | NATIVE | true |
> +-++-+---+
> Fetched 5 row(s) in 0.00s
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8434) Alter Db leads to functions missing unless run "refresh functions"
Xiaomeng Zhang created IMPALA-8434:
--
Summary: Alter Db leads to functions missing unless run "refresh
functions"
Key: IMPALA-8434
URL: https://issues.apache.org/jira/browse/IMPALA-8434
Project: IMPALA
Issue Type: Bug
Reporter: Xiaomeng Zhang
Assignee: Xiaomeng Zhang
I was testing on master branch. In a database with java and native functions.
Run queries below, all functions are missing after alter db until run "refresh
functions" in db.
{code:java}
[localhost:21000] xm> show functions;
Query: show functions
+-++-+---+
| return type | signature | binary type | is persistent |
+-++-+---+
| STRING | add10impala(STRING) | JAVA | true |
| STRING | add10udf(STRING) | JAVA | true |
| INT | add2(INT, INT) | NATIVE | true |
| INT | add2xm(INT, INT) | NATIVE | true |
| INT | addtwomaster(INT, INT) | NATIVE | true |
+-++-+---+
Fetched 5 row(s) in 0.01s
[localhost:21000] xm> alter database xm set owner user impala218;
Query: alter database xm set owner user impala218
+---+
| summary |
+---+
| Updated database. |
+---+
Fetched 1 row(s) in 0.59s
[localhost:21000] xm> show functions;
Query: show functions
Fetched 0 row(s) in 0.01s
[localhost:21000] xm> refresh functions xm;
Query: refresh functions xm
Query submitted at: 2019-04-18 14:19:00 (Coordinator:
http://xiaomeng-OptiPlex-9020:25000)
Query progress can be monitored at:
http://xiaomeng-OptiPlex-9020:25000/query_plan?query_id=fa40cdffde223550:df2a6cc0
Fetched 0 row(s) in 0.08s
[localhost:21000] xm> show functions;
Query: show functions
+-++-+---+
| return type | signature | binary type | is persistent |
+-++-+---+
| STRING | add10impala(STRING) | JAVA | true |
| STRING | add10udf(STRING) | JAVA | true |
| INT | add2(INT, INT) | NATIVE | true |
| INT | add2xm(INT, INT) | NATIVE | true |
| INT | addtwomaster(INT, INT) | NATIVE | true |
+-++-+---+
Fetched 5 row(s) in 0.00s
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-8149) Add support for alter_database events
[ https://issues.apache.org/jira/browse/IMPALA-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaomeng Zhang reassigned IMPALA-8149: -- Assignee: Xiaomeng Zhang (was: Vihang Karajgaonkar) > Add support for alter_database events > - > > Key: IMPALA-8149 > URL: https://issues.apache.org/jira/browse/IMPALA-8149 > Project: IMPALA > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Xiaomeng Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
