[jira] [Created] (IMPALA-8385) Refactor Sentry admin check
Fredy Wijaya created IMPALA-8385: Summary: Refactor Sentry admin check Key: IMPALA-8385 URL: https://issues.apache.org/jira/browse/IMPALA-8385 Project: IMPALA Issue Type: Sub-task Components: Catalog, Frontend Reporter: Fredy Wijaya Currently Sentry admin check is hardcoded, for example: https://github.com/apache/impala/blob/5670f96b828d57f9e36510bb9af02bcc31de775c/be/src/service/client-request-state.cc#L366 This check needs to be moved out to SentryAuthorizationManager instead. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8368) Create database/table with Ranger throws UnsupportedOperationException
[ https://issues.apache.org/jira/browse/IMPALA-8368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-8368: - Component/s: Catalog > Create database/table with Ranger throws UnsupportedOperationException > -- > > Key: IMPALA-8368 > URL: https://issues.apache.org/jira/browse/IMPALA-8368 > Project: IMPALA > Issue Type: Sub-task > Components: Catalog, Frontend >Reporter: Austin Nobis >Assignee: Austin Nobis >Priority: Major > Fix For: Impala 3.3.0 > > > When executing *create database ;* in Impala with Ranger enabled, > an *UnsupportedOperationException* will be thrown. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-7982) Add network I/O throughput to query profiles
[ https://issues.apache.org/jira/browse/IMPALA-7982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Volker resolved IMPALA-7982. - Resolution: Fixed Fix Version/s: Impala 3.3.0 > Add network I/O throughput to query profiles > > > Key: IMPALA-7982 > URL: https://issues.apache.org/jira/browse/IMPALA-7982 > Project: IMPALA > Issue Type: New Feature > Components: Backend >Affects Versions: Impala 3.2.0 >Reporter: Lars Volker >Assignee: Lars Volker >Priority: Major > Labels: observability, supportability > Fix For: Impala 3.3.0 > > > IMPALA-7694 added a framework to collect system resource usage during query > execution and aggregate it at the coordinator. We should add network I/O > throughput there, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8381) Remove branch from ParquetPlainEncoder::Decode()
[ https://issues.apache.org/jira/browse/IMPALA-8381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Volker updated IMPALA-8381: Labels: newbie parquet performance ramp-up (was: newbie parquet performance) > Remove branch from ParquetPlainEncoder::Decode() > > > Key: IMPALA-8381 > URL: https://issues.apache.org/jira/browse/IMPALA-8381 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Csaba Ringhofer >Priority: Minor > Labels: newbie, parquet, performance, ramp-up > > Removing the "if" at > https://github.com/apache/impala/blob/5670f96b828d57f9e36510bb9af02bcc31de775c/be/src/exec/parquet/parquet-common.h#L203 > can lead to 1.5x speed up in plain decoding (type=int32, stride=16). For > primitive types, the same check can be done for a whole batch, so the speedup > can be gained for large batches without loosing safety. The only Parquet type > where this check is needed per element is BYTE_ARRAY (typically used for > STRING columns), which already has a template specialization for > ParquetPlainEncoder::Decode(). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7995) Run end-to-end tests against docker in minicluster
[ https://issues.apache.org/jira/browse/IMPALA-7995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809382#comment-16809382 ] David Knupp commented on IMPALA-7995: - Ah, thanks for clarifying. > Run end-to-end tests against docker in minicluster > -- > > Key: IMPALA-7995 > URL: https://issues.apache.org/jira/browse/IMPALA-7995 > Project: IMPALA > Issue Type: Sub-task > Components: Infrastructure >Reporter: Tim Armstrong >Assignee: Tim Armstrong >Priority: Major > Labels: docker > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7995) Run end-to-end tests against docker in minicluster
[ https://issues.apache.org/jira/browse/IMPALA-7995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809381#comment-16809381 ] Tim Armstrong commented on IMPALA-7995: --- Oh I think my comment was referring to ./fe/src/test/java/org/apache/impala/service/JdbcTest.java which uses the Hive JDBC driver. > Run end-to-end tests against docker in minicluster > -- > > Key: IMPALA-7995 > URL: https://issues.apache.org/jira/browse/IMPALA-7995 > Project: IMPALA > Issue Type: Sub-task > Components: Infrastructure >Reporter: Tim Armstrong >Assignee: Tim Armstrong >Priority: Major > Labels: docker > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7995) Run end-to-end tests against docker in minicluster
[ https://issues.apache.org/jira/browse/IMPALA-7995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809379#comment-16809379 ] David Knupp commented on IMPALA-7995: - [~tarmstrong] -- do you know what version of the JDBC driver we're using? > Run end-to-end tests against docker in minicluster > -- > > Key: IMPALA-7995 > URL: https://issues.apache.org/jira/browse/IMPALA-7995 > Project: IMPALA > Issue Type: Sub-task > Components: Infrastructure >Reporter: Tim Armstrong >Assignee: Tim Armstrong >Priority: Major > Labels: docker > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8384) Insert ACL tests fail on dockerised cluster
Tim Armstrong created IMPALA-8384:
-
Summary: Insert ACL tests fail on dockerised cluster
Key: IMPALA-8384
URL: https://issues.apache.org/jira/browse/IMPALA-8384
Project: IMPALA
Issue Type: Sub-task
Components: Infrastructure
Reporter: Tim Armstrong
{noformat}
$ TEST_START_CLUSTER_ARGS="--docker_network=impala-cluster" impala-py.test
tests/query_test/test_insert_behaviour.py -k acl
...
tests/query_test/test_insert_behaviour.py::TestInsertBehaviour::test_insert_inherit_acls
xfail
tests/query_test/test_insert_behaviour.py::TestInsertBehaviour::test_insert_acl_permissions
FAILED
tests/query_test/test_insert_behaviour.py::TestInsertBehaviour::test_multiple_group_acls
FAILED
{noformat}
{noformat}
_
TestInsertBehaviour.test_insert_acl_permissions
__
tests/query_test/test_insert_behaviour.py:410: in test_insert_acl_permissions
self.execute_query_expect_failure(self.client, insert_query)
tests/common/impala_test_suite.py:607: in wrapper
return function(*args, **kwargs)
tests/common/impala_test_suite.py:629: in execute_query_expect_failure
assert not result.success, "No failure encountered for query %s" % query
E AssertionError: No failure encountered for query INSERT INTO
`test_insert_acl_permissions_4941df88`.`insert_acl_permissions` VALUES(1)
--
Captured stderr setup
---
SET
client_identifier=query_test/test_insert_behaviour.py::TestInsertBehaviour::()::test_insert_acl_permissions;
-- connecting to: localhost:21000
-- connecting to localhost:21050 with impyla
Conn
-- 2019-04-03 16:21:43,525 INFO MainThread: Closing active operation
SET
client_identifier=query_test/test_insert_behaviour.py::TestInsertBehaviour::()::test_insert_acl_permissions;
SET sync_ddl=False;
-- executing against localhost:21000
DROP DATABASE IF EXISTS `test_insert_acl_permissions_4941df88` CASCADE;
-- 2019-04-03 16:21:43,531 INFO MainThread: Started query
0847d4339b358537:c1c6ad23
SET
client_identifier=query_test/test_insert_behaviour.py::TestInsertBehaviour::()::test_insert_acl_permissions;
SET sync_ddl=False;
-- executing against localhost:21000
CREATE DATABASE `test_insert_acl_permissions_4941df88`;
-- 2019-04-03 16:21:43,958 INFO MainThread: Started query
694436ba3cf75303:287b5135
-- 2019-04-03 16:21:43,966 INFO MainThread: Created database
"test_insert_acl_permissions_4941df88" for test ID
"query_test/test_insert_behaviour.py::TestInsertBehaviour::()::test_insert_acl_permissions"
---
Captured stderr call
---
-- executing against localhost:21000
DROP TABLE IF EXISTS
`test_insert_acl_permissions_4941df88`.`insert_acl_permissions`;
-- 2019-04-03 16:21:43,977 INFO MainThread: Started query
ac48b16897d5b622:d96d3999
-- executing against localhost:21000
CREATE TABLE `test_insert_acl_permissions_4941df88`.`insert_acl_permissions`
(col int);
-- 2019-04-03 16:21:44,454 INFO MainThread: Started query
e741c32bf0fbf1f5:84e69d42
-- executing against localhost:21000
INSERT INTO `test_insert_acl_permissions_4941df88`.`insert_acl_permissions`
VALUES(1);
-- 2019-04-03 16:21:47,252 INFO MainThread: Started query
cb44c5c675b81864:3dfa1a50
-- 2019-04-03 16:21:47,371 INFO MainThread: Starting new HTTP connection
(1): 0.0.0.0
-- 2019-04-03 16:21:47,381 INFO MainThread: Starting new HTTP connection
(1): 0.0.0.0
-- executing against localhost:21000
REFRESH `test_insert_acl_permissions_4941df88`.`insert_acl_permissions`;
-- 2019-04-03 16:21:47,482 INFO MainThread: Started query
b049107c5b287ed1:636e8735
-- executing against localhost:21000
INSERT INTO `test_insert_acl_permissions_4941df88`.`insert_acl_permissions`
VALUES(1);
-- 2019-04-03 16:21:47,625 INFO MainThread: Starting new HTTP connection
(1): 0.0.0.0
-- executing against localhost:21000
REFRESH `test_insert_acl_permissions_4941df88`.`insert_acl_permissions`;
-- 2019-04-03 16:21:47,648 INFO MainThread: Started query
134a91aafee52d0c:773f3e7f
-- executing against localhost:21000
INSERT INTO `test_insert_acl_permissions_4941df88`.`insert_acl_permissions`
VALUES(1);
-- 2019-04-03 16:21:47,653 INFO MainThread: Started query
2544546bc391abcb:bd07bba5
-- 2019-04-03 16:21:47,709 INFO MainThread: Starting new HTTP connection
(1): 0.0.0.0
-- executing against localhost:21000
REFRESH
[jira] [Assigned] (IMPALA-8380) The minicluster metastore doesn't start on systems with postgres 10
[
https://issues.apache.org/jira/browse/IMPALA-8380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Volker reassigned IMPALA-8380:
---
Assignee: Lars Volker
> The minicluster metastore doesn't start on systems with postgres 10
> ---
>
> Key: IMPALA-8380
> URL: https://issues.apache.org/jira/browse/IMPALA-8380
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.3.0
>Reporter: Lars Volker
>Assignee: Lars Volker
>Priority: Major
>
> We currently hardcode the postgres client version in at least 2 places:
> {noformat}
> $ git grep 9.0-801
> bin/impala-config.sh:129:export IMPALA_POSTGRES_JDBC_DRIVER_VERSION=9.0-801
> fe/pom.xml:425: 9.0-801.jdbc4
> {noformat}
> Apparently version 9 of the JDBC driver doesn't support connecting to
> postgres 10 ([Some info
> here|https://github.com/brettwooldridge/HikariCP/issues/1103#issuecomment-369767801]).
> We should upgrade to v42+.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-7293) Show tuple layout in explain plan
[
https://issues.apache.org/jira/browse/IMPALA-7293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong reassigned IMPALA-7293:
-
Assignee: Abhishek Rawat
> Show tuple layout in explain plan
> -
>
> Key: IMPALA-7293
> URL: https://issues.apache.org/jira/browse/IMPALA-7293
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
>Reporter: Tim Armstrong
>Assignee: Abhishek Rawat
>Priority: Major
> Labels: observability
>
> There's currently no way to tell in the explain plan what the contents of
> each tuple are. At explain_level>=2 we include "tuple-ids" but no information
> about what is actually in the tuples.
> {noformat}
> [localhost:21000] default> explain select min(regexp_replace(l_comment, ".",
> "x"))
> from tpch.lineitem; summary;
> Query: explain select min(regexp_replace(l_comment, ".", "x"))
> from tpch.lineitem
> +---+
> | Explain String
> |
> +---+
> | Max Per-Host Resource Reservation: Memory=8.00MB Threads=3
> |
> | Per-Host Resource Estimates: Memory=284.00MB
> |
> |
> |
> | F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> |
> | | Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B
> thread-reservation=1 |
> | PLAN-ROOT SINK
> |
> | | mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> | |
> |
> | 03:AGGREGATE [FINALIZE]
> |
> | | output: min:merge(regexp_replace(l_comment, '.', 'x'))
> |
> | | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
> thread-reservation=0 |
> | | tuple-ids=1 row-size=16B cardinality=1
> |
> | |
> |
> | 02:EXCHANGE [UNPARTITIONED]
> |
> | | mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> | | tuple-ids=1 row-size=16B cardinality=1
> |
> | |
> |
> | F00:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
> |
> | Per-Host Resources: mem-estimate=274.00MB mem-reservation=8.00MB
> thread-reservation=2 |
> | 01:AGGREGATE
> |
> | | output: min(regexp_replace(l_comment, '.', 'x'))
> |
> | | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
> thread-reservation=0 |
> | | tuple-ids=1 row-size=16B cardinality=1
> |
> | |
> |
> | 00:SCAN HDFS [tpch.lineitem, RANDOM]
> |
> |partitions=1/1 files=1 size=718.94MB
> |
> |stored statistics:
> |
> | table: rows=6001215 size=718.94MB
> |
> | columns: all
> |
> |extrapolated-rows=disabled max-scan-range-rows=1068457
> |
> |mem-estimate=264.00MB mem-reservation=8.00MB thread-reservation=1
> |
> |tuple-ids=0 row-size=42B cardinality=6001215
> |
> +---+
> Fetched 32 row(s) in 0.01s
> Summary not available
> {noformat}
> We already have a debugString() methods that prints a human-readable
> representation. We could start off by printing a tuple descriptor per line at
> the end of the explain plan with basic information. I think we should only
> print materialised slots and print them in order of offset, e.g. for "select
> l_comment from tpch.lineitem where l_orderkey < 10" we would print something
> like this at the end of the explain plan
> {noformat}
[jira] [Assigned] (IMPALA-4658) Potential race if compiler reorders ReachedLimit() usage
[
https://issues.apache.org/jira/browse/IMPALA-4658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong reassigned IMPALA-4658:
-
Assignee: Abhishek Rawat
> Potential race if compiler reorders ReachedLimit() usage
>
>
> Key: IMPALA-4658
> URL: https://issues.apache.org/jira/browse/IMPALA-4658
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 2.8.0
>Reporter: Matthew Jacobs
>Assignee: Abhishek Rawat
>Priority: Major
>
> There is a potential race in our many usages of {{ExecNode::ReachedLimit()}}
> during execution from a different thread than the one updating rows_returned,
> e.g. scanner threads. The compiler in theory is free to notice that they're
> not updating {{rows_returned_}} and hoist the load up out of loops, etc.
> We should consider AtomicInt64 or NoBarrier_Load(...).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-4658) Potential race if compiler reorders ReachedLimit() usage
[
https://issues.apache.org/jira/browse/IMPALA-4658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong updated IMPALA-4658:
--
Description:
There is a potential race in our many usages of {{ExecNode::ReachedLimit()}}
during execution from a different thread than the one updating rows_returned,
e.g. scanner threads. The compiler in theory is free to notice that they're not
updating {{rows_returned_}} and hoist the load up out of loops, etc.
We should consider AtomicInt64 or NoBarrier_Load(...).
was:
There is a potential race in our many usages of {{ExecNode::ReachedLimit()}}
during execution from a different thread than the one updating rows_returned,
e.g. scanner threads. The compiler in theory is free to notice that they're not
updating {{rows_returned_}} and hoist the load up out of loops, etc.
We should consider atomics or NoBarrier_Load(...).
> Potential race if compiler reorders ReachedLimit() usage
>
>
> Key: IMPALA-4658
> URL: https://issues.apache.org/jira/browse/IMPALA-4658
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 2.8.0
>Reporter: Matthew Jacobs
>Priority: Major
>
> There is a potential race in our many usages of {{ExecNode::ReachedLimit()}}
> during execution from a different thread than the one updating rows_returned,
> e.g. scanner threads. The compiler in theory is free to notice that they're
> not updating {{rows_returned_}} and hoist the load up out of loops, etc.
> We should consider AtomicInt64 or NoBarrier_Load(...).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-4658) Potential race if compiler reorders ReachedLimit() usage
[
https://issues.apache.org/jira/browse/IMPALA-4658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong updated IMPALA-4658:
--
Description:
There is a potential race in our many usages of {{ExecNode::ReachedLimit()}}
during execution from a different thread than the one updating rows_returned,
e.g. scanner threads. The compiler in theory is free to notice that they're not
updating {{rows_returned_}} and hoist the load up out of loops, etc.
We should consider atomics or NoBarrier_Load(...).
was:
There is a potential race in our many usages of {{ReachedLimit()}} during
execution from a different thread than the one updating rows_returned, e.g.
scanner threads. The compiler in theory is free to notice that they're not
updating {{rows_returned_}} and hoist the load up out of loops, etc.
We should consider atomics or NoBarrier_Load(...).
> Potential race if compiler reorders ReachedLimit() usage
>
>
> Key: IMPALA-4658
> URL: https://issues.apache.org/jira/browse/IMPALA-4658
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 2.8.0
>Reporter: Matthew Jacobs
>Priority: Major
>
> There is a potential race in our many usages of {{ExecNode::ReachedLimit()}}
> during execution from a different thread than the one updating rows_returned,
> e.g. scanner threads. The compiler in theory is free to notice that they're
> not updating {{rows_returned_}} and hoist the load up out of loops, etc.
> We should consider atomics or NoBarrier_Load(...).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-1856) Missing datatypes from JDBC DBMD.getTypeInfo() call
[ https://issues.apache.org/jira/browse/IMPALA-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808991#comment-16808991 ] Tim Armstrong commented on IMPALA-1856: --- I think we might be doing some more work around the HS2 endpoint so this would be a good introduction to the thrift interfaces and APIs. > Missing datatypes from JDBC DBMD.getTypeInfo() call > --- > > Key: IMPALA-1856 > URL: https://issues.apache.org/jira/browse/IMPALA-1856 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 2.1 > Environment: Impala 2.1 > C5.3 > hive-jdbc 0.13 >Reporter: Jonathan Seidman >Assignee: Abhishek Rawat >Priority: Minor > Labels: jdbc, odbc, ramp-up, supportability > > Filing this here since it seems to be an Impala defect as opposed to a driver > defect based on observed behavior. The following is seen from > DBMD.getTypeInfo() call against Hive: > VOID > BOOLEAN > TINYINT > SMALLINT > INT > BIGINT > FLOAT > DOUBLE > STRING > CHAR > VARCHAR > DATE > TIMESTAMP > BINARY > DECIMAL > ARRAY > MAP > STRUCT > UNIONTYPE > USER_DEFINED > And against Impala (2.1): > NULL_TYPE > BOOLEAN > TINYINT > SMALLINT > INT > BIGINT > FLOAT > DOUBLE > TIMESTAMP > STRING > BINARY > Note the missing newer datatypes (e.g. decimal, varchar). > This is with the Hive 0.13 driver. Please let me know if there's something > I'm missing and this should be filed against the driver. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-1856) Missing datatypes from JDBC DBMD.getTypeInfo() call
[ https://issues.apache.org/jira/browse/IMPALA-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong reassigned IMPALA-1856: - Assignee: Abhishek Rawat > Missing datatypes from JDBC DBMD.getTypeInfo() call > --- > > Key: IMPALA-1856 > URL: https://issues.apache.org/jira/browse/IMPALA-1856 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 2.1 > Environment: Impala 2.1 > C5.3 > hive-jdbc 0.13 >Reporter: Jonathan Seidman >Assignee: Abhishek Rawat >Priority: Minor > Labels: jdbc, odbc, ramp-up, supportability > > Filing this here since it seems to be an Impala defect as opposed to a driver > defect based on observed behavior. The following is seen from > DBMD.getTypeInfo() call against Hive: > VOID > BOOLEAN > TINYINT > SMALLINT > INT > BIGINT > FLOAT > DOUBLE > STRING > CHAR > VARCHAR > DATE > TIMESTAMP > BINARY > DECIMAL > ARRAY > MAP > STRUCT > UNIONTYPE > USER_DEFINED > And against Impala (2.1): > NULL_TYPE > BOOLEAN > TINYINT > SMALLINT > INT > BIGINT > FLOAT > DOUBLE > TIMESTAMP > STRING > BINARY > Note the missing newer datatypes (e.g. decimal, varchar). > This is with the Hive 0.13 driver. Please let me know if there's something > I'm missing and this should be filed against the driver. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-1856) Missing datatypes from JDBC DBMD.getTypeInfo() call
[ https://issues.apache.org/jira/browse/IMPALA-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808987#comment-16808987 ] Tim Armstrong commented on IMPALA-1856: --- This behaviour is covered by a test - FrontEndTest.TestGetTypeInfo() but the test wasn't updated when new data types would added. It would be good if we could somehow avoid this problem when adding new types so that if we add a new type and forget to update GetTypeInfo(), the test fails. We should also add an end-to-end test for the HS2 GetTypeInfo() RPC to tests/hs2/test_hs2.py. There is a test gap here. > Missing datatypes from JDBC DBMD.getTypeInfo() call > --- > > Key: IMPALA-1856 > URL: https://issues.apache.org/jira/browse/IMPALA-1856 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 2.1 > Environment: Impala 2.1 > C5.3 > hive-jdbc 0.13 >Reporter: Jonathan Seidman >Priority: Minor > Labels: jdbc, odbc, ramp-up, supportability > > Filing this here since it seems to be an Impala defect as opposed to a driver > defect based on observed behavior. The following is seen from > DBMD.getTypeInfo() call against Hive: > VOID > BOOLEAN > TINYINT > SMALLINT > INT > BIGINT > FLOAT > DOUBLE > STRING > CHAR > VARCHAR > DATE > TIMESTAMP > BINARY > DECIMAL > ARRAY > MAP > STRUCT > UNIONTYPE > USER_DEFINED > And against Impala (2.1): > NULL_TYPE > BOOLEAN > TINYINT > SMALLINT > INT > BIGINT > FLOAT > DOUBLE > TIMESTAMP > STRING > BINARY > Note the missing newer datatypes (e.g. decimal, varchar). > This is with the Hive 0.13 driver. Please let me know if there's something > I'm missing and this should be filed against the driver. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-8375) Add metrics for spill disk usage
[ https://issues.apache.org/jira/browse/IMPALA-8375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong reassigned IMPALA-8375: - Assignee: Abhishek Rawat > Add metrics for spill disk usage > > > Key: IMPALA-8375 > URL: https://issues.apache.org/jira/browse/IMPALA-8375 > Project: IMPALA > Issue Type: Sub-task > Components: Backend >Affects Versions: Impala 3.2.0 >Reporter: Tim Armstrong >Assignee: Abhishek Rawat >Priority: Major > > * High water mark of bytes allocated > * Maybe - some metric of utilisation (what % of bytes allocated are actually > being used) > * Maybe - total bytes allocated over the lifetime of the process > * Maybe - statistics about the sizes of blocks allocated -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8383) Bump toolchain version
Hector Acosta created IMPALA-8383: - Summary: Bump toolchain version Key: IMPALA-8383 URL: https://issues.apache.org/jira/browse/IMPALA-8383 Project: IMPALA Issue Type: Bug Reporter: Hector Acosta The current $IMPALA_TOOLCHAIN_BUILD_ID has a bug where the fastbinary shared object is missing for some distributions. We should bump the version to an id that includes fastbinary.so. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8382) Add support for SLES 12 SP3
Hector Acosta created IMPALA-8382: - Summary: Add support for SLES 12 SP3 Key: IMPALA-8382 URL: https://issues.apache.org/jira/browse/IMPALA-8382 Project: IMPALA Issue Type: Bug Reporter: Hector Acosta We already build the native toolchain using [SLES12sp3|https://github.com/cloudera/native-toolchain/blob/master/docker/sles12.df#L1] however bootstrap-toolchain.py only supports the now EOL [SLES 12 sp2|https://github.com/apache/impala/blob/master/bin/bootstrap_toolchain.py#L73]. Since the toolchain is already built on sp3, fetching the same artifacts, should add support for SLES 12 sp3. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8382) Add support for SLES 12 SP3
[ https://issues.apache.org/jira/browse/IMPALA-8382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808935#comment-16808935 ] Hector Acosta commented on IMPALA-8382: --- I'd like to work on this. > Add support for SLES 12 SP3 > --- > > Key: IMPALA-8382 > URL: https://issues.apache.org/jira/browse/IMPALA-8382 > Project: IMPALA > Issue Type: Bug >Reporter: Hector Acosta >Priority: Major > > We already build the native toolchain using > [SLES12sp3|https://github.com/cloudera/native-toolchain/blob/master/docker/sles12.df#L1] > however bootstrap-toolchain.py only supports the now EOL [SLES 12 > sp2|https://github.com/apache/impala/blob/master/bin/bootstrap_toolchain.py#L73]. > > Since the toolchain is already built on sp3, fetching the same artifacts, > should add support for SLES 12 sp3. > > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-8363) Disable column masking and row filtering
[ https://issues.apache.org/jira/browse/IMPALA-8363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-8363 started by Fredy Wijaya. > Disable column masking and row filtering > > > Key: IMPALA-8363 > URL: https://issues.apache.org/jira/browse/IMPALA-8363 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > > Because of policy sharing between Hive and Ranger, Impala needs throw an > authorization exception when column masking and row filtering is enabled in > Hive. This is only temporary until Impala has proper support for column > masking and row filtering. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8381) Remove branch from ParquetPlainEncoder::Decode()
Csaba Ringhofer created IMPALA-8381: --- Summary: Remove branch from ParquetPlainEncoder::Decode() Key: IMPALA-8381 URL: https://issues.apache.org/jira/browse/IMPALA-8381 Project: IMPALA Issue Type: Improvement Components: Backend Reporter: Csaba Ringhofer Removing the "if" at https://github.com/apache/impala/blob/5670f96b828d57f9e36510bb9af02bcc31de775c/be/src/exec/parquet/parquet-common.h#L203 can lead to 1.5x speed up in plain decoding (type=int32, stride=16). For primitive types, the same check can be done for a whole batch, so the speedup can be gained for large batches without loosing safety. The only Parquet type where this check is needed per element is BYTE_ARRAY (typically used for STRING columns), which already has a template specialization for ParquetPlainEncoder::Decode(). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
