[jira] [Resolved] (IMPALA-8750) TestObservability.test_query_profile_contains_query_compilation_events failing when compilation events appear dynamically
[
https://issues.apache.org/jira/browse/IMPALA-8750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya resolved IMPALA-8750.
--
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> TestObservability.test_query_profile_contains_query_compilation_events
> failing when compilation events appear dynamically
> -
>
> Key: IMPALA-8750
> URL: https://issues.apache.org/jira/browse/IMPALA-8750
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.3.0
>Reporter: Tamas Mate
>Assignee: Tamas Mate
>Priority: Major
> Labels: test
> Fix For: Impala 3.3.0
>
>
> {{TestObservability.test_query_profile_contains_query_compilation_events}} is
> failing during builds where the query compilation event differs from the
> specified. This can occur in 2 cases:
> - When {{lineage_event_log_dir}} is specified it adds an extra "Lineage info
> computed:" line.
> - When metadata is not preloaded on the coordinator instead of "Metadata of
> all * tables cached:" it can be multiple lines:
> {noformat}
> Metadata load started:
> Metadata load finished:
> {noformat}
> We should cover these cases as well.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8750) TestObservability.test_query_profile_contains_query_compilation_events failing when compilation events appear dynamically
[
https://issues.apache.org/jira/browse/IMPALA-8750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya resolved IMPALA-8750.
--
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> TestObservability.test_query_profile_contains_query_compilation_events
> failing when compilation events appear dynamically
> -
>
> Key: IMPALA-8750
> URL: https://issues.apache.org/jira/browse/IMPALA-8750
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.3.0
>Reporter: Tamas Mate
>Assignee: Tamas Mate
>Priority: Major
> Labels: test
> Fix For: Impala 3.3.0
>
>
> {{TestObservability.test_query_profile_contains_query_compilation_events}} is
> failing during builds where the query compilation event differs from the
> specified. This can occur in 2 cases:
> - When {{lineage_event_log_dir}} is specified it adds an extra "Lineage info
> computed:" line.
> - When metadata is not preloaded on the coordinator instead of "Metadata of
> all * tables cached:" it can be multiple lines:
> {noformat}
> Metadata load started:
> Metadata load finished:
> {noformat}
> We should cover these cases as well.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (IMPALA-8750) TestObservability.test_query_profile_contains_query_compilation_events failing when compilation events appear dynamically
[
https://issues.apache.org/jira/browse/IMPALA-8750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya updated IMPALA-8750:
-
Component/s: (was: Frontend)
> TestObservability.test_query_profile_contains_query_compilation_events
> failing when compilation events appear dynamically
> -
>
> Key: IMPALA-8750
> URL: https://issues.apache.org/jira/browse/IMPALA-8750
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.3.0
>Reporter: Tamas Mate
>Assignee: Tamas Mate
>Priority: Major
> Labels: test
>
> {{TestObservability.test_query_profile_contains_query_compilation_events}} is
> failing during builds where the query compilation event differs from the
> specified. This can occur in 2 cases:
> - When {{lineage_event_log_dir}} is specified it adds an extra "Lineage info
> computed:" line.
> - When metadata is not preloaded on the coordinator instead of "Metadata of
> all * tables cached:" it can be multiple lines:
> {noformat}
> Metadata load started:
> Metadata load finished:
> {noformat}
> We should cover these cases as well.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8750) TestObservability.test_query_profile_contains_query_compilation_events failing when compilation events appear dynamically
[
https://issues.apache.org/jira/browse/IMPALA-8750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya updated IMPALA-8750:
-
Component/s: Infrastructure
> TestObservability.test_query_profile_contains_query_compilation_events
> failing when compilation events appear dynamically
> -
>
> Key: IMPALA-8750
> URL: https://issues.apache.org/jira/browse/IMPALA-8750
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend, Infrastructure
>Affects Versions: Impala 3.3.0
>Reporter: Tamas Mate
>Assignee: Tamas Mate
>Priority: Major
> Labels: test
>
> {{TestObservability.test_query_profile_contains_query_compilation_events}} is
> failing during builds where the query compilation event differs from the
> specified. This can occur in 2 cases:
> - When {{lineage_event_log_dir}} is specified it adds an extra "Lineage info
> computed:" line.
> - When metadata is not preloaded on the coordinator instead of "Metadata of
> all * tables cached:" it can be multiple lines:
> {noformat}
> Metadata load started:
> Metadata load finished:
> {noformat}
> We should cover these cases as well.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8734) ALTER TABLE ... SET TBLPROPERTIES doesn't propagate until Invalidate Metadata
[
https://issues.apache.org/jira/browse/IMPALA-8734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya resolved IMPALA-8734.
--
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> ALTER TABLE ... SET TBLPROPERTIES doesn't propagate until Invalidate Metadata
> -
>
> Key: IMPALA-8734
> URL: https://issues.apache.org/jira/browse/IMPALA-8734
> Project: IMPALA
> Issue Type: Improvement
>Affects Versions: Impala 3.3.0
>Reporter: Vincent Tran
>Assignee: Fredy Wijaya
>Priority: Major
> Fix For: Impala 3.3.0
>
>
> More particularly, if one needs to change the data representation of NULL in
> the tblproperties, an INVALIDATE METADATA is needed for table to be readable
> with the new `serialization.null.format`
> To setup, create a table and dfs -put some data files into hdfs with some
> null strings:
> {noformat}
> [localhost:21000] default> create table v.t1(c1 string);
> Query: create table v.t1(c1 string)
> +-+
> | summary |
> +-+
> | Table has been created. |
> +-+
> Fetched 1 row(s) in 0.51s
> [localhost:21000] default> show create table v.t1;
> Query: show create table v.t1
> +--+
> | result |
> +--+
> | CREATE TABLE v.t1 ( |
> | c1 STRING |
> | ) |
> | STORED AS TEXTFILE |
> | LOCATION 'hdfs://localhost:20500/test-warehouse/v.db/t1' |
> | |
> +--+
> Fetched 1 row(s) in 4.38s
> [localhost:21000] default> !printf '\nnull\n' > f;
>
> Executed in 0.00s
> [localhost:21000] default> !hdfs dfs -put f /test-warehouse/v.db/t1/f1;
>
> Executed in 1.98s{noformat}
>
> Repro:
> {noformat}
> [localhost:21000] default> select * from v.t1;
> Query: select * from v.t1
> Query submitted at: 2019-06-27 12:48:13 (Coordinator:
> http://blackbox.vpc.cloudera.com:25000)
> Query progress can be monitored at:
> http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=8b4532852b10613e:e9e48475
> +--+
> | c1 |
> +--+
> | |
> | null |
> +--+
> Fetched 2 row(s) in 4.77s
> [localhost:21000] default> alter table v.t1 set
> tblproperties("serialization.null.format"="null");
> Query: alter table v.t1 set tblproperties("serialization.null.format"="null")
> ++
> | summary |
> ++
> | Updated table. |
> ++
> Fetched 1 row(s) in 0.05s
> [localhost:21000] default> select * from v.t1;
> Query: select * from v.t1
> Query submitted at: 2019-06-27 12:48:39 (Coordinator:
> http://blackbox.vpc.cloudera.com:25000)
> Query progress can be monitored at:
> http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=e54749def520fa97:98007283
> +--+
> | c1 |
> +--+
> | |
> | null |
> +--+
> Fetched 2 row(s) in 0.11s
> [localhost:21000] default> invalidate metadata v.t1;
> Query: invalidate metadata v.t1
> Query submitted at: 2019-06-27 12:48:50 (Coordinator:
> http://blackbox.vpc.cloudera.com:25000)
> Query progress can be monitored at:
> http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=9d41e48bc8d5bad3:8020514f
> Fetched 0 row(s) in 0.01s
> [localhost:21000] default> select * from v.t1;
> Query: select * from v.t1
> Query submitted at: 2019-06-27 12:48:51 (Coordinator:
> http://blackbox.vpc.cloudera.com:25000)
> Query progress can be monitored at:
> http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=0446c2c65af3400a:3e3a4849
> +--+
> | c1 |
> +--+
> | |
> | NULL |
> +--+
> Fetched 2 row(s) in 4.48s
> [localhost:21000] default>{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8734) ALTER TABLE ... SET TBLPROPERTIES doesn't propagate until Invalidate Metadata
[
https://issues.apache.org/jira/browse/IMPALA-8734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya resolved IMPALA-8734.
--
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> ALTER TABLE ... SET TBLPROPERTIES doesn't propagate until Invalidate Metadata
> -
>
> Key: IMPALA-8734
> URL: https://issues.apache.org/jira/browse/IMPALA-8734
> Project: IMPALA
> Issue Type: Improvement
>Affects Versions: Impala 3.3.0
>Reporter: Vincent Tran
>Assignee: Fredy Wijaya
>Priority: Major
> Fix For: Impala 3.3.0
>
>
> More particularly, if one needs to change the data representation of NULL in
> the tblproperties, an INVALIDATE METADATA is needed for table to be readable
> with the new `serialization.null.format`
> To setup, create a table and dfs -put some data files into hdfs with some
> null strings:
> {noformat}
> [localhost:21000] default> create table v.t1(c1 string);
> Query: create table v.t1(c1 string)
> +-+
> | summary |
> +-+
> | Table has been created. |
> +-+
> Fetched 1 row(s) in 0.51s
> [localhost:21000] default> show create table v.t1;
> Query: show create table v.t1
> +--+
> | result |
> +--+
> | CREATE TABLE v.t1 ( |
> | c1 STRING |
> | ) |
> | STORED AS TEXTFILE |
> | LOCATION 'hdfs://localhost:20500/test-warehouse/v.db/t1' |
> | |
> +--+
> Fetched 1 row(s) in 4.38s
> [localhost:21000] default> !printf '\nnull\n' > f;
>
> Executed in 0.00s
> [localhost:21000] default> !hdfs dfs -put f /test-warehouse/v.db/t1/f1;
>
> Executed in 1.98s{noformat}
>
> Repro:
> {noformat}
> [localhost:21000] default> select * from v.t1;
> Query: select * from v.t1
> Query submitted at: 2019-06-27 12:48:13 (Coordinator:
> http://blackbox.vpc.cloudera.com:25000)
> Query progress can be monitored at:
> http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=8b4532852b10613e:e9e48475
> +--+
> | c1 |
> +--+
> | |
> | null |
> +--+
> Fetched 2 row(s) in 4.77s
> [localhost:21000] default> alter table v.t1 set
> tblproperties("serialization.null.format"="null");
> Query: alter table v.t1 set tblproperties("serialization.null.format"="null")
> ++
> | summary |
> ++
> | Updated table. |
> ++
> Fetched 1 row(s) in 0.05s
> [localhost:21000] default> select * from v.t1;
> Query: select * from v.t1
> Query submitted at: 2019-06-27 12:48:39 (Coordinator:
> http://blackbox.vpc.cloudera.com:25000)
> Query progress can be monitored at:
> http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=e54749def520fa97:98007283
> +--+
> | c1 |
> +--+
> | |
> | null |
> +--+
> Fetched 2 row(s) in 0.11s
> [localhost:21000] default> invalidate metadata v.t1;
> Query: invalidate metadata v.t1
> Query submitted at: 2019-06-27 12:48:50 (Coordinator:
> http://blackbox.vpc.cloudera.com:25000)
> Query progress can be monitored at:
> http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=9d41e48bc8d5bad3:8020514f
> Fetched 0 row(s) in 0.01s
> [localhost:21000] default> select * from v.t1;
> Query: select * from v.t1
> Query submitted at: 2019-06-27 12:48:51 (Coordinator:
> http://blackbox.vpc.cloudera.com:25000)
> Query progress can be monitored at:
> http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=0446c2c65af3400a:3e3a4849
> +--+
> | c1 |
> +--+
> | |
> | NULL |
> +--+
> Fetched 2 row(s) in 4.48s
> [localhost:21000] default>{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IMPALA-8735) Publish query event hook API to Maven Central
Fredy Wijaya created IMPALA-8735: Summary: Publish query event hook API to Maven Central Key: IMPALA-8735 URL: https://issues.apache.org/jira/browse/IMPALA-8735 Project: IMPALA Issue Type: Sub-task Components: Infrastructure Reporter: Fredy Wijaya We need to publish the query hook API as part of the work done in https://issues.apache.org/jira/browse/IMPALA-8599 to Maven Central. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8735) Publish query event hook API to Maven Central
Fredy Wijaya created IMPALA-8735: Summary: Publish query event hook API to Maven Central Key: IMPALA-8735 URL: https://issues.apache.org/jira/browse/IMPALA-8735 Project: IMPALA Issue Type: Sub-task Components: Infrastructure Reporter: Fredy Wijaya We need to publish the query hook API as part of the work done in https://issues.apache.org/jira/browse/IMPALA-8599 to Maven Central. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (IMPALA-8734) ALTER TABLE ... SET TBLPROPERTIES doesn't propagate until Invalidate Metadata
[
https://issues.apache.org/jira/browse/IMPALA-8734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-8734 started by Fredy Wijaya.
> ALTER TABLE ... SET TBLPROPERTIES doesn't propagate until Invalidate Metadata
> -
>
> Key: IMPALA-8734
> URL: https://issues.apache.org/jira/browse/IMPALA-8734
> Project: IMPALA
> Issue Type: Improvement
>Affects Versions: Impala 3.3.0
>Reporter: Vincent Tran
>Assignee: Fredy Wijaya
>Priority: Major
>
> More particularly, if one needs to change the data representation of NULL in
> the tblproperties, an INVALIDATE METADATA is needed for table to be readable
> with the new `serialization.null.format`
> To setup, create a table and dfs -put some data files into hdfs with some
> null strings:
> {noformat}
> [localhost:21000] default> create table v.t1(c1 string);
> Query: create table v.t1(c1 string)
> +-+
> | summary |
> +-+
> | Table has been created. |
> +-+
> Fetched 1 row(s) in 0.51s
> [localhost:21000] default> show create table v.t1;
> Query: show create table v.t1
> +--+
> | result |
> +--+
> | CREATE TABLE v.t1 ( |
> | c1 STRING |
> | ) |
> | STORED AS TEXTFILE |
> | LOCATION 'hdfs://localhost:20500/test-warehouse/v.db/t1' |
> | |
> +--+
> Fetched 1 row(s) in 4.38s
> [localhost:21000] default> !printf '\nnull\n' > f;
>
> Executed in 0.00s
> [localhost:21000] default> !hdfs dfs -put f /test-warehouse/v.db/t1/f1;
>
> Executed in 1.98s{noformat}
>
> Repro:
> {noformat}
> [localhost:21000] default> select * from v.t1;
> Query: select * from v.t1
> Query submitted at: 2019-06-27 12:48:13 (Coordinator:
> http://blackbox.vpc.cloudera.com:25000)
> Query progress can be monitored at:
> http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=8b4532852b10613e:e9e48475
> +--+
> | c1 |
> +--+
> | |
> | null |
> +--+
> Fetched 2 row(s) in 4.77s
> [localhost:21000] default> alter table v.t1 set
> tblproperties("serialization.null.format"="null");
> Query: alter table v.t1 set tblproperties("serialization.null.format"="null")
> ++
> | summary |
> ++
> | Updated table. |
> ++
> Fetched 1 row(s) in 0.05s
> [localhost:21000] default> select * from v.t1;
> Query: select * from v.t1
> Query submitted at: 2019-06-27 12:48:39 (Coordinator:
> http://blackbox.vpc.cloudera.com:25000)
> Query progress can be monitored at:
> http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=e54749def520fa97:98007283
> +--+
> | c1 |
> +--+
> | |
> | null |
> +--+
> Fetched 2 row(s) in 0.11s
> [localhost:21000] default> invalidate metadata v.t1;
> Query: invalidate metadata v.t1
> Query submitted at: 2019-06-27 12:48:50 (Coordinator:
> http://blackbox.vpc.cloudera.com:25000)
> Query progress can be monitored at:
> http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=9d41e48bc8d5bad3:8020514f
> Fetched 0 row(s) in 0.01s
> [localhost:21000] default> select * from v.t1;
> Query: select * from v.t1
> Query submitted at: 2019-06-27 12:48:51 (Coordinator:
> http://blackbox.vpc.cloudera.com:25000)
> Query progress can be monitored at:
> http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=0446c2c65af3400a:3e3a4849
> +--+
> | c1 |
> +--+
> | |
> | NULL |
> +--+
> Fetched 2 row(s) in 4.48s
> [localhost:21000] default>{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8716) Log a a group of privileges into a single audit event
[ https://issues.apache.org/jira/browse/IMPALA-8716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8716. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Log a a group of privileges into a single audit event > - > > Key: IMPALA-8716 > URL: https://issues.apache.org/jira/browse/IMPALA-8716 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.3.0 > > > Some privileges, such as VIEW_METADATA consists of multiple privileges > (INSERT, SELECT, REFRESH). For example if we run "show partitions > foo.barfoo.bar" and we have SELECT privilege on table "foo.bar", we will be > creating 2 audit logs: > - Attempt to check if there's INSERT privilege on table "foo.bar" -- denied, > INSERT, foo.bar > - Attempt to check if there's SELECT privilege on table "foo.bar" -- allowed, > SELECT, foo.bar > This can be confusing. A better solution is to log this as a single audit > log, e.g. > - allowed, VIEW_METADATA, foo.bar -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8716) Log a a group of privileges into a single audit event
[ https://issues.apache.org/jira/browse/IMPALA-8716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8716. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Log a a group of privileges into a single audit event > - > > Key: IMPALA-8716 > URL: https://issues.apache.org/jira/browse/IMPALA-8716 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.3.0 > > > Some privileges, such as VIEW_METADATA consists of multiple privileges > (INSERT, SELECT, REFRESH). For example if we run "show partitions > foo.barfoo.bar" and we have SELECT privilege on table "foo.bar", we will be > creating 2 audit logs: > - Attempt to check if there's INSERT privilege on table "foo.bar" -- denied, > INSERT, foo.bar > - Attempt to check if there's SELECT privilege on table "foo.bar" -- allowed, > SELECT, foo.bar > This can be confusing. A better solution is to log this as a single audit > log, e.g. > - allowed, VIEW_METADATA, foo.bar -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IMPALA-8399) Batch the authorization requests to Ranger
[ https://issues.apache.org/jira/browse/IMPALA-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-8399: - Issue Type: Sub-task (was: Improvement) Parent: IMPALA-7916 > Batch the authorization requests to Ranger > -- > > Key: IMPALA-8399 > URL: https://issues.apache.org/jira/browse/IMPALA-8399 > Project: IMPALA > Issue Type: Sub-task > Components: Catalog, Frontend >Reporter: Fredy Wijaya >Priority: Major > > To reduce the network round trip we should consider batching authorization > requests to Ranger. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8651) Update grammar for Ranger revoke grant option statement
[ https://issues.apache.org/jira/browse/IMPALA-8651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-8651: - Issue Type: Sub-task (was: Improvement) Parent: IMPALA-7916 > Update grammar for Ranger revoke grant option statement > --- > > Key: IMPALA-8651 > URL: https://issues.apache.org/jira/browse/IMPALA-8651 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Austin Nobis >Priority: Major > > *REVOKE GRANT OPTION FOR SELECT ON DATABASE DB FROM USER USR;* > In Ranger, it is not possible to *REVOKE GRANT OPTION* for a specific > privilege (*SELECT*). The *GRANT OPTION* must be revoked for the entire > *DATABASE DB* resource. The Impala grammar should be updated to support > omitting the *FOR SELECT* if the authorization provider is set to Ranger. If > that grammar is used for Sentry, it should throw an exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8399) Batch the authorization requests to Ranger
[ https://issues.apache.org/jira/browse/IMPALA-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-8399: - Priority: Major (was: Critical) > Batch the authorization requests to Ranger > -- > > Key: IMPALA-8399 > URL: https://issues.apache.org/jira/browse/IMPALA-8399 > Project: IMPALA > Issue Type: Improvement > Components: Catalog, Frontend >Reporter: Fredy Wijaya >Priority: Major > > To reduce the network round trip we should consider batching authorization > requests to Ranger. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8651) Update grammar for Ranger revoke grant option statement
[ https://issues.apache.org/jira/browse/IMPALA-8651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-8651: - Fix Version/s: (was: Impala 3.3.0) > Update grammar for Ranger revoke grant option statement > --- > > Key: IMPALA-8651 > URL: https://issues.apache.org/jira/browse/IMPALA-8651 > Project: IMPALA > Issue Type: Improvement > Components: Frontend >Reporter: Austin Nobis >Priority: Major > > *REVOKE GRANT OPTION FOR SELECT ON DATABASE DB FROM USER USR;* > In Ranger, it is not possible to *REVOKE GRANT OPTION* for a specific > privilege (*SELECT*). The *GRANT OPTION* must be revoked for the entire > *DATABASE DB* resource. The Impala grammar should be updated to support > omitting the *FOR SELECT* if the authorization provider is set to Ranger. If > that grammar is used for Sentry, it should throw an exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8651) Update grammar for Ranger revoke grant option statement
[ https://issues.apache.org/jira/browse/IMPALA-8651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-8651: - Issue Type: Improvement (was: Sub-task) Parent: (was: IMPALA-7916) > Update grammar for Ranger revoke grant option statement > --- > > Key: IMPALA-8651 > URL: https://issues.apache.org/jira/browse/IMPALA-8651 > Project: IMPALA > Issue Type: Improvement > Components: Frontend >Reporter: Austin Nobis >Priority: Major > Fix For: Impala 3.3.0 > > > *REVOKE GRANT OPTION FOR SELECT ON DATABASE DB FROM USER USR;* > In Ranger, it is not possible to *REVOKE GRANT OPTION* for a specific > privilege (*SELECT*). The *GRANT OPTION* must be revoked for the entire > *DATABASE DB* resource. The Impala grammar should be updated to support > omitting the *FOR SELECT* if the authorization provider is set to Ranger. If > that grammar is used for Sentry, it should throw an exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8399) Batch the authorization requests to Ranger
[ https://issues.apache.org/jira/browse/IMPALA-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-8399: - Issue Type: Improvement (was: Sub-task) Parent: (was: IMPALA-7916) > Batch the authorization requests to Ranger > -- > > Key: IMPALA-8399 > URL: https://issues.apache.org/jira/browse/IMPALA-8399 > Project: IMPALA > Issue Type: Improvement > Components: Catalog, Frontend >Reporter: Fredy Wijaya >Priority: Critical > > To reduce the network round trip we should consider batching authorization > requests to Ranger. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8476) Replace Sentry admin check workaround with proper Sentry API
[ https://issues.apache.org/jira/browse/IMPALA-8476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8476. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Replace Sentry admin check workaround with proper Sentry API > > > Key: IMPALA-8476 > URL: https://issues.apache.org/jira/browse/IMPALA-8476 > Project: IMPALA > Issue Type: Improvement > Components: Catalog >Reporter: Fredy Wijaya >Assignee: Fang-Yu Rao >Priority: Minor > Labels: newbie > Fix For: Impala 3.3.0 > > > Impala uses a workaround to detect if a user is a Sentry admin by calling > list privileges API before Sentry didn't provide an API to tell if user is a > Sentry admin: > https://github.com/apache/impala/blob/d820952d86d34ba887c55a09e58b735cbef866c2/fe/src/main/java/org/apache/impala/authorization/sentry/SentryProxy.java#L378-L393 > https://issues.apache.org/jira/browse/SENTRY-2440 supports a new API. We > should update the Impala to use the new proper Sentry API. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8476) Replace Sentry admin check workaround with proper Sentry API
[ https://issues.apache.org/jira/browse/IMPALA-8476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8476. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Replace Sentry admin check workaround with proper Sentry API > > > Key: IMPALA-8476 > URL: https://issues.apache.org/jira/browse/IMPALA-8476 > Project: IMPALA > Issue Type: Improvement > Components: Catalog >Reporter: Fredy Wijaya >Assignee: Fang-Yu Rao >Priority: Minor > Labels: newbie > Fix For: Impala 3.3.0 > > > Impala uses a workaround to detect if a user is a Sentry admin by calling > list privileges API before Sentry didn't provide an API to tell if user is a > Sentry admin: > https://github.com/apache/impala/blob/d820952d86d34ba887c55a09e58b735cbef866c2/fe/src/main/java/org/apache/impala/authorization/sentry/SentryProxy.java#L378-L393 > https://issues.apache.org/jira/browse/SENTRY-2440 supports a new API. We > should update the Impala to use the new proper Sentry API. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IMPALA-8716) Log a a group of privileges into a single audit event
[ https://issues.apache.org/jira/browse/IMPALA-8716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-8716: - Description: Some privileges, such as VIEW_METADATA consists of multiple privileges (INSERT, SELECT, REFRESH). For example if we run "show partitions foo.barfoo.bar" and we have SELECT privilege on table "foo.bar", we will be creating 2 audit logs: - Attempt to check if there's INSERT privilege on table "foo.bar" -- denied, INSERT, foo.bar - Attempt to check if there's SELECT privilege on table "foo.bar" -- allowed, SELECT, foo.bar This can be confusing. A better solution is to log this as a single audit log, e.g. - allowed, VIEW_METADATA, foo.bar was: Some privileges, such as VIEW_METADATA consists of multiple privileges (INSERT, SELECT, REFRESH). For example if we have run "show partitions foo.barfoo.bar" and we have SELECT privilege on table "foo.bar", we will be creating 2 audit logs: - Attempt to check if there's INSERT privilege on table "foo.bar" -- denied, INSERT, foo.bar - Attempt to check if there's SELECT privilege on table "foo.bar" -- allowed, SELECT, foo.bar This can be confusing. A better solution is to log this as a single audit log, e.g. - allowed, VIEW_METADATA, foo.bar > Log a a group of privileges into a single audit event > - > > Key: IMPALA-8716 > URL: https://issues.apache.org/jira/browse/IMPALA-8716 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > > Some privileges, such as VIEW_METADATA consists of multiple privileges > (INSERT, SELECT, REFRESH). For example if we run "show partitions > foo.barfoo.bar" and we have SELECT privilege on table "foo.bar", we will be > creating 2 audit logs: > - Attempt to check if there's INSERT privilege on table "foo.bar" -- denied, > INSERT, foo.bar > - Attempt to check if there's SELECT privilege on table "foo.bar" -- allowed, > SELECT, foo.bar > This can be confusing. A better solution is to log this as a single audit > log, e.g. > - allowed, VIEW_METADATA, foo.bar -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8716) Log a a group of privileges into a single audit event
[ https://issues.apache.org/jira/browse/IMPALA-8716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-8716: - Summary: Log a a group of privileges into a single audit event (was: Log a a group of privileges into a single audit log) > Log a a group of privileges into a single audit event > - > > Key: IMPALA-8716 > URL: https://issues.apache.org/jira/browse/IMPALA-8716 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > > Some privileges, such as VIEW_METADATA consists of multiple privileges > (INSERT, SELECT, REFRESH). For example if we have run "show partitions > foo.barfoo.bar" and we have SELECT privilege on table "foo.bar", we will be > creating 2 audit logs: > - Attempt to check if there's INSERT privilege on table "foo.bar" -- denied, > INSERT, foo.bar > - Attempt to check if there's SELECT privilege on table "foo.bar" -- allowed, > SELECT, foo.bar > This can be confusing. A better solution is to log this as a single audit > log, e.g. > - allowed, VIEW_METADATA, foo.bar -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8716) Log a a group of privileges into a single audit log
Fredy Wijaya created IMPALA-8716: Summary: Log a a group of privileges into a single audit log Key: IMPALA-8716 URL: https://issues.apache.org/jira/browse/IMPALA-8716 Project: IMPALA Issue Type: Sub-task Components: Frontend Reporter: Fredy Wijaya Assignee: Fredy Wijaya Some privileges, such as VIEW_METADATA consists of multiple privileges (INSERT, SELECT, REFRESH). For example if we have run "show partitions foo.barfoo.bar" and we have SELECT privilege on table "foo.bar", we will be creating 2 audit logs: - Attempt to check if there's INSERT privilege on table "foo.bar" -- denied, INSERT, foo.bar - Attempt to check if there's SELECT privilege on table "foo.bar" -- allowed, SELECT, foo.bar This can be confusing. A better solution is to log this as a single audit log, e.g. - allowed, VIEW_METADATA, foo.bar -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-8716) Log a a group of privileges into a single audit log
[ https://issues.apache.org/jira/browse/IMPALA-8716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-8716 started by Fredy Wijaya. > Log a a group of privileges into a single audit log > --- > > Key: IMPALA-8716 > URL: https://issues.apache.org/jira/browse/IMPALA-8716 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > > Some privileges, such as VIEW_METADATA consists of multiple privileges > (INSERT, SELECT, REFRESH). For example if we have run "show partitions > foo.barfoo.bar" and we have SELECT privilege on table "foo.bar", we will be > creating 2 audit logs: > - Attempt to check if there's INSERT privilege on table "foo.bar" -- denied, > INSERT, foo.bar > - Attempt to check if there's SELECT privilege on table "foo.bar" -- allowed, > SELECT, foo.bar > This can be confusing. A better solution is to log this as a single audit > log, e.g. > - allowed, VIEW_METADATA, foo.bar -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8716) Log a a group of privileges into a single audit log
Fredy Wijaya created IMPALA-8716: Summary: Log a a group of privileges into a single audit log Key: IMPALA-8716 URL: https://issues.apache.org/jira/browse/IMPALA-8716 Project: IMPALA Issue Type: Sub-task Components: Frontend Reporter: Fredy Wijaya Assignee: Fredy Wijaya Some privileges, such as VIEW_METADATA consists of multiple privileges (INSERT, SELECT, REFRESH). For example if we have run "show partitions foo.barfoo.bar" and we have SELECT privilege on table "foo.bar", we will be creating 2 audit logs: - Attempt to check if there's INSERT privilege on table "foo.bar" -- denied, INSERT, foo.bar - Attempt to check if there's SELECT privilege on table "foo.bar" -- allowed, SELECT, foo.bar This can be confusing. A better solution is to log this as a single audit log, e.g. - allowed, VIEW_METADATA, foo.bar -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-8589) Fix flaky query event hook tests
[ https://issues.apache.org/jira/browse/IMPALA-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8589. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Fix flaky query event hook tests > > > Key: IMPALA-8589 > URL: https://issues.apache.org/jira/browse/IMPALA-8589 > Project: IMPALA > Issue Type: Sub-task > Components: Infrastructure >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.3.0 > > > The test_query_event_hooks.py tests is flaky. We need to figure out a way to > deflake it. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8589) Fix flaky query event hook tests
[ https://issues.apache.org/jira/browse/IMPALA-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8589. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Fix flaky query event hook tests > > > Key: IMPALA-8589 > URL: https://issues.apache.org/jira/browse/IMPALA-8589 > Project: IMPALA > Issue Type: Sub-task > Components: Infrastructure >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.3.0 > > > The test_query_event_hooks.py tests is flaky. We need to figure out a way to > deflake it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (IMPALA-8589) Fix flaky query event hook tests
[ https://issues.apache.org/jira/browse/IMPALA-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-8589 started by Fredy Wijaya. > Fix flaky query event hook tests > > > Key: IMPALA-8589 > URL: https://issues.apache.org/jira/browse/IMPALA-8589 > Project: IMPALA > Issue Type: Sub-task > Components: Infrastructure >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > > The test_query_event_hooks.py tests is flaky. We need to figure out a way to > deflake it. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8682) Add test coverage for testing authorized proxy user/group with Ranger
[ https://issues.apache.org/jira/browse/IMPALA-8682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8682. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Add test coverage for testing authorized proxy user/group with Ranger > - > > Key: IMPALA-8682 > URL: https://issues.apache.org/jira/browse/IMPALA-8682 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.3.0 > > > There's currently no tests that test authorized proxy user/group with Ranger. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8682) Add test coverage for testing authorized proxy user/group with Ranger
[ https://issues.apache.org/jira/browse/IMPALA-8682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8682. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Add test coverage for testing authorized proxy user/group with Ranger > - > > Key: IMPALA-8682 > URL: https://issues.apache.org/jira/browse/IMPALA-8682 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.3.0 > > > There's currently no tests that test authorized proxy user/group with Ranger. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-8443) Record time spent in authorization in the runtime profile
[
https://issues.apache.org/jira/browse/IMPALA-8443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya resolved IMPALA-8443.
--
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> Record time spent in authorization in the runtime profile
> -
>
> Key: IMPALA-8443
> URL: https://issues.apache.org/jira/browse/IMPALA-8443
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
>Affects Versions: Impala 3.2.0
>Reporter: bharath v
>Assignee: Tamas Mate
>Priority: Major
> Labels: newbie, supportability
> Fix For: Impala 3.3.0
>
>
> Authorization is done as a part of analysis. It helps to have the time spent
> in authorization recorded as a part of the query profile timeline. Something
> like as follows. Helps to debug issues when bottleneck is in authorization.
> {noformat}
> Query Compilation
> Metadata load started: 4ms (4559114)
> Metadata load finished. loaded-tables=1/2 load-requests=1
> catalog-updates=3: 2.60s (2599515306)
> Authorization finished (Ranger|Sentry): x ++
> Analysis finished: 1.9m (113860685080)
> Value transfer graph computed: 1.9m (113863830763)
> Single node plan created: 1.9m (116880551103)
> Runtime filters computed: 1.9m (116880914428)
> Distributed plan created: 1.9m (116882951541)
> Lineage info computed: 1.9m (116884735701)
> Planning finished: 1.9m (116900204008)
> Planning finished: 1.9m (116900208344)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-8682) Add test coverage for testing authorized proxy user/group with Ranger
[ https://issues.apache.org/jira/browse/IMPALA-8682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-8682 started by Fredy Wijaya. > Add test coverage for testing authorized proxy user/group with Ranger > - > > Key: IMPALA-8682 > URL: https://issues.apache.org/jira/browse/IMPALA-8682 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > > There's currently no tests that test authorized proxy user/group with Ranger. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8682) Add test coverage for testing authorized proxy user/group with Ranger
Fredy Wijaya created IMPALA-8682: Summary: Add test coverage for testing authorized proxy user/group with Ranger Key: IMPALA-8682 URL: https://issues.apache.org/jira/browse/IMPALA-8682 Project: IMPALA Issue Type: Improvement Components: Infrastructure Reporter: Fredy Wijaya Assignee: Fredy Wijaya There's currently no tests that test authorized proxy user/group with Ranger. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8682) Add test coverage for testing authorized proxy user/group with Ranger
Fredy Wijaya created IMPALA-8682: Summary: Add test coverage for testing authorized proxy user/group with Ranger Key: IMPALA-8682 URL: https://issues.apache.org/jira/browse/IMPALA-8682 Project: IMPALA Issue Type: Improvement Components: Infrastructure Reporter: Fredy Wijaya Assignee: Fredy Wijaya There's currently no tests that test authorized proxy user/group with Ranger. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-8658) Populate required Ranger audit fields to be consistent with Hive
[ https://issues.apache.org/jira/browse/IMPALA-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8658. -- Resolution: Fixed > Populate required Ranger audit fields to be consistent with Hive > > > Key: IMPALA-8658 > URL: https://issues.apache.org/jira/browse/IMPALA-8658 > Project: IMPALA > Issue Type: Sub-task > Components: Catalog, Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Critical > Fix For: Impala 3.3.0 > > > Some required fields: > - Client IP > - Cluster name > - Access type (will need to be in upper case) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8658) Populate required Ranger audit fields to be consistent with Hive
[ https://issues.apache.org/jira/browse/IMPALA-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8658. -- Resolution: Fixed > Populate required Ranger audit fields to be consistent with Hive > > > Key: IMPALA-8658 > URL: https://issues.apache.org/jira/browse/IMPALA-8658 > Project: IMPALA > Issue Type: Sub-task > Components: Catalog, Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Critical > Fix For: Impala 3.3.0 > > > Some required fields: > - Client IP > - Cluster name > - Access type (will need to be in upper case) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (IMPALA-8658) Populate required Ranger audit fields to be consistent with Hive
[ https://issues.apache.org/jira/browse/IMPALA-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya reopened IMPALA-8658: -- Accidentally clicked "resolved". > Populate required Ranger audit fields to be consistent with Hive > > > Key: IMPALA-8658 > URL: https://issues.apache.org/jira/browse/IMPALA-8658 > Project: IMPALA > Issue Type: Sub-task > Components: Catalog, Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Critical > Fix For: Impala 3.3.0 > > > Some required fields: > - Client IP > - Cluster name > - Access type (will need to be in upper case) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8658) Populate required Ranger audit fields to be consistent with Hive
[ https://issues.apache.org/jira/browse/IMPALA-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8658. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Populate required Ranger audit fields to be consistent with Hive > > > Key: IMPALA-8658 > URL: https://issues.apache.org/jira/browse/IMPALA-8658 > Project: IMPALA > Issue Type: Sub-task > Components: Catalog, Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Critical > Fix For: Impala 3.3.0 > > > Some required fields: > - Client IP > - Cluster name > - Access type (will need to be in upper case) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8658) Populate required Ranger audit fields to be consistent with Hive
[ https://issues.apache.org/jira/browse/IMPALA-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8658. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Populate required Ranger audit fields to be consistent with Hive > > > Key: IMPALA-8658 > URL: https://issues.apache.org/jira/browse/IMPALA-8658 > Project: IMPALA > Issue Type: Sub-task > Components: Catalog, Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Critical > Fix For: Impala 3.3.0 > > > Some required fields: > - Client IP > - Cluster name > - Access type (will need to be in upper case) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-8599) Create a separate Maven module for query event hook API
[ https://issues.apache.org/jira/browse/IMPALA-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8599. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Create a separate Maven module for query event hook API > --- > > Key: IMPALA-8599 > URL: https://issues.apache.org/jira/browse/IMPALA-8599 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Critical > Fix For: Impala 3.3.0 > > > Impala needs to publish this API into Maven Central so that Atlas can consume > it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-8599) Create a separate Maven module for query event hook API
[ https://issues.apache.org/jira/browse/IMPALA-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8599. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Create a separate Maven module for query event hook API > --- > > Key: IMPALA-8599 > URL: https://issues.apache.org/jira/browse/IMPALA-8599 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Critical > Fix For: Impala 3.3.0 > > > Impala needs to publish this API into Maven Central so that Atlas can consume > it. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8671) Do not re-create a new instance of AuthorizationChecker with Ranger authorization
[ https://issues.apache.org/jira/browse/IMPALA-8671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8671. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Do not re-create a new instance of AuthorizationChecker with Ranger > authorization > - > > Key: IMPALA-8671 > URL: https://issues.apache.org/jira/browse/IMPALA-8671 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Critical > Fix For: Impala 3.3.0 > > > RangerAuthorizationChecker is stateless unlike SentryAuthorizationChecker. > However, in some cases, we always re-create a new instance of > RangerAuthorizationChecker due to the logic here: > https://github.com/apache/impala/blob/3bd53972cec3b8a863075cb2344115a950be848a/fe/src/main/java/org/apache/impala/service/Frontend.java#L290-L293. > This causes the plugin to be re-initialized, which can be expensive. The > solution is to reuse the same RangerAuthorizationChecker instance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8671) Do not re-create a new instance of AuthorizationChecker with Ranger authorization
[ https://issues.apache.org/jira/browse/IMPALA-8671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8671. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Do not re-create a new instance of AuthorizationChecker with Ranger > authorization > - > > Key: IMPALA-8671 > URL: https://issues.apache.org/jira/browse/IMPALA-8671 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Critical > Fix For: Impala 3.3.0 > > > RangerAuthorizationChecker is stateless unlike SentryAuthorizationChecker. > However, in some cases, we always re-create a new instance of > RangerAuthorizationChecker due to the logic here: > https://github.com/apache/impala/blob/3bd53972cec3b8a863075cb2344115a950be848a/fe/src/main/java/org/apache/impala/service/Frontend.java#L290-L293. > This causes the plugin to be re-initialized, which can be expensive. The > solution is to reuse the same RangerAuthorizationChecker instance. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IMPALA-8671) Do not re-create a new instance of AuthorizationChecker with Ranger authorization
[ https://issues.apache.org/jira/browse/IMPALA-8671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-8671: - Priority: Critical (was: Major) > Do not re-create a new instance of AuthorizationChecker with Ranger > authorization > - > > Key: IMPALA-8671 > URL: https://issues.apache.org/jira/browse/IMPALA-8671 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Critical > > RangerAuthorizationChecker is stateless unlike SentryAuthorizationChecker. > However, in some cases, we always re-create a new instance of > RangerAuthorizationChecker due to the logic here: > https://github.com/apache/impala/blob/3bd53972cec3b8a863075cb2344115a950be848a/fe/src/main/java/org/apache/impala/service/Frontend.java#L290-L293. > This causes the plugin to be re-initialized, which can be expensive. The > solution is to reuse the same RangerAuthorizationChecker instance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8671) Do not re-create a new instance of AuthorizationChecker with Ranger authorization
Fredy Wijaya created IMPALA-8671: Summary: Do not re-create a new instance of AuthorizationChecker with Ranger authorization Key: IMPALA-8671 URL: https://issues.apache.org/jira/browse/IMPALA-8671 Project: IMPALA Issue Type: Sub-task Components: Frontend Reporter: Fredy Wijaya Assignee: Fredy Wijaya RangerAuthorizationChecker is stateless unlike SentryAuthorizationChecker. However, in some cases, we always re-create a new instance of RangerAuthorizationChecker due to the logic here: https://github.com/apache/impala/blob/3bd53972cec3b8a863075cb2344115a950be848a/fe/src/main/java/org/apache/impala/service/Frontend.java#L290-L293. This causes the plugin to be re-initialized, which can be expensive. The solution is to reuse the same RangerAuthorizationChecker instance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8671) Do not re-create a new instance of AuthorizationChecker with Ranger authorization
Fredy Wijaya created IMPALA-8671: Summary: Do not re-create a new instance of AuthorizationChecker with Ranger authorization Key: IMPALA-8671 URL: https://issues.apache.org/jira/browse/IMPALA-8671 Project: IMPALA Issue Type: Sub-task Components: Frontend Reporter: Fredy Wijaya Assignee: Fredy Wijaya RangerAuthorizationChecker is stateless unlike SentryAuthorizationChecker. However, in some cases, we always re-create a new instance of RangerAuthorizationChecker due to the logic here: https://github.com/apache/impala/blob/3bd53972cec3b8a863075cb2344115a950be848a/fe/src/main/java/org/apache/impala/service/Frontend.java#L290-L293. This causes the plugin to be re-initialized, which can be expensive. The solution is to reuse the same RangerAuthorizationChecker instance. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (IMPALA-8671) Do not re-create a new instance of AuthorizationChecker with Ranger authorization
[ https://issues.apache.org/jira/browse/IMPALA-8671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-8671 started by Fredy Wijaya. > Do not re-create a new instance of AuthorizationChecker with Ranger > authorization > - > > Key: IMPALA-8671 > URL: https://issues.apache.org/jira/browse/IMPALA-8671 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > > RangerAuthorizationChecker is stateless unlike SentryAuthorizationChecker. > However, in some cases, we always re-create a new instance of > RangerAuthorizationChecker due to the logic here: > https://github.com/apache/impala/blob/3bd53972cec3b8a863075cb2344115a950be848a/fe/src/main/java/org/apache/impala/service/Frontend.java#L290-L293. > This causes the plugin to be re-initialized, which can be expensive. The > solution is to reuse the same RangerAuthorizationChecker instance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8670) Restructure Impala Maven projects
[ https://issues.apache.org/jira/browse/IMPALA-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-8670: - Issue Type: Improvement (was: Bug) > Restructure Impala Maven projects > - > > Key: IMPALA-8670 > URL: https://issues.apache.org/jira/browse/IMPALA-8670 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Affects Versions: Impala 3.3.0 >Reporter: Fredy Wijaya >Priority: Major > > impala-parent is the parent for all Impala Maven projects and impala-parent > doesn't include all of its submodule. The goal of this ticket is to refactor > the impala-parent Maven projects. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8670) Restructure Impala Maven projects
Fredy Wijaya created IMPALA-8670: Summary: Restructure Impala Maven projects Key: IMPALA-8670 URL: https://issues.apache.org/jira/browse/IMPALA-8670 Project: IMPALA Issue Type: Bug Components: Infrastructure Affects Versions: Impala 3.3.0 Reporter: Fredy Wijaya impala-parent is the parent for all Impala Maven projects and impala-parent doesn't include all of its submodule. The goal of this ticket is to refactor the impala-parent Maven projects. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8670) Restructure Impala Maven projects
Fredy Wijaya created IMPALA-8670: Summary: Restructure Impala Maven projects Key: IMPALA-8670 URL: https://issues.apache.org/jira/browse/IMPALA-8670 Project: IMPALA Issue Type: Bug Components: Infrastructure Affects Versions: Impala 3.3.0 Reporter: Fredy Wijaya impala-parent is the parent for all Impala Maven projects and impala-parent doesn't include all of its submodule. The goal of this ticket is to refactor the impala-parent Maven projects. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-8662) Timeout in test test_owner_privileges_without_grant
[
https://issues.apache.org/jira/browse/IMPALA-8662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya resolved IMPALA-8662.
--
Resolution: Fixed
I'm not able to reproduce this and it hasn't been an issue so far.
> Timeout in test test_owner_privileges_without_grant
> ---
>
> Key: IMPALA-8662
> URL: https://issues.apache.org/jira/browse/IMPALA-8662
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.3.0
>Reporter: Lars Volker
>Assignee: Fredy Wijaya
>Priority: Blocker
> Labels: broken-build, flaky
>
> test_owner_privileges_without_grant seems to time out:L
> {noformat}
> 11:39:25
> authorization/test_owner_privileges.py::TestOwnerPrivileges::test_owner_privileges_disabled[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none] PASSED
> 11:39:45
> authorization/test_owner_privileges.py::TestOwnerPrivileges::test_owner_privileges_without_grant[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none]
> 11:39:45
> 11:39:45 Tests TIMED OUT!
> 11:39:45
> 11:39:45
> 11:39:45 Generating stacktrace of impalad with process id: 843
> 11:40:09 warning: File
> "/data/jenkins/workspace/impala-asf-master-exhaustive-centos6/Impala-Toolchain/gcc-4.9.2/lib64/libstdc++.so.6.0.20-gdb.py"
> auto-loading has been declined by your `auto-load safe-path' set to
> "$debugdir:$datadir/auto-load".
> 11:40:27 Couldn't write debug register: No such process.
> 11:40:28 Generating stacktrace of impalad with process id: 847
> 11:40:28 ptrace: No such process.
> 11:40:28 Generating stacktrace of impalad with process id: 855
> 11:40:28 ptrace: No such process.
> 11:40:28
> /data/jenkins/workspace/impala-asf-master-exhaustive-centos6/repos/Impala/buildall.sh:
> line 530: 12310 Terminated
> "${IMPALA_HOME}/bin/run-all-tests.sh" -e $EXPLORATION_STRATEGY
> $RUN_ALL_TESTS_ARGS
> 11:40:28 + echo 'buildall.sh ' -format '-testdata failed.'
> 11:40:28 buildall.sh -format -testdata failed.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8662) Timeout in test test_owner_privileges_without_grant
[
https://issues.apache.org/jira/browse/IMPALA-8662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya resolved IMPALA-8662.
--
Resolution: Fixed
I'm not able to reproduce this and it hasn't been an issue so far.
> Timeout in test test_owner_privileges_without_grant
> ---
>
> Key: IMPALA-8662
> URL: https://issues.apache.org/jira/browse/IMPALA-8662
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.3.0
>Reporter: Lars Volker
>Assignee: Fredy Wijaya
>Priority: Blocker
> Labels: broken-build, flaky
>
> test_owner_privileges_without_grant seems to time out:L
> {noformat}
> 11:39:25
> authorization/test_owner_privileges.py::TestOwnerPrivileges::test_owner_privileges_disabled[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none] PASSED
> 11:39:45
> authorization/test_owner_privileges.py::TestOwnerPrivileges::test_owner_privileges_without_grant[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none]
> 11:39:45
> 11:39:45 Tests TIMED OUT!
> 11:39:45
> 11:39:45
> 11:39:45 Generating stacktrace of impalad with process id: 843
> 11:40:09 warning: File
> "/data/jenkins/workspace/impala-asf-master-exhaustive-centos6/Impala-Toolchain/gcc-4.9.2/lib64/libstdc++.so.6.0.20-gdb.py"
> auto-loading has been declined by your `auto-load safe-path' set to
> "$debugdir:$datadir/auto-load".
> 11:40:27 Couldn't write debug register: No such process.
> 11:40:28 Generating stacktrace of impalad with process id: 847
> 11:40:28 ptrace: No such process.
> 11:40:28 Generating stacktrace of impalad with process id: 855
> 11:40:28 ptrace: No such process.
> 11:40:28
> /data/jenkins/workspace/impala-asf-master-exhaustive-centos6/repos/Impala/buildall.sh:
> line 530: 12310 Terminated
> "${IMPALA_HOME}/bin/run-all-tests.sh" -e $EXPLORATION_STRATEGY
> $RUN_ALL_TESTS_ARGS
> 11:40:28 + echo 'buildall.sh ' -format '-testdata failed.'
> 11:40:28 buildall.sh -format -testdata failed.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (IMPALA-8599) Create a separate Maven module for query event hook API
[ https://issues.apache.org/jira/browse/IMPALA-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-8599 started by Fredy Wijaya. > Create a separate Maven module for query event hook API > --- > > Key: IMPALA-8599 > URL: https://issues.apache.org/jira/browse/IMPALA-8599 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Critical > > Impala needs to publish this API into Maven Central so that Atlas can consume > it. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8654) Log the SQL statement in Ranger audit log
[ https://issues.apache.org/jira/browse/IMPALA-8654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8654. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Log the SQL statement in Ranger audit log > - > > Key: IMPALA-8654 > URL: https://issues.apache.org/jira/browse/IMPALA-8654 > Project: IMPALA > Issue Type: Sub-task > Components: Catalog, Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Critical > Fix For: Impala 3.3.0 > > > "requestData" a required field in Ranger audit log and is used for logging > the SQL statement authorized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-8654) Log the SQL statement in Ranger audit log
[ https://issues.apache.org/jira/browse/IMPALA-8654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8654. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Log the SQL statement in Ranger audit log > - > > Key: IMPALA-8654 > URL: https://issues.apache.org/jira/browse/IMPALA-8654 > Project: IMPALA > Issue Type: Sub-task > Components: Catalog, Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Critical > Fix For: Impala 3.3.0 > > > "requestData" a required field in Ranger audit log and is used for logging > the SQL statement authorized. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8654) Log the SQL statement in Ranger audit log
[ https://issues.apache.org/jira/browse/IMPALA-8654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-8654: - Priority: Critical (was: Major) > Log the SQL statement in Ranger audit log > - > > Key: IMPALA-8654 > URL: https://issues.apache.org/jira/browse/IMPALA-8654 > Project: IMPALA > Issue Type: Sub-task > Components: Catalog, Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Critical > > "requestData" a required field in Ranger audit log and is used for logging > the SQL statement authorized. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8658) Populate required Ranger audit fields to be consistent with Hive
[ https://issues.apache.org/jira/browse/IMPALA-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-8658: - Priority: Critical (was: Major) > Populate required Ranger audit fields to be consistent with Hive > > > Key: IMPALA-8658 > URL: https://issues.apache.org/jira/browse/IMPALA-8658 > Project: IMPALA > Issue Type: Sub-task > Components: Catalog, Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Critical > > Some required fields: > - Client IP > - Cluster name > - Access type (will need to be in upper case) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8588) FIx revoke grant option behavior
[ https://issues.apache.org/jira/browse/IMPALA-8588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8588. -- Resolution: Fixed > FIx revoke grant option behavior > > > Key: IMPALA-8588 > URL: https://issues.apache.org/jira/browse/IMPALA-8588 > Project: IMPALA > Issue Type: Sub-task > Components: Catalog, Frontend >Reporter: Austin Nobis >Assignee: Austin Nobis >Priority: Critical > Fix For: Impala 3.3.0 > > > Given permissions have been granted via: > *grant select on database functional for user admin with grant option;* > > > Currently, running the following statement will remove the *grant option*: > *revoke select on database functional from user admin;* > > The following statement will also not remove the *grant option:* > *revoke grant option for select on database functional from user admin;* > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8588) FIx revoke grant option behavior
[ https://issues.apache.org/jira/browse/IMPALA-8588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8588. -- Resolution: Fixed > FIx revoke grant option behavior > > > Key: IMPALA-8588 > URL: https://issues.apache.org/jira/browse/IMPALA-8588 > Project: IMPALA > Issue Type: Sub-task > Components: Catalog, Frontend >Reporter: Austin Nobis >Assignee: Austin Nobis >Priority: Critical > Fix For: Impala 3.3.0 > > > Given permissions have been granted via: > *grant select on database functional for user admin with grant option;* > > > Currently, running the following statement will remove the *grant option*: > *revoke select on database functional from user admin;* > > The following statement will also not remove the *grant option:* > *revoke grant option for select on database functional from user admin;* > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-8649) Confusing error messages in SHOW GRANT statements
[ https://issues.apache.org/jira/browse/IMPALA-8649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8649. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Confusing error messages in SHOW GRANT statements > - > > Key: IMPALA-8649 > URL: https://issues.apache.org/jira/browse/IMPALA-8649 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Affects Versions: Impala 3.2.0 >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.3.0 > > > Some error messages in PrivilegeSpec are confusing. For example: > https://github.com/apache/impala/blob/d753600f9cf4bda0c955f0f5eecc6fb41b0c365e/fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java#L204-L206 > https://github.com/apache/impala/blob/d753600f9cf4bda0c955f0f5eecc6fb41b0c365e/fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java#L254-L258 > https://github.com/apache/impala/blob/d753600f9cf4bda0c955f0f5eecc6fb41b0c365e/fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java#L287-L289 > We should still aim to not reveal the existence of an object, however the > message "Error setting privileges" and "Verify that you have permissions to > issue GRANT/REVOKE statement" can be misleading since PrivilegeSpec is also > used for "SHOW GRANT", which doesn't set any privileges. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-8649) Confusing error messages in SHOW GRANT statements
[ https://issues.apache.org/jira/browse/IMPALA-8649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8649. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Confusing error messages in SHOW GRANT statements > - > > Key: IMPALA-8649 > URL: https://issues.apache.org/jira/browse/IMPALA-8649 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Affects Versions: Impala 3.2.0 >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.3.0 > > > Some error messages in PrivilegeSpec are confusing. For example: > https://github.com/apache/impala/blob/d753600f9cf4bda0c955f0f5eecc6fb41b0c365e/fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java#L204-L206 > https://github.com/apache/impala/blob/d753600f9cf4bda0c955f0f5eecc6fb41b0c365e/fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java#L254-L258 > https://github.com/apache/impala/blob/d753600f9cf4bda0c955f0f5eecc6fb41b0c365e/fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java#L287-L289 > We should still aim to not reveal the existence of an object, however the > message "Error setting privileges" and "Verify that you have permissions to > issue GRANT/REVOKE statement" can be misleading since PrivilegeSpec is also > used for "SHOW GRANT", which doesn't set any privileges. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8655) Some SQL statements do not have proper toSql implementation
[ https://issues.apache.org/jira/browse/IMPALA-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-8655: - Priority: Critical (was: Major) > Some SQL statements do not have proper toSql implementation > --- > > Key: IMPALA-8655 > URL: https://issues.apache.org/jira/browse/IMPALA-8655 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 3.0 >Reporter: Fredy Wijaya >Priority: Critical > > All Impala SQL statements inherit from StatementBase that has an empty string > toSql() implementation. As a result, it is easy to forget implementing > toSql() when adding a new SQL statement. Some existing statements do not have > proper toSql implementation. Having each statement to have proper toSql() > implementation is important for get proper logging, auditing, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-8658) Populate required Ranger audit fields to be consistent with Hive
[ https://issues.apache.org/jira/browse/IMPALA-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-8658 started by Fredy Wijaya. > Populate required Ranger audit fields to be consistent with Hive > > > Key: IMPALA-8658 > URL: https://issues.apache.org/jira/browse/IMPALA-8658 > Project: IMPALA > Issue Type: Sub-task > Components: Catalog, Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > > Some required fields: > - Client IP > - Cluster name > - Access type (will need to be in upper case) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8658) Populate required Ranger audit fields to be consistent with Hive
Fredy Wijaya created IMPALA-8658: Summary: Populate required Ranger audit fields to be consistent with Hive Key: IMPALA-8658 URL: https://issues.apache.org/jira/browse/IMPALA-8658 Project: IMPALA Issue Type: Sub-task Components: Catalog, Frontend Reporter: Fredy Wijaya Assignee: Fredy Wijaya Some required fields: - Client IP - Cluster name - Access type (will need to be in upper case) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8658) Populate required Ranger audit fields to be consistent with Hive
Fredy Wijaya created IMPALA-8658: Summary: Populate required Ranger audit fields to be consistent with Hive Key: IMPALA-8658 URL: https://issues.apache.org/jira/browse/IMPALA-8658 Project: IMPALA Issue Type: Sub-task Components: Catalog, Frontend Reporter: Fredy Wijaya Assignee: Fredy Wijaya Some required fields: - Client IP - Cluster name - Access type (will need to be in upper case) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-8655) Some SQL statements do not have proper toSql implementation
Fredy Wijaya created IMPALA-8655: Summary: Some SQL statements do not have proper toSql implementation Key: IMPALA-8655 URL: https://issues.apache.org/jira/browse/IMPALA-8655 Project: IMPALA Issue Type: Bug Components: Frontend Affects Versions: Impala 3.0 Reporter: Fredy Wijaya All Impala SQL statements inherit from StatementBase that has an empty string toSql() implementation. As a result, it is easy to forget implementing toSql() when adding a new SQL statement. Some existing statements do not have proper toSql implementation. Having each statement to have proper toSql() implementation is important for get proper logging, auditing, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-8655) Some SQL statements do not have proper toSql implementation
Fredy Wijaya created IMPALA-8655: Summary: Some SQL statements do not have proper toSql implementation Key: IMPALA-8655 URL: https://issues.apache.org/jira/browse/IMPALA-8655 Project: IMPALA Issue Type: Bug Components: Frontend Affects Versions: Impala 3.0 Reporter: Fredy Wijaya All Impala SQL statements inherit from StatementBase that has an empty string toSql() implementation. As a result, it is easy to forget implementing toSql() when adding a new SQL statement. Some existing statements do not have proper toSql implementation. Having each statement to have proper toSql() implementation is important for get proper logging, auditing, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-8654) Log the SQL statement in Ranger audit log
[ https://issues.apache.org/jira/browse/IMPALA-8654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-8654 started by Fredy Wijaya. > Log the SQL statement in Ranger audit log > - > > Key: IMPALA-8654 > URL: https://issues.apache.org/jira/browse/IMPALA-8654 > Project: IMPALA > Issue Type: Sub-task > Components: Catalog, Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > > "requestData" a required field in Ranger audit log and is used for logging > the SQL statement authorized. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8654) Log the SQL statement in Ranger audit log
Fredy Wijaya created IMPALA-8654: Summary: Log the SQL statement in Ranger audit log Key: IMPALA-8654 URL: https://issues.apache.org/jira/browse/IMPALA-8654 Project: IMPALA Issue Type: Sub-task Components: Catalog, Frontend Reporter: Fredy Wijaya Assignee: Fredy Wijaya "requestData" a required field in Ranger audit log and is used for logging the SQL statement authorized. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8654) Log the SQL statement in Ranger audit log
Fredy Wijaya created IMPALA-8654: Summary: Log the SQL statement in Ranger audit log Key: IMPALA-8654 URL: https://issues.apache.org/jira/browse/IMPALA-8654 Project: IMPALA Issue Type: Sub-task Components: Catalog, Frontend Reporter: Fredy Wijaya Assignee: Fredy Wijaya "requestData" a required field in Ranger audit log and is used for logging the SQL statement authorized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IMPALA-8649) Confusing error messages in SHOW GRANT statements
[ https://issues.apache.org/jira/browse/IMPALA-8649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-8649: - Summary: Confusing error messages in SHOW GRANT statements (was: Confusing error messages in PrivilegeSpec) > Confusing error messages in SHOW GRANT statements > - > > Key: IMPALA-8649 > URL: https://issues.apache.org/jira/browse/IMPALA-8649 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Affects Versions: Impala 3.2.0 >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > > Some error messages in PrivilegeSpec are confusing. For example: > https://github.com/apache/impala/blob/d753600f9cf4bda0c955f0f5eecc6fb41b0c365e/fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java#L204-L206 > https://github.com/apache/impala/blob/d753600f9cf4bda0c955f0f5eecc6fb41b0c365e/fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java#L254-L258 > https://github.com/apache/impala/blob/d753600f9cf4bda0c955f0f5eecc6fb41b0c365e/fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java#L287-L289 > We should still aim to not reveal the existence of an object, however the > message "Error setting privileges" and "Verify that you have permissions to > issue GRANT/REVOKE statement" can be misleading since PrivilegeSpec is also > used for "SHOW GRANT", which doesn't set any privileges. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-8649) Confusing error messages in PrivilegeSpec
[ https://issues.apache.org/jira/browse/IMPALA-8649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-8649 started by Fredy Wijaya. > Confusing error messages in PrivilegeSpec > - > > Key: IMPALA-8649 > URL: https://issues.apache.org/jira/browse/IMPALA-8649 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Affects Versions: Impala 3.2.0 >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > > Some error messages in PrivilegeSpec are confusing. For example: > https://github.com/apache/impala/blob/d753600f9cf4bda0c955f0f5eecc6fb41b0c365e/fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java#L204-L206 > https://github.com/apache/impala/blob/d753600f9cf4bda0c955f0f5eecc6fb41b0c365e/fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java#L254-L258 > https://github.com/apache/impala/blob/d753600f9cf4bda0c955f0f5eecc6fb41b0c365e/fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java#L287-L289 > We should still aim to not reveal the existence of an object, however the > message "Error setting privileges" and "Verify that you have permissions to > issue GRANT/REVOKE statement" can be misleading since PrivilegeSpec is also > used for "SHOW GRANT", which doesn't set any privileges. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8649) Confusing error messages in PrivilegeSpec
Fredy Wijaya created IMPALA-8649: Summary: Confusing error messages in PrivilegeSpec Key: IMPALA-8649 URL: https://issues.apache.org/jira/browse/IMPALA-8649 Project: IMPALA Issue Type: Sub-task Components: Frontend Affects Versions: Impala 3.2.0 Reporter: Fredy Wijaya Some error messages in PrivilegeSpec are confusing. For example: https://github.com/apache/impala/blob/d753600f9cf4bda0c955f0f5eecc6fb41b0c365e/fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java#L204-L206 https://github.com/apache/impala/blob/d753600f9cf4bda0c955f0f5eecc6fb41b0c365e/fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java#L254-L258 https://github.com/apache/impala/blob/d753600f9cf4bda0c955f0f5eecc6fb41b0c365e/fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java#L287-L289 We should still aim to not reveal the existence of an object, however the message "Error setting privileges" and "Verify that you have permissions to issue GRANT/REVOKE statement" can be misleading since PrivilegeSpec is also used for "SHOW GRANT", which doesn't set any privileges. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-8649) Confusing error messages in PrivilegeSpec
Fredy Wijaya created IMPALA-8649: Summary: Confusing error messages in PrivilegeSpec Key: IMPALA-8649 URL: https://issues.apache.org/jira/browse/IMPALA-8649 Project: IMPALA Issue Type: Sub-task Components: Frontend Affects Versions: Impala 3.2.0 Reporter: Fredy Wijaya Some error messages in PrivilegeSpec are confusing. For example: https://github.com/apache/impala/blob/d753600f9cf4bda0c955f0f5eecc6fb41b0c365e/fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java#L204-L206 https://github.com/apache/impala/blob/d753600f9cf4bda0c955f0f5eecc6fb41b0c365e/fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java#L254-L258 https://github.com/apache/impala/blob/d753600f9cf4bda0c955f0f5eecc6fb41b0c365e/fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java#L287-L289 We should still aim to not reveal the existence of an object, however the message "Error setting privileges" and "Verify that you have permissions to issue GRANT/REVOKE statement" can be misleading since PrivilegeSpec is also used for "SHOW GRANT", which doesn't set any privileges. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8551) GRANT gives confusing error message
[ https://issues.apache.org/jira/browse/IMPALA-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8551. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > GRANT gives confusing error message > --- > > Key: IMPALA-8551 > URL: https://issues.apache.org/jira/browse/IMPALA-8551 > Project: IMPALA > Issue Type: Sub-task > Components: Security >Affects Versions: Impala 3.3.0 >Reporter: Todd Lipcon >Assignee: Fredy Wijaya >Priority: Critical > Fix For: Impala 3.3.0 > > > Tried performing a grant from the Impala shell in a kerberized cluster with > Ranger enabled and got a strange error message: > ```[nightly7x-2:21000] default> grant all on table t to group asdf; > Query: grant all on table t to group asdf > Query submitted at: 2019-05-15 11:24:20 (Coordinator: > https://nightly7x-2.vpc.cloudera.com:25000) > ERROR: InternalException: HTTP 400 Error: syst...@vpc.cloudera.com is Not > Found > ``` > Not sure what's going on here -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8551) GRANT gives confusing error message
[ https://issues.apache.org/jira/browse/IMPALA-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8551. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > GRANT gives confusing error message > --- > > Key: IMPALA-8551 > URL: https://issues.apache.org/jira/browse/IMPALA-8551 > Project: IMPALA > Issue Type: Sub-task > Components: Security >Affects Versions: Impala 3.3.0 >Reporter: Todd Lipcon >Assignee: Fredy Wijaya >Priority: Critical > Fix For: Impala 3.3.0 > > > Tried performing a grant from the Impala shell in a kerberized cluster with > Ranger enabled and got a strange error message: > ```[nightly7x-2:21000] default> grant all on table t to group asdf; > Query: grant all on table t to group asdf > Query submitted at: 2019-05-15 11:24:20 (Coordinator: > https://nightly7x-2.vpc.cloudera.com:25000) > ERROR: InternalException: HTTP 400 Error: syst...@vpc.cloudera.com is Not > Found > ``` > Not sure what's going on here -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IMPALA-8599) Create a separate Maven module for query event hook API
[ https://issues.apache.org/jira/browse/IMPALA-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-8599: - Priority: Critical (was: Major) > Create a separate Maven module for query event hook API > --- > > Key: IMPALA-8599 > URL: https://issues.apache.org/jira/browse/IMPALA-8599 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Critical > > Impala needs to publish this API into Maven Central so that Atlas can consume > it. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8576) Pass lineage object instead of string to query hook
[
https://issues.apache.org/jira/browse/IMPALA-8576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya updated IMPALA-8576:
-
Priority: Critical (was: Major)
> Pass lineage object instead of string to query hook
> ---
>
> Key: IMPALA-8576
> URL: https://issues.apache.org/jira/browse/IMPALA-8576
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend, Frontend
>Reporter: radford nguyen
>Priority: Critical
>
> The {{QueryEventHook}} interface currently takes a {{String}} for the
> {{onQueryComplete}} hook. This string is the JSON representation of the
> lineage graph written to the legacy lineage file.
> It would be better to pass the serialized {{byte[]}} of the lineage thrift
> object itself, so that we can decouple ourselves from any lineage file
> format(s).
> Additionally, hook implementations should use their own version of Thrift to
> deserialize the object so that they are not tied to Impala's Thrift version.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8638) Test failure: test_create_table_timestamp
[
https://issues.apache.org/jira/browse/IMPALA-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya resolved IMPALA-8638.
--
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> Test failure: test_create_table_timestamp
> -
>
> Key: IMPALA-8638
> URL: https://issues.apache.org/jira/browse/IMPALA-8638
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.3.0
>Reporter: Csaba Ringhofer
>Assignee: Fredy Wijaya
>Priority: Blocker
> Labels: broken-build
> Fix For: Impala 3.3.0
>
>
> I saw two recent failures of this test, on without proper explanation for the
> error, and one with the following message:
> 04:41:33.324 FAIL
> custom_cluster/test_lineage.py::TestLineage::()::test_create_table_timestamp[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none]
> 04:41:33.324 === FAILURES
> ===
> 04:41:33.341 TestLineage.test_create_table_timestamp[protocol: beeswax |
> exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none]
> 04:41:33.341 custom_cluster/test_lineage.py:98: in test_create_table_timestamp
> 04:41:33.342 assert lineage_json["queryId"] == profile_query_id
> 04:41:33.342 E assert '75440fbf4d05...d1bd9' ==
> 'fd4d30a1d0696...2b93c'
> 04:41:33.342 E - 75440fbf4d05a7a7:832d1bd9
> 04:41:33.342 E + fd4d30a1d06964f4:99e2b93c
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IMPALA-8638) Test failure: test_create_table_timestamp
[
https://issues.apache.org/jira/browse/IMPALA-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya resolved IMPALA-8638.
--
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> Test failure: test_create_table_timestamp
> -
>
> Key: IMPALA-8638
> URL: https://issues.apache.org/jira/browse/IMPALA-8638
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.3.0
>Reporter: Csaba Ringhofer
>Assignee: Fredy Wijaya
>Priority: Blocker
> Labels: broken-build
> Fix For: Impala 3.3.0
>
>
> I saw two recent failures of this test, on without proper explanation for the
> error, and one with the following message:
> 04:41:33.324 FAIL
> custom_cluster/test_lineage.py::TestLineage::()::test_create_table_timestamp[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none]
> 04:41:33.324 === FAILURES
> ===
> 04:41:33.341 TestLineage.test_create_table_timestamp[protocol: beeswax |
> exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none]
> 04:41:33.341 custom_cluster/test_lineage.py:98: in test_create_table_timestamp
> 04:41:33.342 assert lineage_json["queryId"] == profile_query_id
> 04:41:33.342 E assert '75440fbf4d05...d1bd9' ==
> 'fd4d30a1d0696...2b93c'
> 04:41:33.342 E - 75440fbf4d05a7a7:832d1bd9
> 04:41:33.342 E + fd4d30a1d06964f4:99e2b93c
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-8638) Test failure: test_create_table_timestamp
[
https://issues.apache.org/jira/browse/IMPALA-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-8638 started by Fredy Wijaya.
> Test failure: test_create_table_timestamp
> -
>
> Key: IMPALA-8638
> URL: https://issues.apache.org/jira/browse/IMPALA-8638
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.3.0
>Reporter: Csaba Ringhofer
>Assignee: Fredy Wijaya
>Priority: Blocker
> Labels: broken-build
>
> I saw two recent failures of this test, on without proper explanation for the
> error, and one with the following message:
> 04:41:33.324 FAIL
> custom_cluster/test_lineage.py::TestLineage::()::test_create_table_timestamp[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none]
> 04:41:33.324 === FAILURES
> ===
> 04:41:33.341 TestLineage.test_create_table_timestamp[protocol: beeswax |
> exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none]
> 04:41:33.341 custom_cluster/test_lineage.py:98: in test_create_table_timestamp
> 04:41:33.342 assert lineage_json["queryId"] == profile_query_id
> 04:41:33.342 E assert '75440fbf4d05...d1bd9' ==
> 'fd4d30a1d0696...2b93c'
> 04:41:33.342 E - 75440fbf4d05a7a7:832d1bd9
> 04:41:33.342 E + fd4d30a1d06964f4:99e2b93c
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8640) ImpalaTestSuite::create_impala_client() user session is sticky
[
https://issues.apache.org/jira/browse/IMPALA-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya resolved IMPALA-8640.
--
Resolution: Won't Fix
> ImpalaTestSuite::create_impala_client() user session is sticky
> --
>
> Key: IMPALA-8640
> URL: https://issues.apache.org/jira/browse/IMPALA-8640
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Reporter: Fredy Wijaya
>Priority: Major
> Fix For: Impala 3.3.0
>
>
> ImpalaTestSuite::create_impala_client() user session is sticky, which means
> {noformat}
> client = self.create_impala_client()
> result = self.execute_query_expect_failure(client, statement, user="foo") -->
> the client will be bound to user "foo"
> result = self.execute_query_expect_failure(client, statement, user="bar") -->
> the client will still be bound to user "foo" even though we specify user="bar"
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8640) ImpalaTestSuite::create_impala_client() user session is sticky
[
https://issues.apache.org/jira/browse/IMPALA-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya resolved IMPALA-8640.
--
Resolution: Won't Fix
> ImpalaTestSuite::create_impala_client() user session is sticky
> --
>
> Key: IMPALA-8640
> URL: https://issues.apache.org/jira/browse/IMPALA-8640
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Reporter: Fredy Wijaya
>Priority: Major
> Fix For: Impala 3.3.0
>
>
> ImpalaTestSuite::create_impala_client() user session is sticky, which means
> {noformat}
> client = self.create_impala_client()
> result = self.execute_query_expect_failure(client, statement, user="foo") -->
> the client will be bound to user "foo"
> result = self.execute_query_expect_failure(client, statement, user="bar") -->
> the client will still be bound to user "foo" even though we specify user="bar"
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IMPALA-8640) ImpalaTestSuite::create_impala_client() user session is sticky
[
https://issues.apache.org/jira/browse/IMPALA-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858801#comment-16858801
]
Fredy Wijaya commented on IMPALA-8640:
--
[~tarmstrong] Thanks for the explanation! I'll close this ticket as "Won't Fix".
> ImpalaTestSuite::create_impala_client() user session is sticky
> --
>
> Key: IMPALA-8640
> URL: https://issues.apache.org/jira/browse/IMPALA-8640
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Reporter: Fredy Wijaya
>Priority: Major
> Fix For: Impala 3.3.0
>
>
> ImpalaTestSuite::create_impala_client() user session is sticky, which means
> {noformat}
> client = self.create_impala_client()
> result = self.execute_query_expect_failure(client, statement, user="foo") -->
> the client will be bound to user "foo"
> result = self.execute_query_expect_failure(client, statement, user="bar") -->
> the client will still be bound to user "foo" even though we specify user="bar"
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8640) ImpalaTestSuite::create_impala_client() user session is sticky
Fredy Wijaya created IMPALA-8640:
Summary: ImpalaTestSuite::create_impala_client() user session is
sticky
Key: IMPALA-8640
URL: https://issues.apache.org/jira/browse/IMPALA-8640
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Reporter: Fredy Wijaya
Fix For: Impala 3.3.0
ImpalaTestSuite::create_impala_client() user session is sticky, which means
{noformat}
client = self.create_impala_client()
result = self.execute_query_expect_failure(client, statement, user="foo") -->
the client will be bound to user "foo"
result = self.execute_query_expect_failure(client, statement, user="bar") -->
the client will still be bound to user "foo" even though we specify user="bar"
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IMPALA-8640) ImpalaTestSuite::create_impala_client() user session is sticky
Fredy Wijaya created IMPALA-8640:
Summary: ImpalaTestSuite::create_impala_client() user session is
sticky
Key: IMPALA-8640
URL: https://issues.apache.org/jira/browse/IMPALA-8640
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Reporter: Fredy Wijaya
Fix For: Impala 3.3.0
ImpalaTestSuite::create_impala_client() user session is sticky, which means
{noformat}
client = self.create_impala_client()
result = self.execute_query_expect_failure(client, statement, user="foo") -->
the client will be bound to user "foo"
result = self.execute_query_expect_failure(client, statement, user="bar") -->
the client will still be bound to user "foo" even though we specify user="bar"
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-8599) Create a separate Maven module for query event hook API
[ https://issues.apache.org/jira/browse/IMPALA-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya reassigned IMPALA-8599: Assignee: Fredy Wijaya > Create a separate Maven module for query event hook API > --- > > Key: IMPALA-8599 > URL: https://issues.apache.org/jira/browse/IMPALA-8599 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > > Impala needs to publish this API into Maven Central so that Atlas can consume > it. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8564) Add table's createTime information in the column lineage information
[ https://issues.apache.org/jira/browse/IMPALA-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8564. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Add table's createTime information in the column lineage information > > > Key: IMPALA-8564 > URL: https://issues.apache.org/jira/browse/IMPALA-8564 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Affects Versions: Impala 3.2.0 >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.3.0 > > > This is needed for https://issues.apache.org/jira/browse/ATLAS-3080 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8604) Improve authorization test coverage for update/upsert/delete statements
[ https://issues.apache.org/jira/browse/IMPALA-8604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8604. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Improve authorization test coverage for update/upsert/delete statements > --- > > Key: IMPALA-8604 > URL: https://issues.apache.org/jira/browse/IMPALA-8604 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Affects Versions: Impala 3.2.0 >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.3.0 > > > Currently the authorization test code for update/insert/delete statements > only test against "server" scope which can give an illusion that an ALL > privilege on "server" is required for those statements. The privilege scope > for those statements are at the "table"-level, so we need to make sure we > have tests for that. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8604) Improve authorization test coverage for update/upsert/delete statements
[ https://issues.apache.org/jira/browse/IMPALA-8604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-8604. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Improve authorization test coverage for update/upsert/delete statements > --- > > Key: IMPALA-8604 > URL: https://issues.apache.org/jira/browse/IMPALA-8604 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Affects Versions: Impala 3.2.0 >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.3.0 > > > Currently the authorization test code for update/insert/delete statements > only test against "server" scope which can give an illusion that an ALL > privilege on "server" is required for those statements. The privilege scope > for those statements are at the "table"-level, so we need to make sure we > have tests for that. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-8604) Improve authorization test coverage for update/upsert/delete statements
Fredy Wijaya created IMPALA-8604: Summary: Improve authorization test coverage for update/upsert/delete statements Key: IMPALA-8604 URL: https://issues.apache.org/jira/browse/IMPALA-8604 Project: IMPALA Issue Type: Improvement Components: Infrastructure Affects Versions: Impala 3.2.0 Reporter: Fredy Wijaya Assignee: Fredy Wijaya Currently the authorization test code for update/insert/delete statements only test against "server" scope which can give an illusion that an ALL privilege on "server" is required for those statements. The privilege scope for those statements are at the "table"-level, so we need to make sure we have tests for that. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8604) Improve authorization test coverage for update/upsert/delete statements
Fredy Wijaya created IMPALA-8604: Summary: Improve authorization test coverage for update/upsert/delete statements Key: IMPALA-8604 URL: https://issues.apache.org/jira/browse/IMPALA-8604 Project: IMPALA Issue Type: Improvement Components: Infrastructure Affects Versions: Impala 3.2.0 Reporter: Fredy Wijaya Assignee: Fredy Wijaya Currently the authorization test code for update/insert/delete statements only test against "server" scope which can give an illusion that an ALL privilege on "server" is required for those statements. The privilege scope for those statements are at the "table"-level, so we need to make sure we have tests for that. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IMPALA-8576) Pass lineage object instead of string to query hook
[ https://issues.apache.org/jira/browse/IMPALA-8576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-8576: - Issue Type: Sub-task (was: Improvement) Parent: IMPALA-8598 > Pass lineage object instead of string to query hook > --- > > Key: IMPALA-8576 > URL: https://issues.apache.org/jira/browse/IMPALA-8576 > Project: IMPALA > Issue Type: Sub-task > Components: Backend, Frontend >Reporter: radford nguyen >Priority: Major > > Placeholder: description coming soon -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8599) Create a separate Maven module for query event hook API
[ https://issues.apache.org/jira/browse/IMPALA-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-8599: - Description: Impala needs to publish this API into Maven Central so that Atlas can consume it. > Create a separate Maven module for query event hook API > --- > > Key: IMPALA-8599 > URL: https://issues.apache.org/jira/browse/IMPALA-8599 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Priority: Major > > Impala needs to publish this API into Maven Central so that Atlas can consume > it. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8599) Create a separate Maven module for query event hook API
Fredy Wijaya created IMPALA-8599: Summary: Create a separate Maven module for query event hook API Key: IMPALA-8599 URL: https://issues.apache.org/jira/browse/IMPALA-8599 Project: IMPALA Issue Type: Sub-task Components: Frontend Reporter: Fredy Wijaya -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8599) Create a separate Maven module for query event hook API
Fredy Wijaya created IMPALA-8599: Summary: Create a separate Maven module for query event hook API Key: IMPALA-8599 URL: https://issues.apache.org/jira/browse/IMPALA-8599 Project: IMPALA Issue Type: Sub-task Components: Frontend Reporter: Fredy Wijaya -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-8473) Refactor lineage publication mechanism to allow for different consumers
[
https://issues.apache.org/jira/browse/IMPALA-8473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya resolved IMPALA-8473.
--
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> Refactor lineage publication mechanism to allow for different consumers
> ---
>
> Key: IMPALA-8473
> URL: https://issues.apache.org/jira/browse/IMPALA-8473
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend, Frontend
>Reporter: radford nguyen
>Assignee: radford nguyen
>Priority: Critical
> Fix For: Impala 3.3.0
>
> Attachments: ImpalaPostExecHook-infra.patch
>
>
> Impetus for this change is to allow lineage to be consumed by Atlas via Kafka.
> h3. Design Proposal
> Implement a plugin approach (similar to {{authorization_provider}}) for
> consuming query event hooks, where downstream users can provide their own
> hook implementations as runtime dependencies.
> Keep but deprecate existing lineage event file writing.
> [~mad...@apache.org] has provided a fe patch (attached) with suggested
> mechanism for allowing multiple hooks to be registered with the fe. Hooks
> would be invoked from the be at appropriate places, e.g.
> [https://github.com/apache/impala/blob/c1b0a073938c144e9bf33901bd4df6dcda0f09ec/be/src/service/impala-server.cc#L466].
> The hooks should all be executed asynchronously, so the current thinking is
> that this execution should happen in the fe, since the be does not know about
> what hooks are registered. IOW, the
> {{ImpalaPostExecHookFactory.executeHooks}} method (see patch) should probably
> make use of a thread-pool executor service (or something similar) in order to
> execute all hooks in parallel and in a non-blocking manner, returning to the
> be asap.
>
> h3. Code Review
> [https://gerrit.cloudera.org/#/c/13352/]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8473) Refactor lineage publication mechanism to allow for different consumers
[
https://issues.apache.org/jira/browse/IMPALA-8473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya resolved IMPALA-8473.
--
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> Refactor lineage publication mechanism to allow for different consumers
> ---
>
> Key: IMPALA-8473
> URL: https://issues.apache.org/jira/browse/IMPALA-8473
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend, Frontend
>Reporter: radford nguyen
>Assignee: radford nguyen
>Priority: Critical
> Fix For: Impala 3.3.0
>
> Attachments: ImpalaPostExecHook-infra.patch
>
>
> Impetus for this change is to allow lineage to be consumed by Atlas via Kafka.
> h3. Design Proposal
> Implement a plugin approach (similar to {{authorization_provider}}) for
> consuming query event hooks, where downstream users can provide their own
> hook implementations as runtime dependencies.
> Keep but deprecate existing lineage event file writing.
> [~mad...@apache.org] has provided a fe patch (attached) with suggested
> mechanism for allowing multiple hooks to be registered with the fe. Hooks
> would be invoked from the be at appropriate places, e.g.
> [https://github.com/apache/impala/blob/c1b0a073938c144e9bf33901bd4df6dcda0f09ec/be/src/service/impala-server.cc#L466].
> The hooks should all be executed asynchronously, so the current thinking is
> that this execution should happen in the fe, since the be does not know about
> what hooks are registered. IOW, the
> {{ImpalaPostExecHookFactory.executeHooks}} method (see patch) should probably
> make use of a thread-pool executor service (or something similar) in order to
> execute all hooks in parallel and in a non-blocking manner, returning to the
> be asap.
>
> h3. Code Review
> [https://gerrit.cloudera.org/#/c/13352/]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IMPALA-8572) Move query hook execution to before query unregistration
[ https://issues.apache.org/jira/browse/IMPALA-8572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-8572: - Issue Type: Sub-task (was: Improvement) Parent: IMPALA-8598 > Move query hook execution to before query unregistration > > > Key: IMPALA-8572 > URL: https://issues.apache.org/jira/browse/IMPALA-8572 > Project: IMPALA > Issue Type: Sub-task > Components: Backend >Reporter: radford nguyen >Priority: Major > > Placeholder: description coming soon -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8571) Make query-hook-execution more robust and observable
[ https://issues.apache.org/jira/browse/IMPALA-8571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-8571: - Issue Type: Sub-task (was: Improvement) Parent: IMPALA-8598 > Make query-hook-execution more robust and observable > > > Key: IMPALA-8571 > URL: https://issues.apache.org/jira/browse/IMPALA-8571 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: radford nguyen >Assignee: radford nguyen >Priority: Major > > Placeholder: description coming soon -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8473) Refactor lineage publication mechanism to allow for different consumers
[
https://issues.apache.org/jira/browse/IMPALA-8473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya updated IMPALA-8473:
-
Issue Type: Sub-task (was: Improvement)
Parent: IMPALA-8598
> Refactor lineage publication mechanism to allow for different consumers
> ---
>
> Key: IMPALA-8473
> URL: https://issues.apache.org/jira/browse/IMPALA-8473
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend, Frontend
>Reporter: radford nguyen
>Assignee: radford nguyen
>Priority: Critical
> Attachments: ImpalaPostExecHook-infra.patch
>
>
> Impetus for this change is to allow lineage to be consumed by Atlas via Kafka.
> h3. Design Proposal
> Implement a plugin approach (similar to {{authorization_provider}}) for
> consuming query event hooks, where downstream users can provide their own
> hook implementations as runtime dependencies.
> Keep but deprecate existing lineage event file writing.
> [~mad...@apache.org] has provided a fe patch (attached) with suggested
> mechanism for allowing multiple hooks to be registered with the fe. Hooks
> would be invoked from the be at appropriate places, e.g.
> [https://github.com/apache/impala/blob/c1b0a073938c144e9bf33901bd4df6dcda0f09ec/be/src/service/impala-server.cc#L466].
> The hooks should all be executed asynchronously, so the current thinking is
> that this execution should happen in the fe, since the be does not know about
> what hooks are registered. IOW, the
> {{ImpalaPostExecHookFactory.executeHooks}} method (see patch) should probably
> make use of a thread-pool executor service (or something similar) in order to
> execute all hooks in parallel and in a non-blocking manner, returning to the
> be asap.
>
> h3. Code Review
> [https://gerrit.cloudera.org/#/c/13352/]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
