date:20190514

[jira] [Resolved] (IMPALA-966) Type errors are attributed to wrong expression with insert

2019-05-14 Thread Alice Fan (JIRA)



 [ 
https://issues.apache.org/jira/browse/IMPALA-966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alice Fan resolved IMPALA-966.
--
   Resolution: Fixed
Fix Version/s: Impala 3.2.0

> Type errors are attributed to wrong expression with insert
> --
>
> Key: IMPALA-966
> URL: https://issues.apache.org/jira/browse/IMPALA-966
> Project: IMPALA
>  Issue Type: Bug
>  Components: Frontend
>Affects Versions: Impala 1.3
>Reporter: Henry Robinson
>Assignee: Alice Fan
>Priority: Minor
> Fix For: Impala 3.2.0
>
>
> The type error below belongs to the second row to be inserted ({{sqrt()}} 
> returns {{DOUBLE}}). But the obviously {{FLOAT}} first expression gets blamed 
> for the error.
> {code}
> [localhost:21000] > insert overwrite alltypesnopart_insert(float_col) 
> values(CAST(1.0 AS FLOAT)), (sqrt(-1));
> Query: insert overwrite alltypesnopart_insert(float_col) values(CAST(1.0 AS 
> FLOAT)), (sqrt(-1))
> ERROR: AnalysisException: Possible loss of precision for target table 
> 'functional.alltypesnopart_insert'.
> Expression 'cast(1.0 as float)' (type: DOUBLE) would need to be cast to FLOAT 
> for column 'float_col'
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Resolved] (IMPALA-966) Type errors are attributed to wrong expression with insert

2019-05-14 Thread Alice Fan (JIRA)



 [ 
https://issues.apache.org/jira/browse/IMPALA-966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alice Fan resolved IMPALA-966.
--
   Resolution: Fixed
Fix Version/s: Impala 3.2.0

> Type errors are attributed to wrong expression with insert
> --
>
> Key: IMPALA-966
> URL: https://issues.apache.org/jira/browse/IMPALA-966
> Project: IMPALA
>  Issue Type: Bug
>  Components: Frontend
>Affects Versions: Impala 1.3
>Reporter: Henry Robinson
>Assignee: Alice Fan
>Priority: Minor
> Fix For: Impala 3.2.0
>
>
> The type error below belongs to the second row to be inserted ({{sqrt()}} 
> returns {{DOUBLE}}). But the obviously {{FLOAT}} first expression gets blamed 
> for the error.
> {code}
> [localhost:21000] > insert overwrite alltypesnopart_insert(float_col) 
> values(CAST(1.0 AS FLOAT)), (sqrt(-1));
> Query: insert overwrite alltypesnopart_insert(float_col) values(CAST(1.0 AS 
> FLOAT)), (sqrt(-1))
> ERROR: AnalysisException: Possible loss of precision for target table 
> 'functional.alltypesnopart_insert'.
> Expression 'cast(1.0 as float)' (type: DOUBLE) would need to be cast to FLOAT 
> for column 'float_col'
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IMPALA-8473) Refactor lineage publication mechanism to allow for different consumers

2019-05-14 Thread radford nguyen (JIRA)



[ 
https://issues.apache.org/jira/browse/IMPALA-8473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16840006#comment-16840006
 ] 

radford nguyen commented on IMPALA-8473:


[~LinaAtAustin], any reason you've changed your mind?  Impala team and I 
actually prefer the interface approach.

> Refactor lineage publication mechanism to allow for different consumers
> ---
>
> Key: IMPALA-8473
> URL: https://issues.apache.org/jira/browse/IMPALA-8473
> Project: IMPALA
>  Issue Type: Improvement
>  Components: Backend, Frontend
>Reporter: radford nguyen
>Assignee: radford nguyen
>Priority: Critical
> Attachments: ImpalaPostExecHook-infra.patch
>
>
> Impetus for this change is to allow lineage to be consumed by Atlas via Kafka.
> h3. Design Proposal
> Move lineage logging from be to fe, where we can make use of the same plugin 
> approach as {{authorization_provider}} to allow a downstream user to provide 
> their own lineage consumers as runtime dependencies.
> [~mad...@apache.org] has provided a fe patch (attached) with suggested 
> mechanism for allowing multiple hooks to be registered with the fe.  Hooks 
> would be invoked from the be at appropriate places, e.g. 
> [https://github.com/apache/impala/blob/c1b0a073938c144e9bf33901bd4df6dcda0f09ec/be/src/service/impala-server.cc#L466].
>   The hooks should all be executed asynchronously, so the current thinking is 
> that this execution should happen in the fe, since the be does not know about 
> what hooks are registered.  IOW, the 
> {{ImpalaPostExecHookFactory.executeHooks}} method (see patch) should probably 
> make use of a thread-pool executor service (or something similar) in order to 
> execute all hooks in parallel and in a non-blocking manner, returning to the 
> be asap.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Updated] (IMPALA-8550) Sentry refresh privileges has race conditions

2019-05-14 Thread Vihang Karajgaonkar (JIRA)



 [ 
https://issues.apache.org/jira/browse/IMPALA-8550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vihang Karajgaonkar updated IMPALA-8550:

Description: 
Recently, I encountered a race condition in {{SentryProxy}}'s 
refreshSentryAuthorization loop. The race happens when Sentry server is slow to 
update its information based on changes in HMS. Consider the following scenario:
 # Impala session from user A creates a database/table.
 # AuthorizationManager will updateDatabaseOwnerPrivilege 
[here|[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java#L1159]]
 Note that this add adds the user privilege in Catalog's cache out-of-band 
(without confirming that Sentry has added this privilege in its database)
 # Assume that Sentry is slow to update its database of roles/privileges. 
(Actually depending on the timing of these events, it doesn't really matter but 
likelihood of the issue increases if Sentry is slow.
 # The refreshSentryAuthorization loop is triggered based on a configured 
interval 
[here|[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/sentry/SentryProxy.java#L174]].
 Since Sentry has not yet updated its database of the owner information, this 
loop will remove the privilege from Catalog. Any subsequent SQL which requires 
privileges will fail until Sentry is synced and refresh loop adds this 
privilege again the catalog cache.

  was:
Recently, I encountered a race condition in \{{SentryProxy}}'s 
refreshSentryAuthorization loop. The race happens when Sentry server is slow to 
update its information based on changes in HMS. Consider the following scenario:
 # Impala session from user A creates a database/table.
 # AuthorizationManager will updateDatabaseOwnerPrivilege 
[here|[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java#L1159]]
 Note that this add adds the user privilege in Catalog's cache out-of-band 
(without confirming that Sentry has added this privilege in its database)
 # Assume that Sentry is slow to update its database of roles/privileges. 
(Actually depending on the timing of these events, it doesn't really matter but 
likely increases if Sentry is slow.
 # The refreshSentryAuthorization loop is triggered based on a configured 
interval 
[here|[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/sentry/SentryProxy.java#L174]].
 Since Sentry has not yet updated its database of the owner information, this 
loop will remove the privilege from Catalog. Any subsequent SQL which requires 
privileges will fail until Sentry is synced and refresh loop adds this 
privilege again the catalog cache.


> Sentry refresh privileges has race conditions
> -
>
> Key: IMPALA-8550
> URL: https://issues.apache.org/jira/browse/IMPALA-8550
> Project: IMPALA
>  Issue Type: Bug
>Reporter: Vihang Karajgaonkar
>Priority: Major
>
> Recently, I encountered a race condition in {{SentryProxy}}'s 
> refreshSentryAuthorization loop. The race happens when Sentry server is slow 
> to update its information based on changes in HMS. Consider the following 
> scenario:
>  # Impala session from user A creates a database/table.
>  # AuthorizationManager will updateDatabaseOwnerPrivilege 
> [here|[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java#L1159]]
>  Note that this add adds the user privilege in Catalog's cache out-of-band 
> (without confirming that Sentry has added this privilege in its database)
>  # Assume that Sentry is slow to update its database of roles/privileges. 
> (Actually depending on the timing of these events, it doesn't really matter 
> but likelihood of the issue increases if Sentry is slow.
>  # The refreshSentryAuthorization loop is triggered based on a configured 
> interval 
> [here|[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/sentry/SentryProxy.java#L174]].
>  Since Sentry has not yet updated its database of the owner information, this 
> loop will remove the privilege from Catalog. Any subsequent SQL which 
> requires privileges will fail until Sentry is synced and refresh loop adds 
> this privilege again the catalog cache.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Commented] (IMPALA-8550) Sentry refresh privileges has race conditions

2019-05-14 Thread Fredy Wijaya (JIRA)



[ 
https://issues.apache.org/jira/browse/IMPALA-8550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839908#comment-16839908
 ] 

Fredy Wijaya commented on IMPALA-8550:
--

Yeah, this is a known issue with the Sentry object ownership implementation.

> Sentry refresh privileges has race conditions
> -
>
> Key: IMPALA-8550
> URL: https://issues.apache.org/jira/browse/IMPALA-8550
> Project: IMPALA
>  Issue Type: Bug
>Reporter: Vihang Karajgaonkar
>Priority: Major
>
> Recently, I encountered a race condition in \{{SentryProxy}}'s 
> refreshSentryAuthorization loop. The race happens when Sentry server is slow 
> to update its information based on changes in HMS. Consider the following 
> scenario:
>  # Impala session from user A creates a database/table.
>  # AuthorizationManager will updateDatabaseOwnerPrivilege 
> [here|[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java#L1159]]
>  Note that this add adds the user privilege in Catalog's cache out-of-band 
> (without confirming that Sentry has added this privilege in its database)
>  # Assume that Sentry is slow to update its database of roles/privileges. 
> (Actually depending on the timing of these events, it doesn't really matter 
> but likely increases if Sentry is slow.
>  # The refreshSentryAuthorization loop is triggered based on a configured 
> interval 
> [here|[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/sentry/SentryProxy.java#L174]].
>  Since Sentry has not yet updated its database of the owner information, this 
> loop will remove the privilege from Catalog. Any subsequent SQL which 
> requires privileges will fail until Sentry is synced and refresh loop adds 
> this privilege again the catalog cache.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Commented] (IMPALA-8550) Sentry refresh privileges has race conditions

2019-05-14 Thread Vihang Karajgaonkar (JIRA)



[ 
https://issues.apache.org/jira/browse/IMPALA-8550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839901#comment-16839901
 ] 

Vihang Karajgaonkar commented on IMPALA-8550:
-

The easiest way to reproduce this race is to turn on {{test_owner_privileges}} 
on HMS-3 environment.

> Sentry refresh privileges has race conditions
> -
>
> Key: IMPALA-8550
> URL: https://issues.apache.org/jira/browse/IMPALA-8550
> Project: IMPALA
>  Issue Type: Bug
>Reporter: Vihang Karajgaonkar
>Priority: Major
>
> Recently, I encountered a race condition in \{{SentryProxy}}'s 
> refreshSentryAuthorization loop. The race happens when Sentry server is slow 
> to update its information based on changes in HMS. Consider the following 
> scenario:
>  # Impala session from user A creates a database/table.
>  # AuthorizationManager will updateDatabaseOwnerPrivilege 
> [here|[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java#L1159]]
>  Note that this add adds the user privilege in Catalog's cache out-of-band 
> (without confirming that Sentry has added this privilege in its database)
>  # Assume that Sentry is slow to update its database of roles/privileges. 
> (Actually depending on the timing of these events, it doesn't really matter 
> but likely increases if Sentry is slow.
>  # The refreshSentryAuthorization loop is triggered based on a configured 
> interval 
> [here|[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/sentry/SentryProxy.java#L174]].
>  Since Sentry has not yet updated its database of the owner information, this 
> loop will remove the privilege from Catalog. Any subsequent SQL which 
> requires privileges will fail until Sentry is synced and refresh loop adds 
> this privilege again the catalog cache.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Created] (IMPALA-8550) Sentry refresh privileges has race conditions

2019-05-14 Thread Vihang Karajgaonkar (JIRA)

Vihang Karajgaonkar created IMPALA-8550:
---

Summary: Sentry refresh privileges has race conditions
Key: IMPALA-8550
URL: https://issues.apache.org/jira/browse/IMPALA-8550
Project: IMPALA
Issue Type: Bug
Reporter: Vihang Karajgaonkar

Recently, I encountered a race condition in \{{SentryProxy}}'s
refreshSentryAuthorization loop. The race happens when Sentry server is slow to
update its information based on changes in HMS. Consider the following scenario:
# Impala session from user A creates a database/table.
# AuthorizationManager will updateDatabaseOwnerPrivilege
[here|[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java#L1159]]
Note that this add adds the user privilege in Catalog's cache out-of-band
(without confirming that Sentry has added this privilege in its database)
# Assume that Sentry is slow to update its database of roles/privileges.
(Actually depending on the timing of these events, it doesn't really matter but
likely increases if Sentry is slow.
# The refreshSentryAuthorization loop is triggered based on a configured
interval
[here|[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/sentry/SentryProxy.java#L174]].
Since Sentry has not yet updated its database of the owner information, this
loop will remove the privilege from Catalog. Any subsequent SQL which requires
privileges will fail until Sentry is synced and refresh loop adds this
privilege again the catalog cache.

60 matches

Mail list logo