Re: Review Request 45611: HIVE-13360: Refactoring Hive Authorization

[Ashutosh Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45611/#review126670
---




itests/hive-unit/src/test/java/org/apache/hive/jdbc/authorization/TestHS2AuthzContext.java
 


It will be good to move these tests instead of deleting altogether.



ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java (line 10373)


Can just remove this comment.



ql/src/java/org/apache/hadoop/hive/ql/parse/TableMask.java (line 90)


We should check size of exprs and passed in list of columns and throw 
exception if they don't match.



ql/src/java/org/apache/hadoop/hive/ql/parse/TableMask.java (line 92)


Check for null == expr and use privObj.getCols.get(index) if it is.



ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveAuthorizer.java
 (line 273)


No need to provide QueryContext here.



ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveAuthorizer.java
 (line 288)


I dont think there is need for this call anymore, since we are collecting 
all tables and columns and making one call for it. We can just examine 
List returned from applyRowFilterAndColumnMasking() to 
determine which tables need transforms.



ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HivePrivilegeObject.java
 (lines 110 - 111)


it will be good to add comments for these.


- Ashutosh Chauhan


On April 2, 2016, 12:54 a.m., pengcheng xiong wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45611/
> ---
> 
> (Updated April 2, 2016, 12:54 a.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-13360: Refactoring Hive Authorization
> 
> 
> Diffs
> -
> 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/authorization/plugin/TestHiveAuthorizerCheckInvocation.java
>  9f47f84 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/authorization/plugin/TestHiveAuthorizerShowFilters.java
>  5922a8c 
>   
> itests/hive-unit/src/test/java/org/apache/hive/jdbc/authorization/TestHS2AuthzContext.java
>  c43776b 
>   
> itests/hive-unit/src/test/java/org/apache/hive/jdbc/authorization/TestJdbcMetadataApiAuth.java
>  692bfa0 
>   
> itests/util/src/main/java/org/apache/hadoop/hive/ql/security/DummyAuthenticator.java
>  a296ac5 
>   
> itests/util/src/main/java/org/apache/hadoop/hive/ql/security/InjectableDummyAuthenticator.java
>  322834e 
>   
> itests/util/src/main/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/SQLStdHiveAuthorizationValidatorForTest.java
>  c0387e2 
>   ql/src/java/org/apache/hadoop/hive/ql/Driver.java 7276e31 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/MaskAndFilterInfo.java 
> PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java e81d46e 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TableMask.java c47c2bd 
>   ql/src/java/org/apache/hadoop/hive/ql/processors/CommandUtil.java d98b30c 
>   
> ql/src/java/org/apache/hadoop/hive/ql/security/HadoopDefaultAuthenticator.java
>  18e4e00 
>   
> ql/src/java/org/apache/hadoop/hive/ql/security/HiveAuthenticationProvider.java
>  7befff8 
>   
> ql/src/java/org/apache/hadoop/hive/ql/security/SessionStateConfigUserAuthenticator.java
>  8c7809e 
>   
> ql/src/java/org/apache/hadoop/hive/ql/security/SessionStateUserAuthenticator.java
>  a77e93f 
>   
> ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/AuthorizationMetaStoreFilterHook.java
>  6bad99b 
>   
> ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveAuthorizationValidator.java
>  1b366c2 
>   
> ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveAuthorizer.java
>  6e2ef8d 
>   
> ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveAuthorizerImpl.java
>  c73d667 
>   
> ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveAuthzContext.java
>  195e341 
>   
> ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HivePrivilegeObject.java
>  0364627 
>   
> ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveV1Authorizer.java
>  c8aa9db 
>   
> ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/QueryContext.java
>  PRE-CREATION 
>   
> 

[jira] [Created] (HIVE-13410) PerfLog metrics scopes not closed if there are exceptions on HS2

[Szehon Ho (JIRA)]

Szehon Ho created HIVE-13410:


 Summary: PerfLog metrics scopes not closed if there are exceptions 
on HS2
 Key: HIVE-13410
 URL: https://issues.apache.org/jira/browse/HIVE-13410
 Project: Hive
  Issue Type: Bug
  Components: Diagnosability
Affects Versions: 2.0.0
Reporter: Szehon Ho


If there are errors, the HS2 PerfLog api scopes are not closed.  Then there are 
sometimes messages like 'java.io.IOException: Scope named api_parse is not 
closed, cannot be opened.'

I had simply forgetting to close the dangling scopes if there is an exception.  
Doing so now.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 45611: HIVE-13360: Refactoring Hive Authorization

[pengcheng xiong]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45611/
---

Review request for hive and Ashutosh Chauhan.


Repository: hive-git


Description
---

HIVE-13360: Refactoring Hive Authorization


Diffs
-

  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/authorization/plugin/TestHiveAuthorizerCheckInvocation.java
 9f47f84 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/authorization/plugin/TestHiveAuthorizerShowFilters.java
 5922a8c 
  
itests/hive-unit/src/test/java/org/apache/hive/jdbc/authorization/TestHS2AuthzContext.java
 c43776b 
  
itests/hive-unit/src/test/java/org/apache/hive/jdbc/authorization/TestJdbcMetadataApiAuth.java
 692bfa0 
  
itests/util/src/main/java/org/apache/hadoop/hive/ql/security/DummyAuthenticator.java
 a296ac5 
  
itests/util/src/main/java/org/apache/hadoop/hive/ql/security/InjectableDummyAuthenticator.java
 322834e 
  
itests/util/src/main/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/SQLStdHiveAuthorizationValidatorForTest.java
 c0387e2 
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java 7276e31 
  ql/src/java/org/apache/hadoop/hive/ql/parse/MaskAndFilterInfo.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java e81d46e 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TableMask.java c47c2bd 
  ql/src/java/org/apache/hadoop/hive/ql/processors/CommandUtil.java d98b30c 
  
ql/src/java/org/apache/hadoop/hive/ql/security/HadoopDefaultAuthenticator.java 
18e4e00 
  
ql/src/java/org/apache/hadoop/hive/ql/security/HiveAuthenticationProvider.java 
7befff8 
  
ql/src/java/org/apache/hadoop/hive/ql/security/SessionStateConfigUserAuthenticator.java
 8c7809e 
  
ql/src/java/org/apache/hadoop/hive/ql/security/SessionStateUserAuthenticator.java
 a77e93f 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/AuthorizationMetaStoreFilterHook.java
 6bad99b 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveAuthorizationValidator.java
 1b366c2 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveAuthorizer.java
 6e2ef8d 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveAuthorizerImpl.java
 c73d667 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveAuthzContext.java
 195e341 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HivePrivilegeObject.java
 0364627 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveV1Authorizer.java
 c8aa9db 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/QueryContext.java
 PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/DummyHiveAuthorizationValidator.java
 e4ddc9b 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/SQLStdHiveAuthorizationValidator.java
 c5d60b3 
  service/src/java/org/apache/hive/service/cli/operation/MetadataOperation.java 
285b4f9 

Diff: https://reviews.apache.org/r/45611/diff/


Testing
---


Thanks,

pengcheng xiong

[jira] [Created] (HIVE-13409) Fix JDK8 test failures related to COLUMN_STATS_ACCURATE

[Mohit Sabharwal (JIRA)]

Mohit Sabharwal created HIVE-13409:
--

 Summary: Fix JDK8 test failures related to COLUMN_STATS_ACCURATE
 Key: HIVE-13409
 URL: https://issues.apache.org/jira/browse/HIVE-13409
 Project: Hive
  Issue Type: Bug
  Components: Tests
Reporter: Mohit Sabharwal
Assignee: Mohit Sabharwal


126 failures have crept into JDK8 tests since we resolved HIVE-8607

http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/HIVE-TRUNK-JAVA8/

Majority relate to the ordering of a "COLUMN_STATS_ACCURATE" partition property.

Looks like a simple fix, use ordered map in 
HiveStringUtils.getPropertiesExplain()



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-13408) Issue appending HIVE_QUERY_ID without checking if the prefix already exists

[Vikram Dixit K (JIRA)]

Vikram Dixit K created HIVE-13408:
-

 Summary: Issue appending HIVE_QUERY_ID without checking if the 
prefix already exists
 Key: HIVE-13408
 URL: https://issues.apache.org/jira/browse/HIVE-13408
 Project: Hive
  Issue Type: Bug
  Components: Shims
Affects Versions: 2.0.0
Reporter: Vikram Dixit K
Assignee: Vikram Dixit K


{code}
We are resetting the hadoop caller context to HIVE_QUERY_ID:HIVE_QUERY_ID:
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-13407) Add more subtlety to TezCompiler Perf Logging

[Hari Sankar Sivarama Subramaniyan (JIRA)]

Hari Sankar Sivarama Subramaniyan created HIVE-13407:


 Summary: Add more subtlety to TezCompiler Perf Logging
 Key: HIVE-13407
 URL: https://issues.apache.org/jira/browse/HIVE-13407
 Project: Hive
  Issue Type: Bug
Reporter: Hari Sankar Sivarama Subramaniyan
Assignee: Hari Sankar Sivarama Subramaniyan


We can add more subtlety to perf logging information in TezCompiler



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 45602: TS/Date should have higher precedence in type hierarchy than string group

[Ashutosh Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45602/
---

Review request for hive and Jason Dere.


Bugs: HIVE-13381
https://issues.apache.org/jira/browse/HIVE-13381


Repository: hive-git


Description
---

Comparisons between timestamp/date & string group should happen in ts/date


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 56b96b4 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java 
1eb960d 
  ql/src/test/org/apache/hadoop/hive/ql/exec/TestFunctionRegistry.java 6a83c32 
  
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizationContext.java 
e4c7529 
  ql/src/test/queries/clientpositive/cast_on_constant.q PRE-CREATION 
  ql/src/test/results/clientpositive/cast_on_constant.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/45602/diff/


Testing
---

New test & existing regression suite.


Thanks,

Ashutosh Chauhan

[jira] [Created] (HIVE-13406) HBase metastore needs to support ACID

[Alan Gates (JIRA)]

Alan Gates created HIVE-13406:
-

 Summary: HBase metastore needs to support ACID
 Key: HIVE-13406
 URL: https://issues.apache.org/jira/browse/HIVE-13406
 Project: Hive
  Issue Type: Improvement
  Components: HBase Metastore, Transactions
Affects Versions: 2.0.0
Reporter: Alan Gates
Assignee: Alan Gates


The current HBase metastore code does not support any of the acid operations.  
It needs to.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-13405) Fix Connection Leak in OrcRawRecordMerger

[Thomas Poepping (JIRA)]

Thomas Poepping created HIVE-13405:
--

 Summary: Fix Connection Leak in OrcRawRecordMerger
 Key: HIVE-13405
 URL: https://issues.apache.org/jira/browse/HIVE-13405
 Project: Hive
  Issue Type: Bug
  Components: ORC
Affects Versions: 2.0.0
Reporter: Thomas Poepping


In OrcRawRecordMerger.getLastFlushLength, if the opened stream throws an 
IOException on .available() or on .readLong(), the function will exit without 
closing the stream.

This patch adds a try-with-resources to fix this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

apache hive

[Nikhil Reddy]