[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-10-01 Thread Tao Li (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16187465#comment-16187465
 ] 

Tao Li commented on HIVE-17606:
---

Added a section here: 
https://cwiki.apache.org/confluence/display/Hive/HiveReplicationv2Development#HiveReplicationv2Development-MetastorenotificationAPIsecurity

[~leftylev] Please let me know if you have any suggestions to improve it. 
Thanks!

> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Fix For: 3.0.0
>
> Attachments: HIVE-17606.10.patch, HIVE-17606.1.patch, 
> HIVE-17606.2.patch, HIVE-17606.3.patch, HIVE-17606.4.patch, 
> HIVE-17606.5.patch, HIVE-17606.6.patch, HIVE-17606.7.patch, 
> HIVE-17606.8.patch, HIVE-17606.9.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-30 Thread Tao Li (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16187256#comment-16187256
 ] 

Tao Li commented on HIVE-17606:
---

Thanks [~thejas]. Will update Apache wiki asap.

> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Fix For: 3.0.0
>
> Attachments: HIVE-17606.10.patch, HIVE-17606.1.patch, 
> HIVE-17606.2.patch, HIVE-17606.3.patch, HIVE-17606.4.patch, 
> HIVE-17606.5.patch, HIVE-17606.6.patch, HIVE-17606.7.patch, 
> HIVE-17606.8.patch, HIVE-17606.9.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-28 Thread Thejas M Nair (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16185026#comment-16185026
 ] 

Thejas M Nair commented on HIVE-17606:
--

reverted

> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Fix For: 3.0.0
>
> Attachments: HIVE-17606.10.patch, HIVE-17606.1.patch, 
> HIVE-17606.2.patch, HIVE-17606.3.patch, HIVE-17606.4.patch, 
> HIVE-17606.5.patch, HIVE-17606.6.patch, HIVE-17606.7.patch, 
> HIVE-17606.8.patch, HIVE-17606.9.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-28 Thread Thejas M Nair (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16185021#comment-16185021
 ] 

Thejas M Nair commented on HIVE-17606:
--

[~sershe] [~taoli-hwx]
Looks like I had some partial changes from another patch as well in this commit.
Will revert asap and take a look.

> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Fix For: 3.0.0
>
> Attachments: HIVE-17606.10.patch, HIVE-17606.1.patch, 
> HIVE-17606.2.patch, HIVE-17606.3.patch, HIVE-17606.4.patch, 
> HIVE-17606.5.patch, HIVE-17606.6.patch, HIVE-17606.7.patch, 
> HIVE-17606.8.patch, HIVE-17606.9.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-28 Thread Tao Li (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184995#comment-16184995
 ] 

Tao Li commented on HIVE-17606:
---

[~sershe] I don't see how the change could cause the error since 
ReplChangeManager was not changed. Can you please elaborate?

> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Fix For: 3.0.0
>
> Attachments: HIVE-17606.10.patch, HIVE-17606.1.patch, 
> HIVE-17606.2.patch, HIVE-17606.3.patch, HIVE-17606.4.patch, 
> HIVE-17606.5.patch, HIVE-17606.6.patch, HIVE-17606.7.patch, 
> HIVE-17606.8.patch, HIVE-17606.9.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-28 Thread Sergey Shelukhin (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184988#comment-16184988
 ] 

Sergey Shelukhin commented on HIVE-17606:
-

This appears to break the build:
{noformat}
[ERROR] 
/Users/sergey/git/hivegit/ql/src/java/org/apache/hadoop/hive/ql/parse/repl/CopyUtils.java:[92,100]
 package ReplChangeManager does not exist
[ERROR] 
/Users/sergey/git/hivegit/ql/src/java/org/apache/hadoop/hive/ql/parse/repl/CopyUtils.java:[154,29]
 package ReplChangeManager does not exist
[ERROR] 
/Users/sergey/git/hivegit/ql/src/java/org/apache/hadoop/hive/ql/parse/repl/CopyUtils.java:[153,49]
 package ReplChangeManager does not exist
{noformat}

> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Fix For: 3.0.0
>
> Attachments: HIVE-17606.10.patch, HIVE-17606.1.patch, 
> HIVE-17606.2.patch, HIVE-17606.3.patch, HIVE-17606.4.patch, 
> HIVE-17606.5.patch, HIVE-17606.6.patch, HIVE-17606.7.patch, 
> HIVE-17606.8.patch, HIVE-17606.9.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-28 Thread Tao Li (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184900#comment-16184900
 ] 

Tao Li commented on HIVE-17606:
---

[~thejas] Am I supposed to make the wiki doc change? Thanks for the reminder.

> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Fix For: 3.0.0
>
> Attachments: HIVE-17606.10.patch, HIVE-17606.1.patch, 
> HIVE-17606.2.patch, HIVE-17606.3.patch, HIVE-17606.4.patch, 
> HIVE-17606.5.patch, HIVE-17606.6.patch, HIVE-17606.7.patch, 
> HIVE-17606.8.patch, HIVE-17606.9.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-28 Thread Thejas M Nair (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184894#comment-16184894
 ] 

Thejas M Nair commented on HIVE-17606:
--

[~taoli-hwx]
We should follow up with documentation change in wiki for this. Can you please 
take a look ?

cc [~leftylev]


> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Fix For: 3.0.0
>
> Attachments: HIVE-17606.10.patch, HIVE-17606.1.patch, 
> HIVE-17606.2.patch, HIVE-17606.3.patch, HIVE-17606.4.patch, 
> HIVE-17606.5.patch, HIVE-17606.6.patch, HIVE-17606.7.patch, 
> HIVE-17606.8.patch, HIVE-17606.9.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-28 Thread Tao Li (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184838#comment-16184838
 ] 

Tao Li commented on HIVE-17606:
---

Test result looks good. The failures don't seems related to the change.

> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-17606.10.patch, HIVE-17606.1.patch, 
> HIVE-17606.2.patch, HIVE-17606.3.patch, HIVE-17606.4.patch, 
> HIVE-17606.5.patch, HIVE-17606.6.patch, HIVE-17606.7.patch, 
> HIVE-17606.8.patch, HIVE-17606.9.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-28 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184827#comment-16184827
 ] 

Hive QA commented on HIVE-17606:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12889556/HIVE-17606.10.patch

{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 11089 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_predicate_pushdown]
 (batchId=232)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_single_sourced_multi_insert]
 (batchId=232)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
 (batchId=157)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
 (batchId=170)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
 (batchId=171)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] 
(batchId=101)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.org.apache.hadoop.hive.cli.TestTezPerfCliDriver
 (batchId=240)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=203)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7029/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7029/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7029/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12889556 - PreCommit-HIVE-Build

> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-17606.10.patch, HIVE-17606.1.patch, 
> HIVE-17606.2.patch, HIVE-17606.3.patch, HIVE-17606.4.patch, 
> HIVE-17606.5.patch, HIVE-17606.6.patch, HIVE-17606.7.patch, 
> HIVE-17606.8.patch, HIVE-17606.9.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-28 Thread Thejas M Nair (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184746#comment-16184746
 ] 

Thejas M Nair commented on HIVE-17606:
--

+1

> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-17606.10.patch, HIVE-17606.1.patch, 
> HIVE-17606.2.patch, HIVE-17606.3.patch, HIVE-17606.4.patch, 
> HIVE-17606.5.patch, HIVE-17606.6.patch, HIVE-17606.7.patch, 
> HIVE-17606.8.patch, HIVE-17606.9.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-28 Thread Tao Li (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184674#comment-16184674
 ] 

Tao Li commented on HIVE-17606:
---

[~thejas] Correct. Actually I am fixing that now.

> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-17606.1.patch, HIVE-17606.2.patch, 
> HIVE-17606.3.patch, HIVE-17606.4.patch, HIVE-17606.5.patch, 
> HIVE-17606.6.patch, HIVE-17606.7.patch, HIVE-17606.8.patch, HIVE-17606.9.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-28 Thread Thejas M Nair (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184668#comment-16184668
 ] 

Thejas M Nair commented on HIVE-17606:
--

{code}
 try {
+  rsp = metaStoreClient.getNextNotification(firstEventId, 0, null);
+  assertEquals(1, rsp.getEventsSize());
+  // Turn auth back on. That should fail the call and we do expect the 
exception
+  
hconf.setBoolVar(HiveConf.ConfVars.METASTORE_EVENT_DB_NOTIFICATION_API_AUTH, 
true);
+  rsp = metaStoreClient.getNextNotification(firstEventId, 0, null);
+} 
{code}

In above section, if the first call to metaStoreClient.getNextNotification 
throws an exception, the test would still succeed. Is that right ?


> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-17606.1.patch, HIVE-17606.2.patch, 
> HIVE-17606.3.patch, HIVE-17606.4.patch, HIVE-17606.5.patch, 
> HIVE-17606.6.patch, HIVE-17606.7.patch, HIVE-17606.8.patch, HIVE-17606.9.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-28 Thread Tao Li (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184610#comment-16184610
 ] 

Tao Li commented on HIVE-17606:
---

[~thejas] Thanks for the comments. Regarding the test case, I was specifying 
"expected = TException.class" to expect the exception. I will add some more 
comments to make it more clear. Just uploaded another iteration. 

> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-17606.1.patch, HIVE-17606.2.patch, 
> HIVE-17606.3.patch, HIVE-17606.4.patch, HIVE-17606.5.patch, 
> HIVE-17606.6.patch, HIVE-17606.7.patch, HIVE-17606.8.patch, HIVE-17606.9.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-28 Thread Thejas M Nair (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184573#comment-16184573
 ] 

Thejas M Nair commented on HIVE-17606:
--

MetaStoreUtils.hasPermissionForDbNotificationCalls is implementing a more 
generic function that what the name suggests. Maybe call it something like 
checkUserHasHostProxyPrivileges ?
That would help clarify the purpose and help re-use if some other code path 
also needs same functionality.


> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-17606.1.patch, HIVE-17606.2.patch, 
> HIVE-17606.3.patch, HIVE-17606.4.patch, HIVE-17606.5.patch, 
> HIVE-17606.6.patch, HIVE-17606.7.patch, HIVE-17606.8.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-28 Thread Thejas M Nair (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184569#comment-16184569
 ] 

Thejas M Nair commented on HIVE-17606:
--

I see one possible issue with the test case. 
If the "rsp = metaStoreClient.getNextNotification(firstEventId, 0, null);" that 
is supposed to work throws an exception, the test would still pass. Is that 
right ?


> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-17606.1.patch, HIVE-17606.2.patch, 
> HIVE-17606.3.patch, HIVE-17606.4.patch, HIVE-17606.5.patch, 
> HIVE-17606.6.patch, HIVE-17606.7.patch, HIVE-17606.8.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-28 Thread Tao Li (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184476#comment-16184476
 ] 

Tao Li commented on HIVE-17606:
---

Test result looks good.

> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-17606.1.patch, HIVE-17606.2.patch, 
> HIVE-17606.3.patch, HIVE-17606.4.patch, HIVE-17606.5.patch, 
> HIVE-17606.6.patch, HIVE-17606.7.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-28 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184471#comment-16184471
 ] 

Hive QA commented on HIVE-17606:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12889507/HIVE-17606.7.patch

{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 11090 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_predicate_pushdown]
 (batchId=232)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_single_sourced_multi_insert]
 (batchId=232)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
 (batchId=157)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
 (batchId=171)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] 
(batchId=236)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7026/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7026/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7026/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12889507 - PreCommit-HIVE-Build

> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-17606.1.patch, HIVE-17606.2.patch, 
> HIVE-17606.3.patch, HIVE-17606.4.patch, HIVE-17606.5.patch, 
> HIVE-17606.6.patch, HIVE-17606.7.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-27 Thread Lefty Leverenz (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183695#comment-16183695
 ] 

Lefty Leverenz commented on HIVE-17606:
---

Thanks for fixing the parameter description.  It looks good.

> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-17606.1.patch, HIVE-17606.2.patch, 
> HIVE-17606.3.patch, HIVE-17606.4.patch, HIVE-17606.5.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-27 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183645#comment-16183645
 ] 

Hive QA commented on HIVE-17606:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12889347/HIVE-17606.5.patch

{color:green}SUCCESS:{color} +1 due to 8 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 18 failed/errored test(s), 11089 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_predicate_pushdown]
 (batchId=232)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_single_sourced_multi_insert]
 (batchId=232)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
 (batchId=157)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
 (batchId=171)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] 
(batchId=101)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] 
(batchId=236)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.checkExpectedLocks 
(batchId=285)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.parallelExecutionOfReplicationBootStrapLoad
 (batchId=220)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testBootstrapFunctionReplication
 (batchId=220)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testCreateFunctionIncrementalReplication
 (batchId=220)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testCreateFunctionWithFunctionBinaryJarsOnHDFS
 (batchId=220)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testDropFunctionIncrementalReplication
 (batchId=220)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testMultipleStagesOfReplicationLoadTask
 (batchId=220)
org.apache.hive.hcatalog.api.TestHCatClient.testReplicationTaskIter 
(batchId=181)
org.apache.hive.hcatalog.api.repl.commands.TestCommands.testBasicReplEximCommands
 (batchId=181)
org.apache.hive.hcatalog.api.repl.commands.TestCommands.testNoopReplEximCommands
 (batchId=181)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testReplDumpResultSet (batchId=229)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7014/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7014/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7014/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 18 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12889347 - PreCommit-HIVE-Build

> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-17606.1.patch, HIVE-17606.2.patch, 
> HIVE-17606.3.patch, HIVE-17606.4.patch, HIVE-17606.5.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-27 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183122#comment-16183122
 ] 

Hive QA commented on HIVE-17606:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12889255/HIVE-17606.4.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 145 failed/errored test(s), 11084 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_predicate_pushdown]
 (batchId=231)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_single_sourced_multi_insert]
 (batchId=231)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_addpartition_blobstore_to_blobstore]
 (batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_addpartition_blobstore_to_local]
 (batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_addpartition_blobstore_to_warehouse]
 (batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_addpartition_local_to_blobstore]
 (batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_blobstore]
 (batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_blobstore_nonpart]
 (batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_local]
 (batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_warehouse]
 (batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_warehouse_nonpart]
 (batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_local_to_blobstore]
 (batchId=243)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_00_nonpart_empty] 
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_01_nonpart] 
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_02_00_part_empty] 
(batchId=64)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_02_part] 
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_03_nonpart_over_compat]
 (batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_04_all_part] 
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_04_evolved_parts] 
(batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_05_some_part] 
(batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_06_one_part] 
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_07_all_part_over_nonoverlap]
 (batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_08_nonpart_rename] 
(batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_09_part_spec_nonoverlap]
 (batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_10_external_managed]
 (batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_11_managed_external]
 (batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_12_external_location]
 (batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_13_managed_location]
 (batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_14_managed_location_over_existing]
 (batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_15_external_part] 
(batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_16_part_external] 
(batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_17_part_managed] 
(batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_18_part_external] 
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_19_00_part_external_location]
 (batchId=64)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_19_part_external_location]
 (batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_20_part_managed_location]
 (batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_21_export_authsuccess]
 (batchId=40)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_22_import_exist_authsuccess]
 (batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_23_import_part_authsuccess]
 (batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_24_import_nonexist_authsuccess]
 (batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_25_export_parentpath_has_inaccessible_children]
 (batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_hidden_files] 
(batchId=46)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[repl_2_exim_basic] 
(batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[repl_3_exim_metadata] 
(batchId=55)
org

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-27 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16182742#comment-16182742
 ] 

Hive QA commented on HIVE-17606:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12889255/HIVE-17606.4.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 146 failed/errored test(s), 11078 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_predicate_pushdown]
 (batchId=231)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_single_sourced_multi_insert]
 (batchId=231)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_addpartition_blobstore_to_blobstore]
 (batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_addpartition_blobstore_to_local]
 (batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_addpartition_blobstore_to_warehouse]
 (batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_addpartition_local_to_blobstore]
 (batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_blobstore]
 (batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_blobstore_nonpart]
 (batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_local]
 (batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_warehouse]
 (batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_warehouse_nonpart]
 (batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_local_to_blobstore]
 (batchId=243)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_00_nonpart_empty] 
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_01_nonpart] 
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_02_00_part_empty] 
(batchId=64)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_02_part] 
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_03_nonpart_over_compat]
 (batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_04_all_part] 
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_04_evolved_parts] 
(batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_05_some_part] 
(batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_06_one_part] 
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_07_all_part_over_nonoverlap]
 (batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_08_nonpart_rename] 
(batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_09_part_spec_nonoverlap]
 (batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_10_external_managed]
 (batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_11_managed_external]
 (batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_12_external_location]
 (batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_13_managed_location]
 (batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_14_managed_location_over_existing]
 (batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_15_external_part] 
(batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_16_part_external] 
(batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_17_part_managed] 
(batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_18_part_external] 
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_19_00_part_external_location]
 (batchId=64)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_19_part_external_location]
 (batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_20_part_managed_location]
 (batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_21_export_authsuccess]
 (batchId=40)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_22_import_exist_authsuccess]
 (batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_23_import_part_authsuccess]
 (batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_24_import_nonexist_authsuccess]
 (batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_25_export_parentpath_has_inaccessible_children]
 (batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_hidden_files] 
(batchId=46)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[repl_2_exim_basic] 
(batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[repl_3_exim_metadata] 
(batchId=55)
org

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-27 Thread Tao Li (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16182189#comment-16182189
 ] 

Tao Li commented on HIVE-17606:
---

[~thejas], [~leftylev] Thanks for the comments. Just uploaded another iteration 
and will wait for test results.

> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-17606.1.patch, HIVE-17606.2.patch, 
> HIVE-17606.3.patch, HIVE-17606.4.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-26 Thread Lefty Leverenz (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16182004#comment-16182004
 ] 

Lefty Leverenz commented on HIVE-17606:
---

Config review:  The description of 
*hive.metastore.event.db.notification.api.auth* has a typo ("authroization").  
And while you're at it, please spell out database in the description.

> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-17606.1.patch, HIVE-17606.2.patch, 
> HIVE-17606.3.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-26 Thread Thejas M Nair (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16181754#comment-16181754
 ] 

Thejas M Nair commented on HIVE-17606:
--

Can you also call the authorize method something more specific like 
authorizeProxyPrivilege(), since its doing a very specific authorization check ?
There is more general authorization api provided via pre-event listener, this 
would help do avoid confusing with that.


> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-17606.1.patch, HIVE-17606.2.patch, 
> HIVE-17606.3.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-26 Thread Thejas M Nair (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16181717#comment-16181717
 ] 

Thejas M Nair commented on HIVE-17606:
--

 * Lets secure this out of the box - 
hive.metastore.event.db.notification.api.auth=true. I think we can make this 
slightly incompatible change in 3.0.0 release, in the interest of security.
 * Can you update the description (this setting could be set in hive-site.xml 
also, so the core-site.xml reference is not accurate) ? - "If metastore do 
authorization against db notification related APIs such as 
get_next_notification. If set to true, then only the superusers in proxy user 
settings have the permission" 
 * Can you add comment to each section of test case describing what its testing 
?
 * It would be better to re-use the code in HiveAuthFactory.verifyProxyAccess 
for proxy verification. However, that code is in service package, we might have 
to move that to common.
 * Opening curly braces "{" is put at end of the if statement by hive coding 
conventions, this change has it both ways.


> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-17606.1.patch, HIVE-17606.2.patch, 
> HIVE-17606.3.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-26 Thread Tao Li (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16181607#comment-16181607
 ] 

Tao Li commented on HIVE-17606:
---

Test result looks good now.

> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-17606.1.patch, HIVE-17606.2.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-26 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16181603#comment-16181603
 ] 

Hive QA commented on HIVE-17606:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12889104/HIVE-17606.2.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 11063 tests 
executed
*Failed tests:*
{noformat}
TestAccumuloCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=231)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=231)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
 (batchId=156)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
 (batchId=170)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] 
(batchId=100)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6995/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6995/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6995/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12889104 - PreCommit-HIVE-Build

> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-17606.1.patch, HIVE-17606.2.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17606) Improve security for DB notification related APIs

2017-09-26 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16180631#comment-16180631
 ] 

Hive QA commented on HIVE-17606:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12889021/HIVE-17606.1.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 11064 tests 
executed
*Failed tests:*
{noformat}
TestAccumuloCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=231)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=231)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[optimize_join_ptp] 
(batchId=70)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
 (batchId=156)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
 (batchId=170)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] 
(batchId=235)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] 
(batchId=235)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.sqlDb (batchId=233)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testParallelCompilation (batchId=228)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6988/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6988/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6988/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12889021 - PreCommit-HIVE-Build

> Improve security for DB notification related APIs
> -
>
> Key: HIVE-17606
> URL: https://issues.apache.org/jira/browse/HIVE-17606
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-17606.1.patch
>
>
> The purpose is to make sure only the superusers which are specified in the 
> proxyuser settings can make the db notification related API calls, since this 
> is supposed to be called by superuser/admin instead of any end user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)