[jira] [Commented] (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13013754#comment-13013754 ] Devaraj Das commented on HIVE-78: - BTW, was any thought put in to implement the authorization checks in the ObjectStore? In the model where a MetaStore server is deployed separately, applications (map/reduce tasks for example), can make programmatic calls to the MetaStore to, for example, drop random tables/partitions, and they will pass.. Just wondering whether this usecase was considered. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Security, Server > Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Fix For: 0.7.0 > > Attachments: HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.10.no_thrift.patch, HIVE-78.11.patch, HIVE-78.12.2.patch, > HIVE-78.12.3.patch, HIVE-78.12.4.patch, HIVE-78.12.5.patch, HIVE-78.12.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, HIVE-78.4.complete.patch, > HIVE-78.4.no_thrift.patch, HIVE-78.5.complete.patch, > HIVE-78.5.no_thrift.patch, HIVE-78.6.complete.patch, > HIVE-78.6.no_thrift.patch, HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, > HIVE-78.9.no_thrift.patch, HIVE-78.9.patch, createuser-v1.patch, > hive-78-metadata-v1.patch, hive-78-syntax-v1.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980971#action_12980971 ] He Yongqiang commented on HIVE-78: -- Here is mysql upgrade script: http://wiki.apache.org/hadoop/Hive/AuthDev#A8._Metastore_upgrade_script_for_mysql > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.10.no_thrift.patch, HIVE-78.11.patch, HIVE-78.12.2.patch, > HIVE-78.12.3.patch, HIVE-78.12.4.patch, HIVE-78.12.5.patch, HIVE-78.12.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, HIVE-78.4.complete.patch, > HIVE-78.4.no_thrift.patch, HIVE-78.5.complete.patch, > HIVE-78.5.no_thrift.patch, HIVE-78.6.complete.patch, > HIVE-78.6.no_thrift.patch, HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, > HIVE-78.9.no_thrift.patch, HIVE-78.9.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12979892#action_12979892 ] Ashutosh Chauhan commented on HIVE-78: -- @Namit, Sounds good. Thanks for the info. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.10.no_thrift.patch, HIVE-78.11.patch, HIVE-78.12.2.patch, > HIVE-78.12.patch, HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, > HIVE-78.4.complete.patch, HIVE-78.4.no_thrift.patch, > HIVE-78.5.complete.patch, HIVE-78.5.no_thrift.patch, > HIVE-78.6.complete.patch, HIVE-78.6.no_thrift.patch, > HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, HIVE-78.9.no_thrift.patch, > HIVE-78.9.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12979868#action_12979868 ] Namit Jain commented on HIVE-78: All the tests are passing - we are blocked on the names of the new reserved words, we have introduced. We are trying to get it in asap > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.10.no_thrift.patch, HIVE-78.11.patch, HIVE-78.12.2.patch, > HIVE-78.12.patch, HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, > HIVE-78.4.complete.patch, HIVE-78.4.no_thrift.patch, > HIVE-78.5.complete.patch, HIVE-78.5.no_thrift.patch, > HIVE-78.6.complete.patch, HIVE-78.6.no_thrift.patch, > HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, HIVE-78.9.no_thrift.patch, > HIVE-78.9.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12979827#action_12979827 ] Ashutosh Chauhan commented on HIVE-78: -- John's latest comment on HIVE-1696 https://issues.apache.org/jira/browse/HIVE-1696?focusedCommentId=12978176&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12978176 seems to indicate that HIVE-1696 is blocked on this getting committed. Do we know how far we are on this issue and how long it may take before it gets committed? That will help to estimate commit date for HIVE-1696 > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.10.no_thrift.patch, HIVE-78.11.patch, HIVE-78.12.2.patch, > HIVE-78.12.patch, HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, > HIVE-78.4.complete.patch, HIVE-78.4.no_thrift.patch, > HIVE-78.5.complete.patch, HIVE-78.5.no_thrift.patch, > HIVE-78.6.complete.patch, HIVE-78.6.no_thrift.patch, > HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, HIVE-78.9.no_thrift.patch, > HIVE-78.9.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978978#action_12978978 ] Namit Jain commented on HIVE-78: I am getting some compilation errors - can you regenerate the patch ? > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.10.no_thrift.patch, HIVE-78.11.patch, HIVE-78.12.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, HIVE-78.4.complete.patch, > HIVE-78.4.no_thrift.patch, HIVE-78.5.complete.patch, > HIVE-78.5.no_thrift.patch, HIVE-78.6.complete.patch, > HIVE-78.6.no_thrift.patch, HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, > HIVE-78.9.no_thrift.patch, HIVE-78.9.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978175#action_12978175 ] Namit Jain commented on HIVE-78: My bad, I committed HIVE-1840 just now. Can you regenerate the patch ? > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.10.no_thrift.patch, HIVE-78.11.patch, HIVE-78.2.nothrift.patch, > HIVE-78.2.thrift.patch, HIVE-78.4.complete.patch, HIVE-78.4.no_thrift.patch, > HIVE-78.5.complete.patch, HIVE-78.5.no_thrift.patch, > HIVE-78.6.complete.patch, HIVE-78.6.no_thrift.patch, > HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, HIVE-78.9.no_thrift.patch, > HIVE-78.9.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12975711#action_12975711 ] Namit Jain commented on HIVE-78: Can you check if 'USER', 'ROLE' and 'OPTION' are not used as column names in any table ? > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.10.no_thrift.patch, HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, > HIVE-78.4.complete.patch, HIVE-78.4.no_thrift.patch, > HIVE-78.5.complete.patch, HIVE-78.5.no_thrift.patch, > HIVE-78.6.complete.patch, HIVE-78.6.no_thrift.patch, > HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, HIVE-78.9.no_thrift.patch, > HIVE-78.9.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12975710#action_12975710 ] Namit Jain commented on HIVE-78: I think you can do the following optimization: feel free to do it in a followup. There are many queries which have lots of input partitions for the same input table. If the table under consideration has the same privilege for all the partitions, you dont need to check the permissions for all the partitions. You can find the common tables and skip the partitions altogether > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.10.no_thrift.patch, HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, > HIVE-78.4.complete.patch, HIVE-78.4.no_thrift.patch, > HIVE-78.5.complete.patch, HIVE-78.5.no_thrift.patch, > HIVE-78.6.complete.patch, HIVE-78.6.no_thrift.patch, > HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, HIVE-78.9.no_thrift.patch, > HIVE-78.9.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12975708#action_12975708 ] Namit Jain commented on HIVE-78: HadoopDefaultAuthenticator System.out.println() present PrivilegeObjectDesc.java: @Explain(displayName="privilege subject") can you use Privilege Object instead ? private String object; -> can you change it to tableName ? PrivilegeObjectDesc.java: should contain a list of columns. Remove columns from PrivilegeDesc. -> PrivilegeDesc can be removed all together It is same as Privilege > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.10.no_thrift.patch, HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, > HIVE-78.4.complete.patch, HIVE-78.4.no_thrift.patch, > HIVE-78.5.complete.patch, HIVE-78.5.no_thrift.patch, > HIVE-78.6.complete.patch, HIVE-78.6.no_thrift.patch, > HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, HIVE-78.9.no_thrift.patch, > HIVE-78.9.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[
https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12975689#action_12975689
]
Namit Jain commented on HIVE-78:
hive-default.xml:
hive.variable.substitute
true
This enables substitution using syntax like ${var} ${system:var}
and ${env:var}.
seems like a merge problem.
package.jdo:
no index needed on ROLE_ID
ALTER TABLE authorization_part SET TBLPROPERTIES
("PARTITION_LEVEL_PRIVILEGE"="TRUE");
Dont load partition specific priviliges for tables that do no have a separate
partition level priv.
ObjectStore.java: add comments for getGrantObjects
HiveMetaStoreClient.java: no need for setEmpotyGrantList()
you should always create a empty list for a user,role or group.
DefaultHiveAuthorizationProvider.java:
Can you add comments for all the (private) functions ?
It is not obvious what is the meaning of the return value ?
Still reviewing.
> Authorization infrastructure for Hive
> -
>
> Key: HIVE-78
> URL: https://issues.apache.org/jira/browse/HIVE-78
> Project: Hive
> Issue Type: New Feature
> Components: Metastore, Query Processor, Server Infrastructure
>Reporter: Ashish Thusoo
>Assignee: He Yongqiang
> Attachments: createuser-v1.patch, hive-78-metadata-v1.patch,
> hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch,
> HIVE-78.10.no_thrift.patch, HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch,
> HIVE-78.4.complete.patch, HIVE-78.4.no_thrift.patch,
> HIVE-78.5.complete.patch, HIVE-78.5.no_thrift.patch,
> HIVE-78.6.complete.patch, HIVE-78.6.no_thrift.patch,
> HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, HIVE-78.9.no_thrift.patch,
> HIVE-78.9.patch, hive-78.diff
>
>
> Allow hive to integrate with existing user repositories for authentication
> and authorization infromation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12973388#action_12973388 ] John Sichi commented on HIVE-78: A few more comments on patch 10 in https://reviews.apache.org/r/187/ > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.10.no_thrift.patch, HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, > HIVE-78.4.complete.patch, HIVE-78.4.no_thrift.patch, > HIVE-78.5.complete.patch, HIVE-78.5.no_thrift.patch, > HIVE-78.6.complete.patch, HIVE-78.6.no_thrift.patch, > HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, HIVE-78.9.no_thrift.patch, > HIVE-78.9.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12973289#action_12973289 ] He Yongqiang commented on HIVE-78: -- A new no_thrift patch addressed John's review comments. Thanks John! Running tests. And will upload a new complete patch after tests (and incorporate new comments). > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.10.no_thrift.patch, HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, > HIVE-78.4.complete.patch, HIVE-78.4.no_thrift.patch, > HIVE-78.5.complete.patch, HIVE-78.5.no_thrift.patch, > HIVE-78.6.complete.patch, HIVE-78.6.no_thrift.patch, > HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, HIVE-78.9.no_thrift.patch, > HIVE-78.9.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12972644#action_12972644 ] John Sichi commented on HIVE-78: I added one comment about referring to "grantee" instead of "principal" in some of the API's, but I did not do it consistently. I think this would be clearer across thrift/JDO to distinguish the grantor from the grantee in all cases, but if you want to leave it as is, just ignore that comment. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, HIVE-78.4.complete.patch, > HIVE-78.4.no_thrift.patch, HIVE-78.5.complete.patch, > HIVE-78.5.no_thrift.patch, HIVE-78.6.complete.patch, > HIVE-78.6.no_thrift.patch, HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, > HIVE-78.9.no_thrift.patch, HIVE-78.9.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12972643#action_12972643 ] John Sichi commented on HIVE-78: @Yongqiang: New review comments in https://reviews.apache.org/r/183/ The patch is applying cleanly for me now (I must have forgotten to svn up), so I'll do some testing later. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, HIVE-78.4.complete.patch, > HIVE-78.4.no_thrift.patch, HIVE-78.5.complete.patch, > HIVE-78.5.no_thrift.patch, HIVE-78.6.complete.patch, > HIVE-78.6.no_thrift.patch, HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, > HIVE-78.9.no_thrift.patch, HIVE-78.9.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12972626#action_12972626 ] John Sichi commented on HIVE-78: @Alan: we discussed this in depth at the last Hive contributor meeting: http://wiki.apache.org/hadoop/Hive/Development/ContributorsMeetings/HiveContributorsMinutes101025 Let's talk to Carl about scheduling the next one and make sure we find a timeslot where you can make it. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, HIVE-78.4.complete.patch, > HIVE-78.4.no_thrift.patch, HIVE-78.5.complete.patch, > HIVE-78.5.no_thrift.patch, HIVE-78.6.complete.patch, > HIVE-78.6.no_thrift.patch, HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, > HIVE-78.9.no_thrift.patch, HIVE-78.9.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12972622#action_12972622 ] Alan Gates commented on HIVE-78: Having Hive own all the files and run all the jobs presents serious security issues since UDFs would be running code as root. This would also pose problems for Howl, as Pig and MR can't runs jobs as Hive. Maybe this isn't the right forum for this discussion. If there's a better one, let me know. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, HIVE-78.4.complete.patch, > HIVE-78.4.no_thrift.patch, HIVE-78.5.complete.patch, > HIVE-78.5.no_thrift.patch, HIVE-78.6.complete.patch, > HIVE-78.6.no_thrift.patch, HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, > HIVE-78.9.no_thrift.patch, HIVE-78.9.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12972256#action_12972256 ] He Yongqiang commented on HIVE-78: -- I think this jira is just a first step towards a fulfilled security feature. It just does the meta-store check to see if a given user be able to issue the query or not. There is no integration with HDFS/MR part. So the file owner and the job executer are just the same as now. A long term plan is to set up HiveServer. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, HIVE-78.4.complete.patch, > HIVE-78.4.no_thrift.patch, HIVE-78.5.complete.patch, > HIVE-78.5.no_thrift.patch, HIVE-78.6.complete.patch, > HIVE-78.6.no_thrift.patch, HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, > hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12972233#action_12972233 ] Alan Gates commented on HIVE-78: There's been quite a bit of discussion back and forth in this JIRA on who owns the files (Hive or the user) and who MR jobs execute as. The answers to these questions are very important, but I wasn't able to decipher from the JIRA how they were answered. Was one approach or another selected? > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, HIVE-78.4.complete.patch, > HIVE-78.4.no_thrift.patch, HIVE-78.5.complete.patch, > HIVE-78.5.no_thrift.patch, HIVE-78.6.complete.patch, > HIVE-78.6.no_thrift.patch, HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, > hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[
https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12971793#action_12971793
]
He Yongqiang commented on HIVE-78:
--
@John
Regarding the thrift API's object embedding, do you mean define some new object
in thrift like:
strung TableRef {
string dbname
string tablename
}
and similar to Partition?
That sounds good to me.
> Authorization infrastructure for Hive
> -
>
> Key: HIVE-78
> URL: https://issues.apache.org/jira/browse/HIVE-78
> Project: Hive
> Issue Type: New Feature
> Components: Metastore, Query Processor, Server Infrastructure
>Reporter: Ashish Thusoo
>Assignee: He Yongqiang
> Attachments: createuser-v1.patch, hive-78-metadata-v1.patch,
> hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch,
> HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, HIVE-78.4.complete.patch,
> HIVE-78.4.no_thrift.patch, HIVE-78.5.complete.patch,
> HIVE-78.5.no_thrift.patch, HIVE-78.6.complete.patch,
> HIVE-78.6.no_thrift.patch, HIVE-78.7.no_thrift.patch, HIVE-78.7.patch,
> hive-78.diff
>
>
> Allow hive to integrate with existing user repositories for authentication
> and authorization infromation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12971555#action_12971555 ] John Sichi commented on HIVE-78: Regarding pass-by-name vs pass-by-value for object references in the Thrift API, take a look at how drop table works. We already fetch the table descriptor in DDLTask (so that we can include its info in the posthook). But then, when we drop the table, we pass dbname+tblname (not the actual table object). So I don't see the need to invent a new pattern here. For dealing with compound names, it's fine to define a new struct ObjectReference with object type plus various optional components, then pass that. (In the future, we could also decide to hide an ID in there for the lookup-skipping optimization you mention if it turns out to be warranted.) > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, HIVE-78.4.complete.patch, > HIVE-78.4.no_thrift.patch, HIVE-78.5.complete.patch, > HIVE-78.5.no_thrift.patch, HIVE-78.6.complete.patch, > HIVE-78.6.no_thrift.patch, HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, > hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12971460#action_12971460 ] He Yongqiang commented on HIVE-78: -- By "If you want, do follow ups on them." I meant "if you want, open follow up jiras and assign to me" Here are some points that why they are not easy to do: For JDO embedding, Mostly in the new Objects, there are Table object, Database object, Partition object. If we only keep name for them, It's ok for database. But for Table, need to user dbName, tableName, For partition need dbName, tableName, partName. And need to fetch the object on client side to see the object exist or not. And pass the names to meta-store, the metastore will do another lookup to find ids for db/tbl/part to put into new objects. For thrift apis, one benefit of consolidating into one is reducing the api numbers. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, HIVE-78.4.complete.patch, > HIVE-78.4.no_thrift.patch, HIVE-78.5.complete.patch, > HIVE-78.5.no_thrift.patch, HIVE-78.6.complete.patch, > HIVE-78.6.no_thrift.patch, HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, > hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12971455#action_12971455 ] He Yongqiang commented on HIVE-78: -- No. I do not think i need to make changes in short term for the JDO and thrift apis. If you want, do follow ups on them. 7. Provide a way to make partitions inherit from table (and make it the default) This can be done in a follow-up jira. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, HIVE-78.4.complete.patch, > HIVE-78.4.no_thrift.patch, HIVE-78.5.complete.patch, > HIVE-78.5.no_thrift.patch, HIVE-78.6.complete.patch, > HIVE-78.6.no_thrift.patch, HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, > hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12971442#action_12971442 ] John Sichi commented on HIVE-78: We can't take the size of a patch as a justification for checking in code which doesn't pass review, especially for things like JDO and Thrift API's which are going to be there forever. I discussed it with Namit and his suggestion was to break it down into smaller patches to be committed in sequence so that we can divide-and-conquer the review process. For future projects, it would be great if we can do the same for the design process itself so that the coding doesn't get too far ahead of the design (which is how we end up with giant patches). The items below are OK for followups 2. revokeAllPrivileges should revoke role grants as well 3. Role cycle is not being prevented 6. GRANT should mark WriteEntity for replication etc For this one, we should at least work out the metastore model as part of the JDO changes: 7. Provide a way to make partitions inherit from table (and make it the default) The rest need to be addressed up front as part of the relevant patches. Separately, maybe using git for branch+merge would help make development of a feature of this size more manageable? (If you're not already.) > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, HIVE-78.4.complete.patch, > HIVE-78.4.no_thrift.patch, HIVE-78.5.complete.patch, > HIVE-78.5.no_thrift.patch, HIVE-78.6.complete.patch, > HIVE-78.6.no_thrift.patch, HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, > hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12971428#action_12971428 ] He Yongqiang commented on HIVE-78: -- Let's get this in asap and do follow-ups. It is really painful to maintain it. And there are not a few big changes from the first patch. Just need to update it few weeks later after the previous patches. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, HIVE-78.4.complete.patch, > HIVE-78.4.no_thrift.patch, HIVE-78.5.complete.patch, > HIVE-78.5.no_thrift.patch, HIVE-78.6.complete.patch, > HIVE-78.6.no_thrift.patch, HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, > hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12971419#action_12971419 ] He Yongqiang commented on HIVE-78: -- Needs to open follow-up jiras for: 1. Avoid embedding objects inside of other objects except where necessary. 2. revokeAllPrivileges should revoke role grants as well 3. Role cycle is not being prevented 4. try/finally around transactions in ObjectStore should be used consistently 5. more negative tests 6. GRANT should mark WriteEntity for replication etc 7. Provide a way to make partitions inherit from table (and make it the default) 8: Multiple grants from the same grantor to the same grantee should not result in duplicates > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, HIVE-78.4.complete.patch, > HIVE-78.4.no_thrift.patch, HIVE-78.5.complete.patch, > HIVE-78.5.no_thrift.patch, HIVE-78.6.complete.patch, > HIVE-78.6.no_thrift.patch, HIVE-78.7.no_thrift.patch, HIVE-78.7.patch, > hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12971161#action_12971161 ] John Sichi commented on HIVE-78: Some more from me: * There's a bug when attempting to grant multiple privileges at once; only one of them is getting granted (what I showed you in CLI) * Multiple grants from the same grantor to the same grantee should not result in duplicates (verify against Oracle), and we should collapse everything into one row no matter whether the grants were made at the same or different times (sort privilege names for determinism) * revokeAllPrivileges should revoke role grants as well * Role cycle is not being prevented * try/finally around transactions in ObjectStore should be used consistently (I know there are some cases which were already missing them, but we shouldn't make it worse) * Don't use printStackTrace * show [role] grant role unknown should fail (even though we have to tolerate unknown for user/group since we don't have a table for those) Some additional points noted at code review session: * Need many many negative tests * Provide a way to make partitions inherit from table (and make it the default) * Define a UNIQUE key for the priv tables in JDO * GRANT should mark WriteEntity for replication etc More Typos: * candicate * anaylze I have some more code-level comments but not all of them may be relevant after the issues above have been resolved, so I'll do another pass after the next patch. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, HIVE-78.4.complete.patch, > HIVE-78.4.no_thrift.patch, HIVE-78.5.complete.patch, > HIVE-78.5.no_thrift.patch, HIVE-78.6.complete.patch, > HIVE-78.6.no_thrift.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[
https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12970662#action_12970662
]
John Sichi commented on HIVE-78:
First batch of review comments.
JDO:
* Do we want roles to be contained by databases? Let's discuss this
at next design review.
* Instead of two separate flags (IS_ROLE/IS_GROUP) should we instead use
an enum for principal type { USER, GROUP, ROLE }?
* Naming suggestions (if accepted, propagate to Thrift API also):
** SECURITYROLE -> ROLES
** SECURITYROLEMAP -> ROLE_MAP
** SECURITYUSER -> GLOBAL_PRIVS
** SECURITYDB -> DB_PRIVS
** SECURITYTBLPART -> TBLPART_PRIVS
** SECURITYCOLUMN -> COL_PRIVS
* VARCHAR precision for "privileges" fields should be 4000
* Since we're going to need to record GRANT OPTION eventually, maybe
we should add it now so that we don't have to ALTER TABLE later?
Thrift API:
* Avoid embedding objects inside of other objects except where
necessary. For example, in the definition of struct Role, use
dbName instead of a Database object (assuming we keep roles as
contained by databases). Likewise, in PrivilegeBag, the map keys
should be identifiers, not objects. This applies to quite a few of
the new structs.
* Can we reduce the number of new structs and API calls by
consolidating different object types? For example, for the
get_XXX_privilege_set calls, just have one, and take object
type+identifier.
* Add comments for all new methods.
Config:
* Why is hive.exec.security used for some config params instead of
hive.security? Also, those parameter names should make it clear
that they are default grants. Also, do we really need owner grants
(don't owners automatically have full privileges implicitly)?
* Looks like hive.variable.substitute crept in from some other patch.
* Comments for plugin-loading parameters should make it explicit
exactly which interface they are supposed to implement.
* Comment for role grants says "to some groups" instead.
Pluggable Interfaces:
* I don't think we need the factory classes; just add new methods to
HiveUtils (and follow the classloading pattern used there)
* Rename AuthorizationProvider to HiveAuthorizationProvider
and make it extend Configurable
* Rename AuthorizationProviderManager to AbstractAuthorizationProvider
* All outside references should be to the interface (HiveAuthorizationProvider)
not the abstract class.
* Rename Authenticator to HiveAuthenticationProvider and make it
extend Configurable
* Javadoc?
Typos:
* principla
* Authrization
* GrantInfor
* privielges
* "Table is partitioned, but partition spec found"
* DummpyAuthenticator
* detroy
* wheenve
Implementation:
* why does doAuthorization return a boolean when it just throws
anyway?
* more coming...
> Authorization infrastructure for Hive
> -
>
> Key: HIVE-78
> URL: https://issues.apache.org/jira/browse/HIVE-78
> Project: Hive
> Issue Type: New Feature
> Components: Metastore, Query Processor, Server Infrastructure
>Reporter: Ashish Thusoo
>Assignee: He Yongqiang
> Attachments: createuser-v1.patch, hive-78-metadata-v1.patch,
> hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch,
> HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, HIVE-78.4.complete.patch,
> HIVE-78.4.no_thrift.patch, HIVE-78.5.complete.patch,
> HIVE-78.5.no_thrift.patch, HIVE-78.6.complete.patch,
> HIVE-78.6.no_thrift.patch, hive-78.diff
>
>
> Allow hive to integrate with existing user repositories for authentication
> and authorization infromation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12970002#action_12970002 ] John Sichi commented on HIVE-78: HIVE-78.6.no_thrift.patch has the thrift-generated code. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, HIVE-78.4.complete.patch, > HIVE-78.4.no_thrift.patch, HIVE-78.5.complete.patch, > HIVE-78.5.no_thrift.patch, HIVE-78.6.complete.patch, > HIVE-78.6.no_thrift.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12969896#action_12969896 ] He Yongqiang commented on HIVE-78: -- You can find it from the complete patch. will rebase the patch against the new thrift. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, HIVE-78.4.complete.patch, > HIVE-78.4.no_thrift.patch, HIVE-78.5.complete.patch, > HIVE-78.5.no_thrift.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12969890#action_12969890 ] John Sichi commented on HIVE-78: Taking a first look at this one; I will have a number of suggestions on naming/structure for thrift and JDO. I think you accidentally omitted the org.apache.hadoop.hive.ql.security.authorization package since I see references to it but no code. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, HIVE-78.4.complete.patch, > HIVE-78.4.no_thrift.patch, HIVE-78.5.complete.patch, > HIVE-78.5.no_thrift.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933291#action_12933291 ] He Yongqiang commented on HIVE-78: -- >>Can you check who adds inputs/outputs for locking operations ? It seems no inputs and outputs for lock/unlock. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12932587#action_12932587 ] Namit Jain commented on HIVE-78: In case of dynamic partitions, you can also have DummyPartition outputs. They will contain the correct Table definition. Are you taking care of them ? > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[
https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12932354#action_12932354
]
Namit Jain commented on HIVE-78:
Driver:
//do the authorization check
385 if (HiveConf.getBoolVar(conf,
386 HiveConf.ConfVars.HIVE_AUTHORIZATION_ENABLED)) {
387 boolean pass = doAuthorization(sem);
388 if (!pass) {
389 console.printError("Authrizatio\
n failed (not enough privileges found t?
o run the query.).");
390 return (400);
391 }
392 }
Can we print the reason which privilege was missing ?
Can we optimize the scenario - we are checking for all partitions one-by-one
both for inputs and outputs ? What if the user/group/role has the table
privilege - we dont need to go over all the partitions one by one.
We can even do this in a follow-up
Why do we need the change in QueryPlan ?
showGrants: should the output have a schema ? Going forwad, it will
be easier for JDBC clients to parse.
No need to change WriteEntity etc. ?
user cannot be made a reserved word - ~20 tables have a column called 'user'
in facebook - please check 'role' and 'option'.
SemanticAnalyzer: 3511 not needed
What happens to replication of roles - needs to be done
Where are the privileges copied for a newly created partition ?
> Authorization infrastructure for Hive
> -
>
> Key: HIVE-78
> URL: https://issues.apache.org/jira/browse/HIVE-78
> Project: Hive
> Issue Type: New Feature
> Components: Metastore, Query Processor, Server Infrastructure
>Reporter: Ashish Thusoo
>Assignee: He Yongqiang
> Attachments: createuser-v1.patch, hive-78-metadata-v1.patch,
> hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch,
> HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, hive-78.diff
>
>
> Allow hive to integrate with existing user repositories for authentication
> and authorization infromation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12932306#action_12932306 ] Namit Jain commented on HIVE-78: Few minor comments: 1. Can you add more comments in M* files (the new files in the metastore) ? 2. MRoleEntiry needs a database name - so does the thirft file ? 3. Can you verify that create and create table as select works for hive replication ? 4. Can you check who adds inputs/outputs for locking operations ? > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931518#action_12931518 ] Namit Jain commented on HIVE-78: Also, can you refresh and re-apply the patch ? It does not apply cleanly and is therefore not possible to actually compile/test and understand. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931504#action_12931504 ] Namit Jain commented on HIVE-78: Can you add the tests in the non-thrift patch ? It becomes easier to review > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12930822#action_12930822 ] He Yongqiang commented on HIVE-78: -- >>Will there be a way to turn off authorization (through some configuration >>property) Yes. >>is authorization implementation going to be pluggable? Yes. This is exactly what we wanted. I think Howl can just plug in its own authorization implementation. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12930820#action_12930820 ] Pradeep Kamath commented on HIVE-78: Will there be a way to turn off authorization (through some configuration property) OR is there a way to allow all access OR is authorization implementation going to be pluggable? Since howl is looking at a different authorization model based on dfs permissions, one of these options would be needed for howl. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12929896#action_12929896 ] John Sichi commented on HIVE-78: It looks like HIVE-78.1.nothrift.patch still has a bunch of thrift-generate files in it (metastore/src/gen-javabean/org/apache/hadoop/hive/metastore/api/*) > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12929888#action_12929888 ] John Sichi commented on HIVE-78: https://reviews.apache.org/r/55/diff/#index_header > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924039#action_12924039 ] Carl Steinbach commented on HIVE-78: @Namit: I think it's fine to take an incremental approach with this, but then it's important to spell out what the known security holes are so users and administrators know what they're getting. Otherwise we're going to spend a lot of time answering questions on the hive-user list. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924032#action_12924032 ] John Sichi commented on HIVE-78: (implementation note) If we really need multiple metastore tables, let's name them consistently: user_priv db_priv tbl_priv col_priv > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924026#action_12924026 ] Namit Jain commented on HIVE-78: Overall, there are many security holes in the system. and we are not proposing to close all of them. To start with, it is an attempt for good users, it is not meant for the malicious users - the idea is to prevent good users from committing a mistake. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923734#action_12923734 ] He Yongqiang commented on HIVE-78: -- By-passing the hdfs permission from hive layer is just one option. And the implementation should also support setting user groups in the hdfs side. And let the mapreduce job run as the user. Just a quick update about the authorization rule: In the offline discussion we had internally this afternoon, remove DENY should also another option to be considered. And we examined our use cased with this (without DENY), it works. So remove DENY from the authorization will simplify the implementation a lot. And regarding view and index, for the first version, we should not do that. And we can do them later when we have a better understanding after we implement the first version. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[
https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923733#action_12923733
]
Carl Steinbach commented on HIVE-78:
The issue that Todd raised is pretty important and needs to be addressed in the
proposal.
My personal opinion is that running all queries as a "hive" super-user is the
most
practical approach and will also yield behavior that is familiar to users of
traditional
RDBMS systems (who I expect will increasingly define the average Hive
user/administrator).
There are some other follow-on issues that need to be decided if we end up
settling
on this approach:
* This approach to authorization presupposes that users are accessing Hive
through a HiveServer process. This follows from the fact that A) you want Hive
to execute the query plans as the Hive superuser, and B) that user can
circumvent the authorization model if they are given direct access to the
MetaStore DB. It would be nice if the proposal explicitly stated this
requirement and mentioned some of the follow-on work that this necessitates,
e.g. fixing concurrency issues in HiveServer, reducing the memory requirements
of HiveServer, etc.
* We need to apply the authorization model to the '{{add [archive|file|jar]}}'
commands as well as {{add temorary function}}. {{add jar}} and {{add file}}
both currently allow the user to inject code into MR jobs, and {{add jar}} in
conjunction with {{add temporary function}} allows the user to inject and
execute arbitrary code within the HiveServer process. We may also want to add a
new {{add executable}} command for adding executable scripts that has a
different permission model than {{add file}}.
* I think there also may be security issues stemming from external tables, e.g.
if I create an external table that points to another user's home directory and
then run a query on it which executes with Hive's superuser permissions.
* Loading date into the Hive warehouse from an arbitrary HDFS location and
exporting data to other locations in HDFS are two issues that need to be
considered. In each case I think the correct behavior depends on both the Hive
process's permissions and those of the user.
> Authorization infrastructure for Hive
> -
>
> Key: HIVE-78
> URL: https://issues.apache.org/jira/browse/HIVE-78
> Project: Hive
> Issue Type: New Feature
> Components: Metastore, Query Processor, Server Infrastructure
>Reporter: Ashish Thusoo
>Assignee: He Yongqiang
> Attachments: createuser-v1.patch, hive-78-metadata-v1.patch,
> hive-78-syntax-v1.patch, hive-78.diff
>
>
> Allow hive to integrate with existing user repositories for authentication
> and authorization infromation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923719#action_12923719 ] Todd Lipcon commented on HIVE-78: - I'm a little unclear on how the user identity is passed down to the MR layer. Carl and I had chatted about this a few weeks back -- is the idea now that all hive queries will run MR jobs as a "hive" user, rather than "todd"? If so, we need to add authorization control for UDFs and TRANSFORM as well, since a user could trivially take over the "hive" user credentials from within a UDF. If the MR jobs will continue to run as "todd", then I don't understand how we can apply any permissions model that is any different than HDFS permissions. More restrictive is impossible because I can just read the files myself, and less restrictive is impossible because HDFS is applying permissions based on the "todd" identity. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923671#action_12923671 ] He Yongqiang commented on HIVE-78: -- Sorry, in the previous comment: by "one accept then accept; one deny then deny", i mean "Accept overwrite deny. one accept then accept; no accept then deny" > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923667#action_12923667 ] He Yongqiang commented on HIVE-78: -- The other option we came up from offline discussion is the rule of "one accept then accept" but in a hierarchy style. First check privileges granted the user and groups. One accept then accept; One deny then deny. And then check role level privileges, one accept then accept; one deny then deny. We prefer to go with this rule. Please comment, and if no concerns on this, i will update the wiki. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923642#action_12923642 ] He Yongqiang commented on HIVE-78: -- @dhruba HDFS has its own authorization. So if we allow an access in Hive layer and pass this access to HDFS (by setting the correct hdfs username and groups), the job can fail with HDFS permission problem. So need to solve the problem from 2 layer independent authorization. One way to allow all accesses to HDFS, and let hive do the authorization. So hive runs as root in terms of HDFS. The other way is to plug in HDFS authorization to Hive layer, and only accept one access if both of Hive and HDFS say YES. A user belongs to different unix groups, and set hdfs permission based on the unix group. [ I am not sure about how many groups a user can have in terms of HDFS. I mean how many group settings you can put to a hdfs file. Let's simply say i want these 2 groups to be able to read the file.] The another problem is the column level privileges. This is very open for discussion, please comment on it. About the proposal, there is one authorization rule that we are not sure about. It's the simple rule: one deny then deny. Let's say this example: 5.3.1 I want to grant everyone (new people may join at anytime) to db_name.*, and then later i want to protect one table db_name.T from ALL users but a few 1) Add all users to a group 'users'. (assumption: new users will automatically join this group). And grant 'users' ALL privileges to db_name.* 2) Add those few users to a new group 'users2'. AND REMOVE them from 'users' 3) DENY 'users' to db_name.T 4) Grant ALL on db_name.T to users2 The main problem in this approach is that "REMOVE them from 'users'" is not practicable. The other options that we have thought about is another rule. First try user name: first try to deny this access by look up the deny tables by user name: 1. If there is an entry in 'user' that deny this access, return DENY 2. If there is an entry in 'db' that deny this access, return DENY 3. If there is an entry in 'table' that deny this access, return DENY 4. If there is an entry in 'column' that deny this access, return DENY If we got one deny, will return deny for this attempt. if deny failed, go through all privilege levels with the user name: 5. If there is an entry in 'user' that accept this access, return ACCEPT 6. If there is an entry in 'db' that accept this access, return ACCEPT 7. If there is an entry in 'table' that accept this access, return ACCEPT 8. If there is an entry in 'column' that accept this access, return ACCEPT Second try the user's group/role names one by one until we get an ACCEPT. If we get an ACCEPT from one group/role, will ACCEPT this access. Else deny. For each role/group, we do the same routine as we did for user name. The problem with this approach is it's a little bit complex and we did not find any system that use this. For mysql, there is no deny. For sql server, it's one deny then deny. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923624#action_12923624 ] dhruba borthakur commented on HIVE-78: -- Can somebody pl comment on how this ties in with HDFS permission/authorization? There is a small subsection in the doc about this issue, but I am unable to understand that part. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923598#action_12923598 ] Namit Jain commented on HIVE-78: Please comment - we would like to hear all use cases before finalizing the design. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923597#action_12923597 ] Carl Steinbach commented on HIVE-78: Authorization proposal on the wiki: http://wiki.apache.org/hadoop/Hive/AuthDev > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12919104#action_12919104 ] Namit Jain commented on HIVE-78: Is anyone working on this ? > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hadoop Hive > Issue Type: New Feature > Components: Server Infrastructure >Reporter: Ashish Thusoo >Assignee: Edward Capriolo > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
