[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13660463#comment-13660463 ] Zhuoluo (Clark) Yang commented on HIVE-2616: Hi! I am curious about this patch, what will happen if hive.metastore.sasl.enabled is NOT enabled and hive.metastore.execute.setugi is set. Look into it from the code, I think the ugi is passed to the HMS and meaning nothing. The HMS will create/delete HDFS dir use the server side UGI. Is there a way to use client side ugi to let HMS manipulate HDFS without hive.metastore.sasl.enabled? Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: New Feature Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Fix For: 0.8.1, 0.9.0 Attachments: hive-2616_1.patch, hive-2616_3.patch, hive-2616_4.patch, hive-2616_5.patch, hive-2616.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13660466#comment-13660466 ] Zhuoluo (Clark) Yang commented on HIVE-2616: Is there a way to let user create their table/part dir based on their own UGI? Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: New Feature Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Fix For: 0.8.1, 0.9.0 Attachments: hive-2616_1.patch, hive-2616_3.patch, hive-2616_4.patch, hive-2616_5.patch, hive-2616.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[
https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13660474#comment-13660474
]
Zhuoluo (Clark) Yang commented on HIVE-2616:
I think I've got the point.
Is TUGIBasedProcessor.process() doing this?
try {
shim.doAs(clientUgi, pvea);
return true;
} catch (RuntimeException rte) {
Passing user identity from metastore client to server in non-secure mode
Key: HIVE-2616
URL: https://issues.apache.org/jira/browse/HIVE-2616
Project: Hive
Issue Type: New Feature
Components: Metastore
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
Fix For: 0.8.1, 0.9.0
Attachments: hive-2616_1.patch, hive-2616_3.patch, hive-2616_4.patch,
hive-2616_5.patch, hive-2616.patch
Currently in unsecure mode client don't pass on user identity. As a result
hdfs and other operations done by server gets executed by user running
metastore process instead of being done in context of client. This results in
problem as reported here:
http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13548289#comment-13548289 ] Hudson commented on HIVE-2616: -- Integrated in Hive-trunk-hadoop2 #54 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/54/]) HIVE-2616 : Passing user identity from metastore client to server in non-secure mode (Ashutosh Chauhan) (Revision 1225683) Result = ABORTED hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1225683 Files : * /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/trunk/conf/hive-default.xml.template * /hive/trunk/metastore/if/hive_metastore.thrift * /hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp * /hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h * /hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp * /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java * /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php * /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote * /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py * /hive/trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStore.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestSetUGIOnBothClientServer.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyClient.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyServer.java * /hive/trunk/shims/ivy.xml * /hive/trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/client * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/client/TUGIAssumingTransport.java * /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java * /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TFilterTransport.java * /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TUGIContainingTransport.java Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Fix For: 0.8.1, 0.9.0 Attachments: hive-2616_1.patch, hive-2616_3.patch, hive-2616_4.patch, hive-2616_5.patch, hive-2616.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13189119#comment-13189119 ] Hudson commented on HIVE-2616: -- Integrated in Hive-0.8.1-SNAPSHOT-h0.21 #166 (See [https://builds.apache.org/job/Hive-0.8.1-SNAPSHOT-h0.21/166/]) HIVE-2616. Merge -r 1225682:1225683 https://svn.apache.org/repos/asf/hive/trunk . amareshwari : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1233260 Files : * /hive/branches/branch-0.8-r2/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/branches/branch-0.8-r2/conf/hive-default.xml.template * /hive/branches/branch-0.8-r2/metastore/if/hive_metastore.thrift * /hive/branches/branch-0.8-r2/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp * /hive/branches/branch-0.8-r2/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h * /hive/branches/branch-0.8-r2/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp * /hive/branches/branch-0.8-r2/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java * /hive/branches/branch-0.8-r2/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php * /hive/branches/branch-0.8-r2/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote * /hive/branches/branch-0.8-r2/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py * /hive/branches/branch-0.8-r2/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb * /hive/branches/branch-0.8-r2/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java * /hive/branches/branch-0.8-r2/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java * /hive/branches/branch-0.8-r2/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java * /hive/branches/branch-0.8-r2/metastore/src/test/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStore.java * /hive/branches/branch-0.8-r2/metastore/src/test/org/apache/hadoop/hive/metastore/TestSetUGIOnBothClientServer.java * /hive/branches/branch-0.8-r2/metastore/src/test/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyClient.java * /hive/branches/branch-0.8-r2/metastore/src/test/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyServer.java * /hive/branches/branch-0.8-r2/shims/ivy.xml * /hive/branches/branch-0.8-r2/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java * /hive/branches/branch-0.8-r2/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java * /hive/branches/branch-0.8-r2/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java * /hive/branches/branch-0.8-r2/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/client * /hive/branches/branch-0.8-r2/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/client/TUGIAssumingTransport.java * /hive/branches/branch-0.8-r2/shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java * /hive/branches/branch-0.8-r2/shims/src/common/java/org/apache/hadoop/hive/thrift/TFilterTransport.java * /hive/branches/branch-0.8-r2/shims/src/common/java/org/apache/hadoop/hive/thrift/TUGIContainingTransport.java Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Fix For: 0.8.1, 0.9.0 Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch, hive-2616_4.patch, hive-2616_5.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13188313#comment-13188313 ] Carl Steinbach commented on HIVE-2616: -- +1 Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Fix For: 0.8.1, 0.9.0 Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch, hive-2616_4.patch, hive-2616_5.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13177573#comment-13177573 ] Hudson commented on HIVE-2616: -- Integrated in Hive-trunk-h0.21 #1176 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1176/]) HIVE-2616 : Passing user identity from metastore client to server in non-secure mode (Ashutosh Chauhan) hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1225683 Files : * /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/trunk/conf/hive-default.xml.template * /hive/trunk/metastore/if/hive_metastore.thrift * /hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp * /hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h * /hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp * /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java * /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php * /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote * /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py * /hive/trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStore.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestSetUGIOnBothClientServer.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyClient.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyServer.java * /hive/trunk/shims/ivy.xml * /hive/trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/client * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/client/TUGIAssumingTransport.java * /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java * /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TFilterTransport.java * /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TUGIContainingTransport.java Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Fix For: 0.9.0 Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch, hive-2616_4.patch, hive-2616_5.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175345#comment-13175345 ] Carl Steinbach commented on HIVE-2616: -- +1. Will commit if tests pass. Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch, hive-2616_4.patch, hive-2616_5.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[
https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13172662#comment-13172662
]
Ashutosh Chauhan commented on HIVE-2616:
{code}
BUILD SUCCESSFUL
Total time: 302 minutes 28 seconds
{code}
All the tests passed.
Passing user identity from metastore client to server in non-secure mode
Key: HIVE-2616
URL: https://issues.apache.org/jira/browse/HIVE-2616
Project: Hive
Issue Type: Bug
Components: Metastore
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch,
hive-2616_4.patch
Currently in unsecure mode client don't pass on user identity. As a result
hdfs and other operations done by server gets executed by user running
metastore process instead of being done in context of client. This results in
problem as reported here:
http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13172668#comment-13172668 ] jirapos...@reviews.apache.org commented on HIVE-2616: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2975/#review3984 --- Is it possible to add some testcases? Specifically I'd like to see a test that has a client with setugi enabled connect to a server with setugi disabled and vice-versa. trunk/conf/hive-default.xml.template https://reviews.apache.org/r/2975/#comment9004 This describes the action instead of the effect. Please change to something like In unsecure mode, setting this property to true will cause the metastore to execute DFS operations using the client's reported user and group permissions. Note that this property must be set on both the client and server sides. Also, it may be easier to understand if you separate this out into to separate properties: hive.metastore.client.setugi and hive.metastore.server.setugi trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java https://reviews.apache.org/r/2975/#comment9005 Spacing. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java https://reviews.apache.org/r/2975/#comment9000 The formatting/indentation in this method is still not correct. Please use 2 character indents, nested 'else' operators, etc. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java https://reviews.apache.org/r/2975/#comment9009 Please add some logging statements here, e.g. Starting DB backed MetaStore Server in Secure Mode Starting DB backed MetaStore Server Starting DB backed MetaStore Server with SetUGI enabled trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java https://reviews.apache.org/r/2975/#comment9001 If the call to set_ugi() fails, is logging a message and continuing really the right behavior? Why not just fail outright? Also, if you think that continuing is the correct behavior, then I think the description of metastore.execute.setugi should be updated to explain that this is a best effort approach, and it's possible that your setting will not be honored. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java https://reviews.apache.org/r/2975/#comment8999 There's a TAB character here and on line 285. Please remove. trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java https://reviews.apache.org/r/2975/#comment9013 Add a newline. - Carl On 2011-12-17 02:42:36, Ashutosh Chauhan wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/2975/ bq. --- bq. bq. (Updated 2011-12-17 02:42:36) bq. bq. bq. Review request for hive. bq. bq. bq. Summary bq. --- bq. bq. Pass user identity in metastore connection in unsecure mode bq. bq. bq. This addresses bug HIVE-2616. bq. https://issues.apache.org/jira/browse/HIVE-2616 bq. bq. bq. Diffs bq. - bq. bq.trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1215380 bq.trunk/conf/hive-default.xml.template 1215380 bq.trunk/metastore/if/hive_metastore.thrift 1215380 bq.trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 1215380 bq.trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 1215380 bq. trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 1215380 bq. trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 1215380 bq. trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php 1215380 bq. trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 1215380 bq. trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 1215380 bq.trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 1215380 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1215380 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1215380 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java PRE-CREATION bq.trunk/shims/ivy.xml 1215380 bq. trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 1215380 bq. trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 1215380 bq.
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13172691#comment-13172691 ] jirapos...@reviews.apache.org commented on HIVE-2616: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2975/#review3985 --- About testing, see my earlier comment. To do any real tests for this, one needs to do 'sudo' and then run tests. I have not found an easy way to do that in Hive. HCatalog has test infrastructure which makes this kind of thing possible, where I am adding these tests. See, HCATALOG-181 Also, I have manually verified all four combinations of new/old client and new/old server and results were of my satisfaction. trunk/conf/hive-default.xml.template https://reviews.apache.org/r/2975/#comment9018 I will update the text. But, I think splitting it in two properties will be more confusing then useful. If ever some one uses them and both client and server somehow uses same hive-site.xml, then having one property make sure its either turned on or off for both client and server. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java https://reviews.apache.org/r/2975/#comment9019 will fix trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java https://reviews.apache.org/r/2975/#comment9020 will fix trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java https://reviews.apache.org/r/2975/#comment9024 Will add. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java https://reviews.apache.org/r/2975/#comment9021 In my opinion thats a right behavior because you dont want existing application to break when server is upgraded and is running with setugi on. I will update the text about best effort. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java https://reviews.apache.org/r/2975/#comment9022 Will remove. trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java https://reviews.apache.org/r/2975/#comment9023 Will add. - Ashutosh On 2011-12-17 02:42:36, Ashutosh Chauhan wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/2975/ bq. --- bq. bq. (Updated 2011-12-17 02:42:36) bq. bq. bq. Review request for hive. bq. bq. bq. Summary bq. --- bq. bq. Pass user identity in metastore connection in unsecure mode bq. bq. bq. This addresses bug HIVE-2616. bq. https://issues.apache.org/jira/browse/HIVE-2616 bq. bq. bq. Diffs bq. - bq. bq.trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1215380 bq.trunk/conf/hive-default.xml.template 1215380 bq.trunk/metastore/if/hive_metastore.thrift 1215380 bq.trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 1215380 bq.trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 1215380 bq. trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 1215380 bq. trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 1215380 bq. trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php 1215380 bq. trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 1215380 bq. trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 1215380 bq.trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 1215380 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1215380 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1215380 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java PRE-CREATION bq.trunk/shims/ivy.xml 1215380 bq. trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 1215380 bq. trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 1215380 bq. trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1215380 bq. trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/client/TUGIAssumingTransport.java PRE-CREATION bq. trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java 1215380 bq. trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TFilterTransport.java PRE-CREATION bq. trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TUGIContainingTransport.java PRE-CREATION bq. bq. Diff:
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13172698#comment-13172698 ] jirapos...@reviews.apache.org commented on HIVE-2616: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2975/#review3987 --- trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java https://reviews.apache.org/r/2975/#comment9030 you dont want existing application to break when server is upgraded and is running with setugi on Right, but this is the client-side, which is a little different. If the client requests set_ugi and doesn't get it I think it may be better to throw an exception. Right now it doesn't seem like the client has any way of knowing whether or not their set_ugi request has been honored. - Carl On 2011-12-17 02:42:36, Ashutosh Chauhan wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/2975/ bq. --- bq. bq. (Updated 2011-12-17 02:42:36) bq. bq. bq. Review request for hive. bq. bq. bq. Summary bq. --- bq. bq. Pass user identity in metastore connection in unsecure mode bq. bq. bq. This addresses bug HIVE-2616. bq. https://issues.apache.org/jira/browse/HIVE-2616 bq. bq. bq. Diffs bq. - bq. bq.trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1215380 bq.trunk/conf/hive-default.xml.template 1215380 bq.trunk/metastore/if/hive_metastore.thrift 1215380 bq.trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 1215380 bq.trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 1215380 bq. trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 1215380 bq. trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 1215380 bq. trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php 1215380 bq. trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 1215380 bq. trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 1215380 bq.trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 1215380 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1215380 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1215380 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java PRE-CREATION bq.trunk/shims/ivy.xml 1215380 bq. trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 1215380 bq. trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 1215380 bq. trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1215380 bq. trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/client/TUGIAssumingTransport.java PRE-CREATION bq. trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java 1215380 bq. trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TFilterTransport.java PRE-CREATION bq. trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TUGIContainingTransport.java PRE-CREATION bq. bq. Diff: https://reviews.apache.org/r/2975/diff bq. bq. bq. Testing bq. --- bq. bq. All the tests in metastore dir passes. Manually tested that file on hdfs is owned by user running the client and not by user running metastore server. bq. bq. bq. Thanks, bq. bq. Ashutosh bq. bq. Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch, hive-2616_4.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13172703#comment-13172703 ] jirapos...@reviews.apache.org commented on HIVE-2616: - bq. On 2011-12-19 22:45:48, Carl Steinbach wrote: bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java, line 279 bq. https://reviews.apache.org/r/2975/diff/3/?file=65044#file65044line279 bq. bq.you dont want existing application to break when server is upgraded and is running with setugi on bq. bq. Right, but this is the client-side, which is a little different. If the client requests set_ugi and doesn't get it I think it may be better to throw an exception. Right now it doesn't seem like the client has any way of knowing whether or not their set_ugi request has been honored. bq. Right, but this is the client-side, which is a little different. If the client requests set_ugi and doesn't get it I think it may be better to throw an exception. Since this is a current behavior, I am inclined to keep it that way. If we throw an exception, this will be change of behavior and will surprise those apps. bq. Right now it doesn't seem like the client has any way of knowing whether or not their set_ugi request has been honored. Client catches TException and logs it, so client can know about it, if it wants to. - Ashutosh --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2975/#review3987 --- On 2011-12-17 02:42:36, Ashutosh Chauhan wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/2975/ bq. --- bq. bq. (Updated 2011-12-17 02:42:36) bq. bq. bq. Review request for hive. bq. bq. bq. Summary bq. --- bq. bq. Pass user identity in metastore connection in unsecure mode bq. bq. bq. This addresses bug HIVE-2616. bq. https://issues.apache.org/jira/browse/HIVE-2616 bq. bq. bq. Diffs bq. - bq. bq.trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1215380 bq.trunk/conf/hive-default.xml.template 1215380 bq.trunk/metastore/if/hive_metastore.thrift 1215380 bq.trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 1215380 bq.trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 1215380 bq. trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 1215380 bq. trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 1215380 bq. trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php 1215380 bq. trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 1215380 bq. trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 1215380 bq.trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 1215380 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1215380 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1215380 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java PRE-CREATION bq.trunk/shims/ivy.xml 1215380 bq. trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 1215380 bq. trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 1215380 bq. trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1215380 bq. trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/client/TUGIAssumingTransport.java PRE-CREATION bq. trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java 1215380 bq. trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TFilterTransport.java PRE-CREATION bq. trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TUGIContainingTransport.java PRE-CREATION bq. bq. Diff: https://reviews.apache.org/r/2975/diff bq. bq. bq. Testing bq. --- bq. bq. All the tests in metastore dir passes. Manually tested that file on hdfs is owned by user running the client and not by user running metastore server. bq. bq. bq. Thanks, bq. bq. Ashutosh bq. bq. Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13172705#comment-13172705 ] jirapos...@reviews.apache.org commented on HIVE-2616: - bq. On 2011-12-19 22:35:15, Ashutosh Chauhan wrote: bq. About testing, see my earlier comment. To do any real tests for this, one needs to do 'sudo' and then run tests. I have not found an easy way to do that in Hive. HCatalog has test infrastructure which makes this kind of thing possible, where I am adding these tests. See, HCATALOG-181 Also, I have manually verified all four combinations of new/old client and new/old server and results were of my satisfaction. Wouldn't it make more sense to add this test infrastructure directly to Hive where the feature is implemented? Also, manually testing this feature today does nothing to prevent someone from breaking it tomorrow. That's why we need automated test coverage for this patch. - Carl --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2975/#review3985 --- On 2011-12-17 02:42:36, Ashutosh Chauhan wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/2975/ bq. --- bq. bq. (Updated 2011-12-17 02:42:36) bq. bq. bq. Review request for hive. bq. bq. bq. Summary bq. --- bq. bq. Pass user identity in metastore connection in unsecure mode bq. bq. bq. This addresses bug HIVE-2616. bq. https://issues.apache.org/jira/browse/HIVE-2616 bq. bq. bq. Diffs bq. - bq. bq.trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1215380 bq.trunk/conf/hive-default.xml.template 1215380 bq.trunk/metastore/if/hive_metastore.thrift 1215380 bq.trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 1215380 bq.trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 1215380 bq. trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 1215380 bq. trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 1215380 bq. trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php 1215380 bq. trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 1215380 bq. trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 1215380 bq.trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 1215380 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1215380 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1215380 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java PRE-CREATION bq.trunk/shims/ivy.xml 1215380 bq. trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 1215380 bq. trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 1215380 bq. trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1215380 bq. trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/client/TUGIAssumingTransport.java PRE-CREATION bq. trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java 1215380 bq. trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TFilterTransport.java PRE-CREATION bq. trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TUGIContainingTransport.java PRE-CREATION bq. bq. Diff: https://reviews.apache.org/r/2975/diff bq. bq. bq. Testing bq. --- bq. bq. All the tests in metastore dir passes. Manually tested that file on hdfs is owned by user running the client and not by user running metastore server. bq. bq. bq. Thanks, bq. bq. Ashutosh bq. bq. Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch, hive-2616_4.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here:
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13172848#comment-13172848 ] jirapos...@reviews.apache.org commented on HIVE-2616: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2975/ --- (Updated 2011-12-20 02:05:22.736212) Review request for hive. Changes --- Updated patch incorporating Carl's comments. Also added tests as requested by Carl. Summary --- Pass user identity in metastore connection in unsecure mode This addresses bug HIVE-2616. https://issues.apache.org/jira/browse/HIVE-2616 Diffs (updated) - trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1221059 trunk/conf/hive-default.xml.template 1221059 trunk/metastore/if/hive_metastore.thrift 1221059 trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 1221059 trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 1221059 trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 1221059 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 1221059 trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php 1221059 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 1221059 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 1221059 trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 1221059 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1221059 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1221059 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java PRE-CREATION trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStore.java 1221059 trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestSetUGIOnBothClientServer.java PRE-CREATION trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyClient.java PRE-CREATION trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyServer.java PRE-CREATION trunk/shims/ivy.xml 1221059 trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 1221059 trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 1221059 trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1221059 trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/client/TUGIAssumingTransport.java PRE-CREATION trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java 1221059 trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TFilterTransport.java PRE-CREATION trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TUGIContainingTransport.java PRE-CREATION Diff: https://reviews.apache.org/r/2975/diff Testing --- All the tests in metastore dir passes. Manually tested that file on hdfs is owned by user running the client and not by user running metastore server. Thanks, Ashutosh Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch, hive-2616_4.patch, hive-2616_5.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171419#comment-13171419 ] jirapos...@reviews.apache.org commented on HIVE-2616: - bq. On 2011-12-16 10:03:39, Thomas wrote: bq. Instead of introducing set_ugi into the metastore thrift interface, could this not be solved through SASL (looks like a prime use case for SASL)? bq. bq. Have the server request transmission of ugi when configured to do so and the client react accordingly. Similar to how delegation token is transmitted (SaslClientCallbackHandler). I am not sure, how SASL could be used to solve this problem. Furthermore, even if it does it will require lock-step upgrade of *all* clients, which is not desirable, whereas current approach doesn't have this drawback. - Ashutosh --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2975/#review3947 --- On 2011-12-03 00:07:25, Ashutosh Chauhan wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/2975/ bq. --- bq. bq. (Updated 2011-12-03 00:07:25) bq. bq. bq. Review request for hive. bq. bq. bq. Summary bq. --- bq. bq. Pass user identity in metastore connection in unsecure mode bq. bq. bq. This addresses bug HIVE-2616. bq. https://issues.apache.org/jira/browse/HIVE-2616 bq. bq. bq. Diffs bq. - bq. bq.trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1209772 bq.trunk/metastore/if/hive_metastore.thrift 1209772 bq.trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 1209772 bq.trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 1209772 bq. trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 1209772 bq. trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 1209772 bq. trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php 1209772 bq. trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 1209772 bq. trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 1209772 bq.trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 1209772 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1209772 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1209772 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java PRE-CREATION bq.trunk/shims/ivy.xml 1209772 bq. trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 1209772 bq. trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 1209772 bq. trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1209772 bq. trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/TUGIAssumingTransport.java PRE-CREATION bq. trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java 1209772 bq. trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java 1209772 bq. trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TFilterTransport.java PRE-CREATION bq. trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TUGIContainingTransport.java PRE-CREATION bq. bq. Diff: https://reviews.apache.org/r/2975/diff bq. bq. bq. Testing bq. --- bq. bq. All the tests in metastore dir passes. Manually tested that file on hdfs is owned by user running the client and not by user running metastore server. bq. bq. bq. Thanks, bq. bq. Ashutosh bq. bq. Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here:
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171421#comment-13171421 ] jirapos...@reviews.apache.org commented on HIVE-2616: - bq. On 2011-12-16 03:06:59, Carl Steinbach wrote: bq. trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 237 bq. https://reviews.apache.org/r/2975/diff/2/?file=61777#file61777line237 bq. bq. All properties that appear in HiveConf also need to appear in conf/hive-default.xml.template along with a description. bq. Done. bq. On 2011-12-16 03:06:59, Carl Steinbach wrote: bq. trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 238 bq. https://reviews.apache.org/r/2975/diff/2/?file=61777#file61777line238 bq. bq. bq. I think it would make sense to change the name to 'hive.metastore.client.enable.setugi'. Also, I think this feature should be disabled by default. bq. bq. Done. False by default. bq. On 2011-12-16 03:06:59, Carl Steinbach wrote: bq. trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 239 bq. https://reviews.apache.org/r/2975/diff/2/?file=61777#file61777line239 bq. bq. Please add a new property hive.metastore.server.enable.setugi that allows this RPC to be disabled on the server side, and set the default value to false. I reused same config hive.metastore.execute.setugi at both client and server which is off by default. bq. On 2011-12-16 03:06:59, Carl Steinbach wrote: bq. trunk/metastore/if/hive_metastore.thrift, line 438 bq. https://reviews.apache.org/r/2975/diff/2/?file=61778#file61778line438 bq. bq. Please add a comment explaining what this call does. Added comment. bq. On 2011-12-16 03:06:59, Carl Steinbach wrote: bq. trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java, line 145 bq. https://reviews.apache.org/r/2975/diff/2/?file=61782#file61782line145 bq. bq. When I apply your changes and run the thriftif ant target I see a small diff in this file. Did you use Thrift 0.7.0 to generate these files? I am not sure how that happened. I reran ant thriftif again. So, those should go away now. bq. On 2011-12-16 03:06:59, Carl Steinbach wrote: bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 3589 bq. https://reviews.apache.org/r/2975/diff/2/?file=61787#file61787line3589 bq. bq. Indentation. Fixed. bq. On 2011-12-16 03:06:59, Carl Steinbach wrote: bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 3743 bq. https://reviews.apache.org/r/2975/diff/2/?file=61787#file61787line3743 bq. bq. So instead of checking the hive.metastore.sasl.enabled property we now just check to see if we're using a security enabled shim, and if so assume that the user wants to enable security? I don't think this is correct behavior since the fact that we're using a secure version of Hadoop does not necessarily imply that we actually have security enabled. bq. bq. Also, it looks like this change deprecates the hive.metastore.sasl.enabled configuration property. In line with my comment above I think it makes sense to leave this property in, but if you do remove it then you need to release note the change and remove this property from HiveConf and conf/hive-default.xml.template. Reverted back to use old config variables to avoid the issues outlined. bq. On 2011-12-16 03:06:59, Carl Steinbach wrote: bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 3751 bq. https://reviews.apache.org/r/2975/diff/2/?file=61787#file61787line3751 bq. bq. Indentation. Fixed. bq. On 2011-12-16 03:06:59, Carl Steinbach wrote: bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 3756 bq. https://reviews.apache.org/r/2975/diff/2/?file=61787#file61787line3756 bq. bq. We're initializing SASL even if isSecure=false? Fixed. bq. On 2011-12-16 03:06:59, Carl Steinbach wrote: bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 3758 bq. https://reviews.apache.org/r/2975/diff/2/?file=61787#file61787line3758 bq. bq. Formatting. Fixed. bq. On 2011-12-16 03:06:59, Carl Steinbach wrote: bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java, line 263 bq. https://reviews.apache.org/r/2975/diff/2/?file=61788#file61788line263 bq. bq. Formatting: please add spaces. Fixed. bq. On 2011-12-16 03:06:59, Carl Steinbach wrote: bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java, line 280 bq. https://reviews.apache.org/r/2975/diff/2/?file=61788#file61788line280 bq. bq. Should this be Failed to login to the
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171423#comment-13171423 ] jirapos...@reviews.apache.org commented on HIVE-2616: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2975/ --- (Updated 2011-12-17 02:42:36.039580) Review request for hive. Changes --- Updated the patch to address Carl's comments. Summary --- Pass user identity in metastore connection in unsecure mode This addresses bug HIVE-2616. https://issues.apache.org/jira/browse/HIVE-2616 Diffs (updated) - trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1215380 trunk/conf/hive-default.xml.template 1215380 trunk/metastore/if/hive_metastore.thrift 1215380 trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 1215380 trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 1215380 trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 1215380 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 1215380 trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php 1215380 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 1215380 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 1215380 trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 1215380 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1215380 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1215380 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java PRE-CREATION trunk/shims/ivy.xml 1215380 trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 1215380 trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 1215380 trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1215380 trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/client/TUGIAssumingTransport.java PRE-CREATION trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java 1215380 trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TFilterTransport.java PRE-CREATION trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TUGIContainingTransport.java PRE-CREATION Diff: https://reviews.apache.org/r/2975/diff Testing --- All the tests in metastore dir passes. Manually tested that file on hdfs is owned by user running the client and not by user running metastore server. Thanks, Ashutosh Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch, hive-2616_4.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170521#comment-13170521 ] John Sichi commented on HIVE-2616: -- OK, can you submit the thrift patch? If we'll be able to uptake that eventually, then I'm fine with the current approach. I was thinking dynamic proxy would allow us to do the necessary method interception without needing to futz with method accessibility. Carl, are you good with this? Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170682#comment-13170682 ] Ashutosh Chauhan commented on HIVE-2616: Sure. I uploaded the patch on THRIFT-1465 Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170685#comment-13170685 ] Carl Steinbach commented on HIVE-2616: -- @John: I'm looking at the patch now. Will respond soon with some comments. Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[
https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170726#comment-13170726
]
jirapos...@reviews.apache.org commented on HIVE-2616:
-
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2975/#review3939
---
trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
https://reviews.apache.org/r/2975/#comment8885
All properties that appear in HiveConf also need to appear in
conf/hive-default.xml.template along with a description.
trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
https://reviews.apache.org/r/2975/#comment8893
I think it would make sense to change the name to
'hive.metastore.client.enable.setugi'. Also, I think this feature should be
disabled by default.
trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
https://reviews.apache.org/r/2975/#comment8894
Please add a new property hive.metastore.server.enable.setugi that allows
this RPC to be disabled on the server side, and set the default value to false.
trunk/metastore/if/hive_metastore.thrift
https://reviews.apache.org/r/2975/#comment8895
Please add a comment explaining what this call does.
trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
https://reviews.apache.org/r/2975/#comment8886
When I apply your changes and run the thriftif ant target I see a small
diff in this file. Did you use Thrift 0.7.0 to generate these files?
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
https://reviews.apache.org/r/2975/#comment8887
Indentation.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
https://reviews.apache.org/r/2975/#comment
So instead of checking the hive.metastore.sasl.enabled property we now just
check to see if we're using a security enabled shim, and if so assume that the
user wants to enable security? I don't think this is correct behavior since the
fact that we're using a secure version of Hadoop does not necessarily imply
that we actually have security enabled.
Also, it looks like this change deprecates the hive.metastore.sasl.enabled
configuration property. In line with my comment above I think it makes sense to
leave this property in, but if you do remove it then you need to release note
the change and remove this property from HiveConf and
conf/hive-default.xml.template.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
https://reviews.apache.org/r/2975/#comment8896
Indentation.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
https://reviews.apache.org/r/2975/#comment8897
We're initializing SASL even if isSecure=false?
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
https://reviews.apache.org/r/2975/#comment8889
Formatting.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
https://reviews.apache.org/r/2975/#comment8891
Formatting: please add spaces.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
https://reviews.apache.org/r/2975/#comment8898
new client talking to old metastore seems to imply that we're able to
determine whether or not we're talking to an old server, which isn't true. In
reality, the onus is on the admin to ensure that both sides support this
feature. What happens if the client calls set_ugi(), but the server doesn't
support it? Is the error message helpful?
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
https://reviews.apache.org/r/2975/#comment8892
Should this be Failed to login to the MetaStore Server...?
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java
https://reviews.apache.org/r/2975/#comment8901
I think it's more accurate to say that the processor *checks* to see if
the first call is to set_ugi()... instead of saying that it *expects* the
first call to be to set_ugi().
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java
https://reviews.apache.org/r/2975/#comment8902
+1 to referencing the THRIFT JIRA. I think the class comment should call
out that this is a temporary workaround cite a TODO.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java
https://reviews.apache.org/r/2975/#comment8899
Formatting: '} else {'
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java
https://reviews.apache.org/r/2975/#comment8900
There's a TAB here. Please remove.
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13169510#comment-13169510 ] Ashutosh Chauhan commented on HIVE-2616: @John, I don't see how Dynamic Proxy will be better then current approach. Would you like to expand a bit on that? Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13168668#comment-13168668 ] Ashutosh Chauhan commented on HIVE-2616: Any feedback on this will be appreciated. Since, it was easier to add a test case using hcatalog test harness, I have added a test there on HCATALOG-181 Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13168782#comment-13168782 ] jirapos...@reviews.apache.org commented on HIVE-2616: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2975/#review3886 --- trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java https://reviews.apache.org/r/2975/#comment8718 Typos: it's, don't trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java https://reviews.apache.org/r/2975/#comment8719 Need ASF header on all new files. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java https://reviews.apache.org/r/2975/#comment8720 remove extra spaces trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java https://reviews.apache.org/r/2975/#comment8721 I don't think we should commit this without at least a reference to a Thrift patch to get the necessary enhancement for making this brittle jankiness unnecessary. Alternatively, is it possible to use a dynamic proxy to avoid this? - John On 2011-12-03 00:07:25, Ashutosh Chauhan wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/2975/ bq. --- bq. bq. (Updated 2011-12-03 00:07:25) bq. bq. bq. Review request for hive. bq. bq. bq. Summary bq. --- bq. bq. Pass user identity in metastore connection in unsecure mode bq. bq. bq. This addresses bug HIVE-2616. bq. https://issues.apache.org/jira/browse/HIVE-2616 bq. bq. bq. Diffs bq. - bq. bq.trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1209772 bq.trunk/metastore/if/hive_metastore.thrift 1209772 bq.trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 1209772 bq.trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 1209772 bq. trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 1209772 bq. trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 1209772 bq. trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php 1209772 bq. trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 1209772 bq. trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 1209772 bq.trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 1209772 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1209772 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1209772 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java PRE-CREATION bq.trunk/shims/ivy.xml 1209772 bq. trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 1209772 bq. trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 1209772 bq. trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1209772 bq. trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/TUGIAssumingTransport.java PRE-CREATION bq. trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java 1209772 bq. trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java 1209772 bq. trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TFilterTransport.java PRE-CREATION bq. trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TUGIContainingTransport.java PRE-CREATION bq. bq. Diff: https://reviews.apache.org/r/2975/diff bq. bq. bq. Testing bq. --- bq. bq. All the tests in metastore dir passes. Manually tested that file on hdfs is owned by user running the client and not by user running metastore server. bq. bq. bq. Thanks, bq. bq. Ashutosh bq. bq. Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13161934#comment-13161934 ] jirapos...@reviews.apache.org commented on HIVE-2616: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2975/ --- (Updated 2011-12-02 23:46:56.664874) Review request for hive. Changes --- Updated patch Summary --- Pass user identity in metastore connection in unsecure mode This addresses bug HIVE-2616. https://issues.apache.org/jira/browse/HIVE-2616 Diffs (updated) - trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1209772 trunk/metastore/if/hive_metastore.thrift 1209772 trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 1209772 trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 1209772 trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 1209772 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 1209772 trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php 1209772 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 1209772 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 1209772 trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 1209772 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1209772 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1209772 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java PRE-CREATION trunk/shims/ivy.xml 1209772 trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 1209772 trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 1209772 trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1209772 trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/TUGIAssumingTransport.java PRE-CREATION trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java 1209772 trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java 1209772 trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TFilterTransport.java PRE-CREATION trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TUGIContainingTransport.java PRE-CREATION Diff: https://reviews.apache.org/r/2975/diff Testing --- Design patch, not much tested yet. Thanks, Ashutosh Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159862#comment-13159862 ] Ashutosh Chauhan commented on HIVE-2616: Some details: Currently metastore client when connecting with metastore server doesn't pass on its identity in unsecure mode. In secure mode connection is wrapped into sasl connection which does passes the user identity but only after doing kerberos based authentication. Hadoop also has similar requirements where dfsclient request namenode to perform certain operations on user's behalf. In secure mode, user identity is passed through sasl layer and in unsecure mode it is passed through connection header. Thrift metastore client-server connection however doesn't pass on any connection header at a time of a connection setup. So, mimicking what hadoop does can not yield desired result in backward compatible way. This patch takes an approach where it sends ugi information as a first rpc call from client to server straight after connection setup, which server then caches and uses for subsequent rpcs. As a result a new thrift api set_ugi() is added. This ensures backward compatibility since old client will never make this rpc, so server will continue with its previous behavior, but will perform doAs() when ugi information is indeed made available by new clients. Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2616.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159865#comment-13159865 ] jirapos...@reviews.apache.org commented on HIVE-2616: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2975/ --- Review request for hive. Summary --- Pass user identity in metastore connection in unsecure mode This addresses bug HIVE-2616. https://issues.apache.org/jira/browse/HIVE-2616 Diffs - trunk/metastore/if/hive_metastore.thrift 1205119 trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 1205119 trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 1205119 trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 1205119 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 1205119 trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php 1205119 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 1205119 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 1205119 trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 1205119 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1205119 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1207966 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIContainingTransport.java PRE-CREATION trunk/metastore/src/java/org/apache/hadoop/hive/metastore/UGIBasedProcessor.java PRE-CREATION Diff: https://reviews.apache.org/r/2975/diff Testing --- Design patch, not much tested yet. Thanks, Ashutosh Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2616.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
