[
https://issues.apache.org/jira/browse/HIVE-2467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156234#comment-13156234
]
[email protected] commented on HIVE-2467:
-----------------------------------------------------
bq. On 2011-11-22 02:27:01, Ashutosh Chauhan wrote:
bq. >
trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java,
lines 40-46
bq. > <https://reviews.apache.org/r/2721/diff/2/?file=58992#file58992line40>
bq. >
bq. > If TokenStore is meant to be Configurable, then have a private
variable here to hold the conf in setConf() and return that in getConf()
Added. It was never meant to implement Configurable, but
ReflectionUtils.newInstance(...) does not support constructor injection for
conf.
bq. On 2011-11-22 02:27:01, Ashutosh Chauhan wrote:
bq. >
trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java,
line 33
bq. > <https://reviews.apache.org/r/2721/diff/2/?file=58992#file58992line33>
bq. >
bq. > Instead of java.util.concurrent.ConcurrentHashMap, use
ConcurrentHashMap instead.
done
bq. On 2011-11-22 02:27:01, Ashutosh Chauhan wrote:
bq. >
trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java,
lines 335-343
bq. > <https://reviews.apache.org/r/2721/diff/2/?file=58991#file58991line335>
bq. >
bq. > For consistency reasons, these should be moved to HiveConf. Also,
the other properties which are there in this class.
Ideally yes (tried that originally), but the shim cannot depend on HiveConf...
Perhaps in the future we can define these as part of Hadoop common security.
bq. On 2011-11-22 02:27:01, Ashutosh Chauhan wrote:
bq. >
trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java,
line 123
bq. > <https://reviews.apache.org/r/2721/diff/2/?file=58993#file58993line123>
bq. >
bq. > Either it should return boolean or it should throw Exception. Doing
both is confusing. If you want to throw exception, then throw exception on
failure and return void, else return false in failure scenario. Token must be
removed in all cases if method returns.
Exception is for errors, return boolean to indicate status of successful
operation. Clarified in java doc.
bq. On 2011-11-22 02:27:01, Ashutosh Chauhan wrote:
bq. >
trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java,
line 100
bq. > <https://reviews.apache.org/r/2721/diff/2/?file=58993#file58993line100>
bq. >
bq. > Interface method needs javadoc.
done
bq. On 2011-11-22 02:27:01, Ashutosh Chauhan wrote:
bq. >
trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java,
line 184
bq. > <https://reviews.apache.org/r/2721/diff/2/?file=58993#file58993line184>
bq. >
bq. > Fully qualified classname.
done
bq. On 2011-11-22 02:27:01, Ashutosh Chauhan wrote:
bq. >
trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java,
lines 49-50
bq. > <https://reviews.apache.org/r/2721/diff/2/?file=58994#file58994line49>
bq. >
bq. > Should these come from hiveConf, so they are externally configurable?
these are relative paths, the root is already configurable, added comment in
code
bq. On 2011-11-22 02:27:01, Ashutosh Chauhan wrote:
bq. >
trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java,
line 1
bq. > <https://reviews.apache.org/r/2721/diff/2/?file=58996#file58996line1>
bq. >
bq. > One way to test HA is to start two metastore processes in two
different threads configure to use ZK token store and then do operation on
first kill it and then do operation on second one and then have it succeeds. If
thats not straight forward, may be we can take that up in separate ticket.
That's what I'm doing as integration test (this type of thing is really e2e
(launching multiple servers), not unit testing). We should have the pieces
reasonably covered in unit tests though. The added test covers
DelegationTokenManager with token store interaction, the existing test case
does the full path through the auth bridge.
bq. On 2011-11-22 02:27:01, Ashutosh Chauhan wrote:
bq. >
trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java,
lines 137-152
bq. > <https://reviews.apache.org/r/2721/diff/2/?file=58994#file58994line137>
bq. >
bq. > Contract of configurable is to return conf in getConf(), so you must
store it in a private variable and return it.
ditto MemoryTokenStore
bq. On 2011-11-22 02:27:01, Ashutosh Chauhan wrote:
bq. >
trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java,
line 86
bq. > <https://reviews.apache.org/r/2721/diff/2/?file=58996#file58996line86>
bq. >
bq. > Get rid of commented code.
made this controllable via system property
- Thomas
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2721/#review3417
-----------------------------------------------------------
On 2011-11-23 19:55:34, Thomas wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2721/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-11-23 19:55:34)
bq.
bq.
bq. Review request for Carl Steinbach, Ashutosh Chauhan and Devaraj Das.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. https://issues.apache.org/jira/browse/HIVE-2467
bq.
bq.
bq. This addresses bug HIVE-2467.
bq. https://issues.apache.org/jira/browse/HIVE-2467
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/shims/ivy.xml 1205535
bq.
trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java
PRE-CREATION
bq.
trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java
1205535
bq.
trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java
PRE-CREATION
bq.
trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java
PRE-CREATION
bq.
trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java
PRE-CREATION
bq.
trunk/shims/src/0.20S/java/org/apache/hadoop/security/token/delegation/HiveDelegationTokenSupport.java
PRE-CREATION
bq.
trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java
1205535
bq.
bq. Diff: https://reviews.apache.org/r/2721/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. unit test added, ant clean package test - passed
bq.
bq.
bq. Thanks,
bq.
bq. Thomas
bq.
bq.
> HA Support for Metastore Server
> --------------------------------
>
> Key: HIVE-2467
> URL: https://issues.apache.org/jira/browse/HIVE-2467
> Project: Hive
> Issue Type: Improvement
> Components: Metastore, Security, Server Infrastructure
> Affects Versions: 0.8.0, 0.9.0
> Reporter: Thomas Weise
> Assignee: Thomas Weise
> Fix For: 0.9.0
>
> Attachments: HIVE-2467.2.patch, HIVE-2467.patch
>
>
> We require HA deployment for metastore server for HCatalog:
> * Multiple server instances run behind VIP
> * Database provides HA
> Metastore server instances will need to be able to share any state required
> for VIP outside RDBMS. As of Hive 0.8 affected conversational state that
> needs to support VIP/HA setup is limited to current delegation tokens. Is
> this correct?
> We are planning to use ZooKeeper to share current delegation tokens and
> master keys between nodes of the VIP. ZK is already (optionally) used by Hive
> for concurrency control. Access to ZK would be limited on the network level
> or in the future, when ZooKeeper supports security, through Kerberos, similar
> to NN access.
> Currently Hive taps into Hadoop core security delegation token support
> through extension of
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager<TokenIdent>
> A solution could amend the Hive specific extension to support:
> * Pluggable delegation token and master key store (ZooKeeper as alternative
> for in-memory AbstractDelegationTokenSecretManager)
> * Delegation token retrieval from token store when not found in memory
> (wrap/extend retrievePassword(...))
> * Cancellation of token in token store
> * Purging of expired tokens from token store
> http://www.mail-archive.com/[email protected]/msg00053.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
