[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/203 > That is caused by UserInformationGroup loginUserFromKeytab being called multiple time from different threads if using a multi threaded environment. this fix ensures that there will only be one login per process. `UGI.loginUserFromKeytab` never spawns a renewal thread so as it is. I don't think this change has the effect you intend it to have. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...
Github user dbahir commented on the issue: https://github.com/apache/phoenix/pull/203 This fix has been tested and it solves the issue, the same fix has been applied to the storm hdfs and hbase connectors. https://issues.apache.org/jira/browse/STORM-1521 https://issues.apache.org/jira/browse/STORM-1535 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/203 > This fix has been tested and it solves the issue, the same fix has been applied to the storm hdfs and hbase connectors. But I still don't understand what you're trying to fix. https://github.com/apache/hadoop/blob/94225152399e6e89fa7b4cff6d17d33e544329a3/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java#L957-L958 `UserGroupInformation` does *not* spawn any renewal thread for ticket renewal. Can you clarify what doesn't work? Given your description on JIRA, it doesn't make sense to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...
Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/203 Ignoring the aforementioned issue, I don't think this change is correctly handling multiple users. It would be re-introducing the bug that was talked about in PHOENIX-3126. If there was a user that was already logged in and then a different URL was provided with different credentials, the old user's credentials would be used instead of the new user's credentials. This would be a security vulnerability. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...
Github user dbahir commented on the issue: https://github.com/apache/phoenix/pull/203 Regarding the renewal, I understand from, http://stackoverflow.com/questions/34616676/should-i-call-ugi-checktgtandreloginfromkeytab-before-every-action-on-hadoop, that the RPC layer takes care of that. I am trying to fix the scenario in which multiple threads call loginUserFromKeytab concurrently and then the renewal process no longer works as expected. An example of that scenario is a storm topology that has multiple HBase/Phoenix/HDFS bolts in the same JVM. When the topology starts it will initialize all bolts which will execute a login from each one, when that happens the renewal no longer works. If only one login happens the renewal works properly. In regarding to Phoenix, we came got into a similar situation with a multi-threaded application that caused loginUserFromKeytab to be called concurrently. The code change was made to protect that and works. Your concern regarding security is correct. I looked into PHOENIX-3189 which i was not aware of. The fix can be folded into it however we would need to handle synchronization of the loginUserFromKeytab if multple instances of the driver are created. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...
Github user dbahir commented on the issue: https://github.com/apache/phoenix/pull/203 If you look at https://github.com/hanborq/hadoop/blob/master/src/core/org/apache/hadoop/security/UserGroupInformation.java you can see that this class is not thread safe and not designed to have different users login in the same JVM as loginUser is defined in this way. private static UserGroupInformation loginUser = null; --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...
Github user joshelser commented on the issue:
https://github.com/apache/phoenix/pull/203
> Regarding the renewal, I understand from,
http://stackoverflow.com/questions/34616676/should-i-call-ugi-checktgtandreloginfromkeytab-before-every-action-on-hadoop,
that the RPC layer takes care of that.
Well, if you're talking to HDFS directly it would take care of it :). But
we're talking about accessing HBase here. I'm not sure if the same holds true.
I know there is something similar in the HBase RPC level, but I'd have to find
it again in code to double check.
> I am trying to fix the scenario in which multiple threads call
loginUserFromKeytab concurrently and then the renewal process no longer works
as expected.
> If only one login happens the renewal works properly.
Is this the same principal over and over again? Are you essentially
providing the same principal and keytab in the JDBC URL, expecting Phoenix to
do everything for you instead of doing the login in Storm?
> Your concern regarding security is correct.
Ok. I would like to redirect your efforts to PHOENIX-3189 then. We cannot
sacrifice security for multi-threading (as you can already handle the Kerberos
login yourself). Can you take a look at the changes I have staged on #191? If
this is the above case I outlined, we can add some concurrency control to
prevent concurrent logins from happening.
> you can see that this class is not thread safe and not designed to have
different users login in the same JVM as loginUser is defined in this way.
Phoenix itself is not well-designed to support concurrent (different) users
accessing HBase because of how UGI works. If your application (Storm) needs to
provide this functionality, Storm should perform logins itself, cache the UGI
instances, and use {{UGI.doAs(..)}} instead of relying on the static state in
UGI.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...
Github user dbahir commented on the issue: https://github.com/apache/phoenix/pull/203 HBase renew implementation is similar to the HDFS one. https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java#L658 Thanks for your comments, will look at your changes and see where these changes can fit in. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...
Github user dbahir commented on the issue: https://github.com/apache/phoenix/pull/203 Closing, this issue will be fixed by [#191](https://github.com/apache/phoenix/pull/191) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
