Github user PramodSSImmaneni commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/332#discussion_r63108537
--- Diff: docs/security.md ---
@@ -76,6 +51,72 @@ The property `dt.authentication.principal` specifies the
Kerberos user principal
The subsequent sections talk about how security works in Apex. This
information is not needed by users but is intended for the inquisitive techical
audience who want to know how security works.
+### Token Refresh
+
+Apex applications, at runtime, use delegation tokens to authenticate with
Hadoop services when communicating with them as described in the security
architecture section below. The delegation tokens are originally issued by
these Hadoop services and have an expiry time period which is typically 7 days.
The tokens become invalid beyond this time and the applications will no longer
be able to communicate with the Hadoop services. For long running applications
this presents a problem.
+
+To solve this problem one of the two approaches can be taken. The first
approach is to change the Hadoop configuration itself to extend the token
expiry time period. This may not be possible in all environments as it requires
a change in the security policy as the tokens will now be valid for a longer
period of time and the change also requires administrator privileges to Hadoop.
The second approach is to use a feature available in apex to auto-refresh the
tokens before they expire. Both the approaches are detailed below and the users
can choose the one that works best for them.
+
+####Hadoop configuration approach
+
+An Apex application uses delegation tokens to authenticate with Hadoop
services, Resource Manager (YARN) and Name Node (HDFS), and these tokens are
issued by those services respectively. Since the application is long-running,
the tokens can expire while the application is still running. Hadoop uses
configuration settings to set the maximum lifetime of the tokens. In this
approach these setings are increased to cover the lifetime of the application.
There are separate settings for ResourceManager and NameNode delegation tokens.
+
+The ResourceManager delegation token max lifetime is specified in
`yarn-site.xml` and can be specified as follows for example for a lifetime of 1
year
--- End diff --
I will add 1. Regd 2. are you thinking if there is a mismatch between
actual expiry time set in Hadoop vs the one specified in dt-site.xml or are you
thinking that even with 0.7 there is a chance it may not happen because of some
other errors.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
