[
https://issues.apache.org/jira/browse/MAPREDUCE-1430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12835099#action_12835099
]
Boris Shkolnik commented on MAPREDUCE-1430:
-------------------------------------------
bq.1) For (3), it should be that, cancellation is done for tokens owned by jobs
that have mapreduce.job.cancel.delegation.tokens set to true, when the job is
done. The idea is to immediately cancel the tokens once the job is done unless
the user sets mapreduce.job.cancel.delegation.tokens to false (in which case it
probably means that the user launched jobs from tasks of the job in question,
and these new jobs would be using the same tokens for talking to the hdfs). For
cancelling the token you should invoke the appropriate RPC. Thinking about it,
the config could be named to something like
"mapreduce.job.complete.cancel.delegation.tokens"
Added cancellation for completed jobs (unless
"mapreduce.job.complete.cancel.delegation.tokens" is set to false (default is
true)
bq. 2) Please make the DelegationTokenRenewal class package private.
It is used from JobTracker so it must remain public. Added
@InterfaceAudience.Private.
bq. 3) renewalTimer should be a daemon thread.
Done.
bq. 4) Methods like getDfsForToken could be declared as private.
Done.
bq. 5) The logic for setting the renewal to 60 minutes on an exception should
be within the renewDelegationToken method (newExpirationDate could be
initialized to that).
Done.
bq. 6) You don't need synchronization on delegationTokens in the
setTimerForTokenRenewal method.
Done.
bq. 7) The delegationTokens list could be a HashSet instead
Actually, if we want direct access it should be a MAP keyed by jobId. And that
will point to a list. So I changed it to Map<String, List<DTToRenew>>
I've also
8) modified Test to verify cancellation of the token.
9) moved registration for renewal to job initialization and cancelation to
garbageCollect()
10) specified default for ""mapreduce.job.complete.cancel.delegation.tokens" to
true in mapred-default.xml
11) change all Boolean renew/cancelToken to long
and more..
> JobTracker should be able to renew delegation tokens for the jobs
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1430
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1430
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Devaraj Das
> Assignee: Boris Shkolnik
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-1430-5.patch, MAPREDUCE-1430-6.patch,
> MAPREDUCE-1430-8.patch
>
>
> JobTracker should automatically renew delegation tokens for the jobs it is
> currently running.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
