Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/9814 )
Change subject: KUDU-2379. java: exportAuthenticationCredentials must connect to the cluster ...................................................................... Patch Set 1: > Patch Set 1: > > (1 comment) > > > Patch Set 1: > > > > One thought I had about this: do we need to also do something to ensure > > that, if the Spark driver has connected to the cluster, but the token is > > expired, that it proactively re-connects to get a new token to propagate to > > the executors? > > Prior to KUDU-2259 landing it doesn't appear that we explicitly handled this, > unless I'm missing something. > > In addition, I believe exportAuthenticationCredentials is only called once in > the life of a Kudu/Spark job, when the KuduContext is being constructed (it's > cached as a serializable val). Yea, I don't think the "reacquire" is necessarily a regression, just something I was thinking about in the context of this fix. As for whether export is called multiple times, I think you're right. I think at one point we were discussing hooking serialization so that each time a new task was started and the KuduContext was serialized to that task it would ensure it got a new token. Does this mean we don't support spark jobs which run longer than the token expiry interval? Or are we assuming that once every 7 days, the driver will crash and restart itself and pick up a new token? Meanwhile since I think we agree that the above isn't a regression since 1.6 we can move ahead with this patch, right? -- To view, visit http://gerrit.cloudera.org:8080/9814 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icd1b69abadb4b9c97f6c8f7ed38897c33ca7ff72 Gerrit-Change-Number: 9814 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Dan Burkert <[email protected]> Gerrit-Reviewer: Grant Henke <[email protected]> Gerrit-Reviewer: Hao Hao <[email protected]> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon <[email protected]> Gerrit-Comment-Date: Tue, 27 Mar 2018 18:31:53 +0000 Gerrit-HasComments: No
