Re: Reply：DelegationTokenManager
Hi, Gabor Somogyi! Because my fork repository from apache flink repository was outdated, i did't find in time that some subtasks about KerberosDelegationTokenManager had been done. I accomplished this feature on flink-1.13.5 release used in my scenario, and created a jira FLINK-28291 with patch attached. If this situation is allowed, please assign to related ones or review this patch at your convenience. If not allowed, please drop this jira. And i am sorry to have caused this trouble. Best Regards! 在 2022年6月21日 16:20，Gabor Somogyi 写道： Thanks for pinging me! Yes, this is my main target to finish this feature however there are major code parts which are still missing. Please have a look at the umbrella jira to get better understanding: https://issues.apache.org/jira/browse/FLINK-21232 In general it's not advised to use it for production use-cases but only for testing and/or reporting bugs in test environments. When this will be finished the only thing needs to be done as a framework user is to implement a new provider and all the rest (TGT renewal, token obtain, token propagation, etc.) will be handled automatically in the right time. Related questions I've answered them inline with green, hope I've given answers to all your questions. In general we're open to make the design/implementation better so feel free to contribute any way. If you have Spark knowledge that would answer huge amount of questions because I'm trying to apply the knowledge here (and not re-inventing the wheel). Of course there are some minor differences because obviously these are 2 completely different products but the main concept is the same. BR, G On Tue, Jun 21, 2022 at 8:43 AM Márton Balassi wrote: Hi, For your information G (ccd) is actively working on this topic. [1] He is in the best position to answer your questions as far as I know. :-) [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-211%3A+Kerberos+delegation+token+framework On Tue, Jun 21, 2022 at 8:38 AM vtygoss wrote: Hi, flink community! I don't know much details for KDC. Can different TaskManagers hold different tokens? If so, driver and each worker can renew their tokens in their respective DelegationTokenManager individually. Not sure the intention of this question but as a general rule one cluster means one active DelegationTokenManager which rules all the TaskManagers. So TaskManagers are not intended to be shared between 2 DelegationTokenManagers. > Can different TaskManagers hold different tokens? Physically yes, but it's not intended to do that in the design what we've put together. There is an exotic Hadoop feature which will allow different DTs per TM but it's more like a hack than a real feature (UserGroupInformation.HADOOP_TOKEN_FILE_LOCATION). > If so, driver and each worker can renew their tokens in their respective > DelegationTokenManager individually. If you ask this I'm not sure whether you understand the whole DT concept. The main target of DT is that only one entity (this case JobManager) have Kerberos credentials, obtains tokens which then propagates that set of tokens to all TaskManager. TaskManager simply doesn't have the right to obtain any token since that would be a security hole. Thanks for your any replies. Best Regards! 在 2022年6月21日 13:30，vtygoss 写道： Hi, flink community! I am trying to do some work on renewing DelegationToken periodically for DelegtionTokenManager, and met some problems. 1. Implementations of DelegationTokenProvider There seems to be only two implementations for testing defined by SPI service: TestDelegationTokenProvider and ExceptionThrowingDelegationTokenProvider, no hdfs or hbase. That said implementation is ongoing and far not finished. Currently I'm working on the HadoopFSDelegationTokenProvider which is half-way implemented. The implementation is depending on one of my currently open PR and until that is not merged I'm fully blocked. If you would like to help I suggest to review my PRs. 2. RPCGateway When the renewer thread of DelegationTokenManager in resource manager renews tokens of all providers periodically, DelegationTokenManager should update the tokens of all taskmanagers by RPCGateway that seems to be the appropriate way for now. But the registered taskmanagers are managed by TaskExecutorManager in RM which DelegationTokenManager don't have the pointer of. So, how about using a global context to hold all necessary services, e.g. RPCService, TaskExecutorManager or HA? In my mind, RPCGateway seems to be a little clunky(just privately) for extentions, why not thinking of the async typed message model between driver and worker in design? At the moment there is no DT propagation to TaskManagers since it's not yet implemented. There will be an RPC call which will then transmit all the obtained tokens TaskManagers and all of them will have the same set of DTs. According to the plans it
Re: Reply：DelegationTokenManager
Thanks for pinging me! Yes, this is my main target to finish this feature however there are major code parts which are still missing. Please have a look at the umbrella jira to get better understanding: https://issues.apache.org/jira/browse/FLINK-21232 In general it's not advised to use it for production use-cases but *only for testing* and/or reporting bugs in test environments. When this will be finished the only thing needs to be done as a framework user is to implement a new provider and all the rest (TGT renewal, token obtain, token propagation, etc.) will be handled automatically in the right time. Related questions I've answered them inline with green, hope I've given answers to all your questions. In general we're open to make the design/implementation better so feel free to contribute any way. If you have Spark knowledge that would answer huge amount of questions because I'm trying to apply the knowledge here (and not re-inventing the wheel). Of course there are some minor differences because obviously these are 2 completely different products but the main concept is the same. BR, G On Tue, Jun 21, 2022 at 8:43 AM Márton Balassi wrote: > Hi, > > For your information G (ccd) is actively working on this topic. [1] He is > in the best position to answer your questions as far as I know. :-) > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-211%3A+Kerberos+delegation+token+framework > > On Tue, Jun 21, 2022 at 8:38 AM vtygoss wrote: > >> Hi, flink community! >> >> >> I don't know much details for KDC. Can different TaskManagers hold >> different tokens? If so, driver and each worker can renew their tokens in >> their respective DelegationTokenManager individually. >> > Not sure the intention of this question but as a general rule one cluster means one active DelegationTokenManager which rules all the TaskManagers. So TaskManagers are not intended to be shared between 2 DelegationTokenManagers. > Can different TaskManagers hold different tokens? Physically yes, but it's not intended to do that in the design what we've put together. There is an exotic Hadoop feature which will allow different DTs per TM but it's more like a hack than a real feature (UserGroupInformation.HADOOP_TOKEN_FILE_LOCATION ). > If so, driver and each worker can renew their tokens in their respective DelegationTokenManager individually. If you ask this I'm not sure whether you understand the whole DT concept. The main target of DT is that only one entity (this case JobManager) have Kerberos credentials, obtains tokens which then propagates that set of tokens to all TaskManager. TaskManager simply doesn't have the right to obtain any token since that would be a security hole. > >> Thanks for your any replies. >> >> >> Best Regards! >> >> >> >> 在 2022年6月21日 13:30，vtygoss 写道： >> >> Hi, flink community! >> >> >> I am trying to do some work on renewing DelegationToken periodically for >> DelegtionTokenManager, and met some problems. >> >> >> 1. Implementations of DelegationTokenProvider >> >> There seems to be only two implementations for testing defined by SPI >> service: TestDelegationTokenProvider and >> ExceptionThrowingDelegationTokenProvider, no hdfs or hbase. >> > That said implementation is ongoing and far not finished. Currently I'm working on the HadoopFSDelegationTokenProvider which is half-way implemented. The implementation is depending on one of my currently open PR and until that is not merged I'm fully blocked. If you would like to help I suggest to review my PRs. > >> 2. RPCGateway >> >> When the renewer thread of DelegationTokenManager in resource manager >> renews tokens of all providers periodically, DelegationTokenManager >> should update the tokens of all taskmanagers by RPCGateway that seems to be >> the appropriate way for now. >> >> But the registered taskmanagers are managed by TaskExecutorManager in RM >> which DelegationTokenManager don't have the pointer of. >> >> >> So, how about using a global context to hold all necessary services, e.g. >> RPCService, TaskExecutorManager or HA? >> >> >> In my mind, RPCGateway seems to be a little clunky(just privately) for >> extentions, why not thinking of the async typed message model between >> driver and worker in design? >> >> >> At the moment there is no DT propagation to TaskManagers since it's not yet implemented. There will be an RPC call which will then transmit all the obtained tokens TaskManagers and all of them will have the same set of DTs. According to the plans it will work in HA mode as well. Not fully sure what do you mean async message model but please have a look at Spark where this model works like charm for years. I've not re-invented any wheel but adapting our gathered knowledge here. > Thanks for any replies. >> >> >> Best Regards! >> >> >> >
Re: Reply：DelegationTokenManager
Hi, For your information G (ccd) is actively working on this topic. [1] He is in the best position to answer your questions as far as I know. :-) [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-211%3A+Kerberos+delegation+token+framework On Tue, Jun 21, 2022 at 8:38 AM vtygoss wrote: > Hi, flink community! > > > I don't know much details for KDC. Can different TaskManagers hold > different tokens? If so, driver and each worker can renew their tokens in > their respective DelegationTokenManager individually. > > > Thanks for your any replies. > > > Best Regards! > > > > 在 2022年6月21日 13:30，vtygoss 写道： > > Hi, flink community! > > > I am trying to do some work on renewing DelegationToken periodically for > DelegtionTokenManager, and met some problems. > > > 1. Implementations of DelegationTokenProvider > > There seems to be only two implementations for testing defined by SPI > service: TestDelegationTokenProvider and > ExceptionThrowingDelegationTokenProvider, no hdfs or hbase. > > > 2. RPCGateway > > When the renewer thread of DelegationTokenManager in resource manager > renews tokens of all providers periodically, DelegationTokenManager > should update the tokens of all taskmanagers by RPCGateway that seems to be > the appropriate way for now. > > But the registered taskmanagers are managed by TaskExecutorManager in RM > which DelegationTokenManager don't have the pointer of. > > > So, how about using a global context to hold all necessary services, e.g. > RPCService, TaskExecutorManager or HA? > > > In my mind, RPCGateway seems to be a little clunky(just privately) for > extentions, why not thinking of the async typed message model between > driver and worker in design? > > > Thanks for any replies. > > > Best Regards! > > >
