Sorry I didn't want to offend anybody if it was perceived like this. I can
see that me joining very late into the discussion w/o constructive ideas
was not nice. My motivation for asking for the reasoning behind the current
design proposal is primarily the lack of Kerberos knowledge. Moreover, it
happened before that we moved responsibilities into Flink that we regretted
later.
As I've said, I don't have a better idea right now. If we believe that it
is the right thing to make Flink responsible for distributing the tokens
and we don't find a better solution then we'll go for it. I just wanted to
make sure that we don't overlook an alternative solution that might be
easier to maintain in the long run.
Cheers,
Till
On Thu, Feb 3, 2022 at 7:52 PM Gyula Fóra <gyula.f...@gmail.com> wrote:
Hi Team!
Let's all calm down a little and not let our emotions affect the
discussion
too much.
There has been a lot of effort spent from all involved parties so this is
quite understandable :)
Even though not everyone said this explicitly, it seems that everyone
more
or less agrees that a feature implementing token renewal is necessary and
valuable.
The main point of contention is: where should the token renewal
logic run and how to get the tokens to wherever needed.
From my perspective the current design is very reasonable at first sight
because:
1. It runs the token renewal in a single place avoiding extra CDC
workload
2. Does not introduce new processes, extra communication channels etc
but
piggybacks on existing robust mechanisms.
I understand the concerns about adding new things in the resource manager
but I think that really depends on how we look at it.
We cannot reasonably expect a custom token renewal process to have it's
own
secure distribution logic like Flink has now, that is a complete
overkill.
This practically means that we will not have a slim efficient
implementation for this but something unnecessarily complex. And the only
thing we get in return is a bit less code in the resource manager.
From a logical standpoint the delegation framework needs to run at a
centralized place and need to be able to access new task manager
processes
to achieve all it's design goals.
We can drop a single renewer as a design goal but that might be a
decision
that can affect large scale production runs.
Cheers,
Gyula
On Thu, Feb 3, 2022 at 7:32 PM Chesnay Schepler <ches...@apache.org>
wrote:
First of, at no point have we questioned the use-case and importance of
this feature, and the fact that David, Till and me spent time looking
at
the FLIP, asking questions, and discussing different aspects of it
should make this obvious.
I'd appreciate it if you didn't dismiss our replies that quickly.
> Ok, so we declare that users who try to use delegation tokens in
Flink is dead end code and not supported, right?
No one has said that. Are you claiming that your design is the /only
possible implementation/ that is capable of achieving the stated goals,
that there are 0 alternatives? On of the *main**points* of these
discussion threads is to discover alternative implementations that
maybe
weren't thought of. Yes, that may imply that we amend your design, or
reject it completely and come up with a new one.
Let's clarify what (I think) Till proposed to get the imagination juice
flowing.
At the end of the day, all we need is a way to provide Flink processes
with a token that can be periodically updated. _Who_ issues that token
is irrelevant for the functionality to work. You are proposing for a
new
component in the Flink RM to do that; Till is proposing to have some
external process do it. *That's it*.
How this could look like in practice is fairly straight forwad; add a
pluggable interface (aka, your TokenProvider thing) that is loaded in
each process, which can _somehow_ provide tokens that are then set in
the UserGroupInformation.
_How_ the provider receives token is up to the provider. It _may_ just
talk directly to Kerberos, or it could use some communication channel
to
accept tokens from the outside.
This would for example make it a lot easier to properly integrate this
into the lifecycle of the process, as we'd sidestep the whole "TM is
running but still needs a Token" issue; it could become a proper setup
step of the process that is independent from other Flink processes.
/Discuss/.
On 03/02/2022 18:57, Gabor Somogyi wrote:
And even
if we do it like this, there is no guarantee that it works because
there
can be other applications bombing the KDC with requests.
1. The main issue to solve here is that workloads using delegation
tokens
are stopping after 7 days with default configuration.
2. This is not new design, it's rock stable and performing well in
Spark
for years.
From a
maintainability and separation of concerns perspective I'd rather
have
this
as some kind of external tool/service that makes KDC scale better and
that
Flink processes can talk to to obtain the tokens.
Ok, so we declare that users who try to use delegation tokens in
Flink
is
dead end code and not supported, right? Then this must be explicitely
written in the security documentation that such users who use that
feature
are left behind.
As I see the discussion turned away from facts and started to speak
about
feelings. If you have strategic problems with the feature please put
your
-1 on the vote and we can spare quite some time.
G
On Thu, 3 Feb 2022, 18:34 Till Rohrmann,<trohrm...@apache.org>
wrote:
I don't have a good alternative solution but it sounds to me a bit
as
if we
are trying to solve Kerberos' scalability problems within Flink. And
even
if we do it like this, there is no guarantee that it works because
there
can be other applications bombing the KDC with requests. From a
maintainability and separation of concerns perspective I'd rather
have
this
as some kind of external tool/service that makes KDC scale better
and
that
Flink processes can talk to to obtain the tokens.
Cheers,
Till
On Thu, Feb 3, 2022 at 6:01 PM Gabor Somogyi<
gabor.g.somo...@gmail.com>
wrote:
Oh and the most important reason I've forgotten.
Without the feature in the FLIP all secure workloads with
delegation
tokens
are going to stop when tokens are reaching it's max lifetime 🙂
This is around 7 days with default config...
On Thu, Feb 3, 2022 at 5:30 PM Gabor Somogyi<
gabor.g.somo...@gmail.com
wrote:
That's not the single purpose of the feature but in some
environments
it
caused problems.
The main intention is not to deploy keytab to all the nodes
because
the
attack surface is bigger + reduce the KDC load.
I've already described the situation previously in this thread so
copying
it here.
--------COPY--------
"KDC *may* collapse under some circumstances" is the proper
wording.
We have several customers who are executing workloads on
Spark/Flink.
Most
of the time I'm facing their
daily issues which is heavily environment and use-case dependent.
I've
seen various cases:
* where the mentioned ~1k nodes were working fine
* where KDC thought the number of requests are coming from DDOS
attack
so
discontinued authentication
* where KDC was simply not responding because of the load
* where KDC was intermittently had some outage (this was the most
nasty
thing)
Since you're managing relatively big cluster then you know that
KDC
is
not
only used by Spark/Flink workloads
but the whole company IT infrastructure is bombing it so it really
depends
on other factors too whether KDC is reaching
it's limit or not. Not sure what kind of evidence are you looking
for
but
I'm not authorized to share any information about
our clients data.
One thing is for sure. The more external system types are used in
workloads (for ex. HDFS, HBase, Hive, Kafka) which
are authenticating through KDC the more possibility to reach this
threshold when the cluster is big enough.
--------COPY--------
The FLIP mentions scaling issues with 200 nodes; it's really
surprising
to me that such a small number of requests can already cause
issues.
One node/task doesn't mean 1 request. The following type of
kerberos
auth
types has been seen by me which can run at the same time:
HDFS, Hbase, Hive, Kafka, all DBs (oracle, mariaDB, etc...)
Additionally
one task is not necessarily opens 1 connection.
All in all I don't have steps to reproduce but we've faced this
already...
G
On Thu, Feb 3, 2022 at 5:15 PM Chesnay Schepler<
ches...@apache.org>
wrote:
What I don't understand is how this could overload the KDC.
Aren't
tokens valid for a relatively long time period?
For new deployments where many TMs are started at once I could
imagine
it temporarily, but shouldn't the accesses to the KDC eventually
naturally spread out?
The FLIP mentions scaling issues with 200 nodes; it's really
surprising
to me that such a small number of requests can already cause
issues.
On 03/02/2022 16:14, Gabor Somogyi wrote:
I would prefer not choosing the first option
Then the second option may play only.
I am not a Kerberos expert but is it really so that every
application
that
wants to use Kerberos needs to implement the token propagation
itself?
This
somehow feels as if there is something missing.
OK, so first some kerberos + token intro.
Some basics:
* TGT can be created from keytab
* TGT is needed to obtain TGS (called token)
* Authentication only works with TGS -> all places where
external
system is
needed either a TGT or TGS needed
There are basically 2 ways to authenticate to a kerberos secured
external
system:
1. One needs a kerberos TGT which MUST be propagated to all
JVMs.
Here
each
and every JVM obtains a TGS by itself which bombs the KDC that
may
collapse.
2. One needs a kerberos TGT which exists only on a single place
(in
this
case JM). JM gets a TGS which MUST be propagated to all TMs
because
otherwise authentication fails.
Now the whole system works in a way that keytab file (we can
imagine
that
as plaintext password) is reachable on all nodes.
This is a relatively huge attack surface. Now the main intention
is:
* Instead of propagating keytab file to all nodes propagate a
TGS
which
has
limited lifetime (more secure)
* Do the TGS generation in a single place so KDC may not
collapse
+
having
keytab only on a single node can be better protected
As a final conclusion if there is a place which expects to do
kerberos
authentication then it's a MUST to have either TGT or TGS.
Now it's done in a pretty unsecure way. The questions are the
following:
* Do we want to leave this unsecure keytab propagation like this
and
bomb
KDC?
* If no then how do we propagate the more secure token to TMs.
If the answer to the first question is no then the FLIP can be
abandoned
and doesn't worth the further effort.
If the answer is yes then we can talk about the how part.
G
On Thu, Feb 3, 2022 at 3:42 PM Till Rohrmann<
trohrm...@apache.org
wrote:
I would prefer not choosing the first option
Make the TM accept tasks only after registration(not sure if
it's
possible or makes sense at all)
because it effectively means that we change how Flink's
component
lifecycle
works for distributing Kerberos tokens. It also effectively
means
that
a TM
cannot make progress until connected to a RM.
I am not a Kerberos expert but is it really so that every
application
that
wants to use Kerberos needs to implement the token propagation
itself?
This
somehow feels as if there is something missing.
Cheers,
Till
On Thu, Feb 3, 2022 at 3:29 PM Gabor Somogyi <
gabor.g.somo...@gmail.com>
wrote:
Isn't this something the underlying resource management
system
could
do
or which every process could do on its own?
I was looking for such feature but not found.
Maybe we can solve the propagation easier but then I'm waiting
on
better
suggestion.
If anybody has better/more simple idea then please point to a
specific
feature which works on all resource management systems.
Here's an example for the TM to run workloads without being
connected
to the RM, without ever having a valid token
All in all I see the main problem. Not sure what is the reason
behind
that
a TM accepts tasks w/o registration but clearly not helping
here.
I basically see 2 possible solutions:
* Make the TM accept tasks only after registration(not sure if
it's
possible or makes sense at all)
* We send tokens right after container creation with
"updateDelegationTokens"
Not sure which one is more realistic to do since I'm not
involved
the
new
feature.
WDYT?
On Thu, Feb 3, 2022 at 3:09 PM Till Rohrmann <
trohrm...@apache.org>
wrote:
Hi everyone,
Sorry for joining this discussion late. I also did not read
all
responses
in this thread so my question might already be answered: Why
does
Flink
need to be involved in the propagation of the tokens? Why do
we
need
explicit RPC calls in the Flink domain? Isn't this something
the
underlying
resource management system could do or which every process
could
do
on
its
own? I am a bit worried that we are making Flink responsible
for
something
that it is not really designed to do so.
Cheers,
Till
On Thu, Feb 3, 2022 at 2:54 PM Chesnay Schepler <
ches...@apache.org>
wrote:
Here's an example for the TM to run workloads without being
connected
to
the RM, while potentially having a valid token:
1. TM registers at RM
2. JobMaster requests slot from RM -> TM gets notified
3. JM fails over
4. TM re-offers the slot to the failed over JobMaster
5. TM reconnects to RM at some point
Here's an example for the TM to run workloads without being
connected
to
the RM, without ever having a valid token:
1. TM1 has a valid token and is running some tasks.
2. TM1 crashes
3. TM2 is started to take over, and re-uses the working
directory
of
TM1 (new feature in 1.15!)
4. TM2 recovers the previous slot allocations
5. TM2 is informed about leading JM
6. TM2 starts registration with RM
7. TM2 offers slots to JobMaster
8. TM2 accepts task submission from JobMaster
9. ...some time later the registration completes...
On 03/02/2022 14:24, Gabor Somogyi wrote:
but it can happen that the JobMaster+TM collaborate to run
stuff
without the TM being registered at the RM
Honestly I'm not educated enough within Flink to give an
example
to
such scenario.
Until now I thought JM defines tasks to be done and TM just
blindly
connects to external systems and does the processing.
All in all if external systems can be touched when JM + TM
collaboration happens then we need to consider that in the
design.
Since I don't have an example scenario I don't know what
exactly
needs
to be solved.
I think we need an example case to decide whether we face a
real
issue
or the design is not leaking.
On Thu, Feb 3, 2022 at 2:12 PM Chesnay Schepler <
ches...@apache.org>
wrote:
> Just to learn something new. I think local recovery
is
clear to
me which is not touching external systems like Kafka
or
so
(correct me if I'm wrong). Is it possible that such
case
the
user
code just starts to run blindly w/o JM coordination
and
connects
to external systems to do data processing?
Local recovery itself shouldn't touch external
systems;
the
TM
cannot just run user-code without the JobMaster being
involved,
but it can happen that the JobMaster+TM collaborate
to
run
stuff
without the TM being registered at the RM.
On 03/02/2022 13:48, Gabor Somogyi wrote:
> Any error in loading the provider (be it by
accident
or
explicit checks) then is a setup error and we can
fail
the
cluster.
Fail fast is a good direction in my view. In Spark I
wanted
to
go
to this direction but there were other opinions so
there
if a
provider is not loaded then the workload goes
further.
Of course the processing will fail if the token is
missing...
> Requiring HBase (and Hadoop for that matter) to be
on
the
JM
system classpath would be a bit unfortunate. Have
you
considered
loading the providers as plugins?
Even if it's unfortunate the actual implementation
is
depending
on that already. Moving HBase and/or all token
providers
into
plugins is a possibility.
That way if one wants to use a specific provider
then
a
plugin
need to be added. If we would like to go to this
direction
I
would do that in a separate
FLIP not to have feature creep here. The actual FLIP
already
covers several thousand lines of code changes.
> This is missing from the FLIP. From my experience
with
the
metric reporters, having the implementation rely on
the
configuration is really annoying for testing
purposes.
That's
why
I suggested factories; they can take care of
extracting
all
parameters that the implementation needs, and then
pass
them
nicely via the constructor.
ServiceLoader provided services must have a norarg
constructor
where no parameters can be passed.
As a side note testing delegation token providers is
pain
in
the
ass and not possible with automated tests without
creating
a
fully featured kerberos cluster with KDC, HDFS,
HBase,
Kafka,
etc..
We've had several tries in Spark but then gave it up
because
of
the complexity and the flakyness of it so I wouldn't
care
much
about unit testing.
The sad truth is that most of the token providers
can
be
tested
manually on cluster.
Of course this doesn't mean that the whole code is
not
intended
to be covered with tests. I mean couple of parts can
be
automatically tested but providers are not such.
> This also implies that any fields of the provider
wouldn't
inherently have to be mutable.
I think this is not an issue. A provider connects
to a
service,
obtains token(s) and then close the connection and
never
seen
the
need of an intermediate state.
I've just mentioned the singleton behavior to be
clear.
> One examples is a TM restart + local recovery,
where
the
TM
eagerly offers the previous set of slots to the
leading
JM.
Just to learn something new. I think local recovery
is
clear
to
me which is not touching external systems like Kafka
or
so
(correct me if I'm wrong).
Is it possible that such case the user code just
starts
to
run
blindly w/o JM coordination and connects to external
systems
to
do data processing?
On Thu, Feb 3, 2022 at 1:09 PM Chesnay Schepler
<ches...@apache.org> wrote:
1)
The manager certainly shouldn't check for
specific
implementations.
The problem with classpath-based checks is it
can
easily
happen that the provider can't be loaded in the
first
place
(e.g., if you don't use reflection, which you
currently
kinda
force), and in that case Flink can't tell
whether
the
token
is not required or the cluster isn't set up
correctly.
As I see it we shouldn't try to be clever; if
the
users
wants
kerberos, then have him enable the providers.
Any
error
in
loading the provider (be it by accident or
explicit
checks)
then is a setup error and we can fail the
cluster.
If we still want to auto-detect whether the
provider
should
be used, note that using factories would make
this
easier;
the factory can check the classpath (not having
any
direct
dependencies on HBase avoids the case above),
and
the
provider no longer needs reflection because it
will
only
be
used iff HBase is on the CP.
Requiring HBase (and Hadoop for that matter) to
be
on
the JM
system classpath would be a bit unfortunate.
Have
you
considered loading the providers as plugins?
2) > DelegationTokenProvider#init method
This is missing from the FLIP. From my
experience
with
the
metric reporters, having the implementation rely
on
the
configuration is really annoying for testing
purposes.
That's
why I suggested factories; they can take care of
extracting
all parameters that the implementation needs,
and
then
pass
them nicely via the constructor. This also
implies
that
any
fields of the provider wouldn't inherently have
to
be
mutable.
> workloads are not yet running until the
initial
token
set
is not propagated.
This isn't necessarily true. It can happen that
tasks
are
being deployed to the TM without it having
registered
with
the RM; there is currently no requirement that a
TM
must
be
registered before it may offer slots / accept
task
submissions.
One examples is a TM restart + local recovery,
where
the
TM
eagerly offers the previous set of slots to the
leading
JM.
On 03/02/2022 12:39, Gabor Somogyi wrote:
Thanks for the quick response!
Appreciate your invested time...
G
On Thu, Feb 3, 2022 at 11:12 AM Chesnay
Schepler
<ches...@apache.org> wrote:
Thanks for answering the questions!
1) Does the HBase provider require HBase to
be
on
the
classpath?
To be instantiated no, to obtain a token yes.
If so, then could it even be loaded if
Hbase
is
on
the classpath?
The provider can be loaded but inside the
provider
it
would
detect whether HBase is on classpath.
Just to be crystal clear here this is the
actual
implementation what I would like to take over
into
the
Provider.
Please see:
https://github.com/apache/flink/blob/e6210d40491ff28c779b8604e425f01983f8a3d7/flink-yarn/src/main/java/org/apache/flink/yarn/Utils.java#L243-L254
I've considered to load only the necessary
Providers
but
that would mean a generic Manager need to know
that
if
the
newly loaded Provider is
instanceof HBaseDelegationTokenProvider, then
it
need
to be
skipped.
I think it would add unnecessary complexity to
the
Manager
and it would contain ugly code parts(at least
in
my
view
ugly), like this
if (provider instanceof
HBaseDelegationTokenProvider
&&
hbaseIsNotOnClasspath()) {
// Skip intentionally
} else if (provider instanceof
SomethingElseDelegationTokenProvider &&
somethingElseIsNotOnClasspath()) {
// Skip intentionally
} else {
providers.put(provider.serviceName(),
provider);
}
I think the least code and most clear approach
is
to
load
the providers and decide inside whether
everything
is
given
to obtain a token.
If not, then you're assuming the
classpath
of
the
JM/TM to be the same, which isn't
necessarily
true
(in
general; and also if Hbase is loaded from
the
user-jar).
I'm not assuming that the classpath of JM/TM
must
be
the
same. If the HBase jar is coming from the
user-jar
then
the
HBase code is going to use UGI within the JVM
when
authentication required.
Of course I've not yet tested within Flink but
in
Spark
it
is working fine.
All in all JM/TM classpath may be different but
on
both
side
HBase jar must exists somehow.
2) None of the /Providers/ in your PoC get
access
to
the
configuration. Only the /Manager/ is. Note
that
I
do
not
know whether there is a need for the
providers
to
have
access to the config, as that's very
implementation
specific I suppose.
You're right. Since this is just a POC and I
don't
have
green light I've not put too many effort for a
proper
self-review. DelegationTokenProvider#init
method
must
get
Flink configuration.
The reason behind is that several further
configuration
can
be find out using that. A good example is to
get
Hadoop
conf.
The rationale behind is the same just like
before,
it
would
be good to create a generic Manager as
possible.
To be more specific some code must load Hadoop
conf
which
could be the Manager or the Provider.
If the manager does that then the generic
Manager
must
be
modified all the time when something special
thing
is
needed
for a new provider.
This could be super problematic when a custom
provider
is
written.
10) I'm not sure myself. It could be
something
as
trivial as creating some temporary
directory
in
HDFS I
suppose.
I've not found of such task.YARN and K8S are
not
expecting
such things from executors and workloads are
not
yet
running
until the initial token set is not propagated.
On 03/02/2022 10:23, Gabor Somogyi wrote:
Please see my answers inline. Hope
provided
satisfying
answers to all
questions.
G
On Thu, Feb 3, 2022 at 9:17 AM Chesnay
Schepler<
ches...@apache.org> <mailto:ches...@apache.org> wrote:
I have a few question that I'd appreciate
if
you
could
answer them.
1. How does the Provider know whether
it
is
required or not?
All registered providers which are
registered
properly
are going to be
loaded and asked to obtain tokens. Worth
to
mention
every provider
has the right to decide whether it wants
to
obtain
tokens or not (bool
delegationTokensRequired()). For instance
if
provider
detects that
HBase is not on classpath or not
configured
properly
then no tokens are
obtained from that specific provider.
You may ask how a provider is registered.
Here
it
is:
The provider is on classpath + there is a
META-INF
file
which contains the
name of the provider, for example:
META-INF/services/org.apache.flink.runtime.security.token.DelegationTokenProvider
<
https://github.com/apache/flink/compare/master...gaborgsomogyi:dt?expand=1#diff-b65ee7e64c5d2dfbb683d3569fc3e42f4b5a8052ab83d7ac21de5ab72f428e0b
<
https://github.com/apache/flink/compare/master...gaborgsomogyi:dt?expand=1#diff-b65ee7e64c5d2dfbb683d3569fc3e42f4b5a8052ab83d7ac21de5ab72f428e0b
1. How does the configuration of
Providers
work
(how do they get
access to a configuration)?
Flink configuration is going to be passed
to
all
providers. Please see the
POC here:
https://github.com/apache/flink/compare/master...gaborgsomogyi:dt?expand=1
Service specific configurations are loaded
on-the-fly.
For example in HBase
case it looks for HBase configuration
class
which
will
be instantiated
within the provider.
1. How does a user select providers?
(Is
it
purely
based on the
provider being on the classpath?)
Providers can be explicitly turned off
with
the
following config:
"security.kerberos.tokens.${name}.enabled".
I've
never
seen that 2
different implementation would exist for a
specific
external service, but if this edge case
would
exist
then the mentioned
config need to be added, a new provider
with a
different name need to be
implemented and registered.
All in all we've seen that provider
handling
is
not
user specific task but
a cluster admin one. If a specific
provider
is
needed
then it's implemented
once per company, registered once
to the clusters and then all users may or
may
not
use
the obtained tokens.
Worth to mention the system will know
which
token
need
to be used when HDFS
is accessed, this part is automatic.
1. How can a user override an
existing
provider?
Pease see the previous bulletpoint.
1. What is
DelegationTokenProvider#name()
used
for?
By default all providers which are
registered
properly
(on classpath +
META-INF entry) are on by default. With
"security.kerberos.tokens.${name}.enabled" a
specific
provider can be
turned off.
Additionally I'm intended to use this in
log
entries
later on for debugging
purposes. For example "hadoopfs provider
obtained 2
tokens with ID...".
This would help what and when is happening
with tokens. The same applies to
TaskManager
side:
"2
hadoopfs provider
tokens arrived with ID...". Important to
note
that
the
secret part will be
hidden in the mentioned log entries to
keep
the
attach surface low.
1. What happens if the names of 2
providers
are
identical?
Presume you mean 2 different classes
which
both
registered and having the
same logic inside. This case both will be
loaded
and
both is going to
obtain token(s) for the same service.
Both obtained token(s) are going to be
added
to
the
UGI. As a result the
second will overwrite the first but the
order
is
not
defined. Since both
token(s) are valid no matter which one is
used then access to the external system
will
work.
When the class names are same then service
loader
only
loads a single entry
because services are singletons. That's
the
reason
why
state inside
providers are not advised.
1. Will we directly load the
provider,
or
first
load a factory
(usually preferable)?
Intended to load a provider directly by
DTM.
We
can
add an extra layer to
have factory but after consideration I
came
to
a
conclusion that it would
be and overkill this case.
Please have a look how it's planned to
load
providers
now:
https://github.com/apache/flink/compare/master...gaborgsomogyi:dt?expand=1#diff-d56a0bc77335ff23c0318f6dec1872e7b19b1a9ef6d10fff8fbaab9aecac94faR54-R81
1. What is the Credentials class (it
would
necessarily have to be a
public api as well)?
Credentials class is coming from Hadoop.
My
main
intention was not to bind
the implementation to Hadoop completely.
It
is
not
possible because of the
following reasons:
* Several functionalities are must because
there
are
no
alternatives,
including but not limited to login from
keytab,
proper
TGT cache handling,
passing tokens to Hadoop services like
HDFS,
HBase,
Hive, etc.
* The partial win is that the whole
delegation
token
framework is going to
be initiated if hadoop-common is on
classpath
(Hadoop
is optional in core
libraries)
The possibility to eliminate Credentials
from
API
could
be:
* to convert Credentials to byte array
forth
and
back
while a provider
gives back token(s): I think this would be
an
overkill
and would make the
API less clear what to give back what
Manager
understands
* to re-implement Credentials internal
structure
in a
POJO, here the same
convert forth and back would happen
between
provider
and manager. I think
this case would be the re-invent the wheel
scenario
1. What does the TaskManager do with
the
received
token?
Puts the tokens into the
UserGroupInformation
instance
for the current
user. Such way Hadoop compatible services
can
pick
up
the tokens from there
properly.
This is an existing pattern inside Spark.
1. Is there any functionality in the
TaskManager
that could require a
token on startup (i.e., before
registering
with
the RM)?
Never seen such functionality in Spark
and
after
analysis not seen in
Flink too. If you have something in mind
which
I've
missed plz help me out.
On 11/01/2022 14:58, Gabor Somogyi wrote:
Hi All,
Hope all of you have enjoyed the holiday
season.
I would like to start the discussion on
FLIP-211<
https://cwiki.apache.org/confluence/display/FLINK/FLIP-211%3A+Kerberos+delegation+token+framework
<
https://cwiki.apache.org/confluence/display/FLINK/FLIP-211%3A+Kerberos+delegation+token+framework
<
https://cwiki.apache.org/confluence/display/FLINK/FLIP-211%3A+Kerberos+delegation+token+framework
<
https://cwiki.apache.org/confluence/display/FLINK/FLIP-211%3A+Kerberos+delegation+token+framework
which
aims to provide a
Kerberos delegation token framework that
/obtains/renews/distributes tokens
out-of-the-box.
Please be aware that the FLIP wiki area
is
not
fully
done since the
discussion may
change the feature in major ways. The
proposal
can be
found in a google doc
here<
https://docs.google.com/document/d/1JzMbQ1pCJsLVz8yHrCxroYMRP2GwGwvacLrGyaIx5Yc/edit?fbclid=IwAR0vfeJvAbEUSzHQAAJfnWTaX46L6o7LyXhMfBUCcPrNi-uXNgoOaI8PMDQ
<
https://docs.google.com/document/d/1JzMbQ1pCJsLVz8yHrCxroYMRP2GwGwvacLrGyaIx5Yc/edit?fbclid=IwAR0vfeJvAbEUSzHQAAJfnWTaX46L6o7LyXhMfBUCcPrNi-uXNgoOaI8PMDQ
<
https://docs.google.com/document/d/1JzMbQ1pCJsLVz8yHrCxroYMRP2GwGwvacLrGyaIx5Yc/edit?fbclid=IwAR0vfeJvAbEUSzHQAAJfnWTaX46L6o7LyXhMfBUCcPrNi-uXNgoOaI8PMDQ
<
https://docs.google.com/document/d/1JzMbQ1pCJsLVz8yHrCxroYMRP2GwGwvacLrGyaIx5Yc/edit?fbclid=IwAR0vfeJvAbEUSzHQAAJfnWTaX46L6o7LyXhMfBUCcPrNi-uXNgoOaI8PMDQ
.
As the community agrees on the approach
the
content
will be moved to the
wiki page.
Feel free to add your thoughts to make
this
feature
better!
BR,
G
