steveloughran commented on pull request #2083:
URL: https://github.com/apache/hadoop/pull/2083#issuecomment-661144333
1. S3A DT will have exactly one list it builds up, the s3 one.
2. When anything asks for a new set of AWS creds via
AWSCredentialProviderList shareCredentials, with no dt: return the s3 list.
3. with DT: ask all the way through each DT plugin
3. And aggregate these for the final chain
Every DT should, by default, provide its initial set of AWS credential
providers to all authentication clients. It is only when a client knows that it
wants to provide a different (possibly empty) list of credential providers.
that there's any need to differ.
+define the "official" names of credential chains needed.Proposed: by AWS
service rather than use made in app.
troublespot`shareCredentials()` uses reference counting to decide when to go
through the provider list and close them. This matters for providers which use
close() to release resouces like threads
* we MUST call close on all providers when they are no longer needed
anywhere,
* we MUST NOT close() a provider which may be needed elsewhere
If we have different AWSProviderList lists with different sharing, we can't
use a simple reference count for the list any more.
If a new provider list is generated, then all providers in it MUST be new
*or* we do some complicated and hard to test/maintain tricks to reference count
providers in there.
If we go with "needs new providers" then we will need some two pass
collection for non-default lists, something like
1. ask all entries if they wish to provide a new provider list
1. if all say no: return old one
2. If any one says yes: ask every provider to generate a new provider with
its own lifecycle. It may just be the empty list for an unsupported service
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
