fapifta commented on PR #10742: URL: https://github.com/apache/ozone/pull/10742#issuecomment-5174131657
@fmorg-git @spacemonkd let me pull up the STS vs Custos discussion to the PR level, as that seems to be a key disagreement about the whole concept not just parts of it, also please let me chime in with a few thougths. In #10266 there is an attempt to add OIDC/WebIdentity support to the S3GW as a sidecar to the STS token development. This work proposes a good amount of changes near to the core request processing code without showing the same potential as Custos. From that PR for me it seems that adding a new mechanism requires the same branching of the code again, and adds additional load on the OM to figure out and do the authentication, and leaves authorization as it is today. Custos shows the potential to get rid of Hadoop RPC from our internal and client communication. It provides a separation of the authentication concern, in a way that if we need to add a new authmech we just need to touch one code part which does not affect our core request processing path (after the current code is cleared from the old auth related parts which will live there to provide backward compatibility for some time). And it does this universally without polluting our S3 API either. Custos for me is a complementary feature which does not simply just enable the addition of OIDC based auth, it is much more. Building on the same fundamental idea it moves the complexity of authentication and authorization outside the core request processing makes it horizontally scaleable and simplifies core processing in OM. I simply don't see how the STS development can fulfil this potential on its own, and I don't see how we can move towards a similar design where authn (and maybe authz) is separated from what happens in the core and represented by the same concept regardless of the authentication mechanism used. By releasing STS this way, for me it seems that the path towards this goal is closed by compatibility concerns. On the other hand, what I really miss from the design is how we can offload the current static S3 secret and STS token handling from OM towards a stateless service, because for that currently we use a table within the OM RocksDB, and that is shared state that needs to be protected by a concensus. I might missed/forgot from the design while I got to this point in my comment so @spacemonkd correct me if it is there. Also an interesting question is whether we can use the same Custos token instead of block and container tokens we currently use in the client <-> DN communication, but that is just a wild idea popped up in my mind without any reality check... @fmorg-git I would really love to see some more about how you imagine to steer the STS feature into a direction what is represented and aimed by Custos, can you please share your ideas how to achieve the same goals, or how youd would like to use STS in this concept instead of the CustosToken, as that is not clear for me, and you might have something in mind that we miss here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
