Hello, now might be a good time to revisit an important enhancement to Flink security, so-called service authorization. This means the hardening of a Flink cluster against unauthorized use with some sort of authentication and authorization scheme. Today, Flink relies entirely on network isolation to protect itself from unauthorized job submission and control, and to protect the secrets contained within a Flink cluster. This is a problem in multi-user environments like YARN/Mesos/K8.
Last fall, an effort was made to implement service authorization but the PR was ultimately rejected. The idea was to add a simple secret key to all network communication between the client, JM, and TM. Akka itself has such a feature which formed the basis of the solution. There are usability challenges with this solution, including a dependency on SSL. Since then, the situation has evolved somewhat, and the use of SSL mutual authentication is more viable. Mutual auth is supported in Akka 2.4.12+ (or could be backported to Flakka). My proposal is: 1. Upgrade Akka or backport the functionality to Flakka (see commit 5d03902c5ec3212cd28f26c9b3ef7c3b628b9451). 2. Implement SSL on any endpoint that doesn't yet support it (e.g. queryable state). 3. Enable mutual auth in Akka and implement it on non-Akka endpoints. 4. Implement a simple authorization layer that accepts any authenticated connection. 5. (stretch) generate and store a certificate automatically in YARN mode. 6. (stretch) Develop an alternate authentication method for the Web UI. Are folks interested in this capability? Thoughts on the use of SSL mutual auth versus something else? Thanks! -Eron
