[
https://issues.apache.org/jira/browse/FLINK-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15805602#comment-15805602
]
ASF GitHub Bot commented on FLINK-5364:
---------------------------------------
Github user StephanEwen commented on a diff in the pull request:
https://github.com/apache/flink/pull/3057#discussion_r94994079
--- Diff: docs/internals/flink_security.md ---
@@ -24,64 +24,109 @@ specific language governing permissions and limitations
under the License.
-->
-This document briefly describes how Flink security works in the context of
various deployment mechanism (Standalone/Cluster vs YARN)
-and the connectors that participates in Flink Job execution stage. This
documentation can be helpful for both administrators and developers
-who plans to run Flink on a secure environment.
+This document briefly describes how Flink security works in the context of
various deployment mechanisms (Standalone, YARN, or Mesos),
+filesystems, connectors, and state backends.
## Objective
+The primary goals of the Flink Kerberos security infrastructure are:
+1. to enable secure data access for jobs within a cluster via connectors
(e.g. Kafka)
+2. to authenticate to ZooKeeper (if configured to use SASL)
+3. to authenticate to Hadoop components (e.g. HDFS, HBase)
-The primary goal of Flink security model is to enable secure data access
for jobs within a cluster via connectors. In a production deployment scenario,
-streaming jobs are understood to run for longer period of time
(days/weeks/months) and the system must be able to authenticate against secure
-data sources throughout the life of the job. The current implementation
supports running Flink clusters (Job Manager/Task Manager/Jobs) under the
-context of a Kerberos identity based on Keytab credential supplied during
deployment time. Any jobs submitted will continue to run in the identity of the
cluster.
+In a production deployment scenario, streaming jobs are understood to run
for long periods of time (days/weeks/months) and be able to authenticate to
secure
+data sources throughout the life of the job. Kerberos keytabs do not
expire in that timeframe, unlike a Hadoop delegation token
+or ticket cache entry.
+
+The current implementation supports running Flink clusters (Job
Manager/Task Manager/jobs) with either a configured keytab credential
+or with Hadoop delegation tokens. Keep in mind that all jobs share the
credential configured for a given cluster.
--- End diff --
Maybe point out here that this refers to a "Flink Cluster" (a set of
JobManager/TaskManager processes). One can run different jobs with different
credentials next to each other in YARN by starting different per-job-clusters
or Yarn/Mesos sessions.
> Rework JAAS configuration to support user-supplied entries
> ----------------------------------------------------------
>
> Key: FLINK-5364
> URL: https://issues.apache.org/jira/browse/FLINK-5364
> Project: Flink
> Issue Type: Bug
> Components: Cluster Management
> Reporter: Eron Wright
> Assignee: Eron Wright
> Priority: Critical
> Labels: kerberos, security
>
> Recent issues (see linked) have brought to light a critical deficiency in the
> handling of JAAS configuration.
> 1. the MapR distribution relies on an explicit JAAS conf, rather than
> in-memory conf used by stock Hadoop.
> 2. the ZK/Kafka/Hadoop security configuration is supposed to be independent
> (one can enable each element separately) but isn't.
> Perhaps we should rework the JAAS conf code to merge any user-supplied
> configuration with our defaults, rather than using an all-or-nothing
> approach.
> We should also address some recent regressions:
> 1. The HadoopSecurityContext should be installed regardless of auth mode, to
> login with UserGroupInformation, which:
> - handles the HADOOP_USER_NAME variable.
> - installs an OS-specific user principal (from UnixLoginModule etc.)
> unrelated to Kerberos.
> - picks up the HDFS/HBASE delegation tokens.
> 2. Fix the use of alternative authentication methods - delegation tokens and
> Kerberos ticket cache.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
