[
https://issues.apache.org/jira/browse/MAPREDUCE-7523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18040571#comment-18040571
]
ASF GitHub Bot commented on MAPREDUCE-7523:
-------------------------------------------
steveloughran commented on code in PR #8100:
URL: https://github.com/apache/hadoop/pull/8100#discussion_r2559470037
##########
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/security/authorize/TestTaskLevelSecurityEnforcer.java:
##########
@@ -0,0 +1,140 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.mapreduce.v2.app.security.authorize;
+
+import org.junit.jupiter.api.Test;
+
+import org.apache.hadoop.mapred.JobConf;
+import org.apache.hadoop.mapreduce.MRConfig;
+import org.apache.hadoop.mapreduce.MRJobConfig;
+
+import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+
+public class TestTaskLevelSecurityEnforcer {
Review Comment:
extends 'AbstractHadoopTestBase`
##########
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/security/authorize/TaskLevelSecurityEnforcer.java:
##########
@@ -0,0 +1,103 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.mapreduce.v2.app.security.authorize;
+
+import java.util.Arrays;
+import java.util.List;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hadoop.mapred.JobConf;
+import org.apache.hadoop.mapreduce.MRConfig;
+import org.apache.hadoop.mapreduce.MRJobConfig;
+
+/**
+ * Enforces task-level security rules for MapReduce jobs.
+ *
+ * <p>This security enforcement mechanism validates whether the user who
submitted
+ * a job is allowed to execute the mapper/reducer/task classes defined in the
job
+ * configuration. The check is performed inside the Application Master before
+ * task containers are launched.</p>
+ * <p>If the user is not on the allowed list and any job property within the
configured
+ * security property domain references a denied class/prefix, a
+ * {@link TaskLevelSecurityException} is thrown and the job is rejected.</p>
+ * <p>This prevents unauthorized or unsafe custom code from running inside
+ * cluster containers.</p>
+ */
+public final class TaskLevelSecurityEnforcer {
+ private static final Logger LOG =
LoggerFactory.getLogger(TaskLevelSecurityEnforcer.class);
+
+ /**
+ * Default constructor.
+ */
+ private TaskLevelSecurityEnforcer() {
+ }
+
+ /**
+ * Validates a MapReduce job's configuration against the cluster's task-level
+ * security policy.
+ *
+ * <p>The method performs the following steps:</p>
+ * <ol>
+ * <li>Check whether task-level security is enabled.</li>
+ * <li>Allow the job immediately if the user is on the configured
allowed-users list.</li>
+ * <li>Retrieve the security property domain (list of job configuration
keys to inspect).</li>
+ * <li>Retrieve the list of denied task class prefixes.</li>
+ * <li>For each domain property, check whether its value begins with any
denied prefix.</li>
+ * <li>If a match is found, reject the job by throwing {@link
TaskLevelSecurityException}.</li>
+ * </ol>
+ *
+ * @param conf the job configuration to validate
+ * @throws TaskLevelSecurityException if the user is not authorized to use
one of the task classes
+ */
+ public static void validate(JobConf conf) throws TaskLevelSecurityException {
+ if (!conf.getBoolean(MRConfig.SECURITY_ENABLED,
MRConfig.DEFAULT_SECURITY_ENABLED)) {
+ LOG.debug("The {} is disabled", MRConfig.SECURITY_ENABLED);
+ return;
+ }
+
+ String currentUser = conf.get(MRJobConfig.USER_NAME);
Review Comment:
looking at JobSubmitter, this is set to UGI.currentUser.shortName. Is that
enough? Shouldn't it be on fullName?
if so, JobSubmitter should add that as a property too
##########
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml:
##########
@@ -2282,4 +2282,44 @@
</description>
</property>
+<property>
+ <name>mapreduce.security.enabled</name>
+ <value>false</value>
+ <description>
+ Enables MapReduce Task-Level Security Enforcement
+ When enabled, the Application Master performs validation of user-submitted
mapper, reducer, and other task-related classes before launching containers.
+ This mechanism protects the cluster from running disallowed or unsafe task
implementations as defined by administrator-controlled policies.
+ </description>
+</property>
+
+<property>
+ <name>mapreduce.security.property-domain</name>
+
<value>mapreduce.job.combine.class,mapreduce.job.combiner.group.comparator.class,mapreduce.job.end-notification.custom-notifier-class,mapreduce.job.inputformat.class,mapreduce.job.map.class,mapreduce.job.map.output.collector.class,mapreduce.job.output.group.comparator.class,mapreduce.job.output.key.class,mapreduce.job.output.key.comparator.class,mapreduce.job.output.value.class,mapreduce.job.outputformat.class,mapreduce.job.partitioner.class,mapreduce.job.reduce.class,mapreduce.map.output.key.class,mapreduce.map.output.value.class</value>
+ <description>
Review Comment:
add all the chaining task properties too, presumably
> MapReduce Task-Level Security Enforcement
> -----------------------------------------
>
> Key: MAPREDUCE-7523
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7523
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: mrv2
> Reporter: Bence Kosztolnik
> Priority: Major
> Labels: pull-request-available
>
> h2. Overview
> The goal of this feature to provide a configurable mechanism to control which
> users are allowed to execute specific MapReduce jobs.
> This feature aims to prevent unauthorized or potentially harmful
> mapper/reducer implementations from running within the Hadoop cluster.
> In the standard Hadoop MapReduce execution flow:
> 1) A MapReduce job is submitted by a user.
> 2) The job is registered with the Resource Manager (RM).
> 3) The RM assigns the job to a Node Manager (NM), where the Application
> Master (AM) for the job is launched.
> 4) The AM requests additional containers from the cluster, to be able to
> start tasks.
> 5) The NM launches those containers, and the containers execute the
> mapper/reducer tasks defined by the job.
> The proposed feature introduces a security filtering mechanism inside the
> Application Master.
> Before mapper or reducer tasks are launched, the AM will verify that the
> user-submitted MapReduce code complies with a cluster-defined security
> policy.
> This ensures that only approved classes or packages can be executed inside
> the containers.
> The goal is to protect the cluster from unwanted or unsafe task
> implementations, such as custom code that may introduce performance,
> stability, or security risks.
> Upon receiving job metadata, the Application Master will:
> 1) Check the feature is enabled.
> 2) Check the user who submitted the job is allowed to bypass the security
> check.
> 3) Compare classes in job config against the denied task list.
> 4) If job is not authorised an exception will be thrown and AM will fail.
> h2. New Configs
> h5. Enables MapReduce Task-Level Security Enforcement
> When enabled, the Application Master performs validation of user-submitted
> mapper, reducer, and other task-related classes before launching containers.
> This mechanism protects the cluster from running disallowed or unsafe task
> implementations as defined by administrator-controlled policies.
> - Property name: mapreduce.security.enabled
> - Property type: boolean
> - Default: false (security disabled)
> h5. MapReduce Task-Level Security Enforcement: Property Domain
> Defines the set of MapReduce configuration keys that represent user-supplied
> class names involved in task execution (e.g., mapper, reducer, partitioner).
> The Application Master examines the values of these properties and checks
> whether any referenced class is listed in denied tasks.
> Administrators may override this list to expand or restrict the validation
> domain.
> - Property name: mapreduce.security.property-domain
> - Property type: list of configuration keys
> - Default:
> * mapreduce.job.combine.class
> * mapreduce.job.combiner.group.comparator.class
> * mapreduce.job.end-notification.custom-notifier-class
> * mapreduce.job.inputformat.class
> * mapreduce.job.map.class
> * mapreduce.job.map.output.collector.class
> * mapreduce.job.output.group.comparator.class
> * mapreduce.job.output.key.class
> * mapreduce.job.output.key.comparator.class
> * mapreduce.job.output.value.class
> * mapreduce.job.outputformat.class
> * mapreduce.job.partitioner.class
> * mapreduce.job.reduce.class
> * mapreduce.map.output.key.class
> * mapreduce.map.output.value.class
> h5. MapReduce Task-Level Security Enforcement: Denied Tasks
> Specifies the list of disallowed task implementation classes or packages.
> If a user submits a job whose mapper, reducer, or other task-related classes
> match any entry in this blacklist.
> - Property name: mapreduce.security.denied-tasks
> - Property type: list of class name or package patterns
> - Default: empty
> - Example:
> org.apache.hadoop.streaming,org.apache.hadoop.examples.QuasiMonteCarlo
> h5. MapReduce Task-Level Security Enforcement: Allowed Users
> Specifies users who may bypass the blacklist defined in denied tasks.
> This whitelist is intended for trusted or system-level workflows that may
> legitimately require the use of restricted task implementations.
> If the submitting user is listed here, blacklist enforcement is skipped,
> although standard Hadoop authentication and ACL checks still apply.
> - Property name: mapreduce.security.allowed-users
> - Property type: list of usernames
> - Default: empty
> - Example: alice,bob
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
