[
https://issues.apache.org/jira/browse/HADOOP-11211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14177453#comment-14177453
]
Sangjin Lee commented on HADOOP-11211:
--------------------------------------
Thanks for reporting this [~timyitong]!
Actually this needs a little discussion. I agree that the current behavior is
not as well documented as it should. As you stated, it is a "first-match-wins"
behavior.
What you propose also seems reasonable, but needs more clarification. Your
proposal is to disregard the order and consider all matches and do their
logical AND's. But there can be cases where that rule is counter-intuitive. In
your example, if you have "foo.bar.,-foo.bar." it would mean that the class is
not considered a system class. Does that mean that an exclusion match is
stronger than an inclusion match?
Here is an even more interesting case: how about "foo.bar.tar.,-foo.bar."? One
reasonable interpretation of this may be that "exclude everything that belongs
to foo.bar. but still include foo.bar.tar.". However, using the AND rule, it
would mean that all classes under "foo.bar" would be excluded and the
"foo.bar.tar." rule would be effectively ignored.
Also, how about a exact match (e.g. "foo.bar.MyClass")? Would it need to be
considered over package matches?
> mapreduce.job.classloader.system.classes property behaves differently when
> the exclusion and inclusion order is different
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-11211
> URL: https://issues.apache.org/jira/browse/HADOOP-11211
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Yitong Zhou
> Assignee: Yitong Zhou
>
> If we want to include package foo.bar.* but exclude all sub packages named
> foo.bar.tar.* in system classes, configuring
> "mapreduce.job.classloader.system.classes=foo.bar.,-foo.bar.tar." won't work.
> foo.bar.tar will still be pulled in. But if we change the order:
> "mapreduce.job.classloader.system.classes=-foo.bar.tar.,foo.bar.", then it
> will work.
> This bug is due to the implementation of ApplicationClassLoaser#isSystemClass
> in hadoop-common, where we simply return the matching result immediately when
> the class name hits the first match (either positive or negative).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
