[
https://issues.apache.org/jira/browse/MAPREDUCE-6052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kumar Vavilapalli reopened MAPREDUCE-6052:
------------------------------------------------
I am reopening this JIRA for few minor issues, please post an addendum patch.
h4. Code
- Users can use URI fragments in distributed-cache - for e.g.
hdfs://a/b/c#symlink. Either we should explicitly reject fragments or support
them. See MRApps.parseDistributedCacheArtifacts() for similar changes that we
do for regular dist-cache files.
- MRApps.addLog4jSystemProperties(): If the user passes his own log4j file,
setting YARN_APP_CONTAINER_LOG_SIZE, YARN_APP_CONTAINER_LOG_BACKUPS and
logLevel may not matter at all. But let's set them as you do now, just leave a
code comment that setting them may or may not really take effect.
- job.setWorkingDirectory() getting called repeatedly?
- For the null-scheme input, "File " + file + " does not exist." -> "File " +
file + " does not exist on the local file-system."
- Delete this log statement that you might have added for your testing:
LOG.debug("default FileSystem: " + jtFs.getUri());
- What happens if the user passes a relative path for
MAPREDUCE_JOB_LOG4J_PROPERTIES_FILE? I think it works. If so, let's put this in
the documentation too.
h4. Documentation
- Document the new config in mapred-default.xml
- Mention in that documentation that if no-scheme is given in the path, it
defaults to a log4j file on the local FS.
- Modify the documentation of log-level configs to say that if you override to
have your own log4j.properties file, the log-level configs may not work
> Support overriding log4j.properties per job
> -------------------------------------------
>
> Key: MAPREDUCE-6052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 2.5.0
> Reporter: Junping Du
> Assignee: Junping Du
> Fix For: 2.6.0
>
> Attachments: MAPREDUCE-6052-v2.patch, MAPREDUCE-6052-v3.patch,
> MAPREDUCE-6052-v4.patch, MAPREDUCE-6052.patch
>
>
> For current MR application, the "log4j.configuration" is hard coded to
> container-log4j.properties within each node. We still need flexibility to
> override it per job like what we do in MRV1.
> {code}
> public static void addLog4jSystemProperties(
> String logLevel, long logSize, int numBackups, List<String> vargs) {
> vargs.add("-Dlog4j.configuration=container-log4j.properties");
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
