[jira] [Updated] (YARN-1622) 'bin/yarn' command doesn't behave like 'hadoop' and etc.
[ https://issues.apache.org/jira/browse/YARN-1622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Kamrul Islam updated YARN-1622: Attachment: YARN-1622.1.patch Simple one line patch in script. 'bin/yarn' command doesn't behave like 'hadoop' and etc. - Key: YARN-1622 URL: https://issues.apache.org/jira/browse/YARN-1622 Project: Hadoop YARN Issue Type: Bug Components: client Affects Versions: 2.2.0 Reporter: Mohammad Kamrul Islam Assignee: Mohammad Kamrul Islam Attachments: YARN-1622.1.patch There are few issues with 'bin/yarn' and 'etc/hadoop/yarn-env.sh'. They are loosely related but fixes are minor and will go in the same files. Therefore I combined them into one JIRA. Issues are: 1, bin/yarn has a dangling 'fi' in the last line. Thanks to shell for so compliant! 2. YARN_ROOT_LOGGER is defined as INFO, DFRA in yarn-env.sh. That's why 'bin/yarn' command doesn't show (by default) the log messages in client window. But when we used 'bin/hadoop', it shows the log correctly (because HADOOP_ROOT_LOOGER is INFO,console by default). Need to address this non-consistent behavior. 3. For each client command, yarn creates a log file in $YARN_LOG_DIR/yarn.log own by the 'end-user'. In a multi-tenant environment, the second user will not be able to create its own yarn.log in the same place causing the exception (pasted at the end). By default, it should write to $YARN_LOG_DIR/$USER/yarn.log instead. Note: I plan to address only #1 and #2 in this JIRA. If we default the YARN_ROOT_LOGGER consistent with 'bin/hadoop', the issue #3 will not happen. The scope of this JIRA is come close to 'bin/hadoop' behavior. Exception: log4j:ERROR setFile(null,true) call failed. java.io.FileNotFoundException: /export/apps/hadoop/logs/yarn.log (Permission denied) at java.io.FileOutputStream.open(Native Method) at java.io.FileOutputStream.init(FileOutputStream.java:221) at java.io.FileOutputStream.init(FileOutputStream.java:142) at org.apache.log4j.FileAppender.setFile(FileAppender.java:294) at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165) at org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:223) at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307) at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172) at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104) at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842) at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768) at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580) at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526) at org.apache.log4j.LogManager.clinit(LogManager.java:127) at org.apache.log4j.Logger.getLogger(Logger.java:104) at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:289) at org.apache.commons.logging.impl.Log4JLogger.init(Log4JLogger.java:109) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.commons.logging.impl.LogFactoryImpl.createLogFromClass(LogFactoryImpl.java:1116) at org.apache.commons.logging.impl.LogFactoryImpl.discoverLogImplementation(LogFactoryImpl.java:914) at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:604) at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:336) at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:310) at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:685) at org.apache.hadoop.conf.Configuration.clinit(Configuration.java:165) at org.apache.hadoop.util.RunJar.main(RunJar.java:158) -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1622) 'bin/yarn' command doesn't behave like 'hadoop' and etc.
[ https://issues.apache.org/jira/browse/YARN-1622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Kamrul Islam updated YARN-1622: Affects Version/s: 2.2.0 Summary: 'bin/yarn' command doesn't behave like 'hadoop' and etc. (was: 'yarn' command doesn't behave like 'hadoop' and etc. ) 'bin/yarn' command doesn't behave like 'hadoop' and etc. - Key: YARN-1622 URL: https://issues.apache.org/jira/browse/YARN-1622 Project: Hadoop YARN Issue Type: Bug Components: client Affects Versions: 2.2.0 Reporter: Mohammad Kamrul Islam Assignee: Mohammad Kamrul Islam There are few issues with 'bin/yarn' and 'etc/hadoop/yarn-env.sh'. They are loosely related but fixes are minor and will go in the same files. Therefore I combined them into one JIRA. Issues are: 1, bin/yarn has a dangling 'fi' in the last line. Thanks to shell for so compliant! 2. YARN_ROOT_LOGGER is defined as INFO, DFRA in yarn-env.sh. That's why 'bin/yarn' command doesn't show (by default) the log messages in client window. But when we used 'bin/hadoop', it shows the log correctly (because HADOOP_ROOT_LOOGER is INFO,console by default). Need to address this non-consistent behavior. 3. For each client command, yarn creates a log file in $YARN_LOG_DIR/yarn.log own by the 'end-user'. In a multi-tenant environment, the second user will not be able to create its own yarn.log in the same place causing the exception (pasted at the end). By default, it should write to $YARN_LOG_DIR/$USER/yarn.log instead. Note: I plan to address only #1 and #2 in this JIRA. If we default the YARN_ROOT_LOGGER consistent with 'bin/hadoop', the issue #3 will not happen. The scope of this JIRA is come close to 'bin/hadoop' behavior. Exception: log4j:ERROR setFile(null,true) call failed. java.io.FileNotFoundException: /export/apps/hadoop/logs/yarn.log (Permission denied) at java.io.FileOutputStream.open(Native Method) at java.io.FileOutputStream.init(FileOutputStream.java:221) at java.io.FileOutputStream.init(FileOutputStream.java:142) at org.apache.log4j.FileAppender.setFile(FileAppender.java:294) at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165) at org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:223) at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307) at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172) at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104) at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842) at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768) at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580) at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526) at org.apache.log4j.LogManager.clinit(LogManager.java:127) at org.apache.log4j.Logger.getLogger(Logger.java:104) at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:289) at org.apache.commons.logging.impl.Log4JLogger.init(Log4JLogger.java:109) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.commons.logging.impl.LogFactoryImpl.createLogFromClass(LogFactoryImpl.java:1116) at org.apache.commons.logging.impl.LogFactoryImpl.discoverLogImplementation(LogFactoryImpl.java:914) at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:604) at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:336) at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:310) at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:685) at org.apache.hadoop.conf.Configuration.clinit(Configuration.java:165) at org.apache.hadoop.util.RunJar.main(RunJar.java:158) -- This message was sent by Atlassian JIRA (v6.1.5#6160)
