[jira] [Created] (HADOOP-10625) Configuration: names should be trimmed when putting/getting to properties

2014-05-21 Thread Wangda Tan (JIRA) 
Wangda Tan created HADOOP-10625:
---

 Summary: Configuration: names should be trimmed when 
putting/getting to properties
 Key: HADOOP-10625
 URL: https://issues.apache.org/jira/browse/HADOOP-10625
 Project: Hadoop Common
  Issue Type: Bug
  Components: conf
Affects Versions: 2.4.0
Reporter: Wangda Tan


Currently, Hadoop will not trim name when putting a pair of k/v to property. 
But when loading configuration from file, names will be trimmed:
(In Configuration.java)
{code}
  if ("name".equals(field.getTagName()) && field.hasChildNodes())
attr = StringInterner.weakIntern(
((Text)field.getFirstChild()).getData().trim());
  if ("value".equals(field.getTagName()) && field.hasChildNodes())
value = StringInterner.weakIntern(
((Text)field.getFirstChild()).getData());
{code}
With this behavior, following steps will be problematic:
1. User incorrectly set " hadoop.key=value" (with a space before hadoop.key)
2. User try to get "hadoop.key", cannot get "value"
3. Serialize/deserialize configuration (Like what did in MR)
4. User try to get "hadoop.key", can get "value", which will make inconsistency 
problem.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: HADOOP_ROOT_LOGGER

2014-05-21 Thread Vinayakumar B 
Hi Robert,

I understand your confusion.

HADOOP_ROOT_LOGGER is set to default value "INFO,console" if it hasn't set
for anything and logs will be displayed on the console itself.
This will be true for any client commands you run. For ex: "hdfs dfs -ls /"

But for the server scripts (hadoop-daemon.sh, yarn-daemon.sh, etc)
 HADOOP_ROOT_LOGGER will be set to "INFO, RFA" if HADOOP_ROOT_LOGGER env
variable is not defined.
So that all the log messages of the server daemons goto some log files and
this will be maintained by RollingFileAppender. If you want to override all
these default and set your own loglevel then define that as env variable
HADOOP_ROOT_LOGGER.

For ex:
   export HADOOP_ROOT_LOGGER="DEBUG,RFA"
  export above env variable and then start server scripts or execute client
commands, all logs goto files and will be maintained by RollingFileAppender.


Regards,
Vinay


On Wed, May 21, 2014 at 6:42 PM, Robert Rati  wrote:

> I noticed in hadoop-config.sh there is this line:
>
> HADOOP_OPTS="$HADOOP_OPTS -Dhadoop.root.logger=${HADOOP_
> ROOT_LOGGER:-INFO,console}"
>
> which is setting a root logger if HADOOP_ROOT_LOGGER isn't set.  Why is
> this here.needed?  There is a log4j.properties file provided that defines a
> default logger.  I believe the line above will result in overriding
> whatever is set for the root logger in the log4j.properties file.  This has
> caused some confusion and hacks to work around this.
>
> Is there a reason not to remove the above code and just have all the
> logger definitions in the log4j.properties file?  Is there maybe a
> compatibility concern?
>
> Rob
>



-- 
Regards,
Vinay

HADOOP_ROOT_LOGGER

2014-05-21 Thread Robert Rati 

I noticed in hadoop-config.sh there is this line:

HADOOP_OPTS="$HADOOP_OPTS 
-Dhadoop.root.logger=${HADOOP_ROOT_LOGGER:-INFO,console}"


which is setting a root logger if HADOOP_ROOT_LOGGER isn't set.  Why is 
this here.needed?  There is a log4j.properties file provided that 
defines a default logger.  I believe the line above will result in 
overriding whatever is set for the root logger in the log4j.properties 
file.  This has caused some confusion and hacks to work around this.


Is there a reason not to remove the above code and just have all the 
logger definitions in the log4j.properties file?  Is there maybe a 
compatibility concern?


Rob

[jira] [Created] (HADOOP-10624) Fix some minors typo and add more test cases for hadoop_err

2014-05-21 Thread Wenwu Peng (JIRA) 
Wenwu Peng created HADOOP-10624:
---

 Summary: Fix some minors typo and add more test cases for 
hadoop_err
 Key: HADOOP-10624
 URL: https://issues.apache.org/jira/browse/HADOOP-10624
 Project: Hadoop Common
  Issue Type: Sub-task
Affects Versions: HADOOP-10388
Reporter: Wenwu Peng
Assignee: Wenwu Peng


Changes:
1. Add more test cases to cover method hadoop_lerr_alloc and hadoop_uverr_alloc
2. Fix typo as following:
1) Change hadoop_uverr_alloc(int cod to hadoop_uverr_alloc(int code in 
hadoop_err.h
2) Change OutOfMemory to OutOfMemoryException to consistent with other 
Exception in hadoop_err.c
3) Change DBUG to DEBUG in messenger.c
4) Change DBUG to DEBUG in reactor.c




--
This message was sent by Atlassian JIRA
(v6.2#6252)