Re: should clients be able to override *-site.xml?

2014-09-11 Thread Sangjin Lee
Thanks for the clarification. Yes I agree that there are many ways for users to override *-site.xml regardless of what the hadoop scripts do. It was more of a question of the interpretation of intent, and I got the answer. Thanks! Sangjin On Wed, Sep 10, 2014 at 4:34 PM, Chris Nauroth wrote: >

Re: should clients be able to override *-site.xml?

2014-09-10 Thread Chris Nauroth
I participated on the HADOOP-9450 code review. There was some debate about the significance of keeping HADOOP_CONF_DIR first. Ultimately, there was agreement that "FIRST" really ought to mean first. Apparently, other Hadoop ecosystem projects have made the same interpretation and implemented the

Re: should clients be able to override *-site.xml?

2014-09-10 Thread Allen Wittenauer
On Sep 10, 2014, at 10:14 AM, Sangjin Lee wrote: > if HADOOP_USER_CLASSPATH_FIRST is > set and the user provides his/her version of *-site.xml through > HADOOP_CLASSPATH, the user would end up trumping the hadoop configuration. > And I believe this behavior is preserved after Allen's changes >

Re: should clients be able to override *-site.xml?

2014-09-10 Thread Zhijie Shen
Sangjin, Does "yarn --config " work for you? Zhijie On Wed, Sep 10, 2014 at 10:14 AM, Sangjin Lee wrote: > tl;dr > > This was borne out of an investigation of a hadoop issue at our company. It > appears that prior to https://issues.apache.org/jira/browse/HADOOP-9450 > the > hadoop configurati

should clients be able to override *-site.xml?

2014-09-10 Thread Sangjin Lee
tl;dr This was borne out of an investigation of a hadoop issue at our company. It appears that prior to https://issues.apache.org/jira/browse/HADOOP-9450 the hadoop configuration directory (HADOOP_CONF_DIR) was always the first entry in the classpath, regardless of HADOOP_USER_CLASSPATH_FIRST or H