Re: conf.get("dfs.data.dir") return null when hdfs-site.xml doesn't set it explicitly

nfiguration > variables/values, which one should be returned? > Yong > > Date: Tue, 9 Sep 2014 10:01:14 -0700 > Subject: Re: conf.get("dfs.data.dir") return null when hdfs-site.xml doesn't > set it explicitly > From: nid...@gmail.com > To: user@hadoop.apache.

Re: conf.get("dfs.data.dir") return null when hdfs-site.xml doesn't set it explicitly

computer could have different cpu cores, > memory, disk counts, mount names etc. When you ask configuration > variables/values, which one should be returned? > > Yong > > -- > Date: Tue, 9 Sep 2014 10:01:14 -0700 > Subject: Re: conf.get("

RE: conf.get("dfs.data.dir") return null when hdfs-site.xml doesn't set it explicitly

an get a configuration object reference from the JobContext. Yong Date: Tue, 9 Sep 2014 11:34:03 -0700 Subject: Re: conf.get("dfs.data.dir") return null when hdfs-site.xml doesn't set it explicitly From: nid...@gmail.com To: user@hadoop.apache.org Yong, good point about each node of t

Re: conf.get("dfs.data.dir") return null when hdfs-site.xml doesn't set it explicitly

ent cpu cores, > memory, disk counts, mount names etc. When you ask configuration > variables/values, which one should be returned? > > Yong > > ------ > Date: Tue, 9 Sep 2014 10:01:14 -0700 > Subject: Re: conf.get("dfs.data.dir")

RE: conf.get("dfs.data.dir") return null when hdfs-site.xml doesn't set it explicitly

counts, mount names etc. When you ask configuration variables/values, which one should be returned? Yong Date: Tue, 9 Sep 2014 10:01:14 -0700 Subject: Re: conf.get("dfs.data.dir") return null when hdfs-site.xml doesn't set it explicitly From: nid...@gmail.com To: user@hadoop.apac

Re: conf.get("dfs.data.dir") return null when hdfs-site.xml doesn't set it explicitly

Susheel actually brought up a good point. once the client code connects to the cluster, is there way to get the real cluster configuration variables/values instead of relying on the .xml files on client side? Demai On Mon, Sep 8, 2014 at 10:12 PM, Susheel Kumar Gadalay wrote: > One doubt on bu

Re: conf.get("dfs.data.dir") return null when hdfs-site.xml doesn't set it explicitly

One doubt on building Configuration object. I have a Hadoop remote client and Hadoop cluster. When a client submitted a MR job, the Configuration object is built from Hadoop cluster node xml files, basically the resource manager node core-site.xml and mapred-site.xml and yarn-site.xml. Am I correc

Re: conf.get("dfs.data.dir") return null when hdfs-site.xml doesn't set it explicitly

Bhooshan, Many thanks. I appreciate the help. I will also try out Cloudera mailing list/community Demai On Mon, Sep 8, 2014 at 4:58 PM, Bhooshan Mogal wrote: > Hi Demai, > > conf = new Configuration() > > will create a new Configuration object and only add the properties from > core-default.xm

Re: conf.get("dfs.data.dir") return null when hdfs-site.xml doesn't set it explicitly

Hi Demai, conf = new Configuration() will create a new Configuration object and only add the properties from core-default.xml and core-site.xml in the conf object. This is basically a new configuration object, not the same that the daemons in the hadoop cluster use. I think what you are tryin

Re: conf.get("dfs.data.dir") return null when hdfs-site.xml doesn't set it explicitly

hi, Bhooshan, thanks for your kind response. I run the code on one of the data node of my cluster, with only one hadoop daemon running. I believe my java client code connect to the cluster correctly as I am able to retrieve fileStatus, and list files under a particular hdfs path, and similar thin

Re: conf.get("dfs.data.dir") return null when hdfs-site.xml doesn't set it explicitly

Hi Demai, When you read a property from the conf object, it will only have a value if the conf object contains that property. In your case, you created the conf object as new Configuration() -- adds core-default and core-site.xml. Then you added site.xmls (hdfs-site.xml and core-site.xml) from s