reference document which properties are set in which configuration file
Dear all, while configuring our hadoop cluster I wonder whether there exists a reference document that contains information about which configuration property has to be specified in which properties file. Especially I do not know where the mapred.task.tracker.http.address has to be set. Is it in the mapre-site.xml or in the hdfs-site.xml? any hint will be appreciated thanks Christian 8<-- Siemens AG Corporate Technology Corporate Research and Technologies CT T DE IT3 Otto-Hahn-Ring 6 81739 München, Deutschland Tel.: +49 89 636-42722 Fax: +49 89 636-41423 mailto:christian.kleegr...@siemens.com Siemens Aktiengesellschaft: Vorsitzender des Aufsichtsrats: Gerhard Cromme; Vorstand: Peter Löscher, Vorsitzender; Roland Busch, Brigitte Ederer, Klaus Helmrich, Joe Kaeser, Barbara Kux, Hermann Requardt, Siegfried Russwurm, Peter Y. Solmssen, Michael Süß; Sitz der Gesellschaft: Berlin und München, Deutschland; Registergericht: Berlin Charlottenburg, HRB 12300, München, HRB 6684; WEEE-Reg.-Nr. DE 23691322
Re: reference document which properties are set in which configuration file
As a thumb rule, all properties starting with mapred.* or mapreduce.* go to mapred-site.xml, all properties starting with dfs.* go to hdfs-site.xml, and the rest may be put in core-site.xml to be safe. In case you notice MR or HDFS specific properties being outside of this naming convention, please do report a JIRA so we can deprecate the old name and rename it with a more appropriate prefix. On Sat, Feb 11, 2012 at 9:27 AM, Praveen Sripati wrote: > The mapred.task.tracker.http.address will go in the mapred-site.xml file. > > In the Hadoop installation directory check the core-default.xml, > hdfs-default,xml and mapred-default.xml files to know about the different > properties. Some of the properties which might be in the code may not be > mentioned in the xml files and will be defaulted. > > Praveen > > On Tue, Feb 7, 2012 at 3:30 PM, Kleegrewe, Christian < > christian.kleegr...@siemens.com> wrote: > >> Dear all, >> >> while configuring our hadoop cluster I wonder whether there exists a >> reference document that contains information about which configuration >> property has to be specified in which properties file. Especially I do not >> know where the mapred.task.tracker.http.address has to be set. Is it in the >> mapre-site.xml or in the hdfs-site.xml? >> >> any hint will be appreciated >> >> thanks >> >> Christian >> >> >> 8<-- >> Siemens AG >> Corporate Technology >> Corporate Research and Technologies >> CT T DE IT3 >> Otto-Hahn-Ring 6 >> 81739 München, Deutschland >> Tel.: +49 89 636-42722 >> Fax: +49 89 636-41423 >> mailto:christian.kleegr...@siemens.com >> >> Siemens Aktiengesellschaft: Vorsitzender des Aufsichtsrats: Gerhard >> Cromme; Vorstand: Peter Löscher, Vorsitzender; Roland Busch, Brigitte >> Ederer, Klaus Helmrich, Joe Kaeser, Barbara Kux, Hermann Requardt, >> Siegfried Russwurm, Peter Y. Solmssen, Michael Süß; Sitz der Gesellschaft: >> Berlin und München, Deutschland; Registergericht: Berlin Charlottenburg, >> HRB 12300, München, HRB 6684; WEEE-Reg.-Nr. DE 23691322 >> >> >> -- Harsh J Customer Ops. Engineer Cloudera | http://tiny.cloudera.com/about
Re: reference document which properties are set in which configuration file
Harsh, All This was one of the first questions that I asked. It is sometimes not clear whether some parameters are site related or jab related or whether it belongs to NN, JT , DN or TT. If I get some time during the weekend , I will try and put this into a document and see if it helps Raj > > From: Harsh J >To: common-user@hadoop.apache.org >Sent: Friday, February 10, 2012 8:31 PM >Subject: Re: reference document which properties are set in which >configuration file > >As a thumb rule, all properties starting with mapred.* or mapreduce.* >go to mapred-site.xml, all properties starting with dfs.* go to >hdfs-site.xml, and the rest may be put in core-site.xml to be safe. > >In case you notice MR or HDFS specific properties being outside of >this naming convention, please do report a JIRA so we can deprecate >the old name and rename it with a more appropriate prefix. > >On Sat, Feb 11, 2012 at 9:27 AM, Praveen Sripati > wrote: >> The mapred.task.tracker.http.address will go in the mapred-site.xml file. >> >> In the Hadoop installation directory check the core-default.xml, >> hdfs-default,xml and mapred-default.xml files to know about the different >> properties. Some of the properties which might be in the code may not be >> mentioned in the xml files and will be defaulted. >> >> Praveen >> >> On Tue, Feb 7, 2012 at 3:30 PM, Kleegrewe, Christian < >> christian.kleegr...@siemens.com> wrote: >> >>> Dear all, >>> >>> while configuring our hadoop cluster I wonder whether there exists a >>> reference document that contains information about which configuration >>> property has to be specified in which properties file. Especially I do not >>> know where the mapred.task.tracker.http.address has to be set. Is it in the >>> mapre-site.xml or in the hdfs-site.xml? >>> >>> any hint will be appreciated >>> >>> thanks >>> >>> Christian >>> >>> >>> 8<-- >>> Siemens AG >>> Corporate Technology >>> Corporate Research and Technologies >>> CT T DE IT3 >>> Otto-Hahn-Ring 6 >>> 81739 München, Deutschland >>> Tel.: +49 89 636-42722 >>> Fax: +49 89 636-41423 >>> mailto:christian.kleegr...@siemens.com >>> >>> Siemens Aktiengesellschaft: Vorsitzender des Aufsichtsrats: Gerhard >>> Cromme; Vorstand: Peter Löscher, Vorsitzender; Roland Busch, Brigitte >>> Ederer, Klaus Helmrich, Joe Kaeser, Barbara Kux, Hermann Requardt, >>> Siegfried Russwurm, Peter Y. Solmssen, Michael Süß; Sitz der Gesellschaft: >>> Berlin und München, Deutschland; Registergericht: Berlin Charlottenburg, >>> HRB 12300, München, HRB 6684; WEEE-Reg.-Nr. DE 23691322 >>> >>> >>> > > > >-- >Harsh J >Customer Ops. Engineer >Cloudera | http://tiny.cloudera.com/about > > >
AW: reference document which properties are set in which configuration file
Hi all, thank you for the answers. Especially the hints from Harsh were helpful. Since I was in a little hurry I tried out the different properties in the different files and finally succeeded. I think my worst problem was that I had a typo in the property definition. So if I take care of this in future I think I do not need a reference document any more. Thanks and best regards, Christian 8<-- Siemens AG Corporate Technology Corporate Research and Technologies CT T DE IT3 Otto-Hahn-Ring 6 81739 München, Deutschland Tel.: +49 89 636-42722 Fax: +49 89 636-41423 mailto:christian.kleegr...@siemens.com Siemens Aktiengesellschaft: Vorsitzender des Aufsichtsrats: Gerhard Cromme; Vorstand: Peter Löscher, Vorsitzender; Roland Busch, Brigitte Ederer, Klaus Helmrich, Joe Kaeser, Barbara Kux, Hermann Requardt, Siegfried Russwurm, Peter Y. Solmssen, Michael Süß; Sitz der Gesellschaft: Berlin und München, Deutschland; Registergericht: Berlin Charlottenburg, HRB 12300, München, HRB 6684; WEEE-Reg.-Nr. DE 23691322 -Ursprüngliche Nachricht- Von: Praveen Sripati [mailto:praveensrip...@gmail.com] Gesendet: Samstag, 11. Februar 2012 04:58 An: common-user@hadoop.apache.org Betreff: Re: reference document which properties are set in which configuration file The mapred.task.tracker.http.address will go in the mapred-site.xml file. In the Hadoop installation directory check the core-default.xml, hdfs-default,xml and mapred-default.xml files to know about the different properties. Some of the properties which might be in the code may not be mentioned in the xml files and will be defaulted. Praveen On Tue, Feb 7, 2012 at 3:30 PM, Kleegrewe, Christian < christian.kleegr...@siemens.com> wrote: > Dear all, > > while configuring our hadoop cluster I wonder whether there exists a > reference document that contains information about which configuration > property has to be specified in which properties file. Especially I do not > know where the mapred.task.tracker.http.address has to be set. Is it in the > mapre-site.xml or in the hdfs-site.xml? > > any hint will be appreciated > > thanks > > Christian > > > 8<-- > Siemens AG > Corporate Technology > Corporate Research and Technologies > CT T DE IT3 > Otto-Hahn-Ring 6 > 81739 München, Deutschland > Tel.: +49 89 636-42722 > Fax: +49 89 636-41423 > mailto:christian.kleegr...@siemens.com > > Siemens Aktiengesellschaft: Vorsitzender des Aufsichtsrats: Gerhard > Cromme; Vorstand: Peter Löscher, Vorsitzender; Roland Busch, Brigitte > Ederer, Klaus Helmrich, Joe Kaeser, Barbara Kux, Hermann Requardt, > Siegfried Russwurm, Peter Y. Solmssen, Michael Süß; Sitz der Gesellschaft: > Berlin und München, Deutschland; Registergericht: Berlin Charlottenburg, > HRB 12300, München, HRB 6684; WEEE-Reg.-Nr. DE 23691322 > > >
