Yes you did. Thanks in advance. Is there any way to push this config to all the nodes from master or should I make to script to do this?
Regards Enviado desde mi iPhone > El 12 feb. 2017, a las 02:30, Brahma Reddy Battula <bra...@apache.org> > escribió: > > Hi Alexis Fidalgo > > 1) I did not seen this query recently > > 2) you need to configure this property in slaves ( DataNode). > > dfs.datanode.du.reserved : The number of bytes will be left free on the > volumes used by the DataNodes. By Default,it's zero. > > > For example if the disk capacity is 1TB and dfs.datanode.du.reserved > configured with 100GB.So DataNode will not use 100GB for block allocation,so > this data can be used by nodemanager intermittent files, log files . > May be you can plan your MR jobs accordingly this. Hope I cleared your doubts. > >> On Sat, Feb 11, 2017 at 7:26 PM, Alexis Fidalgo <alz...@gmail.com> wrote: >> Hello, i’ve tried to search archives (and google) regarding this issue but >> had no luck. After some changes in our mapreduce code, it takes all the >> available disk space on datanodes, before this change we hade no problem at >> all, but since then, every few days, disks on datanodes (we have 4, all with >> same configuration regarding disk, memory, processor, OS) becomes full and >> we have no more mapreduce jobs completed. so i need to wipe datanodes and >> format namenode and start all over again. >> >> Reading documentation i found this configuration for hdfs-site.xml >> >> <property> >> <name>dfs.datanode.du.reserved</name> >> <value>32212254720</value> >> </property> >> </configuration> >> >> Questions regarding this >> >> 1. is there any thread already on this issue to read and not to ask again >> about it? >> 2. if not 1, do i need to set up this property only on master or every slave >> too? >> 3. will this fix the problem or just avoid the disk become full but the MR >> jobs will fail the same (no more space to work so we need to review our code) >> >> >> thanks in advance, sorry if im asking about an already discussed issue, i >> just suscribed to the list. >> >> regards >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: user-unsubscr...@hadoop.apache.org >> For additional commands, e-mail: user-h...@hadoop.apache.org >> > > > > -- > > > > --Brahma Reddy Battula
