[JIRA] (COLL-117) seeing warnings "attempt to override final parameter: dfs.data.dir" in tasktracker logs

2010-06-21 Thread Bikash Singhal

Hi Hadoopers,

I have received WARN in the hadoop cluster. Has anybody seen this . Any 
solution?


> 2010-06-06 01:45:04,079 WARN org.apache.hadoop.conf.Configuration: 
> /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201006060025_0003/job.xml:a
>  attempt to override final parameter: dfs.data.dir;  Ignoring.
> 
> 2010-06-06 01:49:41,501 WARN org.apache.hadoop.conf.Configuration: 
> /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201006060025_0004/job.xml:a
>  attempt to override final parameter: dfs.data.dir;  Ignoring.


"Legal Disclaimer: This electronic message and all contents contain information 
from Cybage Software Private Limited which may be privileged, confidential, or 
otherwise protected from disclosure. The information is intended to be for the 
addressee(s) only. If you are not an addressee, any disclosure, copy, 
distribution, or use of the contents of this message is strictly prohibited. If 
you have received this electronic message in error please notify the sender by 
reply e-mail to and destroy the original message and all copies. Cybage has 
taken every reasonable precaution to minimize the risk of malicious content in 
the mail, but is not liable for any damage you may sustain as a result of any 
malicious content in this e-mail. You should carry out your own malicious 
content checks before opening the e-mail or attachment." 
www.cybage.com



FW: hadoop-cluster-1.ic node errors and warnings

2010-06-20 Thread Bikash Singhal
=hadoop:supergroup:rw-r--r--
2010-06-17 18:50:06,187 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
NameSystem.allocateBlock:
/i2a_logs/2010-06-17/i2a-coll-4.ic-100617-182531-access.log.0.
blk_1130168086083501329_28833
2010-06-17 18:50:07,024 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
NameSystem.addStoredBlock: blockMap updated: 10.13.53.127:50010 is added
to blk_1130168086083501329_28833 size 32278706

 - jobtracker errors and warning -- 

2010-06-17 17:00:06,382 WARN org.apache.hadoop.conf.Configuration:
/var/lib/hadoop-0.20/cache/hadoop/mapred/local/jobTracker/job_2010060901
23_0067.xml:a attempt to override final parameter: dfs.data.dir;
Ignoring.
2010-06-17 17:00:06,448 DEBUG org.apache.hadoop.mapred.JobTracker:
Per-Task memory configuration is not set on JT. Not checking the job for
invalid memory requirements.
2010-06-17 17:00:06,449 INFO org.apache.hadoop.mapred.JobTracker:
Initializing job_201006090123_0067
2010-06-17 17:00:06,451 INFO org.apache.hadoop.mapred.JobInProgress:
Initializing job_201006090123_0067
2010-06-17 17:00:06,698 INFO org.apache.hadoop.mapred.JobInProgress:
Input size for job job_201006090123_0067 = 8493462348. Number of splits
= 288
2010-06-17 17:00:06,698 INFO org.apache.hadoop.mapred.JobInProgress:
tip:task_201006090123_0067_m_00 has split on
node:/default-rack/10.13.53.127
2010-06-17 17:00:06,699 INFO org.apache.hadoop.mapred.JobInProgress:
tip:task_201006090123_0067_m_00 has split on
node:/default-rack/10.13.53.156
2010-06-17 17:00:06,699 INFO org.apache.hadoop.mapred.JobInProgress:
tip:task_201006090123_0067_m_00 has split on
node:/default-rack/10.13.53.128
2010-06-17 17:00:06,699 INFO org.apache.hadoop.mapred.JobInProgress:
tip:task_201006090123_0067_m_01 has split on
node:/default-rack/10.13.53.127
2010-06-17 17:00:06,699 INFO org.apache.hadoop.mapred.JobInProgress:
tip:task_201006090123_0067_m_01 has split on
node:/default-rack/10.13.53.129
2010-06-17 17:00:06,699 INFO org.apache.hadoop.mapred.JobInProgress:
tip:task_201006090123_0067_m_01 has split on
node:/default-rack/10.13.53.128
--
2010-06-17 18:35:20,073 WARN org.apache.hadoop.conf.Configuration:
/var/lib/hadoop-0.20/cache/hadoop/mapred/local/jobTracker/job_2010060901
23_0068.xml:a attempt to override final parameter: dfs.data.dir;
Ignoring.
2010-06-17 18:35:20,132 DEBUG org.apache.hadoop.mapred.JobTracker:
Per-Task memory configuration is not set on JT. Not checking the job for
invalid memory requirements.
2010-06-17 18:35:20,133 INFO org.apache.hadoop.mapred.JobTracker:
Initializing job_201006090123_0068
2010-06-17 18:35:20,134 INFO org.apache.hadoop.mapred.JobInProgress:
Initializing job_201006090123_0068
2010-06-17 18:35:21,123 DEBUG org.apache.hadoop.mapred.JobTracker: Got
heartbeat from:
tracker_10.13.53.128:localhost.localdomain/127.0.0.1:35313 (restarted:
false initialContact: false acceptNewTasks: true) with responseId:
-32475
2010-06-17 18:35:21,186 DEBUG org.apache.hadoop.mapred.JobTracker: Got
heartbeat from:
tracker_10.13.53.127:localhost.localdomain/127.0.0.1:45581 (restarted:
false initialContact: false acceptNewTasks: true) with responseId:
-32074
2010-06-17 18:35:21,776 INFO org.apache.hadoop.mapred.JobInProgress:
Input size for job job_201006090123_0068 = 33327229. Number of splits =
288
2010-06-17 18:35:21,776 INFO org.apache.hadoop.mapred.JobInProgress:
tip:task_201006090123_0068_m_00 has split on
node:/default-rack/10.13.53.127
2010-06-17 18:35:21,776 INFO org.apache.hadoop.mapred.JobInProgress:
tip:task_201006090123_0068_m_00 has split on
node:/default-rack/10.13.53.156
2010-06-17 18:35:21,777 INFO org.apache.hadoop.mapred.JobInProgress:
tip:task_201006090123_0068_m_00 has split on
node:/default-rack/10.13.53.129
2010-06-17 18:35:21,777 INFO org.apache.hadoop.mapred.JobInProgress:
tip:task_201006090123_0068_m_01 has split on
node:/default-rack/10.13.53.127

 - datanode errors and warning -- 


Thanks & Regards

Bikash Singhal 



"Legal Disclaimer: This electronic message and all contents contain information 
from Cybage Software Private Limited which may be privileged, confidential, or 
otherwise protected from disclosure. The information is intended to be for the 
addressee(s) only. If you are not an addressee, any disclosure, copy, 
distribution, or use of the contents of this message is strictly prohibited. If 
you have received this electronic message in error please notify the sender by 
reply e-mail to and destroy the original message and all copies. Cybage has 
taken every reasonable precaution to minimize the risk of malicious content in 
the mail, but is not liable for any damage you may sustain as a result of any 
malicious content in this e-mail. You should carry out your own malicious 
content checks before opening the e-mail or attachment."
www.cybage.com