Hi Patai
JobTracker automatically handles this situation by attempting the task
on different nodes.Could you verify the number of attempts that these
failed tasks made. Was that just one? If more whether all the
task attempts were triggered on the same node or not? Did all of them fail
with the same error? You can get this information from the jobtracker web
UI, drill down to task level and then further down a failed task.
Regards
Bejoy
On Thu, Mar 22, 2012 at 11:25 PM, Patai Sangbutsarakum <
silvianhad...@gmail.com> wrote:
> Hi all,
>
> I have a job fail this morning because of 2 tasks were trying to write
> into disk that somehow turned read-only.
> Originally, i was thinking/dreaming that in this case somehow those 2
> tasks will be exported automatically
> to other dn/tt that also has the required data block, and won't fail.
>
> I strongly believe that Hadoop can do that but i just didn't know it
> well enough to enable it.
>
> /dev/sdj1 /hadoop10 ext3 ro,noatime,data=ordered 0 0
>
> Error initializing attempt_201203211854_2633_m_000017_0: EROFS:
> Read-only file system at
> org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method) at
>
> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:496)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:319)
> at
> org.apache.hadoop.mapred.JobLocalizer.createLocalDirs(JobLocalizer.java:144)
> at
> org.apache.hadoop.mapred.DefaultTaskController.initializeJob(DefaultTaskController.java:190)
> at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1199)
> at java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:396) at
>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
> at
> org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1174)
> at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1089)
> at org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:2257)
> at
> org.apache.hadoop.mapred.TaskTracker$TaskLauncher.run(TaskTracker.java:2221)
>
> Hope this make sense.
> Patai
>
