[jira] [Resolved] (MAPREDUCE-6016) hadoop yarn mapreduce skip failed records doesn't work
[ https://issues.apache.org/jira/browse/MAPREDUCE-6016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakub Stransky resolved MAPREDUCE-6016. --- Resolution: Duplicate Not sure what happen but issue report were duplicated MAPREDUCE-6017 > hadoop yarn mapreduce skip failed records doesn't work > -- > > Key: MAPREDUCE-6016 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6016 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.2.0 >Reporter: Jakub Stransky >Priority: Minor > > I am trying to use "skip failed records" map-reduce functionality during the > map phase. I created special testing file with 8 corrupted records. I am > using TextInputFormat and during the processing (of the record) map function > fails with unhandled exception (parsing the record into expected structure). > Job is using the old mapred api. > My job settings for enabling "skip failed records feature": > > mapred.skip.mode.enabled > true > > > mapreduce.map.maxattempts > 10 > > > mapreduce.task.skip.start.attempts > 1 > > > mapreduce.map.skip.maxrecords > 1 > > I verified that those properties are propagated via verification in job.xml. > I am using hadoop 2.2.0 (HDP 2.0). Job is still failing after 10 attempts. > UPDATE: > - obviously job is not entering skip record mode > Q: Does this feature works on RecordReader level only? Hadoop definite guide > (which is for v.1) descibes thais feature at the level of map/reduce funciton -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (MAPREDUCE-6016) hadoop yarn mapreduce skip failed records doesn't work
Jakub Stransky created MAPREDUCE-6016: - Summary: hadoop yarn mapreduce skip failed records doesn't work Key: MAPREDUCE-6016 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6016 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 2.2.0 Reporter: Jakub Stransky Priority: Minor I am trying to use "skip failed records" map-reduce functionality during the map phase. I created special testing file with 8 corrupted records. I am using TextInputFormat and during the processing (of the record) map function fails with unhandled exception (parsing the record into expected structure). Job is using the old mapred api. My job settings for enabling "skip failed records feature": mapred.skip.mode.enabled true mapreduce.map.maxattempts 10 mapreduce.task.skip.start.attempts 1 mapreduce.map.skip.maxrecords 1 I verified that those properties are propagated via verification in job.xml. I am using hadoop 2.2.0 (HDP 2.0). Job is still failing after 10 attempts. UPDATE: - obviously job is not entering skip record mode Q: Does this feature works on RecordReader level only? Hadoop definite guide (which is for v.1) descibes thais feature at the level of map/reduce funciton -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (MAPREDUCE-6017) hadoop yarn mapreduce skip failed records doesn't work
Jakub Stransky created MAPREDUCE-6017: - Summary: hadoop yarn mapreduce skip failed records doesn't work Key: MAPREDUCE-6017 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6017 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 2.2.0 Reporter: Jakub Stransky Priority: Minor I am trying to use "skip failed records" map-reduce functionality during the map phase. I created special testing file with 8 corrupted records. I am using TextInputFormat and during the processing (of the record) map function fails with unhandled exception (parsing the record into expected structure). Job is using the old mapred api. My job settings for enabling "skip failed records feature": mapred.skip.mode.enabled true mapreduce.map.maxattempts 10 mapreduce.task.skip.start.attempts 1 mapreduce.map.skip.maxrecords 1 I verified that those properties are propagated via verification in job.xml. I am using hadoop 2.2.0 (HDP 2.0). Job is still failing after 10 attempts. UPDATE: - obviously job is not entering skip record mode Q: Does this feature works on RecordReader level only? Hadoop definite guide (which is for v.1) descibes thais feature at the level of map/reduce funciton -- This message was sent by Atlassian JIRA (v6.2#6252)
