[
https://issues.apache.org/jira/browse/MAPREDUCE-90?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J resolved MAPREDUCE-90.
------------------------------
Resolution: Won't Fix
I think the resolution that was taken here was that we no longer
support/recommend the skip bad records in the framework. Users should implement
their own tech to handle bad records in their apps.
Given this, I am resolving this as Won't Fix.
> incrementing counters should not be used for triggering record skipping
> -----------------------------------------------------------------------
>
> Key: MAPREDUCE-90
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-90
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Owen O'Malley
>
> The following code is really problematic:
> {code}
> public void incrCounter(String group, String counter, long amount) {
> if (counters != null) {
> counters.incrCounter(group, counter, amount);
> }
> if(skipping && SkipBadRecords.COUNTER_GROUP.equals(group) && (
> SkipBadRecords.COUNTER_MAP_PROCESSED_RECORDS.equals(counter) ||
> SkipBadRecords.COUNTER_REDUCE_PROCESSED_GROUPS.equals(counter))) {
> //if application reports the processed records, move the
> //currentRecStartIndex to the next.
> //currentRecStartIndex is the start index which has not yet been
> //finished and is still in task's stomach.
> for(int i=0;i<amount;i++) {
> currentRecStartIndex = currentRecIndexIterator.next();
> }
> ...
> }
> {code}
> In particular, if the user updates a counter with the wrong name, bad things
> will presumably happen...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
