[jira] Commented: (MAPREDUCE-1915) IndexCache - getIndexInformation - check reduce index Out Of Bounds
[ https://issues.apache.org/jira/browse/MAPREDUCE-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990785#comment-12990785 ] Rares Vernica commented on MAPREDUCE-1915: -- Priyo and Konstantin, thanks for taking care of this! IndexCache - getIndexInformation - check reduce index Out Of Bounds --- Key: MAPREDUCE-1915 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1915 Project: Hadoop Map/Reduce Issue Type: Bug Components: tasktracker Affects Versions: 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.22.0 Reporter: Rares Vernica Assignee: Priyo Mustafi Priority: Trivial Fix For: 0.22.0 Attachments: MAPREDUCE-1915.patch, MAPREDUCE-1915_1.patch When checking if the reduce index is out of bounds the check should be info.mapSpillRecord.size() = reduce instead of: info.mapSpillRecord.size() reduce Not a big bug since an Out Of Bounds is thrown downstream anyway. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (MAPREDUCE-2004) IP address vs host name in updating Counter.DATA_LOCAL_MAPS
IP address vs host name in updating Counter.DATA_LOCAL_MAPS --- Key: MAPREDUCE-2004 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2004 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobtracker Affects Versions: 0.20.2 Reporter: Rares Vernica Priority: Minor Hello, I set mapred.task.cache.levels to 1 so that I have only data-local-map tasks. Still, by looking the the data-local-maps counter it seems not all map tasks are local. I checked each map task to see where it run and what split has been assigned to it and all the maps were actually processing only local data. (BTW, replication was set to 1.) I looked into the JobClient so see what information is there for each split. For each file, the first n-1 splits have an IP address as location while the n-th split has a host name as location. The reason for this is that there is a different code path in deciding the location for the first n-1 splits versus the n-th split. The maps that processed the splits where the location was a host name were counted as data-local-maps while the others were not. So, regardless of the fact that the JobClient gives IP or host names for splits the job works fine. The problem is that the data-local-maps counter does not take this into consideration. Cheers, Rares -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1915) IndexCache - getIndexInformation - check reduce index Out Of Bounds
[ https://issues.apache.org/jira/browse/MAPREDUCE-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rares Vernica updated MAPREDUCE-1915: - Attachment: MAPREDUCE-1915.patch IndexCache - getIndexInformation - check reduce index Out Of Bounds --- Key: MAPREDUCE-1915 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1915 Project: Hadoop Map/Reduce Issue Type: Bug Components: tasktracker Affects Versions: 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.22.0 Reporter: Rares Vernica Priority: Trivial Fix For: 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.22.0 Attachments: MAPREDUCE-1915.patch When checking if the reduce index is out of bounds the check should be info.mapSpillRecord.size() = reduce instead of: info.mapSpillRecord.size() reduce Not a big bug since an Out Of Bounds is thrown downstream anyway. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (MAPREDUCE-1915) IndexCache - getIndexInformation - check reduce index Out Of Bounds
IndexCache - getIndexInformation - check reduce index Out Of Bounds --- Key: MAPREDUCE-1915 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1915 Project: Hadoop Map/Reduce Issue Type: Bug Components: tasktracker Affects Versions: 0.20.2 Reporter: Rares Vernica Priority: Trivial When checking if the reduce index is out of bounds the check should be info.mapSpillRecord.size() = reduce instead of: info.mapSpillRecord.size() reduce Not a big bug since an Out Of Bounds is thrown downstream anyway. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1915) IndexCache - getIndexInformation - check reduce index Out Of Bounds
[ https://issues.apache.org/jira/browse/MAPREDUCE-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rares Vernica updated MAPREDUCE-1915: - Status: Patch Available (was: Open) Affects Version/s: 0.20.1 0.20.3 0.21.0 0.22.0 Fix Version/s: 0.20.3 0.21.0 0.22.0 0.20.2 0.20.1 IndexCache - getIndexInformation - check reduce index Out Of Bounds --- Key: MAPREDUCE-1915 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1915 Project: Hadoop Map/Reduce Issue Type: Bug Components: tasktracker Affects Versions: 0.20.2, 0.20.1, 0.20.3, 0.21.0, 0.22.0 Reporter: Rares Vernica Priority: Trivial Fix For: 0.20.3, 0.21.0, 0.22.0, 0.20.2, 0.20.1 Attachments: MAPREDUCE-1915.patch When checking if the reduce index is out of bounds the check should be info.mapSpillRecord.size() = reduce instead of: info.mapSpillRecord.size() reduce Not a big bug since an Out Of Bounds is thrown downstream anyway. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
