[jira] Created: (MAPREDUCE-2303) RAID BlockFixer should choose targets better
RAID BlockFixer should choose targets better Key: MAPREDUCE-2303 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2303 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/raid Reporter: Ramkumar Vadali Assignee: Ramkumar Vadali The RAID BlockFixer chooses the destination of the generated block at random. It avoids nodes that have a corrupt replica of the block, but does not do anything beyond that. It needs to avoid data nodes that have a replica of any source or parity block in the block's stripe. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (MAPREDUCE-2304) TestMRCLI fails when hostname has a hiphen (-)
TestMRCLI fails when hostname has a hiphen (-) -- Key: MAPREDUCE-2304 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2304 Project: Hadoop Map/Reduce Issue Type: Bug Components: test Affects Versions: 0.22.0 Environment: Linux Reporter: Priyo Mustafi Assignee: Priyo Mustafi Priority: Minor TestMRCLI fails with below Comparator: [RegexpComparator] Comparision result: [fail] Expected output: [mv: Wrong FS: har:/dest/dir0.har/dir0/file0, expected: hdfs://\w+[.a-z]*:[0-9]+] Actual output: [mv: Wrong FS: har:/dest/dir0.har/dir0/file0, expected: hdfs://lab-something.host.com:34039 -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MAPREDUCE-2304) TestMRCLI fails when hostname has a hiphen (-)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyo Mustafi updated MAPREDUCE-2304: - Attachment: MAPREDUCE-2304.patch The host can and have numeric. Interestingly the test works on windows without the patch TestMRCLI fails when hostname has a hiphen (-) -- Key: MAPREDUCE-2304 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2304 Project: Hadoop Map/Reduce Issue Type: Bug Components: test Affects Versions: 0.22.0 Environment: Linux Reporter: Priyo Mustafi Assignee: Priyo Mustafi Priority: Minor Attachments: MAPREDUCE-2304.patch TestMRCLI fails with below Comparator: [RegexpComparator] Comparision result: [fail] Expected output: [mv: Wrong FS: har:/dest/dir0.har/dir0/file0, expected: hdfs://\w+[.a-z]*:[0-9]+] Actual output: [mv: Wrong FS: har:/dest/dir0.har/dir0/file0, expected: hdfs://lab-something.host.com:34039 -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MAPREDUCE-2304) TestMRCLI fails when hostname has a hyphen (-)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated MAPREDUCE-2304: --- Summary: TestMRCLI fails when hostname has a hyphen (-) (was: TestMRCLI fails when hostname has a hiphen (-)) TestMRCLI fails when hostname has a hyphen (-) -- Key: MAPREDUCE-2304 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2304 Project: Hadoop Map/Reduce Issue Type: Bug Components: test Affects Versions: 0.22.0 Environment: Linux Reporter: Priyo Mustafi Assignee: Priyo Mustafi Priority: Minor Attachments: MAPREDUCE-2304.patch TestMRCLI fails with below Comparator: [RegexpComparator] Comparision result: [fail] Expected output: [mv: Wrong FS: har:/dest/dir0.har/dir0/file0, expected: hdfs://\w+[.a-z]*:[0-9]+] Actual output: [mv: Wrong FS: har:/dest/dir0.har/dir0/file0, expected: hdfs://lab-something.host.com:34039 -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MAPREDUCE-2301) TestDataDrivenDBInputFormat fails if port 9001 is occupied
[ https://issues.apache.org/jira/browse/MAPREDUCE-2301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated MAPREDUCE-2301: --- Attachment: mapreduce-2301.txt Talked to Aaron Kimball offline and he pointed out that HSQLDB can be run in an in-memory mode. So this new patch just uses that mode and doesn't need a port at all. TestDataDrivenDBInputFormat fails if port 9001 is occupied -- Key: MAPREDUCE-2301 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2301 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Attachments: mapreduce-2301.txt, mapreduce-2301.txt If you have any process listening on port 9001 (the default hsqldb port) then TestDataDrivenDBInputFormat will fail with: Connection is broken: java.io.EOFException java.sql.SQLException: Connection is broken: java.io.EOFException at org.hsqldb.jdbc.Util.sqlException(Unknown Source) in the logs you see: [Server@42704baa]: [Thread[HSQLDB Server @42704baa,5,main]]: run()/openServerSocket(): java.net.BindException: Address already in use -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-1915) IndexCache - getIndexInformation - check reduce index Out Of Bounds
[ https://issues.apache.org/jira/browse/MAPREDUCE-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990757#comment-12990757 ] Priyo Mustafi commented on MAPREDUCE-1915: -- Got a +1 on release audit as well. It was an environmental issue. Patch-test and test all passed now. IndexCache - getIndexInformation - check reduce index Out Of Bounds --- Key: MAPREDUCE-1915 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1915 Project: Hadoop Map/Reduce Issue Type: Bug Components: tasktracker Affects Versions: 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.22.0 Reporter: Rares Vernica Assignee: Priyo Mustafi Priority: Trivial Fix For: 0.22.0 Attachments: MAPREDUCE-1915.patch, MAPREDUCE-1915_1.patch When checking if the reduce index is out of bounds the check should be info.mapSpillRecord.size() = reduce instead of: info.mapSpillRecord.size() reduce Not a big bug since an Out Of Bounds is thrown downstream anyway. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MAPREDUCE-2301) TestDataDrivenDBInputFormat fails if port 9001 is occupied
[ https://issues.apache.org/jira/browse/MAPREDUCE-2301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated MAPREDUCE-2301: --- Attachment: mapreduce-2301.txt Oops, left some imports in the previous patch. This one should be good, sorry for noise. TestDataDrivenDBInputFormat fails if port 9001 is occupied -- Key: MAPREDUCE-2301 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2301 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Attachments: mapreduce-2301.txt, mapreduce-2301.txt, mapreduce-2301.txt If you have any process listening on port 9001 (the default hsqldb port) then TestDataDrivenDBInputFormat will fail with: Connection is broken: java.io.EOFException java.sql.SQLException: Connection is broken: java.io.EOFException at org.hsqldb.jdbc.Util.sqlException(Unknown Source) in the logs you see: [Server@42704baa]: [Thread[HSQLDB Server @42704baa,5,main]]: run()/openServerSocket(): java.net.BindException: Address already in use -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2301) TestDataDrivenDBInputFormat fails if port 9001 is occupied
[
https://issues.apache.org/jira/browse/MAPREDUCE-2301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990769#comment-12990769
]
Aaron Kimball commented on MAPREDUCE-2301:
--
+1 Changes look reasonable.
FWIW, I don't think this is going to entirely solve your problem. {{TestDBJob}}
should fail for the exact same reason -- it runs the job specified in
{{DBCountPageView.java}} under {{src/examples}}. So you might need to
parameterize the {{DB_URL}} in that process as well and make embedded server
hosting conditional on its value. (If TestDDDBIF fails and TestDBJob doesn't...
that's something else worth pondering.)
TestDataDrivenDBInputFormat fails if port 9001 is occupied
--
Key: MAPREDUCE-2301
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2301
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Minor
Attachments: mapreduce-2301.txt, mapreduce-2301.txt,
mapreduce-2301.txt
If you have any process listening on port 9001 (the default hsqldb port) then
TestDataDrivenDBInputFormat will fail with:
Connection is broken: java.io.EOFException
java.sql.SQLException: Connection is broken: java.io.EOFException
at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
in the logs you see:
[Server@42704baa]: [Thread[HSQLDB Server @42704baa,5,main]]:
run()/openServerSocket():
java.net.BindException: Address already in use
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-1974) FairScheduler can preempt the same task many times
[ https://issues.apache.org/jira/browse/MAPREDUCE-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990775#comment-12990775 ] Priyo Mustafi commented on MAPREDUCE-1974: -- I have tested this as well and all looks good FairScheduler can preempt the same task many times -- Key: MAPREDUCE-1974 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1974 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/fair-share Affects Versions: 0.21.0, 0.21.1, 0.22.0, 0.23.0 Reporter: Scott Chen Assignee: Scott Chen Attachments: MAPREDUCE-1974.txt In FairScheduler.preemptTasks(), tasks are collected from JobInProgress.runningMapCache. But tasks repeat multiple times in JobInProgress.runningMapCache (on rack, node and cluster). This makes FairScheduler preempt the same task many times. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MAPREDUCE-1915) IndexCache - getIndexInformation - check reduce index Out Of Bounds
[ https://issues.apache.org/jira/browse/MAPREDUCE-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated MAPREDUCE-1915: --- Resolution: Fixed Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available) I just committed this. Thank you Priyo. IndexCache - getIndexInformation - check reduce index Out Of Bounds --- Key: MAPREDUCE-1915 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1915 Project: Hadoop Map/Reduce Issue Type: Bug Components: tasktracker Affects Versions: 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.22.0 Reporter: Rares Vernica Assignee: Priyo Mustafi Priority: Trivial Fix For: 0.22.0 Attachments: MAPREDUCE-1915.patch, MAPREDUCE-1915_1.patch When checking if the reduce index is out of bounds the check should be info.mapSpillRecord.size() = reduce instead of: info.mapSpillRecord.size() reduce Not a big bug since an Out Of Bounds is thrown downstream anyway. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-1915) IndexCache - getIndexInformation - check reduce index Out Of Bounds
[ https://issues.apache.org/jira/browse/MAPREDUCE-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990782#comment-12990782 ] Hudson commented on MAPREDUCE-1915: --- Integrated in Hadoop-Mapreduce-trunk-Commit #602 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/602/]) MAPREDUCE-1915. Fix IndexOutOfBoundsException in IndexCache. Contributed by Priyo Mustafi. IndexCache - getIndexInformation - check reduce index Out Of Bounds --- Key: MAPREDUCE-1915 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1915 Project: Hadoop Map/Reduce Issue Type: Bug Components: tasktracker Affects Versions: 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.22.0 Reporter: Rares Vernica Assignee: Priyo Mustafi Priority: Trivial Fix For: 0.22.0 Attachments: MAPREDUCE-1915.patch, MAPREDUCE-1915_1.patch When checking if the reduce index is out of bounds the check should be info.mapSpillRecord.size() = reduce instead of: info.mapSpillRecord.size() reduce Not a big bug since an Out Of Bounds is thrown downstream anyway. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-1915) IndexCache - getIndexInformation - check reduce index Out Of Bounds
[ https://issues.apache.org/jira/browse/MAPREDUCE-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990785#comment-12990785 ] Rares Vernica commented on MAPREDUCE-1915: -- Priyo and Konstantin, thanks for taking care of this! IndexCache - getIndexInformation - check reduce index Out Of Bounds --- Key: MAPREDUCE-1915 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1915 Project: Hadoop Map/Reduce Issue Type: Bug Components: tasktracker Affects Versions: 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.22.0 Reporter: Rares Vernica Assignee: Priyo Mustafi Priority: Trivial Fix For: 0.22.0 Attachments: MAPREDUCE-1915.patch, MAPREDUCE-1915_1.patch When checking if the reduce index is out of bounds the check should be info.mapSpillRecord.size() = reduce instead of: info.mapSpillRecord.size() reduce Not a big bug since an Out Of Bounds is thrown downstream anyway. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2304) TestMRCLI fails when hostname has a hyphen (-)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990789#comment-12990789 ] Konstantin Shvachko commented on MAPREDUCE-2304: It probably has to do something with configuration files on this machine you ran into the issue. Because if you run tests with the default configuration only, then you should just have localhost or 0.0.0.0 as a hostname. But it's a good catch, and extends the applicability of the test. +1 TestMRCLI fails when hostname has a hyphen (-) -- Key: MAPREDUCE-2304 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2304 Project: Hadoop Map/Reduce Issue Type: Bug Components: test Affects Versions: 0.22.0 Environment: Linux Reporter: Priyo Mustafi Assignee: Priyo Mustafi Priority: Minor Attachments: MAPREDUCE-2304.patch TestMRCLI fails with below Comparator: [RegexpComparator] Comparision result: [fail] Expected output: [mv: Wrong FS: har:/dest/dir0.har/dir0/file0, expected: hdfs://\w+[.a-z]*:[0-9]+] Actual output: [mv: Wrong FS: har:/dest/dir0.har/dir0/file0, expected: hdfs://lab-something.host.com:34039 -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MAPREDUCE-1974) FairScheduler can preempt the same task many times
[ https://issues.apache.org/jira/browse/MAPREDUCE-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated MAPREDUCE-1974: --- Resolution: Fixed Fix Version/s: 0.22.0 Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available) I just committed this. Thank you Scott. FairScheduler can preempt the same task many times -- Key: MAPREDUCE-1974 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1974 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/fair-share Affects Versions: 0.21.0, 0.21.1, 0.22.0, 0.23.0 Reporter: Scott Chen Assignee: Scott Chen Fix For: 0.22.0 Attachments: MAPREDUCE-1974.txt In FairScheduler.preemptTasks(), tasks are collected from JobInProgress.runningMapCache. But tasks repeat multiple times in JobInProgress.runningMapCache (on rack, node and cluster). This makes FairScheduler preempt the same task many times. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-1974) FairScheduler can preempt the same task many times
[ https://issues.apache.org/jira/browse/MAPREDUCE-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990827#comment-12990827 ] Hudson commented on MAPREDUCE-1974: --- Integrated in Hadoop-Mapreduce-trunk-Commit #603 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/603/]) MAPREDUCE-1974. Fix multiple preemtions of the same task in FairScheduler. Contributed by Scott Chen. FairScheduler can preempt the same task many times -- Key: MAPREDUCE-1974 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1974 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/fair-share Affects Versions: 0.21.0, 0.21.1, 0.22.0, 0.23.0 Reporter: Scott Chen Assignee: Scott Chen Fix For: 0.22.0 Attachments: MAPREDUCE-1974.txt In FairScheduler.preemptTasks(), tasks are collected from JobInProgress.runningMapCache. But tasks repeat multiple times in JobInProgress.runningMapCache (on rack, node and cluster). This makes FairScheduler preempt the same task many times. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (MAPREDUCE-2304) TestMRCLI fails when hostname has a hyphen (-)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved MAPREDUCE-2304. Resolution: Fixed Fix Version/s: 0.22.0 Hadoop Flags: [Reviewed] I just committed this. Thank you Priyo. TestMRCLI fails when hostname has a hyphen (-) -- Key: MAPREDUCE-2304 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2304 Project: Hadoop Map/Reduce Issue Type: Bug Components: test Affects Versions: 0.22.0 Environment: Linux Reporter: Priyo Mustafi Assignee: Priyo Mustafi Priority: Minor Fix For: 0.22.0 Attachments: MAPREDUCE-2304.patch TestMRCLI fails with below Comparator: [RegexpComparator] Comparision result: [fail] Expected output: [mv: Wrong FS: har:/dest/dir0.har/dir0/file0, expected: hdfs://\w+[.a-z]*:[0-9]+] Actual output: [mv: Wrong FS: har:/dest/dir0.har/dir0/file0, expected: hdfs://lab-something.host.com:34039 -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2304) TestMRCLI fails when hostname has a hyphen (-)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990840#comment-12990840 ] Hudson commented on MAPREDUCE-2304: --- Integrated in Hadoop-Mapreduce-trunk-Commit #604 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/604/]) MAPREDUCE-2304. Fix TestMRCLI to allow hostname with a hyphen (-). Contributed by Priyo Mustafi. TestMRCLI fails when hostname has a hyphen (-) -- Key: MAPREDUCE-2304 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2304 Project: Hadoop Map/Reduce Issue Type: Bug Components: test Affects Versions: 0.22.0 Environment: Linux Reporter: Priyo Mustafi Assignee: Priyo Mustafi Priority: Minor Fix For: 0.22.0 Attachments: MAPREDUCE-2304.patch TestMRCLI fails with below Comparator: [RegexpComparator] Comparision result: [fail] Expected output: [mv: Wrong FS: har:/dest/dir0.har/dir0/file0, expected: hdfs://\w+[.a-z]*:[0-9]+] Actual output: [mv: Wrong FS: har:/dest/dir0.har/dir0/file0, expected: hdfs://lab-something.host.com:34039 -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-1825) jobqueue_details.jsp and FairSchedulerServelet should not call finishedMaps and finishedReduces when job is not initialized
[ https://issues.apache.org/jira/browse/MAPREDUCE-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990847#comment-12990847 ] Priyo Mustafi commented on MAPREDUCE-1825: -- Patch was manually tested. Ant test passed. Looks good. +1 jobqueue_details.jsp and FairSchedulerServelet should not call finishedMaps and finishedReduces when job is not initialized --- Key: MAPREDUCE-1825 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1825 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobtracker Affects Versions: 0.20.1 Reporter: Amareshwari Sriramadasu Assignee: Scott Chen Fix For: 0.22.0 Attachments: MAPREDUCE-1825.txt, MAPREDUCE-1825_1.txt, MAPREDUCE-1825_2.txt, MAPREDUCE-1825_3.txt, jstacks.zip JobInProgress.finishedMaps() and finishedReduces() are synchronized. They are called from jobqueue_details.jsp and FairSchedulerServelet which iterates through all jobs. If any job is in initialization, these pages don't come up until the initialization finishes. See [comment|https://issues.apache.org/jira/browse/MAPREDUCE-1354?focusedCommentId=12834139page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12834139] for more details -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MAPREDUCE-1825) jobqueue_details.jsp and FairSchedulerServelet should not call finishedMaps and finishedReduces when job is not initialized
[ https://issues.apache.org/jira/browse/MAPREDUCE-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated MAPREDUCE-1825: --- Resolution: Fixed Status: Resolved (was: Patch Available) I just committed this. Thank you Scott and Priyo. jobqueue_details.jsp and FairSchedulerServelet should not call finishedMaps and finishedReduces when job is not initialized --- Key: MAPREDUCE-1825 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1825 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobtracker Affects Versions: 0.20.1 Reporter: Amareshwari Sriramadasu Assignee: Scott Chen Fix For: 0.22.0 Attachments: MAPREDUCE-1825.txt, MAPREDUCE-1825_1.txt, MAPREDUCE-1825_2.txt, MAPREDUCE-1825_3.txt, jstacks.zip JobInProgress.finishedMaps() and finishedReduces() are synchronized. They are called from jobqueue_details.jsp and FairSchedulerServelet which iterates through all jobs. If any job is in initialization, these pages don't come up until the initialization finishes. See [comment|https://issues.apache.org/jira/browse/MAPREDUCE-1354?focusedCommentId=12834139page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12834139] for more details -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-1825) jobqueue_details.jsp and FairSchedulerServelet should not call finishedMaps and finishedReduces when job is not initialized
[ https://issues.apache.org/jira/browse/MAPREDUCE-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990857#comment-12990857 ] Hudson commented on MAPREDUCE-1825: --- Integrated in Hadoop-Mapreduce-trunk-Commit #605 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/605/]) MAPREDUCE-1825. jobqueue_details.jsp and FairSchedulerServelet should not call finishedMaps and finishedReduces when job is not initialized. Contributed by Scott Chen. jobqueue_details.jsp and FairSchedulerServelet should not call finishedMaps and finishedReduces when job is not initialized --- Key: MAPREDUCE-1825 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1825 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobtracker Affects Versions: 0.20.1 Reporter: Amareshwari Sriramadasu Assignee: Scott Chen Fix For: 0.22.0 Attachments: MAPREDUCE-1825.txt, MAPREDUCE-1825_1.txt, MAPREDUCE-1825_2.txt, MAPREDUCE-1825_3.txt, jstacks.zip JobInProgress.finishedMaps() and finishedReduces() are synchronized. They are called from jobqueue_details.jsp and FairSchedulerServelet which iterates through all jobs. If any job is in initialization, these pages don't come up until the initialization finishes. See [comment|https://issues.apache.org/jira/browse/MAPREDUCE-1354?focusedCommentId=12834139page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12834139] for more details -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-1825) jobqueue_details.jsp and FairSchedulerServelet should not call finishedMaps and finishedReduces when job is not initialized
[ https://issues.apache.org/jira/browse/MAPREDUCE-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990866#comment-12990866 ] Hudson commented on MAPREDUCE-1825: --- Integrated in Hadoop-Mapreduce-trunk-Commit #606 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/606/]) MAPREDUCE-1825. A dot CHANGES.txt. jobqueue_details.jsp and FairSchedulerServelet should not call finishedMaps and finishedReduces when job is not initialized --- Key: MAPREDUCE-1825 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1825 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobtracker Affects Versions: 0.20.1 Reporter: Amareshwari Sriramadasu Assignee: Scott Chen Fix For: 0.22.0 Attachments: MAPREDUCE-1825.txt, MAPREDUCE-1825_1.txt, MAPREDUCE-1825_2.txt, MAPREDUCE-1825_3.txt, jstacks.zip JobInProgress.finishedMaps() and finishedReduces() are synchronized. They are called from jobqueue_details.jsp and FairSchedulerServelet which iterates through all jobs. If any job is in initialization, these pages don't come up until the initialization finishes. See [comment|https://issues.apache.org/jira/browse/MAPREDUCE-1354?focusedCommentId=12834139page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12834139] for more details -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-1825) jobqueue_details.jsp and FairSchedulerServelet should not call finishedMaps and finishedReduces when job is not initialized
[ https://issues.apache.org/jira/browse/MAPREDUCE-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990871#comment-12990871 ] Scott Chen commented on MAPREDUCE-1825: --- Thanks for the help, Konstantin and Priyo. jobqueue_details.jsp and FairSchedulerServelet should not call finishedMaps and finishedReduces when job is not initialized --- Key: MAPREDUCE-1825 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1825 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobtracker Affects Versions: 0.20.1 Reporter: Amareshwari Sriramadasu Assignee: Scott Chen Fix For: 0.22.0 Attachments: MAPREDUCE-1825.txt, MAPREDUCE-1825_1.txt, MAPREDUCE-1825_2.txt, MAPREDUCE-1825_3.txt, jstacks.zip JobInProgress.finishedMaps() and finishedReduces() are synchronized. They are called from jobqueue_details.jsp and FairSchedulerServelet which iterates through all jobs. If any job is in initialization, these pages don't come up until the initialization finishes. See [comment|https://issues.apache.org/jira/browse/MAPREDUCE-1354?focusedCommentId=12834139page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12834139] for more details -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-1974) FairScheduler can preempt the same task many times
[ https://issues.apache.org/jira/browse/MAPREDUCE-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990874#comment-12990874 ] Scott Chen commented on MAPREDUCE-1974: --- Thanks for the help, guys :) FairScheduler can preempt the same task many times -- Key: MAPREDUCE-1974 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1974 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/fair-share Affects Versions: 0.21.0, 0.21.1, 0.22.0, 0.23.0 Reporter: Scott Chen Assignee: Scott Chen Fix For: 0.22.0 Attachments: MAPREDUCE-1974.txt In FairScheduler.preemptTasks(), tasks are collected from JobInProgress.runningMapCache. But tasks repeat multiple times in JobInProgress.runningMapCache (on rack, node and cluster). This makes FairScheduler preempt the same task many times. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MAPREDUCE-2302) Add static factory methods in GaloisField
[ https://issues.apache.org/jira/browse/MAPREDUCE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Chen updated MAPREDUCE-2302: -- Attachment: MAPREDUCE-2302.2.txt Addressed some offline comments from Ram Add static factory methods in GaloisField - Key: MAPREDUCE-2302 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2302 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/raid Reporter: Scott Chen Assignee: Scott Chen Attachments: MAPREDUCE-2302.2.txt, MAPREDUCE-2302.txt GaloisField is immutable and should be kept reuse after creation to avoid redundant calculation of the multiplication and division tables. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
