[jira] [Updated] (MAPREDUCE-1906) Lower minimum heartbeat interval for tasktracker Jobtracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated MAPREDUCE-1906: -- Attachment: MAPREDUCE-1906.branch-1.patch Lower minimum heartbeat interval for tasktracker Jobtracker - Key: MAPREDUCE-1906 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1906 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobtracker, performance, tasktracker Affects Versions: 0.20.1, 0.20.2 Reporter: Scott Carey Assignee: Todd Lipcon Fix For: 0.23.0 Attachments: MAPREDUCE-1906-0.21.patch, MAPREDUCE-1906.branch-1.patch, mapreduce-1906.txt, mapreduce-1906.txt I get a 0% to 15% performance increase for smaller clusters by making the heartbeat throttle stop penalizing clusters with less than 300 nodes. Between 0.19 and 0.20, the default minimum heartbeat interval increased from 2s to 3s. If a JobTracker is throttled at 100 heartbeats / sec for large clusters, why should a cluster with 10 nodes be throttled to 3.3 heartbeats per second? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-1906) Lower minimum heartbeat interval for tasktracker Jobtracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated MAPREDUCE-1906: -- Affects Version/s: 1.2.0 Lower minimum heartbeat interval for tasktracker Jobtracker - Key: MAPREDUCE-1906 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1906 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobtracker, performance, tasktracker Affects Versions: 0.20.1, 0.20.2, 1.2.0 Reporter: Scott Carey Assignee: Todd Lipcon Fix For: 0.23.0 Attachments: MAPREDUCE-1906-0.21.patch, MAPREDUCE-1906.branch-1.patch, mapreduce-1906.txt, mapreduce-1906.txt I get a 0% to 15% performance increase for smaller clusters by making the heartbeat throttle stop penalizing clusters with less than 300 nodes. Between 0.19 and 0.20, the default minimum heartbeat interval increased from 2s to 3s. If a JobTracker is throttled at 100 heartbeats / sec for large clusters, why should a cluster with 10 nodes be throttled to 3.3 heartbeats per second? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-1906) Lower minimum heartbeat interval for tasktracker Jobtracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated MAPREDUCE-1906: -- Fix Version/s: 1.2.0 Committed to branch-1. Lower minimum heartbeat interval for tasktracker Jobtracker - Key: MAPREDUCE-1906 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1906 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobtracker, performance, tasktracker Affects Versions: 0.20.1, 0.20.2, 1.2.0 Reporter: Scott Carey Assignee: Todd Lipcon Fix For: 0.23.0, 1.2.0 Attachments: MAPREDUCE-1906-0.21.patch, MAPREDUCE-1906.branch-1.patch, mapreduce-1906.txt, mapreduce-1906.txt I get a 0% to 15% performance increase for smaller clusters by making the heartbeat throttle stop penalizing clusters with less than 300 nodes. Between 0.19 and 0.20, the default minimum heartbeat interval increased from 2s to 3s. If a JobTracker is throttled at 100 heartbeats / sec for large clusters, why should a cluster with 10 nodes be throttled to 3.3 heartbeats per second? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-1906) Lower minimum heartbeat interval for tasktracker Jobtracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated MAPREDUCE-1906: --- Component/s: performance Lower minimum heartbeat interval for tasktracker Jobtracker - Key: MAPREDUCE-1906 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1906 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobtracker, performance, tasktracker Affects Versions: 0.20.1, 0.20.2 Reporter: Scott Carey Assignee: Todd Lipcon Fix For: 0.23.0 Attachments: MAPREDUCE-1906-0.21.patch, mapreduce-1906.txt, mapreduce-1906.txt I get a 0% to 15% performance increase for smaller clusters by making the heartbeat throttle stop penalizing clusters with less than 300 nodes. Between 0.19 and 0.20, the default minimum heartbeat interval increased from 2s to 3s. If a JobTracker is throttled at 100 heartbeats / sec for large clusters, why should a cluster with 10 nodes be throttled to 3.3 heartbeats per second? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MAPREDUCE-1906) Lower minimum heartbeat interval for tasktracker Jobtracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated MAPREDUCE-1906: --- Resolution: Fixed Fix Version/s: (was: 0.20.3) (was: 0.22.0) Release Note: The minimum heartbeat interval has been dropped from 3 seconds to 300ms to increase scheduling throughput on large clusters. Users may tune mapreduce.jobtracker.heartbeats.in.second to adjust this value. Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available) Committed to trunk only. Thanks for the original contribution and for your patience, Scott! Lower minimum heartbeat interval for tasktracker Jobtracker - Key: MAPREDUCE-1906 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1906 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobtracker Affects Versions: 0.20.1, 0.20.2 Reporter: Scott Carey Assignee: Todd Lipcon Fix For: 0.23.0 Attachments: MAPREDUCE-1906-0.21.patch, mapreduce-1906.txt, mapreduce-1906.txt I get a 0% to 15% performance increase for smaller clusters by making the heartbeat throttle stop penalizing clusters with less than 300 nodes. Between 0.19 and 0.20, the default minimum heartbeat interval increased from 2s to 3s. If a JobTracker is throttled at 100 heartbeats / sec for large clusters, why should a cluster with 10 nodes be throttled to 3.3 heartbeats per second? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1906) Lower minimum heartbeat interval for tasktracker Jobtracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated MAPREDUCE-1906: --- Component/s: tasktracker Release Note: The minimum heartbeat interval has been dropped from 3 seconds to 300ms to increase scheduling throughput on small clusters. Users may tune mapreduce.jobtracker.heartbeats.in.second to adjust this value. (was: The minimum heartbeat interval has been dropped from 3 seconds to 300ms to increase scheduling throughput on large clusters. Users may tune mapreduce.jobtracker.heartbeats.in.second to adjust this value.) Lower minimum heartbeat interval for tasktracker Jobtracker - Key: MAPREDUCE-1906 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1906 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobtracker, tasktracker Affects Versions: 0.20.1, 0.20.2 Reporter: Scott Carey Assignee: Todd Lipcon Fix For: 0.23.0 Attachments: MAPREDUCE-1906-0.21.patch, mapreduce-1906.txt, mapreduce-1906.txt I get a 0% to 15% performance increase for smaller clusters by making the heartbeat throttle stop penalizing clusters with less than 300 nodes. Between 0.19 and 0.20, the default minimum heartbeat interval increased from 2s to 3s. If a JobTracker is throttled at 100 heartbeats / sec for large clusters, why should a cluster with 10 nodes be throttled to 3.3 heartbeats per second? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1906) Lower minimum heartbeat interval for tasktracker Jobtracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated MAPREDUCE-1906: --- Fix Version/s: 0.23.0 0.22.0 0.20.3 Lower minimum heartbeat interval for tasktracker Jobtracker - Key: MAPREDUCE-1906 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1906 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 0.20.1, 0.20.2 Reporter: Scott Carey Assignee: Todd Lipcon Fix For: 0.20.3, 0.22.0, 0.23.0 Attachments: MAPREDUCE-1906-0.21.patch, mapreduce-1906.txt, mapreduce-1906.txt I get a 0% to 15% performance increase for smaller clusters by making the heartbeat throttle stop penalizing clusters with less than 300 nodes. Between 0.19 and 0.20, the default minimum heartbeat interval increased from 2s to 3s. If a JobTracker is throttled at 100 heartbeats / sec for large clusters, why should a cluster with 10 nodes be throttled to 3.3 heartbeats per second? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1906) Lower minimum heartbeat interval for tasktracker Jobtracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated MAPREDUCE-1906: --- Component/s: jobtracker Lower minimum heartbeat interval for tasktracker Jobtracker - Key: MAPREDUCE-1906 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1906 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobtracker Affects Versions: 0.20.1, 0.20.2 Reporter: Scott Carey Assignee: Todd Lipcon Fix For: 0.20.3, 0.22.0, 0.23.0 Attachments: MAPREDUCE-1906-0.21.patch, mapreduce-1906.txt, mapreduce-1906.txt I get a 0% to 15% performance increase for smaller clusters by making the heartbeat throttle stop penalizing clusters with less than 300 nodes. Between 0.19 and 0.20, the default minimum heartbeat interval increased from 2s to 3s. If a JobTracker is throttled at 100 heartbeats / sec for large clusters, why should a cluster with 10 nodes be throttled to 3.3 heartbeats per second? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1906) Lower minimum heartbeat interval for tasktracker Jobtracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated MAPREDUCE-1906: --- Attachment: mapreduce-1906.txt This is Scott's patch but also makes the minimum interval configurable. I set the default to 300ms as Scott suggests. Lower minimum heartbeat interval for tasktracker Jobtracker - Key: MAPREDUCE-1906 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1906 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 0.20.1, 0.20.2 Reporter: Scott Carey Attachments: MAPREDUCE-1906-0.21.patch, mapreduce-1906.txt I get a 0% to 15% performance increase for smaller clusters by making the heartbeat throttle stop penalizing clusters with less than 300 nodes. Between 0.19 and 0.20, the default minimum heartbeat interval increased from 2s to 3s. If a JobTracker is throttled at 100 heartbeats / sec for large clusters, why should a cluster with 10 nodes be throttled to 3.3 heartbeats per second? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1906) Lower minimum heartbeat interval for tasktracker Jobtracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated MAPREDUCE-1906: --- Attachment: mapreduce-1906.txt Had to update TestMapredHeartbeat to fix an assertion for the new minimum. Lower minimum heartbeat interval for tasktracker Jobtracker - Key: MAPREDUCE-1906 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1906 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 0.20.1, 0.20.2 Reporter: Scott Carey Attachments: MAPREDUCE-1906-0.21.patch, mapreduce-1906.txt, mapreduce-1906.txt I get a 0% to 15% performance increase for smaller clusters by making the heartbeat throttle stop penalizing clusters with less than 300 nodes. Between 0.19 and 0.20, the default minimum heartbeat interval increased from 2s to 3s. If a JobTracker is throttled at 100 heartbeats / sec for large clusters, why should a cluster with 10 nodes be throttled to 3.3 heartbeats per second? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1906) Lower minimum heartbeat interval for tasktracker Jobtracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Carey updated MAPREDUCE-1906: --- Attachment: MAPREDUCE-1906-0.21.patch replaced the original patch with the the latest. Lower minimum heartbeat interval for tasktracker Jobtracker - Key: MAPREDUCE-1906 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1906 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 0.20.1, 0.20.2 Reporter: Scott Carey Attachments: MAPREDUCE-1906-0.21.patch I get a 0% to 15% performance increase for smaller clusters by making the heartbeat throttle stop penalizing clusters with less than 300 nodes. Between 0.19 and 0.20, the default minimum heartbeat interval increased from 2s to 3s. If a JobTracker is throttled at 100 heartbeats / sec for large clusters, why should a cluster with 10 nodes be throttled to 3.3 heartbeats per second? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1906) Lower minimum heartbeat interval for tasktracker Jobtracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Carey updated MAPREDUCE-1906: --- Attachment: (was: MAPREDUCE-1906-0.21.patch) Lower minimum heartbeat interval for tasktracker Jobtracker - Key: MAPREDUCE-1906 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1906 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 0.20.1, 0.20.2 Reporter: Scott Carey Attachments: MAPREDUCE-1906-0.21-v2.patch I get a 0% to 15% performance increase for smaller clusters by making the heartbeat throttle stop penalizing clusters with less than 300 nodes. Between 0.19 and 0.20, the default minimum heartbeat interval increased from 2s to 3s. If a JobTracker is throttled at 100 heartbeats / sec for large clusters, why should a cluster with 10 nodes be throttled to 3.3 heartbeats per second? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1906) Lower minimum heartbeat interval for tasktracker Jobtracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Carey updated MAPREDUCE-1906: --- Status: Open (was: Patch Available) re-subit for hudson. Lower minimum heartbeat interval for tasktracker Jobtracker - Key: MAPREDUCE-1906 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1906 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 0.20.2, 0.20.1 Reporter: Scott Carey Attachments: MAPREDUCE-1906-0.21-v2.patch, MAPREDUCE-1906-0.21.patch I get a 0% to 15% performance increase for smaller clusters by making the heartbeat throttle stop penalizing clusters with less than 300 nodes. Between 0.19 and 0.20, the default minimum heartbeat interval increased from 2s to 3s. If a JobTracker is throttled at 100 heartbeats / sec for large clusters, why should a cluster with 10 nodes be throttled to 3.3 heartbeats per second? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1906) Lower minimum heartbeat interval for tasktracker Jobtracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Carey updated MAPREDUCE-1906: --- Status: Patch Available (was: Open) re-submit for hudson. Lower minimum heartbeat interval for tasktracker Jobtracker - Key: MAPREDUCE-1906 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1906 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 0.20.2, 0.20.1 Reporter: Scott Carey Attachments: MAPREDUCE-1906-0.21-v2.patch, MAPREDUCE-1906-0.21.patch I get a 0% to 15% performance increase for smaller clusters by making the heartbeat throttle stop penalizing clusters with less than 300 nodes. Between 0.19 and 0.20, the default minimum heartbeat interval increased from 2s to 3s. If a JobTracker is throttled at 100 heartbeats / sec for large clusters, why should a cluster with 10 nodes be throttled to 3.3 heartbeats per second? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1906) Lower minimum heartbeat interval for tasktracker Jobtracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Carey updated MAPREDUCE-1906: --- Status: Open (was: Patch Available) Lower minimum heartbeat interval for tasktracker Jobtracker - Key: MAPREDUCE-1906 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1906 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 0.20.2, 0.20.1 Reporter: Scott Carey Attachments: MAPREDUCE-1906-0.21.patch I get a 0% to 15% performance increase for smaller clusters by making the heartbeat throttle stop penalizing clusters with less than 300 nodes. Between 0.19 and 0.20, the default minimum heartbeat interval increased from 2s to 3s. If a JobTracker is throttled at 100 heartbeats / sec for large clusters, why should a cluster with 10 nodes be throttled to 3.3 heartbeats per second? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1906) Lower minimum heartbeat interval for tasktracker Jobtracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Carey updated MAPREDUCE-1906: --- Attachment: MAPREDUCE-1906-0.21-v2.patch Patch adds one line change to JobTracker.java to make the heartbeat interval a smooth function instead of a step function. Total patch is two one-line changes. Lower minimum heartbeat interval for tasktracker Jobtracker - Key: MAPREDUCE-1906 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1906 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 0.20.1, 0.20.2 Reporter: Scott Carey Attachments: MAPREDUCE-1906-0.21-v2.patch, MAPREDUCE-1906-0.21.patch I get a 0% to 15% performance increase for smaller clusters by making the heartbeat throttle stop penalizing clusters with less than 300 nodes. Between 0.19 and 0.20, the default minimum heartbeat interval increased from 2s to 3s. If a JobTracker is throttled at 100 heartbeats / sec for large clusters, why should a cluster with 10 nodes be throttled to 3.3 heartbeats per second? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1906) Lower minimum heartbeat interval for tasktracker Jobtracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Carey updated MAPREDUCE-1906: --- Status: Patch Available (was: Open) MAPREDUCE-1906-0.21-v2.patch changes ping to a smooth function from a step function and lowers the minimum to 300ms. Clusters larger than 300 nodes only see the step-function smooth function change. Clusters between 30 and 300 nodes smoothly increase their ping interval. Clusters with 30 nodes or less have 300ms ping intervals when the TT has nothing to do. This improves scheduling latency on small clusters significantly. The cluster wide ping interval is roughly proportional to how fast the cluster can schedule a job. || cluster size || current ping interval (ms) || current ping rate at JT || patched ping interval (ms) || patched ping rate at JT || | 10 | 3000 | 3.33 /sec | 300 | 33.3 /sec | | 30 | 3000 | 10 /sec | 300 | 100 /sec | | 100 | 3000 | 33.3 /sec | 1000 | 100 /sec | | 300 | 3000 | 100 /sec | 3000 | 100 /sec | | 301 | 4000 | 75 /sec | 3010 | 100 /sec | | 1000 | 1 | 100 /sec | 1 | 100 /sec | | 1001 | 11000 | 91 /sec | 10010 | 100 /sec | Lower minimum heartbeat interval for tasktracker Jobtracker - Key: MAPREDUCE-1906 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1906 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 0.20.2, 0.20.1 Reporter: Scott Carey Attachments: MAPREDUCE-1906-0.21-v2.patch, MAPREDUCE-1906-0.21.patch I get a 0% to 15% performance increase for smaller clusters by making the heartbeat throttle stop penalizing clusters with less than 300 nodes. Between 0.19 and 0.20, the default minimum heartbeat interval increased from 2s to 3s. If a JobTracker is throttled at 100 heartbeats / sec for large clusters, why should a cluster with 10 nodes be throttled to 3.3 heartbeats per second? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1906) Lower minimum heartbeat interval for tasktracker Jobtracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Carey updated MAPREDUCE-1906: --- Status: Patch Available (was: Open) Is it possible to consider this for 0.21? Lower minimum heartbeat interval for tasktracker Jobtracker - Key: MAPREDUCE-1906 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1906 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 0.20.2, 0.20.1 Reporter: Scott Carey Attachments: MAPREDUCE-1906-0.21.patch I get a 0% to 15% performance increase for smaller clusters by making the heartbeat throttle stop penalizing clusters with less than 300 nodes. Between 0.19 and 0.20, the default minimum heartbeat interval increased from 2s to 3s. If a JobTracker is throttled at 100 heartbeats / sec for large clusters, why should a cluster with 10 nodes be throttled to 3.3 heartbeats per second? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1906) Lower minimum heartbeat interval for tasktracker Jobtracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Carey updated MAPREDUCE-1906: --- Attachment: MAPREDUCE-1906-0.21.patch This patch changes the default minimum TaskTracker JobTracker heartbeat interval from 3000ms to 300ms. Effectively, this makes clusters between 30 and 300 nodes increase their heartbeat rate to a cluster-wide 100 heartbeats per second. Clusters larger than 300 nodes remain unchanged at a cluster-wide 100 heartbeats per second. Clusters with less than 30 nodes have a constant 300ms between pings per node. so for a 15 node cluster it is 50 heartbeats per second, and for a 3 node cluster it is 10 heartbeats per second. Lower minimum heartbeat interval for tasktracker Jobtracker - Key: MAPREDUCE-1906 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1906 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 0.20.1, 0.20.2 Reporter: Scott Carey Attachments: MAPREDUCE-1906-0.21.patch I get a 0% to 15% performance increase for smaller clusters by making the heartbeat throttle stop penalizing clusters with less than 300 nodes. Between 0.19 and 0.20, the default minimum heartbeat interval increased from 2s to 3s. If a JobTracker is throttled at 100 heartbeats / sec for large clusters, why should a cluster with 10 nodes be throttled to 3.3 heartbeats per second? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
