[jira] [Updated] (HBASE-4391) Add ability to start RS as root and call mlockall
[ https://issues.apache.org/jira/browse/HBASE-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-4391: - Release Note: A new profile, native, has been added to pom.xml Adding '-Pnative' will build libmlockall_agent.so HBase will look for the .so under ./lib/native/PLATFORM (See JAVA_LIBRARY_PATH in the bin/hbase starup script). mlockall_agent is a simple VM Agent that allows to lock the address space of the region server process. This avoids the process' memory eviction under pressure. You can load the agent by adding it as a jvm option: export HBASE_REGIONSERVER_OPTS="-agentpath:./libmlockall_agent.so=user=hbase" To use this feature, execute the following command as root: hbase --mlock user=hbase regionserver start was: A new profile, native, has been added to pom.xml Adding '-Pnative' will build libmlockall_agent.so mlockall_agent is a simple VM Agent that allows to lock the address space of the region server process. This avoids the process' memory eviction under pressure. You can load the agent by adding it as a jvm option: export HBASE_REGIONSERVER_OPTS="-agentpath:./libmlockall_agent.so=user=hbase" To use this feature, execute the following command as root: hbase --mlock user=hbase regionserver start > Add ability to start RS as root and call mlockall > - > > Key: HBASE-4391 > URL: https://issues.apache.org/jira/browse/HBASE-4391 > Project: HBase > Issue Type: New Feature > Components: regionserver >Affects Versions: 0.94.0 >Reporter: Todd Lipcon >Assignee: Matteo Bertozzi > Fix For: 0.95.0 > > Attachments: 4391-v4.patch, HBASE-4391-v0.patch, HBASE-4391-v1.patch, > HBASE-4391-v2.patch, HBASE-4391-v3.patch > > > A common issue we've seen in practice is that users oversubscribe their > region servers with too many MR tasks, etc. As soon as the machine starts > swapping, the RS grinds to a halt, loses ZK session, aborts, etc. > This can be combatted by starting the RS as root, calling mlockall(), and > then setuid down to the hbase user. We should not require this, but we should > provide it as an option. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4391) Add ability to start RS as root and call mlockall
[ https://issues.apache.org/jira/browse/HBASE-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-4391: - Release Note: A new profile, native, has been added to pom.xml Adding '-Pnative' will build libmlockall_agent.so mlockall_agent is a simple VM Agent that allows to lock the address space of the region server process. This avoids the process' memory eviction under pressure. You can load the agent by adding it as a jvm option: export HBASE_REGIONSERVER_OPTS="-agentpath:./libmlockall_agent.so=user=hbase" To use this feature, execute the following command as root: hbase --mlock user=hbase regionserver start was: A new profile, native, has been added to pom.xml It would build libmlockall_agent.so mlockall_agent is a simple VM Agent that allows to lock the address space of the region server process. This avoids the process' memory eviction under pressure. You can load the agent by adding it as a jvm option: export HBASE_REGIONSERVER_OPTS="-agentpath:./libmlockall_agent.so=user=hbase" To use this feature, execute the following command as root: hbase --mlock user=hbase regionserver start > Add ability to start RS as root and call mlockall > - > > Key: HBASE-4391 > URL: https://issues.apache.org/jira/browse/HBASE-4391 > Project: HBase > Issue Type: New Feature > Components: regionserver >Affects Versions: 0.94.0 >Reporter: Todd Lipcon >Assignee: Matteo Bertozzi > Fix For: 0.95.0 > > Attachments: 4391-v4.patch, HBASE-4391-v0.patch, HBASE-4391-v1.patch, > HBASE-4391-v2.patch, HBASE-4391-v3.patch > > > A common issue we've seen in practice is that users oversubscribe their > region servers with too many MR tasks, etc. As soon as the machine starts > swapping, the RS grinds to a halt, loses ZK session, aborts, etc. > This can be combatted by starting the RS as root, calling mlockall(), and > then setuid down to the hbase user. We should not require this, but we should > provide it as an option. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4391) Add ability to start RS as root and call mlockall
[ https://issues.apache.org/jira/browse/HBASE-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Ted Yu updated HBASE-4391: -- Attachment: 4391-v4.patch Patch v4 corrects some grammatical errors. > Add ability to start RS as root and call mlockall > - > > Key: HBASE-4391 > URL: https://issues.apache.org/jira/browse/HBASE-4391 > Project: HBase > Issue Type: New Feature > Components: regionserver >Affects Versions: 0.94.0 >Reporter: Todd Lipcon >Assignee: Matteo Bertozzi > Fix For: 0.96.0 > > Attachments: 4391-v4.patch, HBASE-4391-v0.patch, HBASE-4391-v1.patch, > HBASE-4391-v2.patch, HBASE-4391-v3.patch > > > A common issue we've seen in practice is that users oversubscribe their > region servers with too many MR tasks, etc. As soon as the machine starts > swapping, the RS grinds to a halt, loses ZK session, aborts, etc. > This can be combatted by starting the RS as root, calling mlockall(), and > then setuid down to the hbase user. We should not require this, but we should > provide it as an option. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4391) Add ability to start RS as root and call mlockall
[ https://issues.apache.org/jira/browse/HBASE-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Ted Yu updated HBASE-4391: -- Status: Open (was: Patch Available) > Add ability to start RS as root and call mlockall > - > > Key: HBASE-4391 > URL: https://issues.apache.org/jira/browse/HBASE-4391 > Project: HBase > Issue Type: New Feature > Components: regionserver >Affects Versions: 0.94.0 >Reporter: Todd Lipcon >Assignee: Matteo Bertozzi > Fix For: 0.96.0 > > Attachments: HBASE-4391-v0.patch, HBASE-4391-v1.patch, > HBASE-4391-v2.patch, HBASE-4391-v3.patch > > > A common issue we've seen in practice is that users oversubscribe their > region servers with too many MR tasks, etc. As soon as the machine starts > swapping, the RS grinds to a halt, loses ZK session, aborts, etc. > This can be combatted by starting the RS as root, calling mlockall(), and > then setuid down to the hbase user. We should not require this, but we should > provide it as an option. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4391) Add ability to start RS as root and call mlockall
[ https://issues.apache.org/jira/browse/HBASE-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Ted Yu updated HBASE-4391: -- Release Note: A new profile, native, has been added to pom.xml It would build libmlockall_agent.so mlockall_agent is a simple VM Agent that allows to lock the address space of the region server process. This avoids the process' memory eviction under pressure. You can load the agent by adding it as a jvm option: export HBASE_REGIONSERVER_OPTS="-agentpath:./libmlockall_agent.so=user=hbase" To use this feature, execute the following command as root: hbase --mlock user=hbase regionserver start was: A new profile, native, has been added to pom.xml It would build libmlockall_agent.so You can load the agent by adding it as a jvm option: export HBASE_REGIONSERVER_OPTS="-agentpath:./libmlockall_agent.so=user=hbase" To use this feature, execute the following command as root: hbase --mlock user=hbase regionserver start > Add ability to start RS as root and call mlockall > - > > Key: HBASE-4391 > URL: https://issues.apache.org/jira/browse/HBASE-4391 > Project: HBase > Issue Type: New Feature > Components: regionserver >Affects Versions: 0.94.0 >Reporter: Todd Lipcon >Assignee: Matteo Bertozzi > Fix For: 0.96.0 > > Attachments: HBASE-4391-v0.patch, HBASE-4391-v1.patch, > HBASE-4391-v2.patch, HBASE-4391-v3.patch > > > A common issue we've seen in practice is that users oversubscribe their > region servers with too many MR tasks, etc. As soon as the machine starts > swapping, the RS grinds to a halt, loses ZK session, aborts, etc. > This can be combatted by starting the RS as root, calling mlockall(), and > then setuid down to the hbase user. We should not require this, but we should > provide it as an option. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4391) Add ability to start RS as root and call mlockall
[ https://issues.apache.org/jira/browse/HBASE-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Ted Yu updated HBASE-4391: -- Release Note: A new profile, native, has been added to pom.xml It would build libmlockall_agent.so You can load the agent by adding it as a jvm option: export HBASE_REGIONSERVER_OPTS="-agentpath:./libmlockall_agent.so=user=hbase" To use this feature, execute the following command as root: hbase --mlock user=hbase regionserver start was: A new profile, native, has been added to pom.xml It would build libmlockall_agent.so You can load the agent by adding it as a jvm option: export HBASE_REGIONSERVER_OPTS="-agentpath:./libmlockall_agent.so=user=hbase" > Add ability to start RS as root and call mlockall > - > > Key: HBASE-4391 > URL: https://issues.apache.org/jira/browse/HBASE-4391 > Project: HBase > Issue Type: New Feature > Components: regionserver >Affects Versions: 0.94.0 >Reporter: Todd Lipcon >Assignee: Matteo Bertozzi > Fix For: 0.96.0 > > Attachments: HBASE-4391-v0.patch, HBASE-4391-v1.patch, > HBASE-4391-v2.patch, HBASE-4391-v3.patch > > > A common issue we've seen in practice is that users oversubscribe their > region servers with too many MR tasks, etc. As soon as the machine starts > swapping, the RS grinds to a halt, loses ZK session, aborts, etc. > This can be combatted by starting the RS as root, calling mlockall(), and > then setuid down to the hbase user. We should not require this, but we should > provide it as an option. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4391) Add ability to start RS as root and call mlockall
[ https://issues.apache.org/jira/browse/HBASE-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Ted Yu updated HBASE-4391: -- Release Note: A new profile, native, has been added to pom.xml It would build libmlockall_agent.so You can load the agent by adding it as a jvm option: export HBASE_REGIONSERVER_OPTS="-agentpath:./libmlockall_agent.so=user=hbase" Hadoop Flags: Reviewed > Add ability to start RS as root and call mlockall > - > > Key: HBASE-4391 > URL: https://issues.apache.org/jira/browse/HBASE-4391 > Project: HBase > Issue Type: New Feature > Components: regionserver >Affects Versions: 0.94.0 >Reporter: Todd Lipcon >Assignee: Matteo Bertozzi > Fix For: 0.96.0 > > Attachments: HBASE-4391-v0.patch, HBASE-4391-v1.patch, > HBASE-4391-v2.patch, HBASE-4391-v3.patch > > > A common issue we've seen in practice is that users oversubscribe their > region servers with too many MR tasks, etc. As soon as the machine starts > swapping, the RS grinds to a halt, loses ZK session, aborts, etc. > This can be combatted by starting the RS as root, calling mlockall(), and > then setuid down to the hbase user. We should not require this, but we should > provide it as an option. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4391) Add ability to start RS as root and call mlockall
[ https://issues.apache.org/jira/browse/HBASE-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-4391: --- Attachment: HBASE-4391-v3.patch > Add ability to start RS as root and call mlockall > - > > Key: HBASE-4391 > URL: https://issues.apache.org/jira/browse/HBASE-4391 > Project: HBase > Issue Type: New Feature > Components: regionserver >Affects Versions: 0.94.0 >Reporter: Todd Lipcon >Assignee: Matteo Bertozzi > Fix For: 0.96.0 > > Attachments: HBASE-4391-v0.patch, HBASE-4391-v1.patch, > HBASE-4391-v2.patch, HBASE-4391-v3.patch > > > A common issue we've seen in practice is that users oversubscribe their > region servers with too many MR tasks, etc. As soon as the machine starts > swapping, the RS grinds to a halt, loses ZK session, aborts, etc. > This can be combatted by starting the RS as root, calling mlockall(), and > then setuid down to the hbase user. We should not require this, but we should > provide it as an option. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4391) Add ability to start RS as root and call mlockall
[ https://issues.apache.org/jira/browse/HBASE-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-4391: --- Assignee: Matteo Bertozzi (was: Todd Lipcon) Status: Patch Available (was: Open) > Add ability to start RS as root and call mlockall > - > > Key: HBASE-4391 > URL: https://issues.apache.org/jira/browse/HBASE-4391 > Project: HBase > Issue Type: New Feature > Components: regionserver >Affects Versions: 0.94.0 >Reporter: Todd Lipcon >Assignee: Matteo Bertozzi > Fix For: 0.96.0 > > Attachments: HBASE-4391-v0.patch, HBASE-4391-v1.patch, > HBASE-4391-v2.patch > > > A common issue we've seen in practice is that users oversubscribe their > region servers with too many MR tasks, etc. As soon as the machine starts > swapping, the RS grinds to a halt, loses ZK session, aborts, etc. > This can be combatted by starting the RS as root, calling mlockall(), and > then setuid down to the hbase user. We should not require this, but we should > provide it as an option. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4391) Add ability to start RS as root and call mlockall
[ https://issues.apache.org/jira/browse/HBASE-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-4391: --- Attachment: HBASE-4391-v2.patch Added profile "native" (-Pnative) to turn on and off the compilation of the mlock agent. hbase --mlock user=hbase regionserver start > Add ability to start RS as root and call mlockall > - > > Key: HBASE-4391 > URL: https://issues.apache.org/jira/browse/HBASE-4391 > Project: HBase > Issue Type: New Feature > Components: regionserver >Affects Versions: 0.94.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon > Fix For: 0.96.0 > > Attachments: HBASE-4391-v0.patch, HBASE-4391-v1.patch, > HBASE-4391-v2.patch > > > A common issue we've seen in practice is that users oversubscribe their > region servers with too many MR tasks, etc. As soon as the machine starts > swapping, the RS grinds to a halt, loses ZK session, aborts, etc. > This can be combatted by starting the RS as root, calling mlockall(), and > then setuid down to the hbase user. We should not require this, but we should > provide it as an option. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4391) Add ability to start RS as root and call mlockall
[ https://issues.apache.org/jira/browse/HBASE-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-4391: --- Attachment: HBASE-4391-v1.patch Moved to src/main/native as Andrew suggested > Add ability to start RS as root and call mlockall > - > > Key: HBASE-4391 > URL: https://issues.apache.org/jira/browse/HBASE-4391 > Project: HBase > Issue Type: New Feature > Components: regionserver >Affects Versions: 0.94.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon > Fix For: 0.96.0 > > Attachments: HBASE-4391-v0.patch, HBASE-4391-v1.patch > > > A common issue we've seen in practice is that users oversubscribe their > region servers with too many MR tasks, etc. As soon as the machine starts > swapping, the RS grinds to a halt, loses ZK session, aborts, etc. > This can be combatted by starting the RS as root, calling mlockall(), and > then setuid down to the hbase user. We should not require this, but we should > provide it as an option. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4391) Add ability to start RS as root and call mlockall
[ https://issues.apache.org/jira/browse/HBASE-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-4391: --- Attachment: HBASE-4391-v0.patch Something like this? where you can start hbase scripts with --mlock user=hbase to enable the mlock agent? > Add ability to start RS as root and call mlockall > - > > Key: HBASE-4391 > URL: https://issues.apache.org/jira/browse/HBASE-4391 > Project: HBase > Issue Type: New Feature > Components: regionserver >Affects Versions: 0.94.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon > Fix For: 0.96.0 > > Attachments: HBASE-4391-v0.patch > > > A common issue we've seen in practice is that users oversubscribe their > region servers with too many MR tasks, etc. As soon as the machine starts > swapping, the RS grinds to a halt, loses ZK session, aborts, etc. > This can be combatted by starting the RS as root, calling mlockall(), and > then setuid down to the hbase user. We should not require this, but we should > provide it as an option. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4391) Add ability to start RS as root and call mlockall
[ https://issues.apache.org/jira/browse/HBASE-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-4391: - Fix Version/s: (was: 0.94.0) 0.96.0 Moving out of 0.94, pull back if feel otherwise. > Add ability to start RS as root and call mlockall > - > > Key: HBASE-4391 > URL: https://issues.apache.org/jira/browse/HBASE-4391 > Project: HBase > Issue Type: New Feature > Components: regionserver >Affects Versions: 0.94.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon > Fix For: 0.96.0 > > > A common issue we've seen in practice is that users oversubscribe their > region servers with too many MR tasks, etc. As soon as the machine starts > swapping, the RS grinds to a halt, loses ZK session, aborts, etc. > This can be combatted by starting the RS as root, calling mlockall(), and > then setuid down to the hbase user. We should not require this, but we should > provide it as an option. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
