[jira] [Updated] (HBASE-4044) [performance]decrease IO calls when flush hlog
[ https://issues.apache.org/jira/browse/HBASE-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mingjian updated HBASE-4044: Attachment: HLog.patch [performance]decrease IO calls when flush hlog -- Key: HBASE-4044 URL: https://issues.apache.org/jira/browse/HBASE-4044 Project: HBase Issue Type: Improvement Components: performance, regionserver Reporter: mingjian Fix For: 0.90.4 Attachments: HLog.patch We found that the hlog flush to disk would block other write threads. When one thread exec doWrite(info, logKey, edit);, the others wait for updateLock in HLog.java. Why not the others add their edits into a list and wait. When sync's time, the whole list sync to disk once. I think it will decrease the IO calls. So Maybe we will make two lists for edits. Each thread write to the waledits and wait for updateLock. Each thread can copy the waledits to flushedits and flush the flushedits to disk once it gets updateLock. In my test, it can increase the write speed of 100% when I set hbase.regionserver.handler.count=100. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4044) [performance]decrease IO calls when flush hlog
[ https://issues.apache.org/jira/browse/HBASE-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mingjian updated HBASE-4044: Attachment: (was: HLog.patch) [performance]decrease IO calls when flush hlog -- Key: HBASE-4044 URL: https://issues.apache.org/jira/browse/HBASE-4044 Project: HBase Issue Type: Improvement Components: performance, regionserver Reporter: mingjian Assignee: mingjian Fix For: 0.90.4 Attachments: HLog-trunk-v1.patch We found that the hlog flush to disk would block other write threads. When one thread exec doWrite(info, logKey, edit);, the others wait for updateLock in HLog.java. Why not the others add their edits into a list and wait. When sync's time, the whole list sync to disk once. I think it will decrease the IO calls. So Maybe we will make two lists for edits. Each thread write to the waledits and wait for updateLock. Each thread can copy the waledits to flushedits and flush the flushedits to disk once it gets updateLock. In my test, it can increase the write speed of 100% when I set hbase.regionserver.handler.count=100. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4044) [performance]decrease IO calls when flush hlog
[ https://issues.apache.org/jira/browse/HBASE-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mingjian updated HBASE-4044: Attachment: HLog-trunk-v1.patch [performance]decrease IO calls when flush hlog -- Key: HBASE-4044 URL: https://issues.apache.org/jira/browse/HBASE-4044 Project: HBase Issue Type: Improvement Components: performance, regionserver Reporter: mingjian Assignee: mingjian Fix For: 0.90.4 Attachments: HLog-trunk-v1.patch We found that the hlog flush to disk would block other write threads. When one thread exec doWrite(info, logKey, edit);, the others wait for updateLock in HLog.java. Why not the others add their edits into a list and wait. When sync's time, the whole list sync to disk once. I think it will decrease the IO calls. So Maybe we will make two lists for edits. Each thread write to the waledits and wait for updateLock. Each thread can copy the waledits to flushedits and flush the flushedits to disk once it gets updateLock. In my test, it can increase the write speed of 100% when I set hbase.regionserver.handler.count=100. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4044) [performance]decrease IO calls when flush hlog
[ https://issues.apache.org/jira/browse/HBASE-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mingjian updated HBASE-4044: Description: We found that the hlog flush to disk would block other write threads. When one thread exec doWrite(info, logKey, edit);, the others wait for updateLock in HLog.java. Why not the others add their edits into a list and wait. When sync's time, the whole list sync to disk once. I think it will decrease the IO calls. So Maybe we will make two lists for edits. Each thread write to the waledits and wait for updateLock. Each thread can copy the waledits to flushedits and flush the flushedits to disk once it gets updateLock. In my test, it can increase the write speed of 100% when I set hbase.regionserver.handler.count=100. was: We found that the hlog flush to disk would block other write threads. When one thread exec doWrite(info, logKey, edit);, the others wait for updateLock in HLog.java. Why not the others add their edits into a list and wait. When sync's time, the whole list sync to disk once. I think it will decrease the IO calls. So Maybe we will make two lists for edits. Each thread write to the waledits and wait for updateLock. Each thread can copy the waledits to flushedits and flush the flushedits to disk once it gets updateLock. In my test, it can increase the write speed of 100% when I set hbase.regionserver.handler.count=100. [performance]decrease IO calls when flush hlog -- Key: HBASE-4044 URL: https://issues.apache.org/jira/browse/HBASE-4044 Project: HBase Issue Type: Improvement Components: performance, regionserver Reporter: mingjian Assignee: mingjian Fix For: 0.90.4 Attachments: HLog-trunk-v1.patch We found that the hlog flush to disk would block other write threads. When one thread exec doWrite(info, logKey, edit);, the others wait for updateLock in HLog.java. Why not the others add their edits into a list and wait. When sync's time, the whole list sync to disk once. I think it will decrease the IO calls. So Maybe we will make two lists for edits. Each thread write to the waledits and wait for updateLock. Each thread can copy the waledits to flushedits and flush the flushedits to disk once it gets updateLock. In my test, it can increase the write speed of 100% when I set hbase.regionserver.handler.count=100. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
