[jira] [Commented] (HBASE-10094) Add batching to HLogPerformanceEvaluation
[ https://issues.apache.org/jira/browse/HBASE-10094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13842157#comment-13842157 ] Hudson commented on HBASE-10094: SUCCESS: Integrated in HBase-TRUNK #4715 (See [https://builds.apache.org/job/HBase-TRUNK/4715/]) HBASE-10094 Add batching to HLogPerformanceEvaluation (stack: rev 1548752) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogPerformanceEvaluation.java Add batching to HLogPerformanceEvaluation - Key: HBASE-10094 URL: https://issues.apache.org/jira/browse/HBASE-10094 Project: HBase Issue Type: Sub-task Components: Performance, wal Reporter: stack Assignee: Himanshu Vashishtha Fix For: 0.98.0, 0.96.1, 0.99.0 Attachments: 10094v2.txt As Himanshu points out in the the parent issue, HLogPE is using an unorthodox API appending edits to the WAL; it is using an API that is meant for tests only that does an append immediately followed by a sync call. In normal deploy, WAL appends are done as a bunch of appends followed by a sync on the tail of the transaction -- not a sync per append. This issue is about changing HLogPE to use append and then sync. It also adds an argument so you can specifying batching of a set of appends before the sync is called. The latter lets HLogPE mimic multi puts that use the minibatch... which appends, appends, appends.. and then syncs. Assigning to Himanshu for review. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-10094) Add batching to HLogPerformanceEvaluation
[ https://issues.apache.org/jira/browse/HBASE-10094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13842173#comment-13842173 ] Hudson commented on HBASE-10094: FAILURE: Integrated in hbase-0.96-hadoop2 #143 (See [https://builds.apache.org/job/hbase-0.96-hadoop2/143/]) HBASE-10094 Add batching to HLogPerformanceEvaluation (stack: rev 1548754) * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogPerformanceEvaluation.java Add batching to HLogPerformanceEvaluation - Key: HBASE-10094 URL: https://issues.apache.org/jira/browse/HBASE-10094 Project: HBase Issue Type: Sub-task Components: Performance, wal Reporter: stack Assignee: Himanshu Vashishtha Fix For: 0.98.0, 0.96.1, 0.99.0 Attachments: 10094v2.txt As Himanshu points out in the the parent issue, HLogPE is using an unorthodox API appending edits to the WAL; it is using an API that is meant for tests only that does an append immediately followed by a sync call. In normal deploy, WAL appends are done as a bunch of appends followed by a sync on the tail of the transaction -- not a sync per append. This issue is about changing HLogPE to use append and then sync. It also adds an argument so you can specifying batching of a set of appends before the sync is called. The latter lets HLogPE mimic multi puts that use the minibatch... which appends, appends, appends.. and then syncs. Assigning to Himanshu for review. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-10094) Add batching to HLogPerformanceEvaluation
[ https://issues.apache.org/jira/browse/HBASE-10094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13842180#comment-13842180 ] Hudson commented on HBASE-10094: FAILURE: Integrated in hbase-0.98-hadoop2 #2 (See [https://builds.apache.org/job/hbase-0.98-hadoop2/2/]) HBASE-10094 Add batching to HLogPerformanceEvaluation (stack: rev 1548753) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogPerformanceEvaluation.java Add batching to HLogPerformanceEvaluation - Key: HBASE-10094 URL: https://issues.apache.org/jira/browse/HBASE-10094 Project: HBase Issue Type: Sub-task Components: Performance, wal Reporter: stack Assignee: Himanshu Vashishtha Fix For: 0.98.0, 0.96.1, 0.99.0 Attachments: 10094v2.txt As Himanshu points out in the the parent issue, HLogPE is using an unorthodox API appending edits to the WAL; it is using an API that is meant for tests only that does an append immediately followed by a sync call. In normal deploy, WAL appends are done as a bunch of appends followed by a sync on the tail of the transaction -- not a sync per append. This issue is about changing HLogPE to use append and then sync. It also adds an argument so you can specifying batching of a set of appends before the sync is called. The latter lets HLogPE mimic multi puts that use the minibatch... which appends, appends, appends.. and then syncs. Assigning to Himanshu for review. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-10094) Add batching to HLogPerformanceEvaluation
[ https://issues.apache.org/jira/browse/HBASE-10094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13842354#comment-13842354 ] Hudson commented on HBASE-10094: SUCCESS: Integrated in hbase-0.96 #218 (See [https://builds.apache.org/job/hbase-0.96/218/]) HBASE-10094 Add batching to HLogPerformanceEvaluation (stack: rev 1548913) * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogPerformanceEvaluation.java Add batching to HLogPerformanceEvaluation - Key: HBASE-10094 URL: https://issues.apache.org/jira/browse/HBASE-10094 Project: HBase Issue Type: Sub-task Components: Performance, wal Reporter: stack Assignee: Himanshu Vashishtha Fix For: 0.98.0, 0.96.1, 0.99.0 Attachments: 10094v2.txt, addendum.0.96.txt As Himanshu points out in the the parent issue, HLogPE is using an unorthodox API appending edits to the WAL; it is using an API that is meant for tests only that does an append immediately followed by a sync call. In normal deploy, WAL appends are done as a bunch of appends followed by a sync on the tail of the transaction -- not a sync per append. This issue is about changing HLogPE to use append and then sync. It also adds an argument so you can specifying batching of a set of appends before the sync is called. The latter lets HLogPE mimic multi puts that use the minibatch... which appends, appends, appends.. and then syncs. Assigning to Himanshu for review. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-10094) Add batching to HLogPerformanceEvaluation
[ https://issues.apache.org/jira/browse/HBASE-10094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13842373#comment-13842373 ] Hudson commented on HBASE-10094: SUCCESS: Integrated in hbase-0.96-hadoop2 #144 (See [https://builds.apache.org/job/hbase-0.96-hadoop2/144/]) HBASE-10094 Add batching to HLogPerformanceEvaluation (stack: rev 1548913) * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogPerformanceEvaluation.java Add batching to HLogPerformanceEvaluation - Key: HBASE-10094 URL: https://issues.apache.org/jira/browse/HBASE-10094 Project: HBase Issue Type: Sub-task Components: Performance, wal Reporter: stack Assignee: Himanshu Vashishtha Fix For: 0.98.0, 0.96.1, 0.99.0 Attachments: 10094v2.txt, addendum.0.96.txt As Himanshu points out in the the parent issue, HLogPE is using an unorthodox API appending edits to the WAL; it is using an API that is meant for tests only that does an append immediately followed by a sync call. In normal deploy, WAL appends are done as a bunch of appends followed by a sync on the tail of the transaction -- not a sync per append. This issue is about changing HLogPE to use append and then sync. It also adds an argument so you can specifying batching of a set of appends before the sync is called. The latter lets HLogPE mimic multi puts that use the minibatch... which appends, appends, appends.. and then syncs. Assigning to Himanshu for review. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-10094) Add batching to HLogPerformanceEvaluation
[
https://issues.apache.org/jira/browse/HBASE-10094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13841801#comment-13841801
]
Hadoop QA commented on HBASE-10094:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12617453/10094v2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8075//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8075//console
This message is automatically generated.
Add batching to HLogPerformanceEvaluation
-
Key: HBASE-10094
URL: https://issues.apache.org/jira/browse/HBASE-10094
Project: HBase
Issue Type: Sub-task
Components: Performance, wal
Reporter: stack
Assignee: Himanshu Vashishtha
Attachments: 10094v2.txt
As Himanshu points out in the the parent issue, HLogPE is using an unorthodox
API appending edits to the WAL; it is using an API that is meant for tests
only that does an append immediately followed by a sync call.
In normal deploy, WAL appends are done as a bunch of appends followed by a
sync on the tail of the transaction -- not a sync per append.
This issue is about changing HLogPE to use append and then sync. It also
adds an argument so you can specifying batching of a set of appends before
the sync is called. The latter lets HLogPE mimic multi puts that use the
minibatch... which appends, appends, appends.. and then syncs.
Assigning to Himanshu for review.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-10094) Add batching to HLogPerformanceEvaluation
[ https://issues.apache.org/jira/browse/HBASE-10094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13841816#comment-13841816 ] Himanshu Vashishtha commented on HBASE-10094: - +1. This would give more insights when comparing various schemes on batching sync calls. minor nit: -long numIterations = 1; +long numIterations = 100; I don't think we need this. It will make TestHLog run longer by almost 90 sec. Add batching to HLogPerformanceEvaluation - Key: HBASE-10094 URL: https://issues.apache.org/jira/browse/HBASE-10094 Project: HBase Issue Type: Sub-task Components: Performance, wal Reporter: stack Assignee: Himanshu Vashishtha Attachments: 10094v2.txt As Himanshu points out in the the parent issue, HLogPE is using an unorthodox API appending edits to the WAL; it is using an API that is meant for tests only that does an append immediately followed by a sync call. In normal deploy, WAL appends are done as a bunch of appends followed by a sync on the tail of the transaction -- not a sync per append. This issue is about changing HLogPE to use append and then sync. It also adds an argument so you can specifying batching of a set of appends before the sync is called. The latter lets HLogPE mimic multi puts that use the minibatch... which appends, appends, appends.. and then syncs. Assigning to Himanshu for review. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-10094) Add batching to HLogPerformanceEvaluation
[ https://issues.apache.org/jira/browse/HBASE-10094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13841821#comment-13841821 ] Himanshu Vashishtha commented on HBASE-10094: - Errr, just ignore my last nit... TestHLog doesn't use default iterations. It's good to go. Thanks. Add batching to HLogPerformanceEvaluation - Key: HBASE-10094 URL: https://issues.apache.org/jira/browse/HBASE-10094 Project: HBase Issue Type: Sub-task Components: Performance, wal Reporter: stack Assignee: Himanshu Vashishtha Attachments: 10094v2.txt As Himanshu points out in the the parent issue, HLogPE is using an unorthodox API appending edits to the WAL; it is using an API that is meant for tests only that does an append immediately followed by a sync call. In normal deploy, WAL appends are done as a bunch of appends followed by a sync on the tail of the transaction -- not a sync per append. This issue is about changing HLogPE to use append and then sync. It also adds an argument so you can specifying batching of a set of appends before the sync is called. The latter lets HLogPE mimic multi puts that use the minibatch... which appends, appends, appends.. and then syncs. Assigning to Himanshu for review. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-10094) Add batching to HLogPerformanceEvaluation
[ https://issues.apache.org/jira/browse/HBASE-10094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13842116#comment-13842116 ] Hudson commented on HBASE-10094: FAILURE: Integrated in hbase-0.96 #217 (See [https://builds.apache.org/job/hbase-0.96/217/]) HBASE-10094 Add batching to HLogPerformanceEvaluation (stack: rev 1548754) * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogPerformanceEvaluation.java Add batching to HLogPerformanceEvaluation - Key: HBASE-10094 URL: https://issues.apache.org/jira/browse/HBASE-10094 Project: HBase Issue Type: Sub-task Components: Performance, wal Reporter: stack Assignee: Himanshu Vashishtha Fix For: 0.98.0, 0.96.1, 0.99.0 Attachments: 10094v2.txt As Himanshu points out in the the parent issue, HLogPE is using an unorthodox API appending edits to the WAL; it is using an API that is meant for tests only that does an append immediately followed by a sync call. In normal deploy, WAL appends are done as a bunch of appends followed by a sync on the tail of the transaction -- not a sync per append. This issue is about changing HLogPE to use append and then sync. It also adds an argument so you can specifying batching of a set of appends before the sync is called. The latter lets HLogPE mimic multi puts that use the minibatch... which appends, appends, appends.. and then syncs. Assigning to Himanshu for review. -- This message was sent by Atlassian JIRA (v6.1#6144)
