[jira] [Commented] (HBASE-8028) Append, Increment don't handle wal-sync exceptions correctly
[ https://issues.apache.org/jira/browse/HBASE-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13600200#comment-13600200 ] Himanshu Vashishtha commented on HBASE-8028: TestHLog passes for me locally. > Append, Increment don't handle wal-sync exceptions correctly > > > Key: HBASE-8028 > URL: https://issues.apache.org/jira/browse/HBASE-8028 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.5 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Fix For: 0.95.0 > > Attachments: HBase-8028-v1.patch, HBase-8028-v2.patch, > HBase-8028-with-Increments-v1.patch > > > In case there is an exception while doing the log-sync, the memstore is not > rollbacked, while the mvcc is _always_ forwarded to the writeentry created at > the beginning of the operation. This may lead to scanners seeing results > which are not synched to the fs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8028) Append, Increment don't handle wal-sync exceptions correctly
[
https://issues.apache.org/jira/browse/HBASE-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13599539#comment-13599539
]
Hadoop QA commented on HBASE-8028:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573215/HBase-8028-with-Increments-v1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.wal.TestHLog
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4769//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4769//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4769//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4769//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4769//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4769//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4769//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4769//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4769//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4769//console
This message is automatically generated.
> Append, Increment don't handle wal-sync exceptions correctly
>
>
> Key: HBASE-8028
> URL: https://issues.apache.org/jira/browse/HBASE-8028
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.94.5
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
> Fix For: 0.95.0
>
> Attachments: HBase-8028-v1.patch, HBase-8028-v2.patch,
> HBase-8028-with-Increments-v1.patch
>
>
> In case there is an exception while doing the log-sync, the memstore is not
> rollbacked, while the mvcc is _always_ forwarded to the writeentry created at
> the beginning of the operation. This may lead to scanners seeing results
> which are not synched to the fs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8028) Append, Increment don't handle wal-sync exceptions correctly
[
https://issues.apache.org/jira/browse/HBASE-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13599515#comment-13599515
]
Hadoop QA commented on HBASE-8028:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573186/HBase-8028-v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4767//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4767//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4767//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4767//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4767//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4767//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4767//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4767//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4767//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4767//console
This message is automatically generated.
> Append, Increment don't handle wal-sync exceptions correctly
>
>
> Key: HBASE-8028
> URL: https://issues.apache.org/jira/browse/HBASE-8028
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.94.5
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
> Fix For: 0.95.0
>
> Attachments: HBase-8028-v1.patch, HBase-8028-v2.patch,
> HBase-8028-with-Increments-v1.patch
>
>
> In case there is an exception while doing the log-sync, the memstore is not
> rollbacked, while the mvcc is _always_ forwarded to the writeentry created at
> the beginning of the operation. This may lead to scanners seeing results
> which are not synched to the fs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8028) Append, Increment don't handle wal-sync exceptions correctly
[
https://issues.apache.org/jira/browse/HBASE-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13599344#comment-13599344
]
Ted Yu commented on HBASE-8028:
---
For deleteKVWithMvccReadPoint():
{code}
+while (iter.hasNext()) {
+ KeyValue cur = iter.next();
+ LOG.warn("kv present is: " + cur);
+ if (cur.matchingRow(referenceKV) && cur.matchingQualifier(referenceKV)) {
{code}
Can we utilize the following ?
http://docs.oracle.com/javase/6/docs/api/java/util/SortedSet.html#tailSet(E)
> Append, Increment don't handle wal-sync exceptions correctly
>
>
> Key: HBASE-8028
> URL: https://issues.apache.org/jira/browse/HBASE-8028
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.94.5
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
> Fix For: 0.95.0
>
> Attachments: HBase-8028-v1.patch
>
>
> In case there is an exception while doing the log-sync, the memstore is not
> rollbacked, while the mvcc is _always_ forwarded to the writeentry created at
> the beginning of the operation. This may lead to scanners seeing results
> which are not synched to the fs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8028) Append, Increment don't handle wal-sync exceptions correctly
[ https://issues.apache.org/jira/browse/HBASE-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13599335#comment-13599335 ] Himanshu Vashishtha commented on HBASE-8028: Ted: will do the edits. Ah, those warn level logs were there for debug purpose only... will remove :) > Append, Increment don't handle wal-sync exceptions correctly > > > Key: HBASE-8028 > URL: https://issues.apache.org/jira/browse/HBASE-8028 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.5 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Fix For: 0.95.0 > > Attachments: HBase-8028-v1.patch > > > In case there is an exception while doing the log-sync, the memstore is not > rollbacked, while the mvcc is _always_ forwarded to the writeentry created at > the beginning of the operation. This may lead to scanners seeing results > which are not synched to the fs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8028) Append, Increment don't handle wal-sync exceptions correctly
[ https://issues.apache.org/jira/browse/HBASE-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13599329#comment-13599329 ] Himanshu Vashishtha commented on HBASE-8028: Lars: Thanks. A 10-15% hit is something one can't ignore. And, moreover, it also reduces the flush size (and eventually compactions i/o), that's why I think it's okay to keep upserts. > Append, Increment don't handle wal-sync exceptions correctly > > > Key: HBASE-8028 > URL: https://issues.apache.org/jira/browse/HBASE-8028 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.5 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Fix For: 0.95.0 > > Attachments: HBase-8028-v1.patch > > > In case there is an exception while doing the log-sync, the memstore is not > rollbacked, while the mvcc is _always_ forwarded to the writeentry created at > the beginning of the operation. This may lead to scanners seeing results > which are not synched to the fs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8028) Append, Increment don't handle wal-sync exceptions correctly
[
https://issues.apache.org/jira/browse/HBASE-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13599326#comment-13599326
]
Ted Yu commented on HBASE-8028:
---
Comment on code style:
{code}
+ if(store.getFamily().getMaxVersions() == 1)
+oldKvs = new ArrayList(family.getValue().size());//
better than default.
{code}
Space should be inserted between if and (
Please add curly braces for the assignment.
This applies to other changes in the patch.
{code}
+ * Since the mvcc readpoint is not forwarded untill we are done with this
rollback, it shouldn't
{code}
Typo: untill
{code}
+ * Rollbacks an upsert transaction. It removes the kvs which share the same
readpoint as the
+ * passed mvccStartPoint, and then replays the oldKVs passed as in
oldKVsToReplay.
{code}
I think 'Rolls back' would be better than 'Rollbacks'
'passed as in' -> 'passed in as'
In deleteKVWithMvccReadPoint(), there're three LOG.warn() statements. Should
they be at DEBUG or TRACE level ?
> Append, Increment don't handle wal-sync exceptions correctly
>
>
> Key: HBASE-8028
> URL: https://issues.apache.org/jira/browse/HBASE-8028
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.94.5
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
> Fix For: 0.95.0
>
> Attachments: HBase-8028-v1.patch
>
>
> In case there is an exception while doing the log-sync, the memstore is not
> rollbacked, while the mvcc is _always_ forwarded to the writeentry created at
> the beginning of the operation. This may lead to scanners seeing results
> which are not synched to the fs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8028) Append, Increment don't handle wal-sync exceptions correctly
[ https://issues.apache.org/jira/browse/HBASE-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13599303#comment-13599303 ] Lars Hofhansl commented on HBASE-8028: -- Approach looks good. Personally I would prefer to remove all that upsert huh-hah and take the performance hit (which is measured about 10-15% before). We're not doing upserts for Puts/Deletes even though that would also be faster; and I do not see in principle how Append/Increment are different (other than the fact that folks are now used to the slightly better performance). > Append, Increment don't handle wal-sync exceptions correctly > > > Key: HBASE-8028 > URL: https://issues.apache.org/jira/browse/HBASE-8028 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.5 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Fix For: 0.95.0 > > Attachments: HBase-8028-v1.patch > > > In case there is an exception while doing the log-sync, the memstore is not > rollbacked, while the mvcc is _always_ forwarded to the writeentry created at > the beginning of the operation. This may lead to scanners seeing results > which are not synched to the fs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8028) Append, Increment don't handle wal-sync exceptions correctly
[ https://issues.apache.org/jira/browse/HBASE-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13599287#comment-13599287 ] Himanshu Vashishtha commented on HBASE-8028: Ted, yes true (its for append, not increment). I mentioned earlier that this patch is for append(). Will add this to increment also, if people agreed with this approach. > Append, Increment don't handle wal-sync exceptions correctly > > > Key: HBASE-8028 > URL: https://issues.apache.org/jira/browse/HBASE-8028 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.5 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Fix For: 0.95.0 > > Attachments: HBase-8028-v1.patch > > > In case there is an exception while doing the log-sync, the memstore is not > rollbacked, while the mvcc is _always_ forwarded to the writeentry created at > the beginning of the operation. This may lead to scanners seeing results > which are not synched to the fs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8028) Append, Increment don't handle wal-sync exceptions correctly
[ https://issues.apache.org/jira/browse/HBASE-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13599284#comment-13599284 ] Ted Yu commented on HBASE-8028: --- Yes, that's true. If I read the patch correctly, tempMemstoreWithOlderKVs is only created inside increment(). What about append() ? > Append, Increment don't handle wal-sync exceptions correctly > > > Key: HBASE-8028 > URL: https://issues.apache.org/jira/browse/HBASE-8028 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.5 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Fix For: 0.95.0 > > Attachments: HBase-8028-v1.patch > > > In case there is an exception while doing the log-sync, the memstore is not > rollbacked, while the mvcc is _always_ forwarded to the writeentry created at > the beginning of the operation. This may lead to scanners seeing results > which are not synched to the fs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8028) Append, Increment don't handle wal-sync exceptions correctly
[
https://issues.apache.org/jira/browse/HBASE-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13599274#comment-13599274
]
Himanshu Vashishtha commented on HBASE-8028:
Thanks for the review Ted.
bq. Can CF version checking be done above so that we don't accumulate KeyValues
which would not be used in rollback ?
The patch is doing that. oldKVs is instantiated only when CF#maxVersion = 1.
{code}
List oldKvs = null;
if(store.getFamily().getMaxVersions() == 1)
oldKvs = new ArrayList(family.getValue().size());//
better than default.
{code}
> Append, Increment don't handle wal-sync exceptions correctly
>
>
> Key: HBASE-8028
> URL: https://issues.apache.org/jira/browse/HBASE-8028
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.94.5
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
> Fix For: 0.95.0
>
> Attachments: HBase-8028-v1.patch
>
>
> In case there is an exception while doing the log-sync, the memstore is not
> rollbacked, while the mvcc is _always_ forwarded to the writeentry created at
> the beginning of the operation. This may lead to scanners seeing results
> which are not synched to the fs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8028) Append, Increment don't handle wal-sync exceptions correctly
[
https://issues.apache.org/jira/browse/HBASE-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13599229#comment-13599229
]
Ted Yu commented on HBASE-8028:
---
{code}
+ if(oldKvs != null)
+tempMemstoreWithOlderKVs.put(store, oldKvs);
{code}
Can CF version checking be done above so that we don't accumulate KeyValues
which would not be used in rollback ?
> Append, Increment don't handle wal-sync exceptions correctly
>
>
> Key: HBASE-8028
> URL: https://issues.apache.org/jira/browse/HBASE-8028
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.94.5
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
> Fix For: 0.95.0
>
> Attachments: HBase-8028-v1.patch
>
>
> In case there is an exception while doing the log-sync, the memstore is not
> rollbacked, while the mvcc is _always_ forwarded to the writeentry created at
> the beginning of the operation. This may lead to scanners seeing results
> which are not synched to the fs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8028) Append, Increment don't handle wal-sync exceptions correctly
[
https://issues.apache.org/jira/browse/HBASE-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13599203#comment-13599203
]
Hadoop QA commented on HBASE-8028:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573143/HBase-8028-v1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4759//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4759//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4759//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4759//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4759//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4759//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4759//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4759//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4759//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4759//console
This message is automatically generated.
> Append, Increment don't handle wal-sync exceptions correctly
>
>
> Key: HBASE-8028
> URL: https://issues.apache.org/jira/browse/HBASE-8028
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.94.5
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
> Fix For: 0.95.0
>
> Attachments: HBase-8028-v1.patch
>
>
> In case there is an exception while doing the log-sync, the memstore is not
> rollbacked, while the mvcc is _always_ forwarded to the writeentry created at
> the beginning of the operation. This may lead to scanners seeing results
> which are not synched to the fs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8028) Append, Increment don't handle wal-sync exceptions correctly
[ https://issues.apache.org/jira/browse/HBASE-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13597536#comment-13597536 ] Lars Hofhansl commented on HBASE-8028: -- That could in theory work. I'd be worried about side-effects, but maybe I shouldn't. > Append, Increment don't handle wal-sync exceptions correctly > > > Key: HBASE-8028 > URL: https://issues.apache.org/jira/browse/HBASE-8028 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.5 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Fix For: 0.95.0 > > > In case there is an exception while doing the log-sync, the memstore is not > rollbacked, while the mvcc is _always_ forwarded to the writeentry created at > the beginning of the operation. This may lead to scanners seeing results > which are not synched to the fs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8028) Append, Increment don't handle wal-sync exceptions correctly
[ https://issues.apache.org/jira/browse/HBASE-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13597524#comment-13597524 ] Himanshu Vashishtha commented on HBASE-8028: [~lhofhansl]: it is a long jira with some interesting discussion. In case the CF version is set to 1, to do the rollback: does it make sense to remove all the keyvalues from the store with the memstoreTs = the mvcc point used in the beginning of the append operation, and inserting the old keyvalues. In case the old keyvalues already exists, it will not be added to the kvskiplist by its very nature (no dups). > Append, Increment don't handle wal-sync exceptions correctly > > > Key: HBASE-8028 > URL: https://issues.apache.org/jira/browse/HBASE-8028 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.5 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Fix For: 0.95.0 > > > In case there is an exception while doing the log-sync, the memstore is not > rollbacked, while the mvcc is _always_ forwarded to the writeentry created at > the beginning of the operation. This may lead to scanners seeing results > which are not synched to the fs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8028) Append, Increment don't handle wal-sync exceptions correctly
[ https://issues.apache.org/jira/browse/HBASE-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13596349#comment-13596349 ] Lars Hofhansl commented on HBASE-8028: -- That's probably the reason. In 0.94 and before it simply could not be done (and whoever pulled the sync outside of the rowlock just ignored this problem to get the performance benefit). When I made this MVCC aware/safe (HBASE-4583) in trunk I did not realize that now this is principle fixable. If I had I would have made a stronger to remove upsert completely and always create new versions (just as do for all other operations). So we should have this discussion again. In HBASE-4583 I have a radical patch that does away with upsert at the expense of performance. The radical patch would the solution you outline (i.e. to the same we do for puts: remove the added KVs if the sync fails). The sentiment was that we should do the less radical version, which has the compromise of using upsert when VERSIONS=1. > Append, Increment don't handle wal-sync exceptions correctly > > > Key: HBASE-8028 > URL: https://issues.apache.org/jira/browse/HBASE-8028 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.5 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Fix For: 0.95.0 > > > In case there is an exception while doing the log-sync, the memstore is not > rollbacked, while the mvcc is _always_ forwarded to the writeentry created at > the beginning of the operation. This may lead to scanners seeing results > which are not synched to the fs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8028) Append, Increment don't handle wal-sync exceptions correctly
[ https://issues.apache.org/jira/browse/HBASE-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13596319#comment-13596319 ] Himanshu Vashishtha commented on HBASE-8028: Looked more into it. For CF which don't have 1-version limitation, we can simply remove the added kvs. Its the upsert logic which makes this tricky: depending on the lowestreadpoint at the time when it is doing the append operation, it decides whether it should do an update (remove the existing kv), or insert a new kv. It doesn't keep a history of its actions, though, which makes it difficult to decide the roll back process. [~lhofhansl]: What's your take on this? Is this the reason why Append doesn't have a rollback scheme? :) > Append, Increment don't handle wal-sync exceptions correctly > > > Key: HBASE-8028 > URL: https://issues.apache.org/jira/browse/HBASE-8028 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.5 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Fix For: 0.95.0 > > > In case there is an exception while doing the log-sync, the memstore is not > rollbacked, while the mvcc is _always_ forwarded to the writeentry created at > the beginning of the operation. This may lead to scanners seeing results > which are not synched to the fs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
