[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13799513#comment-13799513
]
Lars Hofhansl commented on HBASE-9754:
--
Last patch looks good.
Do you have time to make a 0.94 version [~yuzhih...@gmail.com]? If so, I could
do an apples to apples test in 0.94.
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt, 9754-rp-1.txt, 9754-rp-hregion.txt,
9754-rp-hregion-v2.txt, 9754-rp-hregion-v3.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13798055#comment-13798055
]
Hadoop QA commented on HBASE-9754:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12608945/9754-rp-hregion-v2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 27 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7577//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7577//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7577//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7577//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7577//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7577//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7577//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7577//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7577//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7577//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7577//console
This message is automatically generated.
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt, 9754-rp-1.txt, 9754-rp-hregion.txt,
9754-rp-hregion-v2.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13798086#comment-13798086
]
stack commented on HBASE-9754:
--
What are the implications of this change?
-MultiVersionConsistencyControl.resetThreadReadPoint();
Scan scan = new Scan(get);
InternalScanner scanner = (InternalScanner) store.getScanner(scan,
-scan.getFamilyMap().get(store.getFamily().getName()));
+scan.getFamilyMap().get(store.getFamily().getName()), 0);
We no longer do reset and we pass 0 which means all is readable? Would think
we'd retain at least the first line?
Does defaulting 0 in all tests change what is being tested (or do you think the
reset was setting readpoint to 0 across the board anyways)?
This should be final now?
+ private long readPt = 0;
Is this an odd 'public' method to have in HRegion?
+ /*
+* Returns readpoint considering given IsolationLevel
+*/
+ public long getReadpoint(IsolationLevel isolationLevel) {
+ if (isolationLevel == IsolationLevel.READ_UNCOMMITTED) {
+ // This scan can read even uncommitted transactions
+ return Long.MAX_VALUE;
+ }
+ return mvcc.memstoreReadPoint();
+ }
especially as given just above, we can do a getMVCC so can go
'behind-the-scenes'. Should it be private?
It is the readPoint for the region?
Remove
-MultiVersionConsistencyControl.setThreadReadPoint(this.readPt);
+// MultiVersionConsistencyControl.setThreadReadPoint(this.readPt);
ditto
-MultiVersionConsistencyControl.setThreadReadPoint(this.readPt);
+// MultiVersionConsistencyControl.setThreadReadPoint(this.readPt);
Undocumented passing of zero (explaining why this is ok) and the sometimes
passing of smallestreadpoint instead of readpoint is unsettling. Explaination
on why it is ok in the former case and what smallest means in latter would help
the folks who come after understand what is going on.
Otherwise patch is good.
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt, 9754-rp-1.txt, 9754-rp-hregion.txt,
9754-rp-hregion-v2.txt, 9754-rp-hregion-v3.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13798123#comment-13798123
]
Ted Yu commented on HBASE-9754:
---
bq. What are the implications of this change?
You were referring to the change in HBaseTestingUtility.java, right ?
Here is what resetThreadReadPoint() does:
- public static void resetThreadReadPoint() {
-perThreadReadPoint.set(0L);
- }
Meaning, effective readpoint is reset to 0. Hence passing 0 to
store.getScanner() is consistent with the existing test.
bq. Would think we'd retain at least the first line?
Is the first line MultiVersionConsistencyControl.resetThreadReadPoint() ? That
method has been removed.
bq. Does defaulting 0 in all tests change what is being tested
There is no change in semantics. 0 is only used when the two StoreScanner
ctors, marked for testing, are used.
bq. This should be final now?
Done.
bq. Is this an odd 'public' method to have in HRegion?
It is used by NoOpScanPolicyObserver. Compilation passes when I change it to
package private. My reasoning was that if user's RegionObserver overrides
preStoreScannerOpen(), this method is needed. User's code would not be in
org.apache.hadoop.hbase.regionserver package.
bq. It is the readPoint for the region?
Readpoint is associated with mvcc for the region.
bq. Undocumented passing of zero (explaining why this is ok)
Will add doc in next patch.
bq. and what smallest means in latter
Would do this too.
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt, 9754-rp-1.txt, 9754-rp-hregion.txt,
9754-rp-hregion-v2.txt, 9754-rp-hregion-v3.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13798324#comment-13798324
]
Hadoop QA commented on HBASE-9754:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12608987/9754-rp-hregion-v3.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 27 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7581//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7581//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7581//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7581//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7581//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7581//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7581//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7581//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7581//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7581//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7581//console
This message is automatically generated.
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt, 9754-rp-1.txt, 9754-rp-hregion.txt,
9754-rp-hregion-v2.txt, 9754-rp-hregion-v3.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13798719#comment-13798719
]
chunhui shen commented on HBASE-9754:
-
+1 from me for the latest patch
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt, 9754-rp-1.txt, 9754-rp-hregion.txt,
9754-rp-hregion-v2.txt, 9754-rp-hregion-v3.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13796811#comment-13796811
]
Ted Yu commented on HBASE-9754:
---
In ZooKeeperScanPolicyObserver#preCompactScannerOpen(), currently
store.getSmallestReadPoint() is passed.
My change is in accordance with existing code.
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13796821#comment-13796821
]
Ted Yu commented on HBASE-9754:
---
BaseRegionObserver is marked
{code}
@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.COPROC)
@InterfaceStability.Evolving
{code}
We can add a new method which sets the current MVCC readpoint for the observer.
Another method returns MVCC readpoint that would be utilized in constructing
StoreScanner.
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13796835#comment-13796835
]
Lars Hofhansl commented on HBASE-9754:
--
preCompactScannerOpen() is used for compactions where we want the
smalledReadPoint. preStoreScannerOpen is used for user scans where current mvcc
readpoint is what we need.
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13796963#comment-13796963
]
Ted Yu commented on HBASE-9754:
---
Went through the latest patch one more time.
Didn't find where 0 is passed as readpoint.
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt, 9754-rp-1.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13797159#comment-13797159
]
Hadoop QA commented on HBASE-9754:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12608750/9754-rp-hregion.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 27 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7568//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7568//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7568//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7568//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7568//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7568//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7568//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7568//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7568//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7568//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7568//console
This message is automatically generated.
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt, 9754-rp-1.txt, 9754-rp-hregion.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13797416#comment-13797416
]
Enis Soztutar commented on HBASE-9754:
--
I think we should run TestAcidGuarantees a couple of times for some stress
testing before commit.
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt, 9754-rp-1.txt, 9754-rp-hregion.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13797475#comment-13797475
]
Ted Yu commented on HBASE-9754:
---
Here is the tail of one run:
{code}
2013-10-16 17:30:29,951 DEBUG [Thread-55] catalog.CatalogTracker: Stopping
catalog tracker org.apache.hadoop.hbase.catalog.CatalogTracker@6e8ef177
2013-10-16 17:30:29,970 INFO [Thread-55] zookeeper.ZooKeeper: Session:
0x141c373bb9c0035 closed
2013-10-16 17:30:29,970 INFO [Thread-55-EventThread] zookeeper.ClientCnxn:
EventThread shut down
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: Finished test.
Writers:
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 19
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 19
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 21
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 21
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 19
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 20
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 20
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 20
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 21
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 19
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 21
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 22
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 21
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 19
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 20
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 22
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 20
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 21
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 20
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 19
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 18
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 20
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 20
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 17
2013-10-16 17:31:29,971 INFO [main] hbase.TestAcidGuarantees: wrote 19
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 20
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 21
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 21
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 21
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 20
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 20
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 22
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 22
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 20
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 20
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 19
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 22
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 21
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 21
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 22
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 21
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 20
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 20
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 21
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 20
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 20
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 20
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 20
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 21
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: wrote 19
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: Readers:
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: read 18
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: read 16
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: Scanners:
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: scanned 15
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: verified 18
rows
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees: scanned 21
2013-10-16 17:31:29,972 INFO [main] hbase.TestAcidGuarantees:
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13797477#comment-13797477
]
Ted Yu commented on HBASE-9754:
---
Second run:
{code}
2013-10-16 17:33:16,342 DEBUG [Thread-55] catalog.CatalogTracker: Stopping
catalog tracker org.apache.hadoop.hbase.catalog.CatalogTracker@5e4b2b75
2013-10-16 17:33:16,355 INFO [Thread-55] zookeeper.ZooKeeper: Session:
0x141c373bb9c0037 closed
2013-10-16 17:33:16,355 INFO [Thread-55-EventThread] zookeeper.ClientCnxn:
EventThread shut down
2013-10-16 17:34:16,355 INFO [main] hbase.TestAcidGuarantees: Finished test.
Writers:
2013-10-16 17:34:16,355 INFO [main] hbase.TestAcidGuarantees: wrote 38
2013-10-16 17:34:16,355 INFO [main] hbase.TestAcidGuarantees: wrote 38
2013-10-16 17:34:16,355 INFO [main] hbase.TestAcidGuarantees: wrote 38
2013-10-16 17:34:16,355 INFO [main] hbase.TestAcidGuarantees: wrote 36
2013-10-16 17:34:16,355 INFO [main] hbase.TestAcidGuarantees: wrote 39
2013-10-16 17:34:16,355 INFO [main] hbase.TestAcidGuarantees: wrote 37
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 39
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 40
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 40
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 38
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 39
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 40
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 39
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 37
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 39
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 39
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 37
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 37
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 35
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 39
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 39
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 38
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 38
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 38
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 38
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 36
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 40
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 39
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 38
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 39
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 37
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 38
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 37
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 37
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 38
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 37
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 36
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 37
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 37
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 38
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 38
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 38
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 37
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 40
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 38
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 39
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 38
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 41
2013-10-16 17:34:16,356 INFO [main] hbase.TestAcidGuarantees: wrote 39
2013-10-16 17:34:16,357 INFO [main] hbase.TestAcidGuarantees: wrote 38
2013-10-16 17:34:16,357 INFO [main] hbase.TestAcidGuarantees: Readers:
2013-10-16 17:34:16,357 INFO [main] hbase.TestAcidGuarantees: read 61
2013-10-16 17:34:16,357 INFO [main] hbase.TestAcidGuarantees: read 58
2013-10-16 17:34:16,357 INFO [main] hbase.TestAcidGuarantees: Scanners:
2013-10-16 17:34:16,357 INFO [main] hbase.TestAcidGuarantees: scanned 20
2013-10-16 17:34:16,357 INFO [main] hbase.TestAcidGuarantees: verified 60
rows
2013-10-16 17:34:16,357 INFO [main] hbase.TestAcidGuarantees: scanned 19
2013-10-16 17:34:16,357 INFO [main] hbase.TestAcidGuarantees: verified 57
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13797626#comment-13797626
]
chunhui shen commented on HBASE-9754:
-
In Compactor.java
The 'smallestReadPoint' will be calculated in two places now.
If they have different values, is it safe?
Others are ok for me
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt, 9754-rp-1.txt, 9754-rp-hregion.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13796249#comment-13796249
]
Ted Yu commented on HBASE-9754:
---
In StoreScanner.java, I replaced the initial value with the following and reran
test suite:
{code}
+ private long readPt = Long.MAX_VALUE;
{code}
The suite passed - readPt would be assigned correctly in ctor's so its initial
value doesn't matter.
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13796357#comment-13796357
]
chunhui shen commented on HBASE-9754:
-
Patch looks good.
Agree with eliminating the threadlocals
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13796445#comment-13796445
]
Lars Hofhansl commented on HBASE-9754:
--
I think in the *PolicyObserver.preStoreScannerOpen, we want the current MVCC
readpoint, rather than the store smallest readpoint.
I am also a bit concerned about passing 0 as readPt in some places, as that
makes everything visible. And for example it avoids setting the memstoreTs of
KV to 0 even when there are no old scanners open that could see that KV.
(In fact these were the consideration that made me think a correct patch for
this is hard)
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13796447#comment-13796447
]
Hadoop QA commented on HBASE-9754:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12608649/9754-rp-0.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 27 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7559//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7559//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7559//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7559//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7559//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7559//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7559//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7559//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7559//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7559//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7559//console
This message is automatically generated.
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794544#comment-13794544
]
Ted Yu commented on HBASE-9754:
---
w.r.t. the StoreScanner constructors, there're two which don't have Store
argument. These two constructors are for testing.
The other constructors accept Store argument which has reference to HRegion.
Looks like the approach described above can be simplified.
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794755#comment-13794755
]
Ted Yu commented on HBASE-9754:
---
Patch contains patch for HBASE-9753 as well.
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794757#comment-13794757
]
Ted Yu commented on HBASE-9754:
---
Here is review request: https://reviews.apache.org/r/14642/
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794818#comment-13794818
]
Hadoop QA commented on HBASE-9754:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12608406/9754-rp-0.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 27 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to cause Findbugs
(version 1.3.9) to fail.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7540//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7540//console
This message is automatically generated.
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794827#comment-13794827
]
Ted Yu commented on HBASE-9754:
---
I saw the same number of findbugs warnings in:
https://builds.apache.org/job/PreCommit-HBASE-Build/7538/console
which was for HBASE-9748
I think the cause was the following:
{code}
[INFO] Compiling 44 source files to
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-thrift/target/classes
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 90184 bytes for
Arena::Amalloc
# An error report file with more information is saved as:
#
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hs_err_pid8382.log
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/dev-support/test-patch.sh:
line 561: 8382 Aborted $MVN clean package findbugs:findbugs
-D${PROJECT_NAME}PatchProcess -DskipTests /dev/null
{code}
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794855#comment-13794855
]
Lars Hofhansl commented on HBASE-9754:
--
Did a quick read through. Looks good. Need to look in more detail.
Using 0 as readPt when unknown is interesting.
Could we use the readPt to replace the hasMVCC stuff I just added in
HBASE-9751? I.e. pass MAX_LONG as readPt when we would have passed
hasMVCC=true, in which case we know we do not need to check the memstoreTs...?
I'd be in favor of committing HBASE-9753 first as that is simple, and
scrutinize this one a bit more.
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794863#comment-13794863
]
Ted Yu commented on HBASE-9754:
---
bq. scrutinize this one a bit more.
Take your time.
Depending on how long the review takes, we can decide which one to commit
first. HBASE-9753 is a subset of this.
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794865#comment-13794865
]
Ted Yu commented on HBASE-9754:
---
bq. Could we use the readPt to replace the hasMVCC stuff
Looking at the code:
{code}
+ public boolean hasMVCCInfo() {
+return includesMemstoreTS decodeMemstoreTS;
{code}
I think they represent different things.
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794879#comment-13794879
]
Lars Hofhansl commented on HBASE-9754:
--
I meant that say that instead of a boolean, we can set if the passed readPt to
MAX_LONG when the above condition is true, and then avoid the MVCC check as
before when readPt==MAX_LONG. Would safe a parameter and also make sense
(IMHO). Not important, though.
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 9754-rp-0.txt
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793561#comment-13793561
]
Lars Hofhansl commented on HBASE-9754:
--
The idea is this:
* In HRegion.RegionScannerImpl.init pass the current readpoint to
store.getScanner(...)
* From there pass it to StoreScanner.init
* on to StoreScanner.getScannersNoCompaction(...)
* to Store.getScanners()
* and finally memstore.getScanners() and StoreFileScanner
.getScannersForStoreFiles(...)
This will need to tedious refactoring of all the StoreScanner constructors to
do the right thing in all cases, but in principle it is not difficult.
That way MVCC.setThreadReadPoint and MVCC.getThreadReadPoint will be
unnecessary and we can get rid of MVCC.perThreadReadPoint completely. Should be
a nice improvement for performance and readability.
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9754) Consider eliminating threadlocals from MVCC code
[
https://issues.apache.org/jira/browse/HBASE-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793562#comment-13793562
]
Enis Soztutar commented on HBASE-9754:
--
+1 to the idea.
Consider eliminating threadlocals from MVCC code
Key: HBASE-9754
URL: https://issues.apache.org/jira/browse/HBASE-9754
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Brought up by [~vrodionov] and [~yuzhih...@gmail.com].
Currently we use ThreadLocals to communicate the current readpoint between a
RegionScanner and the Store\{File}Scanner's down the stack.
Since ThreadLocals are not cheap we should consider whether it is possible to
pass the readpoint through the call stack instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
