[jira] [Commented] (HBASE-10447) Memstore flusher scans storefiles also when the scanner heap gets reset
[ https://issues.apache.org/jira/browse/HBASE-10447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888495#comment-13888495 ] ramkrishna.s.vasudevan commented on HBASE-10447: +1 for the flag. That is what I had in mind. Memstore flusher scans storefiles also when the scanner heap gets reset --- Key: HBASE-10447 URL: https://issues.apache.org/jira/browse/HBASE-10447 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 0.99.0, 0.96.1.1 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Priority: Blocker Fix For: 0.98.0, 0.96.2, 0.99.0 See the mail thread http://osdir.com/ml/general/2014-01/msg61294.html In case of flush we create a memstore flusher which in turn creates a StoreScanner backed by a Single ton MemstoreScanner. But this scanner also registers for any updates in the reader in the HStore. Is this needed? If this happens then any update on the reader may nullify the current heap and the entire Scanner Stack is reset, but this time with the other scanners for all the files that satisfies the last top key. So the flush that happens on the memstore holds the storefile scanners also in the heap that was recreated but originally the intention was to create a scanner on the memstore alone. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10443) IndexOutOfBoundExceptions when processing compressed tags in HFile
[ https://issues.apache.org/jira/browse/HBASE-10443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888494#comment-13888494 ] ramkrishna.s.vasudevan commented on HBASE-10443: Sorry..bit late to this today.. Will come up with a patch soon. IndexOutOfBoundExceptions when processing compressed tags in HFile -- Key: HBASE-10443 URL: https://issues.apache.org/jira/browse/HBASE-10443 Project: HBase Issue Type: Bug Affects Versions: 0.98.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Priority: Blocker Fix For: 0.98.0 As HBASE-10438 got closed, we still need to fix the Index out of bound exception that occurs. If we have a proper fix will fix this, if the bug was a false alarm would close this. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (HBASE-10451) Enable back Tag compression on HFiles
Anoop Sam John created HBASE-10451: -- Summary: Enable back Tag compression on HFiles Key: HBASE-10451 URL: https://issues.apache.org/jira/browse/HBASE-10451 Project: HBase Issue Type: Bug Reporter: Anoop Sam John Assignee: Anoop Sam John Fix For: 0.98.1 HBASE-10443 disables tag compression on HFiles. This Jira is to fix the issues we have found out in HBASE-10443 and enable it back. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10451) Enable back Tag compression on HFiles
[ https://issues.apache.org/jira/browse/HBASE-10451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888497#comment-13888497 ] Anoop Sam John commented on HBASE-10451: When the store file is being read, as part of reseek to a key if we find the if we have jumped a given key we do a move previous. Incase of DBEs we have already decoded this KV and so tags in that. Suppose this KV had some tags which are like not present in Dict already. So as part of decode we read the actual tag from file buffer and adds an entry into Dict. Now after moving to previous we once again decode this kv and so tags in that and again will make a duplicate entry into Dict (In next index). This makes the Dict structure totally gone. Enable back Tag compression on HFiles - Key: HBASE-10451 URL: https://issues.apache.org/jira/browse/HBASE-10451 Project: HBase Issue Type: Bug Reporter: Anoop Sam John Assignee: Anoop Sam John Fix For: 0.98.1 HBASE-10443 disables tag compression on HFiles. This Jira is to fix the issues we have found out in HBASE-10443 and enable it back. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HBASE-10451) Enable back Tag compression on HFiles
[ https://issues.apache.org/jira/browse/HBASE-10451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-10451: --- Priority: Critical (was: Major) Enable back Tag compression on HFiles - Key: HBASE-10451 URL: https://issues.apache.org/jira/browse/HBASE-10451 Project: HBase Issue Type: Bug Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Critical Fix For: 0.98.1 HBASE-10443 disables tag compression on HFiles. This Jira is to fix the issues we have found out in HBASE-10443 and enable it back. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10354) Add an API for defining consistency per request
[ https://issues.apache.org/jira/browse/HBASE-10354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888501#comment-13888501 ] Nicolas Liochon commented on HBASE-10354: - bq. But I think we should carry it over the PB Result as well I agree. 3 reasons for this: 1) This will be simpler for other clients 2) It will allow to have immutable Result objects (outside of the adapter layer), hence we will be able to share the objects again 3) Sending multiple calls is expensive (more calls to send, more calls to cancel). If the primary is under recovery, we could answer to the get requests with a 'stale' status. Then the client would not have to go to the secondary. This would be available even w/o secondary replica as well. Add an API for defining consistency per request --- Key: HBASE-10354 URL: https://issues.apache.org/jira/browse/HBASE-10354 Project: HBase Issue Type: Sub-task Components: Client Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.99.0 Attachments: hbase-10354_v1.patch We should add an API to be able to define the expected consistency level per operation. This API should also allow to check whether the results coming from a query (get or scan) is stale or not. The defaults should reflect the current semantics. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10451) Enable back Tag compression on HFiles
[ https://issues.apache.org/jira/browse/HBASE-10451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888502#comment-13888502 ] ramkrishna.s.vasudevan commented on HBASE-10451: Interesting part is if we have tags of the same size this does not come out as Index out of bound, but silently will give wrong tag for the given KV which is more dangerous. Enable back Tag compression on HFiles - Key: HBASE-10451 URL: https://issues.apache.org/jira/browse/HBASE-10451 Project: HBase Issue Type: Bug Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Critical Fix For: 0.98.1 HBASE-10443 disables tag compression on HFiles. This Jira is to fix the issues we have found out in HBASE-10443 and enable it back. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10449) Wrong execution pool configuration in HConnectionManager
[ https://issues.apache.org/jira/browse/HBASE-10449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888503#comment-13888503 ] Nicolas Liochon commented on HBASE-10449: - TestAcidGuarantees is likely unrelated. I will run it locally a few times to be sure. [~ndimiduk], [~saint@gmail.com], [~apurtell], what do you think? Should I commit the patch to trunk,.96 .98 as it is, if TestAcidGuarantees is proven to be unrelated? Wrong execution pool configuration in HConnectionManager Key: HBASE-10449 URL: https://issues.apache.org/jira/browse/HBASE-10449 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.98.0, 0.99.0, 0.96.1.1 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Critical Fix For: 0.98.0, 0.96.2, 0.99.0 Attachments: HBASE-10449.v1.patch There is a confusion in the configuration of the pool. The attached patch fixes this. This may change the client performances, as we were using a single thread. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HBASE-10443) IndexOutOfBoundExceptions when processing compressed tags in HFile
[ https://issues.apache.org/jira/browse/HBASE-10443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-10443: --- Attachment: HBASE-10443.patch Patch that defaults COMPRESS_TAGs to false and also logs a msg before setting to HFileContext if found to be true. Should we add this in book also? IndexOutOfBoundExceptions when processing compressed tags in HFile -- Key: HBASE-10443 URL: https://issues.apache.org/jira/browse/HBASE-10443 Project: HBase Issue Type: Bug Affects Versions: 0.98.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Priority: Blocker Fix For: 0.98.0 Attachments: HBASE-10443.patch As HBASE-10438 got closed, we still need to fix the Index out of bound exception that occurs. If we have a proper fix will fix this, if the bug was a false alarm would close this. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HBASE-10443) IndexOutOfBoundExceptions when processing compressed tags in HFile
[ https://issues.apache.org/jira/browse/HBASE-10443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-10443: --- Status: Patch Available (was: Open) IndexOutOfBoundExceptions when processing compressed tags in HFile -- Key: HBASE-10443 URL: https://issues.apache.org/jira/browse/HBASE-10443 Project: HBase Issue Type: Bug Affects Versions: 0.98.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Priority: Blocker Fix For: 0.98.0 Attachments: HBASE-10443.patch As HBASE-10438 got closed, we still need to fix the Index out of bound exception that occurs. If we have a proper fix will fix this, if the bug was a false alarm would close this. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10443) IndexOutOfBoundExceptions when processing compressed tags in HFile
[
https://issues.apache.org/jira/browse/HBASE-10443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888548#comment-13888548
]
Hadoop QA commented on HBASE-10443:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12626465/HBASE-10443.patch
against trunk revision .
ATTACHMENT ID: 12626465
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8571//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8571//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8571//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8571//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8571//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8571//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8571//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8571//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8571//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8571//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8571//console
This message is automatically generated.
IndexOutOfBoundExceptions when processing compressed tags in HFile
--
Key: HBASE-10443
URL: https://issues.apache.org/jira/browse/HBASE-10443
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Blocker
Fix For: 0.98.0
Attachments: HBASE-10443.patch
As HBASE-10438 got closed, we still need to fix the Index out of bound
exception that occurs. If we have a proper fix will fix this, if the bug was
a false alarm would close this.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-10447) Memstore flusher scans storefiles also when the scanner heap gets reset
[ https://issues.apache.org/jira/browse/HBASE-10447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-10447: --- Attachment: HBASE-10447_0.98.patch Patch for 0.98. Will do some more testing once. Memstore flusher scans storefiles also when the scanner heap gets reset --- Key: HBASE-10447 URL: https://issues.apache.org/jira/browse/HBASE-10447 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 0.99.0, 0.96.1.1 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Priority: Blocker Fix For: 0.98.0, 0.96.2, 0.99.0 Attachments: HBASE-10447_0.98.patch See the mail thread http://osdir.com/ml/general/2014-01/msg61294.html In case of flush we create a memstore flusher which in turn creates a StoreScanner backed by a Single ton MemstoreScanner. But this scanner also registers for any updates in the reader in the HStore. Is this needed? If this happens then any update on the reader may nullify the current heap and the entire Scanner Stack is reset, but this time with the other scanners for all the files that satisfies the last top key. So the flush that happens on the memstore holds the storefile scanners also in the heap that was recreated but originally the intention was to create a scanner on the memstore alone. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10443) IndexOutOfBoundExceptions when processing compressed tags in HFile
[ https://issues.apache.org/jira/browse/HBASE-10443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888614#comment-13888614 ] Anoop Sam John commented on HBASE-10443: +1 IndexOutOfBoundExceptions when processing compressed tags in HFile -- Key: HBASE-10443 URL: https://issues.apache.org/jira/browse/HBASE-10443 Project: HBase Issue Type: Bug Affects Versions: 0.98.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Priority: Blocker Fix For: 0.98.0 Attachments: HBASE-10443.patch As HBASE-10438 got closed, we still need to fix the Index out of bound exception that occurs. If we have a proper fix will fix this, if the bug was a false alarm would close this. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10449) Wrong execution pool configuration in HConnectionManager
[ https://issues.apache.org/jira/browse/HBASE-10449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888666#comment-13888666 ] Andrew Purtell commented on HBASE-10449: +1 Wrong execution pool configuration in HConnectionManager Key: HBASE-10449 URL: https://issues.apache.org/jira/browse/HBASE-10449 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.98.0, 0.99.0, 0.96.1.1 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Critical Fix For: 0.98.0, 0.96.2, 0.99.0 Attachments: HBASE-10449.v1.patch There is a confusion in the configuration of the pool. The attached patch fixes this. This may change the client performances, as we were using a single thread. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HBASE-10448) ZKUtil create and watch methods don't set watch in some cases
[
https://issues.apache.org/jira/browse/HBASE-10448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-10448:
---
Fix Version/s: (was: 0.98.1)
0.99.0
0.98.0
bq. A guess is that they don't really depend on the watches being set in the
cases that the watches are not set.
You are right, I don't follow how these *AndWatch methods lived up to the
expectations of their naming.
+1
ZKUtil create and watch methods don't set watch in some cases
-
Key: HBASE-10448
URL: https://issues.apache.org/jira/browse/HBASE-10448
Project: HBase
Issue Type: Bug
Components: Zookeeper
Affects Versions: 0.96.0, 0.96.1.1
Reporter: Jerry He
Fix For: 0.98.0, 0.99.0
Attachments: HBASE-10448-trunk.patch
While using the ZKUtil methods during testing, I found that watch was not set
when it should be set based on the methods and method comments:
createNodeIfNotExistsAndWatch
createEphemeralNodeAndWatch
For example, in createNodeIfNotExistsAndWatch():
{code}
public static boolean createNodeIfNotExistsAndWatch(
ZooKeeperWatcher zkw, String znode, byte [] data)
throws KeeperException {
try {
zkw.getRecoverableZooKeeper().create(znode, data, createACL(zkw, znode),
CreateMode.PERSISTENT);
} catch (KeeperException.NodeExistsException nee) {
try {
zkw.getRecoverableZooKeeper().exists(znode, zkw);
} catch (InterruptedException e) {
zkw.interruptedException(e);
return false;
}
return false;
} catch (InterruptedException e) {
zkw.interruptedException(e);
return false;
}
return true;
}
{code}
The watch is only set via exists() call when the node already exists.
Similarly in createEphemeralNodeAndWatch():
{code}
public static boolean createEphemeralNodeAndWatch(ZooKeeperWatcher zkw,
String znode, byte [] data)
throws KeeperException {
try {
zkw.getRecoverableZooKeeper().create(znode, data, createACL(zkw, znode),
CreateMode.EPHEMERAL);
} catch (KeeperException.NodeExistsException nee) {
if(!watchAndCheckExists(zkw, znode)) {
// It did exist but now it doesn't, try again
return createEphemeralNodeAndWatch(zkw, znode, data);
}
return false;
} catch (InterruptedException e) {
LOG.info(Interrupted, e);
Thread.currentThread().interrupt();
}
return true;
}
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10445) TestRegionObserverInterface occasionally times out
[
https://issues.apache.org/jira/browse/HBASE-10445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888684#comment-13888684
]
Andrew Purtell commented on HBASE-10445:
Ted, could you respond to the question I actually asked? Thanks.
TestRegionObserverInterface occasionally times out
--
Key: HBASE-10445
URL: https://issues.apache.org/jira/browse/HBASE-10445
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10445-alternate.patch, 10445-v1.txt,
TestRegionObserverInterface-output-2.txt,
TestRegionObserverInterface-output-3.txt
TestRegionObserverInterface occasionally times out
Running in a loop, it timed out at 9th iteration twice.
The test starts a cluster with 1 region server. If this server goes down, the
following message would be repeatedly printed:
{code}
2014-01-30 00:35:16,144 WARN [MASTER_META_SERVER_OPERATIONS-kiyo:42930-0]
master.AssignmentManager(2140): Can't move 1588230740, there is no
destination server available.
2014-01-30 00:35:16,144 WARN [MASTER_META_SERVER_OPERATIONS-kiyo:42930-0]
master.AssignmentManager(1863): Unable to determine a plan to assign {ENCODED
= 1588230740, NAME = 'hbase:meta,,1', STARTKEY = '', ENDKEY = ''}
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10451) Enable back Tag compression on HFiles
[ https://issues.apache.org/jira/browse/HBASE-10451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888686#comment-13888686 ] Andrew Purtell commented on HBASE-10451: bq. Interesting part is if we have tags of the same size this does not come out as Index out of bound, but silently will give wrong tag for the given KV which is more dangerous. Agree, and this is how we came so far (in tests) without seeing the issue. Enable back Tag compression on HFiles - Key: HBASE-10451 URL: https://issues.apache.org/jira/browse/HBASE-10451 Project: HBase Issue Type: Bug Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Critical Fix For: 0.98.1 HBASE-10443 disables tag compression on HFiles. This Jira is to fix the issues we have found out in HBASE-10443 and enable it back. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HBASE-10447) Memstore flusher scans storefiles also when the scanner heap gets reset
[ https://issues.apache.org/jira/browse/HBASE-10447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-10447: --- Status: Patch Available (was: Open) HadoopQA might be able to apply this against trunk. Memstore flusher scans storefiles also when the scanner heap gets reset --- Key: HBASE-10447 URL: https://issues.apache.org/jira/browse/HBASE-10447 Project: HBase Issue Type: Bug Affects Versions: 0.96.1.1, 0.98.0, 0.99.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Priority: Blocker Fix For: 0.98.0, 0.96.2, 0.99.0 Attachments: HBASE-10447_0.98.patch See the mail thread http://osdir.com/ml/general/2014-01/msg61294.html In case of flush we create a memstore flusher which in turn creates a StoreScanner backed by a Single ton MemstoreScanner. But this scanner also registers for any updates in the reader in the HStore. Is this needed? If this happens then any update on the reader may nullify the current heap and the entire Scanner Stack is reset, but this time with the other scanners for all the files that satisfies the last top key. So the flush that happens on the memstore holds the storefile scanners also in the heap that was recreated but originally the intention was to create a scanner on the memstore alone. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10447) Memstore flusher scans storefiles also when the scanner heap gets reset
[ https://issues.apache.org/jira/browse/HBASE-10447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888688#comment-13888688 ] Andrew Purtell commented on HBASE-10447: +1, patch looks like it would work. Anyone have an issue with passing down a new flag? Memstore flusher scans storefiles also when the scanner heap gets reset --- Key: HBASE-10447 URL: https://issues.apache.org/jira/browse/HBASE-10447 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 0.99.0, 0.96.1.1 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Priority: Blocker Fix For: 0.98.0, 0.96.2, 0.99.0 Attachments: HBASE-10447_0.98.patch See the mail thread http://osdir.com/ml/general/2014-01/msg61294.html In case of flush we create a memstore flusher which in turn creates a StoreScanner backed by a Single ton MemstoreScanner. But this scanner also registers for any updates in the reader in the HStore. Is this needed? If this happens then any update on the reader may nullify the current heap and the entire Scanner Stack is reset, but this time with the other scanners for all the files that satisfies the last top key. So the flush that happens on the memstore holds the storefile scanners also in the heap that was recreated but originally the intention was to create a scanner on the memstore alone. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10443) IndexOutOfBoundExceptions when processing compressed tags in HFile
[
https://issues.apache.org/jira/browse/HBASE-10443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888693#comment-13888693
]
Andrew Purtell commented on HBASE-10443:
The patch is fine.
We don't need a book update.
{noformat}
+if(family.shouldCompressTags()) {
+ LOG.warn(Compress tags in HFiles disabled in 0.98.0);
+}
{noformat}
Don't put in version specific messages. Instead, do something like this:
{noformat}
+if(family.shouldCompressTags()) {
+ LOG.warn(HFile tag compression attribute ignored for ' +
family.getNameAsString() + ', see HBASE-10443);
+}
{noformat}
Can be fixed up at commit time.
IndexOutOfBoundExceptions when processing compressed tags in HFile
--
Key: HBASE-10443
URL: https://issues.apache.org/jira/browse/HBASE-10443
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Blocker
Fix For: 0.98.0
Attachments: HBASE-10443.patch
As HBASE-10438 got closed, we still need to fix the Index out of bound
exception that occurs. If we have a proper fix will fix this, if the bug was
a false alarm would close this.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10445) TestRegionObserverInterface occasionally times out
[
https://issues.apache.org/jira/browse/HBASE-10445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888706#comment-13888706
]
Ted Yu commented on HBASE-10445:
I am running TestRegionObserverInterface with that change in a loop.
TestRegionObserverInterface occasionally times out
--
Key: HBASE-10445
URL: https://issues.apache.org/jira/browse/HBASE-10445
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10445-alternate.patch, 10445-v1.txt,
TestRegionObserverInterface-output-2.txt,
TestRegionObserverInterface-output-3.txt
TestRegionObserverInterface occasionally times out
Running in a loop, it timed out at 9th iteration twice.
The test starts a cluster with 1 region server. If this server goes down, the
following message would be repeatedly printed:
{code}
2014-01-30 00:35:16,144 WARN [MASTER_META_SERVER_OPERATIONS-kiyo:42930-0]
master.AssignmentManager(2140): Can't move 1588230740, there is no
destination server available.
2014-01-30 00:35:16,144 WARN [MASTER_META_SERVER_OPERATIONS-kiyo:42930-0]
master.AssignmentManager(1863): Unable to determine a plan to assign {ENCODED
= 1588230740, NAME = 'hbase:meta,,1', STARTKEY = '', ENDKEY = ''}
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10448) ZKUtil create and watch methods don't set watch in some cases
[
https://issues.apache.org/jira/browse/HBASE-10448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888710#comment-13888710
]
Ted Yu commented on HBASE-10448:
I tried to apply patch on 0.98 but there was conflict in
hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java.rej
Also, please trim the trailing ^M at line end.
Thanks
ZKUtil create and watch methods don't set watch in some cases
-
Key: HBASE-10448
URL: https://issues.apache.org/jira/browse/HBASE-10448
Project: HBase
Issue Type: Bug
Components: Zookeeper
Affects Versions: 0.96.0, 0.96.1.1
Reporter: Jerry He
Fix For: 0.98.0, 0.99.0
Attachments: HBASE-10448-trunk.patch
While using the ZKUtil methods during testing, I found that watch was not set
when it should be set based on the methods and method comments:
createNodeIfNotExistsAndWatch
createEphemeralNodeAndWatch
For example, in createNodeIfNotExistsAndWatch():
{code}
public static boolean createNodeIfNotExistsAndWatch(
ZooKeeperWatcher zkw, String znode, byte [] data)
throws KeeperException {
try {
zkw.getRecoverableZooKeeper().create(znode, data, createACL(zkw, znode),
CreateMode.PERSISTENT);
} catch (KeeperException.NodeExistsException nee) {
try {
zkw.getRecoverableZooKeeper().exists(znode, zkw);
} catch (InterruptedException e) {
zkw.interruptedException(e);
return false;
}
return false;
} catch (InterruptedException e) {
zkw.interruptedException(e);
return false;
}
return true;
}
{code}
The watch is only set via exists() call when the node already exists.
Similarly in createEphemeralNodeAndWatch():
{code}
public static boolean createEphemeralNodeAndWatch(ZooKeeperWatcher zkw,
String znode, byte [] data)
throws KeeperException {
try {
zkw.getRecoverableZooKeeper().create(znode, data, createACL(zkw, znode),
CreateMode.EPHEMERAL);
} catch (KeeperException.NodeExistsException nee) {
if(!watchAndCheckExists(zkw, znode)) {
// It did exist but now it doesn't, try again
return createEphemeralNodeAndWatch(zkw, znode, data);
}
return false;
} catch (InterruptedException e) {
LOG.info(Interrupted, e);
Thread.currentThread().interrupt();
}
return true;
}
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10447) Memstore flusher scans storefiles also when the scanner heap gets reset
[
https://issues.apache.org/jira/browse/HBASE-10447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888742#comment-13888742
]
Hadoop QA commented on HBASE-10447:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12626475/HBASE-10447_0.98.patch
against trunk revision .
ATTACHMENT ID: 12626475
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 12 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ byte[] dropDeletesFromRow, byte[] dropDeletesToRow, boolean
scannerForFlush) throws IOException {
+ long earliestPutTs, byte[] dropDeletesFromRow, byte[] dropDeletesToRow,
boolean scannerForFlush)
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hbase.TestAcidGuarantees.testMixedAtomicity(TestAcidGuarantees.java:351)
at
org.apache.hadoop.hbase.io.encoding.TestEncodedSeekers.testEncodedSeeker(TestEncodedSeekers.java:129)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8572//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8572//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8572//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8572//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8572//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8572//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8572//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8572//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8572//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8572//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8572//console
This message is automatically generated.
Memstore flusher scans storefiles also when the scanner heap gets reset
---
Key: HBASE-10447
URL: https://issues.apache.org/jira/browse/HBASE-10447
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.0, 0.99.0, 0.96.1.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Blocker
Fix For: 0.98.0, 0.96.2, 0.99.0
Attachments: HBASE-10447_0.98.patch
See the mail thread
http://osdir.com/ml/general/2014-01/msg61294.html
In case of flush we create a memstore flusher which in turn creates a
StoreScanner backed by a Single ton MemstoreScanner.
But this scanner also registers for any updates in the reader in the HStore.
Is this needed?
If this happens then any update on the reader may nullify the current heap
and the entire Scanner Stack is reset, but this time with the other scanners
for all the files that satisfies the last top key. So the flush that happens
on the memstore holds the storefile scanners also in the heap that was
recreated but originally the intention was to create a scanner on the
memstore alone.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-10448) ZKUtil create and watch methods don't set watch in some cases
[
https://issues.apache.org/jira/browse/HBASE-10448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-10448:
---
Status: Open (was: Patch Available)
ZKUtil create and watch methods don't set watch in some cases
-
Key: HBASE-10448
URL: https://issues.apache.org/jira/browse/HBASE-10448
Project: HBase
Issue Type: Bug
Components: Zookeeper
Affects Versions: 0.96.1.1, 0.96.0
Reporter: Jerry He
Fix For: 0.98.0, 0.99.0
Attachments: HBASE-10448-trunk.patch
While using the ZKUtil methods during testing, I found that watch was not set
when it should be set based on the methods and method comments:
createNodeIfNotExistsAndWatch
createEphemeralNodeAndWatch
For example, in createNodeIfNotExistsAndWatch():
{code}
public static boolean createNodeIfNotExistsAndWatch(
ZooKeeperWatcher zkw, String znode, byte [] data)
throws KeeperException {
try {
zkw.getRecoverableZooKeeper().create(znode, data, createACL(zkw, znode),
CreateMode.PERSISTENT);
} catch (KeeperException.NodeExistsException nee) {
try {
zkw.getRecoverableZooKeeper().exists(znode, zkw);
} catch (InterruptedException e) {
zkw.interruptedException(e);
return false;
}
return false;
} catch (InterruptedException e) {
zkw.interruptedException(e);
return false;
}
return true;
}
{code}
The watch is only set via exists() call when the node already exists.
Similarly in createEphemeralNodeAndWatch():
{code}
public static boolean createEphemeralNodeAndWatch(ZooKeeperWatcher zkw,
String znode, byte [] data)
throws KeeperException {
try {
zkw.getRecoverableZooKeeper().create(znode, data, createACL(zkw, znode),
CreateMode.EPHEMERAL);
} catch (KeeperException.NodeExistsException nee) {
if(!watchAndCheckExists(zkw, znode)) {
// It did exist but now it doesn't, try again
return createEphemeralNodeAndWatch(zkw, znode, data);
}
return false;
} catch (InterruptedException e) {
LOG.info(Interrupted, e);
Thread.currentThread().interrupt();
}
return true;
}
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Resolved] (HBASE-10448) ZKUtil create and watch methods don't set watch in some cases
[
https://issues.apache.org/jira/browse/HBASE-10448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HBASE-10448.
Resolution: Fixed
Assignee: Jerry He
Hadoop Flags: Reviewed
Integrated to 0.98 and trunk.
Thanks for the patch, Jerry.
ZKUtil create and watch methods don't set watch in some cases
-
Key: HBASE-10448
URL: https://issues.apache.org/jira/browse/HBASE-10448
Project: HBase
Issue Type: Bug
Components: Zookeeper
Affects Versions: 0.96.0, 0.96.1.1
Reporter: Jerry He
Assignee: Jerry He
Fix For: 0.98.0, 0.99.0
Attachments: HBASE-10448-trunk.patch
While using the ZKUtil methods during testing, I found that watch was not set
when it should be set based on the methods and method comments:
createNodeIfNotExistsAndWatch
createEphemeralNodeAndWatch
For example, in createNodeIfNotExistsAndWatch():
{code}
public static boolean createNodeIfNotExistsAndWatch(
ZooKeeperWatcher zkw, String znode, byte [] data)
throws KeeperException {
try {
zkw.getRecoverableZooKeeper().create(znode, data, createACL(zkw, znode),
CreateMode.PERSISTENT);
} catch (KeeperException.NodeExistsException nee) {
try {
zkw.getRecoverableZooKeeper().exists(znode, zkw);
} catch (InterruptedException e) {
zkw.interruptedException(e);
return false;
}
return false;
} catch (InterruptedException e) {
zkw.interruptedException(e);
return false;
}
return true;
}
{code}
The watch is only set via exists() call when the node already exists.
Similarly in createEphemeralNodeAndWatch():
{code}
public static boolean createEphemeralNodeAndWatch(ZooKeeperWatcher zkw,
String znode, byte [] data)
throws KeeperException {
try {
zkw.getRecoverableZooKeeper().create(znode, data, createACL(zkw, znode),
CreateMode.EPHEMERAL);
} catch (KeeperException.NodeExistsException nee) {
if(!watchAndCheckExists(zkw, znode)) {
// It did exist but now it doesn't, try again
return createEphemeralNodeAndWatch(zkw, znode, data);
}
return false;
} catch (InterruptedException e) {
LOG.info(Interrupted, e);
Thread.currentThread().interrupt();
}
return true;
}
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10448) ZKUtil create and watch methods don't set watch in some cases
[
https://issues.apache.org/jira/browse/HBASE-10448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888790#comment-13888790
]
Hudson commented on HBASE-10448:
SUCCESS: Integrated in HBase-TRUNK #4873 (See
[https://builds.apache.org/job/HBase-TRUNK/4873/])
HBASE-10448 ZKUtil create and watch methods don't set watch in some cases
(Jerry He) (tedyu: rev 1563507)
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
ZKUtil create and watch methods don't set watch in some cases
-
Key: HBASE-10448
URL: https://issues.apache.org/jira/browse/HBASE-10448
Project: HBase
Issue Type: Bug
Components: Zookeeper
Affects Versions: 0.96.0, 0.96.1.1
Reporter: Jerry He
Assignee: Jerry He
Fix For: 0.98.0, 0.99.0
Attachments: HBASE-10448-trunk.patch
While using the ZKUtil methods during testing, I found that watch was not set
when it should be set based on the methods and method comments:
createNodeIfNotExistsAndWatch
createEphemeralNodeAndWatch
For example, in createNodeIfNotExistsAndWatch():
{code}
public static boolean createNodeIfNotExistsAndWatch(
ZooKeeperWatcher zkw, String znode, byte [] data)
throws KeeperException {
try {
zkw.getRecoverableZooKeeper().create(znode, data, createACL(zkw, znode),
CreateMode.PERSISTENT);
} catch (KeeperException.NodeExistsException nee) {
try {
zkw.getRecoverableZooKeeper().exists(znode, zkw);
} catch (InterruptedException e) {
zkw.interruptedException(e);
return false;
}
return false;
} catch (InterruptedException e) {
zkw.interruptedException(e);
return false;
}
return true;
}
{code}
The watch is only set via exists() call when the node already exists.
Similarly in createEphemeralNodeAndWatch():
{code}
public static boolean createEphemeralNodeAndWatch(ZooKeeperWatcher zkw,
String znode, byte [] data)
throws KeeperException {
try {
zkw.getRecoverableZooKeeper().create(znode, data, createACL(zkw, znode),
CreateMode.EPHEMERAL);
} catch (KeeperException.NodeExistsException nee) {
if(!watchAndCheckExists(zkw, znode)) {
// It did exist but now it doesn't, try again
return createEphemeralNodeAndWatch(zkw, znode, data);
}
return false;
} catch (InterruptedException e) {
LOG.info(Interrupted, e);
Thread.currentThread().interrupt();
}
return true;
}
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10448) ZKUtil create and watch methods don't set watch in some cases
[
https://issues.apache.org/jira/browse/HBASE-10448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888792#comment-13888792
]
Jerry He commented on HBASE-10448:
--
Thanks, Ted, Andrew.
Could you also mark HBASE-8937 as resolved as dup so that we have closure on
that one too?
ZKUtil create and watch methods don't set watch in some cases
-
Key: HBASE-10448
URL: https://issues.apache.org/jira/browse/HBASE-10448
Project: HBase
Issue Type: Bug
Components: Zookeeper
Affects Versions: 0.96.0, 0.96.1.1
Reporter: Jerry He
Assignee: Jerry He
Fix For: 0.98.0, 0.99.0
Attachments: HBASE-10448-trunk.patch
While using the ZKUtil methods during testing, I found that watch was not set
when it should be set based on the methods and method comments:
createNodeIfNotExistsAndWatch
createEphemeralNodeAndWatch
For example, in createNodeIfNotExistsAndWatch():
{code}
public static boolean createNodeIfNotExistsAndWatch(
ZooKeeperWatcher zkw, String znode, byte [] data)
throws KeeperException {
try {
zkw.getRecoverableZooKeeper().create(znode, data, createACL(zkw, znode),
CreateMode.PERSISTENT);
} catch (KeeperException.NodeExistsException nee) {
try {
zkw.getRecoverableZooKeeper().exists(znode, zkw);
} catch (InterruptedException e) {
zkw.interruptedException(e);
return false;
}
return false;
} catch (InterruptedException e) {
zkw.interruptedException(e);
return false;
}
return true;
}
{code}
The watch is only set via exists() call when the node already exists.
Similarly in createEphemeralNodeAndWatch():
{code}
public static boolean createEphemeralNodeAndWatch(ZooKeeperWatcher zkw,
String znode, byte [] data)
throws KeeperException {
try {
zkw.getRecoverableZooKeeper().create(znode, data, createACL(zkw, znode),
CreateMode.EPHEMERAL);
} catch (KeeperException.NodeExistsException nee) {
if(!watchAndCheckExists(zkw, znode)) {
// It did exist but now it doesn't, try again
return createEphemeralNodeAndWatch(zkw, znode, data);
}
return false;
} catch (InterruptedException e) {
LOG.info(Interrupted, e);
Thread.currentThread().interrupt();
}
return true;
}
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10440) integration tests fail due to nonce collisions
[ https://issues.apache.org/jira/browse/HBASE-10440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888795#comment-13888795 ] Hudson commented on HBASE-10440: FAILURE: Integrated in HBase-0.98 #119 (See [https://builds.apache.org/job/HBase-0.98/119/]) HBASE-10440 Integration tests fail due to nonce collisions (Sergey Shelukhin) (apurtell: rev 1562677) * /hbase/branches/0.98/hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestIngest.java * /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java * /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/util/MultiThreadedUpdater.java integration tests fail due to nonce collisions -- Key: HBASE-10440 URL: https://issues.apache.org/jira/browse/HBASE-10440 Project: HBase Issue Type: Test Affects Versions: 0.98.0, 0.99.0 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.98.0, 0.99.0 Attachments: HBASE-10440.patch Before HBASE-9899 is implemented, and after HBASE-3787, HBase throws OperationConflictException when client retries an already-successful non-idempotent request because the response didn't reach the client. Integration tests run into this when CM kills servers hard during relatively-recently-added appends and increments. They need to handle this, read verification would make sure the results are still correct -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10442) prepareDelete() isn't called before doPreMutationHook for a row deletion case
[ https://issues.apache.org/jira/browse/HBASE-10442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888796#comment-13888796 ] Hudson commented on HBASE-10442: FAILURE: Integrated in HBase-0.98 #119 (See [https://builds.apache.org/job/HBase-0.98/119/]) HBASE-10442 prepareDelete() isn't called before doPreMutationHook for a row deletion case (Jeffrey Zhong) (apurtell: rev 1562674) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java prepareDelete() isn't called before doPreMutationHook for a row deletion case - Key: HBASE-10442 URL: https://issues.apache.org/jira/browse/HBASE-10442 Project: HBase Issue Type: Bug Components: Coprocessors Affects Versions: 0.98.0, 0.96.0, 0.96.1 Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Priority: Critical Fix For: 0.98.0, 0.96.2, 0.99.0 Attachments: hbase-10442.patch This is a backward incompatibility issue for coprocessors so I marked as critical. When deleting a row, prepareDelete() will add delete marker for all column families before preDelete is called in hbase0.94 while in hbase0.96 and later version we invoke prepareDelete() after preDelete() hook is invoked but before postDelete() is called. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10448) ZKUtil create and watch methods don't set watch in some cases
[
https://issues.apache.org/jira/browse/HBASE-10448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888797#comment-13888797
]
Hudson commented on HBASE-10448:
FAILURE: Integrated in HBase-0.98 #119 (See
[https://builds.apache.org/job/HBase-0.98/119/])
HBASE-10448 ZKUtil create and watch methods don't set watch in some cases
(Jerry He) (tedyu: rev 1563506)
*
/hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
ZKUtil create and watch methods don't set watch in some cases
-
Key: HBASE-10448
URL: https://issues.apache.org/jira/browse/HBASE-10448
Project: HBase
Issue Type: Bug
Components: Zookeeper
Affects Versions: 0.96.0, 0.96.1.1
Reporter: Jerry He
Assignee: Jerry He
Fix For: 0.98.0, 0.99.0
Attachments: HBASE-10448-trunk.patch
While using the ZKUtil methods during testing, I found that watch was not set
when it should be set based on the methods and method comments:
createNodeIfNotExistsAndWatch
createEphemeralNodeAndWatch
For example, in createNodeIfNotExistsAndWatch():
{code}
public static boolean createNodeIfNotExistsAndWatch(
ZooKeeperWatcher zkw, String znode, byte [] data)
throws KeeperException {
try {
zkw.getRecoverableZooKeeper().create(znode, data, createACL(zkw, znode),
CreateMode.PERSISTENT);
} catch (KeeperException.NodeExistsException nee) {
try {
zkw.getRecoverableZooKeeper().exists(znode, zkw);
} catch (InterruptedException e) {
zkw.interruptedException(e);
return false;
}
return false;
} catch (InterruptedException e) {
zkw.interruptedException(e);
return false;
}
return true;
}
{code}
The watch is only set via exists() call when the node already exists.
Similarly in createEphemeralNodeAndWatch():
{code}
public static boolean createEphemeralNodeAndWatch(ZooKeeperWatcher zkw,
String znode, byte [] data)
throws KeeperException {
try {
zkw.getRecoverableZooKeeper().create(znode, data, createACL(zkw, znode),
CreateMode.EPHEMERAL);
} catch (KeeperException.NodeExistsException nee) {
if(!watchAndCheckExists(zkw, znode)) {
// It did exist but now it doesn't, try again
return createEphemeralNodeAndWatch(zkw, znode, data);
}
return false;
} catch (InterruptedException e) {
LOG.info(Interrupted, e);
Thread.currentThread().interrupt();
}
return true;
}
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10448) ZKUtil create and watch methods don't set watch in some cases
[
https://issues.apache.org/jira/browse/HBASE-10448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1300#comment-1300
]
Hudson commented on HBASE-10448:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #111 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/111/])
HBASE-10448 ZKUtil create and watch methods don't set watch in some cases
(Jerry He) (tedyu: rev 1563506)
*
/hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
ZKUtil create and watch methods don't set watch in some cases
-
Key: HBASE-10448
URL: https://issues.apache.org/jira/browse/HBASE-10448
Project: HBase
Issue Type: Bug
Components: Zookeeper
Affects Versions: 0.96.0, 0.96.1.1
Reporter: Jerry He
Assignee: Jerry He
Fix For: 0.98.0, 0.99.0
Attachments: HBASE-10448-trunk.patch
While using the ZKUtil methods during testing, I found that watch was not set
when it should be set based on the methods and method comments:
createNodeIfNotExistsAndWatch
createEphemeralNodeAndWatch
For example, in createNodeIfNotExistsAndWatch():
{code}
public static boolean createNodeIfNotExistsAndWatch(
ZooKeeperWatcher zkw, String znode, byte [] data)
throws KeeperException {
try {
zkw.getRecoverableZooKeeper().create(znode, data, createACL(zkw, znode),
CreateMode.PERSISTENT);
} catch (KeeperException.NodeExistsException nee) {
try {
zkw.getRecoverableZooKeeper().exists(znode, zkw);
} catch (InterruptedException e) {
zkw.interruptedException(e);
return false;
}
return false;
} catch (InterruptedException e) {
zkw.interruptedException(e);
return false;
}
return true;
}
{code}
The watch is only set via exists() call when the node already exists.
Similarly in createEphemeralNodeAndWatch():
{code}
public static boolean createEphemeralNodeAndWatch(ZooKeeperWatcher zkw,
String znode, byte [] data)
throws KeeperException {
try {
zkw.getRecoverableZooKeeper().create(znode, data, createACL(zkw, znode),
CreateMode.EPHEMERAL);
} catch (KeeperException.NodeExistsException nee) {
if(!watchAndCheckExists(zkw, znode)) {
// It did exist but now it doesn't, try again
return createEphemeralNodeAndWatch(zkw, znode, data);
}
return false;
} catch (InterruptedException e) {
LOG.info(Interrupted, e);
Thread.currentThread().interrupt();
}
return true;
}
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-9830) Backport HBASE-9605 to 0.94
[ https://issues.apache.org/jira/browse/HBASE-9830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1317#comment-1317 ] Lars Hofhansl commented on HBASE-9830: -- .bq I don't think 0.96 has Aggregation CP Lars Hofhansl. It was removed so bigtop could compile. Maybe I should put it back? Are you sure it's gone from 0.96 only? I.e. 0.94 has it and 0.98 has it. That would be weird. Let's check and close this one. Backport HBASE-9605 to 0.94 --- Key: HBASE-9830 URL: https://issues.apache.org/jira/browse/HBASE-9830 Project: HBase Issue Type: Improvement Affects Versions: 0.94.3 Reporter: chendihao Assignee: chendihao Priority: Minor Fix For: 0.94.17 Attachments: HBASE-9830-0.94-v1.patch Backport HBASE-9605 which is about Allow AggregationClient to skip specifying column family for row count aggregate -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-9830) Backport HBASE-9605 to 0.94
[ https://issues.apache.org/jira/browse/HBASE-9830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1334#comment-1334 ] stack commented on HBASE-9830: -- [~lhofhansl] Removing aggregation was only way I could make hbase trunk work on bigtop. Bigtop would fail compile in a way that Roman and I could not figure. Undoing it freed us up but then we didn't go much further than a compile... Backport HBASE-9605 to 0.94 --- Key: HBASE-9830 URL: https://issues.apache.org/jira/browse/HBASE-9830 Project: HBase Issue Type: Improvement Affects Versions: 0.94.3 Reporter: chendihao Assignee: chendihao Priority: Minor Fix For: 0.94.17 Attachments: HBASE-9830-0.94-v1.patch Backport HBASE-9605 which is about Allow AggregationClient to skip specifying column family for row count aggregate -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Resolved] (HBASE-8937) createEphemeralNodeAndWatch don't set watcher if the node is created successfully
[
https://issues.apache.org/jira/browse/HBASE-8937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-8937.
--
Resolution: Duplicate
Resolving as dup of HBASE-10448 Reopen please if I have it wrong (At
suggestion of our [~jerryhe])
createEphemeralNodeAndWatch don't set watcher if the node is created
successfully
-
Key: HBASE-8937
URL: https://issues.apache.org/jira/browse/HBASE-8937
Project: HBase
Issue Type: Bug
Components: master
Reporter: Liu Shaohui
Priority: Minor
Labels: master, wacter
CreateEphemeralNodeAndWatch in zkUtil don't set watcher if the node is
created successfully. This is not consistent with the comment and may causes
the ActiveMasterManager cannot get events that master node is deleted or
changed.
{code}
public static boolean createEphemeralNodeAndWatch(ZooKeeperWatcher zkw,
String znode, byte [] data)
throws KeeperException {
try {
zkw.getRecoverableZooKeeper().create(znode, data, createACL(zkw, znode),
CreateMode.EPHEMERAL);
} catch (KeeperException.NodeExistsException nee) {
if(!watchAndCheckExists(zkw, znode)) {
// It did exist but now it doesn't, try again
return createEphemeralNodeAndWatch(zkw, znode, data);
}
return false;
} catch (InterruptedException e) {
LOG.info(Interrupted, e);
Thread.currentThread().interrupt();
}
return true;
}
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-10448) ZKUtil create and watch methods don't set watch in some cases
[
https://issues.apache.org/jira/browse/HBASE-10448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-10448:
--
Fix Version/s: 0.96.2
Good one [~jerryhe] Committed to 0.96 branch.
ZKUtil create and watch methods don't set watch in some cases
-
Key: HBASE-10448
URL: https://issues.apache.org/jira/browse/HBASE-10448
Project: HBase
Issue Type: Bug
Components: Zookeeper
Affects Versions: 0.96.0, 0.96.1.1
Reporter: Jerry He
Assignee: Jerry He
Fix For: 0.98.0, 0.96.2, 0.99.0
Attachments: HBASE-10448-trunk.patch
While using the ZKUtil methods during testing, I found that watch was not set
when it should be set based on the methods and method comments:
createNodeIfNotExistsAndWatch
createEphemeralNodeAndWatch
For example, in createNodeIfNotExistsAndWatch():
{code}
public static boolean createNodeIfNotExistsAndWatch(
ZooKeeperWatcher zkw, String znode, byte [] data)
throws KeeperException {
try {
zkw.getRecoverableZooKeeper().create(znode, data, createACL(zkw, znode),
CreateMode.PERSISTENT);
} catch (KeeperException.NodeExistsException nee) {
try {
zkw.getRecoverableZooKeeper().exists(znode, zkw);
} catch (InterruptedException e) {
zkw.interruptedException(e);
return false;
}
return false;
} catch (InterruptedException e) {
zkw.interruptedException(e);
return false;
}
return true;
}
{code}
The watch is only set via exists() call when the node already exists.
Similarly in createEphemeralNodeAndWatch():
{code}
public static boolean createEphemeralNodeAndWatch(ZooKeeperWatcher zkw,
String znode, byte [] data)
throws KeeperException {
try {
zkw.getRecoverableZooKeeper().create(znode, data, createACL(zkw, znode),
CreateMode.EPHEMERAL);
} catch (KeeperException.NodeExistsException nee) {
if(!watchAndCheckExists(zkw, znode)) {
// It did exist but now it doesn't, try again
return createEphemeralNodeAndWatch(zkw, znode, data);
}
return false;
} catch (InterruptedException e) {
LOG.info(Interrupted, e);
Thread.currentThread().interrupt();
}
return true;
}
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
