[jira] [Updated] (HBASE-13319) Support 64-bits total row number in PerformanceEvaluation
[ https://issues.apache.org/jira/browse/HBASE-13319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Shaohui updated HBASE-13319: Attachment: HBASE-13319-trunk-v1.diff Patch for master Support 64-bits total row number in PerformanceEvaluation - Key: HBASE-13319 URL: https://issues.apache.org/jira/browse/HBASE-13319 Project: HBase Issue Type: Improvement Components: Performance Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Attachments: HBASE-13319-trunk-v1.diff Currently the total row number in PerformanceEvaluation is 32 bits. It's not enough when testing a large hbase cluster using PerformanceEvaluation in mapreduce mode. Suggestions are welcomed~ Thanks -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-12790) Support fairness across parallelized scans
[ https://issues.apache.org/jira/browse/HBASE-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377764#comment-14377764 ] ramkrishna.s.vasudevan edited comment on HBASE-12790 at 3/24/15 12:13 PM: -- Updated patch with test case. The round robin nature for the AbstractPriorityBasedRoundRobinQueue has been corrected as in the first patch we were using one producer under a given priority and once it was over was moving over to the next producer. Which means under Priority A if we had two groups A and B each with 10 task, then we were consuming all the 10 tasks under A and then moving on to B. With the latest patch we ensure that we round robin between the task in A and B. was (Author: ram_krish): Updated patch with test case. The round robin nature for the AbstractRoundRobinQueue has been corrected as in the first patch we were using one producer under a given priority and once it was over was moving over to the next producer. Which means under Priority A if we had two groups A and B each with 10 task, then we were consuming all the 10 tasks under A and then moving on to B. With the latest patch we ensure that we round robin between the task in A and B. Support fairness across parallelized scans -- Key: HBASE-12790 URL: https://issues.apache.org/jira/browse/HBASE-12790 Project: HBase Issue Type: New Feature Reporter: James Taylor Assignee: ramkrishna.s.vasudevan Labels: Phoenix Attachments: AbstractRoundRobinQueue.java, HBASE-12790.patch, HBASE-12790_1.patch Some HBase clients parallelize the execution of a scan to reduce latency in getting back results. This can lead to starvation with a loaded cluster and interleaved scans, since the RPC queue will be ordered and processed on a FIFO basis. For example, if there are two clients, A B that submit largish scans at the same time. Say each scan is broken down into 100 scans by the client (broken down into equal depth chunks along the row key), and the 100 scans of client A are queued first, followed immediately by the 100 scans of client B. In this case, client B will be starved out of getting any results back until the scans for client A complete. One solution to this is to use the attached AbstractRoundRobinQueue instead of the standard FIFO queue. The queue to be used could be (maybe it already is) configurable based on a new config parameter. Using this queue would require the client to have the same identifier for all of the 100 parallel scans that represent a single logical scan from the clients point of view. With this information, the round robin queue would pick off a task from the queue in a round robin fashion (instead of a strictly FIFO manner) to prevent starvation over interleaved parallelized scans. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12790) Support fairness across parallelized scans
[ https://issues.apache.org/jira/browse/HBASE-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-12790: --- Status: Patch Available (was: Open) Support fairness across parallelized scans -- Key: HBASE-12790 URL: https://issues.apache.org/jira/browse/HBASE-12790 Project: HBase Issue Type: New Feature Reporter: James Taylor Labels: Phoenix Attachments: AbstractRoundRobinQueue.java, HBASE-12790.patch, HBASE-12790_1.patch Some HBase clients parallelize the execution of a scan to reduce latency in getting back results. This can lead to starvation with a loaded cluster and interleaved scans, since the RPC queue will be ordered and processed on a FIFO basis. For example, if there are two clients, A B that submit largish scans at the same time. Say each scan is broken down into 100 scans by the client (broken down into equal depth chunks along the row key), and the 100 scans of client A are queued first, followed immediately by the 100 scans of client B. In this case, client B will be starved out of getting any results back until the scans for client A complete. One solution to this is to use the attached AbstractRoundRobinQueue instead of the standard FIFO queue. The queue to be used could be (maybe it already is) configurable based on a new config parameter. Using this queue would require the client to have the same identifier for all of the 100 parallel scans that represent a single logical scan from the clients point of view. With this information, the round robin queue would pick off a task from the queue in a round robin fashion (instead of a strictly FIFO manner) to prevent starvation over interleaved parallelized scans. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13321) Fix flaky TestHBaseFsck
[ https://issues.apache.org/jira/browse/HBASE-13321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-13321: Status: Patch Available (was: Open) Fix flaky TestHBaseFsck --- Key: HBASE-13321 URL: https://issues.apache.org/jira/browse/HBASE-13321 Project: HBase Issue Type: Bug Components: test Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Attachments: HBASE-13321-v0.patch Similar to HBASE-13182 and HBASE-13041 TestHBaseFsck relies on create/deleteTable() to be sync. so, if the master side is slow we get EXPIRED_TABLE_LOCK failures. to workaround that, we can add a master coprocessor used by the test to wait until postOperationHandler is called. This will be removed after HBASE-13204. there is also a znode for the table-lock not deleted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13314) Fix NPE in HMaster.getClusterStatus()
[
https://issues.apache.org/jira/browse/HBASE-13314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377768#comment-14377768
]
Hudson commented on HBASE-13314:
SUCCESS: Integrated in HBase-TRUNK #6296 (See
[https://builds.apache.org/job/HBase-TRUNK/6296/])
HBASE-13314 Fix NPE in HMaster.getClusterStatus() (matteo.bertozzi: rev
80d230e1fbeac24d3dfdac8165e24f35ec26f988)
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/ClusterStatus.java
Fix NPE in HMaster.getClusterStatus()
-
Key: HBASE-13314
URL: https://issues.apache.org/jira/browse/HBASE-13314
Project: HBase
Issue Type: Bug
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Fix For: 2.0.0, 1.0.1, 1.1.0
Attachments: HBASE-13314-v0.patch, HBASE-13314-v1.patch,
HBASE-13314-v1.patch
On a test cluster we got a
{noformat}
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.HMaster.getClusterStatus(HMaster.java:1742)
at
org.apache.hadoop.hbase.master.balancer.ClusterStatusChore.chore(ClusterStatusChore.java:50)
at org.apache.hadoop.hbase.Chore.run(Chore.java:87)
{noformat}
In HMaster.getClusterStatus() we have a couple of NPE.
listChildrenNoWatch() may return null if the node is not found
{code}
try {
backupMasterStrings = ZKUtil.listChildrenNoWatch(this.zooKeeper,
this.zooKeeper.backupMasterAddressesZNode); --- THIS MAY RETURN NULL
} catch (KeeperException e) {
LOG.warn(this.zooKeeper.prefix(Unable to list backup servers), e);
backupMasterStrings = new ArrayListString(0);
}
ListServerName backupMasters = new ArrayListServerName(
backupMasterStrings.size()); --- WE DON'T CHECK FOR NULL
{code}
then below, we build ClusterStatus with args that may be null
{code}
String clusterId = fileSystemManager != null ?
fileSystemManager.getClusterId().toString() : null;
MapString, RegionState regionsInTransition = assignmentManager != null ?
assignmentManager.getRegionStates().getRegionsInTransition() : null;
String[] coprocessors = cpHost != null ? getMasterCoprocessors() : null;
MapServerName, ServerLoad onlineServers = null;
SetServerName deadServers = null;
if (serverManager != null) {
deadServers = serverManager.getDeadServers().copyServerNames();
onlineServers = serverManager.getOnlineServers();
}
return new ClusterStatus(VersionInfo.getVersion(), clusterId,
onlineServers, deadServers, serverName, backupMasters,
regionsInTransition, coprocessors, balancerOn);
{code}
In ClusterStatus equals(), hashCode(), toString() we don't check for nulls
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13320) 'hbase.bucketcache.size' configuration value is not correct in hbase-default.xml
[
https://issues.apache.org/jira/browse/HBASE-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377882#comment-14377882
]
Y. SREENIVASULU REDDY commented on HBASE-13320:
---
yes i will provide the patch for this [~ram_krish]
'hbase.bucketcache.size' configuration value is not correct in
hbase-default.xml
-
Key: HBASE-13320
URL: https://issues.apache.org/jira/browse/HBASE-13320
Project: HBase
Issue Type: Bug
Components: hbase
Affects Versions: 2.0.0
Reporter: Y. SREENIVASULU REDDY
Assignee: Y. SREENIVASULU REDDY
Fix For: 2.0.0
In hbase-default.xml file
* 'hbase.bucketcache.size' is not correct
We either specify it as a float or in MB's and the default value that is
mentioned is never used
{code}
property
namehbase.bucketcache.size/name
value65536/value
sourcehbase-default.xml/source
/property
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13321) Fix flaky TestHBaseFsck
[ https://issues.apache.org/jira/browse/HBASE-13321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-13321: Attachment: HBASE-13321-v0.patch Fix flaky TestHBaseFsck --- Key: HBASE-13321 URL: https://issues.apache.org/jira/browse/HBASE-13321 Project: HBase Issue Type: Bug Components: test Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Attachments: HBASE-13321-v0.patch Similar to HBASE-13182 and HBASE-13041 TestHBaseFsck relies on create/deleteTable() to be sync. so, if the master side is slow we get EXPIRED_TABLE_LOCK failures. to workaround that, we can add a master coprocessor used by the test to wait until postOperationHandler is called. This will be removed after HBASE-13204. there is also a znode for the table-lock not deleted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13147) Load actual META table descriptor, don't use statically defined one.
[
https://issues.apache.org/jira/browse/HBASE-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377856#comment-14377856
]
Hadoop QA commented on HBASE-13147:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12706901/HBASE-13147.v5.patch
against master branch at commit 80d230e1fbeac24d3dfdac8165e24f35ec26f988.
ATTACHMENT ID: 12706901
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 20 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1921 checkstyle errors (more than the master's current 1917 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:red}-1 release audit{color}. The applied patch generated 2 release
audit warnings (more than the master's current 0 warnings).
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestHTableDescriptor
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13384//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13384//artifact/patchprocess/patchReleaseAuditWarnings.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13384//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13384//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13384//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13384//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13384//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13384//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13384//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13384//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13384//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13384//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13384//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13384//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13384//console
This message is automatically generated.
Load actual META table descriptor, don't use statically defined one.
Key: HBASE-13147
URL: https://issues.apache.org/jira/browse/HBASE-13147
Project: HBase
Issue Type: Bug
Components: master, regionserver
Affects Versions: 2.0.0
Reporter: Andrey Stepachev
Assignee: Andrey Stepachev
Attachments: HBASE-13147-branch-1.patch,
HBASE-13147-branch-1.v2.patch, HBASE-13147.patch, HBASE-13147.v2.patch,
HBASE-13147.v3.patch, HBASE-13147.v4.patch, HBASE-13147.v4.patch,
HBASE-13147.v5.patch
In HBASE-13087 stumbled on the fact, that region servers don't see actual
meta descriptor, they use their own, statically compiled.
Need to fix that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13147) Load actual META table descriptor, don't use statically defined one.
[ https://issues.apache.org/jira/browse/HBASE-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13147: - Status: Open (was: Patch Available) Load actual META table descriptor, don't use statically defined one. Key: HBASE-13147 URL: https://issues.apache.org/jira/browse/HBASE-13147 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 2.0.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Attachments: HBASE-13147-branch-1.patch, HBASE-13147-branch-1.v2.patch, HBASE-13147.patch, HBASE-13147.v2.patch, HBASE-13147.v3.patch, HBASE-13147.v4.patch, HBASE-13147.v4.patch In HBASE-13087 stumbled on the fact, that region servers don't see actual meta descriptor, they use their own, statically compiled. Need to fix that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13147) Load actual META table descriptor, don't use statically defined one.
[ https://issues.apache.org/jira/browse/HBASE-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13147: - Attachment: (was: HBASE-13147.v5.patch) Load actual META table descriptor, don't use statically defined one. Key: HBASE-13147 URL: https://issues.apache.org/jira/browse/HBASE-13147 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 2.0.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Attachments: HBASE-13147-branch-1.patch, HBASE-13147-branch-1.v2.patch, HBASE-13147.patch, HBASE-13147.v2.patch, HBASE-13147.v3.patch, HBASE-13147.v4.patch, HBASE-13147.v4.patch In HBASE-13087 stumbled on the fact, that region servers don't see actual meta descriptor, they use their own, statically compiled. Need to fix that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13147) Load actual META table descriptor, don't use statically defined one.
[ https://issues.apache.org/jira/browse/HBASE-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13147: - Status: Open (was: Patch Available) Load actual META table descriptor, don't use statically defined one. Key: HBASE-13147 URL: https://issues.apache.org/jira/browse/HBASE-13147 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 2.0.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Attachments: HBASE-13147-branch-1.patch, HBASE-13147-branch-1.v2.patch, HBASE-13147.patch, HBASE-13147.v2.patch, HBASE-13147.v3.patch, HBASE-13147.v4.patch, HBASE-13147.v4.patch In HBASE-13087 stumbled on the fact, that region servers don't see actual meta descriptor, they use their own, statically compiled. Need to fix that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12790) Support fairness across parallelized scans
[
https://issues.apache.org/jira/browse/HBASE-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377807#comment-14377807
]
Hadoop QA commented on HBASE-12790:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12706892/HBASE-12790_1.patch
against master branch at commit 80d230e1fbeac24d3dfdac8165e24f35ec26f988.
ATTACHMENT ID: 12706892
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 10 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 8
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1923 checkstyle errors (more than the master's current 1917 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1 release
audit warnings (more than the master's current 0 warnings).
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ new java.lang.String[] { Column, Attribute, StartRow,
StopRow, Filter, TimeRange, MaxVersions, CacheBlocks, BatchSize,
MaxResultSize, StoreLimit, StoreOffset, LoadColumnFamiliesOnDemand,
Small, Reversed, Consistency, Caching, GroupingId, });
+ * An {@link RpcExecutor} that will balance requests in a round robin way
across all its queues based on
+boolean callQueueGrouping = conf.getBoolean(CALL_QUEUE_GROUPING,
CALL_QUEUE_GROUPING_DEFAULT_VALUE);
+// TODO : Introduce configuration that strictly allows the Balanced
way of Write QueueExecutor
+ conf, abortable, FairSharePriorityBasedBlockingQueue.class,
maxQueueLength, callPriority);
+// TODO : Introduce configuration that strictly allows the balanced
way of write queue executor
+// we need to adjust the current thread pointer in case it pointed
to this thread list, which is now removed
+AbstractPriorityBasedRoundRobinQueueTestObject testList = new
AbstractRoundRobinPriorityQueueImpl(
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestOperation
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13381//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13381//artifact/patchprocess/patchReleaseAuditWarnings.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13381//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13381//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13381//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13381//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13381//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13381//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13381//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13381//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13381//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13381//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13381//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13381//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13381//artifact/patchprocess/patchJavadocWarnings.txt
[jira] [Commented] (HBASE-13147) Load actual META table descriptor, don't use statically defined one.
[
https://issues.apache.org/jira/browse/HBASE-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377889#comment-14377889
]
Hadoop QA commented on HBASE-13147:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12706905/HBASE-13147.v5.patch
against master branch at commit 80d230e1fbeac24d3dfdac8165e24f35ec26f988.
ATTACHMENT ID: 12706905
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 20 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1921 checkstyle errors (more than the master's current 1917 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:red}-1 release audit{color}. The applied patch generated 2 release
audit warnings (more than the master's current 0 warnings).
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestHTableDescriptor
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13385//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13385//artifact/patchprocess/patchReleaseAuditWarnings.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13385//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13385//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13385//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13385//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13385//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13385//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13385//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13385//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13385//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13385//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13385//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13385//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13385//console
This message is automatically generated.
Load actual META table descriptor, don't use statically defined one.
Key: HBASE-13147
URL: https://issues.apache.org/jira/browse/HBASE-13147
Project: HBase
Issue Type: Bug
Components: master, regionserver
Affects Versions: 2.0.0
Reporter: Andrey Stepachev
Assignee: Andrey Stepachev
Attachments: HBASE-13147-branch-1.patch,
HBASE-13147-branch-1.v2.patch, HBASE-13147.patch, HBASE-13147.v2.patch,
HBASE-13147.v3.patch, HBASE-13147.v4.patch, HBASE-13147.v4.patch,
HBASE-13147.v5.patch
In HBASE-13087 stumbled on the fact, that region servers don't see actual
meta descriptor, they use their own, statically compiled.
Need to fix that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13321) Fix flaky TestHBaseFsck
Matteo Bertozzi created HBASE-13321: --- Summary: Fix flaky TestHBaseFsck Key: HBASE-13321 URL: https://issues.apache.org/jira/browse/HBASE-13321 Project: HBase Issue Type: Bug Components: test Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Attachments: HBASE-13321-v0.patch Similar to HBASE-13182 and HBASE-13041 TestHBaseFsck relies on create/deleteTable() to be sync. so, if the master side is slow we get EXPIRED_TABLE_LOCK failures. to workaround that, we can add a master coprocessor used by the test to wait until postOperationHandler is called. This will be removed after HBASE-13204. there is also a znode for the table-lock not deleted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13147) Load actual META table descriptor, don't use statically defined one.
[ https://issues.apache.org/jira/browse/HBASE-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13147: - Attachment: HBASE-13147.v5.patch Updated patch v5: - added value to descriptor holding meta version - now migration is a bit more intillegent, it compares versions and do minimal updates (for 1 - 2, for any - 2 it will just overwrite descriptor with latest one) - 1 is a version anything prior to this patch, so we always write new descriptor - 2 is a version with table family introduced (opens way to allow relatively safe rolling upgrades) - if master will try start with lesser version or region server will try to open newer meta, they will fail. Load actual META table descriptor, don't use statically defined one. Key: HBASE-13147 URL: https://issues.apache.org/jira/browse/HBASE-13147 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 2.0.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Attachments: HBASE-13147-branch-1.patch, HBASE-13147-branch-1.v2.patch, HBASE-13147.patch, HBASE-13147.v2.patch, HBASE-13147.v3.patch, HBASE-13147.v4.patch, HBASE-13147.v4.patch, HBASE-13147.v5.patch In HBASE-13087 stumbled on the fact, that region servers don't see actual meta descriptor, they use their own, statically compiled. Need to fix that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13147) Load actual META table descriptor, don't use statically defined one.
[ https://issues.apache.org/jira/browse/HBASE-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13147: - Status: Patch Available (was: Open) Load actual META table descriptor, don't use statically defined one. Key: HBASE-13147 URL: https://issues.apache.org/jira/browse/HBASE-13147 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 2.0.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Attachments: HBASE-13147-branch-1.patch, HBASE-13147-branch-1.v2.patch, HBASE-13147.patch, HBASE-13147.v2.patch, HBASE-13147.v3.patch, HBASE-13147.v4.patch, HBASE-13147.v4.patch, HBASE-13147.v5.patch In HBASE-13087 stumbled on the fact, that region servers don't see actual meta descriptor, they use their own, statically compiled. Need to fix that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13147) Load actual META table descriptor, don't use statically defined one.
[ https://issues.apache.org/jira/browse/HBASE-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13147: - Status: Patch Available (was: Open) Load actual META table descriptor, don't use statically defined one. Key: HBASE-13147 URL: https://issues.apache.org/jira/browse/HBASE-13147 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 2.0.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Attachments: HBASE-13147-branch-1.patch, HBASE-13147-branch-1.v2.patch, HBASE-13147.patch, HBASE-13147.v2.patch, HBASE-13147.v3.patch, HBASE-13147.v4.patch, HBASE-13147.v4.patch, HBASE-13147.v5.patch, HBASE-13147.v6.patch In HBASE-13087 stumbled on the fact, that region servers don't see actual meta descriptor, they use their own, statically compiled. Need to fix that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13147) Load actual META table descriptor, don't use statically defined one.
[ https://issues.apache.org/jira/browse/HBASE-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13147: - Status: Open (was: Patch Available) Load actual META table descriptor, don't use statically defined one. Key: HBASE-13147 URL: https://issues.apache.org/jira/browse/HBASE-13147 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 2.0.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Attachments: HBASE-13147-branch-1.patch, HBASE-13147-branch-1.v2.patch, HBASE-13147.patch, HBASE-13147.v2.patch, HBASE-13147.v3.patch, HBASE-13147.v4.patch, HBASE-13147.v4.patch, HBASE-13147.v5.patch, HBASE-13147.v6.patch In HBASE-13087 stumbled on the fact, that region servers don't see actual meta descriptor, they use their own, statically compiled. Need to fix that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13147) Load actual META table descriptor, don't use statically defined one.
[ https://issues.apache.org/jira/browse/HBASE-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13147: - Attachment: HBASE-13147.v6.patch fixing issues with tests and qa. Load actual META table descriptor, don't use statically defined one. Key: HBASE-13147 URL: https://issues.apache.org/jira/browse/HBASE-13147 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 2.0.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Attachments: HBASE-13147-branch-1.patch, HBASE-13147-branch-1.v2.patch, HBASE-13147.patch, HBASE-13147.v2.patch, HBASE-13147.v3.patch, HBASE-13147.v4.patch, HBASE-13147.v4.patch, HBASE-13147.v5.patch, HBASE-13147.v6.patch In HBASE-13087 stumbled on the fact, that region servers don't see actual meta descriptor, they use their own, statically compiled. Need to fix that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13147) Load actual META table descriptor, don't use statically defined one.
[ https://issues.apache.org/jira/browse/HBASE-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13147: - Status: Patch Available (was: Open) Load actual META table descriptor, don't use statically defined one. Key: HBASE-13147 URL: https://issues.apache.org/jira/browse/HBASE-13147 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 2.0.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Attachments: HBASE-13147-branch-1.patch, HBASE-13147-branch-1.v2.patch, HBASE-13147.patch, HBASE-13147.v2.patch, HBASE-13147.v3.patch, HBASE-13147.v4.patch, HBASE-13147.v4.patch, HBASE-13147.v5.patch In HBASE-13087 stumbled on the fact, that region servers don't see actual meta descriptor, they use their own, statically compiled. Need to fix that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13147) Load actual META table descriptor, don't use statically defined one.
[ https://issues.apache.org/jira/browse/HBASE-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13147: - Attachment: HBASE-13147.v5.patch Load actual META table descriptor, don't use statically defined one. Key: HBASE-13147 URL: https://issues.apache.org/jira/browse/HBASE-13147 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 2.0.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Attachments: HBASE-13147-branch-1.patch, HBASE-13147-branch-1.v2.patch, HBASE-13147.patch, HBASE-13147.v2.patch, HBASE-13147.v3.patch, HBASE-13147.v4.patch, HBASE-13147.v4.patch, HBASE-13147.v5.patch In HBASE-13087 stumbled on the fact, that region servers don't see actual meta descriptor, they use their own, statically compiled. Need to fix that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12462) Support deleting all columns of the specified family of a row in hbase shell
[ https://issues.apache.org/jira/browse/HBASE-12462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1436#comment-1436 ] Liu Shaohui commented on HBASE-12462: - [~lhofhansl] The patch just changed the meaning of cmd `delete 't1', 'r1', 'f1'` to delete family from delete a column with null qualifier. If users really want to delete a column with null qualifier, they can use `delete 't1', 'r1', 'f1:'` Support deleting all columns of the specified family of a row in hbase shell Key: HBASE-12462 URL: https://issues.apache.org/jira/browse/HBASE-12462 Project: HBase Issue Type: New Feature Components: shell Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Fix For: 2.0.0 Attachments: HBASE-12462-v1.diff Currently, HBase shell only support deleting a column of a row in a table. In some scenarios, we want to delete all the columns under a a column family of a row, but there may be many columns there. It's difficult to delete the columns one by one in shell. It's easy to add this feature in shell since the Delete have the API of deleting a family. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13319) Support 64-bits total row number in PerformanceEvaluation
[
https://issues.apache.org/jira/browse/HBASE-13319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377787#comment-14377787
]
Hadoop QA commented on HBASE-13319:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12706875/HBASE-13319-trunk-v1.diff
against master branch at commit 80d230e1fbeac24d3dfdac8165e24f35ec26f988.
ATTACHMENT ID: 12706875
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13380//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13380//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13380//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13380//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13380//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13380//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13380//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13380//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13380//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13380//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13380//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13380//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13380//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13380//console
This message is automatically generated.
Support 64-bits total row number in PerformanceEvaluation
-
Key: HBASE-13319
URL: https://issues.apache.org/jira/browse/HBASE-13319
Project: HBase
Issue Type: Improvement
Components: Performance
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Priority: Minor
Fix For: 2.0.0
Attachments: HBASE-13319-trunk-v1.diff
Currently the total row number in PerformanceEvaluation is 32 bits. It's not
enough when testing a large hbase cluster using PerformanceEvaluation in
mapreduce mode.
Suggestions are welcomed~ Thanks
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13319) Support 64-bits total row number in PerformanceEvaluation
[ https://issues.apache.org/jira/browse/HBASE-13319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Shaohui updated HBASE-13319: Fix Version/s: 2.0.0 Status: Patch Available (was: Open) Support 64-bits total row number in PerformanceEvaluation - Key: HBASE-13319 URL: https://issues.apache.org/jira/browse/HBASE-13319 Project: HBase Issue Type: Improvement Components: Performance Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Fix For: 2.0.0 Attachments: HBASE-13319-trunk-v1.diff Currently the total row number in PerformanceEvaluation is 32 bits. It's not enough when testing a large hbase cluster using PerformanceEvaluation in mapreduce mode. Suggestions are welcomed~ Thanks -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13082) Coarsen StoreScanner locks to RegionScanner
[ https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377564#comment-14377564 ] Anoop Sam John commented on HBASE-13082: If scanners can continue read from the old files, it can continue to use the data from cache (if that file blocks were in cache). Otherwise a compaction will result in need for scan from a new file whose data is not in cache. (Assume no cache on write) I was also thinking on this especially wrt Cache usage.. Coarsen StoreScanner locks to RegionScanner --- Key: HBASE-13082 URL: https://issues.apache.org/jira/browse/HBASE-13082 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Assignee: Lars Hofhansl Attachments: 13082-test.txt, 13082-v2.txt, 13082-v3.txt, 13082-v4.txt, 13082.txt, 13082.txt, gc.png, gc.png, gc.png, hits.png, next.png, next.png Continuing where HBASE-10015 left of. We can avoid locking (and memory fencing) inside StoreScanner by deferring to the lock already held by the RegionScanner. In tests this shows quite a scan improvement and reduced CPU (the fences make the cores wait for memory fetches). There are some drawbacks too: * All calls to RegionScanner need to be remain synchronized * Implementors of coprocessors need to be diligent in following the locking contract. For example Phoenix does not lock RegionScanner.nextRaw() and required in the documentation (not picking on Phoenix, this one is my fault as I told them it's OK) * possible starving of flushes and compaction with heavy read load. RegionScanner operations would keep getting the locks and the flushes/compactions would not be able finalize the set of files. I'll have a patch soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13281) 'hbase.bucketcache.size' description in hbase book is not correct
[ https://issues.apache.org/jira/browse/HBASE-13281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377602#comment-14377602 ] Anoop Sam John commented on HBASE-13281: As this is already closed raise another issue and fix in that? 'hbase.bucketcache.size' description in hbase book is not correct - Key: HBASE-13281 URL: https://issues.apache.org/jira/browse/HBASE-13281 Project: HBase Issue Type: Bug Components: documentation Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 2.0.0 Attachments: HBASE-13281.patch The description about 'hbase.bucketcache.size' is not correct. We either specify it as a float or in MB's and the default value that is mentioned is never used. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-5238) Add a log4j category for all edits to META/ROOT
[ https://issues.apache.org/jira/browse/HBASE-5238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377646#comment-14377646 ] Andrey Stepachev commented on HBASE-5238: - toJSON prints only column families and datasize. does it makes sense? Seems harmless for me. (sorry for late answer) Add a log4j category for all edits to META/ROOT --- Key: HBASE-5238 URL: https://issues.apache.org/jira/browse/HBASE-5238 Project: HBase Issue Type: New Feature Components: regionserver Affects Versions: 2.0.0 Reporter: Todd Lipcon Assignee: Andrey Stepachev Priority: Minor Labels: beginner Attachments: HBASE-5238.patch, HBASE-5238.patch, HBASE-5238.v2.patch, meta2.log Occasionally we run into bugs that have corrected META and written some bad data to META/ROOT but it's difficult to understand the order in which things happened. One option is to dump the HLog contents from the servers that hosted META at that time, but then it's interspersed with all other data. It would be nice to add a Log4j Logger to which we log all edits being applied to META and ROOT in textual form at DEBUG level. Then it would be easier to do a cluster-wide log grep to see what happened when. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13320) 'hbase.bucketcache.size' configuration value is not correct in hbase-default.xml
Y. SREENIVASULU REDDY created HBASE-13320:
-
Summary: 'hbase.bucketcache.size' configuration value is not
correct in hbase-default.xml
Key: HBASE-13320
URL: https://issues.apache.org/jira/browse/HBASE-13320
Project: HBase
Issue Type: Bug
Components: hbase
Affects Versions: 2.0.0
Reporter: Y. SREENIVASULU REDDY
Assignee: Y. SREENIVASULU REDDY
Fix For: 2.0.0
In hbase-default.xml file
* 'hbase.bucketcache.size' is not correct
We either specify it as a float or in MB's and the default value that is
mentioned is never used
{code}
property
namehbase.bucketcache.size/name
value65536/value
sourcehbase-default.xml/source
/property
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13320) 'hbase.bucketcache.size' configuration value is not correct in hbase-default.xml
[
https://issues.apache.org/jira/browse/HBASE-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377762#comment-14377762
]
ramkrishna.s.vasudevan commented on HBASE-13320:
Thanks for raising this. You going to give patch for this?
'hbase.bucketcache.size' configuration value is not correct in
hbase-default.xml
-
Key: HBASE-13320
URL: https://issues.apache.org/jira/browse/HBASE-13320
Project: HBase
Issue Type: Bug
Components: hbase
Affects Versions: 2.0.0
Reporter: Y. SREENIVASULU REDDY
Assignee: Y. SREENIVASULU REDDY
Fix For: 2.0.0
In hbase-default.xml file
* 'hbase.bucketcache.size' is not correct
We either specify it as a float or in MB's and the default value that is
mentioned is never used
{code}
property
namehbase.bucketcache.size/name
value65536/value
sourcehbase-default.xml/source
/property
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13314) Fix NPE in HMaster.getClusterStatus()
[
https://issues.apache.org/jira/browse/HBASE-13314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi updated HBASE-13314:
Resolution: Fixed
Fix Version/s: 1.1.0
1.0.1
2.0.0
Status: Resolved (was: Patch Available)
Fix NPE in HMaster.getClusterStatus()
-
Key: HBASE-13314
URL: https://issues.apache.org/jira/browse/HBASE-13314
Project: HBase
Issue Type: Bug
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Fix For: 2.0.0, 1.0.1, 1.1.0
Attachments: HBASE-13314-v0.patch, HBASE-13314-v1.patch,
HBASE-13314-v1.patch
On a test cluster we got a
{noformat}
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.HMaster.getClusterStatus(HMaster.java:1742)
at
org.apache.hadoop.hbase.master.balancer.ClusterStatusChore.chore(ClusterStatusChore.java:50)
at org.apache.hadoop.hbase.Chore.run(Chore.java:87)
{noformat}
In HMaster.getClusterStatus() we have a couple of NPE.
listChildrenNoWatch() may return null if the node is not found
{code}
try {
backupMasterStrings = ZKUtil.listChildrenNoWatch(this.zooKeeper,
this.zooKeeper.backupMasterAddressesZNode); --- THIS MAY RETURN NULL
} catch (KeeperException e) {
LOG.warn(this.zooKeeper.prefix(Unable to list backup servers), e);
backupMasterStrings = new ArrayListString(0);
}
ListServerName backupMasters = new ArrayListServerName(
backupMasterStrings.size()); --- WE DON'T CHECK FOR NULL
{code}
then below, we build ClusterStatus with args that may be null
{code}
String clusterId = fileSystemManager != null ?
fileSystemManager.getClusterId().toString() : null;
MapString, RegionState regionsInTransition = assignmentManager != null ?
assignmentManager.getRegionStates().getRegionsInTransition() : null;
String[] coprocessors = cpHost != null ? getMasterCoprocessors() : null;
MapServerName, ServerLoad onlineServers = null;
SetServerName deadServers = null;
if (serverManager != null) {
deadServers = serverManager.getDeadServers().copyServerNames();
onlineServers = serverManager.getOnlineServers();
}
return new ClusterStatus(VersionInfo.getVersion(), clusterId,
onlineServers, deadServers, serverName, backupMasters,
regionsInTransition, coprocessors, balancerOn);
{code}
In ClusterStatus equals(), hashCode(), toString() we don't check for nulls
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13204) Procedure v2 - client create/delete table sync
[ https://issues.apache.org/jira/browse/HBASE-13204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-13204: Description: client side, part of HBASE-12439/HBASE-13203 it uses the new procedure code to be know when the procedure is completed, and have a proper sync behavior on create/delete table. Review: https://reviews.apache.org/r/32391/ was: client side, part of HBASE-12439/HBASE-13203 it uses the new procedure code to be know when the procedure is completed, and have a proper sync behavior on create/delete table. Procedure v2 - client create/delete table sync -- Key: HBASE-13204 URL: https://issues.apache.org/jira/browse/HBASE-13204 Project: HBase Issue Type: Sub-task Components: master Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor client side, part of HBASE-12439/HBASE-13203 it uses the new procedure code to be know when the procedure is completed, and have a proper sync behavior on create/delete table. Review: https://reviews.apache.org/r/32391/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12790) Support fairness across parallelized scans
[ https://issues.apache.org/jira/browse/HBASE-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-12790: --- Attachment: HBASE-12790_1.patch Updated patch with test case. The round robin nature for the AbstractRoundRobinQueue has been corrected as in the first patch we were using one producer under a given priority and once it was over was moving over to the next producer. Which means under Priority A if we had two groups A and B each with 10 task, then we were consuming all the 10 tasks under A and then moving on to B. With the latest patch we ensure that we round robin between the task in A and B. Support fairness across parallelized scans -- Key: HBASE-12790 URL: https://issues.apache.org/jira/browse/HBASE-12790 Project: HBase Issue Type: New Feature Reporter: James Taylor Labels: Phoenix Attachments: AbstractRoundRobinQueue.java, HBASE-12790.patch, HBASE-12790_1.patch Some HBase clients parallelize the execution of a scan to reduce latency in getting back results. This can lead to starvation with a loaded cluster and interleaved scans, since the RPC queue will be ordered and processed on a FIFO basis. For example, if there are two clients, A B that submit largish scans at the same time. Say each scan is broken down into 100 scans by the client (broken down into equal depth chunks along the row key), and the 100 scans of client A are queued first, followed immediately by the 100 scans of client B. In this case, client B will be starved out of getting any results back until the scans for client A complete. One solution to this is to use the attached AbstractRoundRobinQueue instead of the standard FIFO queue. The queue to be used could be (maybe it already is) configurable based on a new config parameter. Using this queue would require the client to have the same identifier for all of the 100 parallel scans that represent a single logical scan from the clients point of view. With this information, the round robin queue would pick off a task from the queue in a round robin fashion (instead of a strictly FIFO manner) to prevent starvation over interleaved parallelized scans. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-12790) Support fairness across parallelized scans
[ https://issues.apache.org/jira/browse/HBASE-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan reassigned HBASE-12790: -- Assignee: ramkrishna.s.vasudevan Support fairness across parallelized scans -- Key: HBASE-12790 URL: https://issues.apache.org/jira/browse/HBASE-12790 Project: HBase Issue Type: New Feature Reporter: James Taylor Assignee: ramkrishna.s.vasudevan Labels: Phoenix Attachments: AbstractRoundRobinQueue.java, HBASE-12790.patch, HBASE-12790_1.patch Some HBase clients parallelize the execution of a scan to reduce latency in getting back results. This can lead to starvation with a loaded cluster and interleaved scans, since the RPC queue will be ordered and processed on a FIFO basis. For example, if there are two clients, A B that submit largish scans at the same time. Say each scan is broken down into 100 scans by the client (broken down into equal depth chunks along the row key), and the 100 scans of client A are queued first, followed immediately by the 100 scans of client B. In this case, client B will be starved out of getting any results back until the scans for client A complete. One solution to this is to use the attached AbstractRoundRobinQueue instead of the standard FIFO queue. The queue to be used could be (maybe it already is) configurable based on a new config parameter. Using this queue would require the client to have the same identifier for all of the 100 parallel scans that represent a single logical scan from the clients point of view. With this information, the round robin queue would pick off a task from the queue in a round robin fashion (instead of a strictly FIFO manner) to prevent starvation over interleaved parallelized scans. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13319) Support 64-bits total row number in PerformanceEvaluation
Liu Shaohui created HBASE-13319: --- Summary: Support 64-bits total row number in PerformanceEvaluation Key: HBASE-13319 URL: https://issues.apache.org/jira/browse/HBASE-13319 Project: HBase Issue Type: Improvement Components: Performance Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Currently the total row number in PerformanceEvaluation is 32 bits. It's not enough when testing a large hbase cluster using PerformanceEvaluation in mapreduce mode. Suggestions are welcomed~ Thanks -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13314) Fix NPE in HMaster.getClusterStatus()
[
https://issues.apache.org/jira/browse/HBASE-13314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377669#comment-14377669
]
Hudson commented on HBASE-13314:
FAILURE: Integrated in HBase-1.0 #823 (See
[https://builds.apache.org/job/HBase-1.0/823/])
HBASE-13314 Fix NPE in HMaster.getClusterStatus() (matteo.bertozzi: rev
ccbb51d749ea31f13dc8a7c6e008472e4cd2dc5c)
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/ClusterStatus.java
Fix NPE in HMaster.getClusterStatus()
-
Key: HBASE-13314
URL: https://issues.apache.org/jira/browse/HBASE-13314
Project: HBase
Issue Type: Bug
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Fix For: 2.0.0, 1.0.1, 1.1.0
Attachments: HBASE-13314-v0.patch, HBASE-13314-v1.patch,
HBASE-13314-v1.patch
On a test cluster we got a
{noformat}
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.HMaster.getClusterStatus(HMaster.java:1742)
at
org.apache.hadoop.hbase.master.balancer.ClusterStatusChore.chore(ClusterStatusChore.java:50)
at org.apache.hadoop.hbase.Chore.run(Chore.java:87)
{noformat}
In HMaster.getClusterStatus() we have a couple of NPE.
listChildrenNoWatch() may return null if the node is not found
{code}
try {
backupMasterStrings = ZKUtil.listChildrenNoWatch(this.zooKeeper,
this.zooKeeper.backupMasterAddressesZNode); --- THIS MAY RETURN NULL
} catch (KeeperException e) {
LOG.warn(this.zooKeeper.prefix(Unable to list backup servers), e);
backupMasterStrings = new ArrayListString(0);
}
ListServerName backupMasters = new ArrayListServerName(
backupMasterStrings.size()); --- WE DON'T CHECK FOR NULL
{code}
then below, we build ClusterStatus with args that may be null
{code}
String clusterId = fileSystemManager != null ?
fileSystemManager.getClusterId().toString() : null;
MapString, RegionState regionsInTransition = assignmentManager != null ?
assignmentManager.getRegionStates().getRegionsInTransition() : null;
String[] coprocessors = cpHost != null ? getMasterCoprocessors() : null;
MapServerName, ServerLoad onlineServers = null;
SetServerName deadServers = null;
if (serverManager != null) {
deadServers = serverManager.getDeadServers().copyServerNames();
onlineServers = serverManager.getOnlineServers();
}
return new ClusterStatus(VersionInfo.getVersion(), clusterId,
onlineServers, deadServers, serverName, backupMasters,
regionsInTransition, coprocessors, balancerOn);
{code}
In ClusterStatus equals(), hashCode(), toString() we don't check for nulls
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13314) Fix NPE in HMaster.getClusterStatus()
[
https://issues.apache.org/jira/browse/HBASE-13314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377741#comment-14377741
]
Hudson commented on HBASE-13314:
FAILURE: Integrated in HBase-1.1 #315 (See
[https://builds.apache.org/job/HBase-1.1/315/])
HBASE-13314 Fix NPE in HMaster.getClusterStatus() (matteo.bertozzi: rev
266e883daa789c749b815cce2cb97e99ebf8c8d4)
* hbase-client/src/main/java/org/apache/hadoop/hbase/ClusterStatus.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Fix NPE in HMaster.getClusterStatus()
-
Key: HBASE-13314
URL: https://issues.apache.org/jira/browse/HBASE-13314
Project: HBase
Issue Type: Bug
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Fix For: 2.0.0, 1.0.1, 1.1.0
Attachments: HBASE-13314-v0.patch, HBASE-13314-v1.patch,
HBASE-13314-v1.patch
On a test cluster we got a
{noformat}
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.HMaster.getClusterStatus(HMaster.java:1742)
at
org.apache.hadoop.hbase.master.balancer.ClusterStatusChore.chore(ClusterStatusChore.java:50)
at org.apache.hadoop.hbase.Chore.run(Chore.java:87)
{noformat}
In HMaster.getClusterStatus() we have a couple of NPE.
listChildrenNoWatch() may return null if the node is not found
{code}
try {
backupMasterStrings = ZKUtil.listChildrenNoWatch(this.zooKeeper,
this.zooKeeper.backupMasterAddressesZNode); --- THIS MAY RETURN NULL
} catch (KeeperException e) {
LOG.warn(this.zooKeeper.prefix(Unable to list backup servers), e);
backupMasterStrings = new ArrayListString(0);
}
ListServerName backupMasters = new ArrayListServerName(
backupMasterStrings.size()); --- WE DON'T CHECK FOR NULL
{code}
then below, we build ClusterStatus with args that may be null
{code}
String clusterId = fileSystemManager != null ?
fileSystemManager.getClusterId().toString() : null;
MapString, RegionState regionsInTransition = assignmentManager != null ?
assignmentManager.getRegionStates().getRegionsInTransition() : null;
String[] coprocessors = cpHost != null ? getMasterCoprocessors() : null;
MapServerName, ServerLoad onlineServers = null;
SetServerName deadServers = null;
if (serverManager != null) {
deadServers = serverManager.getDeadServers().copyServerNames();
onlineServers = serverManager.getOnlineServers();
}
return new ClusterStatus(VersionInfo.getVersion(), clusterId,
onlineServers, deadServers, serverName, backupMasters,
regionsInTransition, coprocessors, balancerOn);
{code}
In ClusterStatus equals(), hashCode(), toString() we don't check for nulls
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13262) ResultScanner doesn't return all rows in Scan
[
https://issues.apache.org/jira/browse/HBASE-13262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378680#comment-14378680
]
Jonathan Lawlor commented on HBASE-13262:
-
bq. Is there a reason you think that we need to introduce a new attribute on
the message though
Good point. Perhaps the current moreResults flag could be modified to be
relevant to the region. My concern would be that the flag has already been
established as relevant to the scan as a whole rather than per region, so we
would have to be sure that any changes to the way that the flag is set is
understood globally and not misunderstood to mean that the whole scan should
stop. Also, if it does become about the region rather than the scan as a whole,
would there still be a mechanism for the server to indicate to the client that
the whole scan is done (would it still be result == null)?
bq. I think the best we can do is fall back to the old logic and pray we don't
run into the problem
Ya, I think you may be right
ResultScanner doesn't return all rows in Scan
-
Key: HBASE-13262
URL: https://issues.apache.org/jira/browse/HBASE-13262
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 2.0.0, 1.1.0
Environment: Single node, pseduo-distributed 1.1.0-SNAPSHOT
Reporter: Josh Elser
Assignee: Josh Elser
Priority: Blocker
Fix For: 2.0.0, 1.1.0, 0.98.13
Attachments: 13262-0.98-testpatch.txt, HBASE-13262-branch-1-v2.patch,
HBASE-13262-branch-1-v3.patch, HBASE-13262-branch-1.patch,
HBASE-13262-v1.patch, HBASE-13262-v2.patch, HBASE-13262-v3.patch,
HBASE-13262.patch, regionserver-logging.diff, testrun_0.98.txt,
testrun_branch1.0.txt
Tried to write a simple Java client again 1.1.0-SNAPSHOT.
* Write 1M rows, each row with 1 family, and 10 qualifiers (values [0-9]),
for a total of 10M cells written
* Read back the data from the table, ensure I saw 10M cells
Running it against {{04ac1891}} (and earlier) yesterday, I would get ~20% of
the actual rows. Running against 1.0.0, returns all 10M records as expected.
[Code I was
running|https://github.com/joshelser/hbase-hwhat/blob/master/src/main/java/hbase/HBaseTest.java]
for the curious.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13273) Make Result.EMPTY_RESULT read-only; currently it can be modified
[ https://issues.apache.org/jira/browse/HBASE-13273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378681#comment-14378681 ] Sean Busbey commented on HBASE-13273: - I suggested that above, other folks liked the field better and I didn't feel strongly. (the v5 patch is in master, branch-1, and branch-1.0 right now. I'm just waiting on Andrew's decision to close.) Make Result.EMPTY_RESULT read-only; currently it can be modified Key: HBASE-13273 URL: https://issues.apache.org/jira/browse/HBASE-13273 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 1.0.0 Reporter: stack Assignee: Mikhail Antonov Labels: beginner Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13273-v5.patch, HBASE-13273-v5.patch, HBASE-13273.patch, HBASE-13273.patch, HBASE-13273.patch, HBASE-13273.patch Again from [~larsgeorge] Result result2 = Result.EMPTY_RESULT; System.out.println(result2); result2.copyFrom(result1); System.out.println(result2); What do you think happens when result1 has cells? Yep, you just modified the shared public EMPTY_RESULT to be not empty anymore. Fix. Result should be non-modifiable post-construction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13326) Disabled table can't be enabled after HBase is restarted
[ https://issues.apache.org/jira/browse/HBASE-13326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13326: - Attachment: HBASE-13326-branch-1.0.patch Disabled table can't be enabled after HBase is restarted Key: HBASE-13326 URL: https://issues.apache.org/jira/browse/HBASE-13326 Project: HBase Issue Type: Bug Affects Versions: 1.0.0, 1.1.0, 0.98.12 Reporter: Dima Spivak Assignee: Andrey Stepachev Priority: Blocker Fix For: 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13326-branch-1.0.patch, HBASE-13326-branch-1.patch The folks at Intel discovered a pretty nasty bug in 1.0 and 1.1 (but not master). Steps to reproduce: 1. Create a table, any table. 2. Disable the table. 3. Restart HBase. 4. Try enabling the table. The table won't become enabled and the master web UI will indicate a never-ending region in transition. Also worth noting is that [~mbertozzi] dug in and noted that this isn't happening in the master branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13331) Exceptions from DFS client can cause CatalogJanitor to delete referenced files
[ https://issues.apache.org/jira/browse/HBASE-13331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378659#comment-14378659 ] Andrew Purtell commented on HBASE-13331: +1 Exceptions from DFS client can cause CatalogJanitor to delete referenced files -- Key: HBASE-13331 URL: https://issues.apache.org/jira/browse/HBASE-13331 Project: HBase Issue Type: Bug Components: master Affects Versions: 1.0.0, 0.98.12 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Blocker Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13331.patch CatalogJanitor#checkDaughterInFs assumes that there are no references whenever HRegionFileSystem.openRegionFromFileSystem throws IOException. Well Hadoop and HBase throw IOExceptions whenever someone looks in their general direction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13323) Audit behavior heterogenous node capacity
[ https://issues.apache.org/jira/browse/HBASE-13323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378668#comment-14378668 ] Andrew Purtell commented on HBASE-13323: I think currently the HDFS balancer should be considered harmful (in larger deploys), see HDFS-7967 Audit behavior heterogenous node capacity - Key: HBASE-13323 URL: https://issues.apache.org/jira/browse/HBASE-13323 Project: HBase Issue Type: Task Components: Balancer Reporter: Nick Dimiduk Labels: beginner From the thread introducing nodes w/ more storage (http://search-hadoop.com/m/DHED4azyle2), we should have a look at what happens when nodes of varying data density are used in a single cluster. The user would expect that nodes be filled according to their capacity, meaning an even distribution looks like all nodes at the same pct use. This behavior is probably in the intersection of hbase balancer and hdfs balancer. Probably this is made more complex by recent HDFS features such as HDFS-5682. After investigation, let's fix it up to work better (if it's broken), and document the behavior in our awesome book. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13326) Disabled table can't be enabled after HBase is restarted
[ https://issues.apache.org/jira/browse/HBASE-13326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378691#comment-14378691 ] Andrew Purtell commented on HBASE-13326: Reason I ask is we don't have a .12 RC at the moment, and if this is a critical for .13, it should go into .12 at this opportunity. Disabled table can't be enabled after HBase is restarted Key: HBASE-13326 URL: https://issues.apache.org/jira/browse/HBASE-13326 Project: HBase Issue Type: Bug Affects Versions: 1.0.0, 1.1.0, 0.98.12 Reporter: Dima Spivak Assignee: Andrey Stepachev Priority: Blocker Fix For: 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13326-brach-1.0.patch, HBASE-13326-brach-1.patch The folks at Intel discovered a pretty nasty bug in 1.0 and 1.1 (but not master). Steps to reproduce: 1. Create a table, any table. 2. Disable the table. 3. Restart HBase. 4. Try enabling the table. The table won't become enabled and the master web UI will indicate a never-ending region in transition. Also worth noting is that [~mbertozzi] dug in and noted that this isn't happening in the master branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13273) Make Result.EMPTY_RESULT read-only; currently it can be modified
[ https://issues.apache.org/jira/browse/HBASE-13273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378701#comment-14378701 ] Enis Soztutar commented on HBASE-13273: --- bq. I suggested that above, other folks liked the field better and I didn't feel strongly. (the v5 patch is in master, branch-1, and branch-1.0 right now. I'm just waiting on Andrew's decision to close.) Sorry I skimmed through the comments in the jira, went directly to the patch. It seems what we want is the Result object being immutable, but also having a Result.Builder to build immutable Results. Main argument was that we do not want to pay the price (in heap) for all the Result object which are normally not read only. Make Result.EMPTY_RESULT read-only; currently it can be modified Key: HBASE-13273 URL: https://issues.apache.org/jira/browse/HBASE-13273 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 1.0.0 Reporter: stack Assignee: Mikhail Antonov Labels: beginner Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13273-v5.patch, HBASE-13273-v5.patch, HBASE-13273.patch, HBASE-13273.patch, HBASE-13273.patch, HBASE-13273.patch Again from [~larsgeorge] Result result2 = Result.EMPTY_RESULT; System.out.println(result2); result2.copyFrom(result1); System.out.println(result2); What do you think happens when result1 has cells? Yep, you just modified the shared public EMPTY_RESULT to be not empty anymore. Fix. Result should be non-modifiable post-construction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13326) Disabled table can't be enabled after HBase is restarted
[ https://issues.apache.org/jira/browse/HBASE-13326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13326: - Status: Open (was: Patch Available) Disabled table can't be enabled after HBase is restarted Key: HBASE-13326 URL: https://issues.apache.org/jira/browse/HBASE-13326 Project: HBase Issue Type: Bug Affects Versions: 0.98.12, 1.0.0, 1.1.0 Reporter: Dima Spivak Assignee: Andrey Stepachev Priority: Blocker Fix For: 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13326-branch-1.0.patch, HBASE-13326-branch-1.patch The folks at Intel discovered a pretty nasty bug in 1.0 and 1.1 (but not master). Steps to reproduce: 1. Create a table, any table. 2. Disable the table. 3. Restart HBase. 4. Try enabling the table. The table won't become enabled and the master web UI will indicate a never-ending region in transition. Also worth noting is that [~mbertozzi] dug in and noted that this isn't happening in the master branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13326) Disabled table can't be enabled after HBase is restarted
[ https://issues.apache.org/jira/browse/HBASE-13326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13326: - Status: Patch Available (was: Open) Disabled table can't be enabled after HBase is restarted Key: HBASE-13326 URL: https://issues.apache.org/jira/browse/HBASE-13326 Project: HBase Issue Type: Bug Affects Versions: 0.98.12, 1.0.0, 1.1.0 Reporter: Dima Spivak Assignee: Andrey Stepachev Priority: Blocker Fix For: 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13326-branch-1.0.patch, HBASE-13326-branch-1.patch The folks at Intel discovered a pretty nasty bug in 1.0 and 1.1 (but not master). Steps to reproduce: 1. Create a table, any table. 2. Disable the table. 3. Restart HBase. 4. Try enabling the table. The table won't become enabled and the master web UI will indicate a never-ending region in transition. Also worth noting is that [~mbertozzi] dug in and noted that this isn't happening in the master branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13326) Disabled table can't be enabled after HBase is restarted
[
https://issues.apache.org/jira/browse/HBASE-13326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378770#comment-14378770
]
Hadoop QA commented on HBASE-13326:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707021/HBASE-13326-branch-1.0.patch
against branch-1.0 branch at commit 8cb4f89c013b254c6d3e9bee7b63137a391bcaba.
ATTACHMENT ID: 12707021
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:red}-1 javac{color}. The patch appears to cause mvn compile goal to
fail with Hadoop version 2.4.1.
Compilation errors resume:
[ERROR] COMPILATION ERROR :
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/http/ssl/KeyStoreTestUtil.java:[46,24]
AlgorithmId is internal proprietary API and may be removed in a future release
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/http/ssl/KeyStoreTestUtil.java:[47,24]
CertificateAlgorithmId is internal proprietary API and may be removed in a
future release
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/http/ssl/KeyStoreTestUtil.java:[48,24]
CertificateIssuerName is internal proprietary API and may be removed in a
future release
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/http/ssl/KeyStoreTestUtil.java:[49,24]
CertificateSerialNumber is internal proprietary API and may be removed in a
future release
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/http/ssl/KeyStoreTestUtil.java:[50,24]
CertificateSubjectName is internal proprietary API and may be removed in a
future release
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/http/ssl/KeyStoreTestUtil.java:[51,24]
CertificateValidity is internal proprietary API and may be removed in a future
release
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/http/ssl/KeyStoreTestUtil.java:[52,24]
CertificateVersion is internal proprietary API and may be removed in a future
release
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/http/ssl/KeyStoreTestUtil.java:[53,24]
CertificateX509Key is internal proprietary API and may be removed in a future
release
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/http/ssl/KeyStoreTestUtil.java:[54,24]
X500Name is internal proprietary API and may be removed in a future release
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/http/ssl/KeyStoreTestUtil.java:[55,24]
X509CertImpl is internal proprietary API and may be removed in a future release
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/http/ssl/KeyStoreTestUtil.java:[56,24]
X509CertInfo is internal proprietary API and may be removed in a future release
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/http/ssl/KeyStoreTestUtil.java:[85,4]
X509CertInfo is internal proprietary API and may be removed in a future release
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/http/ssl/KeyStoreTestUtil.java:[85,28]
X509CertInfo is internal proprietary API and may be removed in a future release
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/http/ssl/KeyStoreTestUtil.java:[88,4]
CertificateValidity is internal proprietary API and may be removed in a future
release
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/http/ssl/KeyStoreTestUtil.java:[88,39]
CertificateValidity is internal proprietary API and may be removed in a future
release
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/http/ssl/KeyStoreTestUtil.java:[90,4]
X500Name is internal proprietary API and may be removed in a future release
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/http/ssl/KeyStoreTestUtil.java:[90,25]
X500Name is internal proprietary
[jira] [Commented] (HBASE-13331) Exceptions from DFS client can cause CatalogJanitor to delete referenced files
[ https://issues.apache.org/jira/browse/HBASE-13331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378826#comment-14378826 ] Benoit Sigoure commented on HBASE-13331: [How Apache Hadoop is molesting IOException all day|http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html] Exceptions from DFS client can cause CatalogJanitor to delete referenced files -- Key: HBASE-13331 URL: https://issues.apache.org/jira/browse/HBASE-13331 Project: HBase Issue Type: Bug Components: master Affects Versions: 1.0.0, 0.98.12 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Blocker Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13331.patch CatalogJanitor#checkDaughterInFs assumes that there are no references whenever HRegionFileSystem.openRegionFromFileSystem throws IOException. Well Hadoop and HBase throw IOExceptions whenever someone looks in their general direction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13273) Make Result.EMPTY_RESULT read-only; currently it can be modified
[ https://issues.apache.org/jira/browse/HBASE-13273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-13273: Resolution: Fixed Fix Version/s: (was: 0.98.13) 0.98.12 Release Note: The Result.EMPTY_RESULT object is now immutable. In previous releases, the object could be modified by a caller to no longer be empty. Code that relies on this behavior will now receive an UnsupportedOperationException. Status: Resolved (was: Patch Available) Pushed to 0.98+. Thanks Mikhail! Make Result.EMPTY_RESULT read-only; currently it can be modified Key: HBASE-13273 URL: https://issues.apache.org/jira/browse/HBASE-13273 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 1.0.0 Reporter: stack Assignee: Mikhail Antonov Labels: beginner Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.12 Attachments: HBASE-13273-v5.patch, HBASE-13273-v5.patch, HBASE-13273.patch, HBASE-13273.patch, HBASE-13273.patch, HBASE-13273.patch Again from [~larsgeorge] Result result2 = Result.EMPTY_RESULT; System.out.println(result2); result2.copyFrom(result1); System.out.println(result2); What do you think happens when result1 has cells? Yep, you just modified the shared public EMPTY_RESULT to be not empty anymore. Fix. Result should be non-modifiable post-construction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13294) Fix the critical ancient loopholes in security testing infrastructure.
[
https://issues.apache.org/jira/browse/HBASE-13294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378825#comment-14378825
]
Srikanth Srungarapu commented on HBASE-13294:
-
Attached the patch for branch-1.
Fix the critical ancient loopholes in security testing infrastructure.
--
Key: HBASE-13294
URL: https://issues.apache.org/jira/browse/HBASE-13294
Project: HBase
Issue Type: Bug
Reporter: Srikanth Srungarapu
Assignee: Srikanth Srungarapu
Attachments: HBASE-13294-0.98.patch, HBASE-13294-0.98.patch,
HBASE-13294-branch-1.patch, HBASE-13294.patch, HBASE-13294_v2.patch,
HBASE-13294_v3.patch, HBASE-13294_v3.patch, HBASE-13294_v4.patch,
HBASE-13294_v5.patch, HBASE-13294_v6.patch, HBASE-13294_v6.patch
Unfortunately, the verifyDenied method doesn't fail when action parameter
returns null. The relevant code snippet
{code}
try {
Object obj = user.runAs(action);
if (requireException) {
fail(Expected exception was not thrown for user ' +
user.getShortName() + ');
}
if (obj != null obj instanceof List?) {
List? results = (List?) obj;
if (results != null !results.isEmpty()) {
fail(Unexpected results for user ' + user.getShortName() + ');
}
}
}
{code}
As you can see, when obj is null, it returns silently.
Fixing this issue has uncovered another major bug. While constructing
actions, we're using TEST_UTIL.getConnection(), which replaces the doAs
user with the user who initiated the connection. I really am grateful to
[~mbertozzi] without whom debugging this would have been a nightmare.
Now, fixing these two issues have uncovered more issues in our tests :). The
main one is we're allowing the table owner to truncate table in code. But, in
test, we're not allowing him. We should either remove the code that allows
owner or document that the table owner can truncate table.
The other minor issues include granting permissions to namespace, but
checking whether user was able to access tables inside other namespace.
That's it, folks!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13326) Disabled table can't be enabled after HBase is restarted
[ https://issues.apache.org/jira/browse/HBASE-13326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378685#comment-14378685 ] Andrew Purtell commented on HBASE-13326: I see a fix version for 0.98, is that correct? I was planning to check if 0.98 is affected by this later today. Disabled table can't be enabled after HBase is restarted Key: HBASE-13326 URL: https://issues.apache.org/jira/browse/HBASE-13326 Project: HBase Issue Type: Bug Affects Versions: 1.0.0, 1.1.0, 0.98.12 Reporter: Dima Spivak Assignee: Andrey Stepachev Priority: Blocker Fix For: 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13326-brach-1.0.patch, HBASE-13326-brach-1.patch The folks at Intel discovered a pretty nasty bug in 1.0 and 1.1 (but not master). Steps to reproduce: 1. Create a table, any table. 2. Disable the table. 3. Restart HBase. 4. Try enabling the table. The table won't become enabled and the master web UI will indicate a never-ending region in transition. Also worth noting is that [~mbertozzi] dug in and noted that this isn't happening in the master branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13273) Make Result.EMPTY_RESULT read-only; currently it can be modified
[ https://issues.apache.org/jira/browse/HBASE-13273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378714#comment-14378714 ] Mikhail Antonov commented on HBASE-13273: - [~enis] yeah, this object is sure on the hot code path, just looking at the Result class structure (fields) and taking into account common object overhead (header, word alignment) I kind of doubt adding one boolean field to this class provides for measurable difference is object size.. Make Result.EMPTY_RESULT read-only; currently it can be modified Key: HBASE-13273 URL: https://issues.apache.org/jira/browse/HBASE-13273 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 1.0.0 Reporter: stack Assignee: Mikhail Antonov Labels: beginner Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13273-v5.patch, HBASE-13273-v5.patch, HBASE-13273.patch, HBASE-13273.patch, HBASE-13273.patch, HBASE-13273.patch Again from [~larsgeorge] Result result2 = Result.EMPTY_RESULT; System.out.println(result2); result2.copyFrom(result1); System.out.println(result2); What do you think happens when result1 has cells? Yep, you just modified the shared public EMPTY_RESULT to be not empty anymore. Fix. Result should be non-modifiable post-construction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13331) Exceptions from DFS client can cause CatalogJanitor to delete referenced files
[ https://issues.apache.org/jira/browse/HBASE-13331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378712#comment-14378712 ] stack commented on HBASE-13331: --- bq. Well Hadoop and HBase throw IOExceptions whenever someone looks in their general direction. lol [~tsuna] See above. @eclark So when you went to open the daughter region, you got exception trying to read daughter region from fs? Is isExists a canary to test HDFS good before we go to use it? Patch LGTM. Exceptions from DFS client can cause CatalogJanitor to delete referenced files -- Key: HBASE-13331 URL: https://issues.apache.org/jira/browse/HBASE-13331 Project: HBase Issue Type: Bug Components: master Affects Versions: 1.0.0, 0.98.12 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Blocker Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13331.patch CatalogJanitor#checkDaughterInFs assumes that there are no references whenever HRegionFileSystem.openRegionFromFileSystem throws IOException. Well Hadoop and HBase throw IOExceptions whenever someone looks in their general direction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13262) ResultScanner doesn't return all rows in Scan
[
https://issues.apache.org/jira/browse/HBASE-13262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378731#comment-14378731
]
Josh Elser commented on HBASE-13262:
bq. would there still be a mechanism for the server to indicate to the client
that the whole scan is done
Ah, that's a good point. So, as I understand it, we have the following cases
which the RS will return to a client:
* {{Result}}s and more results on this region
* {{Result}}s and no more result on this region (this was the original bug)
* No {{Result}}s and no more results on this region (but we need to check the
other Regions)
* No {{Result}}s and the filter is done (this filter case)
It might be possible to differentiate between the 3 and 4th point if we knew
when {{Result[]}} was null instead of just empty, specifically, for the filter
case, if the client could get a ScanResponse in which the {{Result[]}} was
actually null, we could still keep them separate. I'm not a big fan of that
though, nor am I convinced it would even work. I don't quite understand how the
non-protobuf serialization of {{Result}}s works presently.
That said, hasResults could be left solely for the filter-termination case, and
I could introduce a moreResultsInRegion attribute after all. hasResults would
still be a bit goofy WRT its name, but some comments can make it clear that
it's purpose is explicitly for the filter case and nothing else.
ResultScanner doesn't return all rows in Scan
-
Key: HBASE-13262
URL: https://issues.apache.org/jira/browse/HBASE-13262
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 2.0.0, 1.1.0
Environment: Single node, pseduo-distributed 1.1.0-SNAPSHOT
Reporter: Josh Elser
Assignee: Josh Elser
Priority: Blocker
Fix For: 2.0.0, 1.1.0, 0.98.13
Attachments: 13262-0.98-testpatch.txt, HBASE-13262-branch-1-v2.patch,
HBASE-13262-branch-1-v3.patch, HBASE-13262-branch-1.patch,
HBASE-13262-v1.patch, HBASE-13262-v2.patch, HBASE-13262-v3.patch,
HBASE-13262.patch, regionserver-logging.diff, testrun_0.98.txt,
testrun_branch1.0.txt
Tried to write a simple Java client again 1.1.0-SNAPSHOT.
* Write 1M rows, each row with 1 family, and 10 qualifiers (values [0-9]),
for a total of 10M cells written
* Read back the data from the table, ensure I saw 10M cells
Running it against {{04ac1891}} (and earlier) yesterday, I would get ~20% of
the actual rows. Running against 1.0.0, returns all 10M records as expected.
[Code I was
running|https://github.com/joshelser/hbase-hwhat/blob/master/src/main/java/hbase/HBaseTest.java]
for the curious.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13327) Use Admin in ConnectionCache
[ https://issues.apache.org/jira/browse/HBASE-13327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Solomon Duskis updated HBASE-13327: --- Fix Version/s: 1.1.0 Use Admin in ConnectionCache Key: HBASE-13327 URL: https://issues.apache.org/jira/browse/HBASE-13327 Project: HBase Issue Type: Sub-task Affects Versions: 2.0.0, 1.0.1, 1.1.0 Reporter: Solomon Duskis Assignee: Solomon Duskis Fix For: 2.0.0, 1.0.1, 1.1.0 Attachments: HBASE-13327.patch Replace HBaseAdmin usage with Admin in ConnectionCache and users of ConnectionCache. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13327) Use Admin in ConnectionCache
[ https://issues.apache.org/jira/browse/HBASE-13327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378751#comment-14378751 ] Enis Soztutar commented on HBASE-13327: --- +1. Use Admin in ConnectionCache Key: HBASE-13327 URL: https://issues.apache.org/jira/browse/HBASE-13327 Project: HBase Issue Type: Sub-task Affects Versions: 2.0.0, 1.0.1, 1.1.0 Reporter: Solomon Duskis Assignee: Solomon Duskis Fix For: 2.0.0, 1.0.1, 1.1.0 Attachments: HBASE-13327.patch Replace HBaseAdmin usage with Admin in ConnectionCache and users of ConnectionCache. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13294) Fix the critical ancient loopholes in security testing infrastructure.
[
https://issues.apache.org/jira/browse/HBASE-13294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Srikanth Srungarapu updated HBASE-13294:
Attachment: HBASE-13294-branch-1.patch
Fix the critical ancient loopholes in security testing infrastructure.
--
Key: HBASE-13294
URL: https://issues.apache.org/jira/browse/HBASE-13294
Project: HBase
Issue Type: Bug
Reporter: Srikanth Srungarapu
Assignee: Srikanth Srungarapu
Attachments: HBASE-13294-0.98.patch, HBASE-13294-0.98.patch,
HBASE-13294-branch-1.patch, HBASE-13294.patch, HBASE-13294_v2.patch,
HBASE-13294_v3.patch, HBASE-13294_v3.patch, HBASE-13294_v4.patch,
HBASE-13294_v5.patch, HBASE-13294_v6.patch, HBASE-13294_v6.patch
Unfortunately, the verifyDenied method doesn't fail when action parameter
returns null. The relevant code snippet
{code}
try {
Object obj = user.runAs(action);
if (requireException) {
fail(Expected exception was not thrown for user ' +
user.getShortName() + ');
}
if (obj != null obj instanceof List?) {
List? results = (List?) obj;
if (results != null !results.isEmpty()) {
fail(Unexpected results for user ' + user.getShortName() + ');
}
}
}
{code}
As you can see, when obj is null, it returns silently.
Fixing this issue has uncovered another major bug. While constructing
actions, we're using TEST_UTIL.getConnection(), which replaces the doAs
user with the user who initiated the connection. I really am grateful to
[~mbertozzi] without whom debugging this would have been a nightmare.
Now, fixing these two issues have uncovered more issues in our tests :). The
main one is we're allowing the table owner to truncate table in code. But, in
test, we're not allowing him. We should either remove the code that allows
owner or document that the table owner can truncate table.
The other minor issues include granting permissions to namespace, but
checking whether user was able to access tables inside other namespace.
That's it, folks!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13331) Exceptions from DFS client can cause CatalogJanitor to delete referenced files
[ https://issues.apache.org/jira/browse/HBASE-13331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378858#comment-14378858 ] Elliott Clark commented on HBASE-13331: --- When I went to open the daughter region that failed. The daughter region contained references to the parent. HMaster had deleted the parent because it couldn't open the daughter region and assumed that was because it didn't exist. That's because the best way to signal that a directory doesn't exist is obviously to throw a generic exception. I added the !FSUtils.isExists to explicitly handle the case of the janitor not running before there is another split that removes the daughter region. Exceptions from DFS client can cause CatalogJanitor to delete referenced files -- Key: HBASE-13331 URL: https://issues.apache.org/jira/browse/HBASE-13331 Project: HBase Issue Type: Bug Components: master Affects Versions: 1.0.0, 0.98.12 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Blocker Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13331.patch CatalogJanitor#checkDaughterInFs assumes that there are no references whenever HRegionFileSystem.openRegionFromFileSystem throws IOException. Well Hadoop and HBase throw IOExceptions whenever someone looks in their general direction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13294) Fix the critical ancient loopholes in security testing infrastructure.
[
https://issues.apache.org/jira/browse/HBASE-13294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378676#comment-14378676
]
Andrew Purtell commented on HBASE-13294:
The checkstyle and javadoc audits aren't calibrated in/for 0.98 builds, but
thanks for checking.
+1
Let me try this out a bit locally and then commit.
Fix the critical ancient loopholes in security testing infrastructure.
--
Key: HBASE-13294
URL: https://issues.apache.org/jira/browse/HBASE-13294
Project: HBase
Issue Type: Bug
Reporter: Srikanth Srungarapu
Assignee: Srikanth Srungarapu
Attachments: HBASE-13294-0.98.patch, HBASE-13294-0.98.patch,
HBASE-13294.patch, HBASE-13294_v2.patch, HBASE-13294_v3.patch,
HBASE-13294_v3.patch, HBASE-13294_v4.patch, HBASE-13294_v5.patch,
HBASE-13294_v6.patch, HBASE-13294_v6.patch
Unfortunately, the verifyDenied method doesn't fail when action parameter
returns null. The relevant code snippet
{code}
try {
Object obj = user.runAs(action);
if (requireException) {
fail(Expected exception was not thrown for user ' +
user.getShortName() + ');
}
if (obj != null obj instanceof List?) {
List? results = (List?) obj;
if (results != null !results.isEmpty()) {
fail(Unexpected results for user ' + user.getShortName() + ');
}
}
}
{code}
As you can see, when obj is null, it returns silently.
Fixing this issue has uncovered another major bug. While constructing
actions, we're using TEST_UTIL.getConnection(), which replaces the doAs
user with the user who initiated the connection. I really am grateful to
[~mbertozzi] without whom debugging this would have been a nightmare.
Now, fixing these two issues have uncovered more issues in our tests :). The
main one is we're allowing the table owner to truncate table in code. But, in
test, we're not allowing him. We should either remove the code that allows
owner or document that the table owner can truncate table.
The other minor issues include granting permissions to namespace, but
checking whether user was able to access tables inside other namespace.
That's it, folks!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13326) Disabled table can't be enabled after HBase is restarted
[ https://issues.apache.org/jira/browse/HBASE-13326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13326: - Attachment: (was: HBASE-13326-brach-1.patch) Disabled table can't be enabled after HBase is restarted Key: HBASE-13326 URL: https://issues.apache.org/jira/browse/HBASE-13326 Project: HBase Issue Type: Bug Affects Versions: 1.0.0, 1.1.0, 0.98.12 Reporter: Dima Spivak Assignee: Andrey Stepachev Priority: Blocker Fix For: 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13326-branch-1.patch The folks at Intel discovered a pretty nasty bug in 1.0 and 1.1 (but not master). Steps to reproduce: 1. Create a table, any table. 2. Disable the table. 3. Restart HBase. 4. Try enabling the table. The table won't become enabled and the master web UI will indicate a never-ending region in transition. Also worth noting is that [~mbertozzi] dug in and noted that this isn't happening in the master branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13326) Disabled table can't be enabled after HBase is restarted
[ https://issues.apache.org/jira/browse/HBASE-13326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13326: - Attachment: HBASE-13326-branch-1.patch Disabled table can't be enabled after HBase is restarted Key: HBASE-13326 URL: https://issues.apache.org/jira/browse/HBASE-13326 Project: HBase Issue Type: Bug Affects Versions: 1.0.0, 1.1.0, 0.98.12 Reporter: Dima Spivak Assignee: Andrey Stepachev Priority: Blocker Fix For: 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13326-branch-1.patch The folks at Intel discovered a pretty nasty bug in 1.0 and 1.1 (but not master). Steps to reproduce: 1. Create a table, any table. 2. Disable the table. 3. Restart HBase. 4. Try enabling the table. The table won't become enabled and the master web UI will indicate a never-ending region in transition. Also worth noting is that [~mbertozzi] dug in and noted that this isn't happening in the master branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13326) Disabled table can't be enabled after HBase is restarted
[ https://issues.apache.org/jira/browse/HBASE-13326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378704#comment-14378704 ] Andrey Stepachev commented on HBASE-13326: -- [~dimaspivak] yeah, QA agreed with you :). thanks Disabled table can't be enabled after HBase is restarted Key: HBASE-13326 URL: https://issues.apache.org/jira/browse/HBASE-13326 Project: HBase Issue Type: Bug Affects Versions: 1.0.0, 1.1.0, 0.98.12 Reporter: Dima Spivak Assignee: Andrey Stepachev Priority: Blocker Fix For: 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13326-branch-1.patch The folks at Intel discovered a pretty nasty bug in 1.0 and 1.1 (but not master). Steps to reproduce: 1. Create a table, any table. 2. Disable the table. 3. Restart HBase. 4. Try enabling the table. The table won't become enabled and the master web UI will indicate a never-ending region in transition. Also worth noting is that [~mbertozzi] dug in and noted that this isn't happening in the master branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13326) Disabled table can't be enabled after HBase is restarted
[ https://issues.apache.org/jira/browse/HBASE-13326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13326: - Attachment: (was: HBASE-13326-brach-1.0.patch) Disabled table can't be enabled after HBase is restarted Key: HBASE-13326 URL: https://issues.apache.org/jira/browse/HBASE-13326 Project: HBase Issue Type: Bug Affects Versions: 1.0.0, 1.1.0, 0.98.12 Reporter: Dima Spivak Assignee: Andrey Stepachev Priority: Blocker Fix For: 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13326-branch-1.patch The folks at Intel discovered a pretty nasty bug in 1.0 and 1.1 (but not master). Steps to reproduce: 1. Create a table, any table. 2. Disable the table. 3. Restart HBase. 4. Try enabling the table. The table won't become enabled and the master web UI will indicate a never-ending region in transition. Also worth noting is that [~mbertozzi] dug in and noted that this isn't happening in the master branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11544) [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME
[ https://issues.apache.org/jira/browse/HBASE-11544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378726#comment-14378726 ] stack commented on HBASE-11544: --- bq. I'm thinking about branch-1.0 and 0.98. Or they'll remain susceptible to OOME. Ok. That helps. This patch is branch-1 and master only. Not 0.98 and 1.0. The latter two will not get benefit of this change. That ok [~apurtell] or you still want a patch that goes your route -- would be weird doing special casing when other changes coming in in 1.1 cause breakage (not interested in bringing this back in a patch release). [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME -- Key: HBASE-11544 URL: https://issues.apache.org/jira/browse/HBASE-11544 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jonathan Lawlor Priority: Critical Fix For: 2.0.0, 1.1.0 Attachments: Allocation_Hot_Spots.html, HBASE-11544-branch_1_0-v1.patch, HBASE-11544-branch_1_0-v2.patch, HBASE-11544-v1.patch, HBASE-11544-v2.patch, HBASE-11544-v3.patch, HBASE-11544-v4.patch, HBASE-11544-v5.patch, HBASE-11544-v6.patch, HBASE-11544-v6.patch, HBASE-11544-v6.patch, HBASE-11544-v7.patch, HBASE-11544-v8-branch-1.patch, HBASE-11544-v8.patch, gc.j.png, h.png, hits.j.png, m.png, mean.png, net.j.png, q (2).png Running some tests, I set hbase.client.scanner.caching=1000. Dataset has large cells. I kept OOME'ing. Serverside, we should measure how much we've accumulated and return to the client whatever we've gathered once we pass out a certain size threshold rather than keep accumulating till we OOME. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11544) [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME
[ https://issues.apache.org/jira/browse/HBASE-11544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378740#comment-14378740 ] Enis Soztutar commented on HBASE-11544: --- bq. Would like to see what non-breaking looks like first (and why wouldn't Phoenix want to make use of these amazing newness anyways? Ain't Region Interface coming in in 1.1.0 too so its going to have some work to do anyways? The patch at PHOENIX-1763 already contains some fixes on top of changes related to this patch. I think I can do the necessary changes after the addendum lands if the mapping is clear. If we have the ScannerContext, even the results should go in there, no? bq. and why wouldn't Phoenix want to make use of these amazing newness anyways? Phoenix right now does not make use of limit (in some cases) and remainingResultSize. Some of the Phoenix's scanners are transforming the data completely (group by aggregate scanner) which drives the underlying scan, and buffers in memory (with optional spill), but ignores remainingResultSize because it is not incorporated yet. It should be doable, but will require some work. [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME -- Key: HBASE-11544 URL: https://issues.apache.org/jira/browse/HBASE-11544 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jonathan Lawlor Priority: Critical Fix For: 2.0.0, 1.1.0 Attachments: Allocation_Hot_Spots.html, HBASE-11544-branch_1_0-v1.patch, HBASE-11544-branch_1_0-v2.patch, HBASE-11544-v1.patch, HBASE-11544-v2.patch, HBASE-11544-v3.patch, HBASE-11544-v4.patch, HBASE-11544-v5.patch, HBASE-11544-v6.patch, HBASE-11544-v6.patch, HBASE-11544-v6.patch, HBASE-11544-v7.patch, HBASE-11544-v8-branch-1.patch, HBASE-11544-v8.patch, gc.j.png, h.png, hits.j.png, m.png, mean.png, net.j.png, q (2).png Running some tests, I set hbase.client.scanner.caching=1000. Dataset has large cells. I kept OOME'ing. Serverside, we should measure how much we've accumulated and return to the client whatever we've gathered once we pass out a certain size threshold rather than keep accumulating till we OOME. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-13262) ResultScanner doesn't return all rows in Scan
[
https://issues.apache.org/jira/browse/HBASE-13262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378736#comment-14378736
]
Andrew Purtell edited comment on HBASE-13262 at 3/24/15 10:01 PM:
--
Linking HBASE-13303 here as a reminder that commit should be reverted once this
issue has a more comprehensive fix ready.
Edit: But to be clear, we'd need for that fix to include logic on the server
side that works safely with older clients, or we can't revert it. (We'd risk
skipping data with older clients if cells on the server carry tags.)
was (Author: apurtell):
Linking HBASE-13303 here as a reminder that commit should be reverted once this
issue has a more comprehensive fix ready.
ResultScanner doesn't return all rows in Scan
-
Key: HBASE-13262
URL: https://issues.apache.org/jira/browse/HBASE-13262
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 2.0.0, 1.1.0
Environment: Single node, pseduo-distributed 1.1.0-SNAPSHOT
Reporter: Josh Elser
Assignee: Josh Elser
Priority: Blocker
Fix For: 2.0.0, 1.1.0, 0.98.13
Attachments: 13262-0.98-testpatch.txt, HBASE-13262-branch-1-v2.patch,
HBASE-13262-branch-1-v3.patch, HBASE-13262-branch-1.patch,
HBASE-13262-v1.patch, HBASE-13262-v2.patch, HBASE-13262-v3.patch,
HBASE-13262.patch, regionserver-logging.diff, testrun_0.98.txt,
testrun_branch1.0.txt
Tried to write a simple Java client again 1.1.0-SNAPSHOT.
* Write 1M rows, each row with 1 family, and 10 qualifiers (values [0-9]),
for a total of 10M cells written
* Read back the data from the table, ensure I saw 10M cells
Running it against {{04ac1891}} (and earlier) yesterday, I would get ~20% of
the actual rows. Running against 1.0.0, returns all 10M records as expected.
[Code I was
running|https://github.com/joshelser/hbase-hwhat/blob/master/src/main/java/hbase/HBaseTest.java]
for the curious.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13326) Disabled table can't be enabled after HBase is restarted
[ https://issues.apache.org/jira/browse/HBASE-13326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13326: - Attachment: HBASE-13326-0.98.patch here is for 0.98 Disabled table can't be enabled after HBase is restarted Key: HBASE-13326 URL: https://issues.apache.org/jira/browse/HBASE-13326 Project: HBase Issue Type: Bug Affects Versions: 1.0.0, 1.1.0, 0.98.12 Reporter: Dima Spivak Assignee: Andrey Stepachev Priority: Blocker Fix For: 1.0.1, 1.1.0, 0.98.12 Attachments: HBASE-13326-0.98.patch, HBASE-13326-branch-1.0.patch, HBASE-13326-branch-1.patch The folks at Intel discovered a pretty nasty bug in 1.0 and 1.1 (but not master). Steps to reproduce: 1. Create a table, any table. 2. Disable the table. 3. Restart HBase. 4. Try enabling the table. The table won't become enabled and the master web UI will indicate a never-ending region in transition. Also worth noting is that [~mbertozzi] dug in and noted that this isn't happening in the master branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13326) Disabled table can't be enabled after HBase is restarted
[ https://issues.apache.org/jira/browse/HBASE-13326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13326: - Status: Patch Available (was: Open) Disabled table can't be enabled after HBase is restarted Key: HBASE-13326 URL: https://issues.apache.org/jira/browse/HBASE-13326 Project: HBase Issue Type: Bug Affects Versions: 0.98.12, 1.0.0, 1.1.0 Reporter: Dima Spivak Assignee: Andrey Stepachev Priority: Blocker Fix For: 1.0.1, 1.1.0, 0.98.12 Attachments: HBASE-13326-0.98.patch, HBASE-13326-branch-1.0.patch, HBASE-13326-branch-1.0.patch, HBASE-13326-branch-1.patch The folks at Intel discovered a pretty nasty bug in 1.0 and 1.1 (but not master). Steps to reproduce: 1. Create a table, any table. 2. Disable the table. 3. Restart HBase. 4. Try enabling the table. The table won't become enabled and the master web UI will indicate a never-ending region in transition. Also worth noting is that [~mbertozzi] dug in and noted that this isn't happening in the master branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13294) Fix the critical ancient loopholes in security testing infrastructure.
[
https://issues.apache.org/jira/browse/HBASE-13294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-13294:
---
Fix Version/s: 0.98.12
1.1.0
1.0.1
2.0.0
Fix the critical ancient loopholes in security testing infrastructure.
--
Key: HBASE-13294
URL: https://issues.apache.org/jira/browse/HBASE-13294
Project: HBase
Issue Type: Bug
Reporter: Srikanth Srungarapu
Assignee: Srikanth Srungarapu
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.12
Attachments: HBASE-13294-0.98.patch, HBASE-13294-0.98.patch,
HBASE-13294-branch-1.patch, HBASE-13294.patch, HBASE-13294_v2.patch,
HBASE-13294_v3.patch, HBASE-13294_v3.patch, HBASE-13294_v4.patch,
HBASE-13294_v5.patch, HBASE-13294_v6.patch, HBASE-13294_v6.patch
Unfortunately, the verifyDenied method doesn't fail when action parameter
returns null. The relevant code snippet
{code}
try {
Object obj = user.runAs(action);
if (requireException) {
fail(Expected exception was not thrown for user ' +
user.getShortName() + ');
}
if (obj != null obj instanceof List?) {
List? results = (List?) obj;
if (results != null !results.isEmpty()) {
fail(Unexpected results for user ' + user.getShortName() + ');
}
}
}
{code}
As you can see, when obj is null, it returns silently.
Fixing this issue has uncovered another major bug. While constructing
actions, we're using TEST_UTIL.getConnection(), which replaces the doAs
user with the user who initiated the connection. I really am grateful to
[~mbertozzi] without whom debugging this would have been a nightmare.
Now, fixing these two issues have uncovered more issues in our tests :). The
main one is we're allowing the table owner to truncate table in code. But, in
test, we're not allowing him. We should either remove the code that allows
owner or document that the table owner can truncate table.
The other minor issues include granting permissions to namespace, but
checking whether user was able to access tables inside other namespace.
That's it, folks!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13331) Exceptions from DFS client can cause CatalogJanitor to delete referenced files
[ https://issues.apache.org/jira/browse/HBASE-13331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378831#comment-14378831 ] Benoit Sigoure commented on HBASE-13331: Ah, see HBASE-5796 Exceptions from DFS client can cause CatalogJanitor to delete referenced files -- Key: HBASE-13331 URL: https://issues.apache.org/jira/browse/HBASE-13331 Project: HBase Issue Type: Bug Components: master Affects Versions: 1.0.0, 0.98.12 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Blocker Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13331.patch CatalogJanitor#checkDaughterInFs assumes that there are no references whenever HRegionFileSystem.openRegionFromFileSystem throws IOException. Well Hadoop and HBase throw IOExceptions whenever someone looks in their general direction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13273) Make Result.EMPTY_RESULT read-only; currently it can be modified
[ https://issues.apache.org/jira/browse/HBASE-13273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378833#comment-14378833 ] Mikhail Antonov commented on HBASE-13273: - Thanks for review and comments guys! Make Result.EMPTY_RESULT read-only; currently it can be modified Key: HBASE-13273 URL: https://issues.apache.org/jira/browse/HBASE-13273 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 1.0.0 Reporter: stack Assignee: Mikhail Antonov Labels: beginner Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.12 Attachments: HBASE-13273-v5.patch, HBASE-13273-v5.patch, HBASE-13273.patch, HBASE-13273.patch, HBASE-13273.patch, HBASE-13273.patch Again from [~larsgeorge] Result result2 = Result.EMPTY_RESULT; System.out.println(result2); result2.copyFrom(result1); System.out.println(result2); What do you think happens when result1 has cells? Yep, you just modified the shared public EMPTY_RESULT to be not empty anymore. Fix. Result should be non-modifiable post-construction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13331) Exceptions from DFS client can cause CatalogJanitor to delete referenced files
[ https://issues.apache.org/jira/browse/HBASE-13331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-13331: -- Status: Patch Available (was: Open) Exceptions from DFS client can cause CatalogJanitor to delete referenced files -- Key: HBASE-13331 URL: https://issues.apache.org/jira/browse/HBASE-13331 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.98.12, 1.0.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Blocker Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13331.patch, HBASE-13331.patch CatalogJanitor#checkDaughterInFs assumes that there are no references whenever HRegionFileSystem.openRegionFromFileSystem throws IOException. Well Hadoop and HBase throw IOExceptions whenever someone looks in their general direction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13331) Exceptions from DFS client can cause CatalogJanitor to delete referenced files
[ https://issues.apache.org/jira/browse/HBASE-13331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-13331: -- Attachment: HBASE-13331.patch Trying to kick off jenkins. Exceptions from DFS client can cause CatalogJanitor to delete referenced files -- Key: HBASE-13331 URL: https://issues.apache.org/jira/browse/HBASE-13331 Project: HBase Issue Type: Bug Components: master Affects Versions: 1.0.0, 0.98.12 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Blocker Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13331.patch, HBASE-13331.patch CatalogJanitor#checkDaughterInFs assumes that there are no references whenever HRegionFileSystem.openRegionFromFileSystem throws IOException. Well Hadoop and HBase throw IOExceptions whenever someone looks in their general direction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13262) ResultScanner doesn't return all rows in Scan
[
https://issues.apache.org/jira/browse/HBASE-13262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378931#comment-14378931
]
Jonathan Lawlor commented on HBASE-13262:
-
bq. as I understand it, we have the following cases which the RS will return to
a client
Those cases make sense to me
bq. I'm not a big fan of that though, nor am I convinced it would even work
I agree. I think it may be best to introduce a new flag to communicate this
information on a per region basis, but if anyone else has any ideas please
chime in.
bq. But to be clear, we'd need for that fix to include logic on the server side
that works safely with older clients, or we can't revert it
Ya, in the case that the client does not know to look for the flag, or in the
case that the server does not send the flag, the fix from HBASE-13303 will be
necessary. Because of this, I don't believe the fix currently in the works
would allow us to revert HBASE-13303, correct me if I'm wrong [~elserj].
ResultScanner doesn't return all rows in Scan
-
Key: HBASE-13262
URL: https://issues.apache.org/jira/browse/HBASE-13262
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 2.0.0, 1.1.0
Environment: Single node, pseduo-distributed 1.1.0-SNAPSHOT
Reporter: Josh Elser
Assignee: Josh Elser
Priority: Blocker
Fix For: 2.0.0, 1.1.0, 0.98.13
Attachments: 13262-0.98-testpatch.txt, HBASE-13262-branch-1-v2.patch,
HBASE-13262-branch-1-v3.patch, HBASE-13262-branch-1.patch,
HBASE-13262-v1.patch, HBASE-13262-v2.patch, HBASE-13262-v3.patch,
HBASE-13262.patch, regionserver-logging.diff, testrun_0.98.txt,
testrun_branch1.0.txt
Tried to write a simple Java client again 1.1.0-SNAPSHOT.
* Write 1M rows, each row with 1 family, and 10 qualifiers (values [0-9]),
for a total of 10M cells written
* Read back the data from the table, ensure I saw 10M cells
Running it against {{04ac1891}} (and earlier) yesterday, I would get ~20% of
the actual rows. Running against 1.0.0, returns all 10M records as expected.
[Code I was
running|https://github.com/joshelser/hbase-hwhat/blob/master/src/main/java/hbase/HBaseTest.java]
for the curious.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13262) ResultScanner doesn't return all rows in Scan
[
https://issues.apache.org/jira/browse/HBASE-13262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378736#comment-14378736
]
Andrew Purtell commented on HBASE-13262:
Linking HBASE-13303 here as a reminder that commit should be reverted once this
issue has a more comprehensive fix ready.
ResultScanner doesn't return all rows in Scan
-
Key: HBASE-13262
URL: https://issues.apache.org/jira/browse/HBASE-13262
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 2.0.0, 1.1.0
Environment: Single node, pseduo-distributed 1.1.0-SNAPSHOT
Reporter: Josh Elser
Assignee: Josh Elser
Priority: Blocker
Fix For: 2.0.0, 1.1.0, 0.98.13
Attachments: 13262-0.98-testpatch.txt, HBASE-13262-branch-1-v2.patch,
HBASE-13262-branch-1-v3.patch, HBASE-13262-branch-1.patch,
HBASE-13262-v1.patch, HBASE-13262-v2.patch, HBASE-13262-v3.patch,
HBASE-13262.patch, regionserver-logging.diff, testrun_0.98.txt,
testrun_branch1.0.txt
Tried to write a simple Java client again 1.1.0-SNAPSHOT.
* Write 1M rows, each row with 1 family, and 10 qualifiers (values [0-9]),
for a total of 10M cells written
* Read back the data from the table, ensure I saw 10M cells
Running it against {{04ac1891}} (and earlier) yesterday, I would get ~20% of
the actual rows. Running against 1.0.0, returns all 10M records as expected.
[Code I was
running|https://github.com/joshelser/hbase-hwhat/blob/master/src/main/java/hbase/HBaseTest.java]
for the curious.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13328) LoadIncrementalHFile.doBulkLoad(Path,HTable) should handle managed connections
[ https://issues.apache.org/jira/browse/HBASE-13328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378735#comment-14378735 ] Mikhail Antonov commented on HBASE-13328: - Yeah. +1 (non-binding), i'll link that jira for tracking. LoadIncrementalHFile.doBulkLoad(Path,HTable) should handle managed connections -- Key: HBASE-13328 URL: https://issues.apache.org/jira/browse/HBASE-13328 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 2.0.0, 1.1.0 Attachments: hbase-13328_v1.patch This seems to be a regression from HBASE-12783 discovered in testing Phoenix with 1.1.0-SNAPSHOT. Phoenix uses an HTable (with managed connection) to pass to doBulkLoad() which throws NeedUnmanagedConnectionException (see IndexToolIT.java and IndexTool.java in Phoenix). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11544) [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME
[ https://issues.apache.org/jira/browse/HBASE-11544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378758#comment-14378758 ] Andrew Purtell commented on HBASE-11544: bq. not interested in bringing this back in a patch release A bit odd considering we won't be fixing this issue in any active release, but I can't ask anyone to work on something they're not interested in (smile). It's also true the alternative I suggest is both more work and less clean from a coding perspective. [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME -- Key: HBASE-11544 URL: https://issues.apache.org/jira/browse/HBASE-11544 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jonathan Lawlor Priority: Critical Fix For: 2.0.0, 1.1.0 Attachments: Allocation_Hot_Spots.html, HBASE-11544-branch_1_0-v1.patch, HBASE-11544-branch_1_0-v2.patch, HBASE-11544-v1.patch, HBASE-11544-v2.patch, HBASE-11544-v3.patch, HBASE-11544-v4.patch, HBASE-11544-v5.patch, HBASE-11544-v6.patch, HBASE-11544-v6.patch, HBASE-11544-v6.patch, HBASE-11544-v7.patch, HBASE-11544-v8-branch-1.patch, HBASE-11544-v8.patch, gc.j.png, h.png, hits.j.png, m.png, mean.png, net.j.png, q (2).png Running some tests, I set hbase.client.scanner.caching=1000. Dataset has large cells. I kept OOME'ing. Serverside, we should measure how much we've accumulated and return to the client whatever we've gathered once we pass out a certain size threshold rather than keep accumulating till we OOME. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13294) Fix the critical ancient loopholes in security testing infrastructure.
[
https://issues.apache.org/jira/browse/HBASE-13294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378756#comment-14378756
]
Hadoop QA commented on HBASE-13294:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12706979/HBASE-13294-0.98.patch
against 0.98 branch at commit fcc09dcd38b8543a9dd40212206cb4231620dac9.
ATTACHMENT ID: 12706979
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 18 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
24 warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:red}-1 findbugs{color}. The patch appears to introduce 8 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13395//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13395//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13395//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13395//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13395//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13395//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13395//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13395//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13395//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13395//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13395//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13395//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13395//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13395//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13395//console
This message is automatically generated.
Fix the critical ancient loopholes in security testing infrastructure.
--
Key: HBASE-13294
URL: https://issues.apache.org/jira/browse/HBASE-13294
Project: HBase
Issue Type: Bug
Reporter: Srikanth Srungarapu
Assignee: Srikanth Srungarapu
Attachments: HBASE-13294-0.98.patch, HBASE-13294-0.98.patch,
HBASE-13294.patch, HBASE-13294_v2.patch, HBASE-13294_v3.patch,
HBASE-13294_v3.patch, HBASE-13294_v4.patch, HBASE-13294_v5.patch,
HBASE-13294_v6.patch, HBASE-13294_v6.patch
Unfortunately, the verifyDenied method doesn't fail when action parameter
returns null. The relevant code snippet
{code}
try {
Object obj = user.runAs(action);
if (requireException) {
fail(Expected exception was not thrown for user ' +
user.getShortName() + ');
}
if (obj != null obj instanceof List?) {
List? results = (List?) obj;
if (results != null !results.isEmpty()) {
fail(Unexpected results for user ' + user.getShortName() + ');
[jira] [Updated] (HBASE-13326) Disabled table can't be enabled after HBase is restarted
[ https://issues.apache.org/jira/browse/HBASE-13326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-13326: --- Fix Version/s: (was: 0.98.13) 0.98.12 Disabled table can't be enabled after HBase is restarted Key: HBASE-13326 URL: https://issues.apache.org/jira/browse/HBASE-13326 Project: HBase Issue Type: Bug Affects Versions: 1.0.0, 1.1.0, 0.98.12 Reporter: Dima Spivak Assignee: Andrey Stepachev Priority: Blocker Fix For: 1.0.1, 1.1.0, 0.98.12 Attachments: HBASE-13326-branch-1.0.patch, HBASE-13326-branch-1.patch The folks at Intel discovered a pretty nasty bug in 1.0 and 1.1 (but not master). Steps to reproduce: 1. Create a table, any table. 2. Disable the table. 3. Restart HBase. 4. Try enabling the table. The table won't become enabled and the master web UI will indicate a never-ending region in transition. Also worth noting is that [~mbertozzi] dug in and noted that this isn't happening in the master branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13082) Coarsen StoreScanner locks to RegionScanner
[ https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378784#comment-14378784 ] stack commented on HBASE-13082: --- I ran this patch on my rig and see ~6-7% more throughput (high concurrency, scan1000, medium-sized rows), 7750k/s vs 8250k/s. No big CPU difference that I can discern... perhaps smoother with this patch in place. Coarsen StoreScanner locks to RegionScanner --- Key: HBASE-13082 URL: https://issues.apache.org/jira/browse/HBASE-13082 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Assignee: Lars Hofhansl Attachments: 13082-test.txt, 13082-v2.txt, 13082-v3.txt, 13082-v4.txt, 13082.txt, 13082.txt, gc.png, gc.png, gc.png, hits.png, next.png, next.png Continuing where HBASE-10015 left of. We can avoid locking (and memory fencing) inside StoreScanner by deferring to the lock already held by the RegionScanner. In tests this shows quite a scan improvement and reduced CPU (the fences make the cores wait for memory fetches). There are some drawbacks too: * All calls to RegionScanner need to be remain synchronized * Implementors of coprocessors need to be diligent in following the locking contract. For example Phoenix does not lock RegionScanner.nextRaw() and required in the documentation (not picking on Phoenix, this one is my fault as I told them it's OK) * possible starving of flushes and compaction with heavy read load. RegionScanner operations would keep getting the locks and the flushes/compactions would not be able finalize the set of files. I'll have a patch soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13332) Fix the usage of doAs/runAs in Visibility Controller tests.
Srikanth Srungarapu created HBASE-13332: --- Summary: Fix the usage of doAs/runAs in Visibility Controller tests. Key: HBASE-13332 URL: https://issues.apache.org/jira/browse/HBASE-13332 Project: HBase Issue Type: Sub-task Reporter: Srikanth Srungarapu Assignee: Srikanth Srungarapu The doAs bug is also affecting VC related tests too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13262) ResultScanner doesn't return all rows in Scan
[
https://issues.apache.org/jira/browse/HBASE-13262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378894#comment-14378894
]
Hadoop QA commented on HBASE-13262:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707001/HBASE-13262-v3.patch
against master branch at commit 8cb4f89c013b254c6d3e9bee7b63137a391bcaba.
ATTACHMENT ID: 12707001
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 16 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13396//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13396//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13396//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13396//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13396//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13396//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13396//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13396//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13396//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13396//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13396//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13396//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13396//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13396//console
This message is automatically generated.
ResultScanner doesn't return all rows in Scan
-
Key: HBASE-13262
URL: https://issues.apache.org/jira/browse/HBASE-13262
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 2.0.0, 1.1.0
Environment: Single node, pseduo-distributed 1.1.0-SNAPSHOT
Reporter: Josh Elser
Assignee: Josh Elser
Priority: Blocker
Fix For: 2.0.0, 1.1.0, 0.98.13
Attachments: 13262-0.98-testpatch.txt, HBASE-13262-branch-1-v2.patch,
HBASE-13262-branch-1-v3.patch, HBASE-13262-branch-1.patch,
HBASE-13262-v1.patch, HBASE-13262-v2.patch, HBASE-13262-v3.patch,
HBASE-13262.patch, regionserver-logging.diff, testrun_0.98.txt,
testrun_branch1.0.txt
Tried to write a simple Java client again 1.1.0-SNAPSHOT.
* Write 1M rows, each row with 1 family, and 10 qualifiers (values [0-9]),
for a total of 10M cells written
* Read back the data from the table, ensure I saw 10M cells
Running it against {{04ac1891}} (and earlier) yesterday, I would get ~20% of
the actual rows. Running against 1.0.0, returns all 10M records as expected.
[Code I was
[jira] [Commented] (HBASE-13328) LoadIncrementalHFile.doBulkLoad(Path,HTable) should handle managed connections
[ https://issues.apache.org/jira/browse/HBASE-13328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378672#comment-14378672 ] Enis Soztutar commented on HBASE-13328: --- bq. I assume it's related also to HBASE-13197, as managed connection would go away in 2.0? Indeed. But it is still a regression in branch-1. LoadIncrementalHFile.doBulkLoad(Path,HTable) should handle managed connections -- Key: HBASE-13328 URL: https://issues.apache.org/jira/browse/HBASE-13328 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 2.0.0, 1.1.0 Attachments: hbase-13328_v1.patch This seems to be a regression from HBASE-12783 discovered in testing Phoenix with 1.1.0-SNAPSHOT. Phoenix uses an HTable (with managed connection) to pass to doBulkLoad() which throws NeedUnmanagedConnectionException (see IndexToolIT.java and IndexTool.java in Phoenix). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13326) Disabled table can't be enabled after HBase is restarted
[ https://issues.apache.org/jira/browse/HBASE-13326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378693#comment-14378693 ] Andrey Stepachev commented on HBASE-13326: -- I'll bring fix shortly today. Disabled table can't be enabled after HBase is restarted Key: HBASE-13326 URL: https://issues.apache.org/jira/browse/HBASE-13326 Project: HBase Issue Type: Bug Affects Versions: 1.0.0, 1.1.0, 0.98.12 Reporter: Dima Spivak Assignee: Andrey Stepachev Priority: Blocker Fix For: 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13326-brach-1.0.patch, HBASE-13326-brach-1.patch The folks at Intel discovered a pretty nasty bug in 1.0 and 1.1 (but not master). Steps to reproduce: 1. Create a table, any table. 2. Disable the table. 3. Restart HBase. 4. Try enabling the table. The table won't become enabled and the master web UI will indicate a never-ending region in transition. Also worth noting is that [~mbertozzi] dug in and noted that this isn't happening in the master branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-13326) Disabled table can't be enabled after HBase is restarted
[ https://issues.apache.org/jira/browse/HBASE-13326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378691#comment-14378691 ] Andrew Purtell edited comment on HBASE-13326 at 3/24/15 9:37 PM: - Reason I ask is we don't have a .12 RC at the moment, and if this is a blocker for .13, it should go into .12 at this opportunity. was (Author: apurtell): Reason I ask is we don't have a .12 RC at the moment, and if this is a critical for .13, it should go into .12 at this opportunity. Disabled table can't be enabled after HBase is restarted Key: HBASE-13326 URL: https://issues.apache.org/jira/browse/HBASE-13326 Project: HBase Issue Type: Bug Affects Versions: 1.0.0, 1.1.0, 0.98.12 Reporter: Dima Spivak Assignee: Andrey Stepachev Priority: Blocker Fix For: 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13326-brach-1.0.patch, HBASE-13326-brach-1.patch The folks at Intel discovered a pretty nasty bug in 1.0 and 1.1 (but not master). Steps to reproduce: 1. Create a table, any table. 2. Disable the table. 3. Restart HBase. 4. Try enabling the table. The table won't become enabled and the master web UI will indicate a never-ending region in transition. Also worth noting is that [~mbertozzi] dug in and noted that this isn't happening in the master branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13273) Make Result.EMPTY_RESULT read-only; currently it can be modified
[ https://issues.apache.org/jira/browse/HBASE-13273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378779#comment-14378779 ] Hudson commented on HBASE-13273: SUCCESS: Integrated in HBase-TRUNK #6298 (See [https://builds.apache.org/job/HBase-TRUNK/6298/]) HBASE-13273 Make Result.EMPTY_RESULT read-only; currently it can be modified (busbey: rev 8cb4f89c013b254c6d3e9bee7b63137a391bcaba) * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestResult.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java Make Result.EMPTY_RESULT read-only; currently it can be modified Key: HBASE-13273 URL: https://issues.apache.org/jira/browse/HBASE-13273 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 1.0.0 Reporter: stack Assignee: Mikhail Antonov Labels: beginner Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13273-v5.patch, HBASE-13273-v5.patch, HBASE-13273.patch, HBASE-13273.patch, HBASE-13273.patch, HBASE-13273.patch Again from [~larsgeorge] Result result2 = Result.EMPTY_RESULT; System.out.println(result2); result2.copyFrom(result1); System.out.println(result2); What do you think happens when result1 has cells? Yep, you just modified the shared public EMPTY_RESULT to be not empty anymore. Fix. Result should be non-modifiable post-construction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13273) Make Result.EMPTY_RESULT read-only; currently it can be modified
[ https://issues.apache.org/jira/browse/HBASE-13273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378793#comment-14378793 ] Hudson commented on HBASE-13273: SUCCESS: Integrated in HBase-1.0 #824 (See [https://builds.apache.org/job/HBase-1.0/824/]) HBASE-13273 Make Result.EMPTY_RESULT read-only; currently it can be modified (busbey: rev 04f7122087d8814bcfb26098cd7e2016b8e3f0bf) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestResult.java Make Result.EMPTY_RESULT read-only; currently it can be modified Key: HBASE-13273 URL: https://issues.apache.org/jira/browse/HBASE-13273 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 1.0.0 Reporter: stack Assignee: Mikhail Antonov Labels: beginner Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13273-v5.patch, HBASE-13273-v5.patch, HBASE-13273.patch, HBASE-13273.patch, HBASE-13273.patch, HBASE-13273.patch Again from [~larsgeorge] Result result2 = Result.EMPTY_RESULT; System.out.println(result2); result2.copyFrom(result1); System.out.println(result2); What do you think happens when result1 has cells? Yep, you just modified the shared public EMPTY_RESULT to be not empty anymore. Fix. Result should be non-modifiable post-construction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13273) Make Result.EMPTY_RESULT read-only; currently it can be modified
[ https://issues.apache.org/jira/browse/HBASE-13273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378798#comment-14378798 ] Hudson commented on HBASE-13273: FAILURE: Integrated in HBase-1.1 #317 (See [https://builds.apache.org/job/HBase-1.1/317/]) HBASE-13273 Make Result.EMPTY_RESULT read-only; currently it can be modified (busbey: rev ac6fd86d266e06696e31282de0844b897ef73864) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestResult.java Make Result.EMPTY_RESULT read-only; currently it can be modified Key: HBASE-13273 URL: https://issues.apache.org/jira/browse/HBASE-13273 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 1.0.0 Reporter: stack Assignee: Mikhail Antonov Labels: beginner Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13273-v5.patch, HBASE-13273-v5.patch, HBASE-13273.patch, HBASE-13273.patch, HBASE-13273.patch, HBASE-13273.patch Again from [~larsgeorge] Result result2 = Result.EMPTY_RESULT; System.out.println(result2); result2.copyFrom(result1); System.out.println(result2); What do you think happens when result1 has cells? Yep, you just modified the shared public EMPTY_RESULT to be not empty anymore. Fix. Result should be non-modifiable post-construction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13328) LoadIncrementalHFile.doBulkLoad(Path,HTable) should handle managed connections
[ https://issues.apache.org/jira/browse/HBASE-13328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378663#comment-14378663 ] Mikhail Antonov commented on HBASE-13328: - [~enis] I assume it's related also to HBASE-13197, as managed connection would go away in 2.0? LoadIncrementalHFile.doBulkLoad(Path,HTable) should handle managed connections -- Key: HBASE-13328 URL: https://issues.apache.org/jira/browse/HBASE-13328 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 2.0.0, 1.1.0 Attachments: hbase-13328_v1.patch This seems to be a regression from HBASE-12783 discovered in testing Phoenix with 1.1.0-SNAPSHOT. Phoenix uses an HTable (with managed connection) to pass to doBulkLoad() which throws NeedUnmanagedConnectionException (see IndexToolIT.java and IndexTool.java in Phoenix). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11544) [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME
[ https://issues.apache.org/jira/browse/HBASE-11544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378696#comment-14378696 ] stack commented on HBASE-11544: --- bq. Can we amend this with backwards compatible changes regarding RegionScanner? Chatting we thought nextRaw would actually return ScannerContext -- the above suggestion is a little off here (NextState is actually purged). Please sketch a non-breaking change [~apurtell] and we'll take a looksee. The ScannerContext instance per rpc session will do what NextState did but [~jonathan.lawlor] will also use it in subsequent patches to carry timings for the heartbeating feature (he has a *plan*!) Adding ScannerContext as a param will be breaking change. You are suggesting somehow we'd have override to take ScannerContext? Can revert. Would like to see what non-breaking looks like first (and why wouldn't Phoenix want to make use of these amazing newness anyways? Ain't Region Interface coming in in 1.1.0 too so its going to have some work to do anyways? [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME -- Key: HBASE-11544 URL: https://issues.apache.org/jira/browse/HBASE-11544 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jonathan Lawlor Priority: Critical Fix For: 2.0.0, 1.1.0 Attachments: Allocation_Hot_Spots.html, HBASE-11544-branch_1_0-v1.patch, HBASE-11544-branch_1_0-v2.patch, HBASE-11544-v1.patch, HBASE-11544-v2.patch, HBASE-11544-v3.patch, HBASE-11544-v4.patch, HBASE-11544-v5.patch, HBASE-11544-v6.patch, HBASE-11544-v6.patch, HBASE-11544-v6.patch, HBASE-11544-v7.patch, HBASE-11544-v8-branch-1.patch, HBASE-11544-v8.patch, gc.j.png, h.png, hits.j.png, m.png, mean.png, net.j.png, q (2).png Running some tests, I set hbase.client.scanner.caching=1000. Dataset has large cells. I kept OOME'ing. Serverside, we should measure how much we've accumulated and return to the client whatever we've gathered once we pass out a certain size threshold rather than keep accumulating till we OOME. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13273) Make Result.EMPTY_RESULT read-only; currently it can be modified
[ https://issues.apache.org/jira/browse/HBASE-13273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378695#comment-14378695 ] Andrew Purtell commented on HBASE-13273: +1 for 0.98, any time, I'm holding the RC for HBASE-13326 looks like anyway. Make Result.EMPTY_RESULT read-only; currently it can be modified Key: HBASE-13273 URL: https://issues.apache.org/jira/browse/HBASE-13273 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 1.0.0 Reporter: stack Assignee: Mikhail Antonov Labels: beginner Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13273-v5.patch, HBASE-13273-v5.patch, HBASE-13273.patch, HBASE-13273.patch, HBASE-13273.patch, HBASE-13273.patch Again from [~larsgeorge] Result result2 = Result.EMPTY_RESULT; System.out.println(result2); result2.copyFrom(result1); System.out.println(result2); What do you think happens when result1 has cells? Yep, you just modified the shared public EMPTY_RESULT to be not empty anymore. Fix. Result should be non-modifiable post-construction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11544) [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME
[ https://issues.apache.org/jira/browse/HBASE-11544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378710#comment-14378710 ] Andrew Purtell commented on HBASE-11544: The idea would be to leave RegionScanner alone, introduce a new interface like ExtendedRegonScanner extending RegionScanner (but please... a better name, I know that sucks), put just the new nextRaw into the new interface, and do an instanceof check and cast if the check passes, fall back if it doesn't since we won't get ScannerContexts passed down through via implementors of plain RegionScanner. bq. Ain't Region Interface coming in in 1.1.0 too so its going to have some work to do anyways? Yep. I'm thinking about branch-1.0 and 0.98. Or they'll remain susceptible to OOME. That could be ok. They would still be susceptible if intermediaries like Phoenix wrap the extended RegionScanner with plain RegionScanners. OTOH, we'd be source and binary compatible such that this fix can be slotted in to patch releases. [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME -- Key: HBASE-11544 URL: https://issues.apache.org/jira/browse/HBASE-11544 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jonathan Lawlor Priority: Critical Fix For: 2.0.0, 1.1.0 Attachments: Allocation_Hot_Spots.html, HBASE-11544-branch_1_0-v1.patch, HBASE-11544-branch_1_0-v2.patch, HBASE-11544-v1.patch, HBASE-11544-v2.patch, HBASE-11544-v3.patch, HBASE-11544-v4.patch, HBASE-11544-v5.patch, HBASE-11544-v6.patch, HBASE-11544-v6.patch, HBASE-11544-v6.patch, HBASE-11544-v7.patch, HBASE-11544-v8-branch-1.patch, HBASE-11544-v8.patch, gc.j.png, h.png, hits.j.png, m.png, mean.png, net.j.png, q (2).png Running some tests, I set hbase.client.scanner.caching=1000. Dataset has large cells. I kept OOME'ing. Serverside, we should measure how much we've accumulated and return to the client whatever we've gathered once we pass out a certain size threshold rather than keep accumulating till we OOME. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13326) Disabled table can't be enabled after HBase is restarted
[ https://issues.apache.org/jira/browse/HBASE-13326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13326: - Attachment: HBASE-13326-branch-1.0.patch Disabled table can't be enabled after HBase is restarted Key: HBASE-13326 URL: https://issues.apache.org/jira/browse/HBASE-13326 Project: HBase Issue Type: Bug Affects Versions: 1.0.0, 1.1.0, 0.98.12 Reporter: Dima Spivak Assignee: Andrey Stepachev Priority: Blocker Fix For: 1.0.1, 1.1.0, 0.98.12 Attachments: HBASE-13326-0.98.patch, HBASE-13326-branch-1.0.patch, HBASE-13326-branch-1.0.patch, HBASE-13326-branch-1.patch The folks at Intel discovered a pretty nasty bug in 1.0 and 1.1 (but not master). Steps to reproduce: 1. Create a table, any table. 2. Disable the table. 3. Restart HBase. 4. Try enabling the table. The table won't become enabled and the master web UI will indicate a never-ending region in transition. Also worth noting is that [~mbertozzi] dug in and noted that this isn't happening in the master branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13326) Disabled table can't be enabled after HBase is restarted
[ https://issues.apache.org/jira/browse/HBASE-13326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13326: - Status: Open (was: Patch Available) Disabled table can't be enabled after HBase is restarted Key: HBASE-13326 URL: https://issues.apache.org/jira/browse/HBASE-13326 Project: HBase Issue Type: Bug Affects Versions: 0.98.12, 1.0.0, 1.1.0 Reporter: Dima Spivak Assignee: Andrey Stepachev Priority: Blocker Fix For: 1.0.1, 1.1.0, 0.98.12 Attachments: HBASE-13326-0.98.patch, HBASE-13326-branch-1.0.patch, HBASE-13326-branch-1.0.patch, HBASE-13326-branch-1.patch The folks at Intel discovered a pretty nasty bug in 1.0 and 1.1 (but not master). Steps to reproduce: 1. Create a table, any table. 2. Disable the table. 3. Restart HBase. 4. Try enabling the table. The table won't become enabled and the master web UI will indicate a never-ending region in transition. Also worth noting is that [~mbertozzi] dug in and noted that this isn't happening in the master branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13331) Exceptions from DFS client can cause CatalogJanitor to delete referenced files
[ https://issues.apache.org/jira/browse/HBASE-13331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-13331: -- Status: Open (was: Patch Available) Exceptions from DFS client can cause CatalogJanitor to delete referenced files -- Key: HBASE-13331 URL: https://issues.apache.org/jira/browse/HBASE-13331 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.98.12, 1.0.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Blocker Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13331.patch, HBASE-13331.patch CatalogJanitor#checkDaughterInFs assumes that there are no references whenever HRegionFileSystem.openRegionFromFileSystem throws IOException. Well Hadoop and HBase throw IOExceptions whenever someone looks in their general direction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12990) MetaScanner should be replaced by MetaTableAccessor
[ https://issues.apache.org/jira/browse/HBASE-12990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378040#comment-14378040 ] Andrey Stepachev commented on HBASE-12990: -- MetaTableAccessor wants unmanaged connection, but HBaseAdmin() creates managed. So we need to replace usage of it before we can move further. HBASE-13322 MetaScanner should be replaced by MetaTableAccessor --- Key: HBASE-12990 URL: https://issues.apache.org/jira/browse/HBASE-12990 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 2.0.0, 1.1.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12990-branch-1.v1.patch, HBASE-12990.patch, HBASE-12990.v2.patch, HBASE-12990.v3.patch, HBASE-12990.v4.patch, HBASE-12990.v5.patch, HBASE-12990.v5.patch, HBASE-12990.v5.patch, HBASE-12990.v6.patch, HBASE-12990.v7.patch, HBASE-12990.v7.patch, HBASE-12990.v7.patch, HBASE-12990.v8.patch MetaScanner and MetaTableAccessor do similar things, but seems they tend to diverge. Let's have only one thing to enquiry META. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12999) Make foreground_start return the correct exit code
[ https://issues.apache.org/jira/browse/HBASE-12999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378045#comment-14378045 ] Nick Dimiduk commented on HBASE-12999: -- Understood. Thanks. Make foreground_start return the correct exit code -- Key: HBASE-12999 URL: https://issues.apache.org/jira/browse/HBASE-12999 Project: HBase Issue Type: Bug Components: shell Affects Versions: 2.0.0, 1.1.0, 0.98.11 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.1.0, 0.98.11 Attachments: HBASE-12999-v1.patch, HBASE-12999.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13322) Replace explicit HBaseAdmin creation with connection#getAdmin()
[ https://issues.apache.org/jira/browse/HBASE-13322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13322: - Affects Version/s: (was: 1.1.0) Status: Patch Available (was: Open) Replace explicit HBaseAdmin creation with connection#getAdmin() --- Key: HBASE-13322 URL: https://issues.apache.org/jira/browse/HBASE-13322 Project: HBase Issue Type: Improvement Affects Versions: 2.0.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Priority: Minor Attachments: HBASE-13322.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13322) Replace explicit HBaseAdmin creation with connection#getAdmin()
[ https://issues.apache.org/jira/browse/HBASE-13322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13322: - Attachment: HBASE-13322.patch Replace explicit HBaseAdmin creation with connection#getAdmin() --- Key: HBASE-13322 URL: https://issues.apache.org/jira/browse/HBASE-13322 Project: HBase Issue Type: Improvement Affects Versions: 2.0.0, 1.1.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Priority: Minor Attachments: HBASE-13322.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13082) Coarsen StoreScanner locks to RegionScanner
[ https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378969#comment-14378969 ] Lars Hofhansl commented on HBASE-13082: --- Thanks [~stack]. That's with 1000 bytes values, right? The cost saving is per row, so the smaller the rows the higher the gain (percentage-wise) Coarsen StoreScanner locks to RegionScanner --- Key: HBASE-13082 URL: https://issues.apache.org/jira/browse/HBASE-13082 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Assignee: Lars Hofhansl Attachments: 13082-test.txt, 13082-v2.txt, 13082-v3.txt, 13082-v4.txt, 13082.txt, 13082.txt, gc.png, gc.png, gc.png, hits.png, next.png, next.png Continuing where HBASE-10015 left of. We can avoid locking (and memory fencing) inside StoreScanner by deferring to the lock already held by the RegionScanner. In tests this shows quite a scan improvement and reduced CPU (the fences make the cores wait for memory fetches). There are some drawbacks too: * All calls to RegionScanner need to be remain synchronized * Implementors of coprocessors need to be diligent in following the locking contract. For example Phoenix does not lock RegionScanner.nextRaw() and required in the documentation (not picking on Phoenix, this one is my fault as I told them it's OK) * possible starving of flushes and compaction with heavy read load. RegionScanner operations would keep getting the locks and the flushes/compactions would not be able finalize the set of files. I'll have a patch soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13331) Exceptions from DFS client can cause CatalogJanitor to delete referenced files
[ https://issues.apache.org/jira/browse/HBASE-13331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378968#comment-14378968 ] Enis Soztutar commented on HBASE-13331: --- +1. Exceptions from DFS client can cause CatalogJanitor to delete referenced files -- Key: HBASE-13331 URL: https://issues.apache.org/jira/browse/HBASE-13331 Project: HBase Issue Type: Bug Components: master Affects Versions: 1.0.0, 0.98.12 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Blocker Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.13 Attachments: HBASE-13331.patch, HBASE-13331.patch CatalogJanitor#checkDaughterInFs assumes that there are no references whenever HRegionFileSystem.openRegionFromFileSystem throws IOException. Well Hadoop and HBase throw IOExceptions whenever someone looks in their general direction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
