[jira] [Commented] (HBASE-14684) Try to remove all MiniMapReduceCluster in unit tests
[ https://issues.apache.org/jira/browse/HBASE-14684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067717#comment-15067717 ] Hudson commented on HBASE-14684: FAILURE: Integrated in HBase-1.2-IT #358 (See [https://builds.apache.org/job/HBase-1.2-IT/358/]) HBASE-14684 Try to remove all MiniMapReduceCluster in unit tests (chenheng: rev e20aafbaad3797c3d449c7929f0960b052e0f806) * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultithreadedTableMapper.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHashTable.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestCellCounter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTSVWithTTLs.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatTestBase.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestCopyTable.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTSVWithVisibilityLabels.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/MultiTableInputFormatTestBase.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestSyncTable.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestRowCounter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTimeRangeMapRed.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormat.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultiTableInputFormat.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScanBase.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALPlayer.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHRegionPartitioner.java * hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTSVWithOperationAttributes.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduceBase.java > Try to remove all MiniMapReduceCluster in unit tests > > > Key: HBASE-14684 > URL: https://issues.apache.org/jira/browse/HBASE-14684 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Heng Chen >Assignee: Heng Chen >Priority: Critical > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: 14684.branch-1.txt, 14684.branch-1.txt, > 14684.branch-1.txt, HBASE-14684-branch-1.2.patch, HBASE-14684-branch-1.patch, > HBASE-14684-branch-1.patch, HBASE-14684-branch-1.patch, > HBASE-14684-branch-1_v1.patch, HBASE-14684.patch, HBASE-14684_v1.patch > > > As discussion in dev list, we will try to do MR job without > MiniMapReduceCluster. > Testcases will run faster and more reliable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15026) The default value of "hbase.regions.slop" in docs is obsolete
[
https://issues.apache.org/jira/browse/HBASE-15026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067695#comment-15067695
]
Hadoop QA commented on HBASE-15026:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org
against master branch at commit 6e2c5d216eb1f4cacad7c5d7ed43b67785cabb67.
ATTACHMENT ID: http:
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation, build,
or dev-support patch that doesn't require tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16968//console
This message is automatically generated.
> The default value of "hbase.regions.slop" in docs is obsolete
> -
>
> Key: HBASE-15026
> URL: https://issues.apache.org/jira/browse/HBASE-15026
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Affects Versions: 1.1.2
>Reporter: Tianyin Xu
> Attachments: HBASE.15026.patch.001
>
>
> The default value of {{hbase.regions.slop}} is {{0.001}} in
> {{StochasticLoadBalancer}} (which is the default setting of
> {{hbase.master.loadbalancer.class}}).
> However, in the docs (both {{hbase-default.xml}} and [online
> docs|http://hbase.apache.org/book.html#config.files]), the default value is
> {{0.2}}. This value is specified in {{BaseLoadBalancer}} (which is an
> abstract class) and is inherited by {{SimpleLoadBalancer}}. However, as
> {{SimpleLoadBalancer}} is no longer used as the default load balancer, the
> doc is obsolete.
> The code structure is:
> {code:title=BaseLoadBalancer.java|borderStyle=solid}
> public abstract class BaseLoadBalancer implements LoadBalancer {
>...
>protected void setSlop(Configuration conf) {
> this.slop = conf.getFloat("hbase.regions.slop", (float) 0.2);
>}
> {code}
> {code:title=StochasticLoadBalancer.java|borderStyle=solid}
> public class StochasticLoadBalancer extends BaseLoadBalancer {
>...
>@Override
>protected void setSlop(Configuration conf) {
> this.slop = conf.getFloat("hbase.regions.slop", 0.001F);
>}
> {code}
> I suggest to make the manual entry of {{hbase.regions.slop}} specify the
> different default values in different balancer classes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14684) Try to remove all MiniMapReduceCluster in unit tests
[ https://issues.apache.org/jira/browse/HBASE-14684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067690#comment-15067690 ] Hudson commented on HBASE-14684: FAILURE: Integrated in HBase-1.3 #458 (See [https://builds.apache.org/job/HBase-1.3/458/]) HBASE-14684 Try to remove all MiniMapReduceCluster in unit tests (chenheng: rev b0965b1805f577b634052191d0c337d3cafabbca) * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatTestBase.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultithreadedTableMapper.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTSVWithOperationAttributes.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduceBase.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALPlayer.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTimeRangeMapRed.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultiTableInputFormat.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestRowCounter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestCellCounter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestCopyTable.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormat.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHRegionPartitioner.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTSVWithTTLs.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/MultiTableInputFormatTestBase.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestSyncTable.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHashTable.java > Try to remove all MiniMapReduceCluster in unit tests > > > Key: HBASE-14684 > URL: https://issues.apache.org/jira/browse/HBASE-14684 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Heng Chen >Assignee: Heng Chen >Priority: Critical > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: 14684.branch-1.txt, 14684.branch-1.txt, > 14684.branch-1.txt, HBASE-14684-branch-1.2.patch, HBASE-14684-branch-1.patch, > HBASE-14684-branch-1.patch, HBASE-14684-branch-1.patch, > HBASE-14684-branch-1_v1.patch, HBASE-14684.patch, HBASE-14684_v1.patch > > > As discussion in dev list, we will try to do MR job without > MiniMapReduceCluster. > Testcases will run faster and more reliable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14511) StoreFile.Writer Meta Plugin
[
https://issues.apache.org/jira/browse/HBASE-14511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067685#comment-15067685
]
Hadoop QA commented on HBASE-14511:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12778980/HBASE-14511-v4.patch
against master branch at commit f018c371b574a2d51930ba99f441bd227a77ec23.
ATTACHMENT ID: 12778980
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
new checkstyle errors. Check build console for list of new errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16965//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16965//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16965//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16965//console
This message is automatically generated.
> StoreFile.Writer Meta Plugin
>
>
> Key: HBASE-14511
> URL: https://issues.apache.org/jira/browse/HBASE-14511
> Project: HBase
> Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-14511-v3.patch, HBASE-14511-v4.patch,
> HBASE-14511.v1.patch, HBASE-14511.v2.patch
>
>
> During my work on a new compaction policies (HBASE-14468, HBASE-14477) I had
> to modify the existing code of a StoreFile.Writer to add additional meta-info
> required by these new policies. I think that it should be done by means of a
> new Plugin framework, because this seems to be a general capability/feature.
> As a future enhancement this can become a part of a more general
> StoreFileWriter/Reader plugin architecture. But I need only Meta section of a
> store file.
> This could be used, for example, to collect rowkeys distribution information
> during hfile creation. This info can be used later to find the optimal region
> split key or to create optimal set of sub-regions for M/R jobs or other jobs
> which can operate on a sub-region level.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14654) Reenable TestMultiParallel#testActiveThreadsCount
[ https://issues.apache.org/jira/browse/HBASE-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067676#comment-15067676 ] Hudson commented on HBASE-14654: SUCCESS: Integrated in HBase-Trunk_matrix #575 (See [https://builds.apache.org/job/HBase-Trunk_matrix/575/]) HBASE-14654 Reenable TestMultiParallel#testActiveThreadsCount (chenheng: rev f018c371b574a2d51930ba99f441bd227a77ec23) * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java > Reenable TestMultiParallel#testActiveThreadsCount > - > > Key: HBASE-14654 > URL: https://issues.apache.org/jira/browse/HBASE-14654 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0 >Reporter: Heng Chen >Assignee: Heng Chen > Labels: flakey > Fix For: 2.0.0 > > Attachments: HBASE-14654.patch, HBASE-14654.patch > > > It was disabled in HBASE-14642, this issue should reenable it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14684) Try to remove all MiniMapReduceCluster in unit tests
[ https://issues.apache.org/jira/browse/HBASE-14684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen updated HBASE-14684: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Try to remove all MiniMapReduceCluster in unit tests > > > Key: HBASE-14684 > URL: https://issues.apache.org/jira/browse/HBASE-14684 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Heng Chen >Assignee: Heng Chen >Priority: Critical > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: 14684.branch-1.txt, 14684.branch-1.txt, > 14684.branch-1.txt, HBASE-14684-branch-1.2.patch, HBASE-14684-branch-1.patch, > HBASE-14684-branch-1.patch, HBASE-14684-branch-1.patch, > HBASE-14684-branch-1_v1.patch, HBASE-14684.patch, HBASE-14684_v1.patch > > > As discussion in dev list, we will try to do MR job without > MiniMapReduceCluster. > Testcases will run faster and more reliable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14279) Race condition in ConcurrentIndex
[
https://issues.apache.org/jira/browse/HBASE-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067672#comment-15067672
]
Heng Chen commented on HBASE-14279:
---
{quote}
It is better to remove the sentence because of avoiding a license issue.
{quote}
OK, i will fix it when commit. Any suggestions? [~stack]
> Race condition in ConcurrentIndex
> -
>
> Key: HBASE-14279
> URL: https://issues.apache.org/jira/browse/HBASE-14279
> Project: HBase
> Issue Type: Bug
>Reporter: Hiroshi Ikeda
>Assignee: Heng Chen
>Priority: Minor
> Attachments: HBASE-14279.patch, HBASE-14279_v2.patch,
> HBASE-14279_v3.patch, HBASE-14279_v4.patch, HBASE-14279_v5.patch,
> HBASE-14279_v5.patch, HBASE-14279_v6.patch, LockStripedBag.java
>
>
> {{ConcurrentIndex.put}} and {{remove}} are in race condition. It is possible
> to remove a non-empty set, and to add a value to a removed set. Also
> {{ConcurrentIndex.values}} is vague in sense that the returned set sometimes
> trace the current state and sometimes doesn't.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15026) The default value of "hbase.regions.slop" in docs is obsolete
[
https://issues.apache.org/jira/browse/HBASE-15026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tianyin Xu updated HBASE-15026:
---
Summary: The default value of "hbase.regions.slop" in docs is obsolete
(was: The default value of "hbase.regions.slop" is obsolete)
> The default value of "hbase.regions.slop" in docs is obsolete
> -
>
> Key: HBASE-15026
> URL: https://issues.apache.org/jira/browse/HBASE-15026
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Affects Versions: 1.1.2
>Reporter: Tianyin Xu
> Attachments: HBASE.15026.patch.001
>
>
> The default value of {{hbase.regions.slop}} is {{0.001}} in
> {{StochasticLoadBalancer}} (which is the default setting of
> {{hbase.master.loadbalancer.class}}).
> However, in the docs (both {{hbase-default.xml}} and [online
> docs|http://hbase.apache.org/book.html#config.files]), the default value is
> {{0.2}}. This value is specified in {{BaseLoadBalancer}} (which is an
> abstract class) and is inherited by {{SimpleLoadBalancer}}. However, as
> {{SimpleLoadBalancer}} is no longer used as the default load balancer, the
> doc is obsolete.
> The code structure is:
> {code:title=BaseLoadBalancer.java|borderStyle=solid}
> public abstract class BaseLoadBalancer implements LoadBalancer {
>...
>protected void setSlop(Configuration conf) {
> this.slop = conf.getFloat("hbase.regions.slop", (float) 0.2);
>}
> {code}
> {code:title=StochasticLoadBalancer.java|borderStyle=solid}
> public class StochasticLoadBalancer extends BaseLoadBalancer {
>...
>@Override
>protected void setSlop(Configuration conf) {
> this.slop = conf.getFloat("hbase.regions.slop", 0.001F);
>}
> {code}
> I suggest to make the manual entry of {{hbase.regions.slop}} specify the
> different default values in different balancer classes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14684) Try to remove all MiniMapReduceCluster in unit tests
[ https://issues.apache.org/jira/browse/HBASE-14684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067670#comment-15067670 ] Heng Chen commented on HBASE-14684: --- push to branch-1.2 > Try to remove all MiniMapReduceCluster in unit tests > > > Key: HBASE-14684 > URL: https://issues.apache.org/jira/browse/HBASE-14684 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Heng Chen >Assignee: Heng Chen >Priority: Critical > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: 14684.branch-1.txt, 14684.branch-1.txt, > 14684.branch-1.txt, HBASE-14684-branch-1.2.patch, HBASE-14684-branch-1.patch, > HBASE-14684-branch-1.patch, HBASE-14684-branch-1.patch, > HBASE-14684-branch-1_v1.patch, HBASE-14684.patch, HBASE-14684_v1.patch > > > As discussion in dev list, we will try to do MR job without > MiniMapReduceCluster. > Testcases will run faster and more reliable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11677) Make Logger instance modifiers consistent
[
https://issues.apache.org/jira/browse/HBASE-11677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-11677:
Component/s: util
> Make Logger instance modifiers consistent
> -
>
> Key: HBASE-11677
> URL: https://issues.apache.org/jira/browse/HBASE-11677
> Project: HBase
> Issue Type: Task
> Components: util
>Reporter: Sean Busbey
>Assignee: Usha Kuchibhotla
>Priority: Minor
> Labels: beginner, sonar
> Fix For: 2.0.0, 1.2.0
>
> Attachments: HBASE-11677-v1.patch, HBASE-11677-v2.patch,
> HBASE-11677-v3.patch, HBASE-11677-v4.patch, HBASE-11677.patch,
> HBase-11677-v5.patch, HBase-11677-v6.patch, HBase-11677-v6.patch,
> HBase-11677-v7.patch
>
>
> We have some instances of Logger that are missing one of being private,
> static, and final.
> ex from HealthChecker.java, missing final
> {code}
> private static Log LOG = LogFactory.getLog(HealthChecker.class);
> {code}
> * Clean up where possible by making {{private static final}}
> * If we can't, add a non-javadoc note about why
> One way to look for problematic instances is to grep for initial assignment
> for the commonly used LOG member, e.g.
> * missing final: {{grep -r "LOG =" * | grep -v "final"}}
> * missing static: {{grep -r "LOG =" * | grep -v "static"}}
> * missing private: {{grep -r "LOG =" * | grep -v "private"}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13358) Upgrade VisibilityClient API to accept Connection object.
[
https://issues.apache.org/jira/browse/HBASE-13358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-13358:
Component/s: security
API
> Upgrade VisibilityClient API to accept Connection object.
> -
>
> Key: HBASE-13358
> URL: https://issues.apache.org/jira/browse/HBASE-13358
> Project: HBase
> Issue Type: Improvement
> Components: API, security
>Reporter: Srikanth Srungarapu
>Assignee: Matt Warhaftig
>Priority: Minor
> Fix For: 2.0.0, 1.2.0
>
> Attachments: 13358-addendum.txt, 13358-addendum.txt,
> HBASE-13358-1.2_v5.patch, HBASE-13358-master_v5.patch, HBASE-13358.patch,
> HBASE-13358_v2.patch, HBASE-13358_v3.patch, HBASE-13358_v4.patch
>
>
> From VisibilityClient class:
> {code}
> // TODO: Make it so caller passes in a Connection rather than have us do
> this expensive
> // setup each time. This class only used in test and shell at moment
> though.
> {code}
> Basically, this is the VC's counterpart of HBASE-13171.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14938) Limit to and fro requests size from ZK in bulk loaded hfile replication
[
https://issues.apache.org/jira/browse/HBASE-14938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067661#comment-15067661
]
Ashish Singhi commented on HBASE-14938:
---
I am planning to add a new configuration
{{hbase.replication.max.znodes.per.request}} which will control the number of
znodes we send for create/delete to ZK in a single request default being 5000.
For now we do not have any control on the number of znodes we can request from
ZK in one request. There is a open jira in ZK(ZOOKEEPER-2260) and work is in
progress, once that is complete we can control this part also, I will just add
a TODO for this.
Any concern with above approach or do you have any other thoughts ?
> Limit to and fro requests size from ZK in bulk loaded hfile replication
> ---
>
> Key: HBASE-14938
> URL: https://issues.apache.org/jira/browse/HBASE-14938
> Project: HBase
> Issue Type: Improvement
>Reporter: Ashish Singhi
>Assignee: Ashish Singhi
>
> In ZK the maximum allowable size of the data array is 1 MB. Until we have
> fixed HBASE-10295 we need to handle this.
> Approach to this problem will be discussed in the comments section.
> Note: We have done internal testing with more than 3k nodes in ZK yet to be
> replicated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13158) When client supports CellBlock, return the result Cells as controller payload for get(Get) API also
[ https://issues.apache.org/jira/browse/HBASE-13158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067664#comment-15067664 ] Hudson commented on HBASE-13158: FAILURE: Integrated in HBase-1.3-IT #394 (See [https://builds.apache.org/job/HBase-1.3-IT/394/]) HBASE-13158 When client supports CellBlock, return the result Cells as (anoopsamjohn: rev b7100c934b8ca8e77208949f92a7d173c9e4c558) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java > When client supports CellBlock, return the result Cells as controller payload > for get(Get) API also > --- > > Key: HBASE-13158 > URL: https://issues.apache.org/jira/browse/HBASE-13158 > Project: HBase > Issue Type: Improvement >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0, 1.3.0 > > Attachments: 13158v4.suggestion.txt, HBASE-13158.patch, > HBASE-13158_V2.patch, HBASE-13158_V3.patch, HBASE-13158_V4.patch, > HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V6.patch, > HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V7.patch, > HBASE-13158_V8.patch, HBASE-13158_addendum_branch-1.patch, > HBASE-13158_branch-1.patch, HBASE-13158_branch-1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14684) Try to remove all MiniMapReduceCluster in unit tests
[ https://issues.apache.org/jira/browse/HBASE-14684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067665#comment-15067665 ] Hudson commented on HBASE-14684: FAILURE: Integrated in HBase-1.3-IT #394 (See [https://builds.apache.org/job/HBase-1.3-IT/394/]) HBASE-14684 Try to remove all MiniMapReduceCluster in unit tests (chenheng: rev b0965b1805f577b634052191d0c337d3cafabbca) * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/MultiTableInputFormatTestBase.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatTestBase.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALPlayer.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestCopyTable.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHRegionPartitioner.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormat.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestCellCounter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestRowCounter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTimeRangeMapRed.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHashTable.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTSVWithTTLs.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultiTableInputFormat.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultithreadedTableMapper.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduceBase.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestSyncTable.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTSVWithOperationAttributes.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java > Try to remove all MiniMapReduceCluster in unit tests > > > Key: HBASE-14684 > URL: https://issues.apache.org/jira/browse/HBASE-14684 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Heng Chen >Assignee: Heng Chen >Priority: Critical > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: 14684.branch-1.txt, 14684.branch-1.txt, > 14684.branch-1.txt, HBASE-14684-branch-1.2.patch, HBASE-14684-branch-1.patch, > HBASE-14684-branch-1.patch, HBASE-14684-branch-1.patch, > HBASE-14684-branch-1_v1.patch, HBASE-14684.patch, HBASE-14684_v1.patch > > > As discussion in dev list, we will try to do MR job without > MiniMapReduceCluster. > Testcases will run faster and more reliable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13534) Change HBase master WebUI to explicitly mention if it is a backup master
[ https://issues.apache.org/jira/browse/HBASE-13534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-13534: Component/s: UI master > Change HBase master WebUI to explicitly mention if it is a backup master > > > Key: HBASE-13534 > URL: https://issues.apache.org/jira/browse/HBASE-13534 > Project: HBase > Issue Type: Improvement > Components: master, UI >Reporter: Appy >Assignee: Appy >Priority: Minor > Fix For: 2.0.0, 0.98.13, 1.2.0 > > Attachments: HBASE-13534.patch, after.png, before.png > > > In master-status of both active and backup masters, page title and heading > says 'Master' which is sometimes misleading. > This screenshot is of backup master. > !before.png! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14951) Make hbase.regionserver.maxlogs obsolete
[ https://issues.apache.org/jira/browse/HBASE-14951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14951: Component/s: wal Performance > Make hbase.regionserver.maxlogs obsolete > > > Key: HBASE-14951 > URL: https://issues.apache.org/jira/browse/HBASE-14951 > Project: HBase > Issue Type: Improvement > Components: Performance, wal >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14951-v1.patch, HBASE-14951-v2.patch > > > There was a discussion in HBASE-14388 related to maximum number of log files. > It was an agreement that we should calculate this number in a code but still > need to honor user's setting. > Maximum number of log files now is calculated as following: > maxLogs = HEAP_SIZE * memstoreRatio * 2/ LogRollSize -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15014) Fix filterCellByStore in WALsplitter is awful for performance
[
https://issues.apache.org/jira/browse/HBASE-15014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Elliott Clark updated HBASE-15014:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Fix filterCellByStore in WALsplitter is awful for performance
> -
>
> Key: HBASE-15014
> URL: https://issues.apache.org/jira/browse/HBASE-15014
> Project: HBase
> Issue Type: Bug
> Components: MTTR, Recovery, wal
>Affects Versions: 1.2.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-15014-v1.patch, HBASE-15014-v2.patch,
> HBASE-15014-v3.patch, HBASE-15014.patch
>
>
> Testing the latest 1.2 I see this when there is a regionserver that crashes.
> {code}
> Thread 921 (RS_LOG_REPLAY_OPS-hbase2698:16020-0-Writer-1):
> State: RUNNABLE
> Blocked count: 6354
> Waited count: 6249
> Stack:
> org.apache.hadoop.hbase.KeyValue.equals(KeyValue.java:1128)
> java.util.ArrayList.indexOf(ArrayList.java:317)
> java.util.ArrayList.contains(ArrayList.java:300)
> java.util.ArrayList.batchRemove(ArrayList.java:720)
> java.util.ArrayList.removeAll(ArrayList.java:690)
>
> org.apache.hadoop.hbase.wal.WALSplitter$LogRecoveredEditsOutputSink.filterCellByStore(WALSplitter.java:1529)
>
> org.apache.hadoop.hbase.wal.WALSplitter$LogRecoveredEditsOutputSink.append(WALSplitter.java:1557)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.writeBuffer(WALSplitter.java:1113)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.doRun(WALSplitter.java:1105)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.run(WALSplitter.java:1075)
> Thread 920 (RS_LOG_REPLAY_OPS-hbase2698:16020-0-Writer-0):
> State: TIMED_WAITING
> Blocked count: 17560
> Waited count: 19695
> Stack:
> java.lang.Object.wait(Native Method)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.doRun(WALSplitter.java:1093)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.run(WALSplitter.java:1075)
> Thread 919 (RS_LOG_REPLAY_OPS-hbase2698:16020-0):
> State: TIMED_WAITING
> Blocked count: 115
> Waited count: 976
> Stack:
> java.lang.Object.wait(Native Method)
>
> org.apache.hadoop.hbase.wal.WALSplitter$EntryBuffers.appendEntry(WALSplitter.java:944)
> org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:365)
> org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:236)
>
> org.apache.hadoop.hbase.regionserver.SplitLogWorker$1.exec(SplitLogWorker.java:104)
>
> org.apache.hadoop.hbase.regionserver.handler.WALSplitterHandler.process(WALSplitterHandler.java:72)
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> {code}
> This has been going on for >10 mins.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14999) Remove ref to org.mortbay.log.Log
[ https://issues.apache.org/jira/browse/HBASE-14999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14999: Component/s: dependencies > Remove ref to org.mortbay.log.Log > - > > Key: HBASE-14999 > URL: https://issues.apache.org/jira/browse/HBASE-14999 > Project: HBase > Issue Type: Bug > Components: dependencies >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14999.patch, HBASE-14999_branch-1.patch > > > I could see some 2 or 3 src files and many test files referring to > org.mortbay.log.Log instead of commons Log. Patch correct all such places -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14989) Implementation of Mutation.getWriteToWAL() is backwards
[
https://issues.apache.org/jira/browse/HBASE-14989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-14989:
Component/s: Client
> Implementation of Mutation.getWriteToWAL() is backwards
> ---
>
> Key: HBASE-14989
> URL: https://issues.apache.org/jira/browse/HBASE-14989
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: James Taylor
>Assignee: Enis Soztutar
> Fix For: 1.2.0, 1.3.0, 1.1.4, 0.98.17, 1.0.4
>
> Attachments: HBASE-14989.1.branch-1.patch,
> hbase-14989-branch-1_v1.patch
>
>
> The implementation of the deprecated getWriteToWAL is backwards. It should
> return true if this.durability == Durability.SYNC_WAL:
> {code}
> /**
>* @deprecated Use {@link #getDurability()} instead.
>* @return true if edits should be applied to WAL, false if not
>*/
> @Deprecated
> public boolean getWriteToWAL() {
> return this.durability == Durability.SKIP_WAL;
> }
> {code}
> For example, if mutation.durability is Durability.SYNC_WAL and the following
> code is called {{clonedMutation.setWriteToWAL(mutation.getWriteToWAL())}}, it
> will disable writing to the WAL for clonedMutation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15026) The default value of "hbase.regions.slop" is obsolete
[
https://issues.apache.org/jira/browse/HBASE-15026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tianyin Xu updated HBASE-15026:
---
Status: Patch Available (was: Open)
> The default value of "hbase.regions.slop" is obsolete
> -
>
> Key: HBASE-15026
> URL: https://issues.apache.org/jira/browse/HBASE-15026
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Affects Versions: 1.1.2
>Reporter: Tianyin Xu
> Attachments: HBASE.15026.patch.001
>
>
> The default value of {{hbase.regions.slop}} is {{0.001}} in
> {{StochasticLoadBalancer}} (which is the default setting of
> {{hbase.master.loadbalancer.class}}).
> However, in the docs (both {{hbase-default.xml}} and [online
> docs|http://hbase.apache.org/book.html#config.files]), the default value is
> {{0.2}}. This value is specified in {{BaseLoadBalancer}} (which is an
> abstract class) and is inherited by {{SimpleLoadBalancer}}. However, as
> {{SimpleLoadBalancer}} is no longer used as the default load balancer, the
> doc is obsolete.
> The code structure is:
> {code:title=BaseLoadBalancer.java|borderStyle=solid}
> public abstract class BaseLoadBalancer implements LoadBalancer {
>...
>protected void setSlop(Configuration conf) {
> this.slop = conf.getFloat("hbase.regions.slop", (float) 0.2);
>}
> {code}
> {code:title=StochasticLoadBalancer.java|borderStyle=solid}
> public class StochasticLoadBalancer extends BaseLoadBalancer {
>...
>@Override
>protected void setSlop(Configuration conf) {
> this.slop = conf.getFloat("hbase.regions.slop", 0.001F);
>}
> {code}
> I suggest to make the manual entry of {{hbase.regions.slop}} specify the
> different default values in different balancer classes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14968) ConcurrentModificationException in region close resulting in the region staying in closing state
[
https://issues.apache.org/jira/browse/HBASE-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-14968:
Component/s: regionserver
Region Assignment
> ConcurrentModificationException in region close resulting in the region
> staying in closing state
>
>
> Key: HBASE-14968
> URL: https://issues.apache.org/jira/browse/HBASE-14968
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment, regionserver
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4, 0.98.17, 1.0.4
>
> Attachments: hbase-14968_v1.patch
>
>
> We have seen this in our tests. The region gets closed, but the region close
> handler gets an unexpected exception causing the region to stay in closing
> state until RS is restarted.
> The Phoenix and security coprocessors were loaded in that region. Here is the
> stack trace:
> {code}
> java.util.ConcurrentModificationException
> at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
> at java.util.ArrayList$Itr.next(ArrayList.java:851)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.shutdown(CoprocessorHost.java:442)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionEnvironment.shutdown(RegionCoprocessorHost.java:155)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.shutdown(CoprocessorHost.java:272)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$5.postEnvCall(RegionCoprocessorHost.java:496)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1761)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postClose(RegionCoprocessorHost.java:489)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1502)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1349)
> at
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:138)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14979) Update to the newest Zookeeper release
[ https://issues.apache.org/jira/browse/HBASE-14979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067653#comment-15067653 ] Elliott Clark commented on HBASE-14979: --- Punt to 1.3 > Update to the newest Zookeeper release > -- > > Key: HBASE-14979 > URL: https://issues.apache.org/jira/browse/HBASE-14979 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.2.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14979.patch > > > ZOOKEEPER-706 is nice to have for anyone running replication that sometimes > gets stalled. We should update to the latest patch version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14979) Update to the newest Zookeeper release
[ https://issues.apache.org/jira/browse/HBASE-14979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067642#comment-15067642 ] Sean Busbey commented on HBASE-14979: - do we expect some version of this is going to be ready in the next couple of days, or should I punt it to 1.3? > Update to the newest Zookeeper release > -- > > Key: HBASE-14979 > URL: https://issues.apache.org/jira/browse/HBASE-14979 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.2.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14979.patch > > > ZOOKEEPER-706 is nice to have for anyone running replication that sometimes > gets stalled. We should update to the latest patch version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13270) Setter for Result#getStats is #addResults; confusing!
[
https://issues.apache.org/jira/browse/HBASE-13270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-13270:
-
Component/s: Client
> Setter for Result#getStats is #addResults; confusing!
> -
>
> Key: HBASE-13270
> URL: https://issues.apache.org/jira/browse/HBASE-13270
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Reporter: stack
>Assignee: Mikhail Antonov
> Labels: beginner
> Fix For: 2.0.0, 1.1.0
>
> Attachments: HBASE-13270-v2.patch, HBASE-13270.patch
>
>
> Below is our [~larsgeorge] on a finding he made reviewing our API:
> "Result class having getStats() and addResults(Stats) makes little sense..."
> "...the naming is just weird. You have a getStats() getter and an
> addResults(Stats) setter???"
> "...Especially in the Result class and addResult() is plain misleading..."
> This issue is about deprecating addResults and replacing it with addStats in
> its place.
> The getStats/addResult is recent. It came in with:
> {code}
> commit a411227b0ebf78b4ee8ae7179e162b54734e77de
> Author: Jesse Yates
> Date: Tue Oct 28 16:14:16 2014 -0700
> HBASE-5162 Basic client pushback mechanism
> ...
> {code}
> RegionLoadStats don't belong in Result if you ask me but better in the
> enveloping on invocations... but that is another issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-15001) Thread Safety issues in ReplicationSinkManager and HBaseInterClusterReplicationEndpoint
[
https://issues.apache.org/jira/browse/HBASE-15001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey resolved HBASE-15001.
-
Resolution: Fixed
> Thread Safety issues in ReplicationSinkManager and
> HBaseInterClusterReplicationEndpoint
> ---
>
> Key: HBASE-15001
> URL: https://issues.apache.org/jira/browse/HBASE-15001
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 2.0.0, 1.2.0, 1.3.0, 1.2.1
>Reporter: Ashu Pachauri
>Assignee: Ashu Pachauri
>Priority: Blocker
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-15001-V0.patch, Test.java,
> repro_stuck_replication.diff
>
>
> ReplicationSinkManager is not thread-safe. This can cause problems in
> HBaseInterClusterReplicationEndpoint, when the walprovider is multiwal.
> For example:
> 1. When multiple threads report bad sinks, the sink list can be non-empty but
> report a negative size because the ArrayList itself is not thread-safe.
> 2. HBaseInterClusterReplicationEndpoint depends on the number of sinks to
> batch edits for shipping. However, it's quite possible that the following
> code makes it assume that there are no batches to process (sink size is
> non-zero, but by the time we reach the "batching" part, sink size becomes
> zero.)
> {code}
> if (replicationSinkMgr.getSinks().size() == 0) {
> return false;
> }
> ...
> int n = Math.min(Math.min(this.maxThreads, entries.size()/100+1),
>replicationSinkMgr.getSinks().size());
> {code}
> [Update] This leads to ArithmeticException: division by zero at:
> {code}
> entryLists.get(Math.abs(Bytes.hashCode(e.getKey().getEncodedRegionName())%n)).add(e);
> {code}
> which is benign and will just lead to retries by the ReplicationSource.
> The idea is to make all operations in ReplicationSinkManager thread-safe and
> do a verification on the size of replicated edits before we report success.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14684) Try to remove all MiniMapReduceCluster in unit tests
[
https://issues.apache.org/jira/browse/HBASE-14684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067625#comment-15067625
]
Hadoop QA commented on HBASE-14684:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12778979/HBASE-14684-branch-1.2.patch
against branch-1.2 branch at commit f018c371b574a2d51930ba99f441bd227a77ec23.
ATTACHMENT ID: 12778979
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 89 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16966//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16966//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16966//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16966//console
This message is automatically generated.
> Try to remove all MiniMapReduceCluster in unit tests
>
>
> Key: HBASE-14684
> URL: https://issues.apache.org/jira/browse/HBASE-14684
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Heng Chen
>Assignee: Heng Chen
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14684.branch-1.txt, 14684.branch-1.txt,
> 14684.branch-1.txt, HBASE-14684-branch-1.2.patch, HBASE-14684-branch-1.patch,
> HBASE-14684-branch-1.patch, HBASE-14684-branch-1.patch,
> HBASE-14684-branch-1_v1.patch, HBASE-14684.patch, HBASE-14684_v1.patch
>
>
> As discussion in dev list, we will try to do MR job without
> MiniMapReduceCluster.
> Testcases will run faster and more reliable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15026) The default value of "hbase.regions.slop" is obsolete
[
https://issues.apache.org/jira/browse/HBASE-15026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tianyin Xu updated HBASE-15026:
---
Attachment: HBASE.15026.patch.001
> The default value of "hbase.regions.slop" is obsolete
> -
>
> Key: HBASE-15026
> URL: https://issues.apache.org/jira/browse/HBASE-15026
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Affects Versions: 1.1.2
>Reporter: Tianyin Xu
> Attachments: HBASE.15026.patch.001
>
>
> The default value of {{hbase.regions.slop}} is {{0.001}} in
> {{StochasticLoadBalancer}} (which is the default setting of
> {{hbase.master.loadbalancer.class}}).
> However, in the docs (both {{hbase-default.xml}} and [online
> docs|http://hbase.apache.org/book.html#config.files]), the default value is
> {{0.2}}. This value is specified in {{BaseLoadBalancer}} (which is an
> abstract class) and is inherited by {{SimpleLoadBalancer}}. However, as
> {{SimpleLoadBalancer}} is no longer used as the default load balancer, the
> doc is obsolete.
> The code structure is:
> {code:title=BaseLoadBalancer.java|borderStyle=solid}
> public abstract class BaseLoadBalancer implements LoadBalancer {
>...
>protected void setSlop(Configuration conf) {
> this.slop = conf.getFloat("hbase.regions.slop", (float) 0.2);
>}
> {code}
> {code:title=StochasticLoadBalancer.java|borderStyle=solid}
> public class StochasticLoadBalancer extends BaseLoadBalancer {
>...
>@Override
>protected void setSlop(Configuration conf) {
> this.slop = conf.getFloat("hbase.regions.slop", 0.001F);
>}
> {code}
> I suggest to make the manual entry of {{hbase.regions.slop}} specify the
> different default values in different balancer classes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15018) Inconsistent way of handling TimeoutException in the rpc client implemenations
[
https://issues.apache.org/jira/browse/HBASE-15018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067622#comment-15067622
]
ramkrishna.s.vasudevan commented on HBASE-15018:
+1 on patch.
> Inconsistent way of handling TimeoutException in the rpc client implemenations
> --
>
> Key: HBASE-15018
> URL: https://issues.apache.org/jira/browse/HBASE-15018
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.1.0, 1.2.0
>Reporter: Ashish Singhi
>Assignee: Ashish Singhi
> Fix For: 2.0.0, 1.2.1, 1.1.3
>
> Attachments: HBASE-15018.patch
>
>
> If there is any rpc timeout when using RpcClientImpl then we wrap the
> exception in IOE and throw it,
> {noformat}
> 2015-11-16 04:05:24,935 WARN [main-EventThread.replicationSource,peer2]
> regionserver.HBaseInterClusterReplicationEndpoint: Can't replicate because of
> a local or network error:
> java.io.IOException: Call to host-XX:16040 failed on local exception:
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=510,
> waitTime=180001, operationTimeout=18 expired.
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1271)
> at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1239)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
> at
> org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.replicateWALEntry(AdminProtos.java:25690)
> at
> org.apache.hadoop.hbase.protobuf.ReplicationProtbufUtil.replicateWALEntry(ReplicationProtbufUtil.java:77)
> at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint$Replicator.call(HBaseInterClusterReplicationEndpoint.java:322)
> at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint$Replicator.call(HBaseInterClusterReplicationEndpoint.java:308)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=510,
> waitTime=180001, operationTimeout=18 expired.
> at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:70)
> at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1213)
> ... 10 more
> {noformat}
> But that isn't case with AsyncRpcClient, we don't wrap and throw
> CallTimeoutException as it is.
> {noformat}
> 2015-12-21 14:27:33,093 WARN
> [RS_OPEN_REGION-host-XX:16201-0.replicationSource.host-XX%2C16201%2C1450687255593,1]
> regionserver.HBaseInterClusterReplicationEndpoint: Can't replicate because
> of a local or network error:
> org.apache.hadoop.hbase.ipc.CallTimeoutException: callId=2,
> method=ReplicateWALEntry, rpcTimeout=60, param {TODO: class
> org.apache.hadoop.hbase.protobuf.generated.AdminProtos$ReplicateWALEntryRequest}
> at
> org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:257)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:295)
> at
> org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.replicateWALEntry(AdminProtos.java:23707)
> at
> org.apache.hadoop.hbase.protobuf.ReplicationProtbufUtil.replicateWALEntry(ReplicationProtbufUtil.java:73)
> at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint$Replicator.call(HBaseInterClusterReplicationEndpoint.java:387)
> at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint$Replicator.call(HBaseInterClusterReplicationEndpoint.java:370)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> I think we should have same behavior across both the implementations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13275) Setting hbase.security.authorization to false does not disable authorization
[
https://issues.apache.org/jira/browse/HBASE-13275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-13275:
-
Component/s: security
> Setting hbase.security.authorization to false does not disable authorization
>
>
> Key: HBASE-13275
> URL: https://issues.apache.org/jira/browse/HBASE-13275
> Project: HBase
> Issue Type: Bug
> Components: security
>Reporter: William Watson
>Assignee: Andrew Purtell
> Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.12
>
> Attachments: HBASE-13275-0.98.patch, HBASE-13275-0.98.patch,
> HBASE-13275-branch-1.patch, HBASE-13275-branch-1.patch, HBASE-13275.patch,
> HBASE-13275.patch, HBASE-13275.patch, HBASE-13275.patch
>
>
> According to the docs provided by Cloudera (we're not running Cloudera, BTW),
> this is the list of configs to enable authorization in HBase:
> {code}
>
> hbase.security.authorization
> true
>
>
> hbase.coprocessor.master.classes
> org.apache.hadoop.hbase.security.access.AccessController
>
>
> hbase.coprocessor.region.classes
>
> org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.AccessController
>
> {code}
> We wanted to then disable authorization but simply setting
> hbase.security.authorization to false did not disable the authorization
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13308) Fix flaky TestEndToEndSplitTransaction
[
https://issues.apache.org/jira/browse/HBASE-13308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-13308:
-
Component/s: flakey
> Fix flaky TestEndToEndSplitTransaction
> --
>
> Key: HBASE-13308
> URL: https://issues.apache.org/jira/browse/HBASE-13308
> Project: HBase
> Issue Type: Bug
> Components: flakey, test
>Affects Versions: 2.0.0, 1.1.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0, 1.1.0
>
> Attachments: HBASE-13308.patch
>
>
> https://builds.apache.org/job/HBase-TRUNK-jacoco/24/testReport/junit/org.apache.hadoop.hbase.regionserver/TestEndToEndSplitTransaction/testFromClientSideWhileSplitting/
> First, we split 'e9eb97847340ea7c6b9616d63d62a784' to
> 'abe1973ea732066b12d8e33fce12a951' and '4940dad7ef9b4b699fd13eede5740d9d'.
> And then, we try to split 'abe1973ea732066b12d8e33fce12a951'.
> {noformat}
> 2015-03-21 03:58:46,970 INFO [Thread-191]
> regionserver.TestEndToEndSplitTransaction(399): Initiating region split
> for:testFromClientSideWhileSplitting,,1426910324847.abe1973ea732066b12d8e33fce12a951.
> 2015-03-21 03:58:46,976 INFO
> [PriorityRpcServer.handler=7,queue=1,port=54177]
> regionserver.RSRpcServices(1596): Splitting
> testFromClientSideWhileSplitting,,1426910324847.abe1973ea732066b12d8e33fce12a951.
> 2015-03-21 03:58:46,977 DEBUG
> [PriorityRpcServer.handler=7,queue=1,port=54177]
> regionserver.CompactSplitThread(259): Split requested for
> testFromClientSideWhileSplitting,,1426910324847.abe1973ea732066b12d8e33fce12a951..
> compaction_queue=(0:0), split_queue=1, merge_queue=0
> 2015-03-21 03:58:46,978 INFO [Thread-191]
> regionserver.TestEndToEndSplitTransaction(399): blocking until region is
> split:testFromClientSideWhileSplitting,,1426910324847.abe1973ea732066b12d8e33fce12a951.
> 2015-03-21 03:58:46,985 DEBUG [RS:0;priapus:54177-splits-1426910324832]
> lock.ZKInterProcessLockBase(226): Acquired a lock for
> /hbase/table-lock/testFromClientSideWhileSplitting/read-regionserver:5417702
> 2015-03-21 03:58:46,988 DEBUG [RS:0;priapus:54177-splits-1426910324832]
> lock.ZKInterProcessLockBase(328): Released
> /hbase/table-lock/testFromClientSideWhileSplitting/read-regionserver:5417702
> 2015-03-21 03:58:46,988 DEBUG [Thread-191] ipc.AsyncRpcClient(163): Use
> global event loop group NioEventLoopGroup
> 2015-03-21 03:58:46,988 INFO [RS:0;priapus:54177-splits-1426910324832]
> regionserver.SplitRequest(142): Split transaction journal:
> STARTED at 1426910326977
> {noformat}
> We can see that it failed without any error message.
> I think can only happen when the parent is not splittable or we can not find
> a splitrow.
> {noformat}
> 2015-03-21 03:58:47,019 INFO
> [RS:0;priapus:54177-shortCompactions-1426910324308]
> regionserver.HStore(1334): Completed major compaction of 2 (all) file(s) in
> family of
> testFromClientSideWhileSplitting,,1426910324847.abe1973ea732066b12d8e33fce12a951.
> into e97bccdc4b014c15a52a579cd49ebb31(size=12.6 K), total size for store is
> 12.6 K. This selection was in queue for 0sec, and took 0sec to execute.
> 2015-03-21 03:58:47,019 INFO
> [RS:0;priapus:54177-shortCompactions-1426910324308]
> regionserver.CompactSplitThread$CompactionRunner(523): Completed compaction:
> Request =
> regionName=testFromClientSideWhileSplitting,,1426910324847.abe1973ea732066b12d8e33fce12a951.,
> storeName=family, fileCount=2, fileSize=25.5 K, priority=1,
> time=14542808784655186; duration=0sec
> 2015-03-21 03:58:47,020 DEBUG
> [RS:0;priapus:54177-shortCompactions-1426910324308]
> regionserver.CompactSplitThread$CompactionRunner(546): CompactSplitThread
> Status: compaction_queue=(0:0), split_queue=0, merge_queue=0
> {noformat}
> We can see that, the compaction was completed at 03:58:47,019, but split was
> started at 03:58:46,970 which is earlier.
> So we have a reference file and is not splittable.
> I think the problem is 'compactAndBlockUntilDone' is not reliable, it may
> return before the compaction complete.
> Will try to prepare a patch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15026) The default value of "hbase.regions.slop" is obsolete
Tianyin Xu created HBASE-15026:
--
Summary: The default value of "hbase.regions.slop" is obsolete
Key: HBASE-15026
URL: https://issues.apache.org/jira/browse/HBASE-15026
Project: HBase
Issue Type: Bug
Components: Balancer
Affects Versions: 1.1.2
Reporter: Tianyin Xu
The default value of {{hbase.regions.slop}} is {{0.001}} in
{{StochasticLoadBalancer}} (which is the default setting of
{{hbase.master.loadbalancer.class}}).
However, in the docs (both {{hbase-default.xml}} and [online
docs|http://hbase.apache.org/book.html#config.files]), the default value is
{{0.2}}. This value is specified in {{BaseLoadBalancer}} (which is an abstract
class) and is inherited by {{SimpleLoadBalancer}}. However, as
{{SimpleLoadBalancer}} is no longer used as the default load balancer, the doc
is obsolete.
The code structure is:
{code:title=BaseLoadBalancer.java|borderStyle=solid}
public abstract class BaseLoadBalancer implements LoadBalancer {
...
protected void setSlop(Configuration conf) {
this.slop = conf.getFloat("hbase.regions.slop", (float) 0.2);
}
{code}
{code:title=StochasticLoadBalancer.java|borderStyle=solid}
public class StochasticLoadBalancer extends BaseLoadBalancer {
...
@Override
protected void setSlop(Configuration conf) {
this.slop = conf.getFloat("hbase.regions.slop", 0.001F);
}
{code}
I suggest to make the manual entry of {{hbase.regions.slop}} specify the
different default values in different balancer classes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13340) Include LimitedPrivate interfaces in the API compatibility report
[ https://issues.apache.org/jira/browse/HBASE-13340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-13340: - Component/s: API > Include LimitedPrivate interfaces in the API compatibility report > - > > Key: HBASE-13340 > URL: https://issues.apache.org/jira/browse/HBASE-13340 > Project: HBase > Issue Type: Improvement > Components: API >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-13340.patch > > > The API compatibility checker script added in HBASE-12808 passes a file > containing annotations to the JavaACC tool. When JavaACC is invoked with that > option it will filter out all interfaces that do not have that annotation. > Currently only Public interfaces are checked. We should add LimitedPrivate to > the annotation list, otherwise we will miss changes that impact coprocessors > and other users of those interfaces. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13393) Optimize memstore flushing to avoid writing tag information to hfiles when no tags are present.
[ https://issues.apache.org/jira/browse/HBASE-13393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-13393: - Component/s: Performance HFile > Optimize memstore flushing to avoid writing tag information to hfiles when no > tags are present. > --- > > Key: HBASE-13393 > URL: https://issues.apache.org/jira/browse/HBASE-13393 > Project: HBase > Issue Type: Sub-task > Components: HFile, Performance >Reporter: stack >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 1.2.0 > > Attachments: HBASE-13393.patch, HBASE-13393_1.patch, > HBASE-13393_2.patch, HBASE-13393_3.patch, HBASE-13393_addendum_trunk.patch, > HBase-13393_branch-1.patch > > > We used to specify 'no tags' by choosing to write files that were version 2. > After parent goes in, make it so can ask a v3 file not to write tags. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12593) Tags and Tag dictionary to work with BB
[ https://issues.apache.org/jira/browse/HBASE-12593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067612#comment-15067612 ] stack commented on HBASE-12593: --- Sounds good then [~anoop.hbase] Want to put up a sketch patch? Thanks sir. > Tags and Tag dictionary to work with BB > --- > > Key: HBASE-12593 > URL: https://issues.apache.org/jira/browse/HBASE-12593 > Project: HBase > Issue Type: Sub-task > Components: regionserver, Scanners >Reporter: ramkrishna.s.vasudevan >Assignee: Anoop Sam John > > Adding the subtask so that we don't forget it. Came up while reviewing the > items required for this parent task. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15021) hadoopqa doing false positives
[
https://issues.apache.org/jira/browse/HBASE-15021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-15021:
--
Attachment: 15021.thrownpe.patch
Pushed the attempted fix. Now let me retry my patch that fails on NPE and
should show as failed unit tests.
> hadoopqa doing false positives
> --
>
> Key: HBASE-15021
> URL: https://issues.apache.org/jira/browse/HBASE-15021
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Attachments: 15021.patch, 15021.thrownpe.patch, 15021.thrownpe.patch
>
>
> https://builds.apache.org/job/PreCommit-HBASE-Build/16930/consoleText says:
> {color:green}+1 core tests{color}. The patch passed unit tests in .
> ...but here is what happened:
> {code}
> ...
> Results :
> Tests in error:
> org.apache.hadoop.hbase.regionserver.TestRSStatusServlet.testBasic(org.apache.hadoop.hbase.regionserver.TestRSStatusServlet)
> Run 1: TestRSStatusServlet.testBasic:105 � NullPointer
> Run 2: TestRSStatusServlet.testBasic:105 � NullPointer
> Run 3: TestRSStatusServlet.testBasic:105 � NullPointer
> org.apache.hadoop.hbase.regionserver.TestRSStatusServlet.testWithRegions(org.apache.hadoop.hbase.regionserver.TestRSStatusServlet)
> Run 1: TestRSStatusServlet.testWithRegions:119 � NullPointer
> Run 2: TestRSStatusServlet.testWithRegions:119 � NullPointer
> Run 3: TestRSStatusServlet.testWithRegions:119 � NullPointer
> Tests run: 1033, Failures: 0, Errors: 2, Skipped: 21
> ...
> [INFO] Apache HBase - Server . FAILURE
> [17:54.559s]
> ...
> {code}
> Why we reporting pass when it failed?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15021) hadoopqa doing false positives
[
https://issues.apache.org/jira/browse/HBASE-15021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067605#comment-15067605
]
stack commented on HBASE-15021:
---
I think I broke it adding in an export.. .it overshadowed the return from the
grep of unit test cases. Let me try this patch:
{code}
diff --git a/dev-support/test-patch.sh b/dev-support/test-patch.sh
index 9366319..509edeb 100755
--- a/dev-support/test-patch.sh
+++ b/dev-support/test-patch.sh
@@ -841,14 +841,13 @@ runTests () {
echo "=="
echo ""
echo ""
-
failed_tests=""
echo "$MVN clean test -Dsurefire.rerunFailingTestsCount=2 -P runAllTests
-D${PROJECT_NAME}PatchProcess"
export MAVEN_OPTS="${MAVEN_OPTS}"
ulimit -a
- $MVN clean test -Dsurefire.rerunFailingTestsCount=2 -P runAllTests
-D${PROJECT_NAME}PatchProcess
# Need to export this so the zombie subshell picks up current content
export JIRA_COMMENT
+ $MVN clean test -Dsurefire.rerunFailingTestsCount=2 -P runAllTests
-D${PROJECT_NAME}PatchProcess
if [[ $? != 0 ]] ; then
### Find and format names of failed tests
{code}
> hadoopqa doing false positives
> --
>
> Key: HBASE-15021
> URL: https://issues.apache.org/jira/browse/HBASE-15021
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Attachments: 15021.patch, 15021.thrownpe.patch
>
>
> https://builds.apache.org/job/PreCommit-HBASE-Build/16930/consoleText says:
> {color:green}+1 core tests{color}. The patch passed unit tests in .
> ...but here is what happened:
> {code}
> ...
> Results :
> Tests in error:
> org.apache.hadoop.hbase.regionserver.TestRSStatusServlet.testBasic(org.apache.hadoop.hbase.regionserver.TestRSStatusServlet)
> Run 1: TestRSStatusServlet.testBasic:105 � NullPointer
> Run 2: TestRSStatusServlet.testBasic:105 � NullPointer
> Run 3: TestRSStatusServlet.testBasic:105 � NullPointer
> org.apache.hadoop.hbase.regionserver.TestRSStatusServlet.testWithRegions(org.apache.hadoop.hbase.regionserver.TestRSStatusServlet)
> Run 1: TestRSStatusServlet.testWithRegions:119 � NullPointer
> Run 2: TestRSStatusServlet.testWithRegions:119 � NullPointer
> Run 3: TestRSStatusServlet.testWithRegions:119 � NullPointer
> Tests run: 1033, Failures: 0, Errors: 2, Skipped: 21
> ...
> [INFO] Apache HBase - Server . FAILURE
> [17:54.559s]
> ...
> {code}
> Why we reporting pass when it failed?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15021) hadoopqa doing false positives
[
https://issues.apache.org/jira/browse/HBASE-15021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-15021:
--
Attachment: 15021.patch
Fix to test-patch.sh to try.
> hadoopqa doing false positives
> --
>
> Key: HBASE-15021
> URL: https://issues.apache.org/jira/browse/HBASE-15021
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Attachments: 15021.patch, 15021.thrownpe.patch
>
>
> https://builds.apache.org/job/PreCommit-HBASE-Build/16930/consoleText says:
> {color:green}+1 core tests{color}. The patch passed unit tests in .
> ...but here is what happened:
> {code}
> ...
> Results :
> Tests in error:
> org.apache.hadoop.hbase.regionserver.TestRSStatusServlet.testBasic(org.apache.hadoop.hbase.regionserver.TestRSStatusServlet)
> Run 1: TestRSStatusServlet.testBasic:105 � NullPointer
> Run 2: TestRSStatusServlet.testBasic:105 � NullPointer
> Run 3: TestRSStatusServlet.testBasic:105 � NullPointer
> org.apache.hadoop.hbase.regionserver.TestRSStatusServlet.testWithRegions(org.apache.hadoop.hbase.regionserver.TestRSStatusServlet)
> Run 1: TestRSStatusServlet.testWithRegions:119 � NullPointer
> Run 2: TestRSStatusServlet.testWithRegions:119 � NullPointer
> Run 3: TestRSStatusServlet.testWithRegions:119 � NullPointer
> Tests run: 1033, Failures: 0, Errors: 2, Skipped: 21
> ...
> [INFO] Apache HBase - Server . FAILURE
> [17:54.559s]
> ...
> {code}
> Why we reporting pass when it failed?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13158) When client supports CellBlock, return the result Cells as controller payload for get(Get) API also
[ https://issues.apache.org/jira/browse/HBASE-13158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067591#comment-15067591 ] stack commented on HBASE-13158: --- [~anoop.hbase] Ok. Good. Thanks. Let me try and figure what is up w/ the precommit not failing on bad test. > When client supports CellBlock, return the result Cells as controller payload > for get(Get) API also > --- > > Key: HBASE-13158 > URL: https://issues.apache.org/jira/browse/HBASE-13158 > Project: HBase > Issue Type: Improvement >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0, 1.3.0 > > Attachments: 13158v4.suggestion.txt, HBASE-13158.patch, > HBASE-13158_V2.patch, HBASE-13158_V3.patch, HBASE-13158_V4.patch, > HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V6.patch, > HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V7.patch, > HBASE-13158_V8.patch, HBASE-13158_addendum_branch-1.patch, > HBASE-13158_branch-1.patch, HBASE-13158_branch-1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14279) Race condition in ConcurrentIndex
[
https://issues.apache.org/jira/browse/HBASE-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067585#comment-15067585
]
Hiroshi Ikeda commented on HBASE-14279:
---
I have second thought about the sentence:
{quote}
+ // Copied from ConcurrentHashMap
{quote}
It is better to remove the sentence because of avoiding a license issue. I hope
that the logic of the variant of single-word Wang/Jenkins hash is well-known
ordinary one.
> Race condition in ConcurrentIndex
> -
>
> Key: HBASE-14279
> URL: https://issues.apache.org/jira/browse/HBASE-14279
> Project: HBase
> Issue Type: Bug
>Reporter: Hiroshi Ikeda
>Assignee: Heng Chen
>Priority: Minor
> Attachments: HBASE-14279.patch, HBASE-14279_v2.patch,
> HBASE-14279_v3.patch, HBASE-14279_v4.patch, HBASE-14279_v5.patch,
> HBASE-14279_v5.patch, HBASE-14279_v6.patch, LockStripedBag.java
>
>
> {{ConcurrentIndex.put}} and {{remove}} are in race condition. It is possible
> to remove a non-empty set, and to add a value to a removed set. Also
> {{ConcurrentIndex.values}} is vague in sense that the returned set sometimes
> trace the current state and sometimes doesn't.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14970) Backport HBASE-13082 and its sub-jira to branch-1
[ https://issues.apache.org/jira/browse/HBASE-14970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067571#comment-15067571 ] Anoop Sam John commented on HBASE-14970: IMO we need basic change that will move the ChoreService at Region level to RS level. Because this will make the total threads (dedicated to be running and possibly coming in the system) depend on #regions and #cfs.. When we allow more regions per RS, we will land in issues. So better to have a cleaner chore running at RS level which can make use of a pool of fixed max threads count. So the total #threads which this feature can use is predictable and under better control. I guess that change has to happen in trunk 1st. So we can make a new Jira for that and once it is in trunk, we can make similar change for this backport patch as well. > Backport HBASE-13082 and its sub-jira to branch-1 > - > > Key: HBASE-14970 > URL: https://issues.apache.org/jira/browse/HBASE-14970 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Attachments: HBASE-13082-branch-1.patch, HBASE-14970_branch-1.patch, > HBASE-14970_branch-1.patch, HBASE-14970_branch-1.patch, > HBASE-14970_branch-1_1.patch, HBASE-14970_branch-1_2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14030) HBase Backup/Restore Phase 1
[
https://issues.apache.org/jira/browse/HBASE-14030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067555#comment-15067555
]
Hadoop QA commented on HBASE-14030:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12778959/HBASE-14030-v22.patch
against master branch at commit 9a297ef0a54de736bad53de3b0b518a9eec1865c.
ATTACHMENT ID: 12778959
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 23 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:red}-1 javac{color}. The applied patch generated 43 javac compiler
warnings (more than the master's current 35 warnings).
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
new checkstyle errors. Check build console for list of new errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16962//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16962//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16962//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16962//console
This message is automatically generated.
> HBase Backup/Restore Phase 1
>
>
> Key: HBASE-14030
> URL: https://issues.apache.org/jira/browse/HBASE-14030
> Project: HBase
> Issue Type: Umbrella
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-14030-v0.patch, HBASE-14030-v1.patch,
> HBASE-14030-v10.patch, HBASE-14030-v11.patch, HBASE-14030-v12.patch,
> HBASE-14030-v13.patch, HBASE-14030-v14.patch, HBASE-14030-v15.patch,
> HBASE-14030-v17.patch, HBASE-14030-v18.patch, HBASE-14030-v2.patch,
> HBASE-14030-v20.patch, HBASE-14030-v21.patch, HBASE-14030-v22.patch,
> HBASE-14030-v3.patch, HBASE-14030-v4.patch, HBASE-14030-v5.patch,
> HBASE-14030-v6.patch, HBASE-14030-v7.patch, HBASE-14030-v8.patch
>
>
> This is the umbrella ticket for Backup/Restore Phase 1. See HBASE-7912 design
> doc for the phase description.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13158) When client supports CellBlock, return the result Cells as controller payload for get(Get) API also
[ https://issues.apache.org/jira/browse/HBASE-13158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-13158: --- Resolution: Fixed Status: Resolved (was: Patch Available) > When client supports CellBlock, return the result Cells as controller payload > for get(Get) API also > --- > > Key: HBASE-13158 > URL: https://issues.apache.org/jira/browse/HBASE-13158 > Project: HBase > Issue Type: Improvement >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0, 1.3.0 > > Attachments: 13158v4.suggestion.txt, HBASE-13158.patch, > HBASE-13158_V2.patch, HBASE-13158_V3.patch, HBASE-13158_V4.patch, > HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V6.patch, > HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V7.patch, > HBASE-13158_V8.patch, HBASE-13158_addendum_branch-1.patch, > HBASE-13158_branch-1.patch, HBASE-13158_branch-1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13158) When client supports CellBlock, return the result Cells as controller payload for get(Get) API also
[ https://issues.apache.org/jira/browse/HBASE-13158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067530#comment-15067530 ] Anoop Sam John commented on HBASE-13158: This time checked the test results at https://builds.apache.org/job/PreCommit-HBASE-Build/16963//testReport/ All passed. Will commit now. > When client supports CellBlock, return the result Cells as controller payload > for get(Get) API also > --- > > Key: HBASE-13158 > URL: https://issues.apache.org/jira/browse/HBASE-13158 > Project: HBase > Issue Type: Improvement >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0, 1.3.0 > > Attachments: 13158v4.suggestion.txt, HBASE-13158.patch, > HBASE-13158_V2.patch, HBASE-13158_V3.patch, HBASE-13158_V4.patch, > HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V6.patch, > HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V7.patch, > HBASE-13158_V8.patch, HBASE-13158_addendum_branch-1.patch, > HBASE-13158_branch-1.patch, HBASE-13158_branch-1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15021) hadoopqa doing false positives
[
https://issues.apache.org/jira/browse/HBASE-15021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067527#comment-15067527
]
Anoop Sam John commented on HBASE-15021:
Same issue I got in HBASE-13158 for branch-1 patch as well.
> hadoopqa doing false positives
> --
>
> Key: HBASE-15021
> URL: https://issues.apache.org/jira/browse/HBASE-15021
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Attachments: 15021.thrownpe.patch
>
>
> https://builds.apache.org/job/PreCommit-HBASE-Build/16930/consoleText says:
> {color:green}+1 core tests{color}. The patch passed unit tests in .
> ...but here is what happened:
> {code}
> ...
> Results :
> Tests in error:
> org.apache.hadoop.hbase.regionserver.TestRSStatusServlet.testBasic(org.apache.hadoop.hbase.regionserver.TestRSStatusServlet)
> Run 1: TestRSStatusServlet.testBasic:105 � NullPointer
> Run 2: TestRSStatusServlet.testBasic:105 � NullPointer
> Run 3: TestRSStatusServlet.testBasic:105 � NullPointer
> org.apache.hadoop.hbase.regionserver.TestRSStatusServlet.testWithRegions(org.apache.hadoop.hbase.regionserver.TestRSStatusServlet)
> Run 1: TestRSStatusServlet.testWithRegions:119 � NullPointer
> Run 2: TestRSStatusServlet.testWithRegions:119 � NullPointer
> Run 3: TestRSStatusServlet.testWithRegions:119 � NullPointer
> Tests run: 1033, Failures: 0, Errors: 2, Skipped: 21
> ...
> [INFO] Apache HBase - Server . FAILURE
> [17:54.559s]
> ...
> {code}
> Why we reporting pass when it failed?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13158) When client supports CellBlock, return the result Cells as controller payload for get(Get) API also
[
https://issues.apache.org/jira/browse/HBASE-13158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067526#comment-15067526
]
Hadoop QA commented on HBASE-13158:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12778960/HBASE-13158_addendum_branch-1.patch
against branch-1 branch at commit 9a297ef0a54de736bad53de3b0b518a9eec1865c.
ATTACHMENT ID: 12778960
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16963//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16963//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16963//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16963//console
This message is automatically generated.
> When client supports CellBlock, return the result Cells as controller payload
> for get(Get) API also
> ---
>
> Key: HBASE-13158
> URL: https://issues.apache.org/jira/browse/HBASE-13158
> Project: HBase
> Issue Type: Improvement
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 13158v4.suggestion.txt, HBASE-13158.patch,
> HBASE-13158_V2.patch, HBASE-13158_V3.patch, HBASE-13158_V4.patch,
> HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V6.patch,
> HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V7.patch,
> HBASE-13158_V8.patch, HBASE-13158_addendum_branch-1.patch,
> HBASE-13158_branch-1.patch, HBASE-13158_branch-1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14511) StoreFile.Writer Meta Plugin
[ https://issues.apache.org/jira/browse/HBASE-14511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14511: -- Attachment: HBASE-14511-v4.patch > StoreFile.Writer Meta Plugin > > > Key: HBASE-14511 > URL: https://issues.apache.org/jira/browse/HBASE-14511 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-14511-v3.patch, HBASE-14511-v4.patch, > HBASE-14511.v1.patch, HBASE-14511.v2.patch > > > During my work on a new compaction policies (HBASE-14468, HBASE-14477) I had > to modify the existing code of a StoreFile.Writer to add additional meta-info > required by these new policies. I think that it should be done by means of a > new Plugin framework, because this seems to be a general capability/feature. > As a future enhancement this can become a part of a more general > StoreFileWriter/Reader plugin architecture. But I need only Meta section of a > store file. > This could be used, for example, to collect rowkeys distribution information > during hfile creation. This info can be used later to find the optimal region > split key or to create optimal set of sub-regions for M/R jobs or other jobs > which can operate on a sub-region level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14511) StoreFile.Writer Meta Plugin
[ https://issues.apache.org/jira/browse/HBASE-14511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14511: -- Status: Patch Available (was: Open) > StoreFile.Writer Meta Plugin > > > Key: HBASE-14511 > URL: https://issues.apache.org/jira/browse/HBASE-14511 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-14511-v3.patch, HBASE-14511-v4.patch, > HBASE-14511.v1.patch, HBASE-14511.v2.patch > > > During my work on a new compaction policies (HBASE-14468, HBASE-14477) I had > to modify the existing code of a StoreFile.Writer to add additional meta-info > required by these new policies. I think that it should be done by means of a > new Plugin framework, because this seems to be a general capability/feature. > As a future enhancement this can become a part of a more general > StoreFileWriter/Reader plugin architecture. But I need only Meta section of a > store file. > This could be used, for example, to collect rowkeys distribution information > during hfile creation. This info can be used later to find the optimal region > split key or to create optimal set of sub-regions for M/R jobs or other jobs > which can operate on a sub-region level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14511) StoreFile.Writer Meta Plugin
[ https://issues.apache.org/jira/browse/HBASE-14511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14511: -- Status: Open (was: Patch Available) > StoreFile.Writer Meta Plugin > > > Key: HBASE-14511 > URL: https://issues.apache.org/jira/browse/HBASE-14511 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-14511-v3.patch, HBASE-14511.v1.patch, > HBASE-14511.v2.patch > > > During my work on a new compaction policies (HBASE-14468, HBASE-14477) I had > to modify the existing code of a StoreFile.Writer to add additional meta-info > required by these new policies. I think that it should be done by means of a > new Plugin framework, because this seems to be a general capability/feature. > As a future enhancement this can become a part of a more general > StoreFileWriter/Reader plugin architecture. But I need only Meta section of a > store file. > This could be used, for example, to collect rowkeys distribution information > during hfile creation. This info can be used later to find the optimal region > split key or to create optimal set of sub-regions for M/R jobs or other jobs > which can operate on a sub-region level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14511) StoreFile.Writer Meta Plugin
[ https://issues.apache.org/jira/browse/HBASE-14511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14511: -- Attachment: (was: HBASE-14511-v3.patch) > StoreFile.Writer Meta Plugin > > > Key: HBASE-14511 > URL: https://issues.apache.org/jira/browse/HBASE-14511 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-14511-v3.patch, HBASE-14511.v1.patch, > HBASE-14511.v2.patch > > > During my work on a new compaction policies (HBASE-14468, HBASE-14477) I had > to modify the existing code of a StoreFile.Writer to add additional meta-info > required by these new policies. I think that it should be done by means of a > new Plugin framework, because this seems to be a general capability/feature. > As a future enhancement this can become a part of a more general > StoreFileWriter/Reader plugin architecture. But I need only Meta section of a > store file. > This could be used, for example, to collect rowkeys distribution information > during hfile creation. This info can be used later to find the optimal region > split key or to create optimal set of sub-regions for M/R jobs or other jobs > which can operate on a sub-region level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14684) Try to remove all MiniMapReduceCluster in unit tests
[ https://issues.apache.org/jira/browse/HBASE-14684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen updated HBASE-14684: -- Attachment: HBASE-14684-branch-1.2.patch upload patch for branch-1.2 > Try to remove all MiniMapReduceCluster in unit tests > > > Key: HBASE-14684 > URL: https://issues.apache.org/jira/browse/HBASE-14684 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Heng Chen >Assignee: Heng Chen >Priority: Critical > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: 14684.branch-1.txt, 14684.branch-1.txt, > 14684.branch-1.txt, HBASE-14684-branch-1.2.patch, HBASE-14684-branch-1.patch, > HBASE-14684-branch-1.patch, HBASE-14684-branch-1.patch, > HBASE-14684-branch-1_v1.patch, HBASE-14684.patch, HBASE-14684_v1.patch > > > As discussion in dev list, we will try to do MR job without > MiniMapReduceCluster. > Testcases will run faster and more reliable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14684) Try to remove all MiniMapReduceCluster in unit tests
[ https://issues.apache.org/jira/browse/HBASE-14684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen updated HBASE-14684: -- Fix Version/s: 1.3.0 1.2.0 > Try to remove all MiniMapReduceCluster in unit tests > > > Key: HBASE-14684 > URL: https://issues.apache.org/jira/browse/HBASE-14684 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Heng Chen >Assignee: Heng Chen >Priority: Critical > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: 14684.branch-1.txt, 14684.branch-1.txt, > 14684.branch-1.txt, HBASE-14684-branch-1.2.patch, HBASE-14684-branch-1.patch, > HBASE-14684-branch-1.patch, HBASE-14684-branch-1.patch, > HBASE-14684-branch-1_v1.patch, HBASE-14684.patch, HBASE-14684_v1.patch > > > As discussion in dev list, we will try to do MR job without > MiniMapReduceCluster. > Testcases will run faster and more reliable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14511) StoreFile.Writer Meta Plugin
[
https://issues.apache.org/jira/browse/HBASE-14511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067511#comment-15067511
]
Hadoop QA commented on HBASE-14511:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12778974/HBASE-14511-v3.patch
against master branch at commit 9a297ef0a54de736bad53de3b0b518a9eec1865c.
ATTACHMENT ID: 12778974
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16964//console
This message is automatically generated.
> StoreFile.Writer Meta Plugin
>
>
> Key: HBASE-14511
> URL: https://issues.apache.org/jira/browse/HBASE-14511
> Project: HBase
> Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-14511-v3.patch, HBASE-14511-v3.patch,
> HBASE-14511.v1.patch, HBASE-14511.v2.patch
>
>
> During my work on a new compaction policies (HBASE-14468, HBASE-14477) I had
> to modify the existing code of a StoreFile.Writer to add additional meta-info
> required by these new policies. I think that it should be done by means of a
> new Plugin framework, because this seems to be a general capability/feature.
> As a future enhancement this can become a part of a more general
> StoreFileWriter/Reader plugin architecture. But I need only Meta section of a
> store file.
> This could be used, for example, to collect rowkeys distribution information
> during hfile creation. This info can be used later to find the optimal region
> split key or to create optimal set of sub-regions for M/R jobs or other jobs
> which can operate on a sub-region level.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14279) Race condition in ConcurrentIndex
[
https://issues.apache.org/jira/browse/HBASE-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067510#comment-15067510
]
Heng Chen commented on HBASE-14279:
---
I need one more +1 before commit.
> Race condition in ConcurrentIndex
> -
>
> Key: HBASE-14279
> URL: https://issues.apache.org/jira/browse/HBASE-14279
> Project: HBase
> Issue Type: Bug
>Reporter: Hiroshi Ikeda
>Assignee: Heng Chen
>Priority: Minor
> Attachments: HBASE-14279.patch, HBASE-14279_v2.patch,
> HBASE-14279_v3.patch, HBASE-14279_v4.patch, HBASE-14279_v5.patch,
> HBASE-14279_v5.patch, HBASE-14279_v6.patch, LockStripedBag.java
>
>
> {{ConcurrentIndex.put}} and {{remove}} are in race condition. It is possible
> to remove a non-empty set, and to add a value to a removed set. Also
> {{ConcurrentIndex.values}} is vague in sense that the returned set sometimes
> trace the current state and sometimes doesn't.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14852) Update build env
[
https://issues.apache.org/jira/browse/HBASE-14852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067502#comment-15067502
]
Hadoop QA commented on HBASE-14852:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12778948/HBASE-14852-v10.patch
against master branch at commit 9a297ef0a54de736bad53de3b0b518a9eec1865c.
ATTACHMENT ID: 12778948
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation, build,
or dev-support patch that doesn't require tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16961//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16961//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16961//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16961//console
This message is automatically generated.
> Update build env
>
>
> Key: HBASE-14852
> URL: https://issues.apache.org/jira/browse/HBASE-14852
> Project: HBase
> Issue Type: Sub-task
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Attachments: HBASE-14852-v1.patch, HBASE-14852-v10.patch,
> HBASE-14852-v3.patch, HBASE-14852-v4.patch, HBASE-14852-v5.patch,
> HBASE-14852-v7.patch, HBASE-14852-v8.patch, HBASE-14852-v9.patch,
> HBASE-14852.patch
>
>
> * Use docker to ensure that everyone has the correct versions of libs
> * Use buck to build.
> * Include Folly for IOBuf.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14940) Make our unsafe based ops more safe
[ https://issues.apache.org/jira/browse/HBASE-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067501#comment-15067501 ] Anoop Sam John commented on HBASE-14940: The required code is in Bits.java#unaligned() which is a static private method. Can do reflection call on that. As it is a private class and private method in Java in future if Java make any changes to Class name or method name or signature, we may land in issues.. That is why I decided to copy paste the code. So if there is a consensus to use Reflection based look up, I can change it. Patch is committed to trunk already. I can do the needful change as addendum commit. > Make our unsafe based ops more safe > --- > > Key: HBASE-14940 > URL: https://issues.apache.org/jira/browse/HBASE-14940 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Attachments: HBASE-14940.patch, HBASE-14940_branch-1.patch, > HBASE-14940_branch-1.patch, HBASE-14940_branch-1.patch > > > Thanks for the nice findings [~ikeda] > This jira solves 3 issues with Unsafe operations and ByteBufferUtils > 1. We can do sun unsafe based reads and writes iff unsafe package is > available and underlying platform is having unaligned-access capability. But > we were missing the second check > 2. Java NIO is doing a chunk based copy while doing Unsafe copyMemory. The > max chunk size is 1 MB. This is done for "A limit is imposed to allow for > safepoint polling during a large copy" as mentioned in comments in Bits.java. > We are also going to do same way > 3. In ByteBufferUtils, when Unsafe is not available and ByteBuffers are off > heap, we were doing byte by byte operation (read/copy). We can avoid this and > do better way. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14852) Update build env
[
https://issues.apache.org/jira/browse/HBASE-14852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067499#comment-15067499
]
Hadoop QA commented on HBASE-14852:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12778946/HBASE-14852-v9.patch
against master branch at commit 9a297ef0a54de736bad53de3b0b518a9eec1865c.
ATTACHMENT ID: 12778946
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation, build,
or dev-support patch that doesn't require tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16960//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16960//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16960//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16960//console
This message is automatically generated.
> Update build env
>
>
> Key: HBASE-14852
> URL: https://issues.apache.org/jira/browse/HBASE-14852
> Project: HBase
> Issue Type: Sub-task
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Attachments: HBASE-14852-v1.patch, HBASE-14852-v10.patch,
> HBASE-14852-v3.patch, HBASE-14852-v4.patch, HBASE-14852-v5.patch,
> HBASE-14852-v7.patch, HBASE-14852-v8.patch, HBASE-14852-v9.patch,
> HBASE-14852.patch
>
>
> * Use docker to ensure that everyone has the correct versions of libs
> * Use buck to build.
> * Include Folly for IOBuf.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14654) Reenable TestMultiParallel#testActiveThreadsCount
[ https://issues.apache.org/jira/browse/HBASE-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen updated HBASE-14654: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Reenable TestMultiParallel#testActiveThreadsCount > - > > Key: HBASE-14654 > URL: https://issues.apache.org/jira/browse/HBASE-14654 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0 >Reporter: Heng Chen >Assignee: Heng Chen > Labels: flakey > Fix For: 2.0.0 > > Attachments: HBASE-14654.patch, HBASE-14654.patch > > > It was disabled in HBASE-14642, this issue should reenable it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14654) Reenable TestMultiParallel#testActiveThreadsCount
[ https://issues.apache.org/jira/browse/HBASE-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067494#comment-15067494 ] Heng Chen commented on HBASE-14654: --- push to master. Thanks stack. > Reenable TestMultiParallel#testActiveThreadsCount > - > > Key: HBASE-14654 > URL: https://issues.apache.org/jira/browse/HBASE-14654 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0 >Reporter: Heng Chen >Assignee: Heng Chen > Labels: flakey > Fix For: 2.0.0 > > Attachments: HBASE-14654.patch, HBASE-14654.patch > > > It was disabled in HBASE-14642, this issue should reenable it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15014) Fix filterCellByStore in WALsplitter is awful for performance
[
https://issues.apache.org/jira/browse/HBASE-15014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067493#comment-15067493
]
Anoop Sam John commented on HBASE-15014:
bq.Log replay which is slated to be removed and shouldn't be trusted for
anything.
You mean the DLR right? Not just that but the DLS and its replay while the
region open also uses this METAFAMILY. The pasted above code is in that path
only.
Any way handled it in new version of patch.. +1..
Nice work.
> Fix filterCellByStore in WALsplitter is awful for performance
> -
>
> Key: HBASE-15014
> URL: https://issues.apache.org/jira/browse/HBASE-15014
> Project: HBase
> Issue Type: Bug
> Components: MTTR, Recovery, wal
>Affects Versions: 1.2.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-15014-v1.patch, HBASE-15014-v2.patch,
> HBASE-15014-v3.patch, HBASE-15014.patch
>
>
> Testing the latest 1.2 I see this when there is a regionserver that crashes.
> {code}
> Thread 921 (RS_LOG_REPLAY_OPS-hbase2698:16020-0-Writer-1):
> State: RUNNABLE
> Blocked count: 6354
> Waited count: 6249
> Stack:
> org.apache.hadoop.hbase.KeyValue.equals(KeyValue.java:1128)
> java.util.ArrayList.indexOf(ArrayList.java:317)
> java.util.ArrayList.contains(ArrayList.java:300)
> java.util.ArrayList.batchRemove(ArrayList.java:720)
> java.util.ArrayList.removeAll(ArrayList.java:690)
>
> org.apache.hadoop.hbase.wal.WALSplitter$LogRecoveredEditsOutputSink.filterCellByStore(WALSplitter.java:1529)
>
> org.apache.hadoop.hbase.wal.WALSplitter$LogRecoveredEditsOutputSink.append(WALSplitter.java:1557)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.writeBuffer(WALSplitter.java:1113)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.doRun(WALSplitter.java:1105)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.run(WALSplitter.java:1075)
> Thread 920 (RS_LOG_REPLAY_OPS-hbase2698:16020-0-Writer-0):
> State: TIMED_WAITING
> Blocked count: 17560
> Waited count: 19695
> Stack:
> java.lang.Object.wait(Native Method)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.doRun(WALSplitter.java:1093)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.run(WALSplitter.java:1075)
> Thread 919 (RS_LOG_REPLAY_OPS-hbase2698:16020-0):
> State: TIMED_WAITING
> Blocked count: 115
> Waited count: 976
> Stack:
> java.lang.Object.wait(Native Method)
>
> org.apache.hadoop.hbase.wal.WALSplitter$EntryBuffers.appendEntry(WALSplitter.java:944)
> org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:365)
> org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:236)
>
> org.apache.hadoop.hbase.regionserver.SplitLogWorker$1.exec(SplitLogWorker.java:104)
>
> org.apache.hadoop.hbase.regionserver.handler.WALSplitterHandler.process(WALSplitterHandler.java:72)
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> {code}
> This has been going on for >10 mins.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14684) Try to remove all MiniMapReduceCluster in unit tests
[ https://issues.apache.org/jira/browse/HBASE-14684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067490#comment-15067490 ] Heng Chen commented on HBASE-14684: --- Push to branch-1 > Try to remove all MiniMapReduceCluster in unit tests > > > Key: HBASE-14684 > URL: https://issues.apache.org/jira/browse/HBASE-14684 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Heng Chen >Assignee: Heng Chen >Priority: Critical > Fix For: 2.0.0 > > Attachments: 14684.branch-1.txt, 14684.branch-1.txt, > 14684.branch-1.txt, HBASE-14684-branch-1.patch, HBASE-14684-branch-1.patch, > HBASE-14684-branch-1.patch, HBASE-14684-branch-1_v1.patch, HBASE-14684.patch, > HBASE-14684_v1.patch > > > As discussion in dev list, we will try to do MR job without > MiniMapReduceCluster. > Testcases will run faster and more reliable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14511) StoreFile.Writer Meta Plugin
[ https://issues.apache.org/jira/browse/HBASE-14511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14511: -- Status: Patch Available (was: In Progress) > StoreFile.Writer Meta Plugin > > > Key: HBASE-14511 > URL: https://issues.apache.org/jira/browse/HBASE-14511 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-14511-v3.patch, HBASE-14511-v3.patch, > HBASE-14511.v1.patch, HBASE-14511.v2.patch > > > During my work on a new compaction policies (HBASE-14468, HBASE-14477) I had > to modify the existing code of a StoreFile.Writer to add additional meta-info > required by these new policies. I think that it should be done by means of a > new Plugin framework, because this seems to be a general capability/feature. > As a future enhancement this can become a part of a more general > StoreFileWriter/Reader plugin architecture. But I need only Meta section of a > store file. > This could be used, for example, to collect rowkeys distribution information > during hfile creation. This info can be used later to find the optimal region > split key or to create optimal set of sub-regions for M/R jobs or other jobs > which can operate on a sub-region level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14511) StoreFile.Writer Meta Plugin
[ https://issues.apache.org/jira/browse/HBASE-14511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14511: -- Attachment: HBASE-14511-v3.patch Patch v4. > StoreFile.Writer Meta Plugin > > > Key: HBASE-14511 > URL: https://issues.apache.org/jira/browse/HBASE-14511 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-14511-v3.patch, HBASE-14511-v3.patch, > HBASE-14511.v1.patch, HBASE-14511.v2.patch > > > During my work on a new compaction policies (HBASE-14468, HBASE-14477) I had > to modify the existing code of a StoreFile.Writer to add additional meta-info > required by these new policies. I think that it should be done by means of a > new Plugin framework, because this seems to be a general capability/feature. > As a future enhancement this can become a part of a more general > StoreFileWriter/Reader plugin architecture. But I need only Meta section of a > store file. > This could be used, for example, to collect rowkeys distribution information > during hfile creation. This info can be used later to find the optimal region > split key or to create optimal set of sub-regions for M/R jobs or other jobs > which can operate on a sub-region level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15014) Fix filterCellByStore in WALsplitter is awful for performance
[
https://issues.apache.org/jira/browse/HBASE-15014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067472#comment-15067472
]
Hadoop QA commented on HBASE-15014:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12778929/HBASE-15014-v3.patch
against master branch at commit 9a297ef0a54de736bad53de3b0b518a9eec1865c.
ATTACHMENT ID: 12778929
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16959//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16959//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16959//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16959//console
This message is automatically generated.
> Fix filterCellByStore in WALsplitter is awful for performance
> -
>
> Key: HBASE-15014
> URL: https://issues.apache.org/jira/browse/HBASE-15014
> Project: HBase
> Issue Type: Bug
> Components: MTTR, Recovery, wal
>Affects Versions: 1.2.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-15014-v1.patch, HBASE-15014-v2.patch,
> HBASE-15014-v3.patch, HBASE-15014.patch
>
>
> Testing the latest 1.2 I see this when there is a regionserver that crashes.
> {code}
> Thread 921 (RS_LOG_REPLAY_OPS-hbase2698:16020-0-Writer-1):
> State: RUNNABLE
> Blocked count: 6354
> Waited count: 6249
> Stack:
> org.apache.hadoop.hbase.KeyValue.equals(KeyValue.java:1128)
> java.util.ArrayList.indexOf(ArrayList.java:317)
> java.util.ArrayList.contains(ArrayList.java:300)
> java.util.ArrayList.batchRemove(ArrayList.java:720)
> java.util.ArrayList.removeAll(ArrayList.java:690)
>
> org.apache.hadoop.hbase.wal.WALSplitter$LogRecoveredEditsOutputSink.filterCellByStore(WALSplitter.java:1529)
>
> org.apache.hadoop.hbase.wal.WALSplitter$LogRecoveredEditsOutputSink.append(WALSplitter.java:1557)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.writeBuffer(WALSplitter.java:1113)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.doRun(WALSplitter.java:1105)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.run(WALSplitter.java:1075)
> Thread 920 (RS_LOG_REPLAY_OPS-hbase2698:16020-0-Writer-0):
> State: TIMED_WAITING
> Blocked count: 17560
> Waited count: 19695
> Stack:
> java.lang.Object.wait(Native Method)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.doRun(WALSplitter.java:1093)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.run(WALSplitter.java:1075)
> Thread 919 (RS_LOG_REPLAY_OPS-hbase2698:16020-0):
> State: TIMED_WAITING
> Blocked count: 115
> Waited count: 976
> Stack:
> java.lang.Object.wait(Native Method)
>
> org.apache.hadoop.hbase.wal.WALSplitter$EntryBuffers.appendEntry(WALSplitter.java:944)
> org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:365)
> org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALS
[jira] [Commented] (HBASE-15018) Inconsistent way of handling TimeoutException in the rpc client implemenations
[
https://issues.apache.org/jira/browse/HBASE-15018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067453#comment-15067453
]
Anoop Sam John commented on HBASE-15018:
Ping [~saint@gmail.com]. (He knows AsyncRpcClient very well). Is it good
to go in?
> Inconsistent way of handling TimeoutException in the rpc client implemenations
> --
>
> Key: HBASE-15018
> URL: https://issues.apache.org/jira/browse/HBASE-15018
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.1.0, 1.2.0
>Reporter: Ashish Singhi
>Assignee: Ashish Singhi
> Fix For: 2.0.0, 1.2.1, 1.1.3
>
> Attachments: HBASE-15018.patch
>
>
> If there is any rpc timeout when using RpcClientImpl then we wrap the
> exception in IOE and throw it,
> {noformat}
> 2015-11-16 04:05:24,935 WARN [main-EventThread.replicationSource,peer2]
> regionserver.HBaseInterClusterReplicationEndpoint: Can't replicate because of
> a local or network error:
> java.io.IOException: Call to host-XX:16040 failed on local exception:
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=510,
> waitTime=180001, operationTimeout=18 expired.
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1271)
> at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1239)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
> at
> org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.replicateWALEntry(AdminProtos.java:25690)
> at
> org.apache.hadoop.hbase.protobuf.ReplicationProtbufUtil.replicateWALEntry(ReplicationProtbufUtil.java:77)
> at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint$Replicator.call(HBaseInterClusterReplicationEndpoint.java:322)
> at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint$Replicator.call(HBaseInterClusterReplicationEndpoint.java:308)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=510,
> waitTime=180001, operationTimeout=18 expired.
> at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:70)
> at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1213)
> ... 10 more
> {noformat}
> But that isn't case with AsyncRpcClient, we don't wrap and throw
> CallTimeoutException as it is.
> {noformat}
> 2015-12-21 14:27:33,093 WARN
> [RS_OPEN_REGION-host-XX:16201-0.replicationSource.host-XX%2C16201%2C1450687255593,1]
> regionserver.HBaseInterClusterReplicationEndpoint: Can't replicate because
> of a local or network error:
> org.apache.hadoop.hbase.ipc.CallTimeoutException: callId=2,
> method=ReplicateWALEntry, rpcTimeout=60, param {TODO: class
> org.apache.hadoop.hbase.protobuf.generated.AdminProtos$ReplicateWALEntryRequest}
> at
> org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:257)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:295)
> at
> org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.replicateWALEntry(AdminProtos.java:23707)
> at
> org.apache.hadoop.hbase.protobuf.ReplicationProtbufUtil.replicateWALEntry(ReplicationProtbufUtil.java:73)
> at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint$Replicator.call(HBaseInterClusterReplicationEndpoint.java:387)
> at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint$Replicator.call(HBaseInterClusterReplicationEndpoint.java:370)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> I think we should have same behavior across both the implementations.
--
This message was sent by Atlassian JIRA
(v6.3.4#63
[jira] [Commented] (HBASE-12593) Tags and Tag dictionary to work with BB
[ https://issues.apache.org/jira/browse/HBASE-12593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067427#comment-15067427 ] Anoop Sam John commented on HBASE-12593: Not really Stack.. Then that interface has to be some thing which represent N Tags. We have getTagsByteBuffer() also in Cell. We iterate over byte[] or BB to search for certain tag type. That is fine. But we do have API to convert this to List. Tag is a class and which can be backed by byte[] only. So when the Cell data is in DBB, if any code path uses this API way, we will end up in copying. So my idea was to do the same way what Cell is doing. We have interface and impls of byte[] backing and DBB backing. Same way Tag will be an interface. As it is only in server side, it might be ok to add both APIs of byte[] and BB in same interface and have a hasArray() check. The getter for array can throw Exception if Tag is DBB backed. (This is to simplify things from Cell way where we have 2 interface for byte[] and BB) > Tags and Tag dictionary to work with BB > --- > > Key: HBASE-12593 > URL: https://issues.apache.org/jira/browse/HBASE-12593 > Project: HBase > Issue Type: Sub-task > Components: regionserver, Scanners >Reporter: ramkrishna.s.vasudevan >Assignee: Anoop Sam John > > Adding the subtask so that we don't forget it. Came up while reviewing the > items required for this parent task. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15023) Reenable TestShell and TestStochasticLoadBalancer
[
https://issues.apache.org/jira/browse/HBASE-15023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067423#comment-15067423
]
Hadoop QA commented on HBASE-15023:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12778924/15023.patch
against master branch at commit 9a297ef0a54de736bad53de3b0b518a9eec1865c.
ATTACHMENT ID: 12778924
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 9 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+testWithCluster(numNodes, numRegions, numRegionsPerServer,
replication, numTables, false, false);
+Map> newServerMap = new TreeMap>(serverMap);
+ServerName s2 = ServerName.valueOf(s1.getHostname(), s1.getPort() + i,
1); // create an RS for the same host
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16958//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16958//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16958//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16958//console
This message is automatically generated.
> Reenable TestShell and TestStochasticLoadBalancer
> -
>
> Key: HBASE-15023
> URL: https://issues.apache.org/jira/browse/HBASE-15023
> Project: HBase
> Issue Type: Sub-task
> Components: test
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0, 1.2.1
>
> Attachments: 15023.patch
>
>
> Parent disabled these tests when test runs were flakier than they are now.
> Try reenabling them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15022) undefined method `getZooKeeperClusterKey' for Java::OrgApacheHadoopHbaseZookeeper::ZKUtil:Class
[
https://issues.apache.org/jira/browse/HBASE-15022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067418#comment-15067418
]
Hudson commented on HBASE-15022:
FAILURE: Integrated in HBase-1.2 #464 (See
[https://builds.apache.org/job/HBase-1.2/464/])
HBASE-15022 replication_admin.rb throws undefined method (matteo.bertozzi: rev
f241dc141c87d98775fadcb12dd9c2c44e809e40)
* hbase-shell/src/main/ruby/hbase/replication_admin.rb
> undefined method `getZooKeeperClusterKey' for
> Java::OrgApacheHadoopHbaseZookeeper::ZKUtil:Class
> ---
>
> Key: HBASE-15022
> URL: https://issues.apache.org/jira/browse/HBASE-15022
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Fix For: 2.0.0, 1.2.0
>
> Attachments: HBASE-15022-v0.patch
>
>
> HBASE-14866 moved a bunch of methods from ZKUtil to ZKConfig
> and the shell was not updated
> {noformat}
> replication_admin.rb: cluster_key =
> ZKUtil.getZooKeeperClusterKey(@configuration)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15025) Allow clients configured with insecure fallback to attempt SIMPLE auth when KRB fails
[ https://issues.apache.org/jira/browse/HBASE-15025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067417#comment-15067417 ] Gary Helmling commented on HBASE-15025: --- Originally discussed in HBASE-14700. > Allow clients configured with insecure fallback to attempt SIMPLE auth when > KRB fails > - > > Key: HBASE-15025 > URL: https://issues.apache.org/jira/browse/HBASE-15025 > Project: HBase > Issue Type: Improvement > Components: security >Reporter: Gary Helmling >Assignee: Gary Helmling > > We have separate configurations for both client and server allowing a > "permissive" mode where connections to insecure servers and clients > (respectively) are allowed. However, if both client and server are > configured for Kerberos authentication for a given cluster, and Kerberos > authentication fails, the connection will still fail if the fallback > configurations are set to true. > If the client is configured to allow insecure fallback, and Kerberos > authentication fails, we could instead have the client retry with SIMPLE > auth. If the server is also configured to allow insecure fallback, this > would allow the connection to succeed in the case of transient problems with > Kerberos infrastructure, for example. > There is of course a danger that this would allow misconfigurations of > security to be silently ignored, but we can add some loud logging on the > client side when fallback to SIMPLE auth occurs, plus we have metrics and > logging on the server side for fallbacks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15025) Allow clients configured with insecure fallback to attempt SIMPLE auth when KRB fails
Gary Helmling created HBASE-15025: - Summary: Allow clients configured with insecure fallback to attempt SIMPLE auth when KRB fails Key: HBASE-15025 URL: https://issues.apache.org/jira/browse/HBASE-15025 Project: HBase Issue Type: Improvement Components: security Reporter: Gary Helmling Assignee: Gary Helmling We have separate configurations for both client and server allowing a "permissive" mode where connections to insecure servers and clients (respectively) are allowed. However, if both client and server are configured for Kerberos authentication for a given cluster, and Kerberos authentication fails, the connection will still fail if the fallback configurations are set to true. If the client is configured to allow insecure fallback, and Kerberos authentication fails, we could instead have the client retry with SIMPLE auth. If the server is also configured to allow insecure fallback, this would allow the connection to succeed in the case of transient problems with Kerberos infrastructure, for example. There is of course a danger that this would allow misconfigurations of security to be silently ignored, but we can add some loud logging on the client side when fallback to SIMPLE auth occurs, plus we have metrics and logging on the server side for fallbacks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13158) When client supports CellBlock, return the result Cells as controller payload for get(Get) API also
[ https://issues.apache.org/jira/browse/HBASE-13158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-13158: --- Status: Patch Available (was: Reopened) > When client supports CellBlock, return the result Cells as controller payload > for get(Get) API also > --- > > Key: HBASE-13158 > URL: https://issues.apache.org/jira/browse/HBASE-13158 > Project: HBase > Issue Type: Improvement >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0, 1.3.0 > > Attachments: 13158v4.suggestion.txt, HBASE-13158.patch, > HBASE-13158_V2.patch, HBASE-13158_V3.patch, HBASE-13158_V4.patch, > HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V6.patch, > HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V7.patch, > HBASE-13158_V8.patch, HBASE-13158_addendum_branch-1.patch, > HBASE-13158_branch-1.patch, HBASE-13158_branch-1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HBASE-13158) When client supports CellBlock, return the result Cells as controller payload for get(Get) API also
[ https://issues.apache.org/jira/browse/HBASE-13158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John reopened HBASE-13158: Yes Stack. Thanks. Not sure what is the issue with Pre commit QA. Patch for branch-1 passed all the tests ! I will attach addendum patch. > When client supports CellBlock, return the result Cells as controller payload > for get(Get) API also > --- > > Key: HBASE-13158 > URL: https://issues.apache.org/jira/browse/HBASE-13158 > Project: HBase > Issue Type: Improvement >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0, 1.3.0 > > Attachments: 13158v4.suggestion.txt, HBASE-13158.patch, > HBASE-13158_V2.patch, HBASE-13158_V3.patch, HBASE-13158_V4.patch, > HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V6.patch, > HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V7.patch, > HBASE-13158_V8.patch, HBASE-13158_addendum_branch-1.patch, > HBASE-13158_branch-1.patch, HBASE-13158_branch-1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13158) When client supports CellBlock, return the result Cells as controller payload for get(Get) API also
[ https://issues.apache.org/jira/browse/HBASE-13158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-13158: --- Attachment: HBASE-13158_addendum_branch-1.patch > When client supports CellBlock, return the result Cells as controller payload > for get(Get) API also > --- > > Key: HBASE-13158 > URL: https://issues.apache.org/jira/browse/HBASE-13158 > Project: HBase > Issue Type: Improvement >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0, 1.3.0 > > Attachments: 13158v4.suggestion.txt, HBASE-13158.patch, > HBASE-13158_V2.patch, HBASE-13158_V3.patch, HBASE-13158_V4.patch, > HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V6.patch, > HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V7.patch, > HBASE-13158_V8.patch, HBASE-13158_addendum_branch-1.patch, > HBASE-13158_branch-1.patch, HBASE-13158_branch-1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14030) HBase Backup/Restore Phase 1
[ https://issues.apache.org/jira/browse/HBASE-14030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14030: -- Attachment: HBASE-14030-v22.patch > HBase Backup/Restore Phase 1 > > > Key: HBASE-14030 > URL: https://issues.apache.org/jira/browse/HBASE-14030 > Project: HBase > Issue Type: Umbrella >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-14030-v0.patch, HBASE-14030-v1.patch, > HBASE-14030-v10.patch, HBASE-14030-v11.patch, HBASE-14030-v12.patch, > HBASE-14030-v13.patch, HBASE-14030-v14.patch, HBASE-14030-v15.patch, > HBASE-14030-v17.patch, HBASE-14030-v18.patch, HBASE-14030-v2.patch, > HBASE-14030-v20.patch, HBASE-14030-v21.patch, HBASE-14030-v22.patch, > HBASE-14030-v3.patch, HBASE-14030-v4.patch, HBASE-14030-v5.patch, > HBASE-14030-v6.patch, HBASE-14030-v7.patch, HBASE-14030-v8.patch > > > This is the umbrella ticket for Backup/Restore Phase 1. See HBASE-7912 design > doc for the phase description. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14030) HBase Backup/Restore Phase 1
[ https://issues.apache.org/jira/browse/HBASE-14030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14030: -- Status: Patch Available (was: Open) > HBase Backup/Restore Phase 1 > > > Key: HBASE-14030 > URL: https://issues.apache.org/jira/browse/HBASE-14030 > Project: HBase > Issue Type: Umbrella >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-14030-v0.patch, HBASE-14030-v1.patch, > HBASE-14030-v10.patch, HBASE-14030-v11.patch, HBASE-14030-v12.patch, > HBASE-14030-v13.patch, HBASE-14030-v14.patch, HBASE-14030-v15.patch, > HBASE-14030-v17.patch, HBASE-14030-v18.patch, HBASE-14030-v2.patch, > HBASE-14030-v20.patch, HBASE-14030-v21.patch, HBASE-14030-v22.patch, > HBASE-14030-v3.patch, HBASE-14030-v4.patch, HBASE-14030-v5.patch, > HBASE-14030-v6.patch, HBASE-14030-v7.patch, HBASE-14030-v8.patch > > > This is the umbrella ticket for Backup/Restore Phase 1. See HBASE-7912 design > doc for the phase description. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14030) HBase Backup/Restore Phase 1
[ https://issues.apache.org/jira/browse/HBASE-14030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14030: -- Status: Open (was: Patch Available) > HBase Backup/Restore Phase 1 > > > Key: HBASE-14030 > URL: https://issues.apache.org/jira/browse/HBASE-14030 > Project: HBase > Issue Type: Umbrella >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-14030-v0.patch, HBASE-14030-v1.patch, > HBASE-14030-v10.patch, HBASE-14030-v11.patch, HBASE-14030-v12.patch, > HBASE-14030-v13.patch, HBASE-14030-v14.patch, HBASE-14030-v15.patch, > HBASE-14030-v17.patch, HBASE-14030-v18.patch, HBASE-14030-v2.patch, > HBASE-14030-v20.patch, HBASE-14030-v21.patch, HBASE-14030-v22.patch, > HBASE-14030-v3.patch, HBASE-14030-v4.patch, HBASE-14030-v5.patch, > HBASE-14030-v6.patch, HBASE-14030-v7.patch, HBASE-14030-v8.patch > > > This is the umbrella ticket for Backup/Restore Phase 1. See HBASE-7912 design > doc for the phase description. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14852) Update build env
[ https://issues.apache.org/jira/browse/HBASE-14852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-14852: -- Attachment: HBASE-14852-v10.patch > Update build env > > > Key: HBASE-14852 > URL: https://issues.apache.org/jira/browse/HBASE-14852 > Project: HBase > Issue Type: Sub-task >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-14852-v1.patch, HBASE-14852-v10.patch, > HBASE-14852-v3.patch, HBASE-14852-v4.patch, HBASE-14852-v5.patch, > HBASE-14852-v7.patch, HBASE-14852-v8.patch, HBASE-14852-v9.patch, > HBASE-14852.patch > > > * Use docker to ensure that everyone has the correct versions of libs > * Use buck to build. > * Include Folly for IOBuf. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15022) undefined method `getZooKeeperClusterKey' for Java::OrgApacheHadoopHbaseZookeeper::ZKUtil:Class
[
https://issues.apache.org/jira/browse/HBASE-15022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067333#comment-15067333
]
Hudson commented on HBASE-15022:
FAILURE: Integrated in HBase-1.3 #457 (See
[https://builds.apache.org/job/HBase-1.3/457/])
HBASE-15022 replication_admin.rb throws undefined method (matteo.bertozzi: rev
85257a25282051bb2fa85ed893e308de6c5c3ff4)
* hbase-shell/src/main/ruby/hbase/replication_admin.rb
> undefined method `getZooKeeperClusterKey' for
> Java::OrgApacheHadoopHbaseZookeeper::ZKUtil:Class
> ---
>
> Key: HBASE-15022
> URL: https://issues.apache.org/jira/browse/HBASE-15022
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Fix For: 2.0.0, 1.2.0
>
> Attachments: HBASE-15022-v0.patch
>
>
> HBASE-14866 moved a bunch of methods from ZKUtil to ZKConfig
> and the shell was not updated
> {noformat}
> replication_admin.rb: cluster_key =
> ZKUtil.getZooKeeperClusterKey(@configuration)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14852) Update build env
[ https://issues.apache.org/jira/browse/HBASE-14852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-14852: -- Attachment: HBASE-14852-v9.patch Moved to /usr/local/src/hbase Added some additional info for docker and docker on osx. > Update build env > > > Key: HBASE-14852 > URL: https://issues.apache.org/jira/browse/HBASE-14852 > Project: HBase > Issue Type: Sub-task >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-14852-v1.patch, HBASE-14852-v3.patch, > HBASE-14852-v4.patch, HBASE-14852-v5.patch, HBASE-14852-v7.patch, > HBASE-14852-v8.patch, HBASE-14852-v9.patch, HBASE-14852.patch > > > * Use docker to ensure that everyone has the correct versions of libs > * Use buck to build. > * Include Folly for IOBuf. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14030) HBase Backup/Restore Phase 1
[
https://issues.apache.org/jira/browse/HBASE-14030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067323#comment-15067323
]
Hadoop QA commented on HBASE-14030:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12778905/HBASE-14030-v21.patch
against master branch at commit f56abf730f45823b2b6dd6d2daa5e33c69f3c86b.
ATTACHMENT ID: 12778905
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 23 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:red}-1 javac{color}. The applied patch generated 43 javac compiler
warnings (more than the master's current 35 warnings).
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
new checkstyle errors. Check build console for list of new errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16957//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16957//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16957//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16957//console
This message is automatically generated.
> HBase Backup/Restore Phase 1
>
>
> Key: HBASE-14030
> URL: https://issues.apache.org/jira/browse/HBASE-14030
> Project: HBase
> Issue Type: Umbrella
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-14030-v0.patch, HBASE-14030-v1.patch,
> HBASE-14030-v10.patch, HBASE-14030-v11.patch, HBASE-14030-v12.patch,
> HBASE-14030-v13.patch, HBASE-14030-v14.patch, HBASE-14030-v15.patch,
> HBASE-14030-v17.patch, HBASE-14030-v18.patch, HBASE-14030-v2.patch,
> HBASE-14030-v20.patch, HBASE-14030-v21.patch, HBASE-14030-v3.patch,
> HBASE-14030-v4.patch, HBASE-14030-v5.patch, HBASE-14030-v6.patch,
> HBASE-14030-v7.patch, HBASE-14030-v8.patch
>
>
> This is the umbrella ticket for Backup/Restore Phase 1. See HBASE-7912 design
> doc for the phase description.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14976) Add RPC call queues to the web ui
[ https://issues.apache.org/jira/browse/HBASE-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067297#comment-15067297 ] Hudson commented on HBASE-14976: FAILURE: Integrated in HBase-1.2-IT #357 (See [https://builds.apache.org/job/HBase-1.2-IT/357/]) HBASE-14976 fix tests (antonov: rev 99164b132b13efebd415f3ead4b3c416250fa428) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRSStatusServlet.java > Add RPC call queues to the web ui > - > > Key: HBASE-14976 > URL: https://issues.apache.org/jira/browse/HBASE-14976 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Elliott Clark >Assignee: Pallavi Adusumilli >Priority: Minor > Labels: beginner > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14976-v1.patch, HBASE-14976-v2-addendum.patch, > HBASE-14976-v2.patch, HBASE-14976-v2.patch, HBASE-14976.patch > > > The size of the call queue for the regionserver is a critical metric to see > if things are going too slowly. We should add the call queue size to the ui > under the queues tab. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[ https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067300#comment-15067300 ] Vladimir Rodionov commented on HBASE-14977: --- [~saint@gmail.com] Yes. The addendum to HBASE-14977 will fix HBASE-15024. > ChoreService.shutdown may result in ConcurrentModificationException > --- > > Key: HBASE-14977 > URL: https://issues.apache.org/jira/browse/HBASE-14977 > Project: HBase > Issue Type: Bug > Components: util >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4 > > Attachments: HBASE-14977-add.patch, HBASE-14977-v1.patch, > hbase-14977_addendum2.patch, hbase-14977_addendum2.patch > > > As seen in this test: > https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/425/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt > We need to make shutdown method synchronized to avoid this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15022) undefined method `getZooKeeperClusterKey' for Java::OrgApacheHadoopHbaseZookeeper::ZKUtil:Class
[
https://issues.apache.org/jira/browse/HBASE-15022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067298#comment-15067298
]
Hudson commented on HBASE-15022:
FAILURE: Integrated in HBase-1.2-IT #357 (See
[https://builds.apache.org/job/HBase-1.2-IT/357/])
HBASE-15022 replication_admin.rb throws undefined method (matteo.bertozzi: rev
f241dc141c87d98775fadcb12dd9c2c44e809e40)
* hbase-shell/src/main/ruby/hbase/replication_admin.rb
> undefined method `getZooKeeperClusterKey' for
> Java::OrgApacheHadoopHbaseZookeeper::ZKUtil:Class
> ---
>
> Key: HBASE-15022
> URL: https://issues.apache.org/jira/browse/HBASE-15022
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Fix For: 2.0.0, 1.2.0
>
> Attachments: HBASE-15022-v0.patch
>
>
> HBASE-14866 moved a bunch of methods from ZKUtil to ZKConfig
> and the shell was not updated
> {noformat}
> replication_admin.rb: cluster_key =
> ZKUtil.getZooKeeperClusterKey(@configuration)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14976) Add RPC call queues to the web ui
[ https://issues.apache.org/jira/browse/HBASE-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067290#comment-15067290 ] Hudson commented on HBASE-14976: FAILURE: Integrated in HBase-Trunk_matrix #574 (See [https://builds.apache.org/job/HBase-Trunk_matrix/574/]) HBASE-14976 fix tests (antonov: rev f56abf730f45823b2b6dd6d2daa5e33c69f3c86b) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRSStatusServlet.java > Add RPC call queues to the web ui > - > > Key: HBASE-14976 > URL: https://issues.apache.org/jira/browse/HBASE-14976 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Elliott Clark >Assignee: Pallavi Adusumilli >Priority: Minor > Labels: beginner > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14976-v1.patch, HBASE-14976-v2-addendum.patch, > HBASE-14976-v2.patch, HBASE-14976-v2.patch, HBASE-14976.patch > > > The size of the call queue for the regionserver is a critical metric to see > if things are going too slowly. We should add the call queue size to the ui > under the queues tab. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15022) undefined method `getZooKeeperClusterKey' for Java::OrgApacheHadoopHbaseZookeeper::ZKUtil:Class
[
https://issues.apache.org/jira/browse/HBASE-15022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067291#comment-15067291
]
Hudson commented on HBASE-15022:
FAILURE: Integrated in HBase-Trunk_matrix #574 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/574/])
HBASE-15022 replication_admin.rb throws undefined method (matteo.bertozzi: rev
9a297ef0a54de736bad53de3b0b518a9eec1865c)
* hbase-shell/src/main/ruby/hbase/replication_admin.rb
> undefined method `getZooKeeperClusterKey' for
> Java::OrgApacheHadoopHbaseZookeeper::ZKUtil:Class
> ---
>
> Key: HBASE-15022
> URL: https://issues.apache.org/jira/browse/HBASE-15022
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Fix For: 2.0.0, 1.2.0
>
> Attachments: HBASE-15022-v0.patch
>
>
> HBASE-14866 moved a bunch of methods from ZKUtil to ZKConfig
> and the shell was not updated
> {noformat}
> replication_admin.rb: cluster_key =
> ZKUtil.getZooKeeperClusterKey(@configuration)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15021) hadoopqa doing false positives
[
https://issues.apache.org/jira/browse/HBASE-15021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067283#comment-15067283
]
Hadoop QA commented on HBASE-15021:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12778887/15021.thrownpe.patch
against master branch at commit f56abf730f45823b2b6dd6d2daa5e33c69f3c86b.
ATTACHMENT ID: 12778887
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16954//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16954//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16954//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16954//console
This message is automatically generated.
> hadoopqa doing false positives
> --
>
> Key: HBASE-15021
> URL: https://issues.apache.org/jira/browse/HBASE-15021
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Attachments: 15021.thrownpe.patch
>
>
> https://builds.apache.org/job/PreCommit-HBASE-Build/16930/consoleText says:
> {color:green}+1 core tests{color}. The patch passed unit tests in .
> ...but here is what happened:
> {code}
> ...
> Results :
> Tests in error:
> org.apache.hadoop.hbase.regionserver.TestRSStatusServlet.testBasic(org.apache.hadoop.hbase.regionserver.TestRSStatusServlet)
> Run 1: TestRSStatusServlet.testBasic:105 � NullPointer
> Run 2: TestRSStatusServlet.testBasic:105 � NullPointer
> Run 3: TestRSStatusServlet.testBasic:105 � NullPointer
> org.apache.hadoop.hbase.regionserver.TestRSStatusServlet.testWithRegions(org.apache.hadoop.hbase.regionserver.TestRSStatusServlet)
> Run 1: TestRSStatusServlet.testWithRegions:119 � NullPointer
> Run 2: TestRSStatusServlet.testWithRegions:119 � NullPointer
> Run 3: TestRSStatusServlet.testWithRegions:119 � NullPointer
> Tests run: 1033, Failures: 0, Errors: 2, Skipped: 21
> ...
> [INFO] Apache HBase - Server . FAILURE
> [17:54.559s]
> ...
> {code}
> Why we reporting pass when it failed?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[ https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067273#comment-15067273 ] stack commented on HBASE-14977: --- [~vrodionov] Is this causing HBASE-15024? Thanks. > ChoreService.shutdown may result in ConcurrentModificationException > --- > > Key: HBASE-14977 > URL: https://issues.apache.org/jira/browse/HBASE-14977 > Project: HBase > Issue Type: Bug > Components: util >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4 > > Attachments: HBASE-14977-add.patch, HBASE-14977-v1.patch, > hbase-14977_addendum2.patch, hbase-14977_addendum2.patch > > > As seen in this test: > https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/425/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt > We need to make shutdown method synchronized to avoid this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-9310) Remove slop for Stochastic load balancer
[
https://issues.apache.org/jira/browse/HBASE-9310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067272#comment-15067272
]
Bryan Beaudreault commented on HBASE-9310:
--
Are we sure this is actually fixed?
All this patch seems to have done is ensure that the StochasticLoadBalancer
defaults to 0.001F. Looking at the code, this is not enough to guarantee that
slop is disabled.
Here is a log from my cluster, where I have tried a value of 0.001 and smaller,
but it still does not run:
{quote}
2015-12-21 23:38:01,161 TRACE
org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer: Skipping load
balancing because balanced cluster; servers=14 regions=4501 average=321.5
mostloaded=322 leastloaded=321
{quote}
At this point I looked at the code and see:
{code}
float average = cs.getLoadAverage(); // for logging
int floor = (int) Math.floor(average * (1 - slop));
int ceiling = (int) Math.ceil(average * (1 + slop));
if (!(cs.getMaxLoad() > ceiling || cs.getMinLoad() < floor)) {
NavigableMap> serversByLoad =
cs.getServersByLoad();
if (LOG.isTraceEnabled()) {
// If nothing to balance, then don't say anything unless trace-level
logging.
LOG.trace("Skipping load balancing because balanced cluster; " +
"servers=" + cs.getNumServers() +
" regions=" + cs.getNumRegions() + " average=" + average +
" mostloaded=" + serversByLoad.lastKey().getLoad() +
" leastloaded=" + serversByLoad.firstKey().getLoad());
}
return false;
}
return true;
{code}
In my logline case above, ceiling=Math.ceil(321.8215)=322;
floor=Math.floor(321.1785)=321. My maxloaded is 322 and least loaded is 321, so
the balancer exits.
Considering there are other factors at play such as request load, locality, etc
based on cost functions, we should be able to disable the slop check altogether.
Should I create a new JIRA or re-open this one?
> Remove slop for Stochastic load balancer
>
>
> Key: HBASE-9310
> URL: https://issues.apache.org/jira/browse/HBASE-9310
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Affects Versions: 0.98.0, 0.95.2
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 0.98.0, 0.96.0
>
> Attachments: HBASE-9310-0.patch
>
>
> The new load balancer already has the idea of some slop built in. We
> shouldn't have two layers of it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15022) undefined method `getZooKeeperClusterKey' for Java::OrgApacheHadoopHbaseZookeeper::ZKUtil:Class
[
https://issues.apache.org/jira/browse/HBASE-15022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067261#comment-15067261
]
Hadoop QA commented on HBASE-15022:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12778895/HBASE-15022-v0.patch
against master branch at commit f56abf730f45823b2b6dd6d2daa5e33c69f3c86b.
ATTACHMENT ID: 12778895
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16956//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16956//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16956//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16956//console
This message is automatically generated.
> undefined method `getZooKeeperClusterKey' for
> Java::OrgApacheHadoopHbaseZookeeper::ZKUtil:Class
> ---
>
> Key: HBASE-15022
> URL: https://issues.apache.org/jira/browse/HBASE-15022
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Fix For: 2.0.0, 1.2.0
>
> Attachments: HBASE-15022-v0.patch
>
>
> HBASE-14866 moved a bunch of methods from ZKUtil to ZKConfig
> and the shell was not updated
> {noformat}
> replication_admin.rb: cluster_key =
> ZKUtil.getZooKeeperClusterKey(@configuration)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[ https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067247#comment-15067247 ] Vladimir Rodionov commented on HBASE-14977: --- [~enis], can you commit this? > ChoreService.shutdown may result in ConcurrentModificationException > --- > > Key: HBASE-14977 > URL: https://issues.apache.org/jira/browse/HBASE-14977 > Project: HBase > Issue Type: Bug > Components: util >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4 > > Attachments: HBASE-14977-add.patch, HBASE-14977-v1.patch, > hbase-14977_addendum2.patch, hbase-14977_addendum2.patch > > > As seen in this test: > https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/425/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt > We need to make shutdown method synchronized to avoid this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14976) Add RPC call queues to the web ui
[ https://issues.apache.org/jira/browse/HBASE-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067236#comment-15067236 ] Hudson commented on HBASE-14976: SUCCESS: Integrated in HBase-1.3-IT #393 (See [https://builds.apache.org/job/HBase-1.3-IT/393/]) HBASE-14976 fix tests (antonov: rev b82148b705085bf9a81f3502fd074584494a7a87) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRSStatusServlet.java > Add RPC call queues to the web ui > - > > Key: HBASE-14976 > URL: https://issues.apache.org/jira/browse/HBASE-14976 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Elliott Clark >Assignee: Pallavi Adusumilli >Priority: Minor > Labels: beginner > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14976-v1.patch, HBASE-14976-v2-addendum.patch, > HBASE-14976-v2.patch, HBASE-14976-v2.patch, HBASE-14976.patch > > > The size of the call queue for the regionserver is a critical metric to see > if things are going too slowly. We should add the call queue size to the ui > under the queues tab. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15022) undefined method `getZooKeeperClusterKey' for Java::OrgApacheHadoopHbaseZookeeper::ZKUtil:Class
[
https://issues.apache.org/jira/browse/HBASE-15022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067237#comment-15067237
]
Hudson commented on HBASE-15022:
SUCCESS: Integrated in HBase-1.3-IT #393 (See
[https://builds.apache.org/job/HBase-1.3-IT/393/])
HBASE-15022 replication_admin.rb throws undefined method (matteo.bertozzi: rev
85257a25282051bb2fa85ed893e308de6c5c3ff4)
* hbase-shell/src/main/ruby/hbase/replication_admin.rb
> undefined method `getZooKeeperClusterKey' for
> Java::OrgApacheHadoopHbaseZookeeper::ZKUtil:Class
> ---
>
> Key: HBASE-15022
> URL: https://issues.apache.org/jira/browse/HBASE-15022
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Fix For: 2.0.0, 1.2.0
>
> Attachments: HBASE-15022-v0.patch
>
>
> HBASE-14866 moved a bunch of methods from ZKUtil to ZKConfig
> and the shell was not updated
> {noformat}
> replication_admin.rb: cluster_key =
> ZKUtil.getZooKeeperClusterKey(@configuration)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14976) Add RPC call queues to the web ui
[ https://issues.apache.org/jira/browse/HBASE-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067224#comment-15067224 ] Hudson commented on HBASE-14976: FAILURE: Integrated in HBase-1.3 #456 (See [https://builds.apache.org/job/HBase-1.3/456/]) HBASE-14976 fix tests (antonov: rev b82148b705085bf9a81f3502fd074584494a7a87) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRSStatusServlet.java > Add RPC call queues to the web ui > - > > Key: HBASE-14976 > URL: https://issues.apache.org/jira/browse/HBASE-14976 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Elliott Clark >Assignee: Pallavi Adusumilli >Priority: Minor > Labels: beginner > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14976-v1.patch, HBASE-14976-v2-addendum.patch, > HBASE-14976-v2.patch, HBASE-14976-v2.patch, HBASE-14976.patch > > > The size of the call queue for the regionserver is a critical metric to see > if things are going too slowly. We should add the call queue size to the ui > under the queues tab. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14511) StoreFile.Writer Meta Plugin
[ https://issues.apache.org/jira/browse/HBASE-14511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14511: -- Status: In Progress (was: Patch Available) > StoreFile.Writer Meta Plugin > > > Key: HBASE-14511 > URL: https://issues.apache.org/jira/browse/HBASE-14511 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-14511-v3.patch, HBASE-14511.v1.patch, > HBASE-14511.v2.patch > > > During my work on a new compaction policies (HBASE-14468, HBASE-14477) I had > to modify the existing code of a StoreFile.Writer to add additional meta-info > required by these new policies. I think that it should be done by means of a > new Plugin framework, because this seems to be a general capability/feature. > As a future enhancement this can become a part of a more general > StoreFileWriter/Reader plugin architecture. But I need only Meta section of a > store file. > This could be used, for example, to collect rowkeys distribution information > during hfile creation. This info can be used later to find the optimal region > split key or to create optimal set of sub-regions for M/R jobs or other jobs > which can operate on a sub-region level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-15014) Fix filterCellByStore in WALsplitter is awful for performance
[
https://issues.apache.org/jira/browse/HBASE-15014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067192#comment-15067192
]
Elliott Clark edited comment on HBASE-15014 at 12/21/15 10:44 PM:
--
Re-attaching another patch (after the fix for un-related failing tests went in)
If tests are clean this is the version I plan to commit.
was (Author: eclark):
Re-attaching another patch. If tests are clean this is the version I plan to
commit.
> Fix filterCellByStore in WALsplitter is awful for performance
> -
>
> Key: HBASE-15014
> URL: https://issues.apache.org/jira/browse/HBASE-15014
> Project: HBase
> Issue Type: Bug
> Components: MTTR, Recovery, wal
>Affects Versions: 1.2.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-15014-v1.patch, HBASE-15014-v2.patch,
> HBASE-15014-v3.patch, HBASE-15014.patch
>
>
> Testing the latest 1.2 I see this when there is a regionserver that crashes.
> {code}
> Thread 921 (RS_LOG_REPLAY_OPS-hbase2698:16020-0-Writer-1):
> State: RUNNABLE
> Blocked count: 6354
> Waited count: 6249
> Stack:
> org.apache.hadoop.hbase.KeyValue.equals(KeyValue.java:1128)
> java.util.ArrayList.indexOf(ArrayList.java:317)
> java.util.ArrayList.contains(ArrayList.java:300)
> java.util.ArrayList.batchRemove(ArrayList.java:720)
> java.util.ArrayList.removeAll(ArrayList.java:690)
>
> org.apache.hadoop.hbase.wal.WALSplitter$LogRecoveredEditsOutputSink.filterCellByStore(WALSplitter.java:1529)
>
> org.apache.hadoop.hbase.wal.WALSplitter$LogRecoveredEditsOutputSink.append(WALSplitter.java:1557)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.writeBuffer(WALSplitter.java:1113)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.doRun(WALSplitter.java:1105)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.run(WALSplitter.java:1075)
> Thread 920 (RS_LOG_REPLAY_OPS-hbase2698:16020-0-Writer-0):
> State: TIMED_WAITING
> Blocked count: 17560
> Waited count: 19695
> Stack:
> java.lang.Object.wait(Native Method)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.doRun(WALSplitter.java:1093)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.run(WALSplitter.java:1075)
> Thread 919 (RS_LOG_REPLAY_OPS-hbase2698:16020-0):
> State: TIMED_WAITING
> Blocked count: 115
> Waited count: 976
> Stack:
> java.lang.Object.wait(Native Method)
>
> org.apache.hadoop.hbase.wal.WALSplitter$EntryBuffers.appendEntry(WALSplitter.java:944)
> org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:365)
> org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:236)
>
> org.apache.hadoop.hbase.regionserver.SplitLogWorker$1.exec(SplitLogWorker.java:104)
>
> org.apache.hadoop.hbase.regionserver.handler.WALSplitterHandler.process(WALSplitterHandler.java:72)
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> {code}
> This has been going on for >10 mins.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15014) Fix filterCellByStore in WALsplitter is awful for performance
[
https://issues.apache.org/jira/browse/HBASE-15014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Elliott Clark updated HBASE-15014:
--
Attachment: HBASE-15014-v3.patch
Re-attaching another patch. If tests are clean this is the version I plan to
commit.
> Fix filterCellByStore in WALsplitter is awful for performance
> -
>
> Key: HBASE-15014
> URL: https://issues.apache.org/jira/browse/HBASE-15014
> Project: HBase
> Issue Type: Bug
> Components: MTTR, Recovery, wal
>Affects Versions: 1.2.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-15014-v1.patch, HBASE-15014-v2.patch,
> HBASE-15014-v3.patch, HBASE-15014.patch
>
>
> Testing the latest 1.2 I see this when there is a regionserver that crashes.
> {code}
> Thread 921 (RS_LOG_REPLAY_OPS-hbase2698:16020-0-Writer-1):
> State: RUNNABLE
> Blocked count: 6354
> Waited count: 6249
> Stack:
> org.apache.hadoop.hbase.KeyValue.equals(KeyValue.java:1128)
> java.util.ArrayList.indexOf(ArrayList.java:317)
> java.util.ArrayList.contains(ArrayList.java:300)
> java.util.ArrayList.batchRemove(ArrayList.java:720)
> java.util.ArrayList.removeAll(ArrayList.java:690)
>
> org.apache.hadoop.hbase.wal.WALSplitter$LogRecoveredEditsOutputSink.filterCellByStore(WALSplitter.java:1529)
>
> org.apache.hadoop.hbase.wal.WALSplitter$LogRecoveredEditsOutputSink.append(WALSplitter.java:1557)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.writeBuffer(WALSplitter.java:1113)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.doRun(WALSplitter.java:1105)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.run(WALSplitter.java:1075)
> Thread 920 (RS_LOG_REPLAY_OPS-hbase2698:16020-0-Writer-0):
> State: TIMED_WAITING
> Blocked count: 17560
> Waited count: 19695
> Stack:
> java.lang.Object.wait(Native Method)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.doRun(WALSplitter.java:1093)
>
> org.apache.hadoop.hbase.wal.WALSplitter$WriterThread.run(WALSplitter.java:1075)
> Thread 919 (RS_LOG_REPLAY_OPS-hbase2698:16020-0):
> State: TIMED_WAITING
> Blocked count: 115
> Waited count: 976
> Stack:
> java.lang.Object.wait(Native Method)
>
> org.apache.hadoop.hbase.wal.WALSplitter$EntryBuffers.appendEntry(WALSplitter.java:944)
> org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:365)
> org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:236)
>
> org.apache.hadoop.hbase.regionserver.SplitLogWorker$1.exec(SplitLogWorker.java:104)
>
> org.apache.hadoop.hbase.regionserver.handler.WALSplitterHandler.process(WALSplitterHandler.java:72)
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> {code}
> This has been going on for >10 mins.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15024) TestChoreService is flakey
[ https://issues.apache.org/jira/browse/HBASE-15024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067189#comment-15067189 ] Matteo Bertozzi commented on HBASE-15024: - this will be fixed by the addendum in HBASE-14977 > TestChoreService is flakey > -- > > Key: HBASE-15024 > URL: https://issues.apache.org/jira/browse/HBASE-15024 > Project: HBase > Issue Type: Sub-task > Components: flakey, test >Reporter: stack > > https://builds.apache.org/job/HBase-Trunk_matrix/jdk=latest1.8,label=Hadoop/lastCompletedBuild/testReport/org.apache.hadoop.hbase/TestChoreService/testShutdownRejectsNewSchedules/history/ > Fails a bunch lately -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-1015) HBase Native Client Library
[ https://issues.apache.org/jira/browse/HBASE-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067183#comment-15067183 ] Elliott Clark commented on HBASE-1015: -- I would think so yeah. We're moving to a newer build system, more c++ before the C layer. Moving to wangle/folly vs a C library for networking. etc. So anything here is pretty out of date with the current plan. > HBase Native Client Library > --- > > Key: HBASE-1015 > URL: https://issues.apache.org/jira/browse/HBASE-1015 > Project: HBase > Issue Type: New Feature > Components: Client >Affects Versions: 0.20.6, 1.0.0 >Reporter: Andrew Purtell >Assignee: Aditya Kishore >Priority: Minor > Attachments: HBASE-1015-HBase-native-client.patch, > HBASE-1015-HBase-native-client.patch > > > If via HBASE-794 first class support for talking via Thrift directly to > HMaster and HRS is available, then pure C and C++ client libraries are > possible. > The C client library would wrap a Thrift core. > The C++ client library can provide a class hierarchy quite close to > o.a.h.h.client and, ideally, identical semantics. It should be just a > wrapper around the C API, for economy. > Internally to my employer there is a lot of resistance to HBase because many > dev teams have a strong C/C++ bias. The real issue however is really client > side integration, not a fundamental objection. (What runs server side and how > it is managed is a secondary consideration.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15024) TestChoreService is flakey
stack created HBASE-15024: - Summary: TestChoreService is flakey Key: HBASE-15024 URL: https://issues.apache.org/jira/browse/HBASE-15024 Project: HBase Issue Type: Sub-task Components: flakey, test Reporter: stack https://builds.apache.org/job/HBase-Trunk_matrix/jdk=latest1.8,label=Hadoop/lastCompletedBuild/testReport/org.apache.hadoop.hbase/TestChoreService/testShutdownRejectsNewSchedules/history/ Fails a bunch lately -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-1015) HBase Native Client Library
[ https://issues.apache.org/jira/browse/HBASE-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067179#comment-15067179 ] Mikhail Antonov commented on HBASE-1015: (So this one and all its subtasks shall be closed? ) > HBase Native Client Library > --- > > Key: HBASE-1015 > URL: https://issues.apache.org/jira/browse/HBASE-1015 > Project: HBase > Issue Type: New Feature > Components: Client >Affects Versions: 0.20.6, 1.0.0 >Reporter: Andrew Purtell >Assignee: Aditya Kishore >Priority: Minor > Attachments: HBASE-1015-HBase-native-client.patch, > HBASE-1015-HBase-native-client.patch > > > If via HBASE-794 first class support for talking via Thrift directly to > HMaster and HRS is available, then pure C and C++ client libraries are > possible. > The C client library would wrap a Thrift core. > The C++ client library can provide a class hierarchy quite close to > o.a.h.h.client and, ideally, identical semantics. It should be just a > wrapper around the C API, for economy. > Internally to my employer there is a lot of resistance to HBase because many > dev teams have a strong C/C++ bias. The real issue however is really client > side integration, not a fundamental objection. (What runs server side and how > it is managed is a secondary consideration.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15023) Reenable TestShell and TestStochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-15023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15023: -- Assignee: stack Status: Patch Available (was: Open) > Reenable TestShell and TestStochasticLoadBalancer > - > > Key: HBASE-15023 > URL: https://issues.apache.org/jira/browse/HBASE-15023 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: stack > Fix For: 2.0.0, 1.3.0, 1.2.1 > > Attachments: 15023.patch > > > Parent disabled these tests when test runs were flakier than they are now. > Try reenabling them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
