[jira] [Commented] (HBASE-14830) Fix broken links in 0.94 generated docs
[
https://issues.apache.org/jira/browse/HBASE-14830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059624#comment-15059624
]
Hudson commented on HBASE-14830:
SUCCESS: Integrated in HBase-0.94 #1478 (See
[https://builds.apache.org/job/HBase-0.94/1478/])
HBASE-14830 Make encoding classes @InterfaceAudience.Private (mstanleyjones:
rev 0f35a32ab123ee299f4aaaea02b4ba2d2b43cff2)
* src/main/java/org/apache/hadoop/hbase/io/encoding/PrefixKeyDeltaEncoder.java
* src/main/java/org/apache/hadoop/hbase/io/encoding/DiffKeyDeltaEncoder.java
* src/main/java/org/apache/hadoop/hbase/io/encoding/CopyKeyDataBlockEncoder.java
* src/main/java/org/apache/hadoop/hbase/io/encoding/FastDiffDeltaEncoder.java
> Fix broken links in 0.94 generated docs
> ---
>
> Key: HBASE-14830
> URL: https://issues.apache.org/jira/browse/HBASE-14830
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 0.94.27
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Fix For: 0.94.28
>
> Attachments: HBASE-14830-v1.patch, HBASE-14830.patch
>
>
> {code}
> host: hbase.apache.org
> date: Mon, 16-Nov-2015 10:56:28 (local)
> Linklint version: 2.3.5_ingo_020
> #
> # ERROR 2 missing html files (cross referenced)
> #
> /0.94/devapidocs/org/apache/hadoop/hbase/avro/generated/HBase.html
> used in 1 file:
> /0.94/devapidocs/org/apache/hadoop/hbase/avro/package-summary.html
> /0.94/devapidocs/src-html/org/apache/hadoop/hbase/io/encoding/BufferedDataBlockEncoder.html
> used in 4 files:
>
> /0.94/devapidocs/org/apache/hadoop/hbase/io/encoding/CopyKeyDataBlockEncoder.html
>
> /0.94/devapidocs/org/apache/hadoop/hbase/io/encoding/DiffKeyDeltaEncoder.html
>
> /0.94/devapidocs/org/apache/hadoop/hbase/io/encoding/FastDiffDeltaEncoder.html
>
> /0.94/devapidocs/org/apache/hadoop/hbase/io/encoding/PrefixKeyDeltaEncoder.html
> #
> # ERROR 1 missing other file (cross referenced)
> #
> /0.94/devapidocs/org/apache/hadoop/hbase/HADOOP-1581
> used in 1 file:
> /0.94/devapidocs/org/apache/hadoop/hbase/HTableDescriptor.html
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059609#comment-15059609 ] Francis Liu commented on HBASE-6721: Looks like TestShell has been temporarily removed as part of test stabilization experiment in HBASE-14678. Given how it looks like when it was still there: https://github.com/apache/hbase/commit/f670649f0e2cfba8a2112eb5c1d79b8b7f52c3b2 All I need to do is include the group rb file in the TestShell class. So we can either apply the TestShell change once it's been re-enabled. Or commit the group shell unit tests as a follow-up once TestShell has been re-enabled. Thoughts? > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Francis Liu > Labels: hbase-6721 > Attachments: 6721-master-webUI.patch, HBASE-6721 > GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch, > HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch, > HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch, > HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, > HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, > HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg, > hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch, > hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch, > immediateAssignments Sequence Diagram.svg, randomAssignment Sequence > Diagram.svg, retainAssignment Sequence Diagram.svg, roundRobinAssignment > Sequence Diagram.svg > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-14981) Shell unit tests are not running in trunk?
[ https://issues.apache.org/jira/browse/HBASE-14981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu resolved HBASE-14981. - Resolution: Invalid > Shell unit tests are not running in trunk? > -- > > Key: HBASE-14981 > URL: https://issues.apache.org/jira/browse/HBASE-14981 > Project: HBase > Issue Type: Bug >Reporter: Francis Liu >Assignee: Francis Liu > > TestShell seems to be replaced by an AbstractTestShell class. Thus it does > not seem that any of the shell unit tests are running? If so, is the intent > to create a TestShell which subclasses the abstract class? > On a related note I've written shell unit tests for HBASE-6721 and it > requires a different Shell subclass as I need to have a different balancer > and coprocessor installed. I'm hoping we can address that use case in this > jira as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14981) Shell unit tests are not running in trunk?
[ https://issues.apache.org/jira/browse/HBASE-14981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059605#comment-15059605 ] Francis Liu commented on HBASE-14981: - Looks like this one of the tests that's part of the experiment being done in HBASE-14678 > Shell unit tests are not running in trunk? > -- > > Key: HBASE-14981 > URL: https://issues.apache.org/jira/browse/HBASE-14981 > Project: HBase > Issue Type: Bug >Reporter: Francis Liu >Assignee: Francis Liu > > TestShell seems to be replaced by an AbstractTestShell class. Thus it does > not seem that any of the shell unit tests are running? If so, is the intent > to create a TestShell which subclasses the abstract class? > On a related note I've written shell unit tests for HBASE-6721 and it > requires a different Shell subclass as I need to have a different balancer > and coprocessor installed. I'm hoping we can address that use case in this > jira as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14460) [Perf Regression] Merge of MVCC and SequenceId (HBASE-HBASE-8763) slowed Increments, CheckAndPuts, batch operations
[ https://issues.apache.org/jira/browse/HBASE-14460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059596#comment-15059596 ] stack commented on HBASE-14460: --- First, now that I understand, I like your idea. We should do it. Lets work on your patch. Will give some more feedback tomorrow (need to consider your response on rowOperationContexts... try it). We need more speed up though. Looks like your change could get us from 10x to 3 or 4x what it used be which is close. Trying to figure how to get it closer in. One unformed thought is a means of exploiting the context you've uncovered and somehow using it to do aggregations of counts that are stuck waiting on a rowlock and or mvcc; i.e. if a handler arrives and we are waiting on mvcc and/or row lock, add our increment elsewhere, park until the increment has been applied and then skip out on doing the increment path ourselves. Probably not possible without compounding complexity and synchronization points so could slow more than it speeds us up. bq. ...but this cell is not yet visible? Is it ok? We are out past the sync of the write at this point and while the increment is not yet generally available, as soon as someone needs to read it, they'll be forced to wait till the mvcc catches up. Given this, I think we might be able to get away with this if we narrow the options around when Increments become 'visiible'. We need another 2x to get us within shouting distance of how it used all work in 0.98/0.94 timeframe. Some folks are late upgrading and they've built their apps based on a fast Increment. They are kinda hosed w/o it. > [Perf Regression] Merge of MVCC and SequenceId (HBASE-HBASE-8763) slowed > Increments, CheckAndPuts, batch operations > --- > > Key: HBASE-14460 > URL: https://issues.apache.org/jira/browse/HBASE-14460 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: stack >Assignee: stack >Priority: Critical > Attachments: 0.94.test.patch, 0.98.test.patch, > 1.0.80.flamegraph-7932.svg, 14460.txt, 98.80.flamegraph-11428.svg, > HBASE-14460-discussion.patch, client.test.patch, > flamegraph-13120.svg.master.singlecell.svg, flamegraph-26636.094.100.svg, > flamegraph-28066.098.singlecell.svg, flamegraph-28767.098.100.svg, > flamegraph-31647.master.100.svg, flamegraph-9466.094.singlecell.svg, > m.test.patch, region_lock.png, testincrement.094.patch, > testincrement.098.patch, testincrement.master.patch > > > As reported by 鈴木俊裕 up on the mailing list -- see "Performance degradation > between CDH5.3.1(HBase0.98.6) and CDH5.4.5(HBase1.0.0)" -- our unification of > sequenceid and MVCC slows Increments (and other ops) as the mvcc needs to > 'catch up' to our current point before we can read the last Increment value > that we need to update. > We can say that our Increment is just done wrong, we should just be writing > Increments and summing on read, but checkAndPut as well as batching > operations have the same issue. Fix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14990) Tests in BaseTestHBaseFsck are run by its subclasses redundantly
[
https://issues.apache.org/jira/browse/HBASE-14990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen updated HBASE-14990:
--
Attachment: HBASE-14990_v1.patch
> Tests in BaseTestHBaseFsck are run by its subclasses redundantly
>
>
> Key: HBASE-14990
> URL: https://issues.apache.org/jira/browse/HBASE-14990
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Ted Yu
>Assignee: Heng Chen
>Priority: Minor
> Attachments: HBASE-14990.patch, HBASE-14990_v1.patch
>
>
> Several tests extend BaseTestHBaseFsck :
> public class TestHBaseFsckMOB extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckMOB.java
> public class TestHBaseFsckOneRS extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckOneRS.java
> public class TestHBaseFsckReplicas extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckReplicas.java
> public class TestHBaseFsckTwoRS extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckTwoRS.java
> BaseTestHBaseFsck contains several tests, e.g. testHDFSRegioninfoMissing
> This means that the tests in BaseTestHBaseFsck would be run multiple times in
> the test suite.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059555#comment-15059555 ] Zhan Zhang commented on HBASE-14849: I use following command, but didn't find any javadoc warning. Will fix other issue after gathering review comments. mvn clean package javadoc:javadoc -DskipTests -DHBasePatchProcess > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > Attachments: HBASE-14849.patch > > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14990) Tests in BaseTestHBaseFsck are run by its subclasses redundantly
[
https://issues.apache.org/jira/browse/HBASE-14990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059547#comment-15059547
]
Heng Chen commented on HBASE-14990:
---
{quote}
Should the sub tests be moved into TestHBaseFsckTwoRS ?
{quote}
Sounds reasonable. I will do it.
> Tests in BaseTestHBaseFsck are run by its subclasses redundantly
>
>
> Key: HBASE-14990
> URL: https://issues.apache.org/jira/browse/HBASE-14990
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Ted Yu
>Assignee: Heng Chen
>Priority: Minor
> Attachments: HBASE-14990.patch
>
>
> Several tests extend BaseTestHBaseFsck :
> public class TestHBaseFsckMOB extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckMOB.java
> public class TestHBaseFsckOneRS extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckOneRS.java
> public class TestHBaseFsckReplicas extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckReplicas.java
> public class TestHBaseFsckTwoRS extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckTwoRS.java
> BaseTestHBaseFsck contains several tests, e.g. testHDFSRegioninfoMissing
> This means that the tests in BaseTestHBaseFsck would be run multiple times in
> the test suite.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-6721: --- Attachment: hbase-6721-v20.patch addressed failing non-shell unit tests > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Francis Liu > Labels: hbase-6721 > Attachments: 6721-master-webUI.patch, HBASE-6721 > GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch, > HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch, > HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch, > HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, > HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, > HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg, > hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch, > hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch, > immediateAssignments Sequence Diagram.svg, randomAssignment Sequence > Diagram.svg, retainAssignment Sequence Diagram.svg, roundRobinAssignment > Sequence Diagram.svg > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14991) Fix the feature warning in scala code
[ https://issues.apache.org/jira/browse/HBASE-14991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059534#comment-15059534 ] Jonathan Hsieh commented on HBASE-14991: Thanks! I had just found the args thing and was running a build. Anyway, I will wait for build bot and check the output. before I commit. > Fix the feature warning in scala code > - > > Key: HBASE-14991 > URL: https://issues.apache.org/jira/browse/HBASE-14991 > Project: HBase > Issue Type: Bug >Reporter: Zhan Zhang >Assignee: Zhan Zhang >Priority: Minor > Attachments: HBASE-14991.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14795) Enhance the spark-hbase scan operations
[ https://issues.apache.org/jira/browse/HBASE-14795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059533#comment-15059533 ] Zhan Zhang commented on HBASE-14795: [~jmhsieh] HBASE-14991 is opened for this, and the patch is submitted. Thanks. > Enhance the spark-hbase scan operations > --- > > Key: HBASE-14795 > URL: https://issues.apache.org/jira/browse/HBASE-14795 > Project: HBase > Issue Type: Improvement >Reporter: Ted Malaska >Assignee: Zhan Zhang >Priority: Minor > Fix For: 2.0.0 > > Attachments: > 0001-HBASE-14795-Enhance-the-spark-hbase-scan-operations.patch, > HBASE-14795-1.patch, HBASE-14795-2.patch, HBASE-14795-3.patch, > HBASE-14795-4.patch > > > This is a sub-jira of HBASE-14789. This jira is to focus on the replacement > of TableInputFormat for a more custom scan implementation that will make the > following use case more effective. > Use case: > In the case you have multiple scan ranges on a single table with in a single > query. TableInputFormat will scan the the outer range of the scan start and > end range where this implementation can be more pointed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14991) Fix the feature warning in scala code
[ https://issues.apache.org/jira/browse/HBASE-14991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-14991: --- Status: Patch Available (was: Open) > Fix the feature warning in scala code > - > > Key: HBASE-14991 > URL: https://issues.apache.org/jira/browse/HBASE-14991 > Project: HBase > Issue Type: Bug >Reporter: Zhan Zhang >Assignee: Zhan Zhang >Priority: Minor > Attachments: HBASE-14991.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14991) Fix the feature warning in scala code
[ https://issues.apache.org/jira/browse/HBASE-14991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059531#comment-15059531 ] Zhan Zhang commented on HBASE-14991: Enable feature option and fix feature warning. > Fix the feature warning in scala code > - > > Key: HBASE-14991 > URL: https://issues.apache.org/jira/browse/HBASE-14991 > Project: HBase > Issue Type: Bug >Reporter: Zhan Zhang >Assignee: Zhan Zhang >Priority: Minor > Attachments: HBASE-14991.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14991) Fix the feature warning in scala code
[ https://issues.apache.org/jira/browse/HBASE-14991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059532#comment-15059532 ] Zhan Zhang commented on HBASE-14991: @Jonathan Hsieh Please review. > Fix the feature warning in scala code > - > > Key: HBASE-14991 > URL: https://issues.apache.org/jira/browse/HBASE-14991 > Project: HBase > Issue Type: Bug >Reporter: Zhan Zhang >Assignee: Zhan Zhang >Priority: Minor > Attachments: HBASE-14991.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14991) Fix the feature warning in scala code
[ https://issues.apache.org/jira/browse/HBASE-14991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-14991: --- Attachment: HBASE-14991.patch > Fix the feature warning in scala code > - > > Key: HBASE-14991 > URL: https://issues.apache.org/jira/browse/HBASE-14991 > Project: HBase > Issue Type: Bug >Reporter: Zhan Zhang >Assignee: Zhan Zhang >Priority: Minor > Attachments: HBASE-14991.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14991) Fix the feature warning in scala code
Zhan Zhang created HBASE-14991: -- Summary: Fix the feature warning in scala code Key: HBASE-14991 URL: https://issues.apache.org/jira/browse/HBASE-14991 Project: HBase Issue Type: Bug Reporter: Zhan Zhang Assignee: Zhan Zhang Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14990) Tests in BaseTestHBaseFsck are run by its subclasses redundantly
[
https://issues.apache.org/jira/browse/HBASE-14990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059516#comment-15059516
]
Ted Yu commented on HBASE-14990:
Currently TestHBaseFsckTwoRS takes ~150 sec and TestHBaseFsckOneRS takes ~379
sec (see
https://builds.apache.org/job/HBase-TRUNK_matrix/jdk=latest1.7,label=Hadoop/556/console)
Should the sub tests be moved into TestHBaseFsckTwoRS ?
For TestHBaseFsckTwoRS, it would still be 150 seconds. While the other hbck
tests would run quicker.
> Tests in BaseTestHBaseFsck are run by its subclasses redundantly
>
>
> Key: HBASE-14990
> URL: https://issues.apache.org/jira/browse/HBASE-14990
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Ted Yu
>Assignee: Heng Chen
>Priority: Minor
> Attachments: HBASE-14990.patch
>
>
> Several tests extend BaseTestHBaseFsck :
> public class TestHBaseFsckMOB extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckMOB.java
> public class TestHBaseFsckOneRS extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckOneRS.java
> public class TestHBaseFsckReplicas extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckReplicas.java
> public class TestHBaseFsckTwoRS extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckTwoRS.java
> BaseTestHBaseFsck contains several tests, e.g. testHDFSRegioninfoMissing
> This means that the tests in BaseTestHBaseFsck would be run multiple times in
> the test suite.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14795) Enhance the spark-hbase scan operations
[ https://issues.apache.org/jira/browse/HBASE-14795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059512#comment-15059512 ] Jonathan Hsieh commented on HBASE-14795: [~zhanzhang] about the build warnings, that's fine. My suggestion is to leave this be now and create a new jira to fix this issue and link here. Ideally we should not allow the commit to go in unless both the bot's javadoc and javac -1's were both addressed. > Enhance the spark-hbase scan operations > --- > > Key: HBASE-14795 > URL: https://issues.apache.org/jira/browse/HBASE-14795 > Project: HBase > Issue Type: Improvement >Reporter: Ted Malaska >Assignee: Zhan Zhang >Priority: Minor > Fix For: 2.0.0 > > Attachments: > 0001-HBASE-14795-Enhance-the-spark-hbase-scan-operations.patch, > HBASE-14795-1.patch, HBASE-14795-2.patch, HBASE-14795-3.patch, > HBASE-14795-4.patch > > > This is a sub-jira of HBASE-14789. This jira is to focus on the replacement > of TableInputFormat for a more custom scan implementation that will make the > following use case more effective. > Use case: > In the case you have multiple scan ranges on a single table with in a single > query. TableInputFormat will scan the the outer range of the scan start and > end range where this implementation can be more pointed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14956) [HBase ZKcli] JLine support is disabled. Better to enable this in HBase.
[
https://issues.apache.org/jira/browse/HBASE-14956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059513#comment-15059513
]
Esteban Gutierrez commented on HBASE-14956:
---
That approach will cause issues when we bump JRuby to a newer version and
altering the classpath in that way it will make difficult to keep things
consistent. As I mentioned, this was caused by ZOOKEEPER-1718 and looking at
master and branch-1 we still require ZooKeeper 3.4.6 so it shouldn't be a
problem unless you are using a different version of ZK in your deployment.
> [HBase ZKcli] JLine support is disabled. Better to enable this in HBase.
> -
>
> Key: HBASE-14956
> URL: https://issues.apache.org/jira/browse/HBASE-14956
> Project: HBase
> Issue Type: Bug
> Components: Client, Zookeeper
>Affects Versions: 1.0.2
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>
> To perform the zkcli operations using hbase,
> jline is disabled,
> {code}
> [ERROR] Terminal initialization failed; falling back to unsupported
> java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but
> interface was expected
> at jline.TerminalFactory.create(TerminalFactory.java:101)
> at jline.TerminalFactory.get(TerminalFactory.java:158)
> at jline.console.ConsoleReader.(ConsoleReader.java:229)
> at jline.console.ConsoleReader.(ConsoleReader.java:221)
> at jline.console.ConsoleReader.(ConsoleReader.java:209)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
> at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:335)
> at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:303)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperMainServer.main(ZooKeeperMainServer.java:108)
> JLine support is disabled
> {code}
> To enable this jline-.jar is needed in hbase libraries.
> eg: jline-2.11.jar should be exist in hbase/lib directory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14795) Enhance the spark-hbase scan operations
[ https://issues.apache.org/jira/browse/HBASE-14795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059509#comment-15059509 ] Zhan Zhang commented on HBASE-14795: [~jmhsieh] I am trying to figure out how to "re-run with -feature for details". Do you know how to do it? > Enhance the spark-hbase scan operations > --- > > Key: HBASE-14795 > URL: https://issues.apache.org/jira/browse/HBASE-14795 > Project: HBase > Issue Type: Improvement >Reporter: Ted Malaska >Assignee: Zhan Zhang >Priority: Minor > Fix For: 2.0.0 > > Attachments: > 0001-HBASE-14795-Enhance-the-spark-hbase-scan-operations.patch, > HBASE-14795-1.patch, HBASE-14795-2.patch, HBASE-14795-3.patch, > HBASE-14795-4.patch > > > This is a sub-jira of HBASE-14789. This jira is to focus on the replacement > of TableInputFormat for a more custom scan implementation that will make the > following use case more effective. > Use case: > In the case you have multiple scan ranges on a single table with in a single > query. TableInputFormat will scan the the outer range of the scan start and > end range where this implementation can be more pointed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[ https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059505#comment-15059505 ] Hudson commented on HBASE-14977: FAILURE: Integrated in HBase-1.3 #439 (See [https://builds.apache.org/job/HBase-1.3/439/]) HBASE-14977 ChoreService.shutdown may result in (enis: rev 31d73a4bdec59ba2ca6bdc04d881f2bc3726e903) * hbase-common/src/main/java/org/apache/hadoop/hbase/ChoreService.java > ChoreService.shutdown may result in ConcurrentModificationException > --- > > Key: HBASE-14977 > URL: https://issues.apache.org/jira/browse/HBASE-14977 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4 > > Attachments: HBASE-14977-v1.patch > > > As seen in this test: > https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/425/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt > We need to make shutdown method synchronized to avoid this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14947) WALProcedureStore improvements
[ https://issues.apache.org/jira/browse/HBASE-14947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059506#comment-15059506 ] Hudson commented on HBASE-14947: FAILURE: Integrated in HBase-1.3 #439 (See [https://builds.apache.org/job/HBase-1.3/439/]) HBASE-14947 WALProcedureStore improvements (matteo.bertozzi: rev bf7c36fccac5477d08e296ad93671d2c30ae2dc8) * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureRecovery.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/ProcedureStoreTracker.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/wal/TestWALProcedureStore.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/TestProcedureStoreTracker.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java > WALProcedureStore improvements > -- > > Key: HBASE-14947 > URL: https://issues.apache.org/jira/browse/HBASE-14947 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Ashu Pachauri >Assignee: Matteo Bertozzi >Priority: Blocker > Fix For: 2.0.0, 1.2.0, 1.1.3 > > Attachments: HBASE-14947-v0.patch, HBASE-14947-v1.patch > > > We ended up with a deadlock in HBASE-14943, with the storeTracker and lock > acquired in reverse order by syncLoop() and insert/update/delete. In the > syncLoop() with don't need the lock when we try to roll or removeInactive. > also we can move the insert/update/delete tracker check in the syncLoop > avoiding to the extra lock operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14795) Enhance the spark-hbase scan operations
[ https://issues.apache.org/jira/browse/HBASE-14795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059504#comment-15059504 ] Jonathan Hsieh commented on HBASE-14795: [~zhanzhang] help me out here -- where did the warnings you included come from? was that by adding the -feature flag somehow? is that what the compiler warning turns into? I got the ones I quoted from here[1] and compared against the precommit build of the patch committed right this one here [2]. Looking at the precommit build for HBASE-14849, the warning quote above is still there [3] [1] https://builds.apache.org/job/PreCommit-HBASE-Build/16833//artifact/patchprocess/patchJavadocWarnings.txt [2] https://builds.apache.org/job/PreCommit-HBASE-Build/16839/artifact/patchprocess/patchJavadocWarnings.txt [3] https://builds.apache.org/job/PreCommit-HBASE-Build/16870//artifact/patchprocess/patchJavadocWarnings.txt > Enhance the spark-hbase scan operations > --- > > Key: HBASE-14795 > URL: https://issues.apache.org/jira/browse/HBASE-14795 > Project: HBase > Issue Type: Improvement >Reporter: Ted Malaska >Assignee: Zhan Zhang >Priority: Minor > Fix For: 2.0.0 > > Attachments: > 0001-HBASE-14795-Enhance-the-spark-hbase-scan-operations.patch, > HBASE-14795-1.patch, HBASE-14795-2.patch, HBASE-14795-3.patch, > HBASE-14795-4.patch > > > This is a sub-jira of HBASE-14789. This jira is to focus on the replacement > of TableInputFormat for a more custom scan implementation that will make the > following use case more effective. > Use case: > In the case you have multiple scan ranges on a single table with in a single > query. TableInputFormat will scan the the outer range of the scan start and > end range where this implementation can be more pointed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14795) Enhance the spark-hbase scan operations
[ https://issues.apache.org/jira/browse/HBASE-14795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059502#comment-15059502 ] Zhan Zhang commented on HBASE-14795: My mistake. These warning are different from those three "feature warning", but don't know what build option to enable to get the details of these warnings. > Enhance the spark-hbase scan operations > --- > > Key: HBASE-14795 > URL: https://issues.apache.org/jira/browse/HBASE-14795 > Project: HBase > Issue Type: Improvement >Reporter: Ted Malaska >Assignee: Zhan Zhang >Priority: Minor > Fix For: 2.0.0 > > Attachments: > 0001-HBASE-14795-Enhance-the-spark-hbase-scan-operations.patch, > HBASE-14795-1.patch, HBASE-14795-2.patch, HBASE-14795-3.patch, > HBASE-14795-4.patch > > > This is a sub-jira of HBASE-14789. This jira is to focus on the replacement > of TableInputFormat for a more custom scan implementation that will make the > following use case more effective. > Use case: > In the case you have multiple scan ranges on a single table with in a single > query. TableInputFormat will scan the the outer range of the scan start and > end range where this implementation can be more pointed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-14990) Tests in BaseTestHBaseFsck are run by its subclasses redundantly
[
https://issues.apache.org/jira/browse/HBASE-14990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh reassigned HBASE-14990:
--
Assignee: Heng Chen
Lgtm. Will commit if bot comes back clean.
> Tests in BaseTestHBaseFsck are run by its subclasses redundantly
>
>
> Key: HBASE-14990
> URL: https://issues.apache.org/jira/browse/HBASE-14990
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Ted Yu
>Assignee: Heng Chen
>Priority: Minor
> Attachments: HBASE-14990.patch
>
>
> Several tests extend BaseTestHBaseFsck :
> public class TestHBaseFsckMOB extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckMOB.java
> public class TestHBaseFsckOneRS extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckOneRS.java
> public class TestHBaseFsckReplicas extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckReplicas.java
> public class TestHBaseFsckTwoRS extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckTwoRS.java
> BaseTestHBaseFsck contains several tests, e.g. testHDFSRegioninfoMissing
> This means that the tests in BaseTestHBaseFsck would be run multiple times in
> the test suite.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14795) Enhance the spark-hbase scan operations
[ https://issues.apache.org/jira/browse/HBASE-14795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059498#comment-15059498 ] Zhan Zhang commented on HBASE-14795: [~jmhsieh] My patch in HBASE-14849 will not fix those three warning. But I checked again and think these warning are not generated by the patch of this JIRA, because the patch does not touch any pom.xml file but the warning is actually scoping issue in pom.xml. Correct me if I am wrong. Please following for details. [WARNING] Artifact org.apache.spark:spark-core_2.10:jar:1.3.0:provided retains local artifactScope 'provided' overriding broader artifactScope 'compile' given by a dependency. If this is not intended, modify or remove the local artifactScope. [WARNING] Artifact org.scala-lang:scala-library:jar:2.10.4:provided retains local artifactScope 'provided' overriding broader artifactScope 'compile' given by a dependency. If this is not intended, modify or remove the local artifactScope. [WARNING] Artifact junit:junit:jar:4.12:test retains local artifactScope 'test' overriding broader artifactScope 'compile' given by a dependency. If this is not intended, modify or remove the local artifactScope. > Enhance the spark-hbase scan operations > --- > > Key: HBASE-14795 > URL: https://issues.apache.org/jira/browse/HBASE-14795 > Project: HBase > Issue Type: Improvement >Reporter: Ted Malaska >Assignee: Zhan Zhang >Priority: Minor > Fix For: 2.0.0 > > Attachments: > 0001-HBASE-14795-Enhance-the-spark-hbase-scan-operations.patch, > HBASE-14795-1.patch, HBASE-14795-2.patch, HBASE-14795-3.patch, > HBASE-14795-4.patch > > > This is a sub-jira of HBASE-14789. This jira is to focus on the replacement > of TableInputFormat for a more custom scan implementation that will make the > following use case more effective. > Use case: > In the case you have multiple scan ranges on a single table with in a single > query. TableInputFormat will scan the the outer range of the scan start and > end range where this implementation can be more pointed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14990) Tests in BaseTestHBaseFsck are run by its subclasses redundantly
[
https://issues.apache.org/jira/browse/HBASE-14990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen updated HBASE-14990:
--
Attachment: HBASE-14990.patch
Upload a simple patch. :)
> Tests in BaseTestHBaseFsck are run by its subclasses redundantly
>
>
> Key: HBASE-14990
> URL: https://issues.apache.org/jira/browse/HBASE-14990
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Ted Yu
>Priority: Minor
> Attachments: HBASE-14990.patch
>
>
> Several tests extend BaseTestHBaseFsck :
> public class TestHBaseFsckMOB extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckMOB.java
> public class TestHBaseFsckOneRS extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckOneRS.java
> public class TestHBaseFsckReplicas extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckReplicas.java
> public class TestHBaseFsckTwoRS extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckTwoRS.java
> BaseTestHBaseFsck contains several tests, e.g. testHDFSRegioninfoMissing
> This means that the tests in BaseTestHBaseFsck would be run multiple times in
> the test suite.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-13627) Terminating RS results in redundant CLOSE RPC
[
https://issues.apache.org/jira/browse/HBASE-13627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sreeram Venkatasubramanian reassigned HBASE-13627:
--
Assignee: Sreeram Venkatasubramanian
> Terminating RS results in redundant CLOSE RPC
> -
>
> Key: HBASE-13627
> URL: https://issues.apache.org/jira/browse/HBASE-13627
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 1.1.0
>Reporter: Nick Dimiduk
>Assignee: Sreeram Venkatasubramanian
>Priority: Minor
> Labels: beginner, beginners
> Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.4
>
>
> Noticed while testing the 1.1.0RC0 bits. It seems we're issuing a redundant
> close RPC during shutdown. This results in a logging warning for each region.
> {noformat}
> 2015-05-06 00:07:19,214 INFO [RS:0;ndimiduk-apache-1-1-dist-6:56371]
> regionserver.HRegionServer: Received CLOSE for the region:
> 19cbe4fe2fe5335e7aace05e10e36ede, which we are already trying to CLOSE, but
> not completed yet
> 2015-05-06 00:07:19,214 WARN [RS:0;ndimiduk-apache-1-1-dist-6:56371]
> regionserver.HRegionServer: Failed to close
> cluster_test,,1430869443384.19cbe4fe2fe5335e7aace05e10e36ede. -
> ignoring and continuing
> org.apache.hadoop.hbase.regionserver.RegionAlreadyInTransitionException: The
> region 19cbe4fe2fe5335e7aace05e10e36ede was already closing. New CLOSE
> request is ignored.
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.closeRegion(HRegionServer.java:2769)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.closeRegionIgnoreErrors(HRegionServer.java:2695)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.closeUserRegions(HRegionServer.java:2327)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:937)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> 1. launch a standalone cluster from tgz (./bin/start-hbase.sh)
> 2. load some data (ie, run bin/hbase ltt)
> 3. terminate cluster (./bin/stop-hbase.sh)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14990) Tests in BaseTestHBaseFsck are run by its subclasses redundantly
[
https://issues.apache.org/jira/browse/HBASE-14990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059486#comment-15059486
]
Heng Chen commented on HBASE-14990:
---
Maybe we could move this tests from BaseTestHBaseFsck into another Test case.
> Tests in BaseTestHBaseFsck are run by its subclasses redundantly
>
>
> Key: HBASE-14990
> URL: https://issues.apache.org/jira/browse/HBASE-14990
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Ted Yu
>Priority: Minor
>
> Several tests extend BaseTestHBaseFsck :
> public class TestHBaseFsckMOB extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckMOB.java
> public class TestHBaseFsckOneRS extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckOneRS.java
> public class TestHBaseFsckReplicas extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckReplicas.java
> public class TestHBaseFsckTwoRS extends BaseTestHBaseFsck {
> hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckTwoRS.java
> BaseTestHBaseFsck contains several tests, e.g. testHDFSRegioninfoMissing
> This means that the tests in BaseTestHBaseFsck would be run multiple times in
> the test suite.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[
https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059468#comment-15059468
]
Hadoop QA commented on HBASE-14849:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12777865/HBASE-14849.patch
against master branch at commit 60d33ce34191533bb858852584bd9bddfeb16a23.
ATTACHMENT ID: 12777865
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+sqlContext.sparkContext.getConf.getInt(HBaseSparkConf.CACHE_SIZE,
HBaseSparkConf.defaultCachingSize))
+
scan.setCaching(sqlContext.sparkContext.getConf.getInt(HBaseSparkConf.BATCH_NUM,
HBaseSparkConf.defaultBatchNum))
+ sparkConf.getBoolean(HBaseSparkConf.BLOCK_CACHE_ENABLE,
HBaseSparkConf.defaultBlockCacheEnable))
+ Map("cacheSize" -> "100", "batchNum" -> "100", "blockCacheingEnable" ->
"true", "rowNum" -> "10"))
+ Map("cacheSize" -> "1000", "batchNum" -> "100", "blockCacheingEnable" ->
"true", "rowNum" -> "10"))
+ Map("cacheSize" -> "100", "batchNum" -> "1000", "blockCacheingEnable" ->
"true", "rowNum" -> "10"))
+ Map("cacheSize" -> "100", "batchNum" -> "100", "blockCacheingEnable" ->
"false", "rowNum" -> "10"))
+case class DummyScan(cacheSize: Int, batchNum: Int, blockCachingEnable:
Boolean, rowNum: Int)(@transient val sqlContext: SQLContext)
+ override def buildScan(): RDD[Row] = sqlContext.sparkContext.parallelize(0
until rowNum).map(Row(_)).map{
+sparkConf.getInt(HBaseSparkConf.CACHE_SIZE,
HBaseSparkConf.defaultCachingSize) != cacheSize ||
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16870//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16870//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16870//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16870//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16870//console
This message is automatically generated.
> Add option to set block cache to false on SparkSQL executions
> -
>
> Key: HBASE-14849
> URL: https://issues.apache.org/jira/browse/HBASE-14849
> Project: HBase
> Issue Type: New Feature
> Components: spark
>Reporter: Ted Malaska
>Assignee: Zhan Zhang
> Attachments: HBASE-14849.patch
>
>
> I was working at a client with a ported down version of the Spark module for
> HBase and realized we didn't add an option to turn of block cache for the
> scans.
> At the client I just disabled all caching with Spark SQL, this is an easy but
> very impactful fix.
> The fix for this patch will make this configurable
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14990) Tests in BaseTestHBaseFsck are run by its subclasses redundantly
Ted Yu created HBASE-14990:
--
Summary: Tests in BaseTestHBaseFsck are run by its subclasses
redundantly
Key: HBASE-14990
URL: https://issues.apache.org/jira/browse/HBASE-14990
Project: HBase
Issue Type: Bug
Components: test
Reporter: Ted Yu
Priority: Minor
Several tests extend BaseTestHBaseFsck :
public class TestHBaseFsckMOB extends BaseTestHBaseFsck {
hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckMOB.java
public class TestHBaseFsckOneRS extends BaseTestHBaseFsck {
hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckOneRS.java
public class TestHBaseFsckReplicas extends BaseTestHBaseFsck {
hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckReplicas.java
public class TestHBaseFsckTwoRS extends BaseTestHBaseFsck {
hbase-server/src/test//java/org/apache/hadoop/hbase/util/TestHBaseFsckTwoRS.java
BaseTestHBaseFsck contains several tests, e.g. testHDFSRegioninfoMissing
This means that the tests in BaseTestHBaseFsck would be run multiple times in
the test suite.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14989) Implementation of Mutation.getWriteToWAL() is backwards
[
https://issues.apache.org/jira/browse/HBASE-14989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059418#comment-15059418
]
Enis Soztutar commented on HBASE-14989:
---
Yep, it should be {{!=}} instead of {{==}}.
> Implementation of Mutation.getWriteToWAL() is backwards
> ---
>
> Key: HBASE-14989
> URL: https://issues.apache.org/jira/browse/HBASE-14989
> Project: HBase
> Issue Type: Bug
>Reporter: James Taylor
> Fix For: 1.2.0, 1.3.0, 1.1.4, 0.98.17, 1.0.4
>
>
> The implementation of the deprecated getWriteToWAL is backwards. It should
> return true if this.durability == Durability.SYNC_WAL:
> {code}
> /**
>* @deprecated Use {@link #getDurability()} instead.
>* @return true if edits should be applied to WAL, false if not
>*/
> @Deprecated
> public boolean getWriteToWAL() {
> return this.durability == Durability.SKIP_WAL;
> }
> {code}
> For example, if mutation.durability is Durability.SYNC_WAL and the following
> code is called {{clonedMutation.setWriteToWAL(mutation.getWriteToWAL())}}, it
> will disable writing to the WAL for clonedMutation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14989) Implementation of Mutation.getWriteToWAL() is backwards
[
https://issues.apache.org/jira/browse/HBASE-14989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-14989:
--
Fix Version/s: 1.0.4
0.98.17
1.1.4
1.3.0
1.2.0
> Implementation of Mutation.getWriteToWAL() is backwards
> ---
>
> Key: HBASE-14989
> URL: https://issues.apache.org/jira/browse/HBASE-14989
> Project: HBase
> Issue Type: Bug
>Reporter: James Taylor
> Fix For: 1.2.0, 1.3.0, 1.1.4, 0.98.17, 1.0.4
>
>
> The implementation of the deprecated getWriteToWAL is backwards. It should
> return true if this.durability == Durability.SYNC_WAL:
> {code}
> /**
>* @deprecated Use {@link #getDurability()} instead.
>* @return true if edits should be applied to WAL, false if not
>*/
> @Deprecated
> public boolean getWriteToWAL() {
> return this.durability == Durability.SKIP_WAL;
> }
> {code}
> For example, if mutation.durability is Durability.SYNC_WAL and the following
> code is called {{clonedMutation.setWriteToWAL(mutation.getWriteToWAL())}}, it
> will disable writing to the WAL for clonedMutation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14030) HBase Backup/Restore Phase 1
[
https://issues.apache.org/jira/browse/HBASE-14030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059416#comment-15059416
]
Hadoop QA commented on HBASE-14030:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12777859/HBASE-14030-v20.patch
against master branch at commit 60d33ce34191533bb858852584bd9bddfeb16a23.
ATTACHMENT ID: 12777859
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 23 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:red}-1 javac{color}. The applied patch generated 45 javac compiler
warnings (more than the master's current 37 warnings).
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
new checkstyle errors. Check build console for list of new errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ public BackupHandler(BackupContext backupContext, BackupManager
backupManager, Configuration conf) {
+if (!conf.getBoolean(HConstants.BACKUP_ENABLE_KEY,
HConstants.BACKUP_ENABLE_DEFAULT)) throw new BackupException(
+ throw new IOException("data from HBASE_BACKUP is corrupted: " +
Arrays.toString(flds));
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16869//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16869//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16869//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16869//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16869//console
This message is automatically generated.
> HBase Backup/Restore Phase 1
>
>
> Key: HBASE-14030
> URL: https://issues.apache.org/jira/browse/HBASE-14030
> Project: HBase
> Issue Type: Umbrella
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-14030-v0.patch, HBASE-14030-v1.patch,
> HBASE-14030-v10.patch, HBASE-14030-v11.patch, HBASE-14030-v12.patch,
> HBASE-14030-v13.patch, HBASE-14030-v14.patch, HBASE-14030-v15.patch,
> HBASE-14030-v17.patch, HBASE-14030-v18.patch, HBASE-14030-v2.patch,
> HBASE-14030-v20.patch, HBASE-14030-v3.patch, HBASE-14030-v4.patch,
> HBASE-14030-v5.patch, HBASE-14030-v6.patch, HBASE-14030-v7.patch,
> HBASE-14030-v8.patch
>
>
> This is the umbrella ticket for Backup/Restore Phase 1. See HBASE-7912 design
> doc for the phase description.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14985) TimeRange constructors should set allTime when appropriate
[ https://issues.apache.org/jira/browse/HBASE-14985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059403#comment-15059403 ] Ted Yu commented on HBASE-14985: w.r.t. TestTimeRange.java, do we need to create a new test class ? How about folding the new test into TestTimeRangeTracker ? > TimeRange constructors should set allTime when appropriate > -- > > Key: HBASE-14985 > URL: https://issues.apache.org/jira/browse/HBASE-14985 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.1.2, 0.98.16.1 >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby >Priority: Minor > Attachments: HBASE-14985.patch > > > The default TimeRange constructor creates a range from 0 to Long.MAX_VALUE > and sets an allTime flag to true. This flag allows some performance > optimizations when comparing or using TimeRanges. > This flag is not set, however, if you call "new TimeRange(0L)" or "new > TimeRange(0L, Long.MAX_VALUE)", even though both of these create a logically > equivalent TimeRange to "new TimeRange()". Since TimeRanges are immutable and > detecting this condition is trivial, we should set the flag automatically in > the explicit constructors when appropriate. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14460) [Perf Regression] Merge of MVCC and SequenceId (HBASE-HBASE-8763) slowed Increments, CheckAndPuts, batch operations
[ https://issues.apache.org/jira/browse/HBASE-14460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059394#comment-15059394 ] Jingcheng Du commented on HBASE-14460: -- Thanks for comments! You are right, [~stack]. In the current implementation, region caches a row lock context in lockedRows. In the patch I wrap the lock into a RowContext, and add one more RowOperationContext in RowContext, consequently the lockedRows cache the RowContext instead. bq. could we skip out on the wait in step #6 above in the first list. If we skip the waiting, does it mean a user gets the response, but this cell is not yet visible? Is it ok? bq. Is the intent that all passed in rowOperationContexts all get the highest found nextWrite Number? If so, this loop does not seem like it will achieve this (if rowOperationContexts have write numbers 1 to 3 say, then after the loop is done, all rowOperationContexts will have the same 1 to 3 write numbers... but if write numbers are 3 to 1, then they will have a write number of 3..) Yes, this is done on purpose. It tries to find the biggest lastWriteNumber (which means this current WriteEntry for this batch has to wait for the biggest lastWriteNumber is visible before it can continue in waitForRead). For each RowOperationContext, it has the same nextWriteNumber (by using context.setNextWriteNumber(nextWriteNumber)). Maybe the names are similar, and get mixed up for reading? In the given example, if RowOperationContext has write number 1 to 3, after the loop is done, the lastWritenNumber for the WriteEntry is 3, and the nextWriteNumber in each RowOperationContext has a new number (maybe 4). > [Perf Regression] Merge of MVCC and SequenceId (HBASE-HBASE-8763) slowed > Increments, CheckAndPuts, batch operations > --- > > Key: HBASE-14460 > URL: https://issues.apache.org/jira/browse/HBASE-14460 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: stack >Assignee: stack >Priority: Critical > Attachments: 0.94.test.patch, 0.98.test.patch, > 1.0.80.flamegraph-7932.svg, 14460.txt, 98.80.flamegraph-11428.svg, > HBASE-14460-discussion.patch, client.test.patch, > flamegraph-13120.svg.master.singlecell.svg, flamegraph-26636.094.100.svg, > flamegraph-28066.098.singlecell.svg, flamegraph-28767.098.100.svg, > flamegraph-31647.master.100.svg, flamegraph-9466.094.singlecell.svg, > m.test.patch, region_lock.png, testincrement.094.patch, > testincrement.098.patch, testincrement.master.patch > > > As reported by 鈴木俊裕 up on the mailing list -- see "Performance degradation > between CDH5.3.1(HBase0.98.6) and CDH5.4.5(HBase1.0.0)" -- our unification of > sequenceid and MVCC slows Increments (and other ops) as the mvcc needs to > 'catch up' to our current point before we can read the last Increment value > that we need to update. > We can say that our Increment is just done wrong, we should just be writing > Increments and summing on read, but checkAndPut as well as batching > operations have the same issue. Fix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14989) Implementation of Mutation.getWriteToWAL() is backwards
[
https://issues.apache.org/jira/browse/HBASE-14989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated HBASE-14989:
-
Summary: Implementation of Mutation.getWriteToWAL() is backwards (was:
Mutation.getWriteToWAL is backwards)
> Implementation of Mutation.getWriteToWAL() is backwards
> ---
>
> Key: HBASE-14989
> URL: https://issues.apache.org/jira/browse/HBASE-14989
> Project: HBase
> Issue Type: Bug
>Reporter: James Taylor
>
> The implementation of the deprecated getWriteToWAL is backwards. It should
> return true if this.durability == Durability.SYNC_WAL:
> {code}
> /**
>* @deprecated Use {@link #getDurability()} instead.
>* @return true if edits should be applied to WAL, false if not
>*/
> @Deprecated
> public boolean getWriteToWAL() {
> return this.durability == Durability.SKIP_WAL;
> }
> {code}
> For example, if mutation.durability is Durability.SYNC_WAL and the following
> code is called {{clonedMutation.setWriteToWAL(mutation.getWriteToWAL())}}, it
> will disable writing to the WAL for clonedMutation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14989) Mutation.getWriteToWAL is backwards
James Taylor created HBASE-14989:
Summary: Mutation.getWriteToWAL is backwards
Key: HBASE-14989
URL: https://issues.apache.org/jira/browse/HBASE-14989
Project: HBase
Issue Type: Bug
Reporter: James Taylor
The implementation of the deprecated getWriteToWAL is backwards. It should
return true if this.durability == Durability.SYNC_WAL:
{code}
/**
* @deprecated Use {@link #getDurability()} instead.
* @return true if edits should be applied to WAL, false if not
*/
@Deprecated
public boolean getWriteToWAL() {
return this.durability == Durability.SKIP_WAL;
}
{code}
For example, if mutation.durability is Durability.SYNC_WAL and the following
code is called {{clonedMutation.setWriteToWAL(mutation.getWriteToWAL())}}, it
will disable writing to the WAL for clonedMutation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14955) OOME: cannot create native thread is back
[
https://issues.apache.org/jira/browse/HBASE-14955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059376#comment-15059376
]
Hadoop QA commented on HBASE-14955:
---
{color:green}+1 overall{color}.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16868//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16868//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16868//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16868//console
This message is automatically generated.
> OOME: cannot create native thread is back
> -
>
> Key: HBASE-14955
> URL: https://issues.apache.org/jira/browse/HBASE-14955
> Project: HBase
> Issue Type: Sub-task
> Components: test
>Reporter: stack
>Assignee: Heng Chen
> Attachments: HBASE-14955-branch-1.patch
>
>
> This fail is OOME cannot create native thread. Two MR jobs fail:
>
> org.apache.hadoop.hbase.mapreduce.TestImportTSVWithVisibilityLabels.org.apache.hadoop.hbase.mapreduce.TestImportTSVWithVisibilityLabels46
> ms1
> org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan1.org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan142
> ms1
> Was running 1.3 tests on H0.
> Could try and purge resources used by these tests.
> Making issue in meantime to keep an eye on it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14956) [HBase ZKcli] JLine support is disabled. Better to enable this in HBase.
[
https://issues.apache.org/jira/browse/HBASE-14956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059356#comment-15059356
]
Pankaj Kumar commented on HBASE-14956:
--
Its better to have auto complete and proper zkcli prompt. However it impact
hbase shell if jline jar is placed at hbase/lib, so we can place the jar at
"hbase/lib/jline" and set it in the classpath on zkcli command.
> [HBase ZKcli] JLine support is disabled. Better to enable this in HBase.
> -
>
> Key: HBASE-14956
> URL: https://issues.apache.org/jira/browse/HBASE-14956
> Project: HBase
> Issue Type: Bug
> Components: Client, Zookeeper
>Affects Versions: 1.0.2
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>
> To perform the zkcli operations using hbase,
> jline is disabled,
> {code}
> [ERROR] Terminal initialization failed; falling back to unsupported
> java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but
> interface was expected
> at jline.TerminalFactory.create(TerminalFactory.java:101)
> at jline.TerminalFactory.get(TerminalFactory.java:158)
> at jline.console.ConsoleReader.(ConsoleReader.java:229)
> at jline.console.ConsoleReader.(ConsoleReader.java:221)
> at jline.console.ConsoleReader.(ConsoleReader.java:209)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
> at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:335)
> at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:303)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperMainServer.main(ZooKeeperMainServer.java:108)
> JLine support is disabled
> {code}
> To enable this jline-.jar is needed in hbase libraries.
> eg: jline-2.11.jar should be exist in hbase/lib directory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[ https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059341#comment-15059341 ] Hudson commented on HBASE-14977: SUCCESS: Integrated in HBase-1.2 #445 (See [https://builds.apache.org/job/HBase-1.2/445/]) HBASE-14977 ChoreService.shutdown may result in (enis: rev ef19c2147d9c8cf88d8145bfe9e9bf25ff01206d) * hbase-common/src/main/java/org/apache/hadoop/hbase/ChoreService.java > ChoreService.shutdown may result in ConcurrentModificationException > --- > > Key: HBASE-14977 > URL: https://issues.apache.org/jira/browse/HBASE-14977 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4 > > Attachments: HBASE-14977-v1.patch > > > As seen in this test: > https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/425/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt > We need to make shutdown method synchronized to avoid this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14947) WALProcedureStore improvements
[ https://issues.apache.org/jira/browse/HBASE-14947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059342#comment-15059342 ] Hudson commented on HBASE-14947: SUCCESS: Integrated in HBase-1.2 #445 (See [https://builds.apache.org/job/HBase-1.2/445/]) HBASE-14947 WALProcedureStore improvements (matteo.bertozzi: rev e8698da2a4b6f7b419478485617ec22933f22f3c) * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/TestProcedureStoreTracker.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/wal/TestWALProcedureStore.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureRecovery.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/ProcedureStoreTracker.java > WALProcedureStore improvements > -- > > Key: HBASE-14947 > URL: https://issues.apache.org/jira/browse/HBASE-14947 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Ashu Pachauri >Assignee: Matteo Bertozzi >Priority: Blocker > Fix For: 2.0.0, 1.2.0, 1.1.3 > > Attachments: HBASE-14947-v0.patch, HBASE-14947-v1.patch > > > We ended up with a deadlock in HBASE-14943, with the storeTracker and lock > acquired in reverse order by syncLoop() and insert/update/delete. In the > syncLoop() with don't need the lock when we try to roll or removeInactive. > also we can move the insert/update/delete tracker check in the syncLoop > avoiding to the extra lock operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14984) Allow memcached block cache to set optimze to false
[
https://issues.apache.org/jira/browse/HBASE-14984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059333#comment-15059333
]
Hadoop QA commented on HBASE-14984:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12777827/HBASE-14984.patch
against master branch at commit 60d33ce34191533bb858852584bd9bddfeb16a23.
ATTACHMENT ID: 12777827
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16867//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16867//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16867//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16867//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16867//console
This message is automatically generated.
> Allow memcached block cache to set optimze to false
> ---
>
> Key: HBASE-14984
> URL: https://issues.apache.org/jira/browse/HBASE-14984
> Project: HBase
> Issue Type: Improvement
> Components: BlockCache
>Affects Versions: 2.0.0, 1.2.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14984.patch
>
>
> In order to keep latency consistent it might not be good to allow the spy
> memcached client to optimize.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14985) TimeRange constructors should set allTime when appropriate
[
https://issues.apache.org/jira/browse/HBASE-14985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059310#comment-15059310
]
Hadoop QA commented on HBASE-14985:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12777874/HBASE-14985.patch
against master branch at commit 60d33ce34191533bb858852584bd9bddfeb16a23.
ATTACHMENT ID: 12777874
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16871//console
This message is automatically generated.
> TimeRange constructors should set allTime when appropriate
> --
>
> Key: HBASE-14985
> URL: https://issues.apache.org/jira/browse/HBASE-14985
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.1.2, 0.98.16.1
>Reporter: Geoffrey Jacoby
>Assignee: Geoffrey Jacoby
>Priority: Minor
> Attachments: HBASE-14985.patch
>
>
> The default TimeRange constructor creates a range from 0 to Long.MAX_VALUE
> and sets an allTime flag to true. This flag allows some performance
> optimizations when comparing or using TimeRanges.
> This flag is not set, however, if you call "new TimeRange(0L)" or "new
> TimeRange(0L, Long.MAX_VALUE)", even though both of these create a logically
> equivalent TimeRange to "new TimeRange()". Since TimeRanges are immutable and
> detecting this condition is trivial, we should set the flag automatically in
> the explicit constructors when appropriate.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14460) [Perf Regression] Merge of MVCC and SequenceId (HBASE-HBASE-8763) slowed Increments, CheckAndPuts, batch operations
[ https://issues.apache.org/jira/browse/HBASE-14460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14460: -- Attachment: client.test.patch 98.80.flamegraph-11428.svg 1.0.80.flamegraph-7932.svg Attempts at reproducing the slowdown in the small have failed to pay off. I see the roughly 2x difference but not the 7x claimed by the original poster. With some help from Preston Koprivica, you need to have some friction in place to see the issue; the friction gets amplified by mvcc wait. Here is a test that clearly shows the problem. 0.98 is about 33% slower than 0.94 (0.98 added in mvcc) and then 1.0+ is about 10x the latency WHEN you have 80 clients running external to the regionserver process banging on it. The flame graphs show us spending loads of time in mvcc waiting. The stack trace is the SAME as for the tests in the small but we just seem to be waiting overall longer. There is an amplification going on. Looking at options: [~jingcheng...@intel.com]'s suggestion is a nice one. Will narrow what we have to wait on. I tried disabling completely our wait-on-mvcc before we read at all and this helps; we are only 3x slower than 0.98 (and 4x slower than 0.94). Need some other bit of trickery to take us closer to what was there before. > [Perf Regression] Merge of MVCC and SequenceId (HBASE-HBASE-8763) slowed > Increments, CheckAndPuts, batch operations > --- > > Key: HBASE-14460 > URL: https://issues.apache.org/jira/browse/HBASE-14460 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: stack >Assignee: stack >Priority: Critical > Attachments: 0.94.test.patch, 0.98.test.patch, > 1.0.80.flamegraph-7932.svg, 14460.txt, 98.80.flamegraph-11428.svg, > HBASE-14460-discussion.patch, client.test.patch, > flamegraph-13120.svg.master.singlecell.svg, flamegraph-26636.094.100.svg, > flamegraph-28066.098.singlecell.svg, flamegraph-28767.098.100.svg, > flamegraph-31647.master.100.svg, flamegraph-9466.094.singlecell.svg, > m.test.patch, region_lock.png, testincrement.094.patch, > testincrement.098.patch, testincrement.master.patch > > > As reported by 鈴木俊裕 up on the mailing list -- see "Performance degradation > between CDH5.3.1(HBase0.98.6) and CDH5.4.5(HBase1.0.0)" -- our unification of > sequenceid and MVCC slows Increments (and other ops) as the mvcc needs to > 'catch up' to our current point before we can read the last Increment value > that we need to update. > We can say that our Increment is just done wrong, we should just be writing > Increments and summing on read, but checkAndPut as well as batching > operations have the same issue. Fix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059253#comment-15059253 ] Ted Malaska commented on HBASE-14849: - Can you make the review board? Thanks > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > Attachments: HBASE-14849.patch > > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14795) Enhance the spark-hbase scan operations
[ https://issues.apache.org/jira/browse/HBASE-14795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059230#comment-15059230 ] Jonathan Hsieh commented on HBASE-14795: Thanks Zhan. If you just to the fixes for the warnings I'll review, if it is folded in with HBASE-14849 I'll let ted or some of the others review. IMO patches are easier to review if they each only capture one idea -- so HBASE-14159 feels like the better place to resolve the warnings introduced here. > Enhance the spark-hbase scan operations > --- > > Key: HBASE-14795 > URL: https://issues.apache.org/jira/browse/HBASE-14795 > Project: HBase > Issue Type: Improvement >Reporter: Ted Malaska >Assignee: Zhan Zhang >Priority: Minor > Fix For: 2.0.0 > > Attachments: > 0001-HBASE-14795-Enhance-the-spark-hbase-scan-operations.patch, > HBASE-14795-1.patch, HBASE-14795-2.patch, HBASE-14795-3.patch, > HBASE-14795-4.patch > > > This is a sub-jira of HBASE-14789. This jira is to focus on the replacement > of TableInputFormat for a more custom scan implementation that will make the > following use case more effective. > Use case: > In the case you have multiple scan ranges on a single table with in a single > query. TableInputFormat will scan the the outer range of the scan start and > end range where this implementation can be more pointed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14988) java.lang.reflect.InvocationTargetException when doing batch with large number of increment
[
https://issues.apache.org/jira/browse/HBASE-14988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Keith Lui updated HBASE-14988:
--
Description:
Tried to do a 10K increment with batch. When using
public void batch(List actions, Object[] results)
got
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.simontuffs.onejar.Boot.run(Boot.java:340)
at com.simontuffs.onejar.Boot.main(Boot.java:166)
Caused by: java.lang.AssertionError: results.length
at
org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl.(AsyncProcess.java:763)
at
org.apache.hadoop.hbase.client.AsyncProcess.createAsyncRequestFuture(AsyncProcess.java:1578)
at
org.apache.hadoop.hbase.client.AsyncProcess.submitAll(AsyncProcess.java:554)
at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:1000)
Interestingly no exception is thrown when using the deprecated method
public Object[] batch(List actions)
This is a sample code in Scala:
val table = connection.getTable(TableName.valueOf("test_table"))
val increments = for (i <- 0 until 1) yield {
val increment = new Increment(Random.nextDouble().toString.getBytes)
increment.addColumn(family, qualifier, Random.nextLong())
increment
}
table.batch(increments, Array.empty[Object])
was:
Tried to do a 10K increment with batch. When using
public void batch(List actions, Object[] results)
got
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.simontuffs.onejar.Boot.run(Boot.java:340)
at com.simontuffs.onejar.Boot.main(Boot.java:166)
Caused by: java.lang.AssertionError: results.length
at
org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl.(AsyncProcess.java:763)
at
org.apache.hadoop.hbase.client.AsyncProcess.createAsyncRequestFuture(AsyncProcess.java:1578)
at
org.apache.hadoop.hbase.client.AsyncProcess.submitAll(AsyncProcess.java:554)
at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:1000)
Interestingly no exception is thrown when using the deprecated method
public Object[] batch(List actions)
This is a sample code in Scala:
val table = connection.getTable(TableName.valueOf("test_table"))
val increments = for (i <- 0 until 1) yield {
val rand = Random.nextDouble().toString
val increment = new Increment(rand.getBytes)
increment.addColumn(family, qualifier, Random.nextLong())
increment
}
table.batch(increments, Array.empty[Object])
> java.lang.reflect.InvocationTargetException when doing batch with large
> number of increment
> ---
>
> Key: HBASE-14988
> URL: https://issues.apache.org/jira/browse/HBASE-14988
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Keith Lui
>
> Tried to do a 10K increment with batch. When using
> public void batch(List actions, Object[] results)
> got
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at com.simontuffs.onejar.Boot.run(Boot.java:340)
> at com.simontuffs.onejar.Boot.main(Boot.java:166)
> Caused by: java.lang.AssertionError: results.length
> at
> org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl.(AsyncProcess.java:763)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.createAsyncRequestFuture(AsyncProcess.java:1578)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.submitAll(AsyncProcess.java:554)
> at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:1000)
> Interestingly no exception is thrown when using the deprecated method
> public Object[] batch(List actions)
> This is a sample code in Scala:
> val table = connection.getTable(TableName.valueOf("test_table"))
> val increments = for (i <- 0 until 1) yield {
> val increment = new Increment(Random.nextDouble().toString.getBytes)
> increment.addColumn(family, qualifier, Random.nextLong())
> incremen
[jira] [Created] (HBASE-14988) java.lang.reflect.InvocationTargetException when doing batch with large number of increment
Keith Lui created HBASE-14988:
-
Summary: java.lang.reflect.InvocationTargetException when doing
batch with large number of increment
Key: HBASE-14988
URL: https://issues.apache.org/jira/browse/HBASE-14988
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0
Reporter: Keith Lui
Tried to do a 10K increment with batch. When using
public void batch(List actions, Object[] results)
got
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.simontuffs.onejar.Boot.run(Boot.java:340)
at com.simontuffs.onejar.Boot.main(Boot.java:166)
Caused by: java.lang.AssertionError: results.length
at
org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl.(AsyncProcess.java:763)
at
org.apache.hadoop.hbase.client.AsyncProcess.createAsyncRequestFuture(AsyncProcess.java:1578)
at
org.apache.hadoop.hbase.client.AsyncProcess.submitAll(AsyncProcess.java:554)
at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:1000)
Interestingly no exception is thrown when using the deprecated method
public Object[] batch(List actions)
This is a sample code in Scala:
val table = connection.getTable(TableName.valueOf("test_table"))
val increments = for (i <- 0 until 1) yield {
val rand = Random.nextDouble().toString
val increment = new Increment(rand.getBytes)
increment.addColumn(family, qualifier, Random.nextLong())
increment
}
table.batch(increments, Array.empty[Object])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14985) TimeRange constructors should set allTime when appropriate
[ https://issues.apache.org/jira/browse/HBASE-14985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Geoffrey Jacoby updated HBASE-14985: Status: Patch Available (was: Open) Uploaded the patch with a test > TimeRange constructors should set allTime when appropriate > -- > > Key: HBASE-14985 > URL: https://issues.apache.org/jira/browse/HBASE-14985 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.98.16.1, 1.1.2 >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby >Priority: Minor > Attachments: HBASE-14985.patch > > > The default TimeRange constructor creates a range from 0 to Long.MAX_VALUE > and sets an allTime flag to true. This flag allows some performance > optimizations when comparing or using TimeRanges. > This flag is not set, however, if you call "new TimeRange(0L)" or "new > TimeRange(0L, Long.MAX_VALUE)", even though both of these create a logically > equivalent TimeRange to "new TimeRange()". Since TimeRanges are immutable and > detecting this condition is trivial, we should set the flag automatically in > the explicit constructors when appropriate. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14985) TimeRange constructors should set allTime when appropriate
[ https://issues.apache.org/jira/browse/HBASE-14985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Geoffrey Jacoby updated HBASE-14985: Attachment: HBASE-14985.patch > TimeRange constructors should set allTime when appropriate > -- > > Key: HBASE-14985 > URL: https://issues.apache.org/jira/browse/HBASE-14985 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.1.2, 0.98.16.1 >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby >Priority: Minor > Attachments: HBASE-14985.patch > > > The default TimeRange constructor creates a range from 0 to Long.MAX_VALUE > and sets an allTime flag to true. This flag allows some performance > optimizations when comparing or using TimeRanges. > This flag is not set, however, if you call "new TimeRange(0L)" or "new > TimeRange(0L, Long.MAX_VALUE)", even though both of these create a logically > equivalent TimeRange to "new TimeRange()". Since TimeRanges are immutable and > detecting this condition is trivial, we should set the flag automatically in > the explicit constructors when appropriate. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[ https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059196#comment-15059196 ] Hudson commented on HBASE-14977: FAILURE: Integrated in HBase-Trunk_matrix #556 (See [https://builds.apache.org/job/HBase-Trunk_matrix/556/]) HBASE-14977 ChoreService.shutdown may result in (enis: rev a4bbc461e367994ed7b41673a4accb99bbe5c364) * hbase-common/src/main/java/org/apache/hadoop/hbase/ChoreService.java > ChoreService.shutdown may result in ConcurrentModificationException > --- > > Key: HBASE-14977 > URL: https://issues.apache.org/jira/browse/HBASE-14977 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4 > > Attachments: HBASE-14977-v1.patch > > > As seen in this test: > https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/425/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt > We need to make shutdown method synchronized to avoid this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14968) ConcurrentModificationException in region close resulting in the region staying in closing state
[
https://issues.apache.org/jira/browse/HBASE-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059198#comment-15059198
]
Hudson commented on HBASE-14968:
FAILURE: Integrated in HBase-Trunk_matrix #556 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/556/])
HBASE-14968 ConcurrentModificationException in region close resulting in (enis:
rev 3e260631612b4f41638e663b2e451841f522ae49)
* hbase-server/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/RegionReplicaFlushHandler.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/executor/TestExecutorService.java
> ConcurrentModificationException in region close resulting in the region
> staying in closing state
>
>
> Key: HBASE-14968
> URL: https://issues.apache.org/jira/browse/HBASE-14968
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4, 1.0.4
>
> Attachments: hbase-14968_v1.patch
>
>
> We have seen this in our tests. The region gets closed, but the region close
> handler gets an unexpected exception causing the region to stay in closing
> state until RS is restarted.
> The Phoenix and security coprocessors were loaded in that region. Here is the
> stack trace:
> {code}
> java.util.ConcurrentModificationException
> at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
> at java.util.ArrayList$Itr.next(ArrayList.java:851)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.shutdown(CoprocessorHost.java:442)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionEnvironment.shutdown(RegionCoprocessorHost.java:155)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.shutdown(CoprocessorHost.java:272)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$5.postEnvCall(RegionCoprocessorHost.java:496)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1761)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postClose(RegionCoprocessorHost.java:489)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1502)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1349)
> at
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:138)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14534) Bump yammer/coda/dropwizard metrics dependency version
[ https://issues.apache.org/jira/browse/HBASE-14534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059200#comment-15059200 ] Hudson commented on HBASE-14534: FAILURE: Integrated in HBase-Trunk_matrix #556 (See [https://builds.apache.org/job/HBase-Trunk_matrix/556/]) HBASE-14534 Bump yammer/coda/dropwizard metrics dependency version (antonov: rev abe30b52a8036078f833dc5b3d2b03daa2e93dfc) * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/BlockCacheViewTmpl.jamon * hbase-it/pom.xml * hbase-shell/pom.xml * hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestMetricsConnection.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetricsConnection.java * hbase-server/src/test/java/org/apache/hadoop/hbase/TestPerformanceEvaluation.java * hbase-resource-bundle/src/main/resources/supplemental-models.xml * hbase-server/pom.xml * hbase-server/src/test/java/org/apache/hadoop/hbase/wal/WALPerformanceEvaluation.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/AgeSnapshot.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCacheUtil.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MutableHistogram.java * hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStats.java * hbase-shaded/pom.xml * pom.xml * hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java * hbase-server/src/main/java/org/apache/hadoop/hbase/util/YammerHistogramUtils.java * hbase-hadoop2-compat/pom.xml * hbase-client/pom.xml * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestClientPushback.java * hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestRegionReplicaPerf.java > Bump yammer/coda/dropwizard metrics dependency version > -- > > Key: HBASE-14534 > URL: https://issues.apache.org/jira/browse/HBASE-14534 > Project: HBase > Issue Type: Task >Reporter: Nick Dimiduk >Assignee: Mikhail Antonov >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-14534-v2.patch, HBASE-14534.patch, wip.patch > > > After HBASE-12911 lands, let's update our dependency to the latest > incarnation of this library. I guess they're now calling it Dropwizard > Metrics. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13425) Documentation nit in REST Gateway impersonation section
[
https://issues.apache.org/jira/browse/HBASE-13425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059201#comment-15059201
]
Hudson commented on HBASE-13425:
FAILURE: Integrated in HBase-Trunk_matrix #556 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/556/])
HBASE-13425 Documentation nit in REST Gateway impersonation section
(mstanleyjones: rev f0a97a1fdf0d8386f089306fdc1089080758e973)
* src/main/asciidoc/_chapters/security.adoc
> Documentation nit in REST Gateway impersonation section
> ---
>
> Key: HBASE-13425
> URL: https://issues.apache.org/jira/browse/HBASE-13425
> Project: HBase
> Issue Type: Improvement
> Components: documentation
>Affects Versions: 2.0.0
>Reporter: Jeremie Gomez
>Assignee: Misty Stanley-Jones
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-13425-v1.patch, HBASE-13425.patch
>
>
> In section "55.8. REST Gateway Impersonation Configuration", there is another
> property that needs to be set (and thus documented).
> After this sentence ("To enable REST gateway impersonation, add the following
> to the hbase-site.xml file for every REST gateway."), we should add :
>
>hbase.rest.support.proxyuser
> true
>
> It not set, doing a curl call on the rest gateway gives the error "support
> for proxyuser is not configured".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14947) WALProcedureStore improvements
[ https://issues.apache.org/jira/browse/HBASE-14947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059199#comment-15059199 ] Hudson commented on HBASE-14947: FAILURE: Integrated in HBase-Trunk_matrix #556 (See [https://builds.apache.org/job/HBase-Trunk_matrix/556/]) HBASE-14947 WALProcedureStore improvements (matteo.bertozzi: rev 60d33ce34191533bb858852584bd9bddfeb16a23) * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/TestProcedureStoreTracker.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureRecovery.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/ProcedureStoreTracker.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/wal/TestWALProcedureStore.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java > WALProcedureStore improvements > -- > > Key: HBASE-14947 > URL: https://issues.apache.org/jira/browse/HBASE-14947 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Ashu Pachauri >Assignee: Matteo Bertozzi >Priority: Blocker > Fix For: 2.0.0, 1.2.0, 1.1.3 > > Attachments: HBASE-14947-v0.patch, HBASE-14947-v1.patch > > > We ended up with a deadlock in HBASE-14943, with the storeTracker and lock > acquired in reverse order by syncLoop() and insert/update/delete. In the > syncLoop() with don't need the lock when we try to roll or removeInactive. > also we can move the insert/update/delete tracker check in the syncLoop > avoiding to the extra lock operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13976) release manager list in ref guide is missing 0.94 line
[ https://issues.apache.org/jira/browse/HBASE-13976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059197#comment-15059197 ] Hudson commented on HBASE-13976: FAILURE: Integrated in HBase-Trunk_matrix #556 (See [https://builds.apache.org/job/HBase-Trunk_matrix/556/]) HBASE-13976 Add 0.94 Release Manager to docs (mstanleyjones: rev b8539c62e8d970ac82cf6ace0e8aa258bab1b97c) * src/main/asciidoc/_chapters/developer.adoc > release manager list in ref guide is missing 0.94 line > -- > > Key: HBASE-13976 > URL: https://issues.apache.org/jira/browse/HBASE-13976 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Sean Busbey >Assignee: Misty Stanley-Jones > Fix For: 2.0.0 > > Attachments: HBASE-13976.patch > > > 0.94 has slowed substantially, but we haven't EOLed it yet so we should make > sure folks looking to get a fix included know where to look. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14987) HBaseFsck#mergeRegionDirs() needs to handle compaction marker in recovered edits
[
https://issues.apache.org/jira/browse/HBASE-14987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059194#comment-15059194
]
Ted Yu commented on HBASE-14987:
Approaches I can think of:
1. rewrite compaction marker so that it matches the new region
2. annotate the recovered edits so that log replay can understand the
transition for the compaction marker and handle it properly
> HBaseFsck#mergeRegionDirs() needs to handle compaction marker in recovered
> edits
>
>
> Key: HBASE-14987
> URL: https://issues.apache.org/jira/browse/HBASE-14987
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>
> One customer encountered the following error when replaying recovered edits,
> leading to region open failure:
> {code}
> region=table1,d6b-2282-9223370590058224807-U-9856557-
> EJ452727-16313786400171,1449616291799.fa8a526f2578eb3630bb08a4b1648f5d.,
> starting to roll back the global memstore size.
> org.apache.hadoop.hbase.regionserver.WrongRegionException: Compaction marker
> from WAL table_name: "table1"
> encoded_region_name: "d389c70fde9ec07971d0cfd20ef8f575"
> ...
> region_name:
> "table1,d6b-2282-9223370590058224807-U-9856557-EJ452727-16313786400171,1449089609367.d389c70fde9ec07971d0cfd20ef8f575."
> targetted for region d389c70fde9ec07971d0cfd20ef8f575 does not match this
> region: {ENCODED => fa8a526f2578eb3630bb08a4b1648f5d, NAME =>
> 'table1,d6b-2282-
> 9223370590058224807-U-9856557-EJ452727-16313786400171,1449616291799.fa8a526f2578eb3630bb08a4b1648f5d.',
> STARTKEY => 'd6b-2282-9223370590058224807-U-9856557-EJ452727-
> 16313786400171', ENDKEY =>
> 'd76-2553-9223370588576178807-U-7416904-EK875822-1766218060'}
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkTargetRegion(HRegion.java:4592)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayWALCompactionMarker(HRegion.java:3831)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:3747)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3601)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionStores(HRegion.java:911)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:789)
> at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:762)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5774)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5744)
> {code}
> This was likely caused by the following action of hbck:
> {code}
> 15/12/08 18:11:34 INFO util.HBaseFsck: [hbasefsck-pool1-t37] Moving files
> from
> hdfs://Zealand/hbase/data/default/table1/d389c70fde9ec07971d0cfd20ef8f575/recovered.edits
> into containing region
> hdfs://Zealand/hbase/data/default/table1/fa8a526f2578eb3630bb08a4b1648f5d/recovered.edits
> {code}
> The recovered.edits for d389c70fde9ec07971d0cfd20ef8f575 contained compaction
> marker which couldn't be replayed against fa8a526f2578eb3630bb08a4b1648f5d
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14987) HBaseFsck#mergeRegionDirs() need to handle compaction marker in recovered edits
Ted Yu created HBASE-14987:
--
Summary: HBaseFsck#mergeRegionDirs() need to handle compaction
marker in recovered edits
Key: HBASE-14987
URL: https://issues.apache.org/jira/browse/HBASE-14987
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
One customer encountered the following error when replaying recovered edits,
leading to region open failure:
{code}
region=table1,d6b-2282-9223370590058224807-U-9856557-
EJ452727-16313786400171,1449616291799.fa8a526f2578eb3630bb08a4b1648f5d.,
starting to roll back the global memstore size.
org.apache.hadoop.hbase.regionserver.WrongRegionException: Compaction marker
from WAL table_name: "table1"
encoded_region_name: "d389c70fde9ec07971d0cfd20ef8f575"
...
region_name:
"table1,d6b-2282-9223370590058224807-U-9856557-EJ452727-16313786400171,1449089609367.d389c70fde9ec07971d0cfd20ef8f575."
targetted for region d389c70fde9ec07971d0cfd20ef8f575 does not match this
region: {ENCODED => fa8a526f2578eb3630bb08a4b1648f5d, NAME => 'table1,d6b-2282-
9223370590058224807-U-9856557-EJ452727-16313786400171,1449616291799.fa8a526f2578eb3630bb08a4b1648f5d.',
STARTKEY => 'd6b-2282-9223370590058224807-U-9856557-EJ452727-
16313786400171', ENDKEY =>
'd76-2553-9223370588576178807-U-7416904-EK875822-1766218060'}
at
org.apache.hadoop.hbase.regionserver.HRegion.checkTargetRegion(HRegion.java:4592)
at
org.apache.hadoop.hbase.regionserver.HRegion.replayWALCompactionMarker(HRegion.java:3831)
at
org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:3747)
at
org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3601)
at
org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionStores(HRegion.java:911)
at
org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:789)
at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:762)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5774)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5744)
{code}
This was likely caused by the following action of hbck:
{code}
15/12/08 18:11:34 INFO util.HBaseFsck: [hbasefsck-pool1-t37] Moving files from
hdfs://Zealand/hbase/data/default/table1/d389c70fde9ec07971d0cfd20ef8f575/recovered.edits
into containing region
hdfs://Zealand/hbase/data/default/table1/fa8a526f2578eb3630bb08a4b1648f5d/recovered.edits
{code}
The recovered.edits for d389c70fde9ec07971d0cfd20ef8f575 contained compaction
marker which couldn't be replayed against fa8a526f2578eb3630bb08a4b1648f5d
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14987) HBaseFsck#mergeRegionDirs() needs to handle compaction marker in recovered edits
[
https://issues.apache.org/jira/browse/HBASE-14987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-14987:
---
Summary: HBaseFsck#mergeRegionDirs() needs to handle compaction marker in
recovered edits (was: HBaseFsck#mergeRegionDirs() need to handle compaction
marker in recovered edits)
> HBaseFsck#mergeRegionDirs() needs to handle compaction marker in recovered
> edits
>
>
> Key: HBASE-14987
> URL: https://issues.apache.org/jira/browse/HBASE-14987
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>
> One customer encountered the following error when replaying recovered edits,
> leading to region open failure:
> {code}
> region=table1,d6b-2282-9223370590058224807-U-9856557-
> EJ452727-16313786400171,1449616291799.fa8a526f2578eb3630bb08a4b1648f5d.,
> starting to roll back the global memstore size.
> org.apache.hadoop.hbase.regionserver.WrongRegionException: Compaction marker
> from WAL table_name: "table1"
> encoded_region_name: "d389c70fde9ec07971d0cfd20ef8f575"
> ...
> region_name:
> "table1,d6b-2282-9223370590058224807-U-9856557-EJ452727-16313786400171,1449089609367.d389c70fde9ec07971d0cfd20ef8f575."
> targetted for region d389c70fde9ec07971d0cfd20ef8f575 does not match this
> region: {ENCODED => fa8a526f2578eb3630bb08a4b1648f5d, NAME =>
> 'table1,d6b-2282-
> 9223370590058224807-U-9856557-EJ452727-16313786400171,1449616291799.fa8a526f2578eb3630bb08a4b1648f5d.',
> STARTKEY => 'd6b-2282-9223370590058224807-U-9856557-EJ452727-
> 16313786400171', ENDKEY =>
> 'd76-2553-9223370588576178807-U-7416904-EK875822-1766218060'}
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkTargetRegion(HRegion.java:4592)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayWALCompactionMarker(HRegion.java:3831)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:3747)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3601)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionStores(HRegion.java:911)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:789)
> at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:762)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5774)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5744)
> {code}
> This was likely caused by the following action of hbck:
> {code}
> 15/12/08 18:11:34 INFO util.HBaseFsck: [hbasefsck-pool1-t37] Moving files
> from
> hdfs://Zealand/hbase/data/default/table1/d389c70fde9ec07971d0cfd20ef8f575/recovered.edits
> into containing region
> hdfs://Zealand/hbase/data/default/table1/fa8a526f2578eb3630bb08a4b1648f5d/recovered.edits
> {code}
> The recovered.edits for d389c70fde9ec07971d0cfd20ef8f575 contained compaction
> marker which couldn't be replayed against fa8a526f2578eb3630bb08a4b1648f5d
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14986) Display the key range for each region in the Regionserver UI
[ https://issues.apache.org/jira/browse/HBASE-14986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059189#comment-15059189 ] Matteo Bertozzi commented on HBASE-14986: - I think the number needs to be "cleaned up" a bit. to me, It will be useful to have a quick look at regions and see that "key space of region2 is 5 times larger than the other regions". > Display the key range for each region in the Regionserver UI > > > Key: HBASE-14986 > URL: https://issues.apache.org/jira/browse/HBASE-14986 > Project: HBase > Issue Type: New Feature > Components: regionserver, UI >Reporter: Govind Kamat >Assignee: Govind Kamat >Priority: Minor > Attachments: rs.jpg > > > The regionserver UI shows the start and end keys for each region, but it is > hard for users to deduce the byte range between these. > Unbalanced ranges can potentially lead to hot spots. Giving users an easy > way to view the key range for each region and how these ranges compare across > regions may help alleviate this problem. > Key ranges may become non-uniform as a consequence of auto-splitting or due > to users setting them up incorrectly to begin with. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14944) Minimize or eliminate source incompatible changes due to HBASE-14605, HBASE-14631, and HBASE-14655
[
https://issues.apache.org/jira/browse/HBASE-14944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059174#comment-15059174
]
Nick Dimiduk commented on HBASE-14944:
--
Ping also [~jamestaylor], [~chrajeshbab...@gmail.com], [~eowhadi] -- you fellas
have any concern over these changes in the coproc APIs?
> Minimize or eliminate source incompatible changes due to HBASE-14605,
> HBASE-14631, and HBASE-14655
> --
>
> Key: HBASE-14944
> URL: https://issues.apache.org/jira/browse/HBASE-14944
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Purtell
>Priority: Blocker
> Fix For: 1.1.3, 1.0.4
>
>
> Minimize or eliminate source incompatible changes due to HBASE-14605,
> HBASE-14631, and HBASE-14655. The changes are due to abstract method
> additions to carry the correct (not current) {{User}} through to where
> authoritative decisions or audit is performed.
> HBASE-14605 introduces source incompatible changes to the SplitTransaction
> interface:
> - Adds abstract method execute(Server, RegionServerServices, User)
> - Adds abstract method rollback(Server, RegionServerServices, User)
> HBASE-14631 introduces source incompatible changes to the
> RegionMergeTransaction interface:
> - Adds abstract method execute(Server, RegionServerServices, User)
> - Adds abstract method rollback(Server, RegionServerServices, User)
> HBASE-14655 introduces source incompatible changes to the Store interface:
> - Adds abstract method compact(CompactionContext,
> CompactionThroughputController, User)
> - Adds abstract method requestCompaction( int, CompactionRequest, User)
> Default implementations are provided for binary compatibility but
> implementors of these interface won't recompile until implementations of the
> new methods are added.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14983) Create metrics for per block type hit/miss ratios
[ https://issues.apache.org/jira/browse/HBASE-14983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-14983: -- Attachment: HBASE-14983-v1.patch Yeah it's awful copy paste code, however it is always O(1). I really want a pre-processor. > Create metrics for per block type hit/miss ratios > - > > Key: HBASE-14983 > URL: https://issues.apache.org/jira/browse/HBASE-14983 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-14983-v1.patch, HBASE-14983.patch, Screen Shot > 2015-12-15 at 3.33.09 PM.png > > > Missing a root index block is worse than missing a data block. We should know > the difference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14986) Display the key range for each region in the Regionserver UI
[ https://issues.apache.org/jira/browse/HBASE-14986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059165#comment-15059165 ] Elliott Clark commented on HBASE-14986: --- I'm not sure that's a very usable piece of information for them. Also for users with variable length keys it will be a very misleading number. > Display the key range for each region in the Regionserver UI > > > Key: HBASE-14986 > URL: https://issues.apache.org/jira/browse/HBASE-14986 > Project: HBase > Issue Type: New Feature > Components: regionserver, UI >Reporter: Govind Kamat >Assignee: Govind Kamat >Priority: Minor > Attachments: rs.jpg > > > The regionserver UI shows the start and end keys for each region, but it is > hard for users to deduce the byte range between these. > Unbalanced ranges can potentially lead to hot spots. Giving users an easy > way to view the key range for each region and how these ranges compare across > regions may help alleviate this problem. > Key ranges may become non-uniform as a consequence of auto-splitting or due > to users setting them up incorrectly to begin with. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-14849: --- Status: Open (was: Patch Available) > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > Attachments: HBASE-14849.patch > > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-14849: --- Status: Patch Available (was: Open) > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > Attachments: HBASE-14849.patch > > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-14849: --- Attachment: HBASE-14849.patch > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > Attachments: HBASE-14849.patch > > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[ https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059155#comment-15059155 ] Hudson commented on HBASE-14977: SUCCESS: Integrated in HBase-1.2-IT #342 (See [https://builds.apache.org/job/HBase-1.2-IT/342/]) HBASE-14977 ChoreService.shutdown may result in (enis: rev ef19c2147d9c8cf88d8145bfe9e9bf25ff01206d) * hbase-common/src/main/java/org/apache/hadoop/hbase/ChoreService.java > ChoreService.shutdown may result in ConcurrentModificationException > --- > > Key: HBASE-14977 > URL: https://issues.apache.org/jira/browse/HBASE-14977 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4 > > Attachments: HBASE-14977-v1.patch > > > As seen in this test: > https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/425/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt > We need to make shutdown method synchronized to avoid this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14947) WALProcedureStore improvements
[ https://issues.apache.org/jira/browse/HBASE-14947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059157#comment-15059157 ] Hudson commented on HBASE-14947: SUCCESS: Integrated in HBase-1.2-IT #342 (See [https://builds.apache.org/job/HBase-1.2-IT/342/]) HBASE-14947 WALProcedureStore improvements (matteo.bertozzi: rev e8698da2a4b6f7b419478485617ec22933f22f3c) * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/wal/TestWALProcedureStore.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureRecovery.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/ProcedureStoreTracker.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/TestProcedureStoreTracker.java > WALProcedureStore improvements > -- > > Key: HBASE-14947 > URL: https://issues.apache.org/jira/browse/HBASE-14947 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Ashu Pachauri >Assignee: Matteo Bertozzi >Priority: Blocker > Fix For: 2.0.0, 1.2.0, 1.1.3 > > Attachments: HBASE-14947-v0.patch, HBASE-14947-v1.patch > > > We ended up with a deadlock in HBASE-14943, with the storeTracker and lock > acquired in reverse order by syncLoop() and insert/update/delete. In the > syncLoop() with don't need the lock when we try to roll or removeInactive. > also we can move the insert/update/delete tracker check in the syncLoop > avoiding to the extra lock operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14968) ConcurrentModificationException in region close resulting in the region staying in closing state
[
https://issues.apache.org/jira/browse/HBASE-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059156#comment-15059156
]
Hudson commented on HBASE-14968:
SUCCESS: Integrated in HBase-1.2-IT #342 (See
[https://builds.apache.org/job/HBase-1.2-IT/342/])
HBASE-14968 ConcurrentModificationException in region close resulting (enis:
rev f391b137111d9467b0f3a2930081d41eb2c5f5df)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/executor/TestExecutorService.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/RegionReplicaFlushHandler.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
> ConcurrentModificationException in region close resulting in the region
> staying in closing state
>
>
> Key: HBASE-14968
> URL: https://issues.apache.org/jira/browse/HBASE-14968
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4, 1.0.4
>
> Attachments: hbase-14968_v1.patch
>
>
> We have seen this in our tests. The region gets closed, but the region close
> handler gets an unexpected exception causing the region to stay in closing
> state until RS is restarted.
> The Phoenix and security coprocessors were loaded in that region. Here is the
> stack trace:
> {code}
> java.util.ConcurrentModificationException
> at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
> at java.util.ArrayList$Itr.next(ArrayList.java:851)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.shutdown(CoprocessorHost.java:442)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionEnvironment.shutdown(RegionCoprocessorHost.java:155)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.shutdown(CoprocessorHost.java:272)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$5.postEnvCall(RegionCoprocessorHost.java:496)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1761)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postClose(RegionCoprocessorHost.java:489)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1502)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1349)
> at
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:138)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-14849: --- Attachment: (was: HBASE-14849.patch) > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-14849: --- Status: Patch Available (was: Open) > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > Attachments: HBASE-14849.patch > > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-14849: --- Attachment: HBASE-14849.patch Migrate hbase configuration to SparkConf, and some cleanup. > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > Attachments: HBASE-14849.patch > > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[ https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059133#comment-15059133 ] Hudson commented on HBASE-14977: SUCCESS: Integrated in HBase-1.1-JDK8 #1708 (See [https://builds.apache.org/job/HBase-1.1-JDK8/1708/]) HBASE-14977 ChoreService.shutdown may result in (enis: rev 2c9361602ca4ed12746d992c1161d2dbc08d8d64) * hbase-common/src/main/java/org/apache/hadoop/hbase/ChoreService.java > ChoreService.shutdown may result in ConcurrentModificationException > --- > > Key: HBASE-14977 > URL: https://issues.apache.org/jira/browse/HBASE-14977 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4 > > Attachments: HBASE-14977-v1.patch > > > As seen in this test: > https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/425/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt > We need to make shutdown method synchronized to avoid this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14947) WALProcedureStore improvements
[ https://issues.apache.org/jira/browse/HBASE-14947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059135#comment-15059135 ] Hudson commented on HBASE-14947: SUCCESS: Integrated in HBase-1.1-JDK8 #1708 (See [https://builds.apache.org/job/HBase-1.1-JDK8/1708/]) HBASE-14947 WALProcedureStore improvements (matteo.bertozzi: rev 20b6dba997acd973fb8bd74666c42165bc715dbc) * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/ProcedureStoreTracker.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/wal/TestWALProcedureStore.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/TestProcedureStoreTracker.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java > WALProcedureStore improvements > -- > > Key: HBASE-14947 > URL: https://issues.apache.org/jira/browse/HBASE-14947 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Ashu Pachauri >Assignee: Matteo Bertozzi >Priority: Blocker > Fix For: 2.0.0, 1.2.0, 1.1.3 > > Attachments: HBASE-14947-v0.patch, HBASE-14947-v1.patch > > > We ended up with a deadlock in HBASE-14943, with the storeTracker and lock > acquired in reverse order by syncLoop() and insert/update/delete. In the > syncLoop() with don't need the lock when we try to roll or removeInactive. > also we can move the insert/update/delete tracker check in the syncLoop > avoiding to the extra lock operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14968) ConcurrentModificationException in region close resulting in the region staying in closing state
[
https://issues.apache.org/jira/browse/HBASE-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059134#comment-15059134
]
Hudson commented on HBASE-14968:
SUCCESS: Integrated in HBase-1.1-JDK8 #1708 (See
[https://builds.apache.org/job/HBase-1.1-JDK8/1708/])
HBASE-14968 ConcurrentModificationException in region close resulting in (enis:
rev 6ae430f59649c057efbef9ee23439f2f729e3fd8)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/RegionReplicaFlushHandler.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/executor/TestExecutorService.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
> ConcurrentModificationException in region close resulting in the region
> staying in closing state
>
>
> Key: HBASE-14968
> URL: https://issues.apache.org/jira/browse/HBASE-14968
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4, 1.0.4
>
> Attachments: hbase-14968_v1.patch
>
>
> We have seen this in our tests. The region gets closed, but the region close
> handler gets an unexpected exception causing the region to stay in closing
> state until RS is restarted.
> The Phoenix and security coprocessors were loaded in that region. Here is the
> stack trace:
> {code}
> java.util.ConcurrentModificationException
> at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
> at java.util.ArrayList$Itr.next(ArrayList.java:851)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.shutdown(CoprocessorHost.java:442)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionEnvironment.shutdown(RegionCoprocessorHost.java:155)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.shutdown(CoprocessorHost.java:272)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$5.postEnvCall(RegionCoprocessorHost.java:496)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1761)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postClose(RegionCoprocessorHost.java:489)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1502)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1349)
> at
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:138)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14030) HBase Backup/Restore Phase 1
[ https://issues.apache.org/jira/browse/HBASE-14030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14030: -- Attachment: (was: HBASE-14030-v20.patch) > HBase Backup/Restore Phase 1 > > > Key: HBASE-14030 > URL: https://issues.apache.org/jira/browse/HBASE-14030 > Project: HBase > Issue Type: Umbrella >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-14030-v0.patch, HBASE-14030-v1.patch, > HBASE-14030-v10.patch, HBASE-14030-v11.patch, HBASE-14030-v12.patch, > HBASE-14030-v13.patch, HBASE-14030-v14.patch, HBASE-14030-v15.patch, > HBASE-14030-v17.patch, HBASE-14030-v18.patch, HBASE-14030-v2.patch, > HBASE-14030-v20.patch, HBASE-14030-v3.patch, HBASE-14030-v4.patch, > HBASE-14030-v5.patch, HBASE-14030-v6.patch, HBASE-14030-v7.patch, > HBASE-14030-v8.patch > > > This is the umbrella ticket for Backup/Restore Phase 1. See HBASE-7912 design > doc for the phase description. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14030) HBase Backup/Restore Phase 1
[ https://issues.apache.org/jira/browse/HBASE-14030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14030: -- Status: Patch Available (was: Open) > HBase Backup/Restore Phase 1 > > > Key: HBASE-14030 > URL: https://issues.apache.org/jira/browse/HBASE-14030 > Project: HBase > Issue Type: Umbrella >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-14030-v0.patch, HBASE-14030-v1.patch, > HBASE-14030-v10.patch, HBASE-14030-v11.patch, HBASE-14030-v12.patch, > HBASE-14030-v13.patch, HBASE-14030-v14.patch, HBASE-14030-v15.patch, > HBASE-14030-v17.patch, HBASE-14030-v18.patch, HBASE-14030-v2.patch, > HBASE-14030-v20.patch, HBASE-14030-v3.patch, HBASE-14030-v4.patch, > HBASE-14030-v5.patch, HBASE-14030-v6.patch, HBASE-14030-v7.patch, > HBASE-14030-v8.patch > > > This is the umbrella ticket for Backup/Restore Phase 1. See HBASE-7912 design > doc for the phase description. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14030) HBase Backup/Restore Phase 1
[ https://issues.apache.org/jira/browse/HBASE-14030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14030: -- Attachment: HBASE-14030-v20.patch > HBase Backup/Restore Phase 1 > > > Key: HBASE-14030 > URL: https://issues.apache.org/jira/browse/HBASE-14030 > Project: HBase > Issue Type: Umbrella >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-14030-v0.patch, HBASE-14030-v1.patch, > HBASE-14030-v10.patch, HBASE-14030-v11.patch, HBASE-14030-v12.patch, > HBASE-14030-v13.patch, HBASE-14030-v14.patch, HBASE-14030-v15.patch, > HBASE-14030-v17.patch, HBASE-14030-v18.patch, HBASE-14030-v2.patch, > HBASE-14030-v20.patch, HBASE-14030-v3.patch, HBASE-14030-v4.patch, > HBASE-14030-v5.patch, HBASE-14030-v6.patch, HBASE-14030-v7.patch, > HBASE-14030-v8.patch > > > This is the umbrella ticket for Backup/Restore Phase 1. See HBASE-7912 design > doc for the phase description. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14030) HBase Backup/Restore Phase 1
[ https://issues.apache.org/jira/browse/HBASE-14030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14030: -- Status: Open (was: Patch Available) > HBase Backup/Restore Phase 1 > > > Key: HBASE-14030 > URL: https://issues.apache.org/jira/browse/HBASE-14030 > Project: HBase > Issue Type: Umbrella >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-14030-v0.patch, HBASE-14030-v1.patch, > HBASE-14030-v10.patch, HBASE-14030-v11.patch, HBASE-14030-v12.patch, > HBASE-14030-v13.patch, HBASE-14030-v14.patch, HBASE-14030-v15.patch, > HBASE-14030-v17.patch, HBASE-14030-v18.patch, HBASE-14030-v2.patch, > HBASE-14030-v20.patch, HBASE-14030-v3.patch, HBASE-14030-v4.patch, > HBASE-14030-v5.patch, HBASE-14030-v6.patch, HBASE-14030-v7.patch, > HBASE-14030-v8.patch > > > This is the umbrella ticket for Backup/Restore Phase 1. See HBASE-7912 design > doc for the phase description. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14968) ConcurrentModificationException in region close resulting in the region staying in closing state
[
https://issues.apache.org/jira/browse/HBASE-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059114#comment-15059114
]
Hudson commented on HBASE-14968:
FAILURE: Integrated in HBase-1.1-JDK7 #1620 (See
[https://builds.apache.org/job/HBase-1.1-JDK7/1620/])
HBASE-14968 ConcurrentModificationException in region close resulting in (enis:
rev 6ae430f59649c057efbef9ee23439f2f729e3fd8)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/executor/TestExecutorService.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/RegionReplicaFlushHandler.java
> ConcurrentModificationException in region close resulting in the region
> staying in closing state
>
>
> Key: HBASE-14968
> URL: https://issues.apache.org/jira/browse/HBASE-14968
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4, 1.0.4
>
> Attachments: hbase-14968_v1.patch
>
>
> We have seen this in our tests. The region gets closed, but the region close
> handler gets an unexpected exception causing the region to stay in closing
> state until RS is restarted.
> The Phoenix and security coprocessors were loaded in that region. Here is the
> stack trace:
> {code}
> java.util.ConcurrentModificationException
> at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
> at java.util.ArrayList$Itr.next(ArrayList.java:851)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.shutdown(CoprocessorHost.java:442)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionEnvironment.shutdown(RegionCoprocessorHost.java:155)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.shutdown(CoprocessorHost.java:272)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$5.postEnvCall(RegionCoprocessorHost.java:496)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1761)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postClose(RegionCoprocessorHost.java:489)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1502)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1349)
> at
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:138)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14978) Don't allow Multi to retain too many blocks
[
https://issues.apache.org/jira/browse/HBASE-14978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059111#comment-15059111
]
Hadoop QA commented on HBASE-14978:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12777817/HBASE-14978-v3.patch
against master branch at commit abe30b52a8036078f833dc5b3d2b03daa2e93dfc.
ATTACHMENT ID: 12777817
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16865//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16865//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16865//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16865//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16865//console
This message is automatically generated.
> Don't allow Multi to retain too many blocks
> ---
>
> Key: HBASE-14978
> URL: https://issues.apache.org/jira/browse/HBASE-14978
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0, 1.2.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Critical
> Attachments: HBASE-14978-v1.patch, HBASE-14978-v2.patch,
> HBASE-14978-v3.patch, HBASE-14978.patch
>
>
> Scans and Multi's have limits on the total size of cells that can be
> returned. However if those requests are not all pointing at the same blocks
> then the KeyValues can keep alive a lot more data than their size.
> Take the following example:
> A multi with a list of 1 gets to a fat row. Each column being returned in
> in a different block. Each column is small 32 bytes or so.
> So the total cell size will be 32 * 1 = ~320kb. However if each block is
> 128k then total retained heap size will be almost 2gigs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14947) WALProcedureStore improvements
[ https://issues.apache.org/jira/browse/HBASE-14947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059115#comment-15059115 ] Hudson commented on HBASE-14947: FAILURE: Integrated in HBase-1.1-JDK7 #1620 (See [https://builds.apache.org/job/HBase-1.1-JDK7/1620/]) HBASE-14947 WALProcedureStore improvements (matteo.bertozzi: rev 20b6dba997acd973fb8bd74666c42165bc715dbc) * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/wal/TestWALProcedureStore.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/ProcedureStoreTracker.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/TestProcedureStoreTracker.java > WALProcedureStore improvements > -- > > Key: HBASE-14947 > URL: https://issues.apache.org/jira/browse/HBASE-14947 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Ashu Pachauri >Assignee: Matteo Bertozzi >Priority: Blocker > Fix For: 2.0.0, 1.2.0, 1.1.3 > > Attachments: HBASE-14947-v0.patch, HBASE-14947-v1.patch > > > We ended up with a deadlock in HBASE-14943, with the storeTracker and lock > acquired in reverse order by syncLoop() and insert/update/delete. In the > syncLoop() with don't need the lock when we try to roll or removeInactive. > also we can move the insert/update/delete tracker check in the syncLoop > avoiding to the extra lock operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[ https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059113#comment-15059113 ] Hudson commented on HBASE-14977: FAILURE: Integrated in HBase-1.1-JDK7 #1620 (See [https://builds.apache.org/job/HBase-1.1-JDK7/1620/]) HBASE-14977 ChoreService.shutdown may result in (enis: rev 2c9361602ca4ed12746d992c1161d2dbc08d8d64) * hbase-common/src/main/java/org/apache/hadoop/hbase/ChoreService.java > ChoreService.shutdown may result in ConcurrentModificationException > --- > > Key: HBASE-14977 > URL: https://issues.apache.org/jira/browse/HBASE-14977 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4 > > Attachments: HBASE-14977-v1.patch > > > As seen in this test: > https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/425/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt > We need to make shutdown method synchronized to avoid this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14986) Display the key range for each region in the Regionserver UI
[ https://issues.apache.org/jira/browse/HBASE-14986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Govind Kamat updated HBASE-14986: - Attachment: rs.jpg > Display the key range for each region in the Regionserver UI > > > Key: HBASE-14986 > URL: https://issues.apache.org/jira/browse/HBASE-14986 > Project: HBase > Issue Type: New Feature > Components: regionserver, UI >Reporter: Govind Kamat >Assignee: Govind Kamat >Priority: Minor > Attachments: rs.jpg > > > The regionserver UI shows the start and end keys for each region, but it is > hard for users to deduce the byte range between these. > Unbalanced ranges can potentially lead to hot spots. Giving users an easy > way to view the key range for each region and how these ranges compare across > regions may help alleviate this problem. > Key ranges may become non-uniform as a consequence of auto-splitting or due > to users setting them up incorrectly to begin with. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14983) Create metrics for per block type hit/miss ratios
[ https://issues.apache.org/jira/browse/HBASE-14983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-14983: -- Attachment: HBASE-14983.patch First cut at this. > Create metrics for per block type hit/miss ratios > - > > Key: HBASE-14983 > URL: https://issues.apache.org/jira/browse/HBASE-14983 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-14983.patch, Screen Shot 2015-12-15 at 3.33.09 > PM.png > > > Missing a root index block is worse than missing a data block. We should know > the difference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14983) Create metrics for per block type hit/miss ratios
[ https://issues.apache.org/jira/browse/HBASE-14983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-14983: -- Attachment: Screen Shot 2015-12-15 at 3.33.09 PM.png Results > Create metrics for per block type hit/miss ratios > - > > Key: HBASE-14983 > URL: https://issues.apache.org/jira/browse/HBASE-14983 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-14983.patch, Screen Shot 2015-12-15 at 3.33.09 > PM.png > > > Missing a root index block is worse than missing a data block. We should know > the difference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14986) Display the key range for each region in the Regionserver UI
Govind Kamat created HBASE-14986: Summary: Display the key range for each region in the Regionserver UI Key: HBASE-14986 URL: https://issues.apache.org/jira/browse/HBASE-14986 Project: HBase Issue Type: New Feature Components: regionserver, UI Reporter: Govind Kamat Assignee: Govind Kamat Priority: Minor The regionserver UI shows the start and end keys for each region, but it is hard for users to deduce the byte range between these. Unbalanced ranges can potentially lead to hot spots. Giving users an easy way to view the key range for each region and how these ranges compare across regions may help alleviate this problem. Key ranges may become non-uniform as a consequence of auto-splitting or due to users setting them up incorrectly to begin with. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14947) WALProcedureStore improvements
[ https://issues.apache.org/jira/browse/HBASE-14947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059078#comment-15059078 ] Hudson commented on HBASE-14947: FAILURE: Integrated in HBase-1.3-IT #376 (See [https://builds.apache.org/job/HBase-1.3-IT/376/]) HBASE-14947 WALProcedureStore improvements (matteo.bertozzi: rev bf7c36fccac5477d08e296ad93671d2c30ae2dc8) * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/ProcedureStoreTracker.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/wal/TestWALProcedureStore.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureRecovery.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/TestProcedureStoreTracker.java > WALProcedureStore improvements > -- > > Key: HBASE-14947 > URL: https://issues.apache.org/jira/browse/HBASE-14947 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Ashu Pachauri >Assignee: Matteo Bertozzi >Priority: Blocker > Fix For: 2.0.0, 1.2.0, 1.1.3 > > Attachments: HBASE-14947-v0.patch, HBASE-14947-v1.patch > > > We ended up with a deadlock in HBASE-14943, with the storeTracker and lock > acquired in reverse order by syncLoop() and insert/update/delete. In the > syncLoop() with don't need the lock when we try to roll or removeInactive. > also we can move the insert/update/delete tracker check in the syncLoop > avoiding to the extra lock operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[ https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059076#comment-15059076 ] Hudson commented on HBASE-14977: FAILURE: Integrated in HBase-1.3-IT #376 (See [https://builds.apache.org/job/HBase-1.3-IT/376/]) HBASE-14977 ChoreService.shutdown may result in (enis: rev 31d73a4bdec59ba2ca6bdc04d881f2bc3726e903) * hbase-common/src/main/java/org/apache/hadoop/hbase/ChoreService.java > ChoreService.shutdown may result in ConcurrentModificationException > --- > > Key: HBASE-14977 > URL: https://issues.apache.org/jira/browse/HBASE-14977 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4 > > Attachments: HBASE-14977-v1.patch > > > As seen in this test: > https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/425/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt > We need to make shutdown method synchronized to avoid this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14968) ConcurrentModificationException in region close resulting in the region staying in closing state
[
https://issues.apache.org/jira/browse/HBASE-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059077#comment-15059077
]
Hudson commented on HBASE-14968:
FAILURE: Integrated in HBase-1.3-IT #376 (See
[https://builds.apache.org/job/HBase-1.3-IT/376/])
HBASE-14968 ConcurrentModificationException in region close resulting in (enis:
rev fbad4d2466441d09f45d6fd42486b5e6dc00a893)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/executor/TestExecutorService.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/RegionReplicaFlushHandler.java
> ConcurrentModificationException in region close resulting in the region
> staying in closing state
>
>
> Key: HBASE-14968
> URL: https://issues.apache.org/jira/browse/HBASE-14968
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4, 1.0.4
>
> Attachments: hbase-14968_v1.patch
>
>
> We have seen this in our tests. The region gets closed, but the region close
> handler gets an unexpected exception causing the region to stay in closing
> state until RS is restarted.
> The Phoenix and security coprocessors were loaded in that region. Here is the
> stack trace:
> {code}
> java.util.ConcurrentModificationException
> at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
> at java.util.ArrayList$Itr.next(ArrayList.java:851)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.shutdown(CoprocessorHost.java:442)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionEnvironment.shutdown(RegionCoprocessorHost.java:155)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.shutdown(CoprocessorHost.java:272)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$5.postEnvCall(RegionCoprocessorHost.java:496)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1761)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postClose(RegionCoprocessorHost.java:489)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1502)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1349)
> at
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:138)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13070) Fix possibly zero length family and qualifier is TestCacheOnWrite
[ https://issues.apache.org/jira/browse/HBASE-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-13070: - Summary: Fix possibly zero length family and qualifier is TestCacheOnWrite (was: Fix TestCacheOnWrite) > Fix possibly zero length family and qualifier is TestCacheOnWrite > - > > Key: HBASE-13070 > URL: https://issues.apache.org/jira/browse/HBASE-13070 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11 > > Attachments: HBASE-13070-0.98.patch, HBASE-13070.patch, > HBASE-13070_1.patch, HBASE-13070_2.patch > > > TestCacheOnWrite uses TestHFileWriterV2.randomOrderedKey to generate a random > byte array, then use first 32 bytes as row and remaining part as family and > qualifier. But TestHFileWriterV2.randomOrderedKey may return a byte array > only contains 32 bytes, so there will be zero length family and qualifier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14955) OOME: cannot create native thread is back
[ https://issues.apache.org/jira/browse/HBASE-14955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059063#comment-15059063 ] stack commented on HBASE-14955: --- Patch looks great to me. Submitted it to get a run in. > OOME: cannot create native thread is back > - > > Key: HBASE-14955 > URL: https://issues.apache.org/jira/browse/HBASE-14955 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: Heng Chen > Attachments: HBASE-14955-branch-1.patch > > > This fail is OOME cannot create native thread. Two MR jobs fail: > > org.apache.hadoop.hbase.mapreduce.TestImportTSVWithVisibilityLabels.org.apache.hadoop.hbase.mapreduce.TestImportTSVWithVisibilityLabels46 > ms1 > org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan1.org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan142 > ms1 > Was running 1.3 tests on H0. > Could try and purge resources used by these tests. > Making issue in meantime to keep an eye on it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14955) OOME: cannot create native thread is back
[ https://issues.apache.org/jira/browse/HBASE-14955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14955: -- Assignee: Heng Chen Status: Patch Available (was: Open) > OOME: cannot create native thread is back > - > > Key: HBASE-14955 > URL: https://issues.apache.org/jira/browse/HBASE-14955 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: Heng Chen > Attachments: HBASE-14955-branch-1.patch > > > This fail is OOME cannot create native thread. Two MR jobs fail: > > org.apache.hadoop.hbase.mapreduce.TestImportTSVWithVisibilityLabels.org.apache.hadoop.hbase.mapreduce.TestImportTSVWithVisibilityLabels46 > ms1 > org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan1.org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan142 > ms1 > Was running 1.3 tests on H0. > Could try and purge resources used by these tests. > Making issue in meantime to keep an eye on it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14985) TimeRange constructors should set allTime when appropriate
Geoffrey Jacoby created HBASE-14985: --- Summary: TimeRange constructors should set allTime when appropriate Key: HBASE-14985 URL: https://issues.apache.org/jira/browse/HBASE-14985 Project: HBase Issue Type: Improvement Affects Versions: 0.98.16.1, 1.1.2 Reporter: Geoffrey Jacoby Assignee: Geoffrey Jacoby Priority: Minor The default TimeRange constructor creates a range from 0 to Long.MAX_VALUE and sets an allTime flag to true. This flag allows some performance optimizations when comparing or using TimeRanges. This flag is not set, however, if you call "new TimeRange(0L)" or "new TimeRange(0L, Long.MAX_VALUE)", even though both of these create a logically equivalent TimeRange to "new TimeRange()". Since TimeRanges are immutable and detecting this condition is trivial, we should set the flag automatically in the explicit constructors when appropriate. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14968) ConcurrentModificationException in region close resulting in the region staying in closing state
[
https://issues.apache.org/jira/browse/HBASE-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059006#comment-15059006
]
Hudson commented on HBASE-14968:
SUCCESS: Integrated in HBase-1.3 #438 (See
[https://builds.apache.org/job/HBase-1.3/438/])
HBASE-14968 ConcurrentModificationException in region close resulting in (enis:
rev fbad4d2466441d09f45d6fd42486b5e6dc00a893)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/executor/TestExecutorService.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/RegionReplicaFlushHandler.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
> ConcurrentModificationException in region close resulting in the region
> staying in closing state
>
>
> Key: HBASE-14968
> URL: https://issues.apache.org/jira/browse/HBASE-14968
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4, 1.0.4
>
> Attachments: hbase-14968_v1.patch
>
>
> We have seen this in our tests. The region gets closed, but the region close
> handler gets an unexpected exception causing the region to stay in closing
> state until RS is restarted.
> The Phoenix and security coprocessors were loaded in that region. Here is the
> stack trace:
> {code}
> java.util.ConcurrentModificationException
> at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
> at java.util.ArrayList$Itr.next(ArrayList.java:851)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.shutdown(CoprocessorHost.java:442)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionEnvironment.shutdown(RegionCoprocessorHost.java:155)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.shutdown(CoprocessorHost.java:272)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$5.postEnvCall(RegionCoprocessorHost.java:496)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1761)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postClose(RegionCoprocessorHost.java:489)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1502)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1349)
> at
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:138)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14795) Enhance the spark-hbase scan operations
[ https://issues.apache.org/jira/browse/HBASE-14795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15058997#comment-15058997 ] Zhan Zhang commented on HBASE-14795: [~jmhsieh] Thanks for bring this up. I am working on HBASE-14849, and also doing some cleanup work, which will also fix following warnings. warning: there were 3 feature warning(s); re-run with -feature for details Regarding below warning, it is a legacy one and HBASE-14159 has already open for it. warning: Class org.apache.hadoop.mapred.MiniMRCluster not found - continuing with a stub. > Enhance the spark-hbase scan operations > --- > > Key: HBASE-14795 > URL: https://issues.apache.org/jira/browse/HBASE-14795 > Project: HBase > Issue Type: Improvement >Reporter: Ted Malaska >Assignee: Zhan Zhang >Priority: Minor > Fix For: 2.0.0 > > Attachments: > 0001-HBASE-14795-Enhance-the-spark-hbase-scan-operations.patch, > HBASE-14795-1.patch, HBASE-14795-2.patch, HBASE-14795-3.patch, > HBASE-14795-4.patch > > > This is a sub-jira of HBASE-14789. This jira is to focus on the replacement > of TableInputFormat for a more custom scan implementation that will make the > following use case more effective. > Use case: > In the case you have multiple scan ranges on a single table with in a single > query. TableInputFormat will scan the the outer range of the scan start and > end range where this implementation can be more pointed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14984) Allow memcached block cache to set optimze to false
[ https://issues.apache.org/jira/browse/HBASE-14984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-14984: -- Component/s: BlockCache > Allow memcached block cache to set optimze to false > --- > > Key: HBASE-14984 > URL: https://issues.apache.org/jira/browse/HBASE-14984 > Project: HBase > Issue Type: Improvement > Components: BlockCache >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14984.patch > > > In order to keep latency consistent it might not be good to allow the spy > memcached client to optimize. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14984) Allow memcached block cache to set optimze to false
[ https://issues.apache.org/jira/browse/HBASE-14984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-14984: -- Fix Version/s: 1.3.0 1.2.0 2.0.0 Affects Version/s: 1.3.0 1.2.0 2.0.0 Status: Patch Available (was: Open) > Allow memcached block cache to set optimze to false > --- > > Key: HBASE-14984 > URL: https://issues.apache.org/jira/browse/HBASE-14984 > Project: HBase > Issue Type: Improvement > Components: BlockCache >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14984.patch > > > In order to keep latency consistent it might not be good to allow the spy > memcached client to optimize. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14830) Fix broken links in 0.94 generated docs
[
https://issues.apache.org/jira/browse/HBASE-14830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15058948#comment-15058948
]
Hudson commented on HBASE-14830:
SUCCESS: Integrated in HBase-0.94-security #587 (See
[https://builds.apache.org/job/HBase-0.94-security/587/])
HBASE-14830 Make encoding classes @InterfaceAudience.Private (mstanleyjones:
rev 0f35a32ab123ee299f4aaaea02b4ba2d2b43cff2)
* src/main/java/org/apache/hadoop/hbase/io/encoding/FastDiffDeltaEncoder.java
* src/main/java/org/apache/hadoop/hbase/io/encoding/PrefixKeyDeltaEncoder.java
* src/main/java/org/apache/hadoop/hbase/io/encoding/CopyKeyDataBlockEncoder.java
* src/main/java/org/apache/hadoop/hbase/io/encoding/DiffKeyDeltaEncoder.java
> Fix broken links in 0.94 generated docs
> ---
>
> Key: HBASE-14830
> URL: https://issues.apache.org/jira/browse/HBASE-14830
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 0.94.27
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Fix For: 0.94.28
>
> Attachments: HBASE-14830-v1.patch, HBASE-14830.patch
>
>
> {code}
> host: hbase.apache.org
> date: Mon, 16-Nov-2015 10:56:28 (local)
> Linklint version: 2.3.5_ingo_020
> #
> # ERROR 2 missing html files (cross referenced)
> #
> /0.94/devapidocs/org/apache/hadoop/hbase/avro/generated/HBase.html
> used in 1 file:
> /0.94/devapidocs/org/apache/hadoop/hbase/avro/package-summary.html
> /0.94/devapidocs/src-html/org/apache/hadoop/hbase/io/encoding/BufferedDataBlockEncoder.html
> used in 4 files:
>
> /0.94/devapidocs/org/apache/hadoop/hbase/io/encoding/CopyKeyDataBlockEncoder.html
>
> /0.94/devapidocs/org/apache/hadoop/hbase/io/encoding/DiffKeyDeltaEncoder.html
>
> /0.94/devapidocs/org/apache/hadoop/hbase/io/encoding/FastDiffDeltaEncoder.html
>
> /0.94/devapidocs/org/apache/hadoop/hbase/io/encoding/PrefixKeyDeltaEncoder.html
> #
> # ERROR 1 missing other file (cross referenced)
> #
> /0.94/devapidocs/org/apache/hadoop/hbase/HADOOP-1581
> used in 1 file:
> /0.94/devapidocs/org/apache/hadoop/hbase/HTableDescriptor.html
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
