[jira] [Resolved] (HBASE-17996) HBase master fails to start sometimes on RHEL7
[
https://issues.apache.org/jira/browse/HBASE-17996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh resolved HBASE-17996.
Resolution: Fixed
> HBase master fails to start sometimes on RHEL7
> --
>
> Key: HBASE-17996
> URL: https://issues.apache.org/jira/browse/HBASE-17996
> Project: HBase
> Issue Type: Bug
> Components: master, test
>Reporter: David Knupp
> Attachments:
> hbase-jenkins-master-impala-boost-static-burst-slave-el7-02f4.vpc.cloudera.com.out,
>
> hbase-jenkins-master-impala-boost-static-burst-slave-el7-11ef.vpc.cloudera.com.out
>
>
> Impala includes HBase in its local test environment, and we have found that
> intermittently, the HBase master node fails to start when we are testing on
> RHEL7.
> In these failures, what we typically see in the logs is this:
> {noformat}
> 17/04/29 21:33:47 INFO zookeeper.ClientCnxn: Session establishment complete
> on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x15bbd21b797000a,
> negotiated timeout = 9
> 17/04/29 21:33:47 INFO client.ZooKeeperRegistry: ClusterId read in ZooKeeper
> is null
> 17/04/29 21:33:48 INFO master.ActiveMasterManager: Deleting ZNode for
> /hbase/backup-masters/localhost,16000,1493526758211 from backup master
> directory
> {noformat}
> On a successful startup, the log looks like this:
> {noformat}
> 17/04/16 21:32:29 INFO zookeeper.ClientCnxn: Session establishment complete
> on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x15b7a2ed6860005,
> negotiated timeout = 9
> 17/04/16 21:32:29 INFO client.ZooKeeperRegistry: ClusterId read in ZooKeeper
> is null
> 17/04/16 21:32:30 INFO util.FSUtils: Created version file at
> hdfs://localhost:20500/hbase with version=8
> 17/04/16 21:32:31 INFO master.MasterFileSystem: BOOTSTRAP: creating
> hbase:meta region
> {noformat}
> So the event that we don't see in the failed start up attempts is
> {{master.MasterFileSystem: BOOTSTRAP: creating hbase:meta region}}.
> The full logs will be attached.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-17925) mvn assembly:single fails against hadoop3-alpah2
Jonathan Hsieh created HBASE-17925:
--
Summary: mvn assembly:single fails against hadoop3-alpah2
Key: HBASE-17925
URL: https://issues.apache.org/jira/browse/HBASE-17925
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Hsieh
generally to build tarballs against hadoop3 alpha2 we'd use
{code}
mvn -Dhadoop.profile=3.0 -Dhadoop.three-version=3.0.0-alpha2 assembly:single
-DskipTests
{code}
This currently fails.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (HBASE-17922) TestRegionServerHostname always fails against hadoop 3.0.0-alpha2
Jonathan Hsieh created HBASE-17922: -- Summary: TestRegionServerHostname always fails against hadoop 3.0.0-alpha2 Key: HBASE-17922 URL: https://issues.apache.org/jira/browse/HBASE-17922 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17921) TestBlockReorder always fails against hadoop3.0.0-alpha2
Jonathan Hsieh created HBASE-17921:
--
Summary: TestBlockReorder always fails against hadoop3.0.0-alpha2
Key: HBASE-17921
URL: https://issues.apache.org/jira/browse/HBASE-17921
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Hsieh
{code}
java.lang.ExceptionInInitializerError: null
at
org.apache.hadoop.hbase.fs.TestBlockReorder.(TestBlockReorder.java:83)
testBlockLocation(org.apache.hadoop.hbase.fs.TestBlockReorder) Time elapsed:
0.023 sec <<< ERROR!
java.lang.NoClassDefFoundError: Could not initialize class
org.apache.hadoop.hbase.fs.TestBlockReorder
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.junit.runners.BlockJUnit4ClassRunner.createTest(BlockJUnit4ClassRunner.java:217)
at
org.junit.runners.BlockJUnit4ClassRunner$1.runReflectiveCall(BlockJUnit4ClassRunner.java:266)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.BlockJUnit4ClassRunner.methodBlock(BlockJUnit4ClassRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
testHBaseCluster(org.apache.hadoop.hbase.fs.TestBlockReorder) Time elapsed:
0.011 sec <<< ERROR!
java.lang.NoClassDefFoundError: Could not initialize class
org.apache.hadoop.hbase.fs.TestBlockReorder
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.junit.runners.BlockJUnit4ClassRunner.createTest(BlockJUnit4ClassRunner.java:217)
at
org.junit.runners.BlockJUnit4ClassRunner$1.runReflectiveCall(BlockJUnit4ClassRunner.java:266)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.BlockJUnit4ClassRunner.methodBlock(BlockJUnit4ClassRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
[jira] [Created] (HBASE-17920) TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2
Jonathan Hsieh created HBASE-17920: -- Summary: TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2 Key: HBASE-17920 URL: https://issues.apache.org/jira/browse/HBASE-17920 Project: HBase Issue Type: Sub-task Components: hadoop3 Affects Versions: 2.0.0 Reporter: Jonathan Hsieh TestFSHDFSUtils always fails against hadoop-3.0.0-alpha1 and hadoop-3.0.0-alpha2. This is because HDFS-9427 in those versions change the default nn port from 8020 to 9820. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17919) HBase 2.x over hadoop 3.x umrella
Jonathan Hsieh created HBASE-17919: -- Summary: HBase 2.x over hadoop 3.x umrella Key: HBASE-17919 URL: https://issues.apache.org/jira/browse/HBASE-17919 Project: HBase Issue Type: Umbrella Components: hadoop3 Affects Versions: 2.0.0 Reporter: Jonathan Hsieh We should try to get hadoop 2.x branch working against the recently release hadoop 3.0.0 alphas. These data 3.0.0-alpha2 is the latest. HBASE-16733 and HBASE-17953 got the compile level checks in but we should progress to getting unit tests to pass and a build against hadoop3 up. This umbrella issue will capture issues around this project. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-16763) Remove unintentional dependency on net.sf.ehcache.search.Results
Jonathan Hsieh created HBASE-16763: -- Summary: Remove unintentional dependency on net.sf.ehcache.search.Results Key: HBASE-16763 URL: https://issues.apache.org/jira/browse/HBASE-16763 Project: HBase Issue Type: Bug Reporter: Jonathan Hsieh HBASE-15638 introduced what looks like an inadvertent dependency on net.sf.ehcache.search.Results. This removes it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16733) add hadoop 3.0.0-alpha1 to precommit checks
Jonathan Hsieh created HBASE-16733: -- Summary: add hadoop 3.0.0-alpha1 to precommit checks Key: HBASE-16733 URL: https://issues.apache.org/jira/browse/HBASE-16733 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Fix For: 2.0.0 Been working on getting hadoop3 related build up and running and woudl ike to add a precommit check so that new commits don't break the mvn compile/install. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16728) Add hadoop.profile=3.0 pass to precommit checks.
Jonathan Hsieh created HBASE-16728: -- Summary: Add hadoop.profile=3.0 pass to precommit checks. Key: HBASE-16728 URL: https://issues.apache.org/jira/browse/HBASE-16728 Project: HBase Issue Type: Task Components: build, hadoop3 Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Fix For: 2.0.0 once we get hadoop3 profile compiling (mvn test -DskipTests) and passing licensing muster (mvn install), we should add precommit checks to prevent regressions in these categories -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16712) fix hadoop-3.0 profile mvn install
Jonathan Hsieh created HBASE-16712: -- Summary: fix hadoop-3.0 profile mvn install Key: HBASE-16712 URL: https://issues.apache.org/jira/browse/HBASE-16712 Project: HBase Issue Type: Bug Components: build, hadoop3 Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 2.0.0 After the compile is fixed, mvn install fails due to transitive dependencies coming from hadoop3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16711) Fix hadoop-3.0 profile compile
Jonathan Hsieh created HBASE-16711: -- Summary: Fix hadoop-3.0 profile compile Key: HBASE-16711 URL: https://issues.apache.org/jira/browse/HBASE-16711 Project: HBase Issue Type: Bug Components: hadoop3, build Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 2.0.0 The -Dhadoop.profile=3.0 build is failing currently due to code deprecated in hadoop2 and removed in hadoop3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16669) fix flaky TestAdmin#testmergeRegions
Jonathan Hsieh created HBASE-16669:
--
Summary: fix flaky TestAdmin#testmergeRegions
Key: HBASE-16669
URL: https://issues.apache.org/jira/browse/HBASE-16669
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 2.0.0
Reporter: Jonathan Hsieh
Fix For: 2.0.0
Recent test runs show that this test is failing with these error messages:
{code}
java.lang.AssertionError: expected:<2> but was:<3>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:631)
at
org.apache.hadoop.hbase.client.TestAdmin1.testMergeRegions(TestAdmin1.java:1385)
{code}
or
{code}
java.lang.AssertionError: expected:<1> but was:<2>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:631)
at
org.apache.hadoop.hbase.client.TestAdmin1.testMergeRegions(TestAdmin1.java:1394)
{code}
looking at the code this indicates that merge operation did not complete or
didn't work.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16668) Admin class should have a synchronous Admin#mergeRegions* method
Jonathan Hsieh created HBASE-16668:
--
Summary: Admin class should have a synchronous Admin#mergeRegions*
method
Key: HBASE-16668
URL: https://issues.apache.org/jira/browse/HBASE-16668
Project: HBase
Issue Type: Bug
Components: hbase
Affects Versions: 2.0.0
Reporter: Jonathan Hsieh
Fix For: 2.0.0
In trunk from HBASE-14552, we have deprecated {{void Admin#mergeRegions}} (in
1.x this was an asynchronous call) and replaced it with {{Future
Admin#mergeRegionsAsync}} which is clearly async.
This leaves us only with the async version.
We should have an easy way to make {{mergeRegions}} or an equivalant behave
synchronously.
For normal java Futures, we could just call the future's {{get()}} method.
Unforutnately, the future this method returns doesn't follow java Future
convention and throws Unimplemented operation when a plain {{get()}} is called
and makes the api harder to use and read. We could make this future act more
normally, and have the timeout throw an InterruptedException.
Alternately, we could expose a new method in {{Admin}} that behaves
synchronously such as {{HBaseAdmin#mergeRegionsSync}}. The caveat here is that
we shouldn't use the name {{#mergeRegions}} since it exists in 1.x with async
semantics.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16140) bump owasp.esapi from 2.1.0 to 2.1.0.1
Jonathan Hsieh created HBASE-16140: -- Summary: bump owasp.esapi from 2.1.0 to 2.1.0.1 Key: HBASE-16140 URL: https://issues.apache.org/jira/browse/HBASE-16140 Project: HBase Issue Type: Improvement Components: build Affects Versions: 2.0.0 Reporter: Jonathan Hsieh A small pom change to upgrade the library. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16117) Fix Connection leak in mapred.TableOutputFormat when spark uses it.
Jonathan Hsieh created HBASE-16117: -- Summary: Fix Connection leak in mapred.TableOutputFormat when spark uses it. Key: HBASE-16117 URL: https://issues.apache.org/jira/browse/HBASE-16117 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 2.0.0, 1.3.0, 1.2.2, 1.1.6 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6 Spark seems to instantiate multiple instances of output formats within a single process. When mapred.TableOutputFormat (not mapreduce.TableOutputFormat) is used, this may cause connection leaks that slowly exhaust the cluster's zk connections. This patch fixes that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15791) Improve javadoc in HBASE-15439
Jonathan Hsieh created HBASE-15791: -- Summary: Improve javadoc in HBASE-15439 Key: HBASE-15791 URL: https://issues.apache.org/jira/browse/HBASE-15791 Project: HBase Issue Type: Improvement Components: master Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 2.0.0 Was doing a review of HBASE-15439 and feel that if we had better javadoc about units we would have caught this bug much earlier. Attached patch improves the situation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-15695) Ambari support for bucket caching parameter - hbase.bucketcache.ioengine
[ https://issues.apache.org/jira/browse/HBASE-15695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-15695. Resolution: Invalid closing since it was refiled as an ambari ticket. See link. > Ambari support for bucket caching parameter - hbase.bucketcache.ioengine > > > Key: HBASE-15695 > URL: https://issues.apache.org/jira/browse/HBASE-15695 > Project: HBase > Issue Type: New Feature > Components: BucketCache >Reporter: Nitin Verma >Assignee: Nitin Verma > > In order to utilize SSD as a L2 (bucket) cache, we specify a file location to > hbase.bucketcache.ioengine parameter. > hbase.bucketcache.ioengine=file: > The directory location specified to the above parameter could be non-existent > or non-accessible. > Since ambari-agent runs with root permissions, the proposed change here is to > read the 'hbase.bucketcache.ioengine' parameter from hbase configuration and > see if it points to a file location (by looking at file: prefix). > In case ioengine points to a file, ambari-agent should create the underlying > directory and assign appropriate permissions to hbase:hadoop user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-15468) why Hbase only split regions in one RegionServer
[ https://issues.apache.org/jira/browse/HBASE-15468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-15468. Resolution: Information Provided Assignee: Jonathan Hsieh > why Hbase only split regions in one RegionServer > > > Key: HBASE-15468 > URL: https://issues.apache.org/jira/browse/HBASE-15468 > Project: HBase > Issue Type: Brainstorming > Components: Balancer >Affects Versions: 1.0.2 >Reporter: Jack Wang >Assignee: Jonathan Hsieh >Priority: Minor > Original Estimate: 168h > Remaining Estimate: 168h > > I was writing 300GiB data to my Hbase table user_info, the table I created by > default having only one region. When the writing was going I saw one region > became two regions and more late on it became 8 regions. But my confusion is > that the 8 regions were kept in the same RegionServer. Why Hbase didn't split > the regions to different RegionServer. btw, I had 10 physical RegionsServers > in my Hbase cluster, and the region size I set is 20GiB, Thanks! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15434) [findbugs] Exclude scala generated source and protobuf generated code in hbase-spark module
Jonathan Hsieh created HBASE-15434: -- Summary: [findbugs] Exclude scala generated source and protobuf generated code in hbase-spark module Key: HBASE-15434 URL: https://issues.apache.org/jira/browse/HBASE-15434 Project: HBase Issue Type: Improvement Components: spark Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 2.0.0 Findbugs against Scala compiler generated bytecode is known to generate false positives and spurious warnings. [1] Protobuf generated code has similar issues. This patch excludes these from the hbase-spark module [1] https://github.com/sbt/findbugs4sbt/issues/4 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15310) hbase-spark module has compilation failures with clover profile
Jonathan Hsieh created HBASE-15310: -- Summary: hbase-spark module has compilation failures with clover profile Key: HBASE-15310 URL: https://issues.apache.org/jira/browse/HBASE-15310 Project: HBase Issue Type: Bug Components: spark, test Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 2.0.0 running with a clover profile enabled will fail due to cross compilation ordering issues with the hbase-spark module. Apparently this is a known issue and this page shows a remedy. https://confluence.atlassian.com/display/CLOVERKB/Java-+Scala+cross-compilation+error+-+cannot+find+symbol -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-15286) Revert the API changes to TimeRange constructor and make IA.Private
[ https://issues.apache.org/jira/browse/HBASE-15286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-15286. Resolution: Duplicate Fix Version/s: hbase-15283 Hey [~churromorales], looks like I got a patch in right before you filed this. I'm going to close this out as a dupe. Mind giving the patch over on HBASE-15283 (and HBASE-15284) a review? > Revert the API changes to TimeRange constructor and make IA.Private > --- > > Key: HBASE-15286 > URL: https://issues.apache.org/jira/browse/HBASE-15286 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 1.2.0 >Reporter: churro morales >Assignee: churro morales > Fix For: hbase-15283 > > > Based on the discussion here: > https://mail-archives.apache.org/mod_mbox/hbase-dev/201602.mbox/%3ccan5cbe4rs-2tv3rn1-xhaz0yt3kh3+zkg+8ewk_6kbkfkds...@mail.gmail.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15284) Make TimeRange constructors IA.Private and remove unused TimeRange constructors
Jonathan Hsieh created HBASE-15284: -- Summary: Make TimeRange constructors IA.Private and remove unused TimeRange constructors Key: HBASE-15284 URL: https://issues.apache.org/jira/browse/HBASE-15284 Project: HBase Issue Type: Improvement Affects Versions: 2.0.0 Reporter: Jonathan Hsieh -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15283) Revert to IOException in TimeRange constructor to maintain API compat in 1.x line
Jonathan Hsieh created HBASE-15283:
--
Summary: Revert to IOException in TimeRange constructor to
maintain API compat in 1.x line
Key: HBASE-15283
URL: https://issues.apache.org/jira/browse/HBASE-15283
Project: HBase
Issue Type: Bug
Reporter: Jonathan Hsieh
>From the 1.2 rc3 conversation:
[~busbey]
{quote}
* TimeRange's constructor with a min/max came in HBASE-14355. The
constructor changed from IOException in the case of min > max to
IllegalArgumentException. That's probably a more accurate exception,
but it's a substantial source compat and behavior change for
downstream. I've flagged the issue as incompatible and asked for a
release note. For some reason the javadocs for TimeRange claim it's
"for internal use only" so maybe it shouldn't have been IA.Public in
the first place.
{quote}
Jon:
{quote}
A user could trigger the TimeRange constructed exception change by with the
Get#setTimeRange/Scan#setTimeRange/Increment#setTimeRange apis. The signature
in these classes don't change. (master [1], branch-1.2[2], tag 1.1.3[3])
Doing a little digging, it is worth noting that no public APIs take time range
as an argument (HRegion, TimeRangeTracker do but they are private), however a
few public metrics return it (Get, Scan, Increment). The comment also
basically says don't use the constructor.
{quote}
[1]https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Get.java#L205
[2]https://github.com/apache/hbase/blob/branch-1.2/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Get.java#L195
[3]
https://github.com/apache/hbase/blob/rel/1.1.3/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Get.java#L191
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15282) Bump Spark on Hbase to use Spark 1.6.
Jonathan Hsieh created HBASE-15282: -- Summary: Bump Spark on Hbase to use Spark 1.6. Key: HBASE-15282 URL: https://issues.apache.org/jira/browse/HBASE-15282 Project: HBase Issue Type: Improvement Components: spark Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Fix For: 2.0.0 The latest stable Spark is spark 1.6. [1] Let's bump the version. [1] http://spark.apache.org/news/spark-1-6-0-released.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-15036) Update HBase Spark documentation to include bulk load with thin records
[ https://issues.apache.org/jira/browse/HBASE-15036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-15036. Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.0.0 > Update HBase Spark documentation to include bulk load with thin records > --- > > Key: HBASE-15036 > URL: https://issues.apache.org/jira/browse/HBASE-15036 > Project: HBase > Issue Type: New Feature >Reporter: Ted Malaska >Assignee: Ted Malaska > Fix For: 2.0.0 > > Attachments: HBASE-15036.1.patch, HBASE-15036.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15152) Automatically include prefix-tree module in MR jobs.
Jonathan Hsieh created HBASE-15152:
--
Summary: Automatically include prefix-tree module in MR jobs.
Key: HBASE-15152
URL: https://issues.apache.org/jira/browse/HBASE-15152
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 1.0.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Fix For: 2.0.0, 1.3.0, 1.2.1
I was running some MR jobs tests and ended up with PrefixTreeCodec class not
found in the YarnChildren processes.
{code}
2016-01-21 06:24:26,844 WARN [main] mapreduce.TableMapReduceUtil(785): The
hbase-prefix-tree module jar containing PrefixTreeCodec is not present.
java.lang.ClassNotFoundException:
org.apache.hadoop.hbase.code.prefixtree.PrefixTreeCodec
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
{code}
This is related to HBASE-7434 and HBASE-7936 which address compile time
concerns. This fix makes it so that jar inclusion is done at run time, and
continues if it is not present (for mr unit tests that don't depend on it)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15069) Unify HFile Writer and Reader creation patterns
Jonathan Hsieh created HBASE-15069: -- Summary: Unify HFile Writer and Reader creation patterns Key: HBASE-15069 URL: https://issues.apache.org/jira/browse/HBASE-15069 Project: HBase Issue Type: Bug Components: HFile Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 2.0.0 There are a plethora of different static methods sprinkled through out HStoreFile and HFile, and many tests that have extraneous calls to 'new CacheConfig(conf)' or essentially extraneous FileSystem arguments threaded through out the code. This patch forces all creation to go through, limits the exposure of HFile Reader and Writer Builders, and eliminates all static Builder constructors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15061) Refactor StoreFileScanner creation to builder pattern
Jonathan Hsieh created HBASE-15061:
--
Summary: Refactor StoreFileScanner creation to builder pattern
Key: HBASE-15061
URL: https://issues.apache.org/jira/browse/HBASE-15061
Project: HBase
Issue Type: Improvement
Affects Versions: 2.0.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Fix For: 2.0.0
There are several falvors of calls that creates a list of StoreFileScanners,
and new feature have been added to this recently. This patch converts the
somewhat difficult to read (need to go to javadoc) call:
{code}
// which args are the most relevant to this?
- List sfScanners =
StoreFileScanner.getScannersForStoreFiles(sfs,
-cacheMobBlocks, true, false, false, readPt);
{code}
into one that is more literate:
{code}
// ah, very clearly we are using defaults except for the caching settings
+ List sfScanners = new
StoreFileScanner.ListBuilder(sfs, readPt)
+ .withCacheBlocks(cacheMobBlocks).build();
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15060) Cull TestHFileWriterV2
Jonathan Hsieh created HBASE-15060: -- Summary: Cull TestHFileWriterV2 Key: HBASE-15060 URL: https://issues.apache.org/jira/browse/HBASE-15060 Project: HBase Issue Type: Improvement Components: HFile Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 2.0.0 2.x removes hfile v2. This patch cleans up remnants from unit tests associated with it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15051) Refactor HFileReaderImpl HFIleContext usage
Jonathan Hsieh created HBASE-15051: -- Summary: Refactor HFileReaderImpl HFIleContext usage Key: HBASE-15051 URL: https://issues.apache.org/jira/browse/HBASE-15051 Project: HBase Issue Type: Improvement Reporter: Jonathan Hsieh In discusssion from HBASE-15035, [~ram_krish] noted a different approach for guranteeing that HFileContext settings are passed in bulk load. This patch will take that idea and extend. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15035) bulkloading hfiles with tags that require splits does not preserve tags
Jonathan Hsieh created HBASE-15035: -- Summary: bulkloading hfiles with tags that require splits does not preserve tags Key: HBASE-15035 URL: https://issues.apache.org/jira/browse/HBASE-15035 Project: HBase Issue Type: Bug Components: HFile Affects Versions: 1.1.0, 1.0.0, 2.0.0 Reporter: Jonathan Hsieh Priority: Blocker When an hfile is created with cell tags present and it is bulk loaded into hbase the tags will be present when loaded into a single region. If the bulk load hfile spans multiple regions, bulk load automatically splits the original hfile into a set of split hfiles corresponding to each of the regions that the original covers. Since 0.98, tags are not copied into the newly created split hfiles. (the default for "includeTags" of the HFileContextBuilder [1] is uninitialized which defaults to false). This means acls, ttls, mob pointers and other tag stored values will not be bulk loaded in. [1] https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileContextBuilder.java#L40 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-15011) clean up newly introduced javadoc warning
[
https://issues.apache.org/jira/browse/HBASE-15011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh resolved HBASE-15011.
Resolution: Duplicate
just gonna close as a dupe. Thanks Ashish.
> clean up newly introduced javadoc warning
> -
>
> Key: HBASE-15011
> URL: https://issues.apache.org/jira/browse/HBASE-15011
> Project: HBase
> Issue Type: Bug
> Components: build, documentation
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Labels: beginner
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
>
> there's a new javadoc warning that causes warnings on all of our new patches
> and (I believe) breaks us on jdk8.
> Thanks to [~saint@gmail.com] for chasing it down on HBASE-14849.
> {code}
> 1 warning
> [WARNING] Javadoc Warnings
> [WARNING]
> /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java:430:
> warning - @param argument "hfilesDir" is not a parameter name.
> [INFO]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-9245) Remove dead or deprecated code from hbase 0.96
[ https://issues.apache.org/jira/browse/HBASE-9245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-9245. --- Resolution: Not A Problem 0.96 has long past. closing this out. > Remove dead or deprecated code from hbase 0.96 > -- > > Key: HBASE-9245 > URL: https://issues.apache.org/jira/browse/HBASE-9245 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh > > This is an umbrella issue that will cover the removal or refactoring of > dangling dead code and cruft. Some can make it into 0.96, some may have to > wait for an 0.98. The "great culling" of code will be grouped patches that > are logically related. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-9381) Check that RawComparator implementations use faster UnsafeComparer
[ https://issues.apache.org/jira/browse/HBASE-9381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-9381. --- Resolution: Won't Fix ancient. closing this out. > Check that RawComparator implementations use faster UnsafeComparer > -- > > Key: HBASE-9381 > URL: https://issues.apache.org/jira/browse/HBASE-9381 > Project: HBase > Issue Type: Sub-task >Reporter: Jonathan Hsieh > > Follow on from review of HBASE-9247. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-10588) Remove deprecated public facing classes/methods related to KeyValue
[ https://issues.apache.org/jira/browse/HBASE-10588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-10588. Resolution: Won't Fix closing, other issues have been filed. > Remove deprecated public facing classes/methods related to KeyValue > --- > > Key: HBASE-10588 > URL: https://issues.apache.org/jira/browse/HBASE-10588 > Project: HBase > Issue Type: Sub-task >Reporter: Jimmy Xiang >Assignee: Rekha Joshi > Fix For: 2.0.0 > > > Due to backward compatibility issue, old public facing classes/methods > related to KeyValue are deprecated. We should clean them up after they have > been deprecated for a while. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-9396) Remove deprecation and shims from HBASE-9334/HBASE-9359 from 0.98
[ https://issues.apache.org/jira/browse/HBASE-9396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-9396. --- Resolution: Won't Fix ancient. closing out. > Remove deprecation and shims from HBASE-9334/HBASE-9359 from 0.98 > - > > Key: HBASE-9396 > URL: https://issues.apache.org/jira/browse/HBASE-9396 > Project: HBase > Issue Type: Sub-task >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > > The version commited to 0.96 and trunk were essentially identical and > included deprecation shims. This follow-on patch will remove deprecated > stuff from trunk only. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14780) Integration Tests that run with ChaosMonkey need to specify CFs
Jonathan Hsieh created HBASE-14780: -- Summary: Integration Tests that run with ChaosMonkey need to specify CFs Key: HBASE-14780 URL: https://issues.apache.org/jira/browse/HBASE-14780 Project: HBase Issue Type: Improvement Reporter: Jonathan Hsieh Been running some IT tests and found that some failed because getcfs was null and didn't protecte cfs that were assumed to go unmolested. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14779) Revamp IntegrationTestMTTR
Jonathan Hsieh created HBASE-14779: -- Summary: Revamp IntegrationTestMTTR Key: HBASE-14779 URL: https://issues.apache.org/jira/browse/HBASE-14779 Project: HBase Issue Type: Improvement Components: integration tests Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh I've recently been trying to revive IntegrationTestMTTR runs and found that it tended to not complete in less 6 hours and wasn't written as many of the other Integration Tests. I'm going to revamp it a local it run of it can finish in < 30mins and to make it more configurable for a run against a real cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14763) Remove usages of deprecated HConnection
Jonathan Hsieh created HBASE-14763: -- Summary: Remove usages of deprecated HConnection Key: HBASE-14763 URL: https://issues.apache.org/jira/browse/HBASE-14763 Project: HBase Issue Type: Bug Components: hbase Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 2.0.0 HConnection was deprecated in 1.0.0. There are two interfaces that are supposed to be used instead -- Connection for client programs and ClusterConnection for internal hbaes and special tools (LoadIncremental, HBCK, etc). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14731) Add -DuseMob option to ITBLL
Jonathan Hsieh created HBASE-14731: -- Summary: Add -DuseMob option to ITBLL Key: HBASE-14731 URL: https://issues.apache.org/jira/browse/HBASE-14731 Project: HBase Issue Type: Bug Reporter: Jonathan Hsieh Previously, I would test mob by creating a table, altering to use mob and then running ITBLL. This simplifies it by allowing for a command line arg to enable mob on table creation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14713) Remove simple deprecated since 1.0 code from hbase 2.0
Jonathan Hsieh created HBASE-14713: -- Summary: Remove simple deprecated since 1.0 code from hbase 2.0 Key: HBASE-14713 URL: https://issues.apache.org/jira/browse/HBASE-14713 Project: HBase Issue Type: Task Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 2.0.0 Removed items such as * Migration to 96. * HFileOutputFormat (HFileOutputFormat2 preferred) * MapfilePerformanceEvaluttion * getHTable -> getTable() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-14711) Remove or annotated deprecated methods in HRegionInfo
[ https://issues.apache.org/jira/browse/HBASE-14711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-14711. Resolution: Fixed > Remove or annotated deprecated methods in HRegionInfo > - > > Key: HBASE-14711 > URL: https://issues.apache.org/jira/browse/HBASE-14711 > Project: HBase > Issue Type: Bug > Components: hbase >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: hbase-14711.patch > > > Several methods were deprecated in 1.0 and moved to MetaTableAccessor. This > removes them from the 2.0 branch and converts them to the MetaTableAccessor. > HRegionInfo#getComparator was marked deprecated after 1.0 so needs to > remain. (HBASE-13501) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HBASE-14711) Remove or annotated deprecated methods in HRegionInfo
[ https://issues.apache.org/jira/browse/HBASE-14711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh reopened HBASE-14711: > Remove or annotated deprecated methods in HRegionInfo > - > > Key: HBASE-14711 > URL: https://issues.apache.org/jira/browse/HBASE-14711 > Project: HBase > Issue Type: Bug > Components: hbase >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: hbase-14711.patch > > > Several methods were deprecated in 1.0 and moved to MetaTableAccessor. This > removes them from the 2.0 branch and converts them to the MetaTableAccessor. > HRegionInfo#getComparator was marked deprecated after 1.0 so needs to > remain. (HBASE-13501) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14711) Remove or annotated deprecated methods in HRegionInfo
Jonathan Hsieh created HBASE-14711: -- Summary: Remove or annotated deprecated methods in HRegionInfo Key: HBASE-14711 URL: https://issues.apache.org/jira/browse/HBASE-14711 Project: HBase Issue Type: Bug Components: hbase Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 2.0.0 Several methods were deprecated in 1.0 and moved to MetaTableAccessor. This removes them from the 2.0 branch and converts them to the MetaTableAccessor. HRegionInfo#getComparator was marked deprecated after 1.0 so needs to remain. (HBASE-13501) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14675) Excise deprecated Put#add(...) and replace with Put#addColumn(...)
Jonathan Hsieh created HBASE-14675: -- Summary: Excise deprecated Put#add(...) and replace with Put#addColumn(...) Key: HBASE-14675 URL: https://issues.apache.org/jira/browse/HBASE-14675 Project: HBase Issue Type: Improvement Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 2.0.0 The Put API changed from #add(...) to #addColumn(...). This updates all instances of it and removes it from the Put (which was added for hbase 1.0.0) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14672) Excise deprecated Delete#delete* apis
Jonathan Hsieh created HBASE-14672: -- Summary: Excise deprecated Delete#delete* apis Key: HBASE-14672 URL: https://issues.apache.org/jira/browse/HBASE-14672 Project: HBase Issue Type: Improvement Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 2.0.0 Delete#delete* apis were replaced with Delete#add* apis. This converts all instances of it and removes Delete#delete* apis. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14671) Remove deprecated HBaseTestCase/Put/Delete apis from TestGetClosestAtOrBefore
Jonathan Hsieh created HBASE-14671: -- Summary: Remove deprecated HBaseTestCase/Put/Delete apis from TestGetClosestAtOrBefore Key: HBASE-14671 URL: https://issues.apache.org/jira/browse/HBASE-14671 Project: HBase Issue Type: Improvement Components: test Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 2.0.0 converting from long deprecated apis to modern apis. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14670) Remove deprecated HBaseTestCase from TestBlocksRead
Jonathan Hsieh created HBASE-14670: -- Summary: Remove deprecated HBaseTestCase from TestBlocksRead Key: HBASE-14670 URL: https://issues.apache.org/jira/browse/HBASE-14670 Project: HBase Issue Type: Improvement Reporter: Jonathan Hsieh removing dependency on long deprecated apis -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14668) Remove deprecaetd HBaseTestCase dependency from TestHFile
Jonathan Hsieh created HBASE-14668: -- Summary: Remove deprecaetd HBaseTestCase dependency from TestHFile Key: HBASE-14668 URL: https://issues.apache.org/jira/browse/HBASE-14668 Project: HBase Issue Type: Improvement Components: test Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Fix For: 2.0.0 Remove dependency on long deprecated HBaseTestCase (0.90). modified references to tfiles which haven't been part of any modern hbase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14666) Remove deprecated HBaseTestingUttility#deleteTable methods
Jonathan Hsieh created HBASE-14666: -- Summary: Remove deprecated HBaseTestingUttility#deleteTable methods Key: HBASE-14666 URL: https://issues.apache.org/jira/browse/HBASE-14666 Project: HBase Issue Type: Improvement Components: test Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 2.0.0 HBase has had the TableName APIs since 0.96, and our test code doesn't have privacy/stability markers so we should be able to remove the deprecated methods in 2.0.0. deleteTable has some separate unit test related cleanup. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14665) Remove deprecated HBaseTestingUtility#createTable methods
Jonathan Hsieh created HBASE-14665: -- Summary: Remove deprecated HBaseTestingUtility#createTable methods Key: HBASE-14665 URL: https://issues.apache.org/jira/browse/HBASE-14665 Project: HBase Issue Type: Improvement Components: test Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 2.0.0 HBase has had the TableName APIs since 0.96, and our test code doesn't have privacy/stability markers so we should be able to remove the deprecated methods in 2.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-13804) Revert the changes in pom.xml
[
https://issues.apache.org/jira/browse/HBASE-13804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh resolved HBASE-13804.
Resolution: Fixed
Hadoop Flags: Reviewed
committed. thanks jingcheng and anoop
> Revert the changes in pom.xml
> -
>
> Key: HBASE-13804
> URL: https://issues.apache.org/jira/browse/HBASE-13804
> Project: HBase
> Issue Type: Sub-task
> Components: mob
>Affects Versions: hbase-11339
>Reporter: Jingcheng Du
>Assignee: Jingcheng Du
> Fix For: hbase-11339
>
> Attachments: HBASE-13804.diff
>
>
> Some code were delete in pom.xml.
> {noformat}
>
>target/jacoco.exec
>
> {noformat}
> We can revert the changes if this change is not necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-13531) Flakey failures of TestAcidGuarantees#testMobScanAtomicity
[ https://issues.apache.org/jira/browse/HBASE-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-13531. Resolution: Fixed Hadoop Flags: Reviewed > Flakey failures of TestAcidGuarantees#testMobScanAtomicity > -- > > Key: HBASE-13531 > URL: https://issues.apache.org/jira/browse/HBASE-13531 > Project: HBase > Issue Type: Sub-task > Components: regionserver, Scanners >Affects Versions: hbase-11339 >Reporter: Jonathan Hsieh >Assignee: Jingcheng Du > Fix For: hbase-11339 > > Attachments: HBASE-13531-V2.diff, HBASE-13531-V3.diff, > HBASE-13531-V4.diff, HBASE-13531-V5.diff, HBASE-13531-V6.diff, > HBASE-13531.diff > > > After the merge of master from 4/18/15 with hbase-11339 branch, we encounter > some atomicity violations. We want to fix before calling merge to trunk. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13531) After 4/18/15 merge, we have started encountering failures of TestAcidGuarantees#testMobScanAtomicity
Jonathan Hsieh created HBASE-13531: -- Summary: After 4/18/15 merge, we have started encountering failures of TestAcidGuarantees#testMobScanAtomicity Key: HBASE-13531 URL: https://issues.apache.org/jira/browse/HBASE-13531 Project: HBase Issue Type: Sub-task Affects Versions: hbase-11339 Reporter: Jonathan Hsieh Fix For: hbase-11339 After the merge of master from 4/18/15 with hbase-11339 branch, we encounter some atomicity violations. We want to fix before calling merge to trunk. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-11599) BloomFilterFactory.IO_STOREFILE_DELETEFAMILY_BLOOM_ENABLED is never used
[ https://issues.apache.org/jira/browse/HBASE-11599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-11599. Resolution: Not A Problem Assignee: Apekshit Sharma > BloomFilterFactory.IO_STOREFILE_DELETEFAMILY_BLOOM_ENABLED is never used > > > Key: HBASE-11599 > URL: https://issues.apache.org/jira/browse/HBASE-11599 > Project: HBase > Issue Type: Bug >Affects Versions: 1.0.0, 0.94.21, 0.98.4, 2.0.0 >Reporter: Jean-Marc Spaggiari >Assignee: Apekshit Sharma >Priority: Minor > Labels: beginner > > BloomFilterFactory.IO_STOREFILE_DELETEFAMILY_BLOOM_ENABLED is not used > anywhere. I think we should clean it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-13412) Region split decisions should have jitter
[ https://issues.apache.org/jira/browse/HBASE-13412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-13412. Resolution: Fixed I've committed the addendum to branch-1 and master. > Region split decisions should have jitter > - > > Key: HBASE-13412 > URL: https://issues.apache.org/jira/browse/HBASE-13412 > Project: HBase > Issue Type: New Feature > Components: regionserver >Affects Versions: 1.0.0, 2.0.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.1.0 > > Attachments: HBASE-13412-v1.patch, HBASE-13412-v2.patch, > HBASE-13412-v3.patch, HBASE-13412.patch, hbase-13412.addendum.patch > > > Whenever a region splits it causes lots of IO (compactions are queued for a > while). Because of this it's important to make sure that well distributed > tables don't have all of their regions split at exactly the same time. > This is basically the same as our compaction jitter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-9817) Add throughput quota and enforce hard limits
[ https://issues.apache.org/jira/browse/HBASE-9817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-9817. --- Resolution: Duplicate > Add throughput quota and enforce hard limits > > > Key: HBASE-9817 > URL: https://issues.apache.org/jira/browse/HBASE-9817 > Project: HBase > Issue Type: New Feature > Components: Coprocessors >Reporter: He Liangliang >Assignee: He Liangliang > Attachments: HBASE-9817.patch > > > There is planning to add region count and table count quota mentioned in > HBASE-8410. However, it's also quite useful to control the requesting > throughput inside the region server. For example, we don't want a data > dumping task affecting the online services and it's better to enforce the > throughput quota inside HBase. Another common scenario is multi-tenancy, i.e. > a cluster is shared by multiple applications. > The following rules will be supported: > 1. per user quota > limits the total read/write throughput initiated by a single user on any > table. > 2. per (user, table) quota > limits the total read/write throughput initiated by a single user on a > specified table. > The implementation will use coprocessor to intercept and check each request > on each region. And each region server allocate a portion of quota from the > total specified quota (for that user or user + table) based on the portion of > active regions (the whole cluster or the specified table) assigned on that > region server. The request will be rejected or delayed if the limit is > reached. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13302) fix new javadoc warns introduced by mob.
Jonathan Hsieh created HBASE-13302:
--
Summary: fix new javadoc warns introduced by mob.
Key: HBASE-13302
URL: https://issues.apache.org/jira/browse/HBASE-13302
Project: HBase
Issue Type: Sub-task
Affects Versions: hbase-11339
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Fix For: hbase-11339
now that we have the qabot able to work on branches, we've detected a few
javadoc issues. this will fix it.
{code}
[WARNING] Javadoc Warnings
[WARNING]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java:397:
warning - @param argument "path" is not a parameter name.
[WARNING]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java:578:
warning - @param argument "@param" is not a parameter name.
[WARNING]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java:578:
warning - @param argument "path" is not a parameter name.
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13289) typo in splitSuccessCount metric
Jonathan Hsieh created HBASE-13289:
--
Summary: typo in splitSuccessCount metric
Key: HBASE-13289
URL: https://issues.apache.org/jira/browse/HBASE-13289
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 1.1.0
Reporter: Jonathan Hsieh
Our split metrics have a misspelled Count and it shows up in our jmx metrics
{code}
"splitSuccessCounnt" : 0,
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13277) add mob_threshold option to load test tool
Jonathan Hsieh created HBASE-13277: -- Summary: add mob_threshold option to load test tool Key: HBASE-13277 URL: https://issues.apache.org/jira/browse/HBASE-13277 Project: HBase Issue Type: Sub-task Affects Versions: hbase-11339 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: hbase-11339 This adds '-mob_threshold ' option to the load test tool to simplify mob load testing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13233) add hbase-11339 branch to the patch testing script
Jonathan Hsieh created HBASE-13233: -- Summary: add hbase-11339 branch to the patch testing script Key: HBASE-13233 URL: https://issues.apache.org/jira/browse/HBASE-13233 Project: HBase Issue Type: Task Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh adding hbase-11339 to the BRANCH_NAMES so we can use the apache bot to test patches on that branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13230) [mob] reads hang when trying to read rows with large mobs (>10MB)
Jonathan Hsieh created HBASE-13230: -- Summary: [mob] reads hang when trying to read rows with large mobs (>10MB) Key: HBASE-13230 URL: https://issues.apache.org/jira/browse/HBASE-13230 Project: HBase Issue Type: Bug Components: mob Affects Versions: hbase-11339 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: hbase-11339 Using load tests tool to read and write out 5MB, 10MB, 20MB objects works fine, but problems are encountered when trying to read values >20MB. This is due to the default protobuf size limit of 64MB. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-3743) Throttle major compaction
[ https://issues.apache.org/jira/browse/HBASE-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-3743. --- Resolution: Duplicate This is essentially a dupe of HBASE-8329 and HBASE-5867, both of which are closed out. > Throttle major compaction > - > > Key: HBASE-3743 > URL: https://issues.apache.org/jira/browse/HBASE-3743 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Joep Rottinghuis > > Add the ability to throttle major compaction. > For those use cases when a stop-the-world approach is not practical, it is > useful to be able to throttle the impact that major compaction has on the > cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-13225) SHORT DESCRIPTION
[ https://issues.apache.org/jira/browse/HBASE-13225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-13225. Resolution: Incomplete > SHORT DESCRIPTION > - > > Key: HBASE-13225 > URL: https://issues.apache.org/jira/browse/HBASE-13225 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: david lee >Priority: Blocker > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-13151) IllegalArgumentException in compaction when table has a namespace
[ https://issues.apache.org/jira/browse/HBASE-13151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-13151. Resolution: Fixed Hadoop Flags: Reviewed > IllegalArgumentException in compaction when table has a namespace > - > > Key: HBASE-13151 > URL: https://issues.apache.org/jira/browse/HBASE-13151 > Project: HBase > Issue Type: Sub-task > Components: regionserver, Scanners >Affects Versions: hbase-11339 >Reporter: Jingcheng Du >Assignee: Jingcheng Du > Fix For: hbase-11339 > > Attachments: HBASE-13151-V2.diff, HBASE-13151-V3.diff, > HBASE-13151-V4.diff, HBASE-13151-V5.diff, HBASE-13151.diff > > > In the PartitionedMobFileCompactor we have a bulk load path, currently it > directly use table.getNameAsString as part of the path, if the table has a > namespace, the path contains character ":" which is regarded as a illegal > character in file path. As fixing, we need to divide the namespace and > qualifier into directories like the way in table path. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-13107) Refactor MOB Snapshot logic to reduce code duplication.
[ https://issues.apache.org/jira/browse/HBASE-13107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-13107. Resolution: Fixed Hadoop Flags: Reviewed lgtm. thanks jingcheng! committed to hbase-11339 > Refactor MOB Snapshot logic to reduce code duplication. > --- > > Key: HBASE-13107 > URL: https://issues.apache.org/jira/browse/HBASE-13107 > Project: HBase > Issue Type: Sub-task > Components: mob, snapshots >Affects Versions: hbase-11339 >Reporter: Jonathan Hsieh >Assignee: Jingcheng Du > Fix For: hbase-11339 > > Attachments: HBASE-13107.diff > > > The MOB Snapshot code contains a lot of code duplication with the normal > snapshot code path. We should do some refactoring to clean this up before > merging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-12332) [mob] improve how we resolve mobfiles in reads
[ https://issues.apache.org/jira/browse/HBASE-12332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-12332. Resolution: Fixed Hadoop Flags: Reviewed Thanks for the patience Jiajia, I've committed the patch. > [mob] improve how we resolve mobfiles in reads > -- > > Key: HBASE-12332 > URL: https://issues.apache.org/jira/browse/HBASE-12332 > Project: HBase > Issue Type: Sub-task > Components: mob >Affects Versions: hbase-11339 >Reporter: Jonathan Hsieh >Assignee: Jiajia Li > Fix For: hbase-11339 > > Attachments: HBASE-12332-V1.diff, HBASE-12332-V2.patch, > HBASE-12332-V3.patch, HBASE-12332-V5.patch, HBASE-12332-V6.diff, > hbase-12332.link.v4.patch, hbase-12332.patch > > > in the snapshot code, hmobstore was modified to traverse an hfile link to a > mob. Ideally this should use the transparent filelink code to read the data. > Also there will likely be some issues with the mob file cache with these > links. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-13012) Add shell commands to trigger the mob file compactor
[ https://issues.apache.org/jira/browse/HBASE-13012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-13012. Resolution: Fixed Hadoop Flags: Reviewed > Add shell commands to trigger the mob file compactor > - > > Key: HBASE-13012 > URL: https://issues.apache.org/jira/browse/HBASE-13012 > Project: HBase > Issue Type: Sub-task > Components: regionserver, Scanners >Reporter: Jingcheng Du >Assignee: Jingcheng Du > Fix For: hbase-11339 > > Attachments: HBASE-13012-V5.diff, HBASE-13012-V6.diff, > HBASE-13012-V7.diff, HBASE-13012-V9.diff, HBASE-13012.v8.patch, > hbase-13012-V1.patch, hbase-13012-V2.diff, hbase-13012-V3.diff, > hbase-13012-V4.diff > > > Currently the MobFileCompactor is run by HMaster periodically, we need to add > a shell to trigger the compactor by commands. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13157) Add mob compaction actions and monkeys to Chaos Monkey
Jonathan Hsieh created HBASE-13157: -- Summary: Add mob compaction actions and monkeys to Chaos Monkey Key: HBASE-13157 URL: https://issues.apache.org/jira/browse/HBASE-13157 Project: HBase Issue Type: Sub-task Components: integration tests, mob Affects Versions: hbase-11339 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: hbase-11339 After HBASE-13012 (mob compaction shell and admin api) we can now trigger mob compaction. This adds an action to chaos monkey so we can randomly trigger these compactions during integration test runs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13154) Add support for mob in TestAcidGuarantees and IntegrationTestAcidGuarantees
Jonathan Hsieh created HBASE-13154: -- Summary: Add support for mob in TestAcidGuarantees and IntegrationTestAcidGuarantees Key: HBASE-13154 URL: https://issues.apache.org/jira/browse/HBASE-13154 Project: HBase Issue Type: Sub-task Components: integration tests, mob Affects Versions: hbase-11339 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: hbase-11339 Adding support to run acid guarantees tests on data that is primarily in the mob path. We essentially set the mob size to be very small (4) and run the same test. This is activated using the -DuseMob=true tool style command line arg. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-13152) NPE in ExpiredMobFileCleanerChore
[ https://issues.apache.org/jira/browse/HBASE-13152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-13152. Resolution: Fixed Hadoop Flags: Reviewed lgtm. committing to hbase-11339. Thanks Jingcheng. > NPE in ExpiredMobFileCleanerChore > - > > Key: HBASE-13152 > URL: https://issues.apache.org/jira/browse/HBASE-13152 > Project: HBase > Issue Type: Sub-task > Components: regionserver, Scanners >Affects Versions: hbase-11339 >Reporter: Jingcheng Du >Assignee: Jingcheng Du > Fix For: hbase-11339 > > Attachments: HBASE-13152.diff > > > NullPointerException in ExpiredMobFileCleanerChore. In the > ExpiredMobFileCleanerChore, we invoked ExpiredMobFileCleaner to clean the > expired mob files. But we didn't pass the conf in which led to a NPE when > invoking methods of conf in cleaner. This patch will fix the NPE issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-12670) Add unit tests that exercise the added hfilelink link mob paths
[ https://issues.apache.org/jira/browse/HBASE-12670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-12670. Resolution: Fixed Hadoop Flags: Reviewed thanks jingcheng. committed to hbase-11339 > Add unit tests that exercise the added hfilelink link mob paths > --- > > Key: HBASE-12670 > URL: https://issues.apache.org/jira/browse/HBASE-12670 > Project: HBase > Issue Type: Sub-task > Components: regionserver, Scanners >Affects Versions: hbase-11339 >Reporter: Jonathan Hsieh >Assignee: Jingcheng Du > Fix For: hbase-11339 > > Attachments: HBASE-12670.diff > > > HBASe-12646 introduced the mob path to HFileLink -- we didn't add unit tests > for it however. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13141) IntegrationTestAcidGuarantees returns incorrect values for getColumnFamilies
Jonathan Hsieh created HBASE-13141: -- Summary: IntegrationTestAcidGuarantees returns incorrect values for getColumnFamilies Key: HBASE-13141 URL: https://issues.apache.org/jira/browse/HBASE-13141 Project: HBase Issue Type: Bug Components: integration tests Affects Versions: 1.0.0, 2.0.0, 1.1.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 2.0.0, 1.0.1, 1.1.0 IntegrationTestAcidGuarantees with monkeys on fails occasionally after a RemoveColumnAction because the getColumnFamilies values are incorrect. We actually were naming the cf's "[B@12ef34ab" instead of "A", "B", and "C". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13132) Improve RemoveColumn action debug message
Jonathan Hsieh created HBASE-13132:
--
Summary: Improve RemoveColumn action debug message
Key: HBASE-13132
URL: https://issues.apache.org/jira/browse/HBASE-13132
Project: HBase
Issue Type: Improvement
Components: integration tests
Affects Versions: 2.0.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Trivial
Fix For: 2.0.0, 1.0.1, 1.1.0
Trivial fix for this unsightly log message:
{code}
2015-02-22 14:04:46,357 DEBUG [TwoConcurrentAction-1-thread-2] actions.Action:
Performing action: Removing [B@64275127 from TestAcidGuarantees
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13117) improve mob sweeper javadoc
Jonathan Hsieh created HBASE-13117: -- Summary: improve mob sweeper javadoc Key: HBASE-13117 URL: https://issues.apache.org/jira/browse/HBASE-13117 Project: HBase Issue Type: Sub-task Components: mob Affects Versions: hbase-11339 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: hbase-11339 The command line return values for the sweeper are undocumented and should be explained in javadoc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13107) Refactor MOB Snapshot logic to reduce code duplication.
Jonathan Hsieh created HBASE-13107: -- Summary: Refactor MOB Snapshot logic to reduce code duplication. Key: HBASE-13107 URL: https://issues.apache.org/jira/browse/HBASE-13107 Project: HBase Issue Type: Sub-task Components: mob, snapshots Affects Versions: hbase-11339 Reporter: Jonathan Hsieh The MOB Snapshot code contains a lot of code duplication with the normal snapshot code path. We should do some refactoring to clean this up before merging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-11861) Native MOB Compaction mechanisms.
[ https://issues.apache.org/jira/browse/HBASE-11861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-11861. Resolution: Fixed Fix Version/s: hbase-11339 Hadoop Flags: Reviewed Thanks jingcheng and thanks ram and ted for the reviews. I've commited to hbase-11339 > Native MOB Compaction mechanisms. > - > > Key: HBASE-11861 > URL: https://issues.apache.org/jira/browse/HBASE-11861 > Project: HBase > Issue Type: Sub-task > Components: regionserver, Scanners >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jingcheng Du > Fix For: hbase-11339 > > Attachments: 141030-mob-compaction.pdf, HBASE-11861-V1.diff, > HBASE-11861-V2.diff, HBASE-11861-V3.diff, HBASE-11861-V4.diff, > HBASE-11861-V5.diff, HBASE-11861-V6.diff, HBASE-11861.diff, mob > compaction-out-of-region.pdf, mob compaction.pdf > > > Currently, the first cut of mob will have external processes to age off old > mob data (the ttl cleaner), and to compact away deleted or over written data > (the sweep tool). > From an operational point of view, having two external tools, especially one > that relies on MapReduce is undesirable. In this issue we'll tackle > integrating these into hbase without requiring external processes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-6812) Export dry run mode.
[ https://issues.apache.org/jira/browse/HBASE-6812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-6812. --- Resolution: Won't Fix I filed this way back, closing it. We could likely do a row counter job to scan all the rows which would be roughly equivalent and help diagnose the read side. > Export dry run mode. > > > Key: HBASE-6812 > URL: https://issues.apache.org/jira/browse/HBASE-6812 > Project: HBase > Issue Type: New Feature >Reporter: Jonathan Hsieh > Labels: delete > > We encountered customer experiencing poor performance on Exports and we've > spent some time trying to isolate the performance bottlenecks. It was > suggested to add a "dry run mode" that just scans the table but does not emit > data to hdfs. This would be really useful for determining if problems are on > the write path or the read path. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-12331) Shorten the mob snapshot unit tests
[
https://issues.apache.org/jira/browse/HBASE-12331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh resolved HBASE-12331.
Resolution: Fixed
Assignee: Jiajia Li
Hadoop Flags: Reviewed
> Shorten the mob snapshot unit tests
> ---
>
> Key: HBASE-12331
> URL: https://issues.apache.org/jira/browse/HBASE-12331
> Project: HBase
> Issue Type: Sub-task
> Components: mob
>Affects Versions: hbase-11339
>Reporter: Jonathan Hsieh
>Assignee: Jiajia Li
> Fix For: hbase-11339
>
> Attachments: HBASE-12331-V1.diff, HBASE-12331-V2.patch
>
>
> The mob snapshot patch introduced a whole log of tests that take a long time
> to run and would be better as integration tests.
> {code}
> ---
> T E S T S
> ---
> Running
> org.apache.hadoop.hbase.client.TestMobRestoreSnapshotFromClientWithRegionReplicas
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 394.803 sec -
> in
> org.apache.hadoop.hbase.client.TestMobRestoreSnapshotFromClientWithRegionReplicas
> Running org.apache.hadoop.hbase.client.TestMobRestoreSnapshotFromClient
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 212.377 sec -
> in org.apache.hadoop.hbase.client.TestMobRestoreSnapshotFromClient
> Running
> org.apache.hadoop.hbase.client.TestMobSnapshotFromClientWithRegionReplicas
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 49.463 sec -
> in org.apache.hadoop.hbase.client.TestMobSnapshotFromClientWithRegionReplicas
> Running org.apache.hadoop.hbase.client.TestMobSnapshotFromClient
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 46.724 sec -
> in org.apache.hadoop.hbase.client.TestMobSnapshotFromClient
> Running org.apache.hadoop.hbase.client.TestMobCloneSnapshotFromClient
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 204.03 sec -
> in org.apache.hadoop.hbase.client.TestMobCloneSnapshotFromClient
> Running
> org.apache.hadoop.hbase.client.TestMobCloneSnapshotFromClientWithRegionReplicas
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 214.052 sec -
> in
> org.apache.hadoop.hbase.client.TestMobCloneSnapshotFromClientWithRegionReplicas
> Running org.apache.hadoop.hbase.client.TestMobSnapshotCloneIndependence
> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 105.139 sec -
> in org.apache.hadoop.hbase.client.TestMobSnapshotCloneIndependence
> Running org.apache.hadoop.hbase.regionserver.TestMobStoreScanner
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.42 sec -
> in org.apache.hadoop.hbase.regionserver.TestMobStoreScanner
> Running org.apache.hadoop.hbase.regionserver.TestDeleteMobTable
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.136 sec -
> in org.apache.hadoop.hbase.regionserver.TestDeleteMobTable
> Running org.apache.hadoop.hbase.regionserver.TestHMobStore
> Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.09 sec - in
> org.apache.hadoop.hbase.regionserver.TestHMobStore
> Running org.apache.hadoop.hbase.regionserver.TestMobCompaction
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.629 sec -
> in org.apache.hadoop.hbase.regionserver.TestMobCompaction
> Running org.apache.hadoop.hbase.mob.TestCachedMobFile
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.301 sec -
> in org.apache.hadoop.hbase.mob.TestCachedMobFile
> Running org.apache.hadoop.hbase.mob.mapreduce.TestMobSweepJob
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.752 sec -
> in org.apache.hadoop.hbase.mob.mapreduce.TestMobSweepJob
> Running org.apache.hadoop.hbase.mob.mapreduce.TestMobSweepReducer
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.276 sec -
> in org.apache.hadoop.hbase.mob.mapreduce.TestMobSweepReducer
> Running org.apache.hadoop.hbase.mob.mapreduce.TestMobSweepMapper
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.46 sec -
> in org.apache.hadoop.hbase.mob.mapreduce.TestMobSweepMapper
> Running org.apache.hadoop.hbase.mob.mapreduce.TestMobSweeper
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 173.05 sec -
> in org.apache.hadoop.hbase.mob.mapreduce.TestMobSweeper
> Running org.apache.hadoop.hbase.mob.TestMobDataBlockEncoding
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.86 sec -
> in org.apache.hadoop.hbase.mob.TestMobDataBlockEncoding
> Running org.apache.hadoop.hbase.mob.TestExpiredMobFileCleaner
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.029 sec -
> in org.apache.hadoop.hbase.mob.TestExpiredMobFileCleaner
> Running org.apache.hadoop.hbase.mob.TestMobFile
> Tests run: 2, Failures: 0, Err
[jira] [Resolved] (HBASE-12698) Add mob cell count to the metadata of each mob file
[ https://issues.apache.org/jira/browse/HBASE-12698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-12698. Resolution: Fixed Hadoop Flags: Reviewed > Add mob cell count to the metadata of each mob file > --- > > Key: HBASE-12698 > URL: https://issues.apache.org/jira/browse/HBASE-12698 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Affects Versions: hbase-11339 >Reporter: Jingcheng Du >Assignee: Jingcheng Du > Fix For: hbase-11339 > > Attachments: HBASE-12698.diff > > > Currently we don't know the cell count according to the existing file > information in the mob file. This value is very useful in the mob compaction, > we should have it in the metadata of each mob file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-12758) treat mob region as any other region when generating rs manifest.
[ https://issues.apache.org/jira/browse/HBASE-12758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-12758. Resolution: Fixed Hadoop Flags: Reviewed commited to hbase-11339. thanks for the looksee matteo. > treat mob region as any other region when generating rs manifest. > - > > Key: HBASE-12758 > URL: https://issues.apache.org/jira/browse/HBASE-12758 > Project: HBase > Issue Type: Bug > Components: mob, snapshots >Affects Versions: hbase-11339 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: hbase-11339 > > Attachments: hbase-12758.patch > > > Matteo on inspection noticed that we created two separate manifest files on > the region server that adds the mob region when we snapshot. This patch > makes it so that we only create one on that region server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12758) treat mob region as any other region when generating rs manifest.
Jonathan Hsieh created HBASE-12758: -- Summary: treat mob region as any other region when generating rs manifest. Key: HBASE-12758 URL: https://issues.apache.org/jira/browse/HBASE-12758 Project: HBase Issue Type: Bug Components: mob, snapshots Affects Versions: hbase-11339 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: hbase-11339 Matteo on inspection noticed that we created two separate manifest files on the region server that adds the mob region when we snapshot. This patch makes it so that we only create one on that region server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12749) Tighten HFileLink api to enable non-snapshot other uses
Jonathan Hsieh created HBASE-12749: -- Summary: Tighten HFileLink api to enable non-snapshot other uses Key: HBASE-12749 URL: https://issues.apache.org/jira/browse/HBASE-12749 Project: HBase Issue Type: Bug Components: snapshots Affects Versions: 2.0.0, 1.1.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh In HBASE-12332 we'd like to use the FileLink's IO redirecting powers but want to be able to specify arbitrary alternate link paths and not be tied to the SnapshotFileLink file pattern (aka, table=region-hfile). To do this we need change the constructors and some internals so that it is more generic. Along the way, we remove the FileStatus constructor arguments in favor of Path's and reduce the number of ways to create HFileLinks, and tighten up the scope privacy of many methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12718) Convert TestAcidGurantees from a unit test to an integration test
Jonathan Hsieh created HBASE-12718: -- Summary: Convert TestAcidGurantees from a unit test to an integration test Key: HBASE-12718 URL: https://issues.apache.org/jira/browse/HBASE-12718 Project: HBase Issue Type: Bug Components: hbase, integration tests, test Affects Versions: 1.0.0, 2.0.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh TestAcidGurantees has a main method so that it can be executed via the commandline. In the past this was run and we'd use external tools to inject faults while it executed for an extended period of time. We've had the IT framework we'd like to use the ChaosMonkey automation with it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12709) [mvn] Add unit test excludes command line flag to the build.
Jonathan Hsieh created HBASE-12709: -- Summary: [mvn] Add unit test excludes command line flag to the build. Key: HBASE-12709 URL: https://issues.apache.org/jira/browse/HBASE-12709 Project: HBase Issue Type: New Feature Components: build Affects Versions: 1.0.0, 2.0.0, 0.98.10 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh I've added a simple way to specify unit test classes to skip when executing unit test runs. I've added a -D variable called hbase.excludes.pattern that you can using like this: mvn test -Dexcludes.pattern=**/TestFoo.java,**/TestBar.java to exclude the unit tests form TestFoo and TestBar in this run. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-12693) [docs] nit fix in HBase and MapReduce section
[ https://issues.apache.org/jira/browse/HBASE-12693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-12693. Resolution: Fixed > [docs] nit fix in HBase and MapReduce section > - > > Key: HBASE-12693 > URL: https://issues.apache.org/jira/browse/HBASE-12693 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > > trivial nit fix -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12693) [docs] nit fix in HBase and Mapreduce section of docs
Jonathan Hsieh created HBASE-12693: -- Summary: [docs] nit fix in HBase and Mapreduce section of docs Key: HBASE-12693 URL: https://issues.apache.org/jira/browse/HBASE-12693 Project: HBase Issue Type: Bug Components: documentation Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 2.0.0 trivial nit fix -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12691) sweep job needs to exit non-zero if job fails for any reason.
Jonathan Hsieh created HBASE-12691: -- Summary: sweep job needs to exit non-zero if job fails for any reason. Key: HBASE-12691 URL: https://issues.apache.org/jira/browse/HBASE-12691 Project: HBase Issue Type: Sub-task Components: mob Affects Versions: hbase-11339 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: hbase-11339 When buliding up automated testing I noticed that th sweepjob would not "fail" because it exited 0 even if the job failed. This add the proper exit hygiene adding non-zero exit codes on failure events so that we can rely upon the job in automation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-12540) TestRegionServerMetrics#testMobMetrics test failure
[
https://issues.apache.org/jira/browse/HBASE-12540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh resolved HBASE-12540.
Resolution: Fixed
Assignee: Jingcheng Du (was: Jonathan Hsieh)
Hadoop Flags: Reviewed
> TestRegionServerMetrics#testMobMetrics test failure
> ---
>
> Key: HBASE-12540
> URL: https://issues.apache.org/jira/browse/HBASE-12540
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: hbase-11339
>Reporter: stack
>Assignee: Jingcheng Du
> Fix For: hbase-11339
>
> Attachments: HBASE-12540-V2.patch, HBASE-12540.diff, log.txt
>
>
> Got this on an internal rig run. Maybe you want to take a looksee
> [~jingchengdu]?
> {code}
> Error Message
> Metrics Counters should be equal expected:<5> but was:<2>
> Stacktrace
> java.lang.AssertionError: Metrics Counters should be equal expected:<5> but
> was:<2>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hbase.test.MetricsAssertHelperImpl.assertCounter(MetricsAssertHelperImpl.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestRegionServerMetrics.testMobMetrics(TestRegionServerMetrics.java:448)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12670) Add unit tests that exercise the added hfilelink link mob paths
Jonathan Hsieh created HBASE-12670: -- Summary: Add unit tests that exercise the added hfilelink link mob paths Key: HBASE-12670 URL: https://issues.apache.org/jira/browse/HBASE-12670 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh HBASe-12646 introduced the mob path to HFileLink -- we didn't add unit tests for it however. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12669) Have compaction scanner save info about deleted mob cells
Jonathan Hsieh created HBASE-12669: -- Summary: Have compaction scanner save info about deleted mob cells Key: HBASE-12669 URL: https://issues.apache.org/jira/browse/HBASE-12669 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh for the native mob compaction, we will need a scanner pass used by minor and major compactions that record the cells with references to mobs that have been deleted. This would implement that core section. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12646) SnapshotInfo tool does not find mob data in snapshots
Jonathan Hsieh created HBASE-12646:
--
Summary: SnapshotInfo tool does not find mob data in snapshots
Key: HBASE-12646
URL: https://issues.apache.org/jira/browse/HBASE-12646
Project: HBase
Issue Type: Sub-task
Components: mob, snapshots
Affects Versions: hbase-11339
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Fix For: hbase-11339
After taking a snapshot against a table with mob file data, the SnapshotInfo
tool is not able to find the mob files even though they are present in the
proper place. (/mobdir/data/dfault///)
Here's an example command line execution:
{code}
[jon@a1210 hbase]$ bin/hbase org.apache.hadoop.hbase.snapshot.SnapshotInfo
-snapshot test -files
-
IntegrationTestIngestWithMOB/e9b5d936e7f55a4f1c3246a8d5ce53c2/test_cf/e75380c6db50db845dbdee48ab49381420141205f87d1b47b0f446c7acb3fa0656fa8a57
(NOT FOUND)
**
BAD SNAPSHOT: 145 hfile(s) and 0 log(s) missing.
0 hfile(s) corrupted.
**
54 HFiles (54 in archive), total size 973.6 M (0.00% 0 shared with the source
table)
0 Logs, total size 0
{code}
The missing files are actually present, it is likely that the snapshot tool
doesn't know to look in the mob dir.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12629) Remove hbase.regionsizecalculator.enable from RegionSizeCalculator
Jonathan Hsieh created HBASE-12629: -- Summary: Remove hbase.regionsizecalculator.enable from RegionSizeCalculator Key: HBASE-12629 URL: https://issues.apache.org/jira/browse/HBASE-12629 Project: HBase Issue Type: Improvement Reporter: Jonathan Hsieh Fix For: 1.0.0, 2.0.0 The RegionSizeCalculator has a option to disable it. It is on by default and end-to-end use with it disabled is not tested or used anywhere except for a simple unit test. This removes it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-11645) Snapshot for MOB
[ https://issues.apache.org/jira/browse/HBASE-11645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-11645. Resolution: Fixed Fix Version/s: hbase-11339 Hadoop Flags: Reviewed > Snapshot for MOB > > > Key: HBASE-11645 > URL: https://issues.apache.org/jira/browse/HBASE-11645 > Project: HBase > Issue Type: Sub-task > Components: snapshots >Reporter: Jingcheng Du >Assignee: Jingcheng Du > Fix For: hbase-11339 > > Attachments: HBASE-11645-V2.diff, HBASE-11645-V3.diff, > HBASE-11645-V4.diff, HBASE-11645.diff > > > Add snapshot support for MOB. In the initial implementation, taking a table > snapshot does not preserve the mob data. This issue will make sure that when > a snapshot is taken, mob data is properly preserved and is restorable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12332) [mob] use filelink instad of retry when resolving an hfilelink.
Jonathan Hsieh created HBASE-12332: -- Summary: [mob] use filelink instad of retry when resolving an hfilelink. Key: HBASE-12332 URL: https://issues.apache.org/jira/browse/HBASE-12332 Project: HBase Issue Type: Sub-task Components: mob Affects Versions: hbase-11339 Reporter: Jonathan Hsieh in the snapshot code, hmobstore was modified to traverse an hfile link to a mob. Ideally this should use the transparent filelink code to read the data. Also there will likely be some issues with the mob file cache with these links. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12331) Shorten the mob snapshot unit tests
Jonathan Hsieh created HBASE-12331:
--
Summary: Shorten the mob snapshot unit tests
Key: HBASE-12331
URL: https://issues.apache.org/jira/browse/HBASE-12331
Project: HBase
Issue Type: Sub-task
Components: mob
Affects Versions: hbase-11339
Reporter: Jonathan Hsieh
The mob snapshot patch introduced a whole log of tests that take a long time to
run and would be better as integration tests.
{code}
---
T E S T S
---
Running
org.apache.hadoop.hbase.client.TestMobRestoreSnapshotFromClientWithRegionReplicas
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 394.803 sec -
in
org.apache.hadoop.hbase.client.TestMobRestoreSnapshotFromClientWithRegionReplicas
Running org.apache.hadoop.hbase.client.TestMobRestoreSnapshotFromClient
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 212.377 sec -
in org.apache.hadoop.hbase.client.TestMobRestoreSnapshotFromClient
Running
org.apache.hadoop.hbase.client.TestMobSnapshotFromClientWithRegionReplicas
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 49.463 sec - in
org.apache.hadoop.hbase.client.TestMobSnapshotFromClientWithRegionReplicas
Running org.apache.hadoop.hbase.client.TestMobSnapshotFromClient
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 46.724 sec - in
org.apache.hadoop.hbase.client.TestMobSnapshotFromClient
Running org.apache.hadoop.hbase.client.TestMobCloneSnapshotFromClient
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 204.03 sec - in
org.apache.hadoop.hbase.client.TestMobCloneSnapshotFromClient
Running
org.apache.hadoop.hbase.client.TestMobCloneSnapshotFromClientWithRegionReplicas
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 214.052 sec -
in
org.apache.hadoop.hbase.client.TestMobCloneSnapshotFromClientWithRegionReplicas
Running org.apache.hadoop.hbase.client.TestMobSnapshotCloneIndependence
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 105.139 sec -
in org.apache.hadoop.hbase.client.TestMobSnapshotCloneIndependence
Running org.apache.hadoop.hbase.regionserver.TestMobStoreScanner
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.42 sec - in
org.apache.hadoop.hbase.regionserver.TestMobStoreScanner
Running org.apache.hadoop.hbase.regionserver.TestDeleteMobTable
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.136 sec - in
org.apache.hadoop.hbase.regionserver.TestDeleteMobTable
Running org.apache.hadoop.hbase.regionserver.TestHMobStore
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.09 sec - in
org.apache.hadoop.hbase.regionserver.TestHMobStore
Running org.apache.hadoop.hbase.regionserver.TestMobCompaction
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.629 sec - in
org.apache.hadoop.hbase.regionserver.TestMobCompaction
Running org.apache.hadoop.hbase.mob.TestCachedMobFile
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.301 sec - in
org.apache.hadoop.hbase.mob.TestCachedMobFile
Running org.apache.hadoop.hbase.mob.mapreduce.TestMobSweepJob
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.752 sec - in
org.apache.hadoop.hbase.mob.mapreduce.TestMobSweepJob
Running org.apache.hadoop.hbase.mob.mapreduce.TestMobSweepReducer
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.276 sec - in
org.apache.hadoop.hbase.mob.mapreduce.TestMobSweepReducer
Running org.apache.hadoop.hbase.mob.mapreduce.TestMobSweepMapper
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.46 sec - in
org.apache.hadoop.hbase.mob.mapreduce.TestMobSweepMapper
Running org.apache.hadoop.hbase.mob.mapreduce.TestMobSweeper
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 173.05 sec - in
org.apache.hadoop.hbase.mob.mapreduce.TestMobSweeper
Running org.apache.hadoop.hbase.mob.TestMobDataBlockEncoding
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.86 sec - in
org.apache.hadoop.hbase.mob.TestMobDataBlockEncoding
Running org.apache.hadoop.hbase.mob.TestExpiredMobFileCleaner
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.029 sec - in
org.apache.hadoop.hbase.mob.TestExpiredMobFileCleaner
Running org.apache.hadoop.hbase.mob.TestMobFile
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.562 sec - in
org.apache.hadoop.hbase.mob.TestMobFile
Running org.apache.hadoop.hbase.mob.TestMobFileCache
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.173 sec - in
org.apache.hadoop.hbase.mob.TestMobFileCache
Running org.apache.hadoop.hbase.mob.TestDefaultMobStoreFlusher
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.586 sec - in
org.apache.hadoop.hbase.mob.TestDefaultMobStoreFlusher
Running org.apache.hadoop.hbase.mob.T
[jira] [Resolved] (HBASE-7509) Enable RS to query a secondary datanode in parallel, if the primary takes too long
[ https://issues.apache.org/jira/browse/HBASE-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-7509. --- Resolution: Not a Problem > Enable RS to query a secondary datanode in parallel, if the primary takes too > long > -- > > Key: HBASE-7509 > URL: https://issues.apache.org/jira/browse/HBASE-7509 > Project: HBase > Issue Type: Improvement >Reporter: Amitanand Aiyer >Assignee: Amitanand Aiyer >Priority: Critical > Attachments: quorumDiffs.tgz > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12085) mob status should print human readable numbers.
Jonathan Hsieh created HBASE-12085:
--
Summary: mob status should print human readable numbers.
Key: HBASE-12085
URL: https://issues.apache.org/jira/browse/HBASE-12085
Project: HBase
Issue Type: Bug
Components: mob, UI
Affects Versions: hbase-11339
Reporter: Jonathan Hsieh
Currently, mob cf configuration stuff shows up as ugly byte strings instead of
meaningful numeric values.
{code}
IntegrationTestIngestWithMOB20 'IntegrationTestIngestWithMOB', {NAME
=> 'test_cf', METADATA => {'mobThreshold' =>
'\x00\x00\x00\x00\x00\x01\x90\x00', 'isMob' => '\xFF'}}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-12080) Shorten the run time of integration test by default when using mvn failsafe:integration-test
[
https://issues.apache.org/jira/browse/HBASE-12080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh resolved HBASE-12080.
Resolution: Fixed
Hadoop Flags: Reviewed
Thanks, jingcheng. ran test from the hbase-it dir and it passed this time
around instead of hanging.
{code}
mvn failsafe:integration-test -Dit.test=IntegrationTestIngestWithMOB
{code}
committed to hbase-11339 branch.
> Shorten the run time of integration test by default when using mvn
> failsafe:integration-test
>
>
> Key: HBASE-12080
> URL: https://issues.apache.org/jira/browse/HBASE-12080
> Project: HBase
> Issue Type: Sub-task
> Components: test
>Affects Versions: 2.0.0
>Reporter: Jingcheng Du
>Assignee: Jingcheng Du
> Fix For: hbase-11339
>
> Attachments: HBASE-12080.diff
>
>
> By default, using "mvn failsafe:integration-test" to execute the integration
> test with MOB will run more than 10 minutes. In this JIRA, we'll shorten this
> run time by default.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12066) Avoid major compaction in TestMobSweeper
Jonathan Hsieh created HBASE-12066: -- Summary: Avoid major compaction in TestMobSweeper Key: HBASE-12066 URL: https://issues.apache.org/jira/browse/HBASE-12066 Project: HBase Issue Type: Bug Reporter: Jonathan Hsieh TestMobSweeper is flaky on some machines. This is due to major compactions happening and blocking the unit test's sweep job from running. This patch bumps up the min hfiles before compaction to properly exercise the sweeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
