[jira] [Created] (HBASE-19653) Reduce the default hbase.client.start.log.errors.counter

2017-12-27 Thread Guanghao Zhang (JIRA) 
Guanghao Zhang created HBASE-19653:
--

 Summary: Reduce the default hbase.client.start.log.errors.counter
 Key: HBASE-19653
 URL: https://issues.apache.org/jira/browse/HBASE-19653
 Project: HBase
  Issue Type: Sub-task
Reporter: Guanghao Zhang


As we reduce the default retries number to 10 and now the default start log 
errors counter is 9. So it only log the error at the last retry. So we should 
reduce the default hbase.client.start.log.errors.counter, too.

{code}
  /**
   * Configure the number of failures after which the client will start 
logging. A few failures
   * is fine: region moved, then is not opened, then is overloaded. We try to 
have an acceptable
   * heuristic for the number of errors we don't log. 9 was chosen because we 
wait for 1s at
   * this stage.
   */
  public static final String START_LOG_ERRORS_AFTER_COUNT_KEY =
  "hbase.client.start.log.errors.counter";
  public static final int DEFAULT_START_LOG_ERRORS_AFTER_COUNT = 9;
{code}

{code}
public static final int [] RETRY_BACKOFF = {1, 2, 3, 5, 10, 20, 40, 100, 100, 
100, 100, 200, 200};
public static final long DEFAULT_HBASE_CLIENT_PAUSE = 100;
{code}

The default pause is 100ms and 100ms * 10 = 1s. The old comment of 
DEFAULT_START_LOG_ERRORS_AFTER_COUNT seems not right...

Open this issue to reduce the default hbase.client.start.log.errors.counter to 
5.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-19652) Turn down CleanerChore logging; too chatty

2017-12-27 Thread stack (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-19652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-19652.
---
   Resolution: Fixed
 Assignee: stack
Fix Version/s: 2.0.0-beta-1

.001 is what I pushed to master and branch-2. Logging level changes.

> Turn down CleanerChore logging; too chatty
> --
>
> Key: HBASE-19652
> URL: https://issues.apache.org/jira/browse/HBASE-19652
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19652.master.001.patch
>
>
> Running on cluster, the cleaner chore is too chatty. See over in the tail of 
> HBASE-18309 where [~reidchan] says it was verbose so he could debug. This 
> issue is about dialing it back and reporting errors only unless TRACE is 
> enabled.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19652) Turn down CleanerChore logging; too chatty

2017-12-27 Thread stack (JIRA) 
stack created HBASE-19652:
-

 Summary: Turn down CleanerChore logging; too chatty
 Key: HBASE-19652
 URL: https://issues.apache.org/jira/browse/HBASE-19652
 Project: HBase
  Issue Type: Task
Reporter: stack


Running on cluster, the cleaner chore is too chatty. See over in the tail of 
HBASE-18309 where [~reidchan] says it was verbose so he could debug. This issue 
is about dialing it back and reporting errors only unless TRACE is enabled.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19651) Remove LimitInputStream

2017-12-27 Thread BELUGA BEHR (JIRA) 
BELUGA BEHR created HBASE-19651:
---

 Summary: Remove LimitInputStream
 Key: HBASE-19651
 URL: https://issues.apache.org/jira/browse/HBASE-19651
 Project: HBase
  Issue Type: Improvement
  Components: hbase
Affects Versions: 3.0.0, 2.0.0-beta-2
Reporter: BELUGA BEHR
Priority: Minor


Let us "drink our own champagne" and use the existing Apache Commons 
BoundedInputStream instead.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19650) ExpiredMobFileCleaner has wrong logic about TTL check

2017-12-27 Thread chenxu (JIRA) 
chenxu created HBASE-19650:
--

 Summary: ExpiredMobFileCleaner has wrong logic about TTL check
 Key: HBASE-19650
 URL: https://issues.apache.org/jira/browse/HBASE-19650
 Project: HBase
  Issue Type: Bug
  Components: mob
Reporter: chenxu
Assignee: chenxu


If today is 2017-12-28 00:00:01, and TTL is set to 86400, when 
MobUtils.cleanExpiredMobFiles execute, expireDate will be 1514304000749, but 
fileDate is 151430400. So the fileDate before the expireDate and mobfiles 
generated in is 2017-12-27 will all are deleted. But in fact, we want to delete 
files before 2017-12-27.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-19640) Change in filterAllRemaining() impl for FilterListWithOR

2017-12-27 Thread Zheng Hu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-19640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Hu resolved HBASE-19640.
--
Resolution: Fixed

> Change in filterAllRemaining() impl for FilterListWithOR
> 
>
> Key: HBASE-19640
> URL: https://issues.apache.org/jira/browse/HBASE-19640
> Project: HBase
>  Issue Type: Bug
>  Components: Filters
>Affects Versions: 2.0.0-beta-1
>Reporter: Ankit Singhal
>Assignee: Zheng Hu
>
> filterAllRemaining() in FilterListWithOR will now return false if the filter 
> list is empty whereas earlier it used to return true for 
> Operator.MUST_PASS_ONE.
> *In HBase-2.0*
> {code}
>  @Override
>   public boolean filterAllRemaining() throws IOException {
> if (isEmpty()) {
>   return super.filterAllRemaining();//false
> }
> for (int i = 0, n = filters.size(); i < n; i++) {
>   if (!filters.get(i).filterAllRemaining()) {
> return false;
>   }
> }
> return true;
>   }
> {code}
> Earlier versions(1.3.1 atleast):-
> {code}
> @Override
>   public boolean filterAllRemaining() throws IOException {
> int listize = filters.size();
> for (int i = 0; i < listize; i++) {
>   if (filters.get(i).filterAllRemaining()) {
> if (operator == Operator.MUST_PASS_ALL) {
>   return true;
> }
>   } else {
> if (operator == Operator.MUST_PASS_ONE) {
>   return false;
> }
>   }
> }
> return operator == Operator.MUST_PASS_ONE;
>   }
> {code}
> IMO, the current implementation seems to be right, but probably this change 
> requires a release note at least as some people might have implemented the 
> filters considering the old semantics.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19649) Use Singleton Feature for ImmutableSegment.java

2017-12-27 Thread BELUGA BEHR (JIRA) 
BELUGA BEHR created HBASE-19649:
---

 Summary: Use Singleton Feature for ImmutableSegment.java
 Key: HBASE-19649
 URL: https://issues.apache.org/jira/browse/HBASE-19649
 Project: HBase
  Issue Type: Improvement
  Components: hbase
Affects Versions: 3.0.0, 2.0.0-beta-2
Reporter: BELUGA BEHR
Priority: Trivial


* Cleanup import list
* Use Collections.singletonList instead of the current round-about way of doing 
it



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-19648) Move branch-2 version from 2.0.0-beta-1-SNAPSHOT to 2.0.0-beta-1

2017-12-27 Thread stack (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-19648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-19648.
---
Resolution: Fixed

Pushed to branch-2.

> Move branch-2 version from 2.0.0-beta-1-SNAPSHOT to 2.0.0-beta-1
> 
>
> Key: HBASE-19648
> URL: https://issues.apache.org/jira/browse/HBASE-19648
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19648) Move branch-2 version from 2.0.0-beta-1-SNAPSHOT to 2.0.0-beta-1

2017-12-27 Thread stack (JIRA) 
stack created HBASE-19648:
-

 Summary: Move branch-2 version from 2.0.0-beta-1-SNAPSHOT to 
2.0.0-beta-1
 Key: HBASE-19648
 URL: https://issues.apache.org/jira/browse/HBASE-19648
 Project: HBase
  Issue Type: Sub-task
Reporter: stack
Assignee: stack






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-14562) Fix and reenable zombie TestReplicationShell

2017-12-27 Thread Chia-Ping Tsai (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-14562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai resolved HBASE-14562.

Resolution: Duplicate

hbase-15965

> Fix and reenable zombie TestReplicationShell
> 
>
> Key: HBASE-14562
> URL: https://issues.apache.org/jira/browse/HBASE-14562
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>
> Was disabled over in HBASE-14561 because it hangs with some regularity.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-8053) Reenable TestHBaseFsck#testQuarantineMissingFamdir

2017-12-27 Thread Chia-Ping Tsai (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-8053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai resolved HBASE-8053.
---
Resolution: Won't Fix

The test is gone. HBASE-19272

> Reenable TestHBaseFsck#testQuarantineMissingFamdir
> --
>
> Key: HBASE-8053
> URL: https://issues.apache.org/jira/browse/HBASE-8053
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>
> HBASE-8052 disabled it.  Reenable after figure why it fails sporadically.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-9010) Reenable TestMultiTableInputFormat

2017-12-27 Thread Chia-Ping Tsai (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-9010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai resolved HBASE-9010.
---
Resolution: Duplicate

HBASE-10774

> Reenable TestMultiTableInputFormat
> --
>
> Key: HBASE-9010
> URL: https://issues.apache.org/jira/browse/HBASE-9010
> Project: HBase
>  Issue Type: Task
>  Components: test
>Reporter: stack
>
> See HBASE-9009 where we disable it because it fails in mysterious ways.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-19647) Logging cleanups; emit regionname when RegionTooBusyException inside RetriesExhausted... make netty connect/disconnect TRACE-level

2017-12-27 Thread stack (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-19647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-19647.
---
   Resolution: Fixed
 Assignee: stack
Fix Version/s: 2.0.0-beta-1

Pushed associated patch that adds regionname printing when Retries exhausted, 
makes connect/disconnect from netty trace-level and changes in-memory 
compaction messages to be trace in some instances (its too noisy).

> Logging cleanups; emit regionname when RegionTooBusyException inside 
> RetriesExhausted... make netty connect/disconnect TRACE-level
> --
>
> Key: HBASE-19647
> URL: https://issues.apache.org/jira/browse/HBASE-19647
> Project: HBase
>  Issue Type: Improvement
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
> Attachments: 19647.patch
>
>
> In MR failures, i see this:
>   Error: 
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 
> 365 actions: RegionTooBusyException: 365 times, servers with issues: 
> ve0534.halxg.cloudera.com,16020,1514392912363 at 
> org.apache.hadoop.hbase.client.BatchErrors.makeException(BatchErrors.java:54) 
> at 
> org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:491)
>  at 
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:268)
>  at 
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:225)
>  at 
> org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator$GeneratorMapper.persist(IntegrationTestBigLinkedList.java:541)
>  at 
> org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator$GeneratorMapper.map(IntegrationTestBigLinkedList.java:464)
>  at 
> org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator$GeneratorMapper.map(IntegrationTestBigLinkedList.java:399)
>  at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at 
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at 
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at 
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at 
> java.security.AccessController.doPrivileged(Native Method) at 
> javax.security.auth.Subject.doAs(Subject.java:422) at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746)
>  at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) [2017-12-27 
> 09:09:11.570]Container killed by the ApplicationMaster. [2017-12-27 
> 09:09:11.586]Container killed on request. Exit code is 143 [2017-12-27 
> 09:09:11.600]Container exited with a non-zero exit code 143.
> Its missing the region name which is in the root exception, 
> RegionTooBusyException -- we just skip it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-6127) TestAtomicOperation#testMultiRowMutationMultiThreads occasionally fails

2017-12-27 Thread Chia-Ping Tsai (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai resolved HBASE-6127.
---
Resolution: Duplicate

Looped the test 20 times for branch-1 and master. no failure. Seems it was 
resolved by HBASE-14794.

> TestAtomicOperation#testMultiRowMutationMultiThreads occasionally fails
> ---
>
> Key: HBASE-6127
> URL: https://issues.apache.org/jira/browse/HBASE-6127
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: Ted Yu
>
> TestAtomicOperation occasionally fails.
> Here is one instance:
> https://builds.apache.org/job/HBase-TRUNK/2944/testReport/junit/org.apache.hadoop.hbase.regionserver/TestAtomicOperation/testMultiRowMutationMultiThreads/



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-7141) Cleanup Increment and Append issues

2017-12-27 Thread Chia-Ping Tsai (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-7141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai resolved HBASE-7141.
---
Resolution: Duplicate

bq. Append and Increment should take a TS for their update phase
HBASE-18546

bq. Append should access a timerange for the read phase
HBASE-19427

bq. Increment should no longer implement Writable (in trunk)
HBASE-7215

bq. Increment and Append makes changes visible through the memstore before the 
WAL is sync'ed
HBASE-4583


> Cleanup Increment and Append issues
> ---
>
> Key: HBASE-7141
> URL: https://issues.apache.org/jira/browse/HBASE-7141
> Project: HBase
>  Issue Type: Bug
>Reporter: Lars Hofhansl
>
> * Append and Increment should take a TS for their update phase
> * Append should access a timerange for the read phase
> * Increment should no longer implement Writable (in trunk)
> * Increment and Append makes changes visible through the memstore before the 
> WAL is sync'ed
> This depends on HBASE-4583



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-5851) TestProcessBasedCluster sometimes fails; currently disabled -- needs to be fixed and reenabled

2017-12-27 Thread Chia-Ping Tsai (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai resolved HBASE-5851.
---
Resolution: Won't Fix

see HBASE-14571

> TestProcessBasedCluster sometimes fails; currently disabled -- needs to be 
> fixed and reenabled
> --
>
> Key: HBASE-5851
> URL: https://issues.apache.org/jira/browse/HBASE-5851
> Project: HBase
>  Issue Type: Test
>Reporter: Ted Yu
> Attachments: disable.txt, hbase-5851.patch, hbase-5851_v2.patch, 
> metahang.txt, zkfail.txt
>
>
> TestProcessBasedCluster failed in 
> https://builds.apache.org/job/HBase-TRUNK-security/178
> Looks like cluster failed to start:
> {code}
> 2012-04-21 14:22:32,666 INFO  [Thread-1] 
> util.ProcessBasedLocalHBaseCluster(176): Waiting for HBase to startup. 
> Retries left: 2
> java.io.IOException: Giving up trying to location region in meta: thread is 
> interrupted.
>   at 
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1173)
>   at 
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:956)
>   at 
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:917)
>   at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:252)
>   at org.apache.hadoop.hbase.client.HTable.(HTable.java:192)
>   at 
> org.apache.hadoop.hbase.util.ProcessBasedLocalHBaseCluster.startHBase(ProcessBasedLocalHBaseCluster.java:174)
>   at 
> org.apache.hadoop.hbase.util.TestProcessBasedCluster.testProcessBasedCluster(TestProcessBasedCluster.java:56)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>   at java.lang.reflect.Method.invoke(Method.java:597)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> java.lang.InterruptedException: sleep interrupted at 
> java.lang.Thread.sleep(Native Method)
>   at org.apache.hadoop.hbase.util.Threads.sleep(Threads.java:134)
>   at 
> org.apache.hadoop.hbase.util.ProcessBasedLocalHBaseCluster.startHBase(ProcessBasedLocalHBaseCluster.java:178)
>   at 
> org.apache.hadoop.hbase.util.TestProcessBasedCluster.testProcessBasedCluster(TestProcessBasedCluster.java:56)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19647) Logging cleanups; emit regionname when RegionTooBusyException inside RetriesExhausted... make netty connect/disconnect TRACE-level

2017-12-27 Thread stack (JIRA) 
stack created HBASE-19647:
-

 Summary: Logging cleanups; emit regionname when 
RegionTooBusyException inside RetriesExhausted... make netty connect/disconnect 
TRACE-level
 Key: HBASE-19647
 URL: https://issues.apache.org/jira/browse/HBASE-19647
 Project: HBase
  Issue Type: Improvement
Reporter: stack






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Moving 2.0 forward

2017-12-27 Thread Stack 
Heads-up, I'm working on cutting a beta-1. Shout if there is something you
think needs to make it in (The Chia-Ping Tsai cleanups look good). I'm
mainly waiting on the hbase-thirdparty changes to go in. The rest can wait
it.

Thanks,
S


On Wed, Dec 13, 2017 at 11:28 PM, Stack  wrote:

> Update. See below.
>
> On Tue, Dec 5, 2017 at 10:43 AM, Stack  wrote:
>
>> Update on hbase-2.0.0-beta-1. I'm going to put up a beta-1 RC before
>> Christmas day.
>>
>>
> Still aiming for an hbase2 beta-1 showing up in your christmas stocking.
>
>
>> Here is the list [1]. We're doing pretty good. We've fixed a load since
>> the last update including some nice cleanups  two weeks ago (e.g. undoing
>> client dependency on curator/zk watching making the dependence read-only)
>> but there are still a few hairy ones hanging out there.
>>
>>  * HBASE-18946 Stochastic load balancer assigns replica regions to the
>> same RS
>>This one is proving a little tough. Ram has been banging his head on
>> it. We'll be able to clear a bunch of test failures once this is done.
>>
>>
> This should be going in tomorrow.
>
>
>
>>  * HBASE-19112 Suspect methods on Cell to be deprecated
>>Good discussion up on the JIRA. We need to be crystal clear around
>> Cell and derivatives API and Audience when we ship 2.0.
>>
>>
> Ram, Chia-Ping Tsai, and Anoop are doing a nice job here and it should be
> done in next day or so.
>
>
>
>>  * HBASE-17204 Make L2 off heap cache default ON
>>We need to try this at least so offheaping can be default.
>>
>>
> Chatted with Anoop and Ram. Because it is so late in the day and because
> it critical that the user have a good experience out of the box -- which
> requires time and testing -- we are going to punt this from 2.0. Will look
> at it for 2.1 (or 3.0 if soon). 2.0 will still be carrying all of the boys
> offheap read/write path goodness.
>
>
>
>> Making asyncfswal default is also ongoing, HBASE-16890
>> , making good
>> progress.
>>
>>
>
> Good progress here. In perf testing asyncfswal makes us generally faster.
> Duo making good progress on last few tests that are in the way of our
> making this default for hbase2.
>
> A load of goodness has been landing these last few weeks. Keep up the
> great work.
>
> Your hbase-2 RM.
>
>
>
>
>> I punted HBASE-19147 "All branch-2 unit tests pass" out to beta-2. Our
>> unit test story has gotten better and a few of us are actively working on
>> flakies [2] but we may make a beta-1 w/o shutting down all failures.
>>
>> Speak up if you need help on an issue or if you think we are missing
>> items from our list.
>>
>> Thanks for all the hard work,
>>
>> The hbase-2.0.0 RM.
>>
>>
>> 1. https://issues.apache.org/jira/projects/HBASE/versions/12340861
>> 2. https://builds.apache.org/job/HBASE-Find-Flaky-Tests/last
>> SuccessfulBuild/artifact/dashboard.html
>>
>> On Thu, Nov 16, 2017 at 3:48 PM, Stack  wrote:
>>
>>> hbase-2.0.0-beta-1 update (Reminder, beta-1 is where we finish last
>>> remaining features and apply final polish to API. There will be a beta-2
>>> but it is about upgrade/rolling-upgrade and bug fixes ONLY).
>>>
>>> Myself and Mr. Drob did a pass over the outstanding hbase-2.0.0-beta-1
>>> list this morning. See here [1].
>>>
>>> There are about ~12 issues in progress with most of these about to land.
>>> There are 37 TODO. Many of these are tests we need to run, some are related
>>> to the backup/restore, but a good few are meaty w/o assignees.
>>>
>>> The awkward outstanding ones as I see it are the below:
>>>
>>> HBASE-18946 Stochastic load balancer assigns replica regions to the same
>>> RS
>>> HBASE-17204 Make L2 off heap cache default ON
>>> HBASE-19112 Suspect methods on Cell to be deprecated
>>> HBASE-19147 All branch-2 unit tests pass
>>>
>>> We need to make progress on the above or punt on them (can't punt on the
>>> last one though).
>>>
>>> Any ideas on what configs we should update in hbase2? Dump ideas into:
>>> HBASE-19148 Edit of default configuration
>>>
>>> Still hoping for an early December beta-1 RC. beta-2 hopefully will be
>>> close behind.
>>>
>>> Comments? Thoughts?
>>>
>>> Thanks all,
>>> S
>>>
>>> 1. https://issues.apache.org/jira/projects/HBASE/versions/12340861
>>>
>>> On Tue, Oct 31, 2017 at 2:56 PM, Stack  wrote:
>>>


 On Tue, Oct 31, 2017 at 2:31 PM, Mike Drob  wrote:

> Hoping to keep momentum going from our Stack working on alpha4, I
> tried to
> take a stab at triaging some of the open beta-1 issues.
>
> I moved some docs stuff out form beta-1 to 2.0-GA, if it gets done
> sooner
> then I'm happy to see it pulled back in. Trying to balance optimism
> with
> realism here, and knowing that documentation unfortunately often gets
> pushed to the back-burner.
>
> Also, I poked some folks on unassigned issues that they've

[jira] [Created] (HBASE-19646) Add CRON To Major Compaction

2017-12-27 Thread BELUGA BEHR (JIRA) 
BELUGA BEHR created HBASE-19646:
---

 Summary: Add CRON To Major Compaction
 Key: HBASE-19646
 URL: https://issues.apache.org/jira/browse/HBASE-19646
 Project: HBase
  Issue Type: Bug
  Components: Compaction
Reporter: BELUGA BEHR
Priority: Minor


HBase provides _hbase.hregion.majorcompaction_ 

{quote}
Time between major compactions, expressed in milliseconds. Set to 0 to disable 
time-based automatic major compactions. User-requested and size-based major 
compactions will still run. This value is multiplied by 
hbase.hregion.majorcompaction.jitter to cause compaction to start at a 
somewhat-random time during a given window of time. The default value is 7 
days, expressed in milliseconds. If major compactions are causing disruption in 
your environment, you can configure them to run at off-peak times for your 
deployment, or disable time-based major compactions by setting this parameter 
to 0, and run major compactions in a cron job or by another external mechanism.
{quote}

Instead of this existing mechanism, that adds randomness into a production 
system (ugh), let's simply allow users to specify a cron string and replace 
this simple periodic (+jitter) scheduling mechanism.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19645) Add global barrier for offheap usage

2017-12-27 Thread stack (JIRA) 
stack created HBASE-19645:
-

 Summary: Add global barrier for offheap usage
 Key: HBASE-19645
 URL: https://issues.apache.org/jira/browse/HBASE-19645
 Project: HBase
  Issue Type: Task
Reporter: stack


See discussion at the end of HBASE-16747. In synopsis, we do not have a global 
limit for offheap usage; we do no global accounting. There are the offheap 
buffers, short-circuit read buffers, offheap memstores, etc. Ideally, as for 
onheap, when we approach threshold, we'd bring on flushing to clear out offheap 
usage or make use of onheap. TODO.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19644) add the checkstyle rule to rejects the illegal imports

2017-12-27 Thread Chia-Ping Tsai (JIRA) 
Chia-Ping Tsai created HBASE-19644:
--

 Summary: add the checkstyle rule to rejects the illegal imports
 Key: HBASE-19644
 URL: https://issues.apache.org/jira/browse/HBASE-19644
 Project: HBase
  Issue Type: Task
Affects Versions: 2.0.0
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai


The following imports should be disabled in our code.
# com.google.common.*   (we have the shaded version)
# io.netty.*  (ditto)
# org.apache.curator.shaded.*
# org.apache.htrace.shaded.*



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Reopened] (HBASE-19544) Add UTs for testing concurrent modifications on replication peer

2017-12-27 Thread Duo Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-19544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang reopened HBASE-19544:
---
  Assignee: Guanghao Zhang

Seems the previous concurrent UT for replication peer modification does not 
work properly as before HBASE-19642 we do have problems when modifying peers 
concurrently...

We need to revisit the concurrent UT to see why it does not work.

> Add UTs for testing concurrent modifications on replication peer
> 
>
> Key: HBASE-19544
> URL: https://issues.apache.org/jira/browse/HBASE-19544
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2, Replication, test
>Reporter: Duo Zhang
>Assignee: Guanghao Zhang
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Accessing region state via web UI

2017-12-27 Thread Janos Gub 
I can align with putting it into the rest endpoint if we agree on that exposing 
it is not a problem.

Janos

On 12/25/17, 6:46 AM, "stack"  wrote:

REST endpoint more natural loci for programmatic consumers imo

On Dec 24, 2017 5:22 PM, "Apekshit Sharma"  wrote:

> I started looking into more after last reply.
> I think exposing this info can be useful to the operators - dashboard of
> region states, or tools to quickly dump list of regions which require 
care,
> etc
> But wasn't thinking about hbase rest api so far.
> I think web ui is not the right endpoint to expose json api when we 
already
> have a whole well designed way to do it -  REST apis.
> It should rather be an addition to Table endpoints (table 14 here:
> http://hbase.apache.org/book.html#_using_rest_endpoints).
>
> -- Appy
>
> On Sun, Dec 24, 2017 at 5:16 PM, Apekshit Sharma 
> wrote:
>
> > More details:
> > Proposed interface is json.
> > Pros:
> > - more info exposed to operators, can be used for scripting
> >
> > Cons:
> > - We don't have functions in Admin to expose region states
> > - We don't expose much programatic info through our web ui.
> >
> > -- Appy
> >
> >
> >
> > On Fri, Dec 22, 2017 at 6:01 PM, Ted Yu  wrote:
> >
> >> Hi,
> >> See (tail of) HBASE-17436 for detailed background information.
> >>
> >> Here is the summary:
> >>
> >> In HBASE-17436 (and HBASE-19594), region state is accessed thru web UI.
> >> This is for helping operators gain better insight into the cluster
> status.
> >>
> >> However, since RegionState is marked @InterfaceAudience.Private , we
> (Appy
> >> and I) want to present this discussion to the community to raise
> >> awareness.
> >>
> >> Your comment is welcome.
> >>
> >
> >
> >
> > --
> >
> > -- Appy
> >
>
>
>
> --
>
> -- Appy
>

[jira] [Created] (HBASE-19643) Need to update cache location when get error in AsyncBatchRpcRetryingCaller

2017-12-27 Thread Guanghao Zhang (JIRA) 
Guanghao Zhang created HBASE-19643:
--

 Summary: Need to update cache location when get error in 
AsyncBatchRpcRetryingCaller
 Key: HBASE-19643
 URL: https://issues.apache.org/jira/browse/HBASE-19643
 Project: HBase
  Issue Type: Bug
Reporter: Guanghao Zhang
Assignee: Guanghao Zhang






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19642) Fix locking for peer modification procedure

2017-12-27 Thread Duo Zhang (JIRA) 
Duo Zhang created HBASE-19642:
-

 Summary: Fix locking for peer modification procedure
 Key: HBASE-19642
 URL: https://issues.apache.org/jira/browse/HBASE-19642
 Project: HBase
  Issue Type: Sub-task
Reporter: Duo Zhang


We should hold the lock through the whole execution even if the parent 
procedure is suspended to wait for the refresh result.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19641) AsyncHBaseAdmin should use exponential backoff when polling the procedure result

2017-12-27 Thread Duo Zhang (JIRA) 
Duo Zhang created HBASE-19641:
-

 Summary: AsyncHBaseAdmin should use exponential backoff when 
polling the procedure result
 Key: HBASE-19641
 URL: https://issues.apache.org/jira/browse/HBASE-19641
 Project: HBase
  Issue Type: Improvement
  Components: asyncclient, proc-v2
Reporter: Duo Zhang


Now it polls too frequent and in tests there are so many debug logs...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)