[jira] [Created] (HBASE-19003) Make sure all balancer actions respect decommissioned server

2017-10-12 Thread Jerry He (JIRA)
Jerry He created HBASE-19003:


 Summary: Make sure all balancer actions respect decommissioned 
server
 Key: HBASE-19003
 URL: https://issues.apache.org/jira/browse/HBASE-19003
 Project: HBase
  Issue Type: Sub-task
Reporter: Jerry He


There have been questions raised in HBASE-10367 and other related JIRAs. We 
want to make sure all aspects of the balancer respect the draining flag. We 
will have a good look, and fix if any violation is found.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (HBASE-19002) Introduce more examples to show how to intercept normal region operations

2017-10-12 Thread Duo Zhang (JIRA)
Duo Zhang created HBASE-19002:
-

 Summary: Introduce more examples to show how to intercept normal 
region operations
 Key: HBASE-19002
 URL: https://issues.apache.org/jira/browse/HBASE-19002
 Project: HBase
  Issue Type: Sub-task
Reporter: Duo Zhang






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (HBASE-19001) Remove StoreScanner dependency in our own CP related tests

2017-10-12 Thread Duo Zhang (JIRA)
Duo Zhang created HBASE-19001:
-

 Summary: Remove StoreScanner dependency in our own CP related tests
 Key: HBASE-19001
 URL: https://issues.apache.org/jira/browse/HBASE-19001
 Project: HBase
  Issue Type: Sub-task
  Components: Coprocessors
Reporter: Duo Zhang
 Fix For: 2.0.0-alpha-4






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (HBASE-19000) Group multiple block cache clear requests per server

2017-10-12 Thread Ted Yu (JIRA)
Ted Yu created HBASE-19000:
--

 Summary: Group multiple block cache clear requests per server
 Key: HBASE-19000
 URL: https://issues.apache.org/jira/browse/HBASE-19000
 Project: HBase
  Issue Type: Sub-task
Reporter: Ted Yu


During the review of HBASE-18624, I brought up the following:

There would be multiple regions on the same server whose block cache is to be 
cleared.
Multiple block cache clear requests should be grouped per server to reduce the 
number of RPC calls.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (HBASE-18999) Put in hbase shell cannot do multiple columns

2017-10-12 Thread Mike Drob (JIRA)
Mike Drob created HBASE-18999:
-

 Summary: Put in hbase shell cannot do multiple columns
 Key: HBASE-18999
 URL: https://issues.apache.org/jira/browse/HBASE-18999
 Project: HBase
  Issue Type: Improvement
  Components: shell
Affects Versions: 1.0.0, 2.0.0
Reporter: Mike Drob


A {{Put}} can carry multiple cells, but doing so in the shell is very difficult 
to construct. We should make this easier.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (HBASE-18998) processor.getRowsToLock() always assumes there is some row being locked

2017-10-12 Thread Ted Yu (JIRA)
Ted Yu created HBASE-18998:
--

 Summary: processor.getRowsToLock() always assumes there is some 
row being locked
 Key: HBASE-18998
 URL: https://issues.apache.org/jira/browse/HBASE-18998
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu


During testing, we observed the following exception:
{code}
2017-10-12 02:52:26,683|INFO|MainThread|machine.py:164 - 
run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|1/1  DROP TABLE 
testTable;
2017-10-12 02:52:30,320|INFO|MainThread|machine.py:164 - 
run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|17/10/12 02:52:30 WARN 
ipc.CoprocessorRpcChannel: Call failed on IOException
2017-10-12 02:52:30,320|INFO|MainThread|machine.py:164 - 
run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|org.apache.hadoop.hbase.DoNotRetryIOException:
 org.apache.hadoop.hbase.DoNotRetryIOException: TESTTABLE: null
2017-10-12 02:52:30,321|INFO|MainThread|machine.py:164 - 
run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at 
org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:93)
2017-10-12 02:52:30,321|INFO|MainThread|machine.py:164 - 
run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at 
org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropTable(MetaDataEndpointImpl.java:1671)
2017-10-12 02:52:30,321|INFO|MainThread|machine.py:164 - 
run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at 
org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:14347)
2017-10-12 02:52:30,321|INFO|MainThread|machine.py:164 - 
run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at 
org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7849)
2017-10-12 02:52:30,321|INFO|MainThread|machine.py:164 - 
run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at 
org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1980)
2017-10-12 02:52:30,321|INFO|MainThread|machine.py:164 - 
run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at 
org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1962)
2017-10-12 02:52:30,321|INFO|MainThread|machine.py:164 - 
run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at 
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32389)
2017-10-12 02:52:30,322|INFO|MainThread|machine.py:164 - 
run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at 
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2150)
2017-10-12 02:52:30,322|INFO|MainThread|machine.py:164 - 
run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at 
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
2017-10-12 02:52:30,322|INFO|MainThread|machine.py:164 - 
run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at 
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
2017-10-12 02:52:30,322|INFO|MainThread|machine.py:164 - 
run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at 
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
2017-10-12 02:52:30,322|INFO|MainThread|machine.py:164 - 
run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|Caused by: 
java.util.NoSuchElementException
2017-10-12 02:52:30,322|INFO|MainThread|machine.py:164 - 
run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at 
java.util.Collections$EmptyIterator.next(Collections.java:4189)
2017-10-12 02:52:30,322|INFO|MainThread|machine.py:164 - 
run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at 
org.apache.hadoop.hbase.regionserver.HRegion.processRowsWithLocks(HRegion.java:7137)
2017-10-12 02:52:30,322|INFO|MainThread|machine.py:164 - 
run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at 
org.apache.hadoop.hbase.regionserver.HRegion.mutateRowsWithLocks(HRegion.java:6980)
2017-10-12 02:52:30,322|INFO|MainThread|machine.py:164 - 
run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at 
org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateRowsWithLocks(MetaDataEndpointImpl.java:1966)
2017-10-12 02:52:30,323|INFO|MainThread|machine.py:164 - 
run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at 
org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropTable(MetaDataEndpointImpl.java:1650)
{code}
Here is code from branch-1.1 :
{code}
if (!mutations.isEmpty() && !walSyncSuccessful) {
  LOG.warn("Wal sync failed. Roll back " + mutations.size() +
  " memstore keyvalues for row(s):" + StringUtils.byteToHexString(
  processor.getRowsToLock().iterator().next()) + "...");
{code}
The assumption that processor.getRowsToLock().iterator() would always be 
non-empty was wrong.

In other branches, taking the iterator seems to have the same issue.

Thanks to [~elserj] who spotted this issue.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (HBASE-18997) Remove the redundant methods in RegionInfo

2017-10-12 Thread Chia-Ping Tsai (JIRA)
Chia-Ping Tsai created HBASE-18997:
--

 Summary: Remove the redundant methods in RegionInfo
 Key: HBASE-18997
 URL: https://issues.apache.org/jira/browse/HBASE-18997
 Project: HBase
  Issue Type: Bug
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai


The {{isMetaTable}} and {{isSystemTable}} are short cut to {{TableName}}. Our 
IA.Public interface should be minimalist 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (HBASE-18996) Backport HBASE-17703 (TestThriftServerCmdLine is flaky in master branch) to branch-1

2017-10-12 Thread Peter Somogyi (JIRA)
Peter Somogyi created HBASE-18996:
-

 Summary: Backport HBASE-17703 (TestThriftServerCmdLine is flaky in 
master branch) to branch-1
 Key: HBASE-18996
 URL: https://issues.apache.org/jira/browse/HBASE-18996
 Project: HBase
  Issue Type: Task
  Components: Thrift
Affects Versions: 1.3.2, 1.4.1, 1.5.0, 1.2.7, 1.1.13
Reporter: Peter Somogyi
Assignee: Peter Somogyi


During HBASE-18967 backport to branch-1.3 a test failure came up which was 
fixed in HBASE-17703.
We need to backport it to branch-1. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (HBASE-18995) Move methods that are for internal usage from CellUtil to Private util class

2017-10-12 Thread ramkrishna.s.vasudevan (JIRA)
ramkrishna.s.vasudevan created HBASE-18995:
--

 Summary: Move methods that are for internal usage from CellUtil to 
Private util class
 Key: HBASE-18995
 URL: https://issues.apache.org/jira/browse/HBASE-18995
 Project: HBase
  Issue Type: Improvement
Affects Versions: 2.0.0-alpha-3
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 2.0.0-alpha-4


This was brought up long time back. We need to move some of the public APIs 
from CellUtil to internal private Util class because they are used in some 
internal flow and does not make sense to have it in a @public exposed Util 
class. 
The topic again came in HBASE-18945 RB comments also.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (HBASE-18994) Decide if META should use Compacting Memstore or Default Memstore

2017-10-12 Thread ramkrishna.s.vasudevan (JIRA)
ramkrishna.s.vasudevan created HBASE-18994:
--

 Summary: Decide if META should use Compacting Memstore or Default 
Memstore
 Key: HBASE-18994
 URL: https://issues.apache.org/jira/browse/HBASE-18994
 Project: HBase
  Issue Type: Improvement
Affects Versions: 2.0.0-alpha-3
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 2.0.0-alpha-4


HBASE-18992 brought this topic. Currently META is also using Compacting 
Memstore. We should decide if system tables can go with Default memstore only.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (HBASE-18296) NPE in ProcedureExecutor.removeChore

2017-10-12 Thread Mike Drob (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-18296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob resolved HBASE-18296.
---
Resolution: Cannot Reproduce

I haven't seen this again, and I don't remember where it came from. If it pops 
up again I'll revisit it.

> NPE in ProcedureExecutor.removeChore
> 
>
> Key: HBASE-18296
> URL: https://issues.apache.org/jira/browse/HBASE-18296
> Project: HBase
>  Issue Type: Bug
>  Components: proc-v2
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
>
> Saw this in logs while debugging other failing tests:
> {noformat}
> 2017-06-29 13:54:15,995 ERROR [M:0;172.18.16.40:55484] 
> server.NIOServerCnxnFactory$1(44): Thread 
> Thread[M:0;172.18.16.40:55484,5,FailOnTimeoutGroup] died
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.removeChore(ProcedureExecutor.java:656)
>   at 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.stop(AssignmentManager.java:233)
>   at 
> org.apache.hadoop.hbase.master.HMaster.stopServiceThreads(HMaster.java:1154)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1189)
>   at java.lang.Thread.run(Thread.java:748)
> {noformat}
> I think it was related to an initialization failure, but we should code more 
> defensively anyway



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (HBASE-18993) Backport patches in HBASE-18410 to branch-1.x branches.

2017-10-12 Thread Zheng Hu (JIRA)
Zheng Hu created HBASE-18993:


 Summary: Backport patches in HBASE-18410 to branch-1.x branches.
 Key: HBASE-18993
 URL: https://issues.apache.org/jira/browse/HBASE-18993
 Project: HBase
  Issue Type: Sub-task
Reporter: Zheng Hu
Assignee: Zheng Hu






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (HBASE-18992) Comparators passed to the Memstore's flattened segments seems to be wrong

2017-10-12 Thread ramkrishna.s.vasudevan (JIRA)
ramkrishna.s.vasudevan created HBASE-18992:
--

 Summary: Comparators passed to the Memstore's flattened segments 
seems to be wrong
 Key: HBASE-18992
 URL: https://issues.apache.org/jira/browse/HBASE-18992
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0-alpha-3
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 2.0.0-beta-1


While doing HBASE-18945 found that the comparators passed to the 
CellChunkImmutableSegment and CellArrayImmutableSegment are sometimes wrong. We 
pass the comparator that we got from the Store but when doing 
reinitializeCellSet we always use CellComparator.COMPARATOR.
>From code I don think we are stopping the META region to also go through 
>CompactingMemstore. If we do that then this JIRA is invalid. But considering 
>that is not the case we should fix it and pass the right comparator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (HBASE-18991) Remove RegionMergeRequest

2017-10-12 Thread Chia-Ping Tsai (JIRA)
Chia-Ping Tsai created HBASE-18991:
--

 Summary: Remove RegionMergeRequest
 Key: HBASE-18991
 URL: https://issues.apache.org/jira/browse/HBASE-18991
 Project: HBase
  Issue Type: Task
Reporter: Chia-Ping Tsai
Priority: Trivial
 Fix For: 2.0.0


{code}
/**
 * Handles processing region merges. Put in a queue, owned by HRegionServer.
 */
// TODO:UNUSED: REMOVE!!!
@InterfaceAudience.Private
class RegionMergeRequest implements Runnable {
{code}
HBASE-14614 has discharged {{RegionMergeRequest}} from duty.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)