[jira] [Comment Edited] (HBASE-20095) Redesign single instance pool in CleanerChore

[Reid Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383243#comment-16383243
 ] 

Reid Chan edited comment on HBASE-20095 at 3/2/18 7:57 AM:
---

Only one updates the static pool.

Because all registered {{ConfigurationObserver}} are called sequentially, let's 
say {{LogCleaner}} is called first, then {{poolSize}} is updated; following 
{{HFileCleaner}} which should see the latest {{poolSize}} and should not 
reconfigure the pool and just return. The snippet,
{code:java}
public void onConfigurationChange(Configuration conf) {
int updatedSize = calculatePoolSize(conf.get(CHORE_POOL_SIZE, 
DEFAULT_CHORE_POOL_SIZE));
if (updatedSize == poolSize) {
  LOG.trace("Size from configuration is same as previous={}, no need to 
update.", updatedSize);
  return;
}
poolSize = updatedSize;
...
}
{code}


was (Author: reidchan):
Only one updates the static pool.

Because all registered {{ConfigurationObserver}} are called sequentially, let's 
say {{LogCleaner}} is called first, then size of pool {{poolSize}} is updated; 
following {{HFileCleaner}} which should see the latest {{updatedSize}} and 
should not reconfigure the pool and just return. The snippet,
{code:java}
public void onConfigurationChange(Configuration conf) {
int updatedSize = calculatePoolSize(conf.get(CHORE_POOL_SIZE, 
DEFAULT_CHORE_POOL_SIZE));
if (updatedSize == poolSize) {
  LOG.trace("Size from configuration is same as previous={}, no need to 
update.", updatedSize);
  return;
}
poolSize = updatedSize;
...
}
{code}

> Redesign single instance pool in CleanerChore
> -
>
> Key: HBASE-20095
> URL: https://issues.apache.org/jira/browse/HBASE-20095
> Project: HBase
>  Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Critical
> Attachments: HBASE-20095.master.001.patch, 
> HBASE-20095.master.002.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HBASE-20115) Reimplement serial replication based on the new replication storage layer

[Duo Zhang (JIRA)]

Duo Zhang created HBASE-20115:
-

 Summary: Reimplement serial replication based on the new 
replication storage layer
 Key: HBASE-20115
 URL: https://issues.apache.org/jira/browse/HBASE-20115
 Project: HBase
  Issue Type: Sub-task
Reporter: Duo Zhang






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20050) Reimplement updateReplicationPositions logic in serial replication based on the newly introduced replication storage layer

[Duo Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-20050:
--
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 3.0.0
   Status: Resolved  (was: Patch Available)

Pushed to master. Thanks [~openinx] for contributing.

> Reimplement updateReplicationPositions logic in serial replication based on 
> the newly introduced replication storage layer
> --
>
> Key: HBASE-20050
> URL: https://issues.apache.org/jira/browse/HBASE-20050
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20050.patch, HBASE-20050.v1.patch, 
> HBASE-20050.v2.patch, HBASE-20050.v3.patch
>
>
> Now, in serial replication,  need to update  the replication position both in 
> zookeeper and meta table, which may led to inconsistent problem. I'll 
> re-implement this part based on the uniform replication storage layer. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-8770) deletes and puts with the same ts should be resolved according to mvcc/seqNum

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-8770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-8770:
-
Assignee: stack
  Status: Patch Available  (was: Open)

> deletes and puts with the same ts should be resolved according to mvcc/seqNum
> -
>
> Key: HBASE-8770
> URL: https://issues.apache.org/jira/browse/HBASE-8770
> Project: HBase
>  Issue Type: Brainstorming
>Reporter: Sergey Shelukhin
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: HBASE-8770.branch-2.001.patch
>
>
> This came up during HBASE-8721. Puts with the same ts are resolved by seqNum. 
> It's not clear why deletes with the same ts as a put should always mask the 
> put, rather than also being resolve by seqNum.
> What do you think?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-8770) deletes and puts with the same ts should be resolved according to mvcc/seqNum

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-8770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383310#comment-16383310
 ] 

stack commented on HBASE-8770:
--

.001 is doc on the HBASE-15968 feature, hopefully enough to close out this 
issue.

> deletes and puts with the same ts should be resolved according to mvcc/seqNum
> -
>
> Key: HBASE-8770
> URL: https://issues.apache.org/jira/browse/HBASE-8770
> Project: HBase
>  Issue Type: Brainstorming
>Reporter: Sergey Shelukhin
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: HBASE-8770.branch-2.001.patch
>
>
> This came up during HBASE-8721. Puts with the same ts are resolved by seqNum. 
> It's not clear why deletes with the same ts as a put should always mask the 
> put, rather than also being resolve by seqNum.
> What do you think?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-8770) deletes and puts with the same ts should be resolved according to mvcc/seqNum

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-8770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-8770:
-
Attachment: HBASE-8770.branch-2.001.patch

> deletes and puts with the same ts should be resolved according to mvcc/seqNum
> -
>
> Key: HBASE-8770
> URL: https://issues.apache.org/jira/browse/HBASE-8770
> Project: HBase
>  Issue Type: Brainstorming
>Reporter: Sergey Shelukhin
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: HBASE-8770.branch-2.001.patch
>
>
> This came up during HBASE-8721. Puts with the same ts are resolved by seqNum. 
> It's not clear why deletes with the same ts as a put should always mask the 
> put, rather than also being resolve by seqNum.
> What do you think?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19437) Batch operation can't handle the null result for Append/Increment

[Chia-Ping Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-19437:
---
Release Note: The result from server is changed from null to 
Result.EMPTY_RESULT when Append/Increment operation can't retrieve any data 
from server,   (was: If Mutation#isReturnResults is false and 
RegionObserver#preAppendAfterRowLock doesn't return null, the result from 
server is changed from null to Result.EMPTY_RESULT)

> Batch operation can't handle the null result for Append/Increment
> -
>
> Key: HBASE-19437
> URL: https://issues.apache.org/jira/browse/HBASE-19437
> Project: HBase
>  Issue Type: Bug
>  Components: Usability
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-19437.v0.patch, HBASE-19437.v0.patch
>
>
> But the Table#append and #increment can handle the null result...that is an 
> inconsistent behavior for user.
> I have noticed two scenarios that server will return null result to user.
> # postAppend/postIncrement return null
> # mutation.isReturnResults() is false and 
> preIncrementAfterRowLock/preAppendAfterRowLock doesn't return null
> We should wrap the null to empty result on server side. CP user should throw 
> Exception rather than return null if they intend to say something is broken.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19437) Batch operation can't handle the null result for Append/Increment

[Chia-Ping Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-19437:
---
Hadoop Flags: Incompatible change
Release Note: If Mutation#isReturnResults is false and 
RegionObserver#preAppendAfterRowLock doesn't return null, the result from 
server is changed from null to Result.EMPTY_RESULT

> Batch operation can't handle the null result for Append/Increment
> -
>
> Key: HBASE-19437
> URL: https://issues.apache.org/jira/browse/HBASE-19437
> Project: HBase
>  Issue Type: Bug
>  Components: Usability
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-19437.v0.patch, HBASE-19437.v0.patch
>
>
> But the Table#append and #increment can handle the null result...that is an 
> inconsistent behavior for user.
> I have noticed two scenarios that server will return null result to user.
> # postAppend/postIncrement return null
> # mutation.isReturnResults() is false and 
> preIncrementAfterRowLock/preAppendAfterRowLock doesn't return null
> We should wrap the null to empty result on server side. CP user should throw 
> Exception rather than return null if they intend to say something is broken.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (HBASE-19814) Release hbase-2.0.0-beta-2; "rolling upgrade" release

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383278#comment-16383278
 ] 

stack edited comment on HBASE-19814 at 3/2/18 7:20 AM:
---

Build is failing here... with not a jdk9 in sight...
{code}
...
[ERROR] no module descriptor for org.apache.hbase:hbase
[ERROR] no module descriptor for org.apache.hbase:hbase-checkstyle
[ERROR] no module descriptor for org.apache.hbase:hbase-build-support
[ERROR] no module descriptor for org.apache.hbase:hbase-error-prone
[ERROR] no module descriptor for org.apache.hbase:hbase-annotations
[ERROR] no module descriptor for org.apache.hbase:hbase-build-configuration
[ERROR] no module descriptor for org.apache.hbase:hbase-protocol-shaded
[ERROR] no module descriptor for org.apache.hbase:hbase-common
[ERROR] no module descriptor for org.apache.hbase:hbase-metrics-api
[ERROR] no module descriptor for org.apache.hbase:hbase-hadoop-compat
[ERROR] no module descriptor for org.apache.hbase:hbase-metrics
[ERROR] no module descriptor for org.apache.hbase:hbase-hadoop2-compat
[ERROR] no module descriptor for org.apache.hbase:hbase-protocol
[ERROR] no module descriptor for org.apache.hbase:hbase-client
[ERROR] no module descriptor for org.apache.hbase:hbase-zookeeper
[ERROR] no module descriptor for org.apache.hbase:hbase-replication
[ERROR] no module descriptor for org.apache.hbase:hbase-resource-bundle
[ERROR] no module descriptor for org.apache.hbase:hbase-http
[ERROR] no module descriptor for org.apache.hbase:hbase-procedure
[ERROR] no module descriptor for org.apache.hbase:hbase-server
[ERROR] no module descriptor for org.apache.hbase:hbase-mapreduce
[ERROR] no module descriptor for org.apache.hbase:hbase-testing-util
[ERROR] no module descriptor for org.apache.hbase:hbase-thrift
[ERROR] no module descriptor for org.apache.hbase:hbase-rsgroup
[ERROR] no module descriptor for org.apache.hbase:hbase-shell
[ERROR] no module descriptor for org.apache.hbase:hbase-endpoint
[ERROR] no module descriptor for org.apache.hbase:hbase-it
[ERROR] no module descriptor for org.apache.hbase:hbase-examples
[ERROR] no module descriptor for org.apache.hbase:hbase-rest
[ERROR] no module descriptor for org.apache.hbase:hbase-external-blockcache
[ERROR] no module descriptor for org.apache.hbase:hbase-assembly
[ERROR] no module descriptor for org.apache.hbase:hbase-shaded
[ERROR] no module descriptor for org.apache.hbase:hbase-shaded-client
[ERROR] no module descriptor for org.apache.hbase:hbase-shaded-mapreduce
[ERROR] no module descriptor for org.apache.hbase:hbase-shaded-check-invariants
[ERROR] no module descriptor for org.apache.hbase:hbase-archetypes
[ERROR] no module descriptor for org.apache.hbase:hbase-client-project
[ERROR] no module descriptor for org.apache.hbase:hbase-shaded-client-project
[ERROR] no module descriptor for org.apache.hbase:hbase-archetype-builder
[INFO]
100 errors
100 warnings
...
{code}

Fun.

Running [INFO] <<< maven-javadoc-plugin:3.0.0:test-aggregate @ hbase <<<


was (Author: stack):
Build is failing here... with not a jdk9 in sight...
{code}
...
[ERROR] no module descriptor for org.apache.hbase:hbase
[ERROR] no module descriptor for org.apache.hbase:hbase-checkstyle
[ERROR] no module descriptor for org.apache.hbase:hbase-build-support
[ERROR] no module descriptor for org.apache.hbase:hbase-error-prone
[ERROR] no module descriptor for org.apache.hbase:hbase-annotations
[ERROR] no module descriptor for org.apache.hbase:hbase-build-configuration
[ERROR] no module descriptor for org.apache.hbase:hbase-protocol-shaded
[ERROR] no module descriptor for org.apache.hbase:hbase-common
[ERROR] no module descriptor for org.apache.hbase:hbase-metrics-api
[ERROR] no module descriptor for org.apache.hbase:hbase-hadoop-compat
[ERROR] no module descriptor for org.apache.hbase:hbase-metrics
[ERROR] no module descriptor for org.apache.hbase:hbase-hadoop2-compat
[ERROR] no module descriptor for org.apache.hbase:hbase-protocol
[ERROR] no module descriptor for org.apache.hbase:hbase-client
[ERROR] no module descriptor for org.apache.hbase:hbase-zookeeper
[ERROR] no module descriptor for org.apache.hbase:hbase-replication
[ERROR] no module descriptor for org.apache.hbase:hbase-resource-bundle
[ERROR] no module descriptor for org.apache.hbase:hbase-http
[ERROR] no module descriptor for org.apache.hbase:hbase-procedure
[ERROR] no module descriptor for org.apache.hbase:hbase-server
[ERROR] no module descriptor for org.apache.hbase:hbase-mapreduce
[ERROR] no module descriptor for org.apache.hbase:hbase-testing-util
[ERROR] no module descriptor for org.apache.hbase:hbase-thrift
[ERROR] no module descriptor for org.apache.hbase:hbase-rsgroup
[ERROR] no module descriptor for org.apache.hbase:hbase-shell
[ERROR] no module descriptor for org.apache.hbase:hbase-endpoint
[ERROR] no module descriptor for org.apache.hbase:hbase-it
[ERROR] no module descriptor for 

[jira] [Commented] (HBASE-19504) Add TimeRange support into checkAndMutate

[Chia-Ping Tsai (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383282#comment-16383282
 ] 

Chia-Ping Tsai commented on HBASE-19504:


Not yet. Let me take over it if no one have interest in it.

> Add TimeRange support into checkAndMutate
> -
>
> Key: HBASE-19504
> URL: https://issues.apache.org/jira/browse/HBASE-19504
> Project: HBase
>  Issue Type: New Feature
>Reporter: Chia-Ping Tsai
>Priority: Major
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HBASE-19504) Add TimeRange support into checkAndMutate

[Chia-Ping Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai reassigned HBASE-19504:
--

Assignee: Chia-Ping Tsai

> Add TimeRange support into checkAndMutate
> -
>
> Key: HBASE-19504
> URL: https://issues.apache.org/jira/browse/HBASE-19504
> Project: HBase
>  Issue Type: New Feature
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Major
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19814) Release hbase-2.0.0-beta-2; "rolling upgrade" release

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383278#comment-16383278
 ] 

stack commented on HBASE-19814:
---

Build is failing here... with not a jdk9 in sight...
{code}
...
[ERROR] no module descriptor for org.apache.hbase:hbase
[ERROR] no module descriptor for org.apache.hbase:hbase-checkstyle
[ERROR] no module descriptor for org.apache.hbase:hbase-build-support
[ERROR] no module descriptor for org.apache.hbase:hbase-error-prone
[ERROR] no module descriptor for org.apache.hbase:hbase-annotations
[ERROR] no module descriptor for org.apache.hbase:hbase-build-configuration
[ERROR] no module descriptor for org.apache.hbase:hbase-protocol-shaded
[ERROR] no module descriptor for org.apache.hbase:hbase-common
[ERROR] no module descriptor for org.apache.hbase:hbase-metrics-api
[ERROR] no module descriptor for org.apache.hbase:hbase-hadoop-compat
[ERROR] no module descriptor for org.apache.hbase:hbase-metrics
[ERROR] no module descriptor for org.apache.hbase:hbase-hadoop2-compat
[ERROR] no module descriptor for org.apache.hbase:hbase-protocol
[ERROR] no module descriptor for org.apache.hbase:hbase-client
[ERROR] no module descriptor for org.apache.hbase:hbase-zookeeper
[ERROR] no module descriptor for org.apache.hbase:hbase-replication
[ERROR] no module descriptor for org.apache.hbase:hbase-resource-bundle
[ERROR] no module descriptor for org.apache.hbase:hbase-http
[ERROR] no module descriptor for org.apache.hbase:hbase-procedure
[ERROR] no module descriptor for org.apache.hbase:hbase-server
[ERROR] no module descriptor for org.apache.hbase:hbase-mapreduce
[ERROR] no module descriptor for org.apache.hbase:hbase-testing-util
[ERROR] no module descriptor for org.apache.hbase:hbase-thrift
[ERROR] no module descriptor for org.apache.hbase:hbase-rsgroup
[ERROR] no module descriptor for org.apache.hbase:hbase-shell
[ERROR] no module descriptor for org.apache.hbase:hbase-endpoint
[ERROR] no module descriptor for org.apache.hbase:hbase-it
[ERROR] no module descriptor for org.apache.hbase:hbase-examples
[ERROR] no module descriptor for org.apache.hbase:hbase-rest
[ERROR] no module descriptor for org.apache.hbase:hbase-external-blockcache
[ERROR] no module descriptor for org.apache.hbase:hbase-assembly
[ERROR] no module descriptor for org.apache.hbase:hbase-shaded
[ERROR] no module descriptor for org.apache.hbase:hbase-shaded-client
[ERROR] no module descriptor for org.apache.hbase:hbase-shaded-mapreduce
[ERROR] no module descriptor for org.apache.hbase:hbase-shaded-check-invariants
[ERROR] no module descriptor for org.apache.hbase:hbase-archetypes
[ERROR] no module descriptor for org.apache.hbase:hbase-client-project
[ERROR] no module descriptor for org.apache.hbase:hbase-shaded-client-project
[ERROR] no module descriptor for org.apache.hbase:hbase-archetype-builder
[INFO]
100 errors
100 warnings
...
{code}

Fun.

> Release hbase-2.0.0-beta-2; "rolling upgrade" release
> -
>
> Key: HBASE-19814
> URL: https://issues.apache.org/jira/browse/HBASE-19814
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-8770) deletes and puts with the same ts should be resolved according to mvcc/seqNum

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-8770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383273#comment-16383273
 ] 

stack commented on HBASE-8770:
--

HBASE-15968 does this. You need to set NEW_VERSION_BEHAVIOR on the column 
family. If you are replicating, you need to use serial replication. Let me 
update the doc and then resolve this issue.

> deletes and puts with the same ts should be resolved according to mvcc/seqNum
> -
>
> Key: HBASE-8770
> URL: https://issues.apache.org/jira/browse/HBASE-8770
> Project: HBase
>  Issue Type: Brainstorming
>Reporter: Sergey Shelukhin
>Priority: Blocker
> Fix For: 2.0.0
>
>
> This came up during HBASE-8721. Puts with the same ts are resolved by seqNum. 
> It's not clear why deletes with the same ts as a put should always mask the 
> put, rather than also being resolve by seqNum.
> What do you think?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20095) Redesign single instance pool in CleanerChore

[Reid Chan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Reid Chan updated HBASE-20095:
--
Attachment: HBASE-20095.master.002.patch

> Redesign single instance pool in CleanerChore
> -
>
> Key: HBASE-20095
> URL: https://issues.apache.org/jira/browse/HBASE-20095
> Project: HBase
>  Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Critical
> Attachments: HBASE-20095.master.001.patch, 
> HBASE-20095.master.002.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20095) Redesign single instance pool in CleanerChore

[Reid Chan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Reid Chan updated HBASE-20095:
--
Attachment: (was: HBASE-20095.master.002.patch)

> Redesign single instance pool in CleanerChore
> -
>
> Key: HBASE-20095
> URL: https://issues.apache.org/jira/browse/HBASE-20095
> Project: HBase
>  Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Critical
> Attachments: HBASE-20095.master.001.patch, 
> HBASE-20095.master.002.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20095) Redesign single instance pool in CleanerChore

[Reid Chan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Reid Chan updated HBASE-20095:
--
Attachment: HBASE-20095.master.002.patch

> Redesign single instance pool in CleanerChore
> -
>
> Key: HBASE-20095
> URL: https://issues.apache.org/jira/browse/HBASE-20095
> Project: HBase
>  Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Critical
> Attachments: HBASE-20095.master.001.patch, 
> HBASE-20095.master.002.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20050) Reimplement updateReplicationPositions logic in serial replication based on the newly introduced replication storage layer

[Duo Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383254#comment-16383254
 ] 

Duo Zhang commented on HBASE-20050:
---

{code}
@param lastSeqIds map with {regionName, sequenceId} pairs for serial 
replication.
{code}

Should be encodedRegionName.

And on the directories, encodedRegionName is already a hash result, it does not 
make sense to hash it again... Anyway, can do it in another issue.

+1 on commit now. The comment can be fixed on committing.

Thanks.

> Reimplement updateReplicationPositions logic in serial replication based on 
> the newly introduced replication storage layer
> --
>
> Key: HBASE-20050
> URL: https://issues.apache.org/jira/browse/HBASE-20050
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-20050.patch, HBASE-20050.v1.patch, 
> HBASE-20050.v2.patch, HBASE-20050.v3.patch
>
>
> Now, in serial replication,  need to update  the replication position both in 
> zookeeper and meta table, which may led to inconsistent problem. I'll 
> re-implement this part based on the uniform replication storage layer. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20050) Reimplement updateReplicationPositions logic in serial replication based on the newly introduced replication storage layer

[Zheng Hu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383248#comment-16383248
 ] 

Zheng Hu commented on HBASE-20050:
--

[~Apache9] Any other concerns for patch.v3 ?  The failed UT is unrelated 
(Timeout) and passed under my local .

> Reimplement updateReplicationPositions logic in serial replication based on 
> the newly introduced replication storage layer
> --
>
> Key: HBASE-20050
> URL: https://issues.apache.org/jira/browse/HBASE-20050
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-20050.patch, HBASE-20050.v1.patch, 
> HBASE-20050.v2.patch, HBASE-20050.v3.patch
>
>
> Now, in serial replication,  need to update  the replication position both in 
> zookeeper and meta table, which may led to inconsistent problem. I'll 
> re-implement this part based on the uniform replication storage layer. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19943) Only allow removing sync replication peer which is in DA state

[Duo Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-19943:
--
Attachment: HBASE-19943.HBASE-19064.002.patch

> Only allow removing sync replication peer which is in DA state
> --
>
> Key: HBASE-19943
> URL: https://issues.apache.org/jira/browse/HBASE-19943
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-19943.HBASE-19064.001.patch, 
> HBASE-19943.HBASE-19064.002.patch, HBASE-19943.HBASE-19064.002.patch
>
>
> To simplify the logic of RemovePeerProcedure. Otherwise we may also need to 
> reopen regions which makes the RemovePeerProcedure can not fit for both sync 
> and normal replication peer.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (HBASE-20095) Redesign single instance pool in CleanerChore

[Reid Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383243#comment-16383243
 ] 

Reid Chan edited comment on HBASE-20095 at 3/2/18 6:36 AM:
---

Only one updates the static pool.

Because all registered {{ConfigurationObserver}} are called sequentially, let's 
say {{LogCleaner}} is called first, then size of pool {{poolSize}} is updated; 
following {{HFileCleaner}} which should see the latest {{updatedSize}} and 
should not reconfigure the pool and just return. The snippet,
{code:java}
public void onConfigurationChange(Configuration conf) {
int updatedSize = calculatePoolSize(conf.get(CHORE_POOL_SIZE, 
DEFAULT_CHORE_POOL_SIZE));
if (updatedSize == poolSize) {
  LOG.trace("Size from configuration is same as previous={}, no need to 
update.", updatedSize);
  return;
}
poolSize = updatedSize;
...
}
{code}


was (Author: reidchan):
Only one updates the static pool. Because {{ConfigurationObserver}}s are called 
sequentially, let's say {{LogCleaner}} is called first, then size of pool 
{{updatedSize}} is updated, following {{HFileCleaner}} which should see the 
latest {{updatedSize}} and should not reconfigure the pool and just return. The 
snippet,
{code}
public void onConfigurationChange(Configuration conf) {
int updatedSize = calculatePoolSize(conf.get(CHORE_POOL_SIZE, 
DEFAULT_CHORE_POOL_SIZE));
if (updatedSize == poolSize) {
  LOG.trace("Size from configuration is same as previous={}, no need to 
update.", updatedSize);
  return;
}
poolSize = updatedSize;
...
}
{code}

> Redesign single instance pool in CleanerChore
> -
>
> Key: HBASE-20095
> URL: https://issues.apache.org/jira/browse/HBASE-20095
> Project: HBase
>  Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Critical
> Attachments: HBASE-20095.master.001.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (HBASE-20095) Redesign single instance pool in CleanerChore

[Reid Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383243#comment-16383243
 ] 

Reid Chan edited comment on HBASE-20095 at 3/2/18 6:34 AM:
---

Only one updates the static pool. Because {{ConfigurationObserver}}s are called 
sequentially, let's say {{LogCleaner}} is called first, then size of pool 
{{updatedSize}} is updated, following {{HFileCleaner}} which should see the 
latest {{updatedSize}} and should not reconfigure the pool and just return. The 
snippet,
{code}
public void onConfigurationChange(Configuration conf) {
int updatedSize = calculatePoolSize(conf.get(CHORE_POOL_SIZE, 
DEFAULT_CHORE_POOL_SIZE));
if (updatedSize == poolSize) {
  LOG.trace("Size from configuration is same as previous={}, no need to 
update.", updatedSize);
  return;
}
poolSize = updatedSize;
...
}
{code}


was (Author: reidchan):
Only one updates the static pool. Because {{ConfigurationObserver}} is called 
sequentially, let's say {{LogCleaner}} is called first, then size of pool 
{{updatedSize}} is updated, following {{HFileCleaner}} which should see the 
latest {{updatedSize}} and should not reconfigure the pool and just return. The 
snippet,
{code}
public void onConfigurationChange(Configuration conf) {
int updatedSize = calculatePoolSize(conf.get(CHORE_POOL_SIZE, 
DEFAULT_CHORE_POOL_SIZE));
if (updatedSize == poolSize) {
  LOG.trace("Size from configuration is same as previous={}, no need to 
update.", updatedSize);
  return;
}
poolSize = updatedSize;
...
}
{code}

> Redesign single instance pool in CleanerChore
> -
>
> Key: HBASE-20095
> URL: https://issues.apache.org/jira/browse/HBASE-20095
> Project: HBase
>  Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Critical
> Attachments: HBASE-20095.master.001.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20095) Redesign single instance pool in CleanerChore

[Reid Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383243#comment-16383243
 ] 

Reid Chan commented on HBASE-20095:
---

Only one updates the static pool. Because {{ConfigurationObserver}} is called 
sequentially, let's say {{LogCleaner}} is called first, then size of pool 
{{updatedSize}} is updated, following {{HFileCleaner}} which should see the 
latest {{updatedSize}} and should not reconfigure the pool and just return. The 
snippet,
{code}
public void onConfigurationChange(Configuration conf) {
int updatedSize = calculatePoolSize(conf.get(CHORE_POOL_SIZE, 
DEFAULT_CHORE_POOL_SIZE));
if (updatedSize == poolSize) {
  LOG.trace("Size from configuration is same as previous={}, no need to 
update.", updatedSize);
  return;
}
poolSize = updatedSize;
...
}
{code}

> Redesign single instance pool in CleanerChore
> -
>
> Key: HBASE-20095
> URL: https://issues.apache.org/jira/browse/HBASE-20095
> Project: HBase
>  Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Critical
> Attachments: HBASE-20095.master.001.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-14439) New/Improved Filesystem Abstractions

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-14439:
--
Priority: Critical  (was: Major)

> New/Improved Filesystem Abstractions
> 
>
> Key: HBASE-14439
> URL: https://issues.apache.org/jira/browse/HBASE-14439
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ben Lau
>Priority: Critical
> Attachments: abstraction.patch
>
>
> Ticket for work in progress on new FileSystem abstractions.  Previously, we 
> (Yahoo) submitted a ticket that would add support for humongous (1 million 
> region+) tables via a hierarchical layout (HBASE-13991).  However open source 
> is moving in a similar but not identical direction in the future and so the 
> patch will not be merged into open source.
> We will be working on a different patch now with folks from open source.  It 
> will create/add to 2 layers-- a path abstraction layer and a use-oriented 
> abstraction layer.  The path abstraction layer is epitomized by classes like 
> FsUtils (and in the patch new classes like AFsLayout).  The use oriented 
> abstraction layer is epitomized by existing classes like 
> MasterFileSystem/HRegionFileSystem (and possibly new classes later) that 
> build on the path abstraction layer and focus on 'doing things' (eg creating 
> regions) and less on the gritty details like the paths.
> This work on abstracting and isolating the paths from the use cases will help 
> Yahoo not diverge too much from open source with its internal 'Humongous' 
> table hierarchical layout, while also helping open source move further 
> towards the eventual goal of redoing the FS layout in a similar (but 
> different) hierarchical layout later that focuses on data directory 
> uniformity (unlike the humongous patch) and storing hierarchy in the meta 
> table instead which enables new optimizations (see HBASE-14090.)
> Attached to this ticket is some work we've done at Yahoo so far that will be 
> put into an open source HBase branch for further collaboration.  The patch is 
> not meant to be complete yet and is a work in progress.  (Please wait on 
> patch comments/reviews.)  It also includes some Yahoo-specific 'humongous' 
> layout code that will be removed before submission in open source.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20114) Fix IllegalFormatConversionException in rsgroup.jsp

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383235#comment-16383235
 ] 

Hadoop QA commented on HBASE-20114:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
41s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
39s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
27s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}130m 
33s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
21s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}139m 47s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:9f2f2db |
| JIRA Issue | HBASE-20114 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12912713/HBASE-20114.branch-2.001.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  |
| uname | Linux fa0520b90e34 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4 
19:09:19 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | branch-2 / a2de29560f |
| maven | version: Apache Maven 3.5.2 
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11766/testReport/ |
| Max. process+thread count | 4156 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11766/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> Fix IllegalFormatConversionException in rsgroup.jsp
> ---
>
> Key: HBASE-20114
> URL: https://issues.apache.org/jira/browse/HBASE-20114
> Project: HBase
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 3.0.0, 2.0.0-beta-2
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Attachments: HBASE-20114.branch-2.001.patch
>
>
> IllegalFormatConversionException at
> {code:java}
> String.format("%.0f", sl.getRequestCountPerSecond()) 
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19943) Only allow removing sync replication peer which is in DA state

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383231#comment-16383231
 ] 

Hadoop QA commented on HBASE-19943:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} HBASE-19064 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
12s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
39s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
54s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
12s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  1m 
47s{color} | {color:red} hbase-server in HBASE-19064 has 24 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
26s{color} | {color:green} HBASE-19064 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
10s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red}  6m  
8s{color} | {color:red} The patch causes 10 errors with Hadoop v2.6.5. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red}  8m  
5s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m 
11s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color} 
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m  
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}106m 28s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
22s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}137m 50s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestRegionOpen |
|   | hadoop.hbase.client.TestAsyncTableBatch |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19943 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12912716/HBASE-19943.HBASE-19064.002.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 13569171a46b 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | HBASE-19064 / 0ac7692461 |
| maven | version: Apache Maven 3.5.2 
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 

[jira] [Commented] (HBASE-20095) Redesign single instance pool in CleanerChore

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383226#comment-16383226
 ] 

stack commented on HBASE-20095:
---

[~reidchan] Hard part is what happens on config. change. All instances then 
update the static pool with the new config. Is this a prob? Could be how many 
instances of ChoreCleaner? What happens if they all redo the pool sizing? 
Thanks.

> Redesign single instance pool in CleanerChore
> -
>
> Key: HBASE-20095
> URL: https://issues.apache.org/jira/browse/HBASE-20095
> Project: HBase
>  Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Critical
> Attachments: HBASE-20095.master.001.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19437) Batch operation can't handle the null result for Append/Increment

[Anoop Sam John (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383224#comment-16383224
 ] 

Anoop Sam John commented on HBASE-19437:


Ya pls a latest patch and change as Release notes. Then it should be good.

> Batch operation can't handle the null result for Append/Increment
> -
>
> Key: HBASE-19437
> URL: https://issues.apache.org/jira/browse/HBASE-19437
> Project: HBase
>  Issue Type: Bug
>  Components: Usability
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-19437.v0.patch, HBASE-19437.v0.patch
>
>
> But the Table#append and #increment can handle the null result...that is an 
> inconsistent behavior for user.
> I have noticed two scenarios that server will return null result to user.
> # postAppend/postIncrement return null
> # mutation.isReturnResults() is false and 
> preIncrementAfterRowLock/preAppendAfterRowLock doesn't return null
> We should wrap the null to empty result on server side. CP user should throw 
> Exception rather than return null if they intend to say something is broken.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20050) Reimplement updateReplicationPositions logic in serial replication based on the newly introduced replication storage layer

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383220#comment-16383220
 ] 

Hadoop QA commented on HBASE-20050:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
46s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
59s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
40s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
28s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
23m 41s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
39s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
20s{color} | {color:green} hbase-replication in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}131m 50s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
55s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}183m 19s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.TestDLSAsyncFSWAL |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-20050 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12912710/HBASE-20050.v3.patch |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 7c1a11425133 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19 
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / c415ddb38c |
| maven | version: Apache Maven 3.5.2 

[jira] [Commented] (HBASE-20090) Properly handle Preconditions check failure in MemStoreFlusher$FlushHandler.run

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383215#comment-16383215
 ] 

stack commented on HBASE-20090:
---

Why you posting a patch when you've not said what the problem 
is nor how the patch supposedly fixes this problem 
[~yuzhih...@gmail.com]

> Properly handle Preconditions check failure in 
> MemStoreFlusher$FlushHandler.run
> ---
>
> Key: HBASE-20090
> URL: https://issues.apache.org/jira/browse/HBASE-20090
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20094.v01.patch
>
>
> Here is the code in branch-2 :
> {code}
> try {
>   wakeupPending.set(false); // allow someone to wake us up again
>   fqe = flushQueue.poll(threadWakeFrequency, TimeUnit.MILLISECONDS);
>   if (fqe == null || fqe instanceof WakeupFlushThread) {
> ...
>   if (!flushOneForGlobalPressure()) {
> ...
>   FlushRegionEntry fre = (FlushRegionEntry) fqe;
>   if (!flushRegion(fre)) {
> break;
> ...
> } catch (Exception ex) {
>   LOG.error("Cache flusher failed for entry " + fqe, ex);
>   if (!server.checkFileSystem()) {
> break;
>   }
> }
> {code}
> Inside flushOneForGlobalPressure():
> {code}
>   Preconditions.checkState(
> (regionToFlush != null && regionToFlushSize > 0) ||
> (bestRegionReplica != null && bestRegionReplicaSize > 0));
> {code}
> When the Preconditions check fails, IllegalStateException is caught by the 
> catch block shown above.
> However, the fqe is not flushed, resulting in potential data loss.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20090) Properly handle Preconditions check failure in MemStoreFlusher$FlushHandler.run

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383213#comment-16383213
 ] 

stack commented on HBASE-20090:
---

Patch is for the wrong issue?

> Properly handle Preconditions check failure in 
> MemStoreFlusher$FlushHandler.run
> ---
>
> Key: HBASE-20090
> URL: https://issues.apache.org/jira/browse/HBASE-20090
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20094.v01.patch
>
>
> Here is the code in branch-2 :
> {code}
> try {
>   wakeupPending.set(false); // allow someone to wake us up again
>   fqe = flushQueue.poll(threadWakeFrequency, TimeUnit.MILLISECONDS);
>   if (fqe == null || fqe instanceof WakeupFlushThread) {
> ...
>   if (!flushOneForGlobalPressure()) {
> ...
>   FlushRegionEntry fre = (FlushRegionEntry) fqe;
>   if (!flushRegion(fre)) {
> break;
> ...
> } catch (Exception ex) {
>   LOG.error("Cache flusher failed for entry " + fqe, ex);
>   if (!server.checkFileSystem()) {
> break;
>   }
> }
> {code}
> Inside flushOneForGlobalPressure():
> {code}
>   Preconditions.checkState(
> (regionToFlush != null && regionToFlushSize > 0) ||
> (bestRegionReplica != null && bestRegionReplicaSize > 0));
> {code}
> When the Preconditions check fails, IllegalStateException is caught by the 
> catch block shown above.
> However, the fqe is not flushed, resulting in potential data loss.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-17796) Minor improvements on the javadoc and naming for rpc client implementation

[Yu Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383199#comment-16383199
 ] 

Yu Li commented on HBASE-17796:
---

Sounds good to me to keep it as is for now, let's refine the javadoc and naming 
later.

> Minor improvements on the javadoc and naming for rpc client implementation
> --
>
> Key: HBASE-17796
> URL: https://issues.apache.org/jira/browse/HBASE-17796
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0, 1.4.0
>Reporter: Duo Zhang
>Priority: Major
>
> Address the comments here
> https://reviews.apache.org/r/57459/



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19639) ITBLL can't go big because RegionTooBusyException... Above memstore limit

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19639:
--
Priority: Blocker  (was: Major)

> ITBLL can't go big because RegionTooBusyException... Above memstore limit
> -
>
> Key: HBASE-19639
> URL: https://issues.apache.org/jira/browse/HBASE-19639
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0
>
>
> Running ITBLLs, the basic link generator keeps failing because I run into 
> exceptions like below:
> {code}
> 2017-12-26 19:23:45,284 INFO [main] 
> org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator: 
> Persisting current.length=100, count=100, id=Job: 
> job_1513025868268_0062 Task: attempt_1513025868268_0062_m_06_2, 
> current=\x8B\xDB25\xA7*\x9A\xF5\xDEx\x83\xDF\xDC?\x94\x92, i=100
> 2017-12-26 19:24:18,982 INFO [htable-pool3-t6] 
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl: #2, 
> table=IntegrationTestBigLinkedList, attempt=10/11 failed=524ops, last 
> exception: org.apache.hadoop.hbase.RegionTooBusyException: 
> org.apache.hadoop.hbase.RegionTooBusyException: Above memstore limit, 
> regionName=IntegrationTestBigLinkedList,q\xC7\x1Cq\xC7\x1Cq\xC0,1514342757438.71ef1fbab1576588955f45796e95c08b.,
>  server=ve0538.halxg.cloudera.com,16020,1514343549993, 
> memstoreSize=538084641, blockingMemStoreSize=536870912
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4178)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3799)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3739)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:975)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:894)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2587)
>   at 
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
>   at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
>   at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
>   at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
>   at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
>  on ve0538.halxg.cloudera.com,16020,1514343549993, tracking started null, 
> retrying after=10050ms, replay=524ops
> 2017-12-26 19:24:29,061 INFO [htable-pool3-t6] 
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl: #2, 
> table=IntegrationTestBigLinkedList, attempt=11/11 failed=524ops, last 
> exception: org.apache.hadoop.hbase.RegionTooBusyException: 
> org.apache.hadoop.hbase.RegionTooBusyException: Above memstore limit, 
> regionName=IntegrationTestBigLinkedList,q\xC7\x1Cq\xC7\x1Cq\xC0,1514342757438.71ef1fbab1576588955f45796e95c08b.,
>  server=ve0538.halxg.cloudera.com,16020,1514343549993, 
> memstoreSize=538084641, blockingMemStoreSize=536870912
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4178)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3799)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3739)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:975)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:894)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2587)
>   at 
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
>   at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
>   at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
>   at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
>   at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
>  on ve0538.halxg.cloudera.com,16020,1514343549993, tracking started null, 
> retrying after=10033ms, replay=524ops
> 2017-12-26 19:24:37,183 INFO [ReadOnlyZKClient] 
> org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient: 0x015051a0 no activities 
> for 6 ms, close active connection. Will reconnect next time when there 
> are new requests.
> 2017-12-26 19:24:39,122 WARN [htable-pool3-t6] 
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl: #2, 
> table=IntegrationTestBigLinkedList, attempt=12/11 failed=524ops, last 
> exception: org.apache.hadoop.hbase.RegionTooBusyException: 
> 

[jira] [Updated] (HBASE-20111) Able to split region explicitly even on shouldSplit return false from split policy

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-20111:
--
Fix Version/s: (was: 2.0.0-beta-2)
   2.0.0

> Able to split region explicitly even on shouldSplit return false from split 
> policy
> --
>
> Key: HBASE-20111
> URL: https://issues.apache.org/jira/browse/HBASE-20111
> Project: HBase
>  Issue Type: Bug
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
>Priority: Critical
> Fix For: 2.0.0
>
>
> Currently able to split the region explicitly even when the split policy 
> returns from shouldSplit.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19639) ITBLL can't go big because RegionTooBusyException... Above memstore limit

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19639:
--
Fix Version/s: (was: 2.0.0-beta-2)
   2.0.0

> ITBLL can't go big because RegionTooBusyException... Above memstore limit
> -
>
> Key: HBASE-19639
> URL: https://issues.apache.org/jira/browse/HBASE-19639
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0
>
>
> Running ITBLLs, the basic link generator keeps failing because I run into 
> exceptions like below:
> {code}
> 2017-12-26 19:23:45,284 INFO [main] 
> org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator: 
> Persisting current.length=100, count=100, id=Job: 
> job_1513025868268_0062 Task: attempt_1513025868268_0062_m_06_2, 
> current=\x8B\xDB25\xA7*\x9A\xF5\xDEx\x83\xDF\xDC?\x94\x92, i=100
> 2017-12-26 19:24:18,982 INFO [htable-pool3-t6] 
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl: #2, 
> table=IntegrationTestBigLinkedList, attempt=10/11 failed=524ops, last 
> exception: org.apache.hadoop.hbase.RegionTooBusyException: 
> org.apache.hadoop.hbase.RegionTooBusyException: Above memstore limit, 
> regionName=IntegrationTestBigLinkedList,q\xC7\x1Cq\xC7\x1Cq\xC0,1514342757438.71ef1fbab1576588955f45796e95c08b.,
>  server=ve0538.halxg.cloudera.com,16020,1514343549993, 
> memstoreSize=538084641, blockingMemStoreSize=536870912
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4178)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3799)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3739)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:975)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:894)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2587)
>   at 
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
>   at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
>   at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
>   at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
>   at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
>  on ve0538.halxg.cloudera.com,16020,1514343549993, tracking started null, 
> retrying after=10050ms, replay=524ops
> 2017-12-26 19:24:29,061 INFO [htable-pool3-t6] 
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl: #2, 
> table=IntegrationTestBigLinkedList, attempt=11/11 failed=524ops, last 
> exception: org.apache.hadoop.hbase.RegionTooBusyException: 
> org.apache.hadoop.hbase.RegionTooBusyException: Above memstore limit, 
> regionName=IntegrationTestBigLinkedList,q\xC7\x1Cq\xC7\x1Cq\xC0,1514342757438.71ef1fbab1576588955f45796e95c08b.,
>  server=ve0538.halxg.cloudera.com,16020,1514343549993, 
> memstoreSize=538084641, blockingMemStoreSize=536870912
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4178)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3799)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3739)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:975)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:894)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2587)
>   at 
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
>   at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
>   at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
>   at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
>   at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
>  on ve0538.halxg.cloudera.com,16020,1514343549993, tracking started null, 
> retrying after=10033ms, replay=524ops
> 2017-12-26 19:24:37,183 INFO [ReadOnlyZKClient] 
> org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient: 0x015051a0 no activities 
> for 6 ms, close active connection. Will reconnect next time when there 
> are new requests.
> 2017-12-26 19:24:39,122 WARN [htable-pool3-t6] 
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl: #2, 
> table=IntegrationTestBigLinkedList, attempt=12/11 failed=524ops, last 
> exception: 

[jira] [Updated] (HBASE-19997) [rolling upgrade] 1.x => 2.x

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19997:
--
Priority: Blocker  (was: Major)

> [rolling upgrade] 1.x => 2.x
> 
>
> Key: HBASE-19997
> URL: https://issues.apache.org/jira/browse/HBASE-19997
> Project: HBase
>  Issue Type: Umbrella
>Reporter: stack
>Priority: Blocker
> Fix For: 2.0.0
>
>
> An umbrella issue of issues needed so folks can do a rolling upgrade from 
> hbase-1.x to hbase-2.x.
> (Recent) Notables:
>  * hbase-1.x can't read hbase-2.x WALs -- hbase-1.x doesn't know the 
> AsyncProtobufLogWriter class used writing the WAL -- see 
> https://issues.apache.org/jira/browse/HBASE-19166?focusedCommentId=16362897=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16362897
>  for exception.
>  ** Might be ok... means WAL split fails on an hbase1 RS... must wait till an 
> hbase-2.x RS picks up the WAL for it to be split.
>  * hbase-1 can't open regions from tables created by hbase-2; it can't find 
> the Table descriptor. See 
> https://issues.apache.org/jira/browse/HBASE-19116?focusedCommentId=16363276=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16363276
>  ** This might be ok if the tables we are doing rolling upgrade over were 
> written with hbase-1.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19997) [rolling upgrade] 1.x => 2.x

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19997:
--
Fix Version/s: (was: 2.0.0-beta-2)
   2.0.0

> [rolling upgrade] 1.x => 2.x
> 
>
> Key: HBASE-19997
> URL: https://issues.apache.org/jira/browse/HBASE-19997
> Project: HBase
>  Issue Type: Umbrella
>Reporter: stack
>Priority: Blocker
> Fix For: 2.0.0
>
>
> An umbrella issue of issues needed so folks can do a rolling upgrade from 
> hbase-1.x to hbase-2.x.
> (Recent) Notables:
>  * hbase-1.x can't read hbase-2.x WALs -- hbase-1.x doesn't know the 
> AsyncProtobufLogWriter class used writing the WAL -- see 
> https://issues.apache.org/jira/browse/HBASE-19166?focusedCommentId=16362897=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16362897
>  for exception.
>  ** Might be ok... means WAL split fails on an hbase1 RS... must wait till an 
> hbase-2.x RS picks up the WAL for it to be split.
>  * hbase-1 can't open regions from tables created by hbase-2; it can't find 
> the Table descriptor. See 
> https://issues.apache.org/jira/browse/HBASE-19116?focusedCommentId=16363276=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16363276
>  ** This might be ok if the tables we are doing rolling upgrade over were 
> written with hbase-1.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-18882) [TEST] Run MR branch-1 jobs against hbase2 cluster

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-18882:
--
Fix Version/s: (was: 2.0.0-beta-2)
   2.0.0

> [TEST] Run MR branch-1 jobs against hbase2 cluster
> --
>
> Key: HBASE-18882
> URL: https://issues.apache.org/jira/browse/HBASE-18882
> Project: HBase
>  Issue Type: Task
>  Components: mapreduce, test
>Reporter: stack
>Assignee: Josh Elser
>Priority: Major
> Fix For: 2.0.0
>
>
> Ensure this works still. Run all our bundled MR tools at least. Find some 
> custom if time.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20111) Able to split region explicitly even on shouldSplit return false from split policy

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383154#comment-16383154
 ] 

stack commented on HBASE-20111:
---

Tell us more. Give us an example. You have a patch [~rajeshbabu] Thank you.

> Able to split region explicitly even on shouldSplit return false from split 
> policy
> --
>
> Key: HBASE-20111
> URL: https://issues.apache.org/jira/browse/HBASE-20111
> Project: HBase
>  Issue Type: Bug
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> Currently able to split the region explicitly even when the split policy 
> returns from shouldSplit.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-17449) Add explicit document on different timeout settings

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-17449:
--
Fix Version/s: (was: 2.0.0-beta-2)
   2.0.0

> Add explicit document on different timeout settings
> ---
>
> Key: HBASE-17449
> URL: https://issues.apache.org/jira/browse/HBASE-17449
> Project: HBase
>  Issue Type: Improvement
>  Components: documentation
>Reporter: Yu Li
>Assignee: Yu Li
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 0001-Standalone.patch, WIP-HBASE-17499.master.001.patch
>
>
> Currently we have more than one timeout settings, mainly includes:
> * hbase.rpc.timeout
> * hbase.client.operation.timeout
> * hbase.client.scanner.timeout.period
> And in latest branch-1 or master branch code, we will have two other 
> properties:
> * hbase.rpc.read.timeout
> * hbase.rpc.write.timeout
> However, in current refguid we don't have explicit instruction on the 
> difference of these timeout settings (there're explanations for each 
> property, but no instruction on when to use which)
> In my understanding, for RPC layer timeout, or say each rpc call:
> * Scan (openScanner/next): controlled by hbase.client.scanner.timeout.period
> * Other operations:
>1. For released versions: controlled by hbase.rpc.timeout
>2. For 1.4+ versions: read operation controlled by hbase.rpc.read.timeout, 
> write operation controlled by hbase.rpc.write.timeout, or hbase.rpc.timeout 
> if the previous two are not set.
> And hbase.client.operation.timeout is a higher-level control counting retry 
> in, or say the overall control for one user call.
> After this JIRA, I hope when users ask questions like "What settings I should 
> use if I don't want to wait for more than 1 second for a single 
> put/get/scan.next call", we could give a neat answer.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-17730) [DOC] Migration to 2.0 for coprocessors

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-17730:
--
Fix Version/s: (was: 2.0.0-beta-2)
   2.0.0

> [DOC] Migration to 2.0 for coprocessors 
> 
>
> Key: HBASE-17730
> URL: https://issues.apache.org/jira/browse/HBASE-17730
> Project: HBase
>  Issue Type: Sub-task
>  Components: documentation, migration
>Reporter: Appy
>Priority: Blocker
> Fix For: 2.0.0
>
>
> Jiras breaking coprocessor compatibility should be marked with component ' 
> Coprocessor', and label 'incompatible'.
> Close to releasing 2.0, we should go through all such jiras and write down 
> steps for migrating coprocessor easily.
> The idea is, it might be very hard to fix coprocessor breakages by reverse 
> engineering errors,  but will be easier we suggest easiest way to fix 
> breakages resulting from each individual incompatible change.
> For eg. HBASE-17312 is incompatible change. It'll result in 100s of errors 
> because BaseXXXObserver classes are gone and will probably result in a lot of 
> confusion, but if we explicitly mention the fix which is just one line change 
> - replace "Foo extends BaseXXXObserver" with "Foo implements XXXObserver" - 
> it makes it very easy.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-18622) Mitigate API compatibility concerns between branch-1 and branch-2

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-18622:
--
Fix Version/s: (was: 2.0.0-beta-2)
   2.0.0

> Mitigate API compatibility concerns between branch-1 and branch-2
> -
>
> Key: HBASE-18622
> URL: https://issues.apache.org/jira/browse/HBASE-18622
> Project: HBase
>  Issue Type: Bug
>  Components: API
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: report.1.2_2.0.html.gz
>
>
> This project is to do what [~apurtell] did in the issue "HBASE-18431 Mitigate 
> compatibility concerns between branch-1.3 and branch-1.4" only do it between 
> branch-1 and branch-2.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-18758) [TEST][compat 1-2] Test delegation tokens continue to work when hbase1 going against hbase2 cluster

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-18758:
--
Fix Version/s: (was: 2.0.0-beta-2)
   2.0.0

> [TEST][compat 1-2] Test delegation tokens continue to work when hbase1 going 
> against hbase2 cluster
> ---
>
> Key: HBASE-18758
> URL: https://issues.apache.org/jira/browse/HBASE-18758
> Project: HBase
>  Issue Type: Sub-task
>  Components: API
>Reporter: stack
>Priority: Critical
> Fix For: 2.0.0
>
>
> Suggested by [~toffer] Need to test.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20108) `hbase zkcli` falls into a non-interactive prompt after HBASE-15199

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-20108:
--
Fix Version/s: (was: 2.0.0-beta-2)
   2.0.0

> `hbase zkcli` falls into a non-interactive prompt after HBASE-15199
> ---
>
> Key: HBASE-20108
> URL: https://issues.apache.org/jira/browse/HBASE-20108
> Project: HBase
>  Issue Type: Bug
>  Components: Usability
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20108.001.branch-2.patch, 
> HBASE-20108.002.branch-2.patch, HBASE-20108.003.branch-2.patch
>
>
> HBASE-15199 pulls the jruby-complete jar out of the normal classpath for 
> commands run in HBase. Jruby-complete bundles jline inside. ZK uses jline for 
> its nice shell-like usage.
> The problem is that this uncovered a bug where we're not explicitly bundling 
> a version of jline to make sure that {{hbase zkcli}} actually works. As long 
> as we're expecting {{zkcli}} to be there, we should provide jline on the 
> classpath to make sure the users get a real cli.
> Thanks to [~sergey.soldatov] for getting to the bottom of it quickly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-13428) [DOC] Migration to hbase-2.0.0

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-13428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-13428:
--
Fix Version/s: (was: 2.0.0-beta-2)
   2.0.0

> [DOC] Migration to hbase-2.0.0
> --
>
> Key: HBASE-13428
> URL: https://issues.apache.org/jira/browse/HBASE-13428
> Project: HBase
>  Issue Type: Umbrella
>  Components: migration
>Reporter: stack
>Priority: Blocker
> Fix For: 2.0.0
>
>
> Opening a 2.0 umbrella migration issue. Lets hang off this one any tools and 
> expectations migrating from 1.0 (or earlier) to 2.0. So far there are none 
> that I know of though there is an expectation in HBASE-13373 that hfiles are 
> at least major version 2 and minor version 3.  Lets list all such 
> expectations, etc., here.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19093) Check Admin/Table to ensure all operations go via AccessControl

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19093:
--
Fix Version/s: (was: 2.0.0-beta-2)
   2.0.0

> Check Admin/Table to ensure all operations go via AccessControl
> ---
>
> Key: HBASE-19093
> URL: https://issues.apache.org/jira/browse/HBASE-19093
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Assignee: Balazs Meszaros
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: HBASE-19093.master.001.patch, 
> HBASE-19093.master.002.patch, RegionObserver.txt
>
>
> A cursory review of Admin Interface has a bunch of methods as open, with out 
> AccessControl checks. For example, procedure executor has not check on it.
> This issue is about given the Admin and Table Interfaces a once-over to see 
> what is missing and to fill in access control where missing.
> This is a follow-on from work over in HBASE-19048



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19093) Check Admin/Table to ensure all operations go via AccessControl

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383152#comment-16383152
 ] 

stack commented on HBASE-19093:
---

[~appy] what else is to be done in here?

> Check Admin/Table to ensure all operations go via AccessControl
> ---
>
> Key: HBASE-19093
> URL: https://issues.apache.org/jira/browse/HBASE-19093
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Assignee: Balazs Meszaros
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: HBASE-19093.master.001.patch, 
> HBASE-19093.master.002.patch, RegionObserver.txt
>
>
> A cursory review of Admin Interface has a bunch of methods as open, with out 
> AccessControl checks. For example, procedure executor has not check on it.
> This issue is about given the Admin and Table Interfaces a once-over to see 
> what is missing and to fill in access control where missing.
> This is a follow-on from work over in HBASE-19048



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-18562) [AMv2] expireServers and ServerCrashProcedure cleanup

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-18562:
--
Fix Version/s: (was: 2.0.0)

> [AMv2] expireServers and ServerCrashProcedure cleanup
> -
>
> Key: HBASE-18562
> URL: https://issues.apache.org/jira/browse/HBASE-18562
> Project: HBase
>  Issue Type: Bug
>  Components: Region Assignment
>Reporter: stack
>Priority: Critical
>
> In review of HBASE-18551, [~uagashe] posed a scenario that revealed a hole in 
> our processing of unassigns; there is case where a UP might not get 
> notification from ServerCrashProcedure if the UP is scheduled AFTER a SCP has 
> gotten past its handleRIT call (No new SCP will be queued because 
> expireServer won't let it happen if crashed server is in dead server list 
> which it will be).
> Chatting on it, expireServers is doing checks that belong inside 
> ServerCrashProcedure. expireServers scheduling an SCP each time it is called 
> would make it so SCP processing is serialized one behind the other. If the 
> first does the clean up all subsequent will do no work but Procedures 
> dependent on them will get their wakeup call.
> This issue is about implementing the above cleanup.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-17214) Add metrics on responseQueue length and size

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-17214:
--
Component/s: Operability

> Add metrics on responseQueue length and size
> 
>
> Key: HBASE-17214
> URL: https://issues.apache.org/jira/browse/HBASE-17214
> Project: HBase
>  Issue Type: Improvement
>  Components: Operability
>Reporter: Yu Li
>Assignee: Yu Li
>Priority: Critical
>
> Currently in RpcServer we will use a response queue if multiple response to 
> write to the same connection, but there's no view of whether it's used and 
> how much response might be queued. In this JIRA we will add some metrics for 
> monitoring



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-17214) Add metrics on responseQueue length and size

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-17214:
--
Fix Version/s: (was: 2.0.0)

> Add metrics on responseQueue length and size
> 
>
> Key: HBASE-17214
> URL: https://issues.apache.org/jira/browse/HBASE-17214
> Project: HBase
>  Issue Type: Improvement
>  Components: Operability
>Reporter: Yu Li
>Assignee: Yu Li
>Priority: Critical
>
> Currently in RpcServer we will use a response queue if multiple response to 
> write to the same connection, but there's no view of whether it's used and 
> how much response might be queued. In this JIRA we will add some metrics for 
> monitoring



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-17009) Revisiting the removement of managed connection and connection caching

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-17009:
--
Fix Version/s: (was: 2.0.0)

> Revisiting the removement of managed connection and connection caching
> --
>
> Key: HBASE-17009
> URL: https://issues.apache.org/jira/browse/HBASE-17009
> Project: HBase
>  Issue Type: Task
>  Components: Operability
>Reporter: Yu Li
>Assignee: Yu Li
>Priority: Critical
>
> In HBASE-13197 we have done lots of good cleanups for Connection API, but 
> among which HBASE-13252 dropped the feature of managed connection and 
> connection caching, and this JIRA propose to have a revisit on this decision 
> for below reasons.
> Assume we have a long running process with multiple threads accessing HBase 
> (a common case for streaming application), let's see what happens previously 
> and now.
> Previously:
> User could create an HTable instance whenever they want w/o worrying about 
> the underlying connections because HBase client will mange it automatically, 
> say no matter how many threads there will be only one Connection instance
> {code}
>   @Deprecated
>   public HTable(Configuration conf, final TableName tableName)
>   throws IOException {
> ...
> this.connection = ConnectionManager.getConnectionInternal(conf);
> ...
>   }
>   static ClusterConnection getConnectionInternal(final Configuration conf)
> throws IOException {
> HConnectionKey connectionKey = new HConnectionKey(conf);
> synchronized (CONNECTION_INSTANCES) {
>   HConnectionImplementation connection = 
> CONNECTION_INSTANCES.get(connectionKey);
>   if (connection == null) {
> connection = (HConnectionImplementation)createConnection(conf, true);
> CONNECTION_INSTANCES.put(connectionKey, connection);
>   } else if (connection.isClosed()) {
> ConnectionManager.deleteConnection(connectionKey, true);
> connection = (HConnectionImplementation)createConnection(conf, true);
> CONNECTION_INSTANCES.put(connectionKey, connection);
>   }
>   connection.incCount();
>   return connection;
> }
>   }
> {code}
> Now:
> User has to create the connection by themselves, using below codes like 
> indicated in our recommendations
> {code}
> Connection connection = ConnectionFactory.createConnection(conf);
> Table table = connection.getTable(tableName);
> {code}
> And they must make sure *only one* single connection created in one *process* 
> instead of creating HTable instance freely, or else there might be many 
> connections setup to zookeeper/RS with multiple threads. Also user might ask 
> "when I should close the connection I close?" and the answer is "make sure 
> don't close it until the *process* shutdown"
> So now we have much more things for user to "Make sure", but custom is 
> something hard to change. User used to create table instance in each thread 
> (according to which table to access per requested) so probably they will 
> still create connections everywhere, and then operators will have to crazily 
> resolve all kinds of problems...
> So I'm proposing to add back the managed connection and connection caching 
> support. IMHO it's something good and ever existed in our implementation, so 
> let's bring it back and save the workload for operators when they decided to 
> upgrade from 1.x to 2.x
> Thoughts?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-15666) shaded dependencies for hbase-testing-util

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-15666:
--
Fix Version/s: (was: 1.5.0)
   (was: 2.0.0)

> shaded dependencies for hbase-testing-util
> --
>
> Key: HBASE-15666
> URL: https://issues.apache.org/jira/browse/HBASE-15666
> Project: HBase
>  Issue Type: New Feature
>  Components: test
>Affects Versions: 1.1.0, 1.2.0
>Reporter: Sean Busbey
>Priority: Critical
>
> Folks that make use of our shaded client but then want to test things using 
> the hbase-testing-util end up getting all of our dependencies again in the 
> test scope.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-14583) Enabled client-side metrics by default

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-14583:
--
Fix Version/s: (was: 1.4.3)
   (was: 1.3.3)
   (was: 1.5.0)
   (was: 2.0.0)

> Enabled client-side metrics by default
> --
>
> Key: HBASE-14583
> URL: https://issues.apache.org/jira/browse/HBASE-14583
> Project: HBase
>  Issue Type: Task
>  Components: Client, Operability, Performance
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Critical
> Attachments: 14583.00.branch-1.patch, 14583.00.patch
>
>
> Enabling this feature by default for master and branch-1.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-13772) Replication endpoints should restrict access to a limited set of principals

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-13772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-13772:
--
Fix Version/s: (was: 2.0.0)

> Replication endpoints should restrict access to a limited set of principals
> ---
>
> Key: HBASE-13772
> URL: https://issues.apache.org/jira/browse/HBASE-13772
> Project: HBase
>  Issue Type: Bug
>Reporter: Andrew Purtell
>Priority: Critical
>
> Replication endpoints will accept RPC connections from any Kerberos principal 
> that is trusted by the endpoint's local KDC. This is far too open and may 
> allow for the establishment of rogue endpoints (in conjunction with 
> HBASE-13771). 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (HBASE-11236) Last flushed sequence id is ignored by ServerManager

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-11236.
---
Resolution: Fixed
  Assignee: Chia-Ping Tsai

Resolving as [~chia7712] suggests after research. Assigning issue to Chia-Ping 
since he did the work.

> Last flushed sequence id is ignored by ServerManager
> 
>
> Key: HBASE-11236
> URL: https://issues.apache.org/jira/browse/HBASE-11236
> Project: HBase
>  Issue Type: Bug
>Reporter: Jimmy Xiang
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0
>
>
> I got lots of error messages like this:
> {quote}
> 2014-05-22 08:58:59,793 DEBUG [RpcServer.handler=1,port=20020] 
> master.ServerManager: RegionServer 
> a2428.halxg.cloudera.com,20020,1400742071109 indicates a last flushed 
> sequence id (numberOfStores=9, numberOfStorefiles=2, 
> storefileUncompressedSizeMB=517, storefileSizeMB=517, 
> compressionRatio=1., memstoreSizeMB=0, storefileIndexSizeMB=0, 
> readRequestsCount=0, writeRequestsCount=0, rootIndexSizeKB=34, 
> totalStaticIndexSizeKB=381, totalStaticBloomSizeKB=0, totalCompactingKVs=0, 
> currentCompactedKVs=0, compactionProgressPct=NaN) that is less than the 
> previous last flushed sequence id (605446) for region 
> IntegrationTestBigLinkedList, 
> �A��*t�^FU�2��0,1400740489477.a44d3e309b5a7e29355f6faa0d3a4095. Ignoring.
> {quote}
> RegionLoad.toString doesn't print out the last flushed sequence id passed in. 
> Why is it less than the previous one?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-8963) Add configuration option to skip HFile archiving

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-8963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-8963:
-
Fix Version/s: (was: 2.0.0)

> Add configuration option to skip HFile archiving
> 
>
> Key: HBASE-8963
> URL: https://issues.apache.org/jira/browse/HBASE-8963
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
>  Labels: usability
> Attachments: 8963-v10.txt, HBASE-8963.trunk.v1.patch, 
> HBASE-8963.trunk.v2.patch, HBASE-8963.trunk.v3.patch, 
> HBASE-8963.trunk.v4.patch, HBASE-8963.trunk.v5.patch, 
> HBASE-8963.trunk.v6.patch, HBASE-8963.trunk.v7.patch, 
> HBASE-8963.trunk.v8.patch, HBASE-8963.trunk.v9.patch
>
>
> Currently HFileArchiver is always called when a table is dropped or compacted.
> A configuration option (either global or per table) should be provided so 
> that archiving can be skipped when table is deleted or compacted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-11208) Remove the hbase.hstore.blockingStoreFiles setting

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-11208:
--
Fix Version/s: (was: 2.0.0)

> Remove the hbase.hstore.blockingStoreFiles setting
> --
>
> Key: HBASE-11208
> URL: https://issues.apache.org/jira/browse/HBASE-11208
> Project: HBase
>  Issue Type: Brainstorming
>  Components: Compaction, regionserver
>Affects Versions: 0.99.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
>Priority: Critical
>
> It's a little bit of a provocation, but the rational is:
>  - there are some bugs around the delayed flush. For example, if the periodic 
> scheduler has asked for a delayed flush, and that we need to flush, we will 
> have to wait
>  - if the number of WAL files increases, we won't flush immediately if the 
> blockingFile number has been reached. This impacts the MTTR.
>  - We don't write to limit the compaction impact, but they are many cases 
> where we would want to flush anyway, as the writes cannot wait.
>  - this obviously leads to huge write latency peaks.
> So I'm questioning this setting, it leads to multiple intricate cases, 
> unpredictable write latency, and looks like a workaround for compaction 
> performances. With all the work done on compaction, I think we can get rid of 
> it.  A solution in the middle would be to deprecate it and to set it to a 
> large value...
> Any opinion before I shoot :-) ? 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-7013) Avoid reusing a input stream that stumbles on rpc-timeout in HBaseClient

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-7013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-7013:
-
Fix Version/s: (was: 2.0.0)

> Avoid reusing a input stream that stumbles on rpc-timeout in HBaseClient
> 
>
> Key: HBASE-7013
> URL: https://issues.apache.org/jira/browse/HBASE-7013
> Project: HBase
>  Issue Type: Bug
>  Components: Client
>Reporter: Hiroshi Ikeda
>Priority: Critical
>  Labels: delete
>
> HBASE-2937 introduces rpc-timeout and sets SO_TIMEOUT parameter of the socket 
> to throw SocketTimeoutException. That means the exception can be thrown from 
> any code that reads data directly/indirectly from the socket. If the 
> exception is thrown in the middle of reading a set of data, it is required to 
> drag out and drop the rest part of the set of data from the socket and make 
> ready to read the next data, in order to reuse the socket. It seems 
> difficult, and I can't find such recovering code in HBaseClient.
> I think, when IO streams wrapping the socket throw an exception, the nesting 
> connection instance should be discarded, and rpc-timeout should be handled 
> separately from SO_TIMEOUT parameter.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20090) Properly handle Preconditions check failure in MemStoreFlusher$FlushHandler.run

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383145#comment-16383145
 ] 

Ted Yu commented on HBASE-20090:


It seems the Preconditions check can be converted to a normal condition check.
[~ram_krish] [~anoop.hbase] [~anastas] :
Can you take a look at the patch ?
Here was snippet from region server log during PE randomWrite:
{code}
2018-03-02 03:55:19,232 INFO  [MemStoreFlusher.1] regionserver.MemStoreFlusher: 
Flush of region atlas_janus,,1519927429371.fbcb5e495344542daf8b499e4bac03ae. 
due to global heap pressure. Flush type=ABOVE_ONHEAP_HIGHER_MARKTotal 
Memstore Heap size=403.9 MTotal Memstore Off-Heap size=0, Region memstore size=0
2018-03-02 03:55:19,232 INFO  [MemStoreFlusher.1] regionserver.MemStoreFlusher: 
Nothing to flush for 
atlas_janus,,1519927429371.fbcb5e495344542daf8b499e4bac03ae.
2018-03-02 03:55:19,232 INFO  [MemStoreFlusher.1] regionserver.MemStoreFlusher: 
Excluding unflushable region 
atlas_janus,,1519927429371.fbcb5e495344542daf8b499e4bac03ae. -trying to 
find a different region to flush.
{code}
Note atlas_janus was not the table being written.
TestTable was being written to.

> Properly handle Preconditions check failure in 
> MemStoreFlusher$FlushHandler.run
> ---
>
> Key: HBASE-20090
> URL: https://issues.apache.org/jira/browse/HBASE-20090
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20094.v01.patch
>
>
> Here is the code in branch-2 :
> {code}
> try {
>   wakeupPending.set(false); // allow someone to wake us up again
>   fqe = flushQueue.poll(threadWakeFrequency, TimeUnit.MILLISECONDS);
>   if (fqe == null || fqe instanceof WakeupFlushThread) {
> ...
>   if (!flushOneForGlobalPressure()) {
> ...
>   FlushRegionEntry fre = (FlushRegionEntry) fqe;
>   if (!flushRegion(fre)) {
> break;
> ...
> } catch (Exception ex) {
>   LOG.error("Cache flusher failed for entry " + fqe, ex);
>   if (!server.checkFileSystem()) {
> break;
>   }
> }
> {code}
> Inside flushOneForGlobalPressure():
> {code}
>   Preconditions.checkState(
> (regionToFlush != null && regionToFlushSize > 0) ||
> (bestRegionReplica != null && bestRegionReplicaSize > 0));
> {code}
> When the Preconditions check fails, IllegalStateException is caught by the 
> catch block shown above.
> However, the fqe is not flushed, resulting in potential data loss.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19663) site build fails complaining "javadoc: error - class file for javax.annotation.meta.TypeQualifierNickname not found"

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19663:
--
Fix Version/s: (was: 1.5.0)
   (was: 2.0.0)

> site build fails complaining "javadoc: error - class file for 
> javax.annotation.meta.TypeQualifierNickname not found"
> 
>
> Key: HBASE-19663
> URL: https://issues.apache.org/jira/browse/HBASE-19663
> Project: HBase
>  Issue Type: Bug
>  Components: site
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Attachments: script.sh
>
>
> Cryptic failure trying to build beta-1 RC. Fails like this:
> {code}
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 03:54 min
> [INFO] Finished at: 2017-12-29T01:13:15-08:00
> [INFO] Final Memory: 381M/9165M
> [INFO] 
> 
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on project 
> hbase: Error generating maven-javadoc-plugin:2.10.3:aggregate:
> [ERROR] Exit code: 1 - warning: unknown enum constant When.ALWAYS
> [ERROR] reason: class file for javax.annotation.meta.When not found
> [ERROR] warning: unknown enum constant When.UNKNOWN
> [ERROR] warning: unknown enum constant When.MAYBE
> [ERROR] 
> /home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java:762:
>  warning - Tag @link: malformed: "#matchingRows(Cell, byte[]))"
> [ERROR] 
> /home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java:762:
>  warning - Tag @link: reference not found: #matchingRows(Cell, byte[]))
> [ERROR] 
> /home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java:762:
>  warning - Tag @link: reference not found: #matchingRows(Cell, byte[]))
> [ERROR] javadoc: warning - Class javax.annotation.Nonnull not found.
> [ERROR] javadoc: error - class file for 
> javax.annotation.meta.TypeQualifierNickname not found
> [ERROR]
> [ERROR] Command line was: /home/stack/bin/jdk1.8.0_151/jre/../bin/javadoc 
> -J-Xmx2G @options @packages
> [ERROR]
> [ERROR] Refer to the generated Javadoc files in 
> '/home/stack/hbase.git/target/site/apidocs' dir.
> [ERROR] -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please 
> read the following articles:
> [ERROR] [Help 1] 
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> {code}
> javax.annotation.meta.TypeQualifierNickname is out of jsr305 but we don't 
> include this anywhere according to mvn dependency.
> Happens building the User API both test and main.
> Excluding these lines gets us passing again:
> {code}
>   3511   
>   3512 
> org.apache.yetus.audience.tools.IncludePublicAnnotationsStandardDoclet
>   3513   
>   3514   
>   3515 org.apache.yetus
>   3516 audience-annotations
>   3517 ${audience-annotations.version}
>   3518   
> + 3519   true
> {code}
> Tried upgrading to newer mvn site (ours is three years old) but that a 
> different set of problems.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20090) Properly handle Preconditions check failure in MemStoreFlusher$FlushHandler.run

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-20090:
---
Attachment: 20094.v01.patch

> Properly handle Preconditions check failure in 
> MemStoreFlusher$FlushHandler.run
> ---
>
> Key: HBASE-20090
> URL: https://issues.apache.org/jira/browse/HBASE-20090
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20094.v01.patch
>
>
> Here is the code in branch-2 :
> {code}
> try {
>   wakeupPending.set(false); // allow someone to wake us up again
>   fqe = flushQueue.poll(threadWakeFrequency, TimeUnit.MILLISECONDS);
>   if (fqe == null || fqe instanceof WakeupFlushThread) {
> ...
>   if (!flushOneForGlobalPressure()) {
> ...
>   FlushRegionEntry fre = (FlushRegionEntry) fqe;
>   if (!flushRegion(fre)) {
> break;
> ...
> } catch (Exception ex) {
>   LOG.error("Cache flusher failed for entry " + fqe, ex);
>   if (!server.checkFileSystem()) {
> break;
>   }
> }
> {code}
> Inside flushOneForGlobalPressure():
> {code}
>   Preconditions.checkState(
> (regionToFlush != null && regionToFlushSize > 0) ||
> (bestRegionReplica != null && bestRegionReplicaSize > 0));
> {code}
> When the Preconditions check fails, IllegalStateException is caught by the 
> catch block shown above.
> However, the fqe is not flushed, resulting in potential data loss.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HBASE-20090) Properly handle Preconditions check failure in MemStoreFlusher$FlushHandler.run

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu reassigned HBASE-20090:
--

Assignee: Ted Yu

> Properly handle Preconditions check failure in 
> MemStoreFlusher$FlushHandler.run
> ---
>
> Key: HBASE-20090
> URL: https://issues.apache.org/jira/browse/HBASE-20090
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
>
> Here is the code in branch-2 :
> {code}
> try {
>   wakeupPending.set(false); // allow someone to wake us up again
>   fqe = flushQueue.poll(threadWakeFrequency, TimeUnit.MILLISECONDS);
>   if (fqe == null || fqe instanceof WakeupFlushThread) {
> ...
>   if (!flushOneForGlobalPressure()) {
> ...
>   FlushRegionEntry fre = (FlushRegionEntry) fqe;
>   if (!flushRegion(fre)) {
> break;
> ...
> } catch (Exception ex) {
>   LOG.error("Cache flusher failed for entry " + fqe, ex);
>   if (!server.checkFileSystem()) {
> break;
>   }
> }
> {code}
> Inside flushOneForGlobalPressure():
> {code}
>   Preconditions.checkState(
> (regionToFlush != null && regionToFlushSize > 0) ||
> (bestRegionReplica != null && bestRegionReplicaSize > 0));
> {code}
> When the Preconditions check fails, IllegalStateException is caught by the 
> catch block shown above.
> However, the fqe is not flushed, resulting in potential data loss.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19943) Only allow removing sync replication peer which is in DA state

[Zheng Hu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383139#comment-16383139
 ] 

Zheng Hu commented on HBASE-19943:
--

Patch v2. fix the failed TestReplicationAdmin UT.

> Only allow removing sync replication peer which is in DA state
> --
>
> Key: HBASE-19943
> URL: https://issues.apache.org/jira/browse/HBASE-19943
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-19943.HBASE-19064.001.patch, 
> HBASE-19943.HBASE-19064.002.patch
>
>
> To simplify the logic of RemovePeerProcedure. Otherwise we may also need to 
> reopen regions which makes the RemovePeerProcedure can not fit for both sync 
> and normal replication peer.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19943) Only allow removing sync replication peer which is in DA state

[Zheng Hu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Hu updated HBASE-19943:
-
Attachment: HBASE-19943.HBASE-19064.002.patch

> Only allow removing sync replication peer which is in DA state
> --
>
> Key: HBASE-19943
> URL: https://issues.apache.org/jira/browse/HBASE-19943
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-19943.HBASE-19064.001.patch, 
> HBASE-19943.HBASE-19064.002.patch
>
>
> To simplify the logic of RemovePeerProcedure. Otherwise we may also need to 
> reopen regions which makes the RemovePeerProcedure can not fit for both sync 
> and normal replication peer.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19943) Only allow removing sync replication peer which is in DA state

[Zheng Hu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383135#comment-16383135
 ] 

Zheng Hu commented on HBASE-19943:
--

The TestRegionOpen UT is broken after HBASE-19391 , and the HBASE-20052 fix the 
failed UT.   So our sync replication branch need rebase to the lastest master 
now ..

> Only allow removing sync replication peer which is in DA state
> --
>
> Key: HBASE-19943
> URL: https://issues.apache.org/jira/browse/HBASE-19943
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-19943.HBASE-19064.001.patch
>
>
> To simplify the logic of RemovePeerProcedure. Otherwise we may also need to 
> reopen regions which makes the RemovePeerProcedure can not fit for both sync 
> and normal replication peer.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20114) Fix IllegalFormatConversionException in rsgroup.jsp

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-20114:
---
Status: Patch Available  (was: Open)

> Fix IllegalFormatConversionException in rsgroup.jsp
> ---
>
> Key: HBASE-20114
> URL: https://issues.apache.org/jira/browse/HBASE-20114
> Project: HBase
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 3.0.0, 2.0.0-beta-2
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Attachments: HBASE-20114.branch-2.001.patch
>
>
> IllegalFormatConversionException at
> {code:java}
> String.format("%.0f", sl.getRequestCountPerSecond()) 
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20095) Redesign single instance pool in CleanerChore

[Reid Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383128#comment-16383128
 ] 

Reid Chan commented on HBASE-20095:
---

bq. have static getPool(Configuration) method called from constructor that does 
the init steps?

bq. 
@edu.umd.cs.findbugs.annotations.SuppressWarnings(value="ST_WRITE_TO_STATIC_FROM_INSTANCE_METHOD",
 justification="TODO: Fix. It is wonky have static pool initialized from 
instance")

That's why i move to eager initialization, thanks for any comment and welcome 
more.

> Redesign single instance pool in CleanerChore
> -
>
> Key: HBASE-20095
> URL: https://issues.apache.org/jira/browse/HBASE-20095
> Project: HBase
>  Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Critical
> Attachments: HBASE-20095.master.001.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20006) TestRestoreSnapshotFromClientWithRegionReplicas is flakey

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-20006:
--
Fix Version/s: (was: 2.0.0)

> TestRestoreSnapshotFromClientWithRegionReplicas is flakey
> -
>
> Key: HBASE-20006
> URL: https://issues.apache.org/jira/browse/HBASE-20006
> Project: HBase
>  Issue Type: Bug
>  Components: read replicas
>Reporter: stack
>Assignee: Toshihiro Suzuki
>Priority: Critical
> Attachments: HBASE-20006.branch-2.001.patch
>
>
> Failing 10% of the time. Interestingly, it is below that causes fail. We go 
> to split but it is already split. We will then fail the split with an 
> internal assert which messes up procedures; at a minimum we should just not 
> split (this is in the prepare stage).
> {code}
> 2018-02-15 23:21:42,162 INFO  [PEWorker-12] 
> procedure.MasterProcedureScheduler(571): pid=105, 
> state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure 
> table=testOnlineSnapshotAfterSplittingRegions-1518736887838, 
> parent=3f850cea7d71a7ebd019f2f009efca4d, 
> daughterA=06b5e6366efbef155d70e56cfdf58dc9, 
> daughterB=8c175de1b33765a5683ac1e502edb0bd, 
> table=testOnlineSnapshotAfterSplittingRegions-1518736887838, 
> testOnlineSnapshotAfterSplittingRegions-1518736887838,,1518736887882.3f850cea7d71a7ebd019f2f009efca4d.
> 2018-02-15 23:21:42,162 INFO  [PEWorker-12] 
> assignment.SplitTableRegionProcedure(440): Split of {ENCODED => 
> 3f850cea7d71a7ebd019f2f009efca4d, NAME => 
> 'testOnlineSnapshotAfterSplittingRegions-1518736887838,,1518736887882.3f850cea7d71a7ebd019f2f009efca4d.',
>  STARTKEY => '', ENDKEY => '1'} skipped; state is already SPLIT
> 2018-02-15 23:21:42,163 ERROR [PEWorker-12] 
> procedure2.ProcedureExecutor(1480): CODE-BUG: Uncaught runtime exception: 
> pid=105, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure 
> table=testOnlineSnapshotAfterSplittingRegions-1518736887838, 
> parent=3f850cea7d71a7ebd019f2f009efca4d, 
> daughterA=06b5e6366efbef155d70e56cfdf58dc9, 
> daughterB=8c175de1b33765a5683ac1e502edb0bd
> java.lang.AssertionError: split region should have an exception here
>   at 
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:228)
>   at 
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:89)
>   at 
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:180)
>   at 
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1455)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1224)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1734)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (HBASE-20067) Clean up findbugs warnings

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-20067.
---
Resolution: Fixed
  Assignee: Duo Zhang

Resolving as done. All subtasks complete.

> Clean up findbugs warnings
> --
>
> Key: HBASE-20067
> URL: https://issues.apache.org/jira/browse/HBASE-20067
> Project: HBase
>  Issue Type: Task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.0.0
>
>
> It seems that our pre commit build fails to execute findbugs for a while and 
> now it is back. Need to do a cleanup otherwise we can not get a green result 
> at all.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (HBASE-20094) Import$CellWritableComparable should define equals()

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-20094.
---
Resolution: Duplicate

Resolved by the below


commit 03f9cb89a62de38abf2402869807be5d42813b82
Author: Michael Stack 
Date:   Wed Feb 28 23:38:35 2018 -0800

HBASE-20110 Findbugs in zk and mr caused nightly #409 branch-2 to fail


Resolving as dupe.

> Import$CellWritableComparable should define equals()
> 
>
> Key: HBASE-20094
> URL: https://issues.apache.org/jira/browse/HBASE-20094
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
>
> Bug type EQ_COMPARETO_USE_OBJECT_EQUALS
> {code}
> In class org.apache.hadoop.hbase.mapreduce.Import$CellWritableComparable
> In method 
> org.apache.hadoop.hbase.mapreduce.Import$CellWritableComparable.compareTo(Import$CellWritableComparable)
> At Import.java:[line 149]
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-6970) hbase-deamon.sh creates/updates pid file even when that start failed.

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-6970:
-
Fix Version/s: (was: 2.0.0)

> hbase-deamon.sh creates/updates pid file even when that start failed.
> -
>
> Key: HBASE-6970
> URL: https://issues.apache.org/jira/browse/HBASE-6970
> Project: HBase
>  Issue Type: Bug
>  Components: Usability
>Reporter: Lars Hofhansl
>Assignee: Ashish Singhi
>Priority: Major
>
> We just ran into a strange issue where could neither start nor stop services 
> with hbase-deamon.sh.
> The problem is this:
> {code}
> nohup nice -n $HBASE_NICENESS "$HBASE_HOME"/bin/hbase \
> --config "${HBASE_CONF_DIR}" \
> $command "$@" $startStop > "$logout" 2>&1 < /dev/null &
> echo $! > $pid
> {code}
> So the pid file is created or updated even when the start of the service 
> failed. The next stop command will then fail, because the pid file has the 
> wrong pid in it.
> Edit: Spelling and more spelling errors.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-5839) Backup master not starting up due to Bind Exception while starting HttpServer

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5839:
-
Fix Version/s: (was: 2.0.0)

> Backup master not starting up due to Bind Exception while starting HttpServer
> -
>
> Key: HBASE-5839
> URL: https://issues.apache.org/jira/browse/HBASE-5839
> Project: HBase
>  Issue Type: Bug
>Reporter: ramkrishna.s.vasudevan
>Priority: Major
>
> Backup master tries to bind to the info port 60010.
> This is done once the back up master becomes active.  Even before that the 
> Data Xceviers threads (IPC handlers) are started and they are started at 
> random port.  If already 60010 is used then when standby master comes up then 
> it fails due to bind exception.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-8073) HFileOutputFormat support for offline operation

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-8073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-8073:
-
Fix Version/s: (was: 2.0.0)

> HFileOutputFormat support for offline operation
> ---
>
> Key: HBASE-8073
> URL: https://issues.apache.org/jira/browse/HBASE-8073
> Project: HBase
>  Issue Type: Sub-task
>  Components: mapreduce
>Reporter: Nick Dimiduk
>Priority: Major
> Attachments: HBASE-8073-trunk-v0.patch, HBASE-8073-trunk-v1.patch
>
>
> When using HFileOutputFormat to generate HFiles, it inspects the region 
> topology of the target table. The split points from that table are used to 
> guide the TotalOrderPartitioner. If the target table does not exist, it is 
> first created. This imposes an unnecessary dependence on an online HBase and 
> existing table.
> If the table exists, it can be used. However, the job can be smarter. For 
> example, if there's far more data going into the HFiles than the table 
> currently contains, the table regions aren't very useful for data split 
> points. Instead, the input data can be sampled to produce split points more 
> meaningful to the dataset. LoadIncrementalHFiles is already capable of 
> handling divergence between HFile boundaries and table regions, so this 
> should not pose any additional burdon at load time.
> The proper method of sampling the data likely requires a custom input format 
> and an additional map-reduce job perform the sampling. See a relevant 
> implementation: 
> https://github.com/alexholmes/hadoop-book/blob/master/src/main/java/com/manning/hip/ch4/sampler/ReservoirSamplerInputFormat.java



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-7245) Recovery on failed snapshot restore

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-7245:
-
Fix Version/s: (was: 2.0.0)

> Recovery on failed snapshot restore
> ---
>
> Key: HBASE-7245
> URL: https://issues.apache.org/jira/browse/HBASE-7245
> Project: HBase
>  Issue Type: Bug
>  Components: Client, master, regionserver, snapshots, Zookeeper
>Reporter: Jonathan Hsieh
>Priority: Major
>
> Restore will do updates to the file system and to meta.  it seems that an 
> inopportune failure before meta is completely updated could result in an 
> inconsistent state that would require hbck to fix.
> We should define what the semantics are for recovering from this.  Some 
> suggestions:
> 1) Fail Forward (see some log saying restore's meta edits not completed, then 
> gather information necessary to build it all from fs, and complete meta 
> edits.).
> 2) Fail backwards (see some log saying restore's meta edits not completed, 
> delete incomplete snapshot region entries from meta.)  
> I think I prefer 1 -- if two processes end somehow updating  (somehow the 
> original master didn't die, and a new one started up) they would be 
> idempotent.  If we used 2, we could still have a race and still be in a bad 
> place.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-7743) Replace *SortReducers with Hadoop Secondary Sort

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-7743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-7743:
-
Fix Version/s: (was: 2.0.0)

> Replace *SortReducers with Hadoop Secondary Sort
> 
>
> Key: HBASE-7743
> URL: https://issues.apache.org/jira/browse/HBASE-7743
> Project: HBase
>  Issue Type: Sub-task
>  Components: mapreduce, Performance
>Reporter: Nick Dimiduk
>Priority: Major
>
> The mapreduce package provides two Reducer implementations, 
> KeyValueSortReducer and PutSortReducer, which are used by Import, ImportTsv, 
> and WALPlayer in conjunction with the HFileOutputFormat. Both of these 
> implementations make use of a TreeSet to sort values matching a key. This 
> reducer will OOM when rows are large.
> A better solution would be to implement secondary sort of the values. That 
> way hadoop sorts the records, spilling to disk when necessary.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-7115) [shell] Provide a way to register custom filters with the Filter Language Parser

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-7115:
-
Fix Version/s: (was: 1.5.0)
   (was: 2.0.0)

> [shell] Provide a way to register custom filters with the Filter Language 
> Parser
> 
>
> Key: HBASE-7115
> URL: https://issues.apache.org/jira/browse/HBASE-7115
> Project: HBase
>  Issue Type: Improvement
>  Components: Filters, shell
>Affects Versions: 0.95.2
>Reporter: Aditya Kishore
>Assignee: Aditya Kishore
>Priority: Major
> Attachments: HBASE-7115_trunk.patch, HBASE-7115_trunk.patch, 
> HBASE-7115_trunk_v2.patch
>
>
> HBASE-5428 added this capability to thrift interface but the configuration 
> parameter name is "thrift" specific.
> This patch introduces a more generic parameter "hbase.user.filters" using 
> which the user defined custom filters can be specified in the configuration 
> and loaded in any client that needs to use the filter language parser.
> The patch then uses this new parameter to register any user specified filters 
> while invoking the HBase shell.
> Example usage: Let's say I have written a couple of custom filters with class 
> names *{{org.apache.hadoop.hbase.filter.custom.SuperDuperFilter}}* and 
> *{{org.apache.hadoop.hbase.filter.custom.SilverBulletFilter}}* and I want to 
> use them from HBase shell using the filter language.
> To do that, I would add the following configuration to {{hbase-site.xml}}
> {panel}{{}}
> {{  hbase.user.filters}}
> {{  
> }}*{{SuperDuperFilter}}*{{:org.apache.hadoop.hbase.filter.custom.SuperDuperFilter,}}*{{SilverBulletFilter}}*{{:org.apache.hadoop.hbase.filter.custom.SilverBulletFilter}}
> {{}}{panel}
> Once this is configured, I can launch HBase shell and use these filters in my 
> {{get}} or {{scan}} just the way I would use a built-in filter.
> {code}
> hbase(main):001:0> scan 't', {FILTER => "SuperDuperFilter(true) AND 
> SilverBulletFilter(42)"}
> ROW  COLUMN+CELL
>  status  column=cf:a, 
> timestamp=30438552, value=world_peace
> 1 row(s) in 0. seconds
> {code}
> To use this feature in any client, the client needs to make the following 
> function call as part of its initialization.
> {code}
> ParseFilter.registerUserFilters(configuration);
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (HBASE-8593) Type support in ImportTSV tool

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-8593.
--
Resolution: REMIND

Resolving as old, for a different time. No progress. Resolving.

> Type support in ImportTSV tool
> --
>
> Key: HBASE-8593
> URL: https://issues.apache.org/jira/browse/HBASE-8593
> Project: HBase
>  Issue Type: Sub-task
>  Components: mapreduce
>Reporter: Anoop Sam John
>Assignee: rajeshbabu
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-8593.patch, HBASE-8593_v2.patch, 
> HBASE-8593_v4.patch, ReportMapper.java
>
>
> Now the ImportTSV tool treats all the table column to be of type String. It 
> converts the input data into bytes considering its type to be String. Some 
> times user will need a type of say int/float to get added to table by using 
> this tool.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-8091) Goraci test might rewrite data in case of map task failures, which eclipses data loss problems

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-8091:
-
Fix Version/s: (was: 2.0.0)

> Goraci test might rewrite data in case of map task failures, which eclipses 
> data loss problems
> --
>
> Key: HBASE-8091
> URL: https://issues.apache.org/jira/browse/HBASE-8091
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Major
>
> As discussed in HBASE-8031, goraci map tasks will rewite the same data if the 
> map task fails. There are some conditions that this might cause overwriting 
> data that would have been reported as loss otherwise. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-10184) [Online Schema Change]: Add additional tests for online schema change

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-10184:
--
Fix Version/s: (was: 2.0.0)

> [Online Schema Change]: Add additional tests for online schema change
> -
>
> Key: HBASE-10184
> URL: https://issues.apache.org/jira/browse/HBASE-10184
> Project: HBase
>  Issue Type: Task
>  Components: test
>Affects Versions: 0.96.1, 0.99.0
>Reporter: Aleksandr Shulman
>Priority: Major
>  Labels: online_schema_change
> Attachments: 10184-4.patch, 10184.addendum, HBASE-10184-trunk.diff
>
>
> There are some gaps in testing for Online Schema Change:
> Examples of some tests that should be added:
> 1. Splits with online schema change
> 2. Merge during online schema change
> 3. MR over HBase during online schema change
> 4. Bulk Load during online schema change
> 5. Online change table owner
> 6. Online Replication scope change
> 7. Online Bloom Filter change
> 8. Snapshots during online schema change (HBASE-10136)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-10619) Don't allow non super users to do DDL ops on system tables

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-10619:
--
Fix Version/s: (was: 2.0.0)

> Don't allow non super users to do DDL ops on system tables
> --
>
> Key: HBASE-10619
> URL: https://issues.apache.org/jira/browse/HBASE-10619
> Project: HBase
>  Issue Type: Bug
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
>Priority: Major
> Attachments: HBASE-10619.patch, HBASE-10619_V2.patch, 
> HBASE-10619_V3.patch, HBASE-10619_V4.patch, HBASE-10619_V5.patch, 
> HBASE-10619_V6.patch
>
>
> Don't allow non super users to do DDL ops on system tables
> For META and NS tables, we should allow them to be disabled even by super 
> users.  With out these tables online the cluster will be non operational.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-10919) [VisibilityController] ScanLabelGenerator using LDAP

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-10919:
--
Fix Version/s: (was: 2.0.0)

> [VisibilityController] ScanLabelGenerator using LDAP
> 
>
> Key: HBASE-10919
> URL: https://issues.apache.org/jira/browse/HBASE-10919
> Project: HBase
>  Issue Type: New Feature
>Reporter: Andrew Purtell
>Priority: Major
> Attachments: slides-10919.pdf
>
>
> A ScanLabelGenerator that queries an external service, using the LDAP 
> protocol, for a set of attributes corresponding to the principal represented 
> by the request UGI, and converts any returned in the response to additional 
> auths in the effective set.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-10974) Improve DBEs read performance by avoiding byte array deep copies for key[] and value[]

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-10974:
--
Fix Version/s: (was: 1.5.0)
   (was: 2.0.0)

> Improve DBEs read performance by avoiding byte array deep copies for key[] 
> and value[]
> --
>
> Key: HBASE-10974
> URL: https://issues.apache.org/jira/browse/HBASE-10974
> Project: HBase
>  Issue Type: Improvement
>  Components: Scanners
>Affects Versions: 0.99.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Major
> Attachments: HBASE-10974_1.patch
>
>
> As part of HBASE-10801, we  tried to reduce the copy of the value [] in 
> forming the KV from the DBEs. 
> The keys required copying and this was restricting us in using Cells and 
> always wanted to copy to be done.
> The idea here is to replace the key byte[] as ByteBuffer and create a 
> consecutive stream of the keys (currently the same byte[] is used and hence 
> the copy).  Use offset and length to track this key bytebuffer.
> The copy of the encoded format to normal Key format is definitely needed and 
> can't be avoided but we could always avoid the deep copy of the bytes to form 
> a KV and thus use cells effectively. Working on a patch, will post it soon.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-11084) Handle deletions and column tracking using internal filters rather than using DeleteTrackers and ColumnTrackers

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-11084:
--
Fix Version/s: (was: 2.0.0)

> Handle deletions and column tracking using internal filters rather than using 
> DeleteTrackers and ColumnTrackers
> ---
>
> Key: HBASE-11084
> URL: https://issues.apache.org/jira/browse/HBASE-11084
> Project: HBase
>  Issue Type: Improvement
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Major
>
> See HBASE-11054 for discussion.  
> Currently the delete tracking is done by DeleteTracker and its subclasses.  
> column tracking and its versions are handled inside the ColumnTrackers and 
> its subclasses. 
> This JIRA aims at providing internal filters and attaching them to 
> scans/gets, including minor and major compaction scans so that all the logic 
> of deletes and version counting goes into it rather than having trackers.  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (HBASE-11850) RpcClient can get stuck when closing

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-11850.
---
Resolution: Invalid

Old issue. Haven't seen this. Context has changed. Resolving as invalid.

> RpcClient can get stuck when closing
> 
>
> Key: HBASE-11850
> URL: https://issues.apache.org/jira/browse/HBASE-11850
> Project: HBase
>  Issue Type: Bug
>  Components: Client
>Affects Versions: 0.99.0, 2.0.0
>Reporter: Nicolas Liochon
>Priority: Major
> Fix For: 2.0.0
>
>
> we don't stop until the map in 'connections' is empty.
> the new connection is put in this map at creation, but if this connection is 
> not used it will never be removed.
> No totally sure of the fix yet. Probability is low (but not zero. I saw it 
> happening).
> The code is different in 0.98. It's 0.99+ issue only.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (HBASE-11827) Encryption support for bulkloading data into table with encryption configured for hfile format 3

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-11827.
---
Resolution: Later

Old issue. No progress. Resolving as LATER.

> Encryption support for bulkloading data into table with encryption configured 
> for hfile format 3
> 
>
> Key: HBASE-11827
> URL: https://issues.apache.org/jira/browse/HBASE-11827
> Project: HBase
>  Issue Type: Improvement
>  Components: mapreduce
>Affects Versions: 0.98.5
>Reporter: Kashif
>Assignee: Kashif
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-11827-98-v1.patch, HBASE-11827-trunk-v1.patch
>
>
> The solution would be to add support to auto detect encryption parameters 
> similar to other parameters like compression, datablockencoding, etc when 
> encryption is enabled for hfile format 3. 
> The current patch does the following:
> 1. Automatically detects encryption type and key in HFileOutputFormat & 
> HFileOutputFormat2.
> 2. Uses Base64encoder/decoder for url passing of Encryption key which is in 
> bytes format



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (HBASE-11860) Restore UpgradeTo in order to upgrade a HBase cluster to the latest release

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-11860.
---
Resolution: REMIND

Old issue. Resolving as invalid. Context has changed.

> Restore UpgradeTo in order to upgrade a HBase cluster to the latest release
> ---
>
> Key: HBASE-11860
> URL: https://issues.apache.org/jira/browse/HBASE-11860
> Project: HBase
>  Issue Type: Improvement
>  Components: HFile, master
>Reporter: Esteban Gutierrez
>Assignee: Esteban Gutierrez
>Priority: Major
> Fix For: 2.0.0
>
>
> Users should have the option to upgrade to a newer release of HBase without 
> performing a transitive upgrade to HBase 0.96 (EOL'd) or HBase 0.98 and then 
> HBase 1.x or 2.x
> In HBASE-11611 UpgradeTo96 was removed and it won't be possible to migrate a 
> HBase 0.94 cluster to HBase 1.x or 2.x  without the extra step. 
> Initially we can restore part of the functionality of the tool supporting 
> migration to namespaces and report for the existence of any HFileV1 file and 
> ZKDataMigrator could be safely removed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20114) Fix IllegalFormatConversionException in rsgroup.jsp

[Xiaolin Ha (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiaolin Ha updated HBASE-20114:
---
Attachment: HBASE-20114.branch-2.001.patch

> Fix IllegalFormatConversionException in rsgroup.jsp
> ---
>
> Key: HBASE-20114
> URL: https://issues.apache.org/jira/browse/HBASE-20114
> Project: HBase
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 3.0.0, 2.0.0-beta-2
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Attachments: HBASE-20114.branch-2.001.patch
>
>
> IllegalFormatConversionException at
> {code:java}
> String.format("%.0f", sl.getRequestCountPerSecond()) 
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (HBASE-11894) MetaEntries from coprocessor hooks in split and merge are not getting added to hbase:meta after HBASE-11611

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-11894.
---
Resolution: Invalid

Resolving as invalid. Context has changed.

> MetaEntries from coprocessor hooks in split and merge are not getting added 
> to hbase:meta after HBASE-11611
> ---
>
> Key: HBASE-11894
> URL: https://issues.apache.org/jira/browse/HBASE-11894
> Project: HBase
>  Issue Type: Bug
>  Components: Region Assignment
>Reporter: rajeshbabu
>Assignee: rajeshbabu
>Priority: Major
> Fix For: 2.0.0
>
>
> As part of HBASE-9249 & HBASE-9489, added new hooks in split and merge which 
> take meta entries from coprocessor hooks if any. These meta entries helps to 
> ensure atomicity of split(merge) of regions by server and split(merge) of the 
> regions handled in coprocessors(This is required in secondary indexing case).
> After HBASE-11611 the meta entries are not getting added to meta both in 
> split and merge.
> {code}
> @MetaMutationAnnotation
> List metaEntries = new ArrayList();
> if (rsCoprocessorHost != null) {
>   if (rsCoprocessorHost.preMergeCommit(this.region_a, this.region_b, 
> metaEntries)) {
> throw new IOException("Coprocessor bypassing regions " + 
> this.region_a + " "
> + this.region_b + " merge.");
>   }
>   try {
> for (Mutation p : metaEntries) {
>   HRegionInfo.parseRegionName(p.getRow());
> }
>   } catch (IOException e) {
> LOG.error("Row key of mutation from coprocessor is not parsable as 
> region name."
> + "Mutations from coprocessor should only be for hbase:meta 
> table.", e);
> throw e;
>   }
> }
> // This is the point of no return. Similar with SplitTransaction.
> // IF we reach the PONR then subsequent failures need to crash out this
> // regionserver
> this.journal.add(JournalEntry.PONR);
> // Add merged region and delete region_a and region_b
> // as an atomic update. See HBASE-7721. This update to hbase:meta makes 
> the region
> // will determine whether the region is merged or not in case of failures.
> // If it is successful, master will roll-forward, if not, master will
> // rollback
> if (services != null && 
> !services.reportRegionStateTransition(TransitionCode.MERGE_PONR,
> mergedRegionInfo, region_a.getRegionInfo(), 
> region_b.getRegionInfo())) {
>   // Passed PONR, let SSH clean it up
>   throw new IOException("Failed to notify master that merge passed PONR: "
> + region_a.getRegionInfo().getRegionNameAsString() + " and "
> + region_b.getRegionInfo().getRegionNameAsString());
> }
> {code}
> I think while reporting region state transition to master we need to pass 
> meta entries also so that we can add them to meta along with split or merge 
> updates.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-12208) Integerating Jacoco code coverage report into the hadoop qa bot

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-12208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-12208:
--
Fix Version/s: (was: 2.0.0)

> Integerating Jacoco code coverage report into the hadoop qa bot
> ---
>
> Key: HBASE-12208
> URL: https://issues.apache.org/jira/browse/HBASE-12208
> Project: HBase
>  Issue Type: New Feature
>  Components: test
>Affects Versions: 2.0.0
>Reporter: Manukranth Kolloju
>Priority: Major
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> HBASE-8936 introduced jacoco code coverage report into the pom, but its 
> disabled by default. The code coverage tool would help in finding the extent 
> of code coverage we have in new patches. The hadoop qa bot should be able to 
> stamp a -1 on diffs that decrease the coverage. This jira is to do the same. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-12254) Document limitations related to pluggable replication endpoint feature usage in 0.98

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-12254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-12254:
--
Fix Version/s: (was: 2.0.0)

> Document limitations related to pluggable replication endpoint feature usage 
> in 0.98
> 
>
> Key: HBASE-12254
> URL: https://issues.apache.org/jira/browse/HBASE-12254
> Project: HBase
>  Issue Type: Sub-task
>  Components: documentation
>Affects Versions: 0.98.7
>Reporter: ramkrishna.s.vasudevan
>Priority: Major
> Attachments: HBASE-12254.patch
>
>
> The pluggable Replication endpoint in 0.98 will need to be documented as how 
> exactly it can be used because of limitations that we may have due to mixed 
> version compatability where the peers may be in an older version of 0.98 
> where pluggable replication endpoint is not there. 
> Also this feature adds some more data to the znodes like the name of the 
> Endpoint impl, data and the Replication config.  A peer cluster with the 
> older version will not be able to read this data particularly when there is a 
> custom replication configured.  This JIRA aims at documenting such cases for 
> the ease of user.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (HBASE-12563) After Starting up mini hbase cluster, Real Configuration of Cluster never set to HBaseTestingUtility

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-12563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-12563.
---
Resolution: Later

No progress. Resolving as LATER

> After Starting up mini hbase cluster, Real Configuration of Cluster never set 
> to HBaseTestingUtility 
> -
>
> Key: HBASE-12563
> URL: https://issues.apache.org/jira/browse/HBASE-12563
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.0.0, 0.99.1
>Reporter: Talat UYARER
>Assignee: Talat UYARER
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-12563-v2.patch, HBASE-12563-v3.patch, 
> HBASE-12563.patch
>
>
> When I use startMiniHBaseCluster method. It starts up a Mini Cluster for the 
> tests. While starting It changes some configuration. For example Master's 
> port or Region Server's. After Cluster starting We should set its new 
> configuration to HbaseTestingUtils conf.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-12269) Add support for Scan.setRowPrefixFilter to thrift

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-12269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-12269:
--
Fix Version/s: (was: 1.5.0)
   (was: 2.0.0)

> Add support for Scan.setRowPrefixFilter to thrift
> -
>
> Key: HBASE-12269
> URL: https://issues.apache.org/jira/browse/HBASE-12269
> Project: HBase
>  Issue Type: New Feature
>  Components: Thrift
>Reporter: Niels Basjes
>Assignee: Niels Basjes
>Priority: Major
> Attachments: HBASE-12269-2014-10-15-v1.patch, 
> HBASE-12269-2014-10-16-v2-INCOMPLETE.patch, HBASE-12269-2014-12-05-v2.patch
>
>
> I think having the feature introduced in HBASE-11990 in the hbase thrift 
> interface would be very useful.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HBASE-20114) Fix IllegalFormatConversionException in rsgroup.jsp

[Xiaolin Ha (JIRA)]

Xiaolin Ha created HBASE-20114:
--

 Summary: Fix IllegalFormatConversionException in rsgroup.jsp
 Key: HBASE-20114
 URL: https://issues.apache.org/jira/browse/HBASE-20114
 Project: HBase
  Issue Type: Bug
  Components: UI
Affects Versions: 3.0.0, 2.0.0-beta-2
Reporter: Xiaolin Ha
Assignee: Xiaolin Ha


IllegalFormatConversionException at

{code:java}
String.format("%.0f", sl.getRequestCountPerSecond()) 
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-12807) Implement 0.89fb-style "roll on slow sync"

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-12807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-12807:
--
Fix Version/s: (was: 1.5.0)
   (was: 2.0.0)

> Implement 0.89fb-style "roll on slow sync"
> --
>
> Key: HBASE-12807
> URL: https://issues.apache.org/jira/browse/HBASE-12807
> Project: HBase
>  Issue Type: Sub-task
>  Components: wal
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
>
> As a first step towards bringing down wal latency, implement the version of 
> slow sync rolling done in the 0.89-fb branch.
> That is, instead of interrupting a slow sync operation, just request a log 
> roll after whenever we detect one.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-12621) Explain in the book when KEEP_DELETED_CELLS is useful

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-12621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-12621:
--
Fix Version/s: (was: 2.0.0)

> Explain in the book when KEEP_DELETED_CELLS is useful
> -
>
> Key: HBASE-12621
> URL: https://issues.apache.org/jira/browse/HBASE-12621
> Project: HBase
>  Issue Type: Sub-task
>  Components: documentation
>Reporter: Lars Hofhansl
>Priority: Major
>
> KEEP_DELETED_CELLS seems to be very confusing.
> The books need further clarification when this setting is useful and what the 
> implications are.
> (and maybe we should discuss if there's a simpler way to achieve what I 
> intended to achieve with this when I implemented it first)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-12629) Remove hbase.regionsizecalculator.enable from RegionSizeCalculator

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-12629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-12629:
--
Fix Version/s: (was: 2.0.0)

> Remove hbase.regionsizecalculator.enable from RegionSizeCalculator
> --
>
> Key: HBASE-12629
> URL: https://issues.apache.org/jira/browse/HBASE-12629
> Project: HBase
>  Issue Type: Improvement
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
>Priority: Major
> Attachments: 
> 0001-HBASE-12629-Remove-hbase.regionsizecalculator.enable.patch
>
>
> The RegionSizeCalculator has a option to disable it.  It is on by default and 
> end-to-end use with it disabled is not tested or used anywhere except for a 
> simple unit test.  This removes it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-12909) Junit listed at compile scope instead of test

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-12909:
--
Fix Version/s: (was: 2.0.0)

> Junit listed at compile scope instead of test
> -
>
> Key: HBASE-12909
> URL: https://issues.apache.org/jira/browse/HBASE-12909
> Project: HBase
>  Issue Type: Bug
>  Components: build
>Affects Versions: 1.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-12909.1.patch.txt
>
>
> Right now our top level pom lists junit as a dependency for every module in 
> the compile scope, which makes it subject to our compatibility promises.
> It should instead be test scope.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-12823) Visibility label security at limited localized level

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-12823:
--
Fix Version/s: (was: 2.0.0)

> Visibility label security at limited localized level
> 
>
> Key: HBASE-12823
> URL: https://issues.apache.org/jira/browse/HBASE-12823
> Project: HBase
>  Issue Type: Improvement
>  Components: security
>Affects Versions: 1.0.0, 2.0.0, 0.98.10
>Reporter: Jerry He
>Priority: Major
>
> Currently, if visibility label security is enabled for a HBase instance, 
> after VisibilityController is configured, the cell level visibility label 
> filtering will kick in across the HBase instance.
> Cell level visibility label filtering has non-negligible performance impact.
> On the other hand, in many use cases, only a small portion of the overall 
> data needs visibility label protection.
> If we can support  visibility label security at a limited and localized 
> level, we will broaden the use cases and the adoption of this feature.
> We should be able to support visibility label security at per table or per 
> column family level. This is quite common in many other HBase features.
> Cell level visibility label filtering will only be enabled and kick in for 
> the tables or column families that the user designates.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-13390) Document the implications of installing security coprocessors with hbase.security.authorization as false

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-13390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-13390:
--
Fix Version/s: (was: 2.0.0)

> Document the implications of installing security coprocessors with 
> hbase.security.authorization as false
> 
>
> Key: HBASE-13390
> URL: https://issues.apache.org/jira/browse/HBASE-13390
> Project: HBase
>  Issue Type: Sub-task
>  Components: documentation
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Major
>
> Add a new section to the security section of the online manual documenting 
> the implications of installing security coprocessors with 
> hbase.security.authorization set to false in site configuration. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-13354) Add documentation and tests for external block cache.

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-13354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-13354:
--
Fix Version/s: (was: 2.0.0)

> Add documentation and tests for external block cache.
> -
>
> Key: HBASE-13354
> URL: https://issues.apache.org/jira/browse/HBASE-13354
> Project: HBase
>  Issue Type: Bug
>  Components: BlockCache, documentation, test
>Affects Versions: 2.0.0, 1.1.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Major
>
> The new memcached integration needs some documentation and some testing 
> showing how it works and what can go wrong.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-13557) Special WAL handling for system tables

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-13557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-13557:
--
Fix Version/s: (was: 2.0.0)

> Special WAL handling for system tables
> --
>
> Key: HBASE-13557
> URL: https://issues.apache.org/jira/browse/HBASE-13557
> Project: HBase
>  Issue Type: Sub-task
>  Components: wal
>Affects Versions: 0.98.0
>Reporter: Sean Busbey
>Priority: Major
>
> We need to be able to recover system tables after meta and before user 
> tables, since they block things like setting a new active master.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)