[jira] [Commented] (HBASE-14885) NullPointerException in HMaster#normalizeRegions() due to missing TableDescriptor
[
https://issues.apache.org/jira/browse/HBASE-14885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028319#comment-15028319
]
Hudson commented on HBASE-14885:
SUCCESS: Integrated in HBase-1.2 #403 (See
[https://builds.apache.org/job/HBase-1.2/403/])
HBASE-14885 NullPointerException in HMaster#normalizeRegions() due to (tedyu:
rev 1e654241f20d4417290f643a8a80bd9be059a1f8)
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
> NullPointerException in HMaster#normalizeRegions() due to missing
> TableDescriptor
> -
>
> Key: HBASE-14885
> URL: https://issues.apache.org/jira/browse/HBASE-14885
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14885-branch-1.txt, 14885-v1.txt
>
>
> During system test on Windows, we observed the following in master log:
> {code}
> 2015-11-23 11:31:38,853 ERROR
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore: Caught
> error^M
> java.lang.NullPointerException^M
> at
> org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:1396)^M
> at
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:49)^M
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:185)^M
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)^M
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)^M
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)^M
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)^M
> at java.lang.Thread.run(Thread.java:745)^M
> {code}
> The NullPointerException came from the second line below:
> {code}
> for(TableName table : allEnabledTables) {
> if (table.isSystemTable() ||
> !getTableDescriptors().get(table).isNormalizationEnabled()) {
> {code}
> It seems TableDescriptor for some table was absent.
> normalizeRegions() should deal with such scenario without producing NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14355) Scan different TimeRange for each column family
[ https://issues.apache.org/jira/browse/HBASE-14355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028318#comment-15028318 ] Hudson commented on HBASE-14355: FAILURE: Integrated in HBase-1.2-IT #310 (See [https://builds.apache.org/job/HBase-1.2-IT/310/]) HBASE-14355 Scan different TimeRange for each column family - fixing (stack: rev 0cfd893e12ee56cf025d950e4163b00dff6985fc) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDefaultMemStore.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultMemStore.java > Scan different TimeRange for each column family > --- > > Key: HBASE-14355 > URL: https://issues.apache.org/jira/browse/HBASE-14355 > Project: HBase > Issue Type: New Feature > Components: Client, regionserver, Scanners >Reporter: Dave Latham >Assignee: churro morales > Fix For: 2.0.0, 1.3.0, 0.98.17 > > Attachments: HBASE-14355-addendum.patch, HBASE-14355-v1.patch, > HBASE-14355-v10.patch, HBASE-14355-v11.patch, HBASE-14355-v2.patch, > HBASE-14355-v3.patch, HBASE-14355-v4.patch, HBASE-14355-v5.patch, > HBASE-14355-v6.patch, HBASE-14355-v7.patch, HBASE-14355-v8.patch, > HBASE-14355-v9.patch, HBASE-14355.branch-1.patch, HBASE-14355.patch > > > At present the Scan API supports only table level time range. We have > specific use cases that will benefit from per column family time range. (See > background discussion at > https://mail-archives.apache.org/mod_mbox/hbase-user/201508.mbox/%3ccaa4mzom00ef5eoxstk0hetxeby8mqss61gbvgttgpaspmhq...@mail.gmail.com%3E) > There are a couple of choices that would be good to validate. First - how to > update the Scan API to support family and table level updates. One proposal > would be to add Scan.setTimeRange(byte family, long minTime, long maxTime), > then store it in a Map. When executing the scan, if a > family has a specified TimeRange, then use it, otherwise fall back to using > the table level TimeRange. Clients using the new API against old region > servers would not get the families correctly filterd. Old clients sending > scans to new region servers would work correctly. > The other question is how to get StoreFileScanner.shouldUseScanner to match > up the proper family and time range. It has the Scan available but doesn't > currently have available which family it is a part of. One option would be > to try to pass down the column family in each constructor path. Another > would be to instead alter shouldUseScanner to pass down the specific > TimeRange to use (similar to how it currently passes down the columns to use > which also appears to be a workaround for not having the family available). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14777) Fix Inter Cluster Replication Future ordering issues
[
https://issues.apache.org/jira/browse/HBASE-14777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028317#comment-15028317
]
Hudson commented on HBASE-14777:
FAILURE: Integrated in HBase-1.2-IT #310 (See
[https://builds.apache.org/job/HBase-1.2-IT/310/])
HBASE-14777 second addendum, better fix using CompletionService. (larsh: rev
89e34f36a4bf28c569e3b83757365c84dd1997b6)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HBaseInterClusterReplicationEndpoint.java
> Fix Inter Cluster Replication Future ordering issues
>
>
> Key: HBASE-14777
> URL: https://issues.apache.org/jira/browse/HBASE-14777
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Bhupendra Kumar Jain
>Assignee: Ashu Pachauri
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14777-alternative.txt, HBASE-14777-1.patch,
> HBASE-14777-2.patch, HBASE-14777-3.patch, HBASE-14777-4.patch,
> HBASE-14777-5.patch, HBASE-14777-6.patch, HBASE-14777-addendum.patch,
> HBASE-14777.patch
>
>
> Replication fails with IndexOutOfBoundsException
> {code}
> regionserver.ReplicationSource$ReplicationSourceWorkerThread(939):
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint
> threw unknown exception:java.lang.IndexOutOfBoundsException: Index: 1, Size:
> 1
> at java.util.ArrayList.rangeCheck(Unknown Source)
> at java.util.ArrayList.remove(Unknown Source)
> at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicate(HBaseInterClusterReplicationEndpoint.java:222)
> {code}
> Its happening due to incorrect removal of entries from the replication
> entries list.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen updated HBASE-14703:
--
Attachment: HBASE-14703_v9.patch
changes:
* remove Enable option in Client.proto
* fix some checkstyle errors.
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-14702_v5.2_addendum-addendum.patch,
> HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch,
> HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch,
> HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch,
> HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch,
> HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch,
> HBASE-14703_v6-addendum.patch, HBASE-14703_v6.patch, HBASE-14703_v7.patch,
> HBASE-14703_v8.patch, HBASE-14703_v9.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14888) ClusterSchema: Add Namespace Operations
[
https://issues.apache.org/jira/browse/HBASE-14888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028306#comment-15028306
]
Hadoop QA commented on HBASE-14888:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12774469/14888.patch
against master branch at commit 6531b465a757f8f6dad08d924c0a7a88bd52e5a9.
ATTACHMENT ID: 12774469
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 11 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
new checkstyle errors. Check build console for list of new errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestInterfaceAudienceAnnotations
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16676//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16676//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16676//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16676//console
This message is automatically generated.
> ClusterSchema: Add Namespace Operations
> ---
>
> Key: HBASE-14888
> URL: https://issues.apache.org/jira/browse/HBASE-14888
> Project: HBase
> Issue Type: Sub-task
> Components: API
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0
>
> Attachments: 14888.patch
>
>
> Add in a ClusterSchema Interface. It will have all API for all cluster
> manipulation; adding namespaces, tables, amending column families, etc. The
> idea is to gather up our mess and put it all behind a tidy API that all works
> the same way whatever you changing returning a Future to wait on and behind
> the scenes driving Procedures.
> This patch does namespace operations first.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14889) Region stuck in transition in OPEN state indefinitely in corner scenario
Abhishek Singh Chouhan created HBASE-14889:
--
Summary: Region stuck in transition in OPEN state indefinitely in
corner scenario
Key: HBASE-14889
URL: https://issues.apache.org/jira/browse/HBASE-14889
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.14
Reporter: Abhishek Singh Chouhan
During a failure scenario when a RS dies and the bulk assigner(BA) is assigning
its regions to others RSs, if another RS dies(on which some regions are being
moved) on which region is in pending open state, we end up in a situation where
two bulk assigners try to assign the same region on the Same RS.
The following happened -
1. While one BA was opening the region the second one sees it in pending open
state, retries and calls unassign(...) thereby sending CLOSE RPC to the RS.
2. The RS meanwhile has already opened the region, hence changing the znode
state to RS_ZK_REGION_OPENED which triggers event on master.
3. On master after the unassign is successful we go on to deleting the znode,
change region state to Pending open and send open RPC to RS.
4. The earlier triggered event now sees the state as Pending open and happily
changes it to OPEN, but is unable to delete the znode which by this time is not
in RS_ZK_REGION_OPENED state but is in M_ZK_REGION_OFFLINE state. Hence the
region remains in transition in the OPEN state.
5. RS goes on to changing the znode states and successfully opens the region
(changes znode state to RS_ZK_REGION_OPENED)
6. This again triggers event on master but this time since the state is OPEN
the folloing code path is taken
{noformat}
case RS_ZK_REGION_OPENED:
// Should see OPENED after OPENING but possible after PENDING_OPEN.
if (regionState == null
|| !regionState.isPendingOpenOrOpeningOnServer(sn)) {
LOG.warn("Received OPENED for " + prettyPrintedRegionName
+ " from " + sn + " but the region isn't PENDING_OPEN/OPENING
here: "
+ regionStates.getRegionState(encodedName));
if (regionState != null) {
// Close it without updating the internal region states,
// so as not to create double assignments in unlucky scenarios
// mentioned in OpenRegionHandler#process
unassign(regionState.getRegion(), null, -1, null, false, sn);
}
return;
}
{noformat}
We call unassign here with transitionInZK=false and state=null
7. RS closes the region but doesn't update the ZK, also state is not changed in
master. Region remains in transition in OPEN state, when its actually closed.
We have to restart the RS post which it opens correctly on some other RS.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028296#comment-15028296
]
Hadoop QA commented on HBASE-14703:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12774461/HBASE-14703_v8.patch
against master branch at commit e73a9594c218ed969a2f5b0b356d7b8d0e1474c0.
ATTACHMENT ID: 12774461
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 20 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
new checkstyle errors. Check build console for list of new errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+protected RpcRetryingCaller
createCaller(PayloadCarryingServerCallable callable) {
+ * configuration flag, so they don't have to use it, even if its sent.
This flag is only useful
+ * configuration flag, so they don't have to use it, even if its sent.
This flag is only useful
+ * configuration flag, so they don't have to use it, even if its sent.
This flag is only useful
+ * configuration flag, so they don't have to use it, even if its sent.
This flag is only useful
+ new java.lang.String[] { "MemstoreLoad", "HeapOccupancy",
"CompactionPressure", "Enabled", });
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16675//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16675//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16675//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16675//console
This message is automatically generated.
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-14702_v5.2_addendum-addendum.patch,
> HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch,
> HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch,
> HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch,
> HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch,
> HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch,
> HBASE-14703_v6-addendum.patch, HBASE-14703_v6.patch, HBASE-14703_v7.patch,
> HBASE-14703_v8.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connec
[jira] [Commented] (HBASE-14885) NullPointerException in HMaster#normalizeRegions() due to missing TableDescriptor
[
https://issues.apache.org/jira/browse/HBASE-14885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028285#comment-15028285
]
Hudson commented on HBASE-14885:
SUCCESS: Integrated in HBase-Trunk_matrix #503 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/503/])
HBASE-14885 NullPointerException in HMaster#normalizeRegions() due to (tedyu:
rev e73a9594c218ed969a2f5b0b356d7b8d0e1474c0)
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
> NullPointerException in HMaster#normalizeRegions() due to missing
> TableDescriptor
> -
>
> Key: HBASE-14885
> URL: https://issues.apache.org/jira/browse/HBASE-14885
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14885-branch-1.txt, 14885-v1.txt
>
>
> During system test on Windows, we observed the following in master log:
> {code}
> 2015-11-23 11:31:38,853 ERROR
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore: Caught
> error^M
> java.lang.NullPointerException^M
> at
> org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:1396)^M
> at
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:49)^M
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:185)^M
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)^M
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)^M
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)^M
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)^M
> at java.lang.Thread.run(Thread.java:745)^M
> {code}
> The NullPointerException came from the second line below:
> {code}
> for(TableName table : allEnabledTables) {
> if (table.isSystemTable() ||
> !getTableDescriptors().get(table).isNormalizationEnabled()) {
> {code}
> It seems TableDescriptor for some table was absent.
> normalizeRegions() should deal with such scenario without producing NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14355) Scan different TimeRange for each column family
[ https://issues.apache.org/jira/browse/HBASE-14355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028281#comment-15028281 ] Hudson commented on HBASE-14355: SUCCESS: Integrated in HBase-1.3-IT #341 (See [https://builds.apache.org/job/HBase-1.3-IT/341/]) HBASE-14355 Scan different TimeRange for each column family - fixing (stack: rev 9a11c025f2cba39baf472cb9959542f04eef) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultMemStore.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDefaultMemStore.java > Scan different TimeRange for each column family > --- > > Key: HBASE-14355 > URL: https://issues.apache.org/jira/browse/HBASE-14355 > Project: HBase > Issue Type: New Feature > Components: Client, regionserver, Scanners >Reporter: Dave Latham >Assignee: churro morales > Fix For: 2.0.0, 1.3.0, 0.98.17 > > Attachments: HBASE-14355-addendum.patch, HBASE-14355-v1.patch, > HBASE-14355-v10.patch, HBASE-14355-v11.patch, HBASE-14355-v2.patch, > HBASE-14355-v3.patch, HBASE-14355-v4.patch, HBASE-14355-v5.patch, > HBASE-14355-v6.patch, HBASE-14355-v7.patch, HBASE-14355-v8.patch, > HBASE-14355-v9.patch, HBASE-14355.branch-1.patch, HBASE-14355.patch > > > At present the Scan API supports only table level time range. We have > specific use cases that will benefit from per column family time range. (See > background discussion at > https://mail-archives.apache.org/mod_mbox/hbase-user/201508.mbox/%3ccaa4mzom00ef5eoxstk0hetxeby8mqss61gbvgttgpaspmhq...@mail.gmail.com%3E) > There are a couple of choices that would be good to validate. First - how to > update the Scan API to support family and table level updates. One proposal > would be to add Scan.setTimeRange(byte family, long minTime, long maxTime), > then store it in a Map. When executing the scan, if a > family has a specified TimeRange, then use it, otherwise fall back to using > the table level TimeRange. Clients using the new API against old region > servers would not get the families correctly filterd. Old clients sending > scans to new region servers would work correctly. > The other question is how to get StoreFileScanner.shouldUseScanner to match > up the proper family and time range. It has the Scan available but doesn't > currently have available which family it is a part of. One option would be > to try to pass down the column family in each constructor path. Another > would be to instead alter shouldUseScanner to pass down the specific > TimeRange to use (similar to how it currently passes down the columns to use > which also appears to be a workaround for not having the family available). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14876) Provide maven archetypes
[ https://issues.apache.org/jira/browse/HBASE-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028266#comment-15028266 ] Daniel Vimont commented on HBASE-14876: --- Besides wanting to get my hands dirty and start making some coding contributions to the HBase project, I've also been wanting to deepen my knowledge and understanding of Maven. While I'm not free to assign myself to this issue right at the moment, I could be available in a December/January timeframe to do some R&D and see what I could get going on this issue. With that said, if there are any Maven mavens around who want to deal with this more expeditiously, they should go for it now. > Provide maven archetypes > > > Key: HBASE-14876 > URL: https://issues.apache.org/jira/browse/HBASE-14876 > Project: HBase > Issue Type: New Feature > Components: build, Usability >Reporter: Nick Dimiduk > Labels: beginner > > To help onboard new users, we should provide maven archetypes for hbase > client applications. Off the top of my head, we should have templates for > - hbase client application with all dependencies > - hbase client application using client-shaded jar > - mapreduce application with hbase as input and output (ie, copy table) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14777) Fix Inter Cluster Replication Future ordering issues
[
https://issues.apache.org/jira/browse/HBASE-14777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028259#comment-15028259
]
Hudson commented on HBASE-14777:
FAILURE: Integrated in HBase-1.3-IT #340 (See
[https://builds.apache.org/job/HBase-1.3-IT/340/])
HBASE-14777 second addendum, better fix using CompletionService. (larsh: rev
dbbdb7b2e8fcf5f7b2cfc895fd6dfc8f85e4259c)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HBaseInterClusterReplicationEndpoint.java
> Fix Inter Cluster Replication Future ordering issues
>
>
> Key: HBASE-14777
> URL: https://issues.apache.org/jira/browse/HBASE-14777
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Bhupendra Kumar Jain
>Assignee: Ashu Pachauri
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14777-alternative.txt, HBASE-14777-1.patch,
> HBASE-14777-2.patch, HBASE-14777-3.patch, HBASE-14777-4.patch,
> HBASE-14777-5.patch, HBASE-14777-6.patch, HBASE-14777-addendum.patch,
> HBASE-14777.patch
>
>
> Replication fails with IndexOutOfBoundsException
> {code}
> regionserver.ReplicationSource$ReplicationSourceWorkerThread(939):
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint
> threw unknown exception:java.lang.IndexOutOfBoundsException: Index: 1, Size:
> 1
> at java.util.ArrayList.rangeCheck(Unknown Source)
> at java.util.ArrayList.remove(Unknown Source)
> at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicate(HBaseInterClusterReplicationEndpoint.java:222)
> {code}
> Its happening due to incorrect removal of entries from the replication
> entries list.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14734) TestGenerateDelegationToken fails with BindAddress clash
[ https://issues.apache.org/jira/browse/HBASE-14734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028254#comment-15028254 ] stack commented on HBASE-14734: --- bq. In any event it's out of our hands I think. Yeah. Head of the MiniKDC class says: bq. ...multiple MiniKdc instances cannot be started in parallel. For example, running testcases in parallel that start a KDC each. To accomplish this a single MiniKdc should be used for all testcases running It does just one test so maybe I can move the setup/teardown into the test and if a bind address issue, log a WARN but not fail the test, or try again with a new port myself. > TestGenerateDelegationToken fails with BindAddress clash > > > Key: HBASE-14734 > URL: https://issues.apache.org/jira/browse/HBASE-14734 > Project: HBase > Issue Type: Sub-task > Components: flakey, test >Reporter: stack > > From > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/330/jdk=latest1.7,label=Hadoop/testReport/junit/org.apache.hadoop.hbase.security.token/TestGenerateDelegationToken/org_apache_hadoop_hbase_security_token_TestGenerateDelegationToken/ > Error Message > Address already in use > Stacktrace > java.net.BindException: Address already in use > at sun.nio.ch.Net.bind0(Native Method) > at sun.nio.ch.Net.bind(Net.java:444) > at sun.nio.ch.Net.bind(Net.java:436) > at > sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214) > at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) > at > org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:198) > at > org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:51) > at > org.apache.mina.core.polling.AbstractPollingIoAcceptor.registerHandles(AbstractPollingIoAcceptor.java:547) > at > org.apache.mina.core.polling.AbstractPollingIoAcceptor.access$400(AbstractPollingIoAcceptor.java:68) > at > org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.run(AbstractPollingIoAcceptor.java:422) > at > org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Can this utility be made to not fail if address taken? Try another? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14872) Scan different timeRange per column family doesn't percolate down to the memstore
[ https://issues.apache.org/jira/browse/HBASE-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028245#comment-15028245 ] stack commented on HBASE-14872: --- I pushed to branch-1.2+. It does not go back into 0.98 cleanly. Take a look if you don't mind [~churromorales]? Thanks. > Scan different timeRange per column family doesn't percolate down to the > memstore > -- > > Key: HBASE-14872 > URL: https://issues.apache.org/jira/browse/HBASE-14872 > Project: HBase > Issue Type: Bug > Components: Client, regionserver, Scanners >Affects Versions: 2.0.0, 1.3.0 >Reporter: churro morales >Assignee: churro morales > Fix For: 2.0.0, 1.3.0, 0.98.17 > > Attachments: HBASE-14872-v1.patch, HBASE-14872.patch > > > HBASE-14355 The scan different time range for column family feature was not > applied to the memstore it was only done for the store files. This breaks > the contract. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14869) Better request latency histograms
[ https://issues.apache.org/jira/browse/HBASE-14869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028244#comment-15028244 ] Lars Hofhansl commented on HBASE-14869: --- Hmm... Looking at all the metrics, I also see metrics like: ResponseSize, RequestSize, AppendSize, etc. Those also get the ranges now, which doesn't make sense (the numbers mean bytes, not ms), in the end we'll want the same there (byte 90'iles cannot be combined either, but we'd want different ranges there I assume). So I might have to invent a new metric type for this. Something like MutableTimeHistogram. > Better request latency histograms > - > > Key: HBASE-14869 > URL: https://issues.apache.org/jira/browse/HBASE-14869 > Project: HBase > Issue Type: Brainstorming >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Attachments: 14869-test-0.98.txt, 14869-v1-0.98.txt > > > I just discussed this with a colleague. > The get, put, etc, histograms that each region server keeps are somewhat > useless (depending on what you want to achieve of course), as they are > aggregated and calculated by each region server. > It would be better to record the number of requests in certainly latency > bands in addition to what we do now. > For example the number of gets that took 0-5ms, 6-10ms, 10-20ms, 20-50ms, > 50-100ms, 100-1000ms, > 1000ms, etc. (just as an example, should be > configurable). > That way we can do further calculations after the fact, and answer questions > like: How often did we miss our SLA? Percentage of requests that missed an > SLA, etc. > Comments? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14884) TestSplitTransactionOnCluster.testSSHCleanupDaugtherRegionsOfAbortedSplit is flakey
[
https://issues.apache.org/jira/browse/HBASE-14884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028239#comment-15028239
]
Heng Chen commented on HBASE-14884:
---
We should ensure server is dead on master side,
{{cluster.getRegionServerThreads().get(serverWith).join()}} is not enough.
> TestSplitTransactionOnCluster.testSSHCleanupDaugtherRegionsOfAbortedSplit is
> flakey
> ---
>
> Key: HBASE-14884
> URL: https://issues.apache.org/jira/browse/HBASE-14884
> Project: HBase
> Issue Type: Sub-task
> Components: flakey, test
>Reporter: stack
> Attachments: HBASE-14884.patch
>
>
> This one fails a bit on master at least:
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Trunk_matrix/496/jdk=latest1.8,label=Hadoop/testReport/junit/org.apache.hadoop.hbase.regionserver/TestSplitTransactionOnCluster/testSSHCleanupDaugtherRegionsOfAbortedSplit/history/
> ... about 15% of the time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14884) TestSplitTransactionOnCluster.testSSHCleanupDaugtherRegionsOfAbortedSplit is flakey
[ https://issues.apache.org/jira/browse/HBASE-14884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen updated HBASE-14884: -- Status: Patch Available (was: Open) > TestSplitTransactionOnCluster.testSSHCleanupDaugtherRegionsOfAbortedSplit is > flakey > --- > > Key: HBASE-14884 > URL: https://issues.apache.org/jira/browse/HBASE-14884 > Project: HBase > Issue Type: Sub-task > Components: flakey, test >Reporter: stack > Attachments: HBASE-14884.patch > > > This one fails a bit on master at least: > https://builds.apache.org/view/H-L/view/HBase/job/HBase-Trunk_matrix/496/jdk=latest1.8,label=Hadoop/testReport/junit/org.apache.hadoop.hbase.regionserver/TestSplitTransactionOnCluster/testSSHCleanupDaugtherRegionsOfAbortedSplit/history/ > ... about 15% of the time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14884) TestSplitTransactionOnCluster.testSSHCleanupDaugtherRegionsOfAbortedSplit is flakey
[ https://issues.apache.org/jira/browse/HBASE-14884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen updated HBASE-14884: -- Attachment: HBASE-14884.patch > TestSplitTransactionOnCluster.testSSHCleanupDaugtherRegionsOfAbortedSplit is > flakey > --- > > Key: HBASE-14884 > URL: https://issues.apache.org/jira/browse/HBASE-14884 > Project: HBase > Issue Type: Sub-task > Components: flakey, test >Reporter: stack > Attachments: HBASE-14884.patch > > > This one fails a bit on master at least: > https://builds.apache.org/view/H-L/view/HBase/job/HBase-Trunk_matrix/496/jdk=latest1.8,label=Hadoop/testReport/junit/org.apache.hadoop.hbase.regionserver/TestSplitTransactionOnCluster/testSSHCleanupDaugtherRegionsOfAbortedSplit/history/ > ... about 15% of the time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14888) ClusterSchema: Add Namespace Operations
[ https://issues.apache.org/jira/browse/HBASE-14888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14888: -- Status: Patch Available (was: Open) See what I broke. > ClusterSchema: Add Namespace Operations > --- > > Key: HBASE-14888 > URL: https://issues.apache.org/jira/browse/HBASE-14888 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: 14888.patch > > > Add in a ClusterSchema Interface. It will have all API for all cluster > manipulation; adding namespaces, tables, amending column families, etc. The > idea is to gather up our mess and put it all behind a tidy API that all works > the same way whatever you changing returning a Future to wait on and behind > the scenes driving Procedures. > This patch does namespace operations first. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14885) NullPointerException in HMaster#normalizeRegions() due to missing TableDescriptor
[
https://issues.apache.org/jira/browse/HBASE-14885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028219#comment-15028219
]
Hudson commented on HBASE-14885:
SUCCESS: Integrated in HBase-1.2-IT #309 (See
[https://builds.apache.org/job/HBase-1.2-IT/309/])
HBASE-14885 NullPointerException in HMaster#normalizeRegions() due to (tedyu:
rev 1e654241f20d4417290f643a8a80bd9be059a1f8)
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
> NullPointerException in HMaster#normalizeRegions() due to missing
> TableDescriptor
> -
>
> Key: HBASE-14885
> URL: https://issues.apache.org/jira/browse/HBASE-14885
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14885-branch-1.txt, 14885-v1.txt
>
>
> During system test on Windows, we observed the following in master log:
> {code}
> 2015-11-23 11:31:38,853 ERROR
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore: Caught
> error^M
> java.lang.NullPointerException^M
> at
> org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:1396)^M
> at
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:49)^M
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:185)^M
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)^M
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)^M
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)^M
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)^M
> at java.lang.Thread.run(Thread.java:745)^M
> {code}
> The NullPointerException came from the second line below:
> {code}
> for(TableName table : allEnabledTables) {
> if (table.isSystemTable() ||
> !getTableDescriptors().get(table).isNormalizationEnabled()) {
> {code}
> It seems TableDescriptor for some table was absent.
> normalizeRegions() should deal with such scenario without producing NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-14869) Better request latency histograms
[ https://issues.apache.org/jira/browse/HBASE-14869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl reassigned HBASE-14869: - Assignee: Lars Hofhansl > Better request latency histograms > - > > Key: HBASE-14869 > URL: https://issues.apache.org/jira/browse/HBASE-14869 > Project: HBase > Issue Type: Brainstorming >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Attachments: 14869-test-0.98.txt, 14869-v1-0.98.txt > > > I just discussed this with a colleague. > The get, put, etc, histograms that each region server keeps are somewhat > useless (depending on what you want to achieve of course), as they are > aggregated and calculated by each region server. > It would be better to record the number of requests in certainly latency > bands in addition to what we do now. > For example the number of gets that took 0-5ms, 6-10ms, 10-20ms, 20-50ms, > 50-100ms, 100-1000ms, > 1000ms, etc. (just as an example, should be > configurable). > That way we can do further calculations after the fact, and answer questions > like: How often did we miss our SLA? Percentage of requests that missed an > SLA, etc. > Comments? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14869) Better request latency histograms
[ https://issues.apache.org/jira/browse/HBASE-14869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-14869: -- Attachment: 14869-v1-0.98.txt Here's a patch that I tested a bit. Reports the right values. Changes: * Renamed "bands" to "ranges". * Does not report values for range for it hasn't seen a value. That allows us to use this for operations that take very short (few ms) or very long (many minutes) times, without reporting time ranges that make no sense for the operation. Question: How should I name these metrics? Presumably they'd be processed mostly by software, and I have to give them some name. I chose: "metricname"_start-end, i.e."Get_0-1", "Get_1-3", "Get_10-30" , etc, and "Get_>60". Any better ideas? > Better request latency histograms > - > > Key: HBASE-14869 > URL: https://issues.apache.org/jira/browse/HBASE-14869 > Project: HBase > Issue Type: Brainstorming >Reporter: Lars Hofhansl > Attachments: 14869-test-0.98.txt, 14869-v1-0.98.txt > > > I just discussed this with a colleague. > The get, put, etc, histograms that each region server keeps are somewhat > useless (depending on what you want to achieve of course), as they are > aggregated and calculated by each region server. > It would be better to record the number of requests in certainly latency > bands in addition to what we do now. > For example the number of gets that took 0-5ms, 6-10ms, 10-20ms, 20-50ms, > 50-100ms, 100-1000ms, > 1000ms, etc. (just as an example, should be > configurable). > That way we can do further calculations after the fact, and answer questions > like: How often did we miss our SLA? Percentage of requests that missed an > SLA, etc. > Comments? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14888) ClusterSchema: Add Namespace Operations
[
https://issues.apache.org/jira/browse/HBASE-14888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-14888:
--
Attachment: 14888.patch
{code}
Add in a ClusterSchema Interface. It will have all API for all cluster
manipulation; adding namespaces, tables, amending column
families, etc. The idea is to gather up our mess and put it all behind a tidy
API that all works the same way returning a Future to wait on.
This patch does namespace operations first.
M
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
Just remove a hyphen from thread name. We were doubling up.
A hbase-common/src/main/java/org/apache/hadoop/hbase/Service.java
A very basic 'Service' Interface. To be enhanced later or replaced by
the guava Service (our guava is too old currently and its Service is not
as )
M
hbase-common/src/main/java/org/apache/hadoop/hbase/ServiceNotRunningException.java
Moved this back to hbase-common so coul use it in new Service Interface.
A hbase-server/src/main/java/org/apache/hadoop/hbase/master/ClusterSchema.java
New ClusterSchema Interface. Has namespace API only and some utility as
well as a low-level API to fetch namespace table manager that probably
belongs elsewhere.
A
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ClusterSchemaException.java
A ClusterSchemeException. I don't think this is really needed so will
probably remove in later patch.
A
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ClusterSchemaService.java
Interface that is made of ClusterSchema and Service
A
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ClusterSchemaServiceImpl.java
Implementation. Moved stuff here from Master.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Mix in ClusterSchema for namespace ops. All that remains in here is pre
and post coprocessor calls. Some cleanup around namespace init; make it
less complicated. Namespace ops don't have to be public anymore.
M
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java
Use new ClusterSchema when doing namespace ops.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java
Remove namespace particular ops and add getting of a ClusterSchema
instance instead.
A hbase-server/src/main/java/org/apache/hadoop/hbase/master/ProcedureFuture.java
Map procedure processing to a Future model. Probably belongs over in
procedure space. Can move later if generally useful.
M
hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableNamespaceManager.java
Cleanup
{code}
> ClusterSchema: Add Namespace Operations
> ---
>
> Key: HBASE-14888
> URL: https://issues.apache.org/jira/browse/HBASE-14888
> Project: HBase
> Issue Type: Sub-task
> Components: API
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0
>
> Attachments: 14888.patch
>
>
> Add in a ClusterSchema Interface. It will have all API for all cluster
> manipulation; adding namespaces, tables, amending column families, etc. The
> idea is to gather up our mess and put it all behind a tidy API that all works
> the same way whatever you changing returning a Future to wait on and behind
> the scenes driving Procedures.
> This patch does namespace operations first.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14888) ClusterSchema: Add Namespace Operations
stack created HBASE-14888: - Summary: ClusterSchema: Add Namespace Operations Key: HBASE-14888 URL: https://issues.apache.org/jira/browse/HBASE-14888 Project: HBase Issue Type: Sub-task Components: API Reporter: stack Assignee: stack Add in a ClusterSchema Interface. It will have all API for all cluster manipulation; adding namespaces, tables, amending column families, etc. The idea is to gather up our mess and put it all behind a tidy API that all works the same way whatever you changing returning a Future to wait on and behind the scenes driving Procedures. This patch does namespace operations first. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14887) Use Orientated Abstractions
stack created HBASE-14887: - Summary: Use Orientated Abstractions Key: HBASE-14887 URL: https://issues.apache.org/jira/browse/HBASE-14887 Project: HBase Issue Type: Umbrella Components: API Affects Versions: 2.0.0 Reporter: stack Fix For: 2.0.0 This umbrella issue is about adding clean server-side "user orientated" internal APIs that are about 'doing things' (as [~benlau] says over in HBASE-14439 "New/Improved Filesystem Abstractions") such as creating and deleting tables, etc. Initially, we'll retrofit in a ClusterSchema Interface/API that cleans up Master internals and abstracts away implementation detail around the manipulation of namespace and tables. This project operates at a level above the undertaking described in HBASE-7806 "Isolate the FileSystem calls" and related issues, but has the same intent of making it so we can the more easily swap in alternate implementations and representations; e.g. changing filesystem layout under the covers. It is a complement to HBASE-7806 and related lower-level undertakings (look out for the gold spike when this top-down vector crosses the bottom-up endeavor). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14866) VerifyReplication should use peer configuration in peer connection
[
https://issues.apache.org/jira/browse/HBASE-14866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028195#comment-15028195
]
Hadoop QA commented on HBASE-14866:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12774438/hbase-14866_v3.patch
against master branch at commit e73a9594c218ed969a2f5b0b356d7b8d0e1474c0.
ATTACHMENT ID: 12774438
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
new checkstyle errors. Check build console for list of new errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16673//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16673//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16673//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16673//console
This message is automatically generated.
> VerifyReplication should use peer configuration in peer connection
> --
>
> Key: HBASE-14866
> URL: https://issues.apache.org/jira/browse/HBASE-14866
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14866.patch, HBASE-14866_v1.patch,
> hbase-14866_v2.patch, hbase-14866_v3.patch
>
>
> VerifyReplication uses the replication peer's configuration to construct the
> ZooKeeper quorum address for the peer connection. However, other
> configuration properties in the peer's configuration are dropped. It should
> merge all configuration properties from the {{ReplicationPeerConfig}} when
> creating the peer connection and obtaining a credentials for the peer cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-14777) Fix Inter Cluster Replication Future ordering issues
[
https://issues.apache.org/jira/browse/HBASE-14777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl resolved HBASE-14777.
---
Resolution: Fixed
Pushed to all branches. Thanks for bearing with me.
> Fix Inter Cluster Replication Future ordering issues
>
>
> Key: HBASE-14777
> URL: https://issues.apache.org/jira/browse/HBASE-14777
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Bhupendra Kumar Jain
>Assignee: Ashu Pachauri
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14777-alternative.txt, HBASE-14777-1.patch,
> HBASE-14777-2.patch, HBASE-14777-3.patch, HBASE-14777-4.patch,
> HBASE-14777-5.patch, HBASE-14777-6.patch, HBASE-14777-addendum.patch,
> HBASE-14777.patch
>
>
> Replication fails with IndexOutOfBoundsException
> {code}
> regionserver.ReplicationSource$ReplicationSourceWorkerThread(939):
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint
> threw unknown exception:java.lang.IndexOutOfBoundsException: Index: 1, Size:
> 1
> at java.util.ArrayList.rangeCheck(Unknown Source)
> at java.util.ArrayList.remove(Unknown Source)
> at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicate(HBaseInterClusterReplicationEndpoint.java:222)
> {code}
> Its happening due to incorrect removal of entries from the replication
> entries list.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14777) Fix Inter Cluster Replication Future ordering issues
[
https://issues.apache.org/jira/browse/HBASE-14777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028183#comment-15028183
]
Lars Hofhansl commented on HBASE-14777:
---
Cool... I'll commit this as an addendum to 1.2, 1.3, 2.0.
> Fix Inter Cluster Replication Future ordering issues
>
>
> Key: HBASE-14777
> URL: https://issues.apache.org/jira/browse/HBASE-14777
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Bhupendra Kumar Jain
>Assignee: Ashu Pachauri
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14777-alternative.txt, HBASE-14777-1.patch,
> HBASE-14777-2.patch, HBASE-14777-3.patch, HBASE-14777-4.patch,
> HBASE-14777-5.patch, HBASE-14777-6.patch, HBASE-14777-addendum.patch,
> HBASE-14777.patch
>
>
> Replication fails with IndexOutOfBoundsException
> {code}
> regionserver.ReplicationSource$ReplicationSourceWorkerThread(939):
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint
> threw unknown exception:java.lang.IndexOutOfBoundsException: Index: 1, Size:
> 1
> at java.util.ArrayList.rangeCheck(Unknown Source)
> at java.util.ArrayList.remove(Unknown Source)
> at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicate(HBaseInterClusterReplicationEndpoint.java:222)
> {code}
> Its happening due to incorrect removal of entries from the replication
> entries list.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14885) NullPointerException in HMaster#normalizeRegions() due to missing TableDescriptor
[
https://issues.apache.org/jira/browse/HBASE-14885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028178#comment-15028178
]
Hudson commented on HBASE-14885:
FAILURE: Integrated in HBase-1.3-IT #339 (See
[https://builds.apache.org/job/HBase-1.3-IT/339/])
HBASE-14885 NullPointerException in HMaster#normalizeRegions() due to (tedyu:
rev 06c233834f1d0e2fac3659d3d243641cabf72961)
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
> NullPointerException in HMaster#normalizeRegions() due to missing
> TableDescriptor
> -
>
> Key: HBASE-14885
> URL: https://issues.apache.org/jira/browse/HBASE-14885
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14885-branch-1.txt, 14885-v1.txt
>
>
> During system test on Windows, we observed the following in master log:
> {code}
> 2015-11-23 11:31:38,853 ERROR
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore: Caught
> error^M
> java.lang.NullPointerException^M
> at
> org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:1396)^M
> at
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:49)^M
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:185)^M
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)^M
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)^M
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)^M
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)^M
> at java.lang.Thread.run(Thread.java:745)^M
> {code}
> The NullPointerException came from the second line below:
> {code}
> for(TableName table : allEnabledTables) {
> if (table.isSystemTable() ||
> !getTableDescriptors().get(table).isNormalizationEnabled()) {
> {code}
> It seems TableDescriptor for some table was absent.
> normalizeRegions() should deal with such scenario without producing NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14866) VerifyReplication should use peer configuration in peer connection
[
https://issues.apache.org/jira/browse/HBASE-14866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028138#comment-15028138
]
Heng Chen commented on HBASE-14866:
---
Should we extract one method to generate peerConf, it includes set clusterKey
and configuration map.
I see in HBASE-14886, you do the same thing.
> VerifyReplication should use peer configuration in peer connection
> --
>
> Key: HBASE-14866
> URL: https://issues.apache.org/jira/browse/HBASE-14866
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14866.patch, HBASE-14866_v1.patch,
> hbase-14866_v2.patch, hbase-14866_v3.patch
>
>
> VerifyReplication uses the replication peer's configuration to construct the
> ZooKeeper quorum address for the peer connection. However, other
> configuration properties in the peer's configuration are dropped. It should
> merge all configuration properties from the {{ReplicationPeerConfig}} when
> creating the peer connection and obtaining a credentials for the peer cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-14886) ReplicationAdmin does not use full peer configuration
[
https://issues.apache.org/jira/browse/HBASE-14886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028126#comment-15028126
]
Heng Chen edited comment on HBASE-14886 at 11/26/15 4:11 AM:
-
btw.
As elliott said, should we move applyClusterKeyToConf into
ReplicationPeersZKImpl and make it private.
Each place which ZKUtil.applyClusterKeyToConf is used should be replaced.
was (Author: chenheng):
btw.
As elliott said, should we move applyClusterKeyToConf into
ReplicationPeersZKImpl and make it to private.
Each place which ZKUtil.applyClusterKeyToConf is used should be replaced.
> ReplicationAdmin does not use full peer configuration
> -
>
> Key: HBASE-14886
> URL: https://issues.apache.org/jira/browse/HBASE-14886
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: hbase-14886-v1.patch, hbase-14886.patch
>
>
> In {{listValidReplicationPeers()}}, we're creating the peer {{Configuration}}
> based on the source connection configuration and simply applying the peer ZK
> cluster key. This causes any additional properties present in the
> {{ReplicationPeerConfig}} configuration to not be applied.
> We should instead be using the configuration returned by
> {{ReplicationPeers.getPeerConf()}}, which we already call in that method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-14886) ReplicationAdmin does not use full peer configuration
[
https://issues.apache.org/jira/browse/HBASE-14886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028126#comment-15028126
]
Heng Chen edited comment on HBASE-14886 at 11/26/15 4:11 AM:
-
btw.
As elliott said, should we move applyClusterKeyToConf into
ReplicationPeersZKImpl and make it to private.
Each place which ZKUtil.applyClusterKeyToConf is used should be replaced.
was (Author: chenheng):
btw.
As elliott said, should we move applyClusterKeyToConf into
ReplicationPeersZKImpl and make it to private.
Each place which ZKUtil.applyClusterKeyToConf should be replaced.
> ReplicationAdmin does not use full peer configuration
> -
>
> Key: HBASE-14886
> URL: https://issues.apache.org/jira/browse/HBASE-14886
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: hbase-14886-v1.patch, hbase-14886.patch
>
>
> In {{listValidReplicationPeers()}}, we're creating the peer {{Configuration}}
> based on the source connection configuration and simply applying the peer ZK
> cluster key. This causes any additional properties present in the
> {{ReplicationPeerConfig}} configuration to not be applied.
> We should instead be using the configuration returned by
> {{ReplicationPeers.getPeerConf()}}, which we already call in that method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14886) ReplicationAdmin does not use full peer configuration
[
https://issues.apache.org/jira/browse/HBASE-14886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028126#comment-15028126
]
Heng Chen commented on HBASE-14886:
---
btw.
As elliott said, should we move applyClusterKeyToConf into
ReplicationPeersZKImpl and make it to private.
Each place which ZKUtil.applyClusterKeyToConf should be replaced.
> ReplicationAdmin does not use full peer configuration
> -
>
> Key: HBASE-14886
> URL: https://issues.apache.org/jira/browse/HBASE-14886
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: hbase-14886-v1.patch, hbase-14886.patch
>
>
> In {{listValidReplicationPeers()}}, we're creating the peer {{Configuration}}
> based on the source connection configuration and simply applying the peer ZK
> cluster key. This causes any additional properties present in the
> {{ReplicationPeerConfig}} configuration to not be applied.
> We should instead be using the configuration returned by
> {{ReplicationPeers.getPeerConf()}}, which we already call in that method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14886) ReplicationAdmin does not use full peer configuration
[
https://issues.apache.org/jira/browse/HBASE-14886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028120#comment-15028120
]
Heng Chen commented on HBASE-14886:
---
LGTM. +1
> ReplicationAdmin does not use full peer configuration
> -
>
> Key: HBASE-14886
> URL: https://issues.apache.org/jira/browse/HBASE-14886
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: hbase-14886-v1.patch, hbase-14886.patch
>
>
> In {{listValidReplicationPeers()}}, we're creating the peer {{Configuration}}
> based on the source connection configuration and simply applying the peer ZK
> cluster key. This causes any additional properties present in the
> {{ReplicationPeerConfig}} configuration to not be applied.
> We should instead be using the configuration returned by
> {{ReplicationPeers.getPeerConf()}}, which we already call in that method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Issue Comment Deleted] (HBASE-14886) ReplicationAdmin does not use full peer configuration
[
https://issues.apache.org/jira/browse/HBASE-14886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen updated HBASE-14886:
--
Comment: was deleted
(was: ReplicationPeerZKImpl is not used to create peer, peer was created in
ReplicationPeersZKImpl.
I am not sure it is a bug.
Thanks [~ghelmling]. :))
> ReplicationAdmin does not use full peer configuration
> -
>
> Key: HBASE-14886
> URL: https://issues.apache.org/jira/browse/HBASE-14886
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: hbase-14886-v1.patch, hbase-14886.patch
>
>
> In {{listValidReplicationPeers()}}, we're creating the peer {{Configuration}}
> based on the source connection configuration and simply applying the peer ZK
> cluster key. This causes any additional properties present in the
> {{ReplicationPeerConfig}} configuration to not be applied.
> We should instead be using the configuration returned by
> {{ReplicationPeers.getPeerConf()}}, which we already call in that method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14885) NullPointerException in HMaster#normalizeRegions() due to missing TableDescriptor
[
https://issues.apache.org/jira/browse/HBASE-14885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028115#comment-15028115
]
Hudson commented on HBASE-14885:
SUCCESS: Integrated in HBase-1.3 #399 (See
[https://builds.apache.org/job/HBase-1.3/399/])
HBASE-14885 NullPointerException in HMaster#normalizeRegions() due to (tedyu:
rev 06c233834f1d0e2fac3659d3d243641cabf72961)
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
> NullPointerException in HMaster#normalizeRegions() due to missing
> TableDescriptor
> -
>
> Key: HBASE-14885
> URL: https://issues.apache.org/jira/browse/HBASE-14885
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14885-branch-1.txt, 14885-v1.txt
>
>
> During system test on Windows, we observed the following in master log:
> {code}
> 2015-11-23 11:31:38,853 ERROR
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore: Caught
> error^M
> java.lang.NullPointerException^M
> at
> org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:1396)^M
> at
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:49)^M
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:185)^M
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)^M
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)^M
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)^M
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)^M
> at java.lang.Thread.run(Thread.java:745)^M
> {code}
> The NullPointerException came from the second line below:
> {code}
> for(TableName table : allEnabledTables) {
> if (table.isSystemTable() ||
> !getTableDescriptors().get(table).isNormalizationEnabled()) {
> {code}
> It seems TableDescriptor for some table was absent.
> normalizeRegions() should deal with such scenario without producing NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-14886) ReplicationAdmin does not use full peer configuration
[
https://issues.apache.org/jira/browse/HBASE-14886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028101#comment-15028101
]
Heng Chen edited comment on HBASE-14886 at 11/26/15 3:41 AM:
-
ReplicationPeerZKImpl is not used to create peer, peer was created in
ReplicationPeersZKImpl.
I am not sure it is a bug.
Thanks [~ghelmling]. :)
was (Author: chenheng):
ReplicationPeerZKImpl is not used to create peer, peer was created in
ReplicationPeersZKImpl.
And the configuration in ReplicationPeerZKImpl is useless currently. So it is
not a bug.
The code here is some confused. And it has some changes in HBASE-11393
Thanks [~ghelmling]. :)
> ReplicationAdmin does not use full peer configuration
> -
>
> Key: HBASE-14886
> URL: https://issues.apache.org/jira/browse/HBASE-14886
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: hbase-14886-v1.patch, hbase-14886.patch
>
>
> In {{listValidReplicationPeers()}}, we're creating the peer {{Configuration}}
> based on the source connection configuration and simply applying the peer ZK
> cluster key. This causes any additional properties present in the
> {{ReplicationPeerConfig}} configuration to not be applied.
> We should instead be using the configuration returned by
> {{ReplicationPeers.getPeerConf()}}, which we already call in that method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14886) ReplicationAdmin does not use full peer configuration
[
https://issues.apache.org/jira/browse/HBASE-14886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028101#comment-15028101
]
Heng Chen commented on HBASE-14886:
---
ReplicationPeerZKImpl is not used to create peer, peer was created in
ReplicationPeersZKImpl.
And the configuration in ReplicationPeerZKImpl is useless currently. So it is
not a bug.
The code here is some confused. And it has some changes in HBASE-11393
Thanks [~ghelmling]. :)
> ReplicationAdmin does not use full peer configuration
> -
>
> Key: HBASE-14886
> URL: https://issues.apache.org/jira/browse/HBASE-14886
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: hbase-14886-v1.patch, hbase-14886.patch
>
>
> In {{listValidReplicationPeers()}}, we're creating the peer {{Configuration}}
> based on the source connection configuration and simply applying the peer ZK
> cluster key. This causes any additional properties present in the
> {{ReplicationPeerConfig}} configuration to not be applied.
> We should instead be using the configuration returned by
> {{ReplicationPeers.getPeerConf()}}, which we already call in that method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14872) Scan different timeRange per column family doesn't percolate down to the memstore
[
https://issues.apache.org/jira/browse/HBASE-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028088#comment-15028088
]
Hadoop QA commented on HBASE-14872:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12774434/HBASE-14872-v1.patch
against master branch at commit e73a9594c218ed969a2f5b0b356d7b8d0e1474c0.
ATTACHMENT ID: 12774434
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
new checkstyle errors. Check build console for list of new errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16672//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16672//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16672//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16672//console
This message is automatically generated.
> Scan different timeRange per column family doesn't percolate down to the
> memstore
> --
>
> Key: HBASE-14872
> URL: https://issues.apache.org/jira/browse/HBASE-14872
> Project: HBase
> Issue Type: Bug
> Components: Client, regionserver, Scanners
>Affects Versions: 2.0.0, 1.3.0
>Reporter: churro morales
>Assignee: churro morales
> Fix For: 2.0.0, 1.3.0, 0.98.17
>
> Attachments: HBASE-14872-v1.patch, HBASE-14872.patch
>
>
> HBASE-14355 The scan different time range for column family feature was not
> applied to the memstore it was only done for the store files. This breaks
> the contract.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen updated HBASE-14703:
--
Attachment: HBASE-14703_v8.patch
changes:
* remove AbstractMultiCallable, just left repeated code in HTable, we will
fix it in future.
* fix some checkstyle errors.
Thanks [~jesse_yates] for your nice review.
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-14702_v5.2_addendum-addendum.patch,
> HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch,
> HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch,
> HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch,
> HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch,
> HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch,
> HBASE-14703_v6-addendum.patch, HBASE-14703_v6.patch, HBASE-14703_v7.patch,
> HBASE-14703_v8.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14885) NullPointerException in HMaster#normalizeRegions() due to missing TableDescriptor
[
https://issues.apache.org/jira/browse/HBASE-14885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028082#comment-15028082
]
Hadoop QA commented on HBASE-14885:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12774432/14885-branch-1.txt
against branch-1 branch at commit efb5917f2247b4cc1951ba950a9b9ff1a4606c92.
ATTACHMENT ID: 12774432
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16671//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16671//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16671//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16671//console
This message is automatically generated.
> NullPointerException in HMaster#normalizeRegions() due to missing
> TableDescriptor
> -
>
> Key: HBASE-14885
> URL: https://issues.apache.org/jira/browse/HBASE-14885
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14885-branch-1.txt, 14885-v1.txt
>
>
> During system test on Windows, we observed the following in master log:
> {code}
> 2015-11-23 11:31:38,853 ERROR
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore: Caught
> error^M
> java.lang.NullPointerException^M
> at
> org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:1396)^M
> at
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:49)^M
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:185)^M
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)^M
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)^M
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)^M
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)^M
> at java.lang.Thread.run(Thread.java:745)^M
> {code}
> The NullPointerException came from the second line below:
> {code}
> for(TableName table : allEnabledTables) {
> if (table.isSystemTable() ||
> !getTableDescriptors().get(table).isNormalizationEnabled()) {
> {code}
> It seems TableDescriptor for some table was absent.
> normalizeRegions() should deal with such scenario without producing NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14886) ReplicationAdmin does not use full peer configuration
[
https://issues.apache.org/jira/browse/HBASE-14886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028060#comment-15028060
]
Hadoop QA commented on HBASE-14886:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12774429/hbase-14886-v1.patch
against master branch at commit efb5917f2247b4cc1951ba950a9b9ff1a4606c92.
ATTACHMENT ID: 12774429
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
new checkstyle errors. Check build console for list of new errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16670//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16670//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16670//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16670//console
This message is automatically generated.
> ReplicationAdmin does not use full peer configuration
> -
>
> Key: HBASE-14886
> URL: https://issues.apache.org/jira/browse/HBASE-14886
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: hbase-14886-v1.patch, hbase-14886.patch
>
>
> In {{listValidReplicationPeers()}}, we're creating the peer {{Configuration}}
> based on the source connection configuration and simply applying the peer ZK
> cluster key. This causes any additional properties present in the
> {{ReplicationPeerConfig}} configuration to not be applied.
> We should instead be using the configuration returned by
> {{ReplicationPeers.getPeerConf()}}, which we already call in that method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14730) region server needs to log warnings when there are attributes configured for cells with hfile v2
[
https://issues.apache.org/jira/browse/HBASE-14730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028059#comment-15028059
]
Hadoop QA commented on HBASE-14730:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12774446/HBASE-14730-v001.patch
against master branch at commit e73a9594c218ed969a2f5b0b356d7b8d0e1474c0.
ATTACHMENT ID: 12774446
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16674//console
This message is automatically generated.
> region server needs to log warnings when there are attributes configured for
> cells with hfile v2
>
>
> Key: HBASE-14730
> URL: https://issues.apache.org/jira/browse/HBASE-14730
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Affects Versions: 1.0.0, 1.1.0, 1.2.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
> Attachments: HBASE-14730-v001.patch
>
>
> User can configure cell attributes with hfile.format.version 2. When cells
> are flushed from memStore to Hfile, attributes are not saved. Warnings need
> to be logged.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14876) Provide maven archetypes
[ https://issues.apache.org/jira/browse/HBASE-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028057#comment-15028057 ] Nick Dimiduk commented on HBASE-14876: -- Published via maven repository just like the rest of our jars. Unless I miss-understand how archetypes work... > Provide maven archetypes > > > Key: HBASE-14876 > URL: https://issues.apache.org/jira/browse/HBASE-14876 > Project: HBase > Issue Type: New Feature > Components: build, Usability >Reporter: Nick Dimiduk > Labels: beginner > > To help onboard new users, we should provide maven archetypes for hbase > client applications. Off the top of my head, we should have templates for > - hbase client application with all dependencies > - hbase client application using client-shaded jar > - mapreduce application with hbase as input and output (ie, copy table) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14859) Better checkstyle reporting
[
https://issues.apache.org/jira/browse/HBASE-14859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028022#comment-15028022
]
Hudson commented on HBASE-14859:
FAILURE: Integrated in HBase-Trunk_matrix #502 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/502/])
HBASE-14859 Better checkstyle reporting. Reports file names, error names
(stack: rev efb5917f2247b4cc1951ba950a9b9ff1a4606c92)
* dev-support/test-patch.sh
* dev-support/checkstyle_report.py
* hbase-checkstyle/src/main/resources/hbase/checkstyle.xml
> Better checkstyle reporting
> ---
>
> Key: HBASE-14859
> URL: https://issues.apache.org/jira/browse/HBASE-14859
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-14859-master-v2.patch, HBASE-14859.patch
>
>
> With additional checkstyles in place, I believe "-1 checkstyle" will fire
> more often now. Trying to make hunting down exact errors easier.
> This change will dump error like these in the checkstyle part of console
> itself:
> {noformat}
> hbase-common/src/main/java/org/apache/hadoop/hbase/Tag.java
> NonEmptyAtclauseDescriptionCheck10 11
> hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> NonEmptyAtclauseDescriptionCheck0 1
> hbase-client/src/main/java/org/apache/hadoop/hbase/client/Query.java
> NonEmptyAtclauseDescriptionCheck3 4
> hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AsyncRpcChannel.java
> UnusedImportsCheck 0 1
> hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java
> NonEmptyAtclauseDescriptionCheck2 3
> hbase-client/src/main/java/org/apache/hadoop/hbase/security/SaslUtil.java
> ArrayTypeStyleCheck 0 1
> hbase-client/src/main/java/org/apache/hadoop/hbase/security/SaslUtil.java
> UnusedImportsCheck 0 1
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14886) ReplicationAdmin does not use full peer configuration
[
https://issues.apache.org/jira/browse/HBASE-14886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028005#comment-15028005
]
Hadoop QA commented on HBASE-14886:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12774429/hbase-14886-v1.patch
against master branch at commit efb5917f2247b4cc1951ba950a9b9ff1a4606c92.
ATTACHMENT ID: 12774429
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
new checkstyle errors. Check build console for list of new errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16669//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16669//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16669//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16669//console
This message is automatically generated.
> ReplicationAdmin does not use full peer configuration
> -
>
> Key: HBASE-14886
> URL: https://issues.apache.org/jira/browse/HBASE-14886
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: hbase-14886-v1.patch, hbase-14886.patch
>
>
> In {{listValidReplicationPeers()}}, we're creating the peer {{Configuration}}
> based on the source connection configuration and simply applying the peer ZK
> cluster key. This causes any additional properties present in the
> {{ReplicationPeerConfig}} configuration to not be applied.
> We should instead be using the configuration returned by
> {{ReplicationPeers.getPeerConf()}}, which we already call in that method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14886) ReplicationAdmin does not use full peer configuration
[
https://issues.apache.org/jira/browse/HBASE-14886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027918#comment-15027918
]
Elliott Clark commented on HBASE-14886:
---
Would this same change apply to just about everywhere that
applyClusterKeyToConf is used ?
> ReplicationAdmin does not use full peer configuration
> -
>
> Key: HBASE-14886
> URL: https://issues.apache.org/jira/browse/HBASE-14886
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: hbase-14886-v1.patch, hbase-14886.patch
>
>
> In {{listValidReplicationPeers()}}, we're creating the peer {{Configuration}}
> based on the source connection configuration and simply applying the peer ZK
> cluster key. This causes any additional properties present in the
> {{ReplicationPeerConfig}} configuration to not be applied.
> We should instead be using the configuration returned by
> {{ReplicationPeers.getPeerConf()}}, which we already call in that method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14730) region server needs to log warnings when there are attributes configured for cells with hfile v2
[ https://issues.apache.org/jira/browse/HBASE-14730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun updated HBASE-14730: - Status: Patch Available (was: Open) > region server needs to log warnings when there are attributes configured for > cells with hfile v2 > > > Key: HBASE-14730 > URL: https://issues.apache.org/jira/browse/HBASE-14730 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 1.1.0, 1.0.0, 1.2.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-14730-v001.patch > > > User can configure cell attributes with hfile.format.version 2. When cells > are flushed from memStore to Hfile, attributes are not saved. Warnings need > to be logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14730) region server needs to log warnings when there are attributes configured for cells with hfile v2
[ https://issues.apache.org/jira/browse/HBASE-14730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun updated HBASE-14730: - Attachment: HBASE-14730-v001.patch > region server needs to log warnings when there are attributes configured for > cells with hfile v2 > > > Key: HBASE-14730 > URL: https://issues.apache.org/jira/browse/HBASE-14730 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 1.0.0, 1.1.0, 1.2.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-14730-v001.patch > > > User can configure cell attributes with hfile.format.version 2. When cells > are flushed from memStore to Hfile, attributes are not saved. Warnings need > to be logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14730) region server needs to log warnings when there are attributes configured for cells with hfile v2
[ https://issues.apache.org/jira/browse/HBASE-14730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun updated HBASE-14730: - Affects Version/s: 1.2.0 1.0.0 1.1.0 > region server needs to log warnings when there are attributes configured for > cells with hfile v2 > > > Key: HBASE-14730 > URL: https://issues.apache.org/jira/browse/HBASE-14730 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 1.0.0, 1.1.0, 1.2.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > > User can configure cell attributes with hfile.format.version 2. When cells > are flushed from memStore to Hfile, attributes are not saved. Warnings need > to be logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14769) Remove unused functions and duplicate javadocs from HBaseAdmin
[
https://issues.apache.org/jira/browse/HBASE-14769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027852#comment-15027852
]
Hadoop QA commented on HBASE-14769:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12774413/HBASE-14769-master-v6.patch
against master branch at commit efb5917f2247b4cc1951ba950a9b9ff1a4606c92.
ATTACHMENT ID: 12774413
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 33 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
new checkstyle errors. Check build console for list of new errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16668//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16668//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16668//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16668//console
This message is automatically generated.
> Remove unused functions and duplicate javadocs from HBaseAdmin
> ---
>
> Key: HBASE-14769
> URL: https://issues.apache.org/jira/browse/HBASE-14769
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
> Fix For: 2.0.0
>
> Attachments: HBASE-14769-master-v2.patch,
> HBASE-14769-master-v3.patch, HBASE-14769-master-v4.patch,
> HBASE-14769-master-v5.patch, HBASE-14769-master-v6.patch,
> HBASE-14769-master.patch
>
>
> HBaseAdmin is marked private, so removing the functions not being used
> anywhere.
> Also, the javadocs of overridden functions are same as corresponding ones in
> Admin.java. Since javadocs are automatically inherited from the interface
> class, we can remove these redundant 100s of lines.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14885) NullPointerException in HMaster#normalizeRegions() due to missing TableDescriptor
[
https://issues.apache.org/jira/browse/HBASE-14885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-14885:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
Thanks for the review, Stephen
> NullPointerException in HMaster#normalizeRegions() due to missing
> TableDescriptor
> -
>
> Key: HBASE-14885
> URL: https://issues.apache.org/jira/browse/HBASE-14885
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14885-branch-1.txt, 14885-v1.txt
>
>
> During system test on Windows, we observed the following in master log:
> {code}
> 2015-11-23 11:31:38,853 ERROR
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore: Caught
> error^M
> java.lang.NullPointerException^M
> at
> org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:1396)^M
> at
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:49)^M
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:185)^M
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)^M
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)^M
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)^M
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)^M
> at java.lang.Thread.run(Thread.java:745)^M
> {code}
> The NullPointerException came from the second line below:
> {code}
> for(TableName table : allEnabledTables) {
> if (table.isSystemTable() ||
> !getTableDescriptors().get(table).isNormalizationEnabled()) {
> {code}
> It seems TableDescriptor for some table was absent.
> normalizeRegions() should deal with such scenario without producing NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14885) NullPointerException in HMaster#normalizeRegions() due to missing TableDescriptor
[
https://issues.apache.org/jira/browse/HBASE-14885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027827#comment-15027827
]
Hadoop QA commented on HBASE-14885:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12774410/14885-v1.txt
against master branch at commit efb5917f2247b4cc1951ba950a9b9ff1a4606c92.
ATTACHMENT ID: 12774410
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16667//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16667//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16667//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16667//console
This message is automatically generated.
> NullPointerException in HMaster#normalizeRegions() due to missing
> TableDescriptor
> -
>
> Key: HBASE-14885
> URL: https://issues.apache.org/jira/browse/HBASE-14885
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14885-branch-1.txt, 14885-v1.txt
>
>
> During system test on Windows, we observed the following in master log:
> {code}
> 2015-11-23 11:31:38,853 ERROR
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore: Caught
> error^M
> java.lang.NullPointerException^M
> at
> org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:1396)^M
> at
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:49)^M
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:185)^M
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)^M
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)^M
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)^M
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)^M
> at java.lang.Thread.run(Thread.java:745)^M
> {code}
> The NullPointerException came from the second line below:
> {code}
> for(TableName table : allEnabledTables) {
> if (table.isSystemTable() ||
> !getTableDescriptors().get(table).isNormalizationEnabled()) {
> {code}
> It seems TableDescriptor for some table was absent.
> normalizeRegions() should deal with such scenario without producing NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14825) HBase Ref Guide corrections of typos/misspellings
[
https://issues.apache.org/jira/browse/HBASE-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027810#comment-15027810
]
Daniel Vimont commented on HBASE-14825:
---
Thanks!
> HBase Ref Guide corrections of typos/misspellings
> -
>
> Key: HBASE-14825
> URL: https://issues.apache.org/jira/browse/HBASE-14825
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 2.0.0
>Reporter: Daniel Vimont
>Assignee: Daniel Vimont
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-14825-v2.patch, HBASE-14825-v3.patch,
> HBASE-14825-v4.patch, HBASE-14825-v5-test.patch, HBASE-14825-v6.patch,
> HBASE-14825.patch, HBASE-14825_misty_example.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Found the following list of typos/misspellings on the book.html page, and
> thought I would make corrections to the appropriate src/main/asciidoc files
> in which they are located. (This is just a good opportunity for me to become
> familiar with submission of fixes/patches as a prelude to beginning to make
> some coding contributions. This is also my first submission to the JIRA
> system, so corrections to content/conventions are welcome!)
> [Note: I see that [~misty] may be in the midst of a reformatting task --
> HBASE-14823 -- that might involve these same asciidoc files. Please advise
> if I should wait on this task to avoid a possibly cumbersome Git
> reconciliation mess. (?)]
> Here is the list of typos/misspellings. The format of each item is (a) the
> problem is presented in brackets on the first line, and (b) the phrase (as it
> currently appears in the text) is on the second line.
> ===
> ["you" should be "your", and "Kimballs'" should be "Kimball's" (move the
> apostrophe) in the following:]
> A useful read setting config on you hadoop cluster is Aaron Kimballs'
> Configuration Parameters: What can you just ignore?
> [Period needed after "a"]
> a.k.a pseudo-distributed
> ["empty" is misspelled]
> The default value in this configuration has been intentionally left emtpy in
> order to honor the old hbase.regionserver.global.memstore.upperLimit property
> if present.
> [All occurrences of "a HBase" should be changed to "an HBase" -- 15
> occurrences found]
> ["file path are" should be "file paths are"]
> By default, all of HBase's ZooKeeper file path are configured with a relative
> path, so they will all go under this directory unless changed.
> ["times" -- plural required]
> How many time to retry attempting to write a version file before just
> aborting.
> ["separated" is misspelled]
> Each attempt is seperated by the hbase.server.thread.wakefrequency
> milliseconds.
> [space needed after quotation mark (include"limit)]
> Because this limit represents the "automatic include"limit...
> [space needed ("ashbase:metadata" should be "as hbase:metadata")]
> This helps to keep compaction of lean tables (such ashbase:meta) fast.
> [Acronym "ide" should be capitalized for clarity: IDE]
> Setting this to true can be useful in contexts other than the other side of a
> maven generation; i.e. running in an ide.
> [RuntimeException missing an "e"]
> You'll want to set this boolean to true to avoid seeing the RuntimException
> complaint:
> [Space missing after "secure"]
> FS Permissions for the root directory in a secure(kerberos) setup.
> ["mutations" misspelled]
> ...will be created which will tail the logs and replicate the mutatations to
> region replicas for tables that have region replication > 1.
> ["it such that" should be "is such that"]
> If your working set it such that block cache does you no good...
> ["an" should be "and"]
> See the Deveraj Das an Nicolas Liochon blog post...
> [Tag "" should be ""]
> hbase.coprocessor.master.classes
> [Misspelling of "implementations"]
> Those consumers are coprocessors, phoenix, replication endpoint
> implemnetations or similar.
> [Misspelling of "cluster"]
> On upgrade, before running a rolling restart over the cluser...
> ["effect" should be "affect"]
> If NOT using BucketCache, this change does not effect you.
> [Need space after "throw"]
> This will throw`java.lang.NoSuchMethodError...
> ["habasee" should be "hbase"]
> You can pass commands to the HBase Shell in non-interactive mode (see
> hbasee.shell.noninteractive)...
> ["ie" should be "i.e."]
> Restrict the amount of resources (ie regions, tables) a namespace can consume.
> ["an" should be "and"]
> ...but can be conjured on the fly while the table is up an running.
> [Malformed link (text appears as follows when rendered in a browser):]
> Puts are executed via Table.put (writeBuffer) or
> link:http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Table.html#batch(java.util.List,
>
[jira] [Updated] (HBASE-14866) VerifyReplication should use peer configuration in peer connection
[
https://issues.apache.org/jira/browse/HBASE-14866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Helmling updated HBASE-14866:
--
Attachment: hbase-14866_v3.patch
Fix checkstyle issue.
> VerifyReplication should use peer configuration in peer connection
> --
>
> Key: HBASE-14866
> URL: https://issues.apache.org/jira/browse/HBASE-14866
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14866.patch, HBASE-14866_v1.patch,
> hbase-14866_v2.patch, hbase-14866_v3.patch
>
>
> VerifyReplication uses the replication peer's configuration to construct the
> ZooKeeper quorum address for the peer connection. However, other
> configuration properties in the peer's configuration are dropped. It should
> merge all configuration properties from the {{ReplicationPeerConfig}} when
> creating the peer connection and obtaining a credentials for the peer cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14865) Support passing multiple QOPs to SaslClient/Server via hbase.rpc.protection
[
https://issues.apache.org/jira/browse/HBASE-14865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027770#comment-15027770
]
Hadoop QA commented on HBASE-14865:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12774403/HBASE-14865-master-v4.patch
against master branch at commit 90bdb0dc7412f3ae8fc1e14ce033bf115c6332f3.
ATTACHMENT ID: 12774403
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 27 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
18686 checkstyle errors (more than the master's current 18685 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16665//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16665//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16665//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16665//console
This message is automatically generated.
> Support passing multiple QOPs to SaslClient/Server via hbase.rpc.protection
> ---
>
> Key: HBASE-14865
> URL: https://issues.apache.org/jira/browse/HBASE-14865
> Project: HBase
> Issue Type: Improvement
> Components: security
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-14865-master-v2.patch,
> HBASE-14865-master-v3.patch, HBASE-14865-master-v4.patch,
> HBASE-14865-master.patch
>
>
> Currently, we can set the value of hbase.rpc.protection to one of
> authentication/integrity/privacy. It is the used to set
> {{javax.security.sasl.qop}} in SaslUtil.java.
> The problem is, if a cluster wants to switch from one qop to another, it'll
> have to take a downtime. Rolling upgrade will create a situation where some
> nodes have old value and some have new, which'll prevent any communication
> between them. There will be similar issue when clients will try to connect.
> {{javax.security.sasl.qop}} can take in a list of QOP in preferences order.
> So a transition from qop1 to qop2 can be easily done like this
> "qop1" --> "qop2,qop1" --> rolling restart --> "qop2" --> rolling restart
> Need to change hbase.rpc.protection to accept a list too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14866) VerifyReplication should use peer configuration in peer connection
[
https://issues.apache.org/jira/browse/HBASE-14866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027759#comment-15027759
]
Hadoop QA commented on HBASE-14866:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12774398/hbase-14866_v2.patch
against master branch at commit 90bdb0dc7412f3ae8fc1e14ce033bf115c6332f3.
ATTACHMENT ID: 12774398
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
18686 checkstyle errors (more than the master's current 18685 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/1//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1//console
This message is automatically generated.
> VerifyReplication should use peer configuration in peer connection
> --
>
> Key: HBASE-14866
> URL: https://issues.apache.org/jira/browse/HBASE-14866
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14866.patch, HBASE-14866_v1.patch,
> hbase-14866_v2.patch
>
>
> VerifyReplication uses the replication peer's configuration to construct the
> ZooKeeper quorum address for the peer connection. However, other
> configuration properties in the peer's configuration are dropped. It should
> merge all configuration properties from the {{ReplicationPeerConfig}} when
> creating the peer connection and obtaining a credentials for the peer cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14876) Provide maven archetypes
[ https://issues.apache.org/jira/browse/HBASE-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027754#comment-15027754 ] Gabor Liptak commented on HBASE-14876: -- [~ndimiduk] How do you see archetypes packaged (and published)? Thanks https://maven.apache.org/guides/mini/guide-creating-archetypes.html > Provide maven archetypes > > > Key: HBASE-14876 > URL: https://issues.apache.org/jira/browse/HBASE-14876 > Project: HBase > Issue Type: New Feature > Components: build, Usability >Reporter: Nick Dimiduk > Labels: beginner > > To help onboard new users, we should provide maven archetypes for hbase > client applications. Off the top of my head, we should have templates for > - hbase client application with all dependencies > - hbase client application using client-shaded jar > - mapreduce application with hbase as input and output (ie, copy table) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14872) Scan different timeRange per column family doesn't percolate down to the memstore
[ https://issues.apache.org/jira/browse/HBASE-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HBASE-14872: --- Status: Patch Available (was: Open) > Scan different timeRange per column family doesn't percolate down to the > memstore > -- > > Key: HBASE-14872 > URL: https://issues.apache.org/jira/browse/HBASE-14872 > Project: HBase > Issue Type: Bug > Components: Client, regionserver, Scanners >Affects Versions: 2.0.0, 1.3.0 >Reporter: churro morales >Assignee: churro morales > Fix For: 2.0.0, 1.3.0, 0.98.17 > > Attachments: HBASE-14872-v1.patch, HBASE-14872.patch > > > HBASE-14355 The scan different time range for column family feature was not > applied to the memstore it was only done for the store files. This breaks > the contract. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14872) Scan different timeRange per column family doesn't percolate down to the memstore
[ https://issues.apache.org/jira/browse/HBASE-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HBASE-14872: --- Status: Open (was: Patch Available) > Scan different timeRange per column family doesn't percolate down to the > memstore > -- > > Key: HBASE-14872 > URL: https://issues.apache.org/jira/browse/HBASE-14872 > Project: HBase > Issue Type: Bug > Components: Client, regionserver, Scanners >Affects Versions: 2.0.0, 1.3.0 >Reporter: churro morales >Assignee: churro morales > Fix For: 2.0.0, 1.3.0, 0.98.17 > > Attachments: HBASE-14872-v1.patch, HBASE-14872.patch > > > HBASE-14355 The scan different time range for column family feature was not > applied to the memstore it was only done for the store files. This breaks > the contract. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14872) Scan different timeRange per column family doesn't percolate down to the memstore
[ https://issues.apache.org/jira/browse/HBASE-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HBASE-14872: --- Attachment: HBASE-14872-v1.patch > Scan different timeRange per column family doesn't percolate down to the > memstore > -- > > Key: HBASE-14872 > URL: https://issues.apache.org/jira/browse/HBASE-14872 > Project: HBase > Issue Type: Bug > Components: Client, regionserver, Scanners >Affects Versions: 2.0.0, 1.3.0 >Reporter: churro morales >Assignee: churro morales > Fix For: 2.0.0, 1.3.0, 0.98.17 > > Attachments: HBASE-14872-v1.patch, HBASE-14872.patch > > > HBASE-14355 The scan different time range for column family feature was not > applied to the memstore it was only done for the store files. This breaks > the contract. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14885) NullPointerException in HMaster#normalizeRegions() due to missing TableDescriptor
[
https://issues.apache.org/jira/browse/HBASE-14885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-14885:
---
Hadoop Flags: Reviewed
Fix Version/s: 1.3.0
1.2.0
2.0.0
> NullPointerException in HMaster#normalizeRegions() due to missing
> TableDescriptor
> -
>
> Key: HBASE-14885
> URL: https://issues.apache.org/jira/browse/HBASE-14885
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14885-branch-1.txt, 14885-v1.txt
>
>
> During system test on Windows, we observed the following in master log:
> {code}
> 2015-11-23 11:31:38,853 ERROR
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore: Caught
> error^M
> java.lang.NullPointerException^M
> at
> org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:1396)^M
> at
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:49)^M
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:185)^M
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)^M
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)^M
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)^M
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)^M
> at java.lang.Thread.run(Thread.java:745)^M
> {code}
> The NullPointerException came from the second line below:
> {code}
> for(TableName table : allEnabledTables) {
> if (table.isSystemTable() ||
> !getTableDescriptors().get(table).isNormalizationEnabled()) {
> {code}
> It seems TableDescriptor for some table was absent.
> normalizeRegions() should deal with such scenario without producing NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14883) TestSplitTransactionOnCluster#testFailedSplit flakey
[
https://issues.apache.org/jira/browse/HBASE-14883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027722#comment-15027722
]
Hudson commented on HBASE-14883:
FAILURE: Integrated in HBase-1.2 #401 (See
[https://builds.apache.org/job/HBase-1.2/401/])
HBASE-14883 TestSplitTransactionOnCluster#testFailedSplit flakey (stack: rev
bd63a59c60838d297c0a2c5cc50f2bc74796ecfd)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java
> TestSplitTransactionOnCluster#testFailedSplit flakey
>
>
> Key: HBASE-14883
> URL: https://issues.apache.org/jira/browse/HBASE-14883
> Project: HBase
> Issue Type: Sub-task
> Components: flakey, test
>Affects Versions: 1.2.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Attachments: 14883-branch-1.txt
>
>
> Only in branch-1 and branch-1.2.
> Fails look like this:
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.3/jdk=latest1.8,label=Hadoop/397/
> TEST-org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.xml.
> If I look in the xml, I see this:
> {code}
>classname="org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster"
> time="8.275">
> java.lang.AssertionError:
> null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testFailedSplit(TestSplitTransactionOnCluster.java:1339)
>
[jira] [Commented] (HBASE-14872) Scan different timeRange per column family doesn't percolate down to the memstore
[ https://issues.apache.org/jira/browse/HBASE-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027715#comment-15027715 ] churro morales commented on HBASE-14872: Sure thing will file a ticket after I put up a patch to add that feature to the REST API, thanks Andrew! > Scan different timeRange per column family doesn't percolate down to the > memstore > -- > > Key: HBASE-14872 > URL: https://issues.apache.org/jira/browse/HBASE-14872 > Project: HBase > Issue Type: Bug > Components: Client, regionserver, Scanners >Affects Versions: 2.0.0, 1.3.0 >Reporter: churro morales >Assignee: churro morales > Fix For: 2.0.0, 1.3.0, 0.98.17 > > Attachments: HBASE-14872.patch > > > HBASE-14355 The scan different time range for column family feature was not > applied to the memstore it was only done for the store files. This breaks > the contract. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14885) NullPointerException in HMaster#normalizeRegions() due to missing TableDescriptor
[
https://issues.apache.org/jira/browse/HBASE-14885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-14885:
---
Attachment: 14885-branch-1.txt
> NullPointerException in HMaster#normalizeRegions() due to missing
> TableDescriptor
> -
>
> Key: HBASE-14885
> URL: https://issues.apache.org/jira/browse/HBASE-14885
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 14885-branch-1.txt, 14885-v1.txt
>
>
> During system test on Windows, we observed the following in master log:
> {code}
> 2015-11-23 11:31:38,853 ERROR
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore: Caught
> error^M
> java.lang.NullPointerException^M
> at
> org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:1396)^M
> at
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:49)^M
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:185)^M
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)^M
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)^M
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)^M
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)^M
> at java.lang.Thread.run(Thread.java:745)^M
> {code}
> The NullPointerException came from the second line below:
> {code}
> for(TableName table : allEnabledTables) {
> if (table.isSystemTable() ||
> !getTableDescriptors().get(table).isNormalizationEnabled()) {
> {code}
> It seems TableDescriptor for some table was absent.
> normalizeRegions() should deal with such scenario without producing NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14872) Scan different timeRange per column family doesn't percolate down to the memstore
[ https://issues.apache.org/jira/browse/HBASE-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027708#comment-15027708 ] Andrew Purtell commented on HBASE-14872: Sure, just please consider filing an issue for adding support for per CF scan timeranges to the REST API at some future time. > Scan different timeRange per column family doesn't percolate down to the > memstore > -- > > Key: HBASE-14872 > URL: https://issues.apache.org/jira/browse/HBASE-14872 > Project: HBase > Issue Type: Bug > Components: Client, regionserver, Scanners >Affects Versions: 2.0.0, 1.3.0 >Reporter: churro morales >Assignee: churro morales > Fix For: 2.0.0, 1.3.0, 0.98.17 > > Attachments: HBASE-14872.patch > > > HBASE-14355 The scan different time range for column family feature was not > applied to the memstore it was only done for the store files. This breaks > the contract. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14886) ReplicationAdmin does not use full peer configuration
[
https://issues.apache.org/jira/browse/HBASE-14886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Helmling updated HBASE-14886:
--
Status: Patch Available (was: Open)
> ReplicationAdmin does not use full peer configuration
> -
>
> Key: HBASE-14886
> URL: https://issues.apache.org/jira/browse/HBASE-14886
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: hbase-14886-v1.patch, hbase-14886.patch
>
>
> In {{listValidReplicationPeers()}}, we're creating the peer {{Configuration}}
> based on the source connection configuration and simply applying the peer ZK
> cluster key. This causes any additional properties present in the
> {{ReplicationPeerConfig}} configuration to not be applied.
> We should instead be using the configuration returned by
> {{ReplicationPeers.getPeerConf()}}, which we already call in that method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14886) ReplicationAdmin does not use full peer configuration
[
https://issues.apache.org/jira/browse/HBASE-14886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Helmling updated HBASE-14886:
--
Attachment: hbase-14886-v1.patch
Attaching a new patch that adds a simple test for the problem.
> ReplicationAdmin does not use full peer configuration
> -
>
> Key: HBASE-14886
> URL: https://issues.apache.org/jira/browse/HBASE-14886
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: hbase-14886-v1.patch, hbase-14886.patch
>
>
> In {{listValidReplicationPeers()}}, we're creating the peer {{Configuration}}
> based on the source connection configuration and simply applying the peer ZK
> cluster key. This causes any additional properties present in the
> {{ReplicationPeerConfig}} configuration to not be applied.
> We should instead be using the configuration returned by
> {{ReplicationPeers.getPeerConf()}}, which we already call in that method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14886) ReplicationAdmin does not use full peer configuration
[
https://issues.apache.org/jira/browse/HBASE-14886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Helmling updated HBASE-14886:
--
Status: Open (was: Patch Available)
> ReplicationAdmin does not use full peer configuration
> -
>
> Key: HBASE-14886
> URL: https://issues.apache.org/jira/browse/HBASE-14886
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: hbase-14886-v1.patch, hbase-14886.patch
>
>
> In {{listValidReplicationPeers()}}, we're creating the peer {{Configuration}}
> based on the source connection configuration and simply applying the peer ZK
> cluster key. This causes any additional properties present in the
> {{ReplicationPeerConfig}} configuration to not be applied.
> We should instead be using the configuration returned by
> {{ReplicationPeers.getPeerConf()}}, which we already call in that method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14872) Scan different timeRange per column family doesn't percolate down to the memstore
[ https://issues.apache.org/jira/browse/HBASE-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027663#comment-15027663 ] churro morales commented on HBASE-14872: Looks like changing the ScannerModel will be a bit of work. Right now it doesn't look like the rest API supports the everything the scan does, do you guys think its fine leaving this column family time range feature for the rest client? If not I can take a stab at making it work but it will take some time, I can get a patch for everything else to fix the other issues right now. What do you think [~stack] and [~apurtell] ? > Scan different timeRange per column family doesn't percolate down to the > memstore > -- > > Key: HBASE-14872 > URL: https://issues.apache.org/jira/browse/HBASE-14872 > Project: HBase > Issue Type: Bug > Components: Client, regionserver, Scanners >Affects Versions: 2.0.0, 1.3.0 >Reporter: churro morales >Assignee: churro morales > Fix For: 2.0.0, 1.3.0, 0.98.17 > > Attachments: HBASE-14872.patch > > > HBASE-14355 The scan different time range for column family feature was not > applied to the memstore it was only done for the store files. This breaks > the contract. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14886) ReplicationAdmin does not use full peer configuration
[
https://issues.apache.org/jira/browse/HBASE-14886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Helmling updated HBASE-14886:
--
Status: Patch Available (was: Open)
> ReplicationAdmin does not use full peer configuration
> -
>
> Key: HBASE-14886
> URL: https://issues.apache.org/jira/browse/HBASE-14886
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: hbase-14886.patch
>
>
> In {{listValidReplicationPeers()}}, we're creating the peer {{Configuration}}
> based on the source connection configuration and simply applying the peer ZK
> cluster key. This causes any additional properties present in the
> {{ReplicationPeerConfig}} configuration to not be applied.
> We should instead be using the configuration returned by
> {{ReplicationPeers.getPeerConf()}}, which we already call in that method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14886) ReplicationAdmin does not use full peer configuration
[
https://issues.apache.org/jira/browse/HBASE-14886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Helmling updated HBASE-14886:
--
Attachment: hbase-14886.patch
Attaching a patch which uses the Configuration returned by
ReplicationPeers.getPeerConf(), which has the ReplicationPeerConfig config
properties already merged in.
> ReplicationAdmin does not use full peer configuration
> -
>
> Key: HBASE-14886
> URL: https://issues.apache.org/jira/browse/HBASE-14886
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: hbase-14886.patch
>
>
> In {{listValidReplicationPeers()}}, we're creating the peer {{Configuration}}
> based on the source connection configuration and simply applying the peer ZK
> cluster key. This causes any additional properties present in the
> {{ReplicationPeerConfig}} configuration to not be applied.
> We should instead be using the configuration returned by
> {{ReplicationPeers.getPeerConf()}}, which we already call in that method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14885) NullPointerException in HMaster#normalizeRegions() due to missing TableDescriptor
[
https://issues.apache.org/jira/browse/HBASE-14885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027586#comment-15027586
]
Stephen Yuan Jiang commented on HBASE-14885:
+1 LGTM (based on [~tedyu]'s research, new table could have TableDescriptor
missing and "looking at FSTableDescriptors#readTableDescriptor(), if content
cannot be parsed, null is returned")
> NullPointerException in HMaster#normalizeRegions() due to missing
> TableDescriptor
> -
>
> Key: HBASE-14885
> URL: https://issues.apache.org/jira/browse/HBASE-14885
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 14885-v1.txt
>
>
> During system test on Windows, we observed the following in master log:
> {code}
> 2015-11-23 11:31:38,853 ERROR
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore: Caught
> error^M
> java.lang.NullPointerException^M
> at
> org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:1396)^M
> at
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:49)^M
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:185)^M
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)^M
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)^M
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)^M
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)^M
> at java.lang.Thread.run(Thread.java:745)^M
> {code}
> The NullPointerException came from the second line below:
> {code}
> for(TableName table : allEnabledTables) {
> if (table.isSystemTable() ||
> !getTableDescriptors().get(table).isNormalizationEnabled()) {
> {code}
> It seems TableDescriptor for some table was absent.
> normalizeRegions() should deal with such scenario without producing NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14769) Remove unused functions and duplicate javadocs from HBaseAdmin
[ https://issues.apache.org/jira/browse/HBASE-14769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-14769: - Attachment: HBASE-14769-master-v6.patch fixing merge error. > Remove unused functions and duplicate javadocs from HBaseAdmin > --- > > Key: HBASE-14769 > URL: https://issues.apache.org/jira/browse/HBASE-14769 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Fix For: 2.0.0 > > Attachments: HBASE-14769-master-v2.patch, > HBASE-14769-master-v3.patch, HBASE-14769-master-v4.patch, > HBASE-14769-master-v5.patch, HBASE-14769-master-v6.patch, > HBASE-14769-master.patch > > > HBaseAdmin is marked private, so removing the functions not being used > anywhere. > Also, the javadocs of overridden functions are same as corresponding ones in > Admin.java. Since javadocs are automatically inherited from the interface > class, we can remove these redundant 100s of lines. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14883) TestSplitTransactionOnCluster#testFailedSplit flakey
[
https://issues.apache.org/jira/browse/HBASE-14883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027548#comment-15027548
]
Hudson commented on HBASE-14883:
SUCCESS: Integrated in HBase-1.2-IT #308 (See
[https://builds.apache.org/job/HBase-1.2-IT/308/])
HBASE-14883 TestSplitTransactionOnCluster#testFailedSplit flakey (stack: rev
bd63a59c60838d297c0a2c5cc50f2bc74796ecfd)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java
> TestSplitTransactionOnCluster#testFailedSplit flakey
>
>
> Key: HBASE-14883
> URL: https://issues.apache.org/jira/browse/HBASE-14883
> Project: HBase
> Issue Type: Sub-task
> Components: flakey, test
>Affects Versions: 1.2.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Attachments: 14883-branch-1.txt
>
>
> Only in branch-1 and branch-1.2.
> Fails look like this:
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.3/jdk=latest1.8,label=Hadoop/397/
> TEST-org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.xml.
> If I look in the xml, I see this:
> {code}
>classname="org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster"
> time="8.275">
> java.lang.AssertionError:
> null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testFailedSplit(TestSplitTransactionOnCluster.java:1339)
>
[jira] [Created] (HBASE-14886) ReplicationAdmin does not use full peer configuration
Gary Helmling created HBASE-14886:
-
Summary: ReplicationAdmin does not use full peer configuration
Key: HBASE-14886
URL: https://issues.apache.org/jira/browse/HBASE-14886
Project: HBase
Issue Type: Bug
Components: Replication
Reporter: Gary Helmling
Assignee: Gary Helmling
Priority: Critical
Fix For: 2.0.0, 1.2.0, 1.3.0
In {{listValidReplicationPeers()}}, we're creating the peer {{Configuration}}
based on the source connection configuration and simply applying the peer ZK
cluster key. This causes any additional properties present in the
{{ReplicationPeerConfig}} configuration to not be applied.
We should instead be using the configuration returned by
{{ReplicationPeers.getPeerConf()}}, which we already call in that method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14883) TestSplitTransactionOnCluster#testFailedSplit flakey
[
https://issues.apache.org/jira/browse/HBASE-14883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027513#comment-15027513
]
Hudson commented on HBASE-14883:
SUCCESS: Integrated in HBase-1.3 #398 (See
[https://builds.apache.org/job/HBase-1.3/398/])
HBASE-14883 TestSplitTransactionOnCluster#testFailedSplit flakey (stack: rev
30604761c4f2e72496c8cdd804536e239444012a)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java
> TestSplitTransactionOnCluster#testFailedSplit flakey
>
>
> Key: HBASE-14883
> URL: https://issues.apache.org/jira/browse/HBASE-14883
> Project: HBase
> Issue Type: Sub-task
> Components: flakey, test
>Affects Versions: 1.2.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Attachments: 14883-branch-1.txt
>
>
> Only in branch-1 and branch-1.2.
> Fails look like this:
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.3/jdk=latest1.8,label=Hadoop/397/
> TEST-org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.xml.
> If I look in the xml, I see this:
> {code}
>classname="org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster"
> time="8.275">
> java.lang.AssertionError:
> null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testFailedSplit(TestSplitTransactionOnCluster.java:1339)
>
[jira] [Commented] (HBASE-14883) TestSplitTransactionOnCluster#testFailedSplit flakey
[
https://issues.apache.org/jira/browse/HBASE-14883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027514#comment-15027514
]
Hadoop QA commented on HBASE-14883:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12774381/14883-branch-1.txt
against branch-1 branch at commit 90bdb0dc7412f3ae8fc1e14ce033bf115c6332f3.
ATTACHMENT ID: 12774381
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16664//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16664//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16664//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16664//console
This message is automatically generated.
> TestSplitTransactionOnCluster#testFailedSplit flakey
>
>
> Key: HBASE-14883
> URL: https://issues.apache.org/jira/browse/HBASE-14883
> Project: HBase
> Issue Type: Sub-task
> Components: flakey, test
>Affects Versions: 1.2.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Attachments: 14883-branch-1.txt
>
>
> Only in branch-1 and branch-1.2.
> Fails look like this:
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.3/jdk=latest1.8,label=Hadoop/397/
> TEST-org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.xml.
> If I look in the xml, I see this:
> {code}
>classname="org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster"
> time="8.275">
> java.lang.AssertionError:
> null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testFailedSplit(TestSplitTransactionOnCluster.java:1339)
>
[jira] [Commented] (HBASE-14207) Region was hijacked and remained in transition when RS failed to open a region and later regionplan changed to new RS on retry
[
https://issues.apache.org/jira/browse/HBASE-14207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027506#comment-15027506
]
Stephen Yuan Jiang commented on HBASE-14207:
[~enis], master branch only has zk-less assignment. This part of code does not
exist
> Region was hijacked and remained in transition when RS failed to open a
> region and later regionplan changed to new RS on retry
> --
>
> Key: HBASE-14207
> URL: https://issues.apache.org/jira/browse/HBASE-14207
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.98.6
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 0.98.15
>
> Attachments: HBASE-14207-0.98-V2.patch, HBASE-14207-0.98-V2.patch,
> HBASE-14207-0.98.patch
>
>
> On production environment, following events happened
> 1. Master is trying to assign a region to RS, but due to
> KeeperException$SessionExpiredException RS failed to open the region.
> In RS log, saw multiple WARN log related to
> KeeperException$SessionExpiredException
> > KeeperErrorCode = Session expired for
> /hbase/region-in-transition/08f1935d652e5dbdac09b423b8f9401b
> > Unable to get data of znode
> /hbase/region-in-transition/08f1935d652e5dbdac09b423b8f9401b
> 2. Master retried to assign the region to same RS, but RS again failed.
> 3. On second retry new plan formed and this time plan destination (RS) is
> different, so master send the request to new RS to open the region. But new
> RS failed to open the region as there was server mismatch in ZNODE than the
> expected current server name.
> Logs Snippet:
> {noformat}
> HM
> 2015-07-14 03:50:29,759 | INFO | master:T101PC03VM13:21300 | Processing
> 08f1935d652e5dbdac09b423b8f9401b in state: M_ZK_REGION_OFFLINE |
> org.apache.hadoop.hbase.master.AssignmentManager.processRegionsInTransition(AssignmentManager.java:644)
> 2015-07-14 03:50:29,759 | INFO | master:T101PC03VM13:21300 | Transitioned
> {08f1935d652e5dbdac09b423b8f9401b state=OFFLINE, ts=1436817029679,
> server=null} to {08f1935d652e5dbdac09b423b8f9401b state=PENDING_OPEN,
> ts=1436817029759, server=T101PC03VM13,21302,1436816690692} |
> org.apache.hadoop.hbase.master.RegionStates.updateRegionState(RegionStates.java:327)
> 2015-07-14 03:50:29,760 | INFO | master:T101PC03VM13:21300 | Processed
> region 08f1935d652e5dbdac09b423b8f9401b in state M_ZK_REGION_OFFLINE, on
> server: T101PC03VM13,21302,1436816690692 |
> org.apache.hadoop.hbase.master.AssignmentManager.processRegionsInTransition(AssignmentManager.java:768)
> 2015-07-14 03:50:29,800 | INFO |
> MASTER_SERVER_OPERATIONS-T101PC03VM13:21300-3 | Assigning
> INTER_CONCURRENCY_SETTING,,1436596137981.08f1935d652e5dbdac09b423b8f9401b. to
> T101PC03VM13,21302,1436816690692 |
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1983)
> 2015-07-14 03:50:29,801 | WARN |
> MASTER_SERVER_OPERATIONS-T101PC03VM13:21300-3 | Failed assignment of
> INTER_CONCURRENCY_SETTING,,1436596137981.08f1935d652e5dbdac09b423b8f9401b. to
> T101PC03VM13,21302,1436816690692, trying to assign elsewhere instead; try=1
> of 10 |
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:2077)
> 2015-07-14 03:50:29,802 | INFO |
> MASTER_SERVER_OPERATIONS-T101PC03VM13:21300-3 | Trying to re-assign
> INTER_CONCURRENCY_SETTING,,1436596137981.08f1935d652e5dbdac09b423b8f9401b. to
> the same failed server. |
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:2123)
> 2015-07-14 03:50:31,804 | INFO |
> MASTER_SERVER_OPERATIONS-T101PC03VM13:21300-3 | Assigning
> INTER_CONCURRENCY_SETTING,,1436596137981.08f1935d652e5dbdac09b423b8f9401b. to
> T101PC03VM13,21302,1436816690692 |
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1983)
> 2015-07-14 03:50:31,806 | WARN |
> MASTER_SERVER_OPERATIONS-T101PC03VM13:21300-3 | Failed assignment of
> INTER_CONCURRENCY_SETTING,,1436596137981.08f1935d652e5dbdac09b423b8f9401b. to
> T101PC03VM13,21302,1436816690692, trying to assign elsewhere instead; try=2
> of 10 |
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:2077)
> 2015-07-14 03:50:31,807 | INFO |
> MASTER_SERVER_OPERATIONS-T101PC03VM13:21300-3 | Transitioned
> {08f1935d652e5dbdac09b423b8f9401b state=PENDING_OPEN, ts=1436817031804,
> server=T101PC03VM13,21302,1436816690692} to {08f1935d652e5dbdac09b423b8f9401b
> state=OFFLINE, ts=1436817031807, server=T101PC03VM13,21302,1436816690692} |
> org.apache.hadoop.hbase.master.RegionStates.updateRegionState(RegionStates.java:327)
> 2015-07-14 03:50:31,807 |
[jira] [Updated] (HBASE-14885) NullPointerException in HMaster#normalizeRegions() due to missing TableDescriptor
[
https://issues.apache.org/jira/browse/HBASE-14885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-14885:
---
Attachment: 14885-v1.txt
> NullPointerException in HMaster#normalizeRegions() due to missing
> TableDescriptor
> -
>
> Key: HBASE-14885
> URL: https://issues.apache.org/jira/browse/HBASE-14885
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 14885-v1.txt
>
>
> During system test on Windows, we observed the following in master log:
> {code}
> 2015-11-23 11:31:38,853 ERROR
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore: Caught
> error^M
> java.lang.NullPointerException^M
> at
> org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:1396)^M
> at
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:49)^M
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:185)^M
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)^M
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)^M
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)^M
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)^M
> at java.lang.Thread.run(Thread.java:745)^M
> {code}
> The NullPointerException came from the second line below:
> {code}
> for(TableName table : allEnabledTables) {
> if (table.isSystemTable() ||
> !getTableDescriptors().get(table).isNormalizationEnabled()) {
> {code}
> It seems TableDescriptor for some table was absent.
> normalizeRegions() should deal with such scenario without producing NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14885) NullPointerException in HMaster#normalizeRegions() due to missing TableDescriptor
[
https://issues.apache.org/jira/browse/HBASE-14885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-14885:
---
Status: Patch Available (was: Open)
> NullPointerException in HMaster#normalizeRegions() due to missing
> TableDescriptor
> -
>
> Key: HBASE-14885
> URL: https://issues.apache.org/jira/browse/HBASE-14885
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 14885-v1.txt
>
>
> During system test on Windows, we observed the following in master log:
> {code}
> 2015-11-23 11:31:38,853 ERROR
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore: Caught
> error^M
> java.lang.NullPointerException^M
> at
> org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:1396)^M
> at
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:49)^M
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:185)^M
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)^M
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)^M
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)^M
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)^M
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)^M
> at java.lang.Thread.run(Thread.java:745)^M
> {code}
> The NullPointerException came from the second line below:
> {code}
> for(TableName table : allEnabledTables) {
> if (table.isSystemTable() ||
> !getTableDescriptors().get(table).isNormalizationEnabled()) {
> {code}
> It seems TableDescriptor for some table was absent.
> normalizeRegions() should deal with such scenario without producing NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14883) TestSplitTransactionOnCluster#testFailedSplit flakey
[
https://issues.apache.org/jira/browse/HBASE-14883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027481#comment-15027481
]
Hudson commented on HBASE-14883:
FAILURE: Integrated in HBase-1.3-IT #337 (See
[https://builds.apache.org/job/HBase-1.3-IT/337/])
HBASE-14883 TestSplitTransactionOnCluster#testFailedSplit flakey (stack: rev
30604761c4f2e72496c8cdd804536e239444012a)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java
> TestSplitTransactionOnCluster#testFailedSplit flakey
>
>
> Key: HBASE-14883
> URL: https://issues.apache.org/jira/browse/HBASE-14883
> Project: HBase
> Issue Type: Sub-task
> Components: flakey, test
>Affects Versions: 1.2.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Attachments: 14883-branch-1.txt
>
>
> Only in branch-1 and branch-1.2.
> Fails look like this:
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.3/jdk=latest1.8,label=Hadoop/397/
> TEST-org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.xml.
> If I look in the xml, I see this:
> {code}
>classname="org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster"
> time="8.275">
> java.lang.AssertionError:
> null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testFailedSplit(TestSplitTransactionOnCluster.java:1339)
>
[jira] [Updated] (HBASE-14859) Better checkstyle reporting
[
https://issues.apache.org/jira/browse/HBASE-14859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-14859:
-
Description:
With additional checkstyles in place, I believe "-1 checkstyle" will fire more
often now. Trying to make hunting down exact errors easier.
This change will dump error like these in the checkstyle part of console itself:
{noformat}
hbase-common/src/main/java/org/apache/hadoop/hbase/Tag.java
NonEmptyAtclauseDescriptionCheck10 11
hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
NonEmptyAtclauseDescriptionCheck0 1
hbase-client/src/main/java/org/apache/hadoop/hbase/client/Query.java
NonEmptyAtclauseDescriptionCheck3 4
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AsyncRpcChannel.java
UnusedImportsCheck 0 1
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java
NonEmptyAtclauseDescriptionCheck2 3
hbase-client/src/main/java/org/apache/hadoop/hbase/security/SaslUtil.java
ArrayTypeStyleCheck 0 1
hbase-client/src/main/java/org/apache/hadoop/hbase/security/SaslUtil.java
UnusedImportsCheck 0 1
{noformat}
was:With additional checkstyles in place, I believe "-1 checkstyle" will fire
more often now. Trying to make hunting down exact errors easier.
> Better checkstyle reporting
> ---
>
> Key: HBASE-14859
> URL: https://issues.apache.org/jira/browse/HBASE-14859
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-14859-master-v2.patch, HBASE-14859.patch
>
>
> With additional checkstyles in place, I believe "-1 checkstyle" will fire
> more often now. Trying to make hunting down exact errors easier.
> This change will dump error like these in the checkstyle part of console
> itself:
> {noformat}
> hbase-common/src/main/java/org/apache/hadoop/hbase/Tag.java
> NonEmptyAtclauseDescriptionCheck10 11
> hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> NonEmptyAtclauseDescriptionCheck0 1
> hbase-client/src/main/java/org/apache/hadoop/hbase/client/Query.java
> NonEmptyAtclauseDescriptionCheck3 4
> hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AsyncRpcChannel.java
> UnusedImportsCheck 0 1
> hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java
> NonEmptyAtclauseDescriptionCheck2 3
> hbase-client/src/main/java/org/apache/hadoop/hbase/security/SaslUtil.java
> ArrayTypeStyleCheck 0 1
> hbase-client/src/main/java/org/apache/hadoop/hbase/security/SaslUtil.java
> UnusedImportsCheck 0 1
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13153) Bulk Loaded HFile Replication
[
https://issues.apache.org/jira/browse/HBASE-13153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027470#comment-15027470
]
Hadoop QA commented on HBASE-13153:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12774378/HBASE-13153-branch-1-v18.patch
against branch-1 branch at commit 90bdb0dc7412f3ae8fc1e14ce033bf115c6332f3.
ATTACHMENT ID: 12774378
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 42 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16663//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16663//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16663//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16663//console
This message is automatically generated.
> Bulk Loaded HFile Replication
> -
>
> Key: HBASE-13153
> URL: https://issues.apache.org/jira/browse/HBASE-13153
> Project: HBase
> Issue Type: New Feature
> Components: Replication
>Reporter: sunhaitao
>Assignee: Ashish Singhi
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-13153-branch-1-v18.patch, HBASE-13153-v1.patch,
> HBASE-13153-v10.patch, HBASE-13153-v11.patch, HBASE-13153-v12.patch,
> HBASE-13153-v13.patch, HBASE-13153-v14.patch, HBASE-13153-v15.patch,
> HBASE-13153-v16.patch, HBASE-13153-v17.patch, HBASE-13153-v18.patch,
> HBASE-13153-v2.patch, HBASE-13153-v3.patch, HBASE-13153-v4.patch,
> HBASE-13153-v5.patch, HBASE-13153-v6.patch, HBASE-13153-v7.patch,
> HBASE-13153-v8.patch, HBASE-13153-v9.patch, HBASE-13153.patch, HBase Bulk
> Load Replication-v1-1.pdf, HBase Bulk Load Replication-v2.pdf, HBase Bulk
> Load Replication-v3.pdf, HBase Bulk Load Replication.pdf, HDFS_HA_Solution.PNG
>
>
> Currently we plan to use HBase Replication feature to deal with disaster
> tolerance scenario.But we encounter an issue that we will use bulkload very
> frequently,because bulkload bypass write path, and will not generate WAL, so
> the data will not be replicated to backup cluster. It's inappropriate to
> bukload twice both on active cluster and backup cluster. So i advise do some
> modification to bulkload feature to enable bukload to both active cluster and
> backup cluster
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14885) NullPointerException in HMaster#normalizeRegions() due to missing TableDescriptor
Ted Yu created HBASE-14885:
--
Summary: NullPointerException in HMaster#normalizeRegions() due to
missing TableDescriptor
Key: HBASE-14885
URL: https://issues.apache.org/jira/browse/HBASE-14885
Project: HBase
Issue Type: Bug
Affects Versions: 1.3.0
Reporter: Ted Yu
Assignee: Ted Yu
During system test on Windows, we observed the following in master log:
{code}
2015-11-23 11:31:38,853 ERROR
org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore: Caught error^M
java.lang.NullPointerException^M
at
org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:1396)^M
at
org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:49)^M
at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:185)^M
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)^M
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)^M
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)^M
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)^M
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)^M
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)^M
at java.lang.Thread.run(Thread.java:745)^M
{code}
The NullPointerException came from the second line below:
{code}
for(TableName table : allEnabledTables) {
if (table.isSystemTable() ||
!getTableDescriptors().get(table).isNormalizationEnabled()) {
{code}
It seems TableDescriptor for some table was absent.
normalizeRegions() should deal with such scenario without producing NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14865) Support passing multiple QOPs to SaslClient/Server via hbase.rpc.protection
[
https://issues.apache.org/jira/browse/HBASE-14865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-14865:
-
Attachment: HBASE-14865-master-v4.patch
> Support passing multiple QOPs to SaslClient/Server via hbase.rpc.protection
> ---
>
> Key: HBASE-14865
> URL: https://issues.apache.org/jira/browse/HBASE-14865
> Project: HBase
> Issue Type: Improvement
> Components: security
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-14865-master-v2.patch,
> HBASE-14865-master-v3.patch, HBASE-14865-master-v4.patch,
> HBASE-14865-master.patch
>
>
> Currently, we can set the value of hbase.rpc.protection to one of
> authentication/integrity/privacy. It is the used to set
> {{javax.security.sasl.qop}} in SaslUtil.java.
> The problem is, if a cluster wants to switch from one qop to another, it'll
> have to take a downtime. Rolling upgrade will create a situation where some
> nodes have old value and some have new, which'll prevent any communication
> between them. There will be similar issue when clients will try to connect.
> {{javax.security.sasl.qop}} can take in a list of QOP in preferences order.
> So a transition from qop1 to qop2 can be easily done like this
> "qop1" --> "qop2,qop1" --> rolling restart --> "qop2" --> rolling restart
> Need to change hbase.rpc.protection to accept a list too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14866) VerifyReplication should use peer configuration in peer connection
[
https://issues.apache.org/jira/browse/HBASE-14866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Helmling updated HBASE-14866:
--
Status: Patch Available (was: Open)
> VerifyReplication should use peer configuration in peer connection
> --
>
> Key: HBASE-14866
> URL: https://issues.apache.org/jira/browse/HBASE-14866
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14866.patch, HBASE-14866_v1.patch,
> hbase-14866_v2.patch
>
>
> VerifyReplication uses the replication peer's configuration to construct the
> ZooKeeper quorum address for the peer connection. However, other
> configuration properties in the peer's configuration are dropped. It should
> merge all configuration properties from the {{ReplicationPeerConfig}} when
> creating the peer connection and obtaining a credentials for the peer cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14866) VerifyReplication should use peer configuration in peer connection
[
https://issues.apache.org/jira/browse/HBASE-14866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Helmling updated HBASE-14866:
--
Attachment: hbase-14866_v2.patch
Attaching a new patch making use of HBaseConfiguration.subset() to pass through
the peer configuration.
> VerifyReplication should use peer configuration in peer connection
> --
>
> Key: HBASE-14866
> URL: https://issues.apache.org/jira/browse/HBASE-14866
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14866.patch, HBASE-14866_v1.patch,
> hbase-14866_v2.patch
>
>
> VerifyReplication uses the replication peer's configuration to construct the
> ZooKeeper quorum address for the peer connection. However, other
> configuration properties in the peer's configuration are dropped. It should
> merge all configuration properties from the {{ReplicationPeerConfig}} when
> creating the peer connection and obtaining a credentials for the peer cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14866) VerifyReplication should use peer configuration in peer connection
[
https://issues.apache.org/jira/browse/HBASE-14866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Helmling updated HBASE-14866:
--
Status: Open (was: Patch Available)
> VerifyReplication should use peer configuration in peer connection
> --
>
> Key: HBASE-14866
> URL: https://issues.apache.org/jira/browse/HBASE-14866
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14866.patch, HBASE-14866_v1.patch
>
>
> VerifyReplication uses the replication peer's configuration to construct the
> ZooKeeper quorum address for the peer connection. However, other
> configuration properties in the peer's configuration are dropped. It should
> merge all configuration properties from the {{ReplicationPeerConfig}} when
> creating the peer connection and obtaining a credentials for the peer cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14872) Scan different timeRange per column family doesn't percolate down to the memstore
[ https://issues.apache.org/jira/browse/HBASE-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027410#comment-15027410 ] churro morales commented on HBASE-14872: Thanks for taking a look [~stack] and [~apurtell] I'll make the changes to ScanQueryMatcher and ScannerModel and will keep it backwards compatible for older clients. > Scan different timeRange per column family doesn't percolate down to the > memstore > -- > > Key: HBASE-14872 > URL: https://issues.apache.org/jira/browse/HBASE-14872 > Project: HBase > Issue Type: Bug > Components: Client, regionserver, Scanners >Affects Versions: 2.0.0, 1.3.0 >Reporter: churro morales >Assignee: churro morales > Fix For: 2.0.0, 1.3.0, 0.98.17 > > Attachments: HBASE-14872.patch > > > HBASE-14355 The scan different time range for column family feature was not > applied to the memstore it was only done for the store files. This breaks > the contract. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-14866) VerifyReplication should use peer configuration in peer connection
[
https://issues.apache.org/jira/browse/HBASE-14866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Helmling reassigned HBASE-14866:
-
Assignee: Gary Helmling
> VerifyReplication should use peer configuration in peer connection
> --
>
> Key: HBASE-14866
> URL: https://issues.apache.org/jira/browse/HBASE-14866
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14866.patch, HBASE-14866_v1.patch
>
>
> VerifyReplication uses the replication peer's configuration to construct the
> ZooKeeper quorum address for the peer connection. However, other
> configuration properties in the peer's configuration are dropped. It should
> merge all configuration properties from the {{ReplicationPeerConfig}} when
> creating the peer connection and obtaining a credentials for the peer cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14777) Fix Inter Cluster Replication Future ordering issues
[
https://issues.apache.org/jira/browse/HBASE-14777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027371#comment-15027371
]
Andrew Purtell commented on HBASE-14777:
{quote}
- Uses a proper CompletionService now (like my first - unpublished - version
had). Futures are handled in the proper order.
- Does not alter the entryLists indexes at all, just set any completed chunk to
an emptyLiist (instead of .clear() on the existing set so as not to leave a
large array around longer than needed).
{quote}
+1
> Fix Inter Cluster Replication Future ordering issues
>
>
> Key: HBASE-14777
> URL: https://issues.apache.org/jira/browse/HBASE-14777
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Bhupendra Kumar Jain
>Assignee: Ashu Pachauri
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14777-alternative.txt, HBASE-14777-1.patch,
> HBASE-14777-2.patch, HBASE-14777-3.patch, HBASE-14777-4.patch,
> HBASE-14777-5.patch, HBASE-14777-6.patch, HBASE-14777-addendum.patch,
> HBASE-14777.patch
>
>
> Replication fails with IndexOutOfBoundsException
> {code}
> regionserver.ReplicationSource$ReplicationSourceWorkerThread(939):
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint
> threw unknown exception:java.lang.IndexOutOfBoundsException: Index: 1, Size:
> 1
> at java.util.ArrayList.rangeCheck(Unknown Source)
> at java.util.ArrayList.remove(Unknown Source)
> at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicate(HBaseInterClusterReplicationEndpoint.java:222)
> {code}
> Its happening due to incorrect removal of entries from the replication
> entries list.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14872) Scan different timeRange per column family doesn't percolate down to the memstore
[ https://issues.apache.org/jira/browse/HBASE-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027363#comment-15027363 ] Andrew Purtell commented on HBASE-14872: bq. Aside from that looks like ScannerModel is the last reference for scan.getTimeRange() that needs to be updated. Looks like this is used for the rest API, do we want to go ahead and make the changes there too? Any changes to ScannerModel are fine as long as they are backwards compatible with requests which might be made by older clients. > Scan different timeRange per column family doesn't percolate down to the > memstore > -- > > Key: HBASE-14872 > URL: https://issues.apache.org/jira/browse/HBASE-14872 > Project: HBase > Issue Type: Bug > Components: Client, regionserver, Scanners >Affects Versions: 2.0.0, 1.3.0 >Reporter: churro morales >Assignee: churro morales > Fix For: 2.0.0, 1.3.0, 0.98.17 > > Attachments: HBASE-14872.patch > > > HBASE-14355 The scan different time range for column family feature was not > applied to the memstore it was only done for the store files. This breaks > the contract. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14872) Scan different timeRange per column family doesn't percolate down to the memstore
[ https://issues.apache.org/jira/browse/HBASE-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027360#comment-15027360 ] stack commented on HBASE-14872: --- Change it if you don't mind [~churromorales] You can blame me if it goes wrong. We might get [~apurtell] to have a look before it goes in too (he is Mr. REST). > Scan different timeRange per column family doesn't percolate down to the > memstore > -- > > Key: HBASE-14872 > URL: https://issues.apache.org/jira/browse/HBASE-14872 > Project: HBase > Issue Type: Bug > Components: Client, regionserver, Scanners >Affects Versions: 2.0.0, 1.3.0 >Reporter: churro morales >Assignee: churro morales > Fix For: 2.0.0, 1.3.0, 0.98.17 > > Attachments: HBASE-14872.patch > > > HBASE-14355 The scan different time range for column family feature was not > applied to the memstore it was only done for the store files. This breaks > the contract. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13082) Coarsen StoreScanner locks to RegionScanner
[ https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027356#comment-15027356 ] stack commented on HBASE-13082: --- +1 Thanks [~ram_krish] StoreFileManager and StoreFileInfo all needs cleaning up -- where does StoreFile accounting get done, where is StoreFile metadata kept -- but not as part of this patch. > Coarsen StoreScanner locks to RegionScanner > --- > > Key: HBASE-13082 > URL: https://issues.apache.org/jira/browse/HBASE-13082 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: ramkrishna.s.vasudevan > Attachments: 13082-test.txt, 13082-v2.txt, 13082-v3.txt, > 13082-v4.txt, 13082.txt, 13082.txt, HBASE-13082.pdf, HBASE-13082_1.pdf, > HBASE-13082_12.patch, HBASE-13082_13.patch, HBASE-13082_14.patch, > HBASE-13082_15.patch, HBASE-13082_16.patch, HBASE-13082_17.patch, > HBASE-13082_1_WIP.patch, HBASE-13082_2.pdf, HBASE-13082_2_WIP.patch, > HBASE-13082_3.patch, HBASE-13082_4.patch, HBASE-13082_9.patch, > HBASE-13082_9.patch, HBASE-13082_withoutpatch.jpg, HBASE-13082_withpatch.jpg, > LockVsSynchronized.java, gc.png, gc.png, gc.png, hits.png, next.png, next.png > > > Continuing where HBASE-10015 left of. > We can avoid locking (and memory fencing) inside StoreScanner by deferring to > the lock already held by the RegionScanner. > In tests this shows quite a scan improvement and reduced CPU (the fences make > the cores wait for memory fetches). > There are some drawbacks too: > * All calls to RegionScanner need to be remain synchronized > * Implementors of coprocessors need to be diligent in following the locking > contract. For example Phoenix does not lock RegionScanner.nextRaw() and > required in the documentation (not picking on Phoenix, this one is my fault > as I told them it's OK) > * possible starving of flushes and compaction with heavy read load. > RegionScanner operations would keep getting the locks and the > flushes/compactions would not be able finalize the set of files. > I'll have a patch soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14207) Region was hijacked and remained in transition when RS failed to open a region and later regionplan changed to new RS on retry
[
https://issues.apache.org/jira/browse/HBASE-14207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027330#comment-15027330
]
Enis Soztutar commented on HBASE-14207:
---
Do we need this in master as well?
> Region was hijacked and remained in transition when RS failed to open a
> region and later regionplan changed to new RS on retry
> --
>
> Key: HBASE-14207
> URL: https://issues.apache.org/jira/browse/HBASE-14207
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.98.6
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 0.98.15
>
> Attachments: HBASE-14207-0.98-V2.patch, HBASE-14207-0.98-V2.patch,
> HBASE-14207-0.98.patch
>
>
> On production environment, following events happened
> 1. Master is trying to assign a region to RS, but due to
> KeeperException$SessionExpiredException RS failed to open the region.
> In RS log, saw multiple WARN log related to
> KeeperException$SessionExpiredException
> > KeeperErrorCode = Session expired for
> /hbase/region-in-transition/08f1935d652e5dbdac09b423b8f9401b
> > Unable to get data of znode
> /hbase/region-in-transition/08f1935d652e5dbdac09b423b8f9401b
> 2. Master retried to assign the region to same RS, but RS again failed.
> 3. On second retry new plan formed and this time plan destination (RS) is
> different, so master send the request to new RS to open the region. But new
> RS failed to open the region as there was server mismatch in ZNODE than the
> expected current server name.
> Logs Snippet:
> {noformat}
> HM
> 2015-07-14 03:50:29,759 | INFO | master:T101PC03VM13:21300 | Processing
> 08f1935d652e5dbdac09b423b8f9401b in state: M_ZK_REGION_OFFLINE |
> org.apache.hadoop.hbase.master.AssignmentManager.processRegionsInTransition(AssignmentManager.java:644)
> 2015-07-14 03:50:29,759 | INFO | master:T101PC03VM13:21300 | Transitioned
> {08f1935d652e5dbdac09b423b8f9401b state=OFFLINE, ts=1436817029679,
> server=null} to {08f1935d652e5dbdac09b423b8f9401b state=PENDING_OPEN,
> ts=1436817029759, server=T101PC03VM13,21302,1436816690692} |
> org.apache.hadoop.hbase.master.RegionStates.updateRegionState(RegionStates.java:327)
> 2015-07-14 03:50:29,760 | INFO | master:T101PC03VM13:21300 | Processed
> region 08f1935d652e5dbdac09b423b8f9401b in state M_ZK_REGION_OFFLINE, on
> server: T101PC03VM13,21302,1436816690692 |
> org.apache.hadoop.hbase.master.AssignmentManager.processRegionsInTransition(AssignmentManager.java:768)
> 2015-07-14 03:50:29,800 | INFO |
> MASTER_SERVER_OPERATIONS-T101PC03VM13:21300-3 | Assigning
> INTER_CONCURRENCY_SETTING,,1436596137981.08f1935d652e5dbdac09b423b8f9401b. to
> T101PC03VM13,21302,1436816690692 |
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1983)
> 2015-07-14 03:50:29,801 | WARN |
> MASTER_SERVER_OPERATIONS-T101PC03VM13:21300-3 | Failed assignment of
> INTER_CONCURRENCY_SETTING,,1436596137981.08f1935d652e5dbdac09b423b8f9401b. to
> T101PC03VM13,21302,1436816690692, trying to assign elsewhere instead; try=1
> of 10 |
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:2077)
> 2015-07-14 03:50:29,802 | INFO |
> MASTER_SERVER_OPERATIONS-T101PC03VM13:21300-3 | Trying to re-assign
> INTER_CONCURRENCY_SETTING,,1436596137981.08f1935d652e5dbdac09b423b8f9401b. to
> the same failed server. |
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:2123)
> 2015-07-14 03:50:31,804 | INFO |
> MASTER_SERVER_OPERATIONS-T101PC03VM13:21300-3 | Assigning
> INTER_CONCURRENCY_SETTING,,1436596137981.08f1935d652e5dbdac09b423b8f9401b. to
> T101PC03VM13,21302,1436816690692 |
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1983)
> 2015-07-14 03:50:31,806 | WARN |
> MASTER_SERVER_OPERATIONS-T101PC03VM13:21300-3 | Failed assignment of
> INTER_CONCURRENCY_SETTING,,1436596137981.08f1935d652e5dbdac09b423b8f9401b. to
> T101PC03VM13,21302,1436816690692, trying to assign elsewhere instead; try=2
> of 10 |
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:2077)
> 2015-07-14 03:50:31,807 | INFO |
> MASTER_SERVER_OPERATIONS-T101PC03VM13:21300-3 | Transitioned
> {08f1935d652e5dbdac09b423b8f9401b state=PENDING_OPEN, ts=1436817031804,
> server=T101PC03VM13,21302,1436816690692} to {08f1935d652e5dbdac09b423b8f9401b
> state=OFFLINE, ts=1436817031807, server=T101PC03VM13,21302,1436816690692} |
> org.apache.hadoop.hbase.master.RegionStates.updateRegionState(RegionStates.java:327)
> 2015-07-14 03:50:31,807 | INFO |
> MASTER_SERVER_OPERATIONS-T101PC03VM13:21300-3 | As
[jira] [Commented] (HBASE-14825) HBase Ref Guide corrections of typos/misspellings
[
https://issues.apache.org/jira/browse/HBASE-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027290#comment-15027290
]
stack commented on HBASE-14825:
---
bq. Is this "FAILURE" message anything to be concerned about?
The FAILURE refers to a run of the total test suite after this patch was
applied. If you look at the 'failure', you'll see that the 1.7 test suite
passed but 1.8 failed on the test
org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover.testSecondaryRegionKillWhilePrimaryIsAcceptingWrites[0]
This test looks unrelated to documentation change and if you click on history
for the test, it passes all but this last run.
Its not you [~daniel_vimont]
> HBase Ref Guide corrections of typos/misspellings
> -
>
> Key: HBASE-14825
> URL: https://issues.apache.org/jira/browse/HBASE-14825
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 2.0.0
>Reporter: Daniel Vimont
>Assignee: Daniel Vimont
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-14825-v2.patch, HBASE-14825-v3.patch,
> HBASE-14825-v4.patch, HBASE-14825-v5-test.patch, HBASE-14825-v6.patch,
> HBASE-14825.patch, HBASE-14825_misty_example.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Found the following list of typos/misspellings on the book.html page, and
> thought I would make corrections to the appropriate src/main/asciidoc files
> in which they are located. (This is just a good opportunity for me to become
> familiar with submission of fixes/patches as a prelude to beginning to make
> some coding contributions. This is also my first submission to the JIRA
> system, so corrections to content/conventions are welcome!)
> [Note: I see that [~misty] may be in the midst of a reformatting task --
> HBASE-14823 -- that might involve these same asciidoc files. Please advise
> if I should wait on this task to avoid a possibly cumbersome Git
> reconciliation mess. (?)]
> Here is the list of typos/misspellings. The format of each item is (a) the
> problem is presented in brackets on the first line, and (b) the phrase (as it
> currently appears in the text) is on the second line.
> ===
> ["you" should be "your", and "Kimballs'" should be "Kimball's" (move the
> apostrophe) in the following:]
> A useful read setting config on you hadoop cluster is Aaron Kimballs'
> Configuration Parameters: What can you just ignore?
> [Period needed after "a"]
> a.k.a pseudo-distributed
> ["empty" is misspelled]
> The default value in this configuration has been intentionally left emtpy in
> order to honor the old hbase.regionserver.global.memstore.upperLimit property
> if present.
> [All occurrences of "a HBase" should be changed to "an HBase" -- 15
> occurrences found]
> ["file path are" should be "file paths are"]
> By default, all of HBase's ZooKeeper file path are configured with a relative
> path, so they will all go under this directory unless changed.
> ["times" -- plural required]
> How many time to retry attempting to write a version file before just
> aborting.
> ["separated" is misspelled]
> Each attempt is seperated by the hbase.server.thread.wakefrequency
> milliseconds.
> [space needed after quotation mark (include"limit)]
> Because this limit represents the "automatic include"limit...
> [space needed ("ashbase:metadata" should be "as hbase:metadata")]
> This helps to keep compaction of lean tables (such ashbase:meta) fast.
> [Acronym "ide" should be capitalized for clarity: IDE]
> Setting this to true can be useful in contexts other than the other side of a
> maven generation; i.e. running in an ide.
> [RuntimeException missing an "e"]
> You'll want to set this boolean to true to avoid seeing the RuntimException
> complaint:
> [Space missing after "secure"]
> FS Permissions for the root directory in a secure(kerberos) setup.
> ["mutations" misspelled]
> ...will be created which will tail the logs and replicate the mutatations to
> region replicas for tables that have region replication > 1.
> ["it such that" should be "is such that"]
> If your working set it such that block cache does you no good...
> ["an" should be "and"]
> See the Deveraj Das an Nicolas Liochon blog post...
> [Tag "" should be ""]
> hbase.coprocessor.master.classes
> [Misspelling of "implementations"]
> Those consumers are coprocessors, phoenix, replication endpoint
> implemnetations or similar.
> [Misspelling of "cluster"]
> On upgrade, before running a rolling restart over the cluser...
> ["effect" should be "affect"]
> If NOT using BucketCache, this change does not effect you.
> [Need space after "throw"]
> This will throw`java.lang.NoSuchMethodError...
> ["habasee" should be "hbase"]
> You can pass commands to the HBase
[jira] [Created] (HBASE-14884) TestSplitTransactionOnCluster.testSSHCleanupDaugtherRegionsOfAbortedSplit is flakey
stack created HBASE-14884: - Summary: TestSplitTransactionOnCluster.testSSHCleanupDaugtherRegionsOfAbortedSplit is flakey Key: HBASE-14884 URL: https://issues.apache.org/jira/browse/HBASE-14884 Project: HBase Issue Type: Sub-task Components: flakey, test Reporter: stack This one fails a bit on master at least: https://builds.apache.org/view/H-L/view/HBase/job/HBase-Trunk_matrix/496/jdk=latest1.8,label=Hadoop/testReport/junit/org.apache.hadoop.hbase.regionserver/TestSplitTransactionOnCluster/testSSHCleanupDaugtherRegionsOfAbortedSplit/history/ ... about 15% of the time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14883) TestSplitTransactionOnCluster#testFailedSplit flakey
[
https://issues.apache.org/jira/browse/HBASE-14883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-14883:
--
Attachment: 14883-branch-1.txt
Remove balance assert on end of test. Just do assert that no RIT. Thats what
the test is about, not about balancing. Balance is being used to assert no
RIT.. just cut to the chase. Table is being removed on end of test anyways so
balance is just work.
> TestSplitTransactionOnCluster#testFailedSplit flakey
>
>
> Key: HBASE-14883
> URL: https://issues.apache.org/jira/browse/HBASE-14883
> Project: HBase
> Issue Type: Sub-task
> Components: flakey, test
>Affects Versions: 1.2.0, 1.3.0
>Reporter: stack
> Attachments: 14883-branch-1.txt
>
>
> Only in branch-1 and branch-1.2.
> Fails look like this:
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.3/jdk=latest1.8,label=Hadoop/397/
> TEST-org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.xml.
> If I look in the xml, I see this:
> {code}
>classname="org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster"
> time="8.275">
> java.lang.AssertionError:
> null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testFailedSplit(TestSplitTransactionOnCluster.java:1339)
>
[jira] [Commented] (HBASE-14883) TestSplitTransactionOnCluster#testFailedSplit flakey
[
https://issues.apache.org/jira/browse/HBASE-14883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027258#comment-15027258
]
stack commented on HBASE-14883:
---
Related, I see this failing on master branch
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testSSHCleanupDaugtherRegionsOfAbortedSplit
Failing for the past 1 build (Since Failed#498 )
Took 4.9 sec.
add description
Error Message
{1ae71d5d6e6159824119325977ade60c={1ae71d5d6e6159824119325977ade60c
state=SPLITTING_NEW, ts=1448422880340,
server=asf901.gq1.ygridcore.net,56231,1448422869502},
e76418269e624d3707be3217f88470ee={e76418269e624d3707be3217f88470ee
state=SPLITTING, ts=1448422880336,
server=asf901.gq1.ygridcore.net,56231,1448422869502},
8455f044bf42015cb0c2b13a4d6fc932={8455f044bf42015cb0c2b13a4d6fc932
state=SPLITTING_NEW, ts=1448422880340,
server=asf901.gq1.ygridcore.net,56231,1448422869502}} expected:<0> but was:<3>
Stacktrace
java.lang.AssertionError:
{1ae71d5d6e6159824119325977ade60c={1ae71d5d6e6159824119325977ade60c
state=SPLITTING_NEW, ts=1448422880340,
server=asf901.gq1.ygridcore.net,56231,1448422869502},
e76418269e624d3707be3217f88470ee={e76418269e624d3707be3217f88470ee
state=SPLITTING, ts=1448422880336,
server=asf901.gq1.ygridcore.net,56231,1448422869502},
8455f044bf42015cb0c2b13a4d6fc932={8455f044bf42015cb0c2b13a4d6fc932
state=SPLITTING_NEW, ts=1448422880340,
server=asf901.gq1.ygridcore.net,56231,1448422869502}} expected:<0> but was:<3>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:645)
at
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testSSHCleanupDaugtherRegionsOfAbortedSplit(TestSplitTransactionOnCluster.java:995)
Let me open an issue for it since it happens a bit.
> TestSplitTransactionOnCluster#testFailedSplit flakey
>
>
> Key: HBASE-14883
> URL: https://issues.apache.org/jira/browse/HBASE-14883
> Project: HBase
> Issue Type: Sub-task
> Components: flakey, test
>Affects Versions: 1.2.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Attachments: 14883-branch-1.txt
>
>
> Only in branch-1 and branch-1.2.
> Fails look like this:
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.3/jdk=latest1.8,label=Hadoop/397/
> TEST-org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.xml.
> If I look in the xml, I see this:
> {code}
>classname="org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster"
> time="8.275">
> java.lang.AssertionError:
> null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testFailedSplit(TestSplitTransactionOnCluster.java:1339)
>
[jira] [Commented] (HBASE-14883) TestSplitTransactionOnCluster#testFailedSplit flakey
[
https://issues.apache.org/jira/browse/HBASE-14883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027247#comment-15027247
]
stack commented on HBASE-14883:
---
Pushed the patch to branch-1.2 and branch-1. Leaving open to see if we fail
again any time soon.
> TestSplitTransactionOnCluster#testFailedSplit flakey
>
>
> Key: HBASE-14883
> URL: https://issues.apache.org/jira/browse/HBASE-14883
> Project: HBase
> Issue Type: Sub-task
> Components: flakey, test
>Affects Versions: 1.2.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Attachments: 14883-branch-1.txt
>
>
> Only in branch-1 and branch-1.2.
> Fails look like this:
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.3/jdk=latest1.8,label=Hadoop/397/
> TEST-org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.xml.
> If I look in the xml, I see this:
> {code}
>classname="org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster"
> time="8.275">
> java.lang.AssertionError:
> null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testFailedSplit(TestSplitTransactionOnCluster.java:1339)
>
