[jira] [Commented] (HBASE-12471) Task 4. replace internal ConnectionManager#{delete,get}Connection use with #close, #createConnection (0.98, 0.99) under src/main/java
[
https://issues.apache.org/jira/browse/HBASE-12471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221865#comment-14221865
]
Hadoop QA commented on HBASE-12471:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683035/12471.reapplyv2.txt
against master branch at commit 7ee4df600bb7ec8d794dd3b4ab2cb933ab15b988.
ATTACHMENT ID: 12683035
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 49 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ if
(!region.getRegionInfo().getEncodedName().equals(toMove.getRegionInfo().getEncodedName())
+
Assert.assertFalse(destServer.getRegionsInTransitionInRS().containsKey(encodedRegionNameBytes));
+
Assert.assertFalse(curServer.getRegionsInTransitionInRS().containsKey(encodedRegionNameBytes));
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestHCM
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11796//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11796//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11796//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11796//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11796//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11796//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11796//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11796//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11796//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11796//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11796//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11796//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/11796//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11796//console
This message is automatically generated.
> Task 4. replace internal ConnectionManager#{delete,get}Connection use with
> #close, #createConnection (0.98, 0.99) under src/main/java
> -
>
> Key: HBASE-12471
> URL: https://issues.apache.org/jira/browse/HBASE-12471
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 0.99.2
>
> Attachments:
> 0001-HBASE-12471-Task-4.-replace-internal-ConnectionManag.patch, 12404v9.txt,
> 124471v4.txt, 12471.reapply.txt, 12471.reapplyv2.txt, 12471.reapplyv2.txt,
> 12471v10.txt, 12471v10.txt, 12471v11.txt, 12471v2.txt, 12471v3.txt,
> 12471v6.txt, 12471v7.txt, 12471v9.txt
>
>
> Let me do this. A bunch of this was done in HBASE-12404 Let me see if can
> find more.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-11992) Backport HBASE-11367 (Pluggable replication endpoint) to 0.98
[ https://issues.apache.org/jira/browse/HBASE-11992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-11992: --- Attachment: HBASE-11992_0.98_3.patch Updated patch that addresses [~virag]'s comment. > Backport HBASE-11367 (Pluggable replication endpoint) to 0.98 > - > > Key: HBASE-11992 > URL: https://issues.apache.org/jira/browse/HBASE-11992 > Project: HBase > Issue Type: Task >Reporter: Andrew Purtell >Assignee: ramkrishna.s.vasudevan > Fix For: 0.98.9 > > Attachments: HBASE-11992_0.98_1.patch, HBASE-11992_0.98_2.patch, > HBASE-11992_0.98_3.patch, hbase-11367_0.98.patch > > > ReplicationSource tails the logs for each peer. HBASE-11367 introduces > ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is > run in the same RS process and instantiated per replication peer per region > server. Implementations of this interface handle the actual shipping of WAL > edits to the remote cluster. > This issue is for backporting HBASE-11367 to 0.98. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12563) After Starting up mini hbase cluster, Real Configuration of Cluster never set to HBaseTestingUtility
[ https://issues.apache.org/jira/browse/HBASE-12563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Talat UYARER updated HBASE-12563: - Attachment: HBASE-12563.patch Hi [~ndimiduk], You know the problem. Can you review my patch ? It should submit all branches. > After Starting up mini hbase cluster, Real Configuration of Cluster never set > to HBaseTestingUtility > - > > Key: HBASE-12563 > URL: https://issues.apache.org/jira/browse/HBASE-12563 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 0.99.1 >Reporter: Talat UYARER >Assignee: Talat UYARER > Attachments: HBASE-12563.patch > > > When I use startMiniHBaseCluster method. It starts up a Mini Cluster for the > tests. While starting It changes some configuration. For example Master's > port or Region Server's. After Cluster starting We should set its new > configuration to HbaseTestingUtils conf. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12563) After Starting up mini hbase cluster, Real Configuration of Cluster never set to HBaseTestingUtility
Talat UYARER created HBASE-12563: Summary: After Starting up mini hbase cluster, Real Configuration of Cluster never set to HBaseTestingUtility Key: HBASE-12563 URL: https://issues.apache.org/jira/browse/HBASE-12563 Project: HBase Issue Type: Bug Affects Versions: 0.99.1, 2.0.0 Reporter: Talat UYARER Assignee: Talat UYARER When I use startMiniHBaseCluster method. It starts up a Mini Cluster for the tests. While starting It changes some configuration. For example Master's port or Region Server's. After Cluster starting We should set its new configuration to HbaseTestingUtils conf. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11992) Backport HBASE-11367 (Pluggable replication endpoint) to 0.98
[ https://issues.apache.org/jira/browse/HBASE-11992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221857#comment-14221857 ] ramkrishna.s.vasudevan commented on HBASE-11992: This JIRA https://issues.apache.org/jira/browse/HBASE-12136 has done some change here. I will update the patch accordingly. Many thanks to @apurtell for the testing and to [~virag] for the reviews. > Backport HBASE-11367 (Pluggable replication endpoint) to 0.98 > - > > Key: HBASE-11992 > URL: https://issues.apache.org/jira/browse/HBASE-11992 > Project: HBase > Issue Type: Task >Reporter: Andrew Purtell >Assignee: ramkrishna.s.vasudevan > Fix For: 0.98.9 > > Attachments: HBASE-11992_0.98_1.patch, HBASE-11992_0.98_2.patch, > hbase-11367_0.98.patch > > > ReplicationSource tails the logs for each peer. HBASE-11367 introduces > ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is > run in the same RS process and instantiated per replication peer per region > server. Implementations of this interface handle the actual shipping of WAL > edits to the remote cluster. > This issue is for backporting HBASE-11367 to 0.98. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12562) Handling memory pressure for secondary region replicas
Enis Soztutar created HBASE-12562: - Summary: Handling memory pressure for secondary region replicas Key: HBASE-12562 URL: https://issues.apache.org/jira/browse/HBASE-12562 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 2.0.0 This issue will track the implementation of how to handle the memory pressure for secondary region replicas. Since the replicas cannot flush by themselves, the region server might get blocked or cause extensive flushing for its primary regions. The design doc attached at HBASE-11183 contains two possible solutions that we can pursue. The first one is to not allow secondary region replicas to not flush by themselves, but instead of needed allow them to refresh their store files on demand (which possibly allows them to drop their memstore snapshots or memstores). The second approach is to allow the secondaries to flush to a temporary space. Both have pros and cons, but for simplicity and to not cause extra write amplification, we have implemented the first approach. More details can be found in the design doc, but we can also discuss other options here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11183) Timeline Consistent region replicas - Phase 2 design
[ https://issues.apache.org/jira/browse/HBASE-11183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-11183: -- Attachment: PhaseIIDesignforHBASE-10070.pdf A little update is in order. Good news is that, we were able to implement most of the stuff in this second phase design, and get it stable enough and useable (although on an earlier code base based on 0.98). I think, over the following weeks (hopefully), we can get most of the patches in master and resolve the remaining subtasks for HBASE-10070. I am attaching an updated design doc, which contains some more details about remaining open tickets under HBASE-10070, esp HBASE-11569, HBASE-11580, HBASE-11842 and memory pressure handling for secondary region replicas. > Timeline Consistent region replicas - Phase 2 design > > > Key: HBASE-11183 > URL: https://issues.apache.org/jira/browse/HBASE-11183 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: PhaseIIDesignforHBASE-10070.pdf, > PhaseIIDesignforHBASE-10070.pdf > > > Now that Phase 1 of parent issue HBASE-10070 is mostly done, it is time to > think about remaining items in Phase 2 as per the design doc > https://issues.apache.org/jira/secure/attachment/12616659/HighAvailabilityDesignforreadsApachedoc.pdf > Phase 2 will conclude the work and include at least the following major > features: > - Async WAL Replication > - Replication Interface changes (HBASE-10504) > - Replication should guarantee seqId order (for region moves and RS failure) > - Flush / Compaction events should be written to WAL > - Flush / Memstore handling from secondary regions > - Region split / merge handling for replicated regions > In this issue, we can discuss the proposed design, and keep it as like a > parent jira for Phase 2 work. We'll open subtasks agains the HBASE-10070 jira > for individual patches. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12556) Create a golden file for testing client API source/binary compatibility
[ https://issues.apache.org/jira/browse/HBASE-12556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221855#comment-14221855 ] Sean Busbey commented on HBASE-12556: - My concern is that we'll be ad-hoc recreating compatibility checks that others have spent way more effort maturing. For example, ATM we don't take into account # sub-class-level audience or stability annotations # source incompatible changes to generics # changes to members # incompatible changes to extends / implements I realize this is a rough draft and we can build these things, but why do it ourselves when we can build on the work of others? We can achieve the automation by expanding on the scripts used to verify patches pre-commit and post-integration with trunk. We can automate it to individual developers builds either by gymnastics to make java api compliance checker work or use something that's already Maven friendly like [Clirr|http://mojo.codehaus.org/clirr-maven-plugin/index.html]. Adding or changing InterfaceAudience annotations is already us explicitly changing our API. Reviewers _should_ be treating it carefully. If they're not, making them do one additional step isn't going to fix that. > Create a golden file for testing client API source/binary compatibility > --- > > Key: HBASE-12556 > URL: https://issues.apache.org/jira/browse/HBASE-12556 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.99.2 > > Attachments: hbase-12556-wip.patch > > > [~lhofhansl] had a suggestion (in some other jira I forgot) for doing a > golden file for the HBase API so that we can compare between releases to > ensure that we are keeping source and binary compatibility as defined in this > document : > https://docs.google.com/document/d/1p5pP7v2OuzSSaomK2S2v7sfKky1Hex6OqwsJO0sZTUY/edit > > I think we can generate a file, commit it to the repo, and create a unit test > to check whether any API's are broken. Adding new InterfaceAudience.Public > interfaces has to modify this file so that it becomes an explicit decision. > The downside is that we have to modify the file every time we add a new API, > but it should be fine since it will force us to think more before committing > to supporting new interfaces within the same major versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12549) Fix TestAssignmentManagerOnCluster#testAssignRacingWithSSH() flaky test
[ https://issues.apache.org/jira/browse/HBASE-12549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221853#comment-14221853 ] Hudson commented on HBASE-12549: FAILURE: Integrated in HBase-1.0 #494 (See [https://builds.apache.org/job/HBase-1.0/494/]) HBASE-12549 Fix TestAssignmentManagerOnCluster#testAssignRacingWithSSH() flaky test (Virag Kothari) --ADDENDUM (stack: rev 7a3e8b55f8b331451c80da93e2f4a6d881db6d35) * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManagerOnCluster.java > Fix TestAssignmentManagerOnCluster#testAssignRacingWithSSH() flaky test > --- > > Key: HBASE-12549 > URL: https://issues.apache.org/jira/browse/HBASE-12549 > Project: HBase > Issue Type: Test >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 0.98.9, 0.99.2 > > Attachments: HBASE-12459-addendum.patch, hbase-12549_v1.patch > > > TestAssignmentManagerOnCluster#testAssignRacingWithSSH() runs on average 40 > sec, with 60 sec timeout. This causes flakiness. > The fix is easy. One of the earlier tests increases the > hbase.assignment.maximum.attempts from 3 to 10. Then > testAssignRacingWithSSH() is ran with 10 attempts causing slowness. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12556) Create a golden file for testing client API source/binary compatibility
[ https://issues.apache.org/jira/browse/HBASE-12556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-12556: -- Attachment: hbase-12556-wip.patch Here is a rough idea for what I had in mind. A simple class to generate the golden file, a unit test to check against this file, and a sample golden file in simple text format. If a patch adds a new API for example, the test will fail, and the author will be responsible to re-create the golden file again (containing the new interfaces) which will make it explicit. If an interface gets deleted, or changed, this UT will catch it. RM's can also do a simple diff against this file to double-check changes between releases. Let me know what you guys think. > Create a golden file for testing client API source/binary compatibility > --- > > Key: HBASE-12556 > URL: https://issues.apache.org/jira/browse/HBASE-12556 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.99.2 > > Attachments: hbase-12556-wip.patch > > > [~lhofhansl] had a suggestion (in some other jira I forgot) for doing a > golden file for the HBase API so that we can compare between releases to > ensure that we are keeping source and binary compatibility as defined in this > document : > https://docs.google.com/document/d/1p5pP7v2OuzSSaomK2S2v7sfKky1Hex6OqwsJO0sZTUY/edit > > I think we can generate a file, commit it to the repo, and create a unit test > to check whether any API's are broken. Adding new InterfaceAudience.Public > interfaces has to modify this file so that it becomes an explicit decision. > The downside is that we have to modify the file every time we add a new API, > but it should be fine since it will force us to think more before committing > to supporting new interfaces within the same major versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12404) Task 5 from parent: Replace internal HTable constructor use with HConnection#getTable (0.98, 0.99)
[
https://issues.apache.org/jira/browse/HBASE-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221846#comment-14221846
]
Hadoop QA commented on HBASE-12404:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683036/12404.v17.txt
against master branch at commit 5911c030a509b8fc6ad5a3735f2e1279712485f7.
ATTACHMENT ID: 12683036
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 117
new or modified tests.
{color:red}-1 javac{color}. The patch appears to cause mvn compile goal to
fail.
Compilation errors resume:
[ERROR] COMPILATION ERROR :
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionManager.java:[777,14]
cannot find symbol
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on
project hbase-client: Compilation failure
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionManager.java:[777,14]
cannot find symbol
[ERROR] symbol: variable RegistryFactory
[ERROR] location: class
org.apache.hadoop.hbase.client.ConnectionManager.HConnectionImplementation
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :hbase-client
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11797//console
This message is automatically generated.
> Task 5 from parent: Replace internal HTable constructor use with
> HConnection#getTable (0.98, 0.99)
> --
>
> Key: HBASE-12404
> URL: https://issues.apache.org/jira/browse/HBASE-12404
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
> Fix For: 0.99.2
>
> Attachments:
> 0001-HBASE-12404-Task-5-from-parent-Replace-internal-HTab.patch, 12404.txt,
> 12404.v16.txt, 12404.v16.txt, 12404.v17.txt, 12404v10.txt, 12404v11.txt,
> 12404v12.txt, 12404v12.txt, 12404v13.txt, 12404v13.txt, 12404v14.txt,
> 12404v15.txt, 12404v2.txt, 12404v3.txt, 12404v5.txt, 12404v6.txt,
> 12404v7.txt, 12404v9.txt
>
>
> Do the step 5. from the [~ndimiduk] list in parent issue. Go through src
> code and change all new HTable to instead be connection.getTable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12534) Wrong region location cache in client after regions are moved
[
https://issues.apache.org/jira/browse/HBASE-12534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221844#comment-14221844
]
Lars Hofhansl commented on HBASE-12534:
---
The purging of the cache is always safe (and cheap) and we got a NSRE.
Does anybody remember the use for MIN_RPC_TIMEOUT? [~stack], [~nkeywal]?
The trunk patch is a bit more involved. We want to make we *only* revoke the
cache when we got a NSRE (or the RegionServer is down).
> Wrong region location cache in client after regions are moved
> -
>
> Key: HBASE-12534
> URL: https://issues.apache.org/jira/browse/HBASE-12534
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Critical
> Labels: client
> Attachments: HBASE-12534-0.94-v1.diff, HBASE-12534-v1.diff
>
>
> In our 0.94 hbase cluster, we found that client got wrong region location
> cache and did not update it after a region is moved to another regionserver.
> The reason is wrong client config and bug in RpcRetryingCaller of hbase
> client.
> The rpc configs are following:
> {code}
> hbase.rpc.timeout=1000
> hbase.client.pause=200
> hbase.client.operation.timeout=1200
> {code}
> But the client retry number is 3
> {code}
> hbase.client.retries.number=3
> {code}
> Assumed that a region is at regionserver A before, and then it is moved to
> regionserver B. The client try to make a call to regionserver A and get an
> NotServingRegionException. For the rety number is not 1, the region server
> location cache is not cleaned. See: RpcRetryingCaller.java#141 and
> RegionServerCallable.java#127
> {code}
> @Override
> public void throwable(Throwable t, boolean retrying) {
> if (t instanceof SocketTimeoutException ||
>
> } else if (t instanceof NotServingRegionException && !retrying) {
> // Purge cache entries for this specific region from hbase:meta cache
> // since we don't call connect(true) when number of retries is 1.
> getConnection().deleteCachedRegionLocation(location);
> }
> }
> {code}
> But the call did not retry and throw an SocketTimeoutException for the time
> the call will take is larger than the operation timeout.See
> RpcRetryingCaller.java#152
> {code}
> expectedSleep = callable.sleep(pause, tries + 1);
> // If, after the planned sleep, there won't be enough time left, we
> stop now.
> long duration = singleCallDuration(expectedSleep);
> if (duration > callTimeout) {
> String msg = "callTimeout=" + callTimeout + ", callDuration=" +
> duration +
> ": " + callable.getExceptionMessageAdditionalDetail();
> throw (SocketTimeoutException)(new
> SocketTimeoutException(msg).initCause(t));
> }
> {code}
> At last, the wrong region location will never be not cleaned up .
> [~lhofhansl]
> In hbase 0.94, the MIN_RPC_TIMEOUT in singleCallDuration is 2000 in default,
> which trigger this bug.
> {code}
> private long singleCallDuration(final long expectedSleep) {
> return (EnvironmentEdgeManager.currentTimeMillis() - this.globalStartTime)
> + MIN_RPC_TIMEOUT + expectedSleep;
> }
> {code}
> But there is risk in master code too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12556) Create a golden file for testing client API source/binary compatibility
[ https://issues.apache.org/jira/browse/HBASE-12556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221842#comment-14221842 ] Enis Soztutar commented on HBASE-12556: --- I would like to make checking of source compatibility automated. And also, I want to make adding new Public APIs explicit so that authors and reviewers think twice about adding new stuff that we have to maintain over the long term. jdiff is great, but it is manual (for now) and it is only done at the time of the release candidate (which is too late I think). That's why I think we should have an internal unit test for testing this. I already have a patch for creating and checking golden files for the client API without any more dependencies (using HBASE-10671 tests). It checks for source compat, and it checks class names with "class" or "interface" prepended. I am not sure whether that it is enough for binary compat check (or we should mark abstract classes as well). In cases where this tool creates false negatives, the author can regenerate the file, and review can verify that the changes are in fact compatible I think. Let me attach what I have now. It is not done yet, but gives an idea of what I am thinking. > Create a golden file for testing client API source/binary compatibility > --- > > Key: HBASE-12556 > URL: https://issues.apache.org/jira/browse/HBASE-12556 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.99.2 > > > [~lhofhansl] had a suggestion (in some other jira I forgot) for doing a > golden file for the HBase API so that we can compare between releases to > ensure that we are keeping source and binary compatibility as defined in this > document : > https://docs.google.com/document/d/1p5pP7v2OuzSSaomK2S2v7sfKky1Hex6OqwsJO0sZTUY/edit > > I think we can generate a file, commit it to the repo, and create a unit test > to check whether any API's are broken. Adding new InterfaceAudience.Public > interfaces has to modify this file so that it becomes an explicit decision. > The downside is that we have to modify the file every time we add a new API, > but it should be fine since it will force us to think more before committing > to supporting new interfaces within the same major versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12404) Task 5 from parent: Replace internal HTable constructor use with HConnection#getTable (0.98, 0.99)
[ https://issues.apache.org/jira/browse/HBASE-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12404: -- Attachment: 12404.v17.txt Last patch missed my fix. > Task 5 from parent: Replace internal HTable constructor use with > HConnection#getTable (0.98, 0.99) > -- > > Key: HBASE-12404 > URL: https://issues.apache.org/jira/browse/HBASE-12404 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack > Fix For: 0.99.2 > > Attachments: > 0001-HBASE-12404-Task-5-from-parent-Replace-internal-HTab.patch, 12404.txt, > 12404.v16.txt, 12404.v16.txt, 12404.v17.txt, 12404v10.txt, 12404v11.txt, > 12404v12.txt, 12404v12.txt, 12404v13.txt, 12404v13.txt, 12404v14.txt, > 12404v15.txt, 12404v2.txt, 12404v3.txt, 12404v5.txt, 12404v6.txt, > 12404v7.txt, 12404v9.txt > > > Do the step 5. from the [~ndimiduk] list in parent issue. Go through src > code and change all new HTable to instead be connection.getTable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-12560) [WINDOWS] Append the classpath from Hadoop to HBase classpath in bin/hbase.cmd
[ https://issues.apache.org/jira/browse/HBASE-12560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar resolved HBASE-12560. --- Resolution: Fixed Hadoop Flags: Reviewed Thanks Stack for looking. I've pushed this to 0.98+. > [WINDOWS] Append the classpath from Hadoop to HBase classpath in bin/hbase.cmd > -- > > Key: HBASE-12560 > URL: https://issues.apache.org/jira/browse/HBASE-12560 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 0.98.9, 0.99.2 > > Attachments: hbase-12560_v1.patch > > > On windows, bin/hbase.cmd does not append the classpath from hadoop (only > uses whatever in hbase's lib dir) while on linux bin/hbase does append the > hadoop classpath to the hbase's classpath. > This patch will do the similar thing for bin/hbase.cmd as well. If lzo or > other library is installed in hadoop, this way we can use it from hbase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12550) Check all storefiles are referenced before splitting
[ https://issues.apache.org/jira/browse/HBASE-12550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221838#comment-14221838 ] stack commented on HBASE-12550: --- The TestBaseLoadBalancer fail is not this patch. > Check all storefiles are referenced before splitting > > > Key: HBASE-12550 > URL: https://issues.apache.org/jira/browse/HBASE-12550 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-12550-v1.patch, HBASE-12550-v2.patch, > HBASE-12550-v3.patch, HBASE-12550.patch > > > Make double extra sure all storefiles are referenced before -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12404) Task 5 from parent: Replace internal HTable constructor use with HConnection#getTable (0.98, 0.99)
[
https://issues.apache.org/jira/browse/HBASE-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221837#comment-14221837
]
Hadoop QA commented on HBASE-12404:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683034/12404.v16.txt
against master branch at commit 7ee4df600bb7ec8d794dd3b4ab2cb933ab15b988.
ATTACHMENT ID: 12683034
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 117
new or modified tests.
{color:red}-1 javac{color}. The patch appears to cause mvn compile goal to
fail.
Compilation errors resume:
[ERROR] COMPILATION ERROR :
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTableWrapper.java:[118,21]
method createGetClosestRowOrBeforeReverseScan in class
org.apache.hadoop.hbase.client.Scan cannot be applied to given types;
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on
project hbase-server: Compilation failure
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTableWrapper.java:[118,21]
method createGetClosestRowOrBeforeReverseScan in class
org.apache.hadoop.hbase.client.Scan cannot be applied to given types;
[ERROR] required: byte[]
[ERROR] found: byte[],byte[]
[ERROR] reason: actual and formal argument lists differ in length
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :hbase-server
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11795//console
This message is automatically generated.
> Task 5 from parent: Replace internal HTable constructor use with
> HConnection#getTable (0.98, 0.99)
> --
>
> Key: HBASE-12404
> URL: https://issues.apache.org/jira/browse/HBASE-12404
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
> Fix For: 0.99.2
>
> Attachments:
> 0001-HBASE-12404-Task-5-from-parent-Replace-internal-HTab.patch, 12404.txt,
> 12404.v16.txt, 12404.v16.txt, 12404v10.txt, 12404v11.txt, 12404v12.txt,
> 12404v12.txt, 12404v13.txt, 12404v13.txt, 12404v14.txt, 12404v15.txt,
> 12404v2.txt, 12404v3.txt, 12404v5.txt, 12404v6.txt, 12404v7.txt, 12404v9.txt
>
>
> Do the step 5. from the [~ndimiduk] list in parent issue. Go through src
> code and change all new HTable to instead be connection.getTable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12522) Backport WAL refactoring to branch-1
[
https://issues.apache.org/jira/browse/HBASE-12522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221836#comment-14221836
]
Enis Soztutar commented on HBASE-12522:
---
Sorry, I've read below as 1 million and one. The last character is lower case
L, not 1? {{100L}} is much more readable.
{code}
source.incrementSyncTime(timeInNanos/100l);
{code}
> Backport WAL refactoring to branch-1
>
>
> Key: HBASE-12522
> URL: https://issues.apache.org/jira/browse/HBASE-12522
> Project: HBase
> Issue Type: Task
> Components: wal
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Fix For: 0.99.2
>
>
> backport HBASE-10378 to branch-1.
> This will let us remove the Deprecated stuff in master, allow some baking
> time within the 1.x line, and will give us the option of pulling back follow
> on performance improvements.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12476) HydraBase Consensus Protocol
[ https://issues.apache.org/jira/browse/HBASE-12476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221834#comment-14221834 ] stack commented on HBASE-12476: --- Thanks for putting the code into a module (thanks also for posting) What direction do you think we should take this contrib? Toward the layout described in the hydrabase doc or are you thinking we'd recast it as an in-cluster quorum-of-regions? If the latter, would it be on all the time -- a different sort of hbase -- or would it be something you could enable? HBase 2.0 or HBase 1.0? Sorry for the big questions. No need of an answer now but we need to all chat. What should we do about swift out here in apache hbase? We'd redo it as pb and rpc calls? airlift is not for REST services, right? You are just using it for stats? You like it? Better than the histograms we have going on out herein apache hbase? You have a Counter, Duration, TimeDistribution, and ExpotentialDecay.. Need airlift for this or could redo doing our current metrics dependencies? Should apache hbase make use of jmxutils too? How you folks using it (Not reviewed the patch yet). Let me get you more feedback. Above are first impression. Thanks. > HydraBase Consensus Protocol > > > Key: HBASE-12476 > URL: https://issues.apache.org/jira/browse/HBASE-12476 > Project: HBase > Issue Type: Sub-task > Components: Consensus, wal >Reporter: Gaurav Menghani >Assignee: Gaurav Menghani > Attachments: 0001-HydraBase-consensus-protocol.patch > > > This is the first patch for the HydraBase consensus protocol implemented > according to the Raft consensus protocol > (https://ramcloud.stanford.edu/raft.pdf), as advertised in (HBASE-12259) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12549) Fix TestAssignmentManagerOnCluster#testAssignRacingWithSSH() flaky test
[ https://issues.apache.org/jira/browse/HBASE-12549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221831#comment-14221831 ] Enis Soztutar commented on HBASE-12549: --- Thanks for tracking this. I've run the test on master and 0.98, but not on branch-1. > Fix TestAssignmentManagerOnCluster#testAssignRacingWithSSH() flaky test > --- > > Key: HBASE-12549 > URL: https://issues.apache.org/jira/browse/HBASE-12549 > Project: HBase > Issue Type: Test >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 0.98.9, 0.99.2 > > Attachments: HBASE-12459-addendum.patch, hbase-12549_v1.patch > > > TestAssignmentManagerOnCluster#testAssignRacingWithSSH() runs on average 40 > sec, with 60 sec timeout. This causes flakiness. > The fix is easy. One of the earlier tests increases the > hbase.assignment.maximum.attempts from 3 to 10. Then > testAssignRacingWithSSH() is ran with 10 attempts causing slowness. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12471) Task 4. replace internal ConnectionManager#{delete,get}Connection use with #close, #createConnection (0.98, 0.99) under src/main/java
[
https://issues.apache.org/jira/browse/HBASE-12471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-12471:
--
Attachment: 12471.reapplyv2.txt
Retry. The TestHCM is an incidence of HBASE-12558. The zombie is just weird.
> Task 4. replace internal ConnectionManager#{delete,get}Connection use with
> #close, #createConnection (0.98, 0.99) under src/main/java
> -
>
> Key: HBASE-12471
> URL: https://issues.apache.org/jira/browse/HBASE-12471
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 0.99.2
>
> Attachments:
> 0001-HBASE-12471-Task-4.-replace-internal-ConnectionManag.patch, 12404v9.txt,
> 124471v4.txt, 12471.reapply.txt, 12471.reapplyv2.txt, 12471.reapplyv2.txt,
> 12471v10.txt, 12471v10.txt, 12471v11.txt, 12471v2.txt, 12471v3.txt,
> 12471v6.txt, 12471v7.txt, 12471v9.txt
>
>
> Let me do this. A bunch of this was done in HBASE-12404 Let me see if can
> find more.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12560) [WINDOWS] Append the classpath from Hadoop to HBase classpath in bin/hbase.cmd
[ https://issues.apache.org/jira/browse/HBASE-12560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221829#comment-14221829 ] stack commented on HBASE-12560: --- LGTM though I know very little about windows .cmd files so take it as is if you tried it @enis... good enough for me. > [WINDOWS] Append the classpath from Hadoop to HBase classpath in bin/hbase.cmd > -- > > Key: HBASE-12560 > URL: https://issues.apache.org/jira/browse/HBASE-12560 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 0.98.9, 0.99.2 > > Attachments: hbase-12560_v1.patch > > > On windows, bin/hbase.cmd does not append the classpath from hadoop (only > uses whatever in hbase's lib dir) while on linux bin/hbase does append the > hadoop classpath to the hbase's classpath. > This patch will do the similar thing for bin/hbase.cmd as well. If lzo or > other library is installed in hadoop, this way we can use it from hbase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12404) Task 5 from parent: Replace internal HTable constructor use with HConnection#getTable (0.98, 0.99)
[ https://issues.apache.org/jira/browse/HBASE-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12404: -- Attachment: 12404.v16.txt Compilation issue. > Task 5 from parent: Replace internal HTable constructor use with > HConnection#getTable (0.98, 0.99) > -- > > Key: HBASE-12404 > URL: https://issues.apache.org/jira/browse/HBASE-12404 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack > Fix For: 0.99.2 > > Attachments: > 0001-HBASE-12404-Task-5-from-parent-Replace-internal-HTab.patch, 12404.txt, > 12404.v16.txt, 12404.v16.txt, 12404v10.txt, 12404v11.txt, 12404v12.txt, > 12404v12.txt, 12404v13.txt, 12404v13.txt, 12404v14.txt, 12404v15.txt, > 12404v2.txt, 12404v3.txt, 12404v5.txt, 12404v6.txt, 12404v7.txt, 12404v9.txt > > > Do the step 5. from the [~ndimiduk] list in parent issue. Go through src > code and change all new HTable to instead be connection.getTable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12479) Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1
[ https://issues.apache.org/jira/browse/HBASE-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221819#comment-14221819 ] stack commented on HBASE-12479: --- I committed the [~virag] addendum over on HBASE-12549. Indeed, stuff was broke on branch-1 ahead of the patch attached to this issue. > Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1 > > > Key: HBASE-12479 > URL: https://issues.apache.org/jira/browse/HBASE-12479 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Reporter: Virag Kothari >Assignee: Virag Kothari > Fix For: 0.98.9, 0.99.2 > > Attachments: HBASE-12479-0.98.patch, HBASE-12479-0.98_v2.patch, > HBASE-12479-branch-1.patch > > > Required for zk-less assignment -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12549) Fix TestAssignmentManagerOnCluster#testAssignRacingWithSSH() flaky test
[ https://issues.apache.org/jira/browse/HBASE-12549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221818#comment-14221818 ] stack commented on HBASE-12549: --- Thank you for the addendum [~virag] That fixed broke branch-1. You the man. Applied to branch-1. I tried 0.98 and it doesn't seem to have this issue. > Fix TestAssignmentManagerOnCluster#testAssignRacingWithSSH() flaky test > --- > > Key: HBASE-12549 > URL: https://issues.apache.org/jira/browse/HBASE-12549 > Project: HBase > Issue Type: Test >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 0.98.9, 0.99.2 > > Attachments: HBASE-12459-addendum.patch, hbase-12549_v1.patch > > > TestAssignmentManagerOnCluster#testAssignRacingWithSSH() runs on average 40 > sec, with 60 sec timeout. This causes flakiness. > The fix is easy. One of the earlier tests increases the > hbase.assignment.maximum.attempts from 3 to 10. Then > testAssignRacingWithSSH() is ran with 10 attempts causing slowness. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12404) Task 5 from parent: Replace internal HTable constructor use with HConnection#getTable (0.98, 0.99)
[
https://issues.apache.org/jira/browse/HBASE-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221816#comment-14221816
]
Hadoop QA commented on HBASE-12404:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683033/12404.v16.txt
against master branch at commit 7ee4df600bb7ec8d794dd3b4ab2cb933ab15b988.
ATTACHMENT ID: 12683033
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 117
new or modified tests.
{color:red}-1 javac{color}. The patch appears to cause mvn compile goal to
fail.
Compilation errors resume:
[ERROR] COMPILATION ERROR :
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTableWrapper.java:[118,21]
method createGetClosestRowOrBeforeReverseScan in class
org.apache.hadoop.hbase.client.Scan cannot be applied to given types;
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on
project hbase-server: Compilation failure
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTableWrapper.java:[118,21]
method createGetClosestRowOrBeforeReverseScan in class
org.apache.hadoop.hbase.client.Scan cannot be applied to given types;
[ERROR] required: byte[]
[ERROR] found: byte[],byte[]
[ERROR] reason: actual and formal argument lists differ in length
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :hbase-server
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11794//console
This message is automatically generated.
> Task 5 from parent: Replace internal HTable constructor use with
> HConnection#getTable (0.98, 0.99)
> --
>
> Key: HBASE-12404
> URL: https://issues.apache.org/jira/browse/HBASE-12404
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
> Fix For: 0.99.2
>
> Attachments:
> 0001-HBASE-12404-Task-5-from-parent-Replace-internal-HTab.patch, 12404.txt,
> 12404.v16.txt, 12404v10.txt, 12404v11.txt, 12404v12.txt, 12404v12.txt,
> 12404v13.txt, 12404v13.txt, 12404v14.txt, 12404v15.txt, 12404v2.txt,
> 12404v3.txt, 12404v5.txt, 12404v6.txt, 12404v7.txt, 12404v9.txt
>
>
> Do the step 5. from the [~ndimiduk] list in parent issue. Go through src
> code and change all new HTable to instead be connection.getTable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12404) Task 5 from parent: Replace internal HTable constructor use with HConnection#getTable (0.98, 0.99)
[ https://issues.apache.org/jira/browse/HBASE-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12404: -- Attachment: 12404.v16.txt Rebase (again!). > Task 5 from parent: Replace internal HTable constructor use with > HConnection#getTable (0.98, 0.99) > -- > > Key: HBASE-12404 > URL: https://issues.apache.org/jira/browse/HBASE-12404 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack > Fix For: 0.99.2 > > Attachments: > 0001-HBASE-12404-Task-5-from-parent-Replace-internal-HTab.patch, 12404.txt, > 12404.v16.txt, 12404v10.txt, 12404v11.txt, 12404v12.txt, 12404v12.txt, > 12404v13.txt, 12404v13.txt, 12404v14.txt, 12404v15.txt, 12404v2.txt, > 12404v3.txt, 12404v5.txt, 12404v6.txt, 12404v7.txt, 12404v9.txt > > > Do the step 5. from the [~ndimiduk] list in parent issue. Go through src > code and change all new HTable to instead be connection.getTable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12557) Introduce timeout mechanism for IP to rack resolution
[
https://issues.apache.org/jira/browse/HBASE-12557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221810#comment-14221810
]
Ted Yu commented on HBASE-12557:
ThreadPoolExecutor can provide count of threads that are executing tasks.
In the following code:
{code}
InetSocketAddress isa = new InetSocketAddress("zxy.org", 321);
try {
InetAddress addr = isa.getAddress();
String ipAddressString = DNS.reverseDns(addr, null);
{code}
if I replace "zxy.org" with "zxyy.org" , isa.getAddress() returns null.
Still looking for a way to make lengthy DNS related call.
> Introduce timeout mechanism for IP to rack resolution
> -
>
> Key: HBASE-12557
> URL: https://issues.apache.org/jira/browse/HBASE-12557
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 12557-v1.txt
>
>
> Config parameter, "hbase.util.ip.to.rack.determiner", determines the class
> which does IP to rack resolution.
> The actual resolution may be lengthy.
> This JIRA is continuation of HBASE-12554 where a mock DNSToSwitchMapping is
> used for rack resolution.
> A timeout parameter, "hbase.ip.to.rack.determiner.timeout", is proposed whose
> value governs the duration which RackManager waits before rack resolution is
> stopped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12522) Backport WAL refactoring to branch-1
[
https://issues.apache.org/jira/browse/HBASE-12522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221797#comment-14221797
]
Sean Busbey commented on HBASE-12522:
-
Sorry, missed this part in my original response.
{quote}
I had a bit hard time understanding the WalProvider vs WalFactory though.
{quote}
Providers are implementations around a particular kind of WAL. They're the
interface a new implementation would use to provide all the integration points
we currently think we need.
Factory is just the entry point in the rest of the code base interacting with
the wal code. It's mostly a matter of abstracting away the instantiation and
management of Providers. That management could have been done with static
methods if we had made WALProvider an abstract class instead of an interface,
but historically I've found it harder to properly handle testing when there's a
bunch of logic tied up in shared static state.
I think some of this distinction is obscured by not having yet taken care of
the replay side and the resultant reaching into the DefaultWALProvider.
> Backport WAL refactoring to branch-1
>
>
> Key: HBASE-12522
> URL: https://issues.apache.org/jira/browse/HBASE-12522
> Project: HBase
> Issue Type: Task
> Components: wal
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Fix For: 0.99.2
>
>
> backport HBASE-10378 to branch-1.
> This will let us remove the Deprecated stuff in master, allow some baking
> time within the 1.x line, and will give us the option of pulling back follow
> on performance improvements.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12522) Backport WAL refactoring to branch-1
[
https://issues.apache.org/jira/browse/HBASE-12522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221793#comment-14221793
]
Sean Busbey commented on HBASE-12522:
-
{quote}
Sean Busbey is the branch-1 patch that different from master? It should not be.
I am going over the master patch in the review board.
{quote}
I don't expect it to be, but I haven't checked how non-clean the pick is yet.
{quote}
I've noticed that you are using yoda conditionals which is not the practice in
hbase. We do not have to fix that for this patch, but it is good to be
consistent throughout the code base.
{code}
if (null == providerId) {
{code}
{quote}
Ah yes. Habit from my C days; hadn't noticed a conscientious choice in HBase.
I'll change my habit and update the code style section of the ref guide.
{quote}
Is this a typo?
{code}
+ public void postSync(final long timeInNanos, final int handlerSyncs) {
+source.incrementSyncTime(timeInNanos/100l);
}
{code}
{quote}
I don't think so? which file is this in? Presuming {{source.incrementSyncTime}}
wants milliseconds that would be the correct conversion from nanos.
> Backport WAL refactoring to branch-1
>
>
> Key: HBASE-12522
> URL: https://issues.apache.org/jira/browse/HBASE-12522
> Project: HBase
> Issue Type: Task
> Components: wal
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Fix For: 0.99.2
>
>
> backport HBASE-10378 to branch-1.
> This will let us remove the Deprecated stuff in master, allow some baking
> time within the 1.x line, and will give us the option of pulling back follow
> on performance improvements.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12479) Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1
[ https://issues.apache.org/jira/browse/HBASE-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221776#comment-14221776 ] Virag Kothari commented on HBASE-12479: --- Trying to fix this on HBASE-12549. Please take a look there. Thanks > Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1 > > > Key: HBASE-12479 > URL: https://issues.apache.org/jira/browse/HBASE-12479 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Reporter: Virag Kothari >Assignee: Virag Kothari > Fix For: 0.98.9, 0.99.2 > > Attachments: HBASE-12479-0.98.patch, HBASE-12479-0.98_v2.patch, > HBASE-12479-branch-1.patch > > > Required for zk-less assignment -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12554) TestBaseLoadBalancer may timeout due to lengthy rack lookup
[
https://issues.apache.org/jira/browse/HBASE-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221772#comment-14221772
]
Hudson commented on HBASE-12554:
FAILURE: Integrated in HBase-1.0 #493 (See
[https://builds.apache.org/job/HBase-1.0/493/])
HBASE-12554 TestBaseLoadBalancer may timeout due to lengthy rack lookup (tedyu:
rev d5bcf338d67fb160cc81e3fcf3284b6c887db11a)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestBaseLoadBalancer.java
> TestBaseLoadBalancer may timeout due to lengthy rack lookup
> ---
>
> Key: HBASE-12554
> URL: https://issues.apache.org/jira/browse/HBASE-12554
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 0.99.2
>
> Attachments: 12554-v1.txt, 12554-v2.txt, 12554-v3.txt, 12554-v4.txt,
> 12554-v5.txt
>
>
> Here is one of the recent occurrences
> (https://builds.apache.org/job/PreCommit-HBASE-Build/11778/console):
> {code}
> testImmediateAssignment(org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer)
> Time elapsed: 30.019 sec <<< ERROR!
> java.lang.Exception: test timed out after 3 milliseconds
> at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
> at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901)
> at
> java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293)
> at java.net.InetAddress.getAllByName0(InetAddress.java:1246)
> at java.net.InetAddress.getAllByName(InetAddress.java:1162)
> at java.net.InetAddress.getAllByName(InetAddress.java:1098)
> at java.net.InetAddress.getByName(InetAddress.java:1048)
> at org.apache.hadoop.net.NetUtils.normalizeHostName(NetUtils.java:561)
> at org.apache.hadoop.net.NetUtils.normalizeHostNames(NetUtils.java:578)
> at
> org.apache.hadoop.net.CachedDNSToSwitchMapping.resolve(CachedDNSToSwitchMapping.java:109)
> at
> org.apache.hadoop.hbase.master.RackManager.getRack(RackManager.java:66)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.(BaseLoadBalancer.java:273)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.createCluster(BaseLoadBalancer.java:1113)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.randomAssignment(BaseLoadBalancer.java:1175)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.immediateAssignment(BaseLoadBalancer.java:1145)
> at
> org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer.testImmediateAssignment(TestBaseLoadBalancer.java:136)
> {code}
> One possible fix is to submit CachedDNSToSwitchMapping.resolve() to executor
> pool for execution. RackManager.getRack() can set a timeout beyond which
> UNKNOWN_RACK is returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12561) Replicas of regions can be cached from different instances of the table in MetaCache
Enis Soztutar created HBASE-12561: - Summary: Replicas of regions can be cached from different instances of the table in MetaCache Key: HBASE-12561 URL: https://issues.apache.org/jira/browse/HBASE-12561 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 2.0.0 In testing, we have observed that if a client caches some regions with replicas in MetaCache, and then the table is recreated, the cache does not get invalidated. Upon caching more entries from a new instance of the table we can have entries for replicas from different instances of a table (remember that metacache is row based) On async wal replay (HBASE-11568), we have an elaborate mechanism to track region replica locations and skip replaying wal entries if the current region replica is not the same region replica from the original wal edit. This prevents replaying entries of older tables with the same name to the region replicas as well as replaying older entries from split (or merged) regions from old parent to new daughters. Trace level logging in my test setup showed that we were skipping some entries because of a case, where we would have region locations (of the same range) cached from different instances of the table (meaning some replicas are from already deleted table vs some entries from new table). This was causing superfluous skipping entries. The fix can be when we are merging entries for the same range, we simply check the other entries and delete those that do not match the new region id. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12549) Fix TestAssignmentManagerOnCluster#testAssignRacingWithSSH() flaky test
[ https://issues.apache.org/jira/browse/HBASE-12549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari updated HBASE-12549: -- Attachment: HBASE-12459-addendum.patch TestAssignmentManagerOnCluster.testAssignRegionBySSH is failing in 1.0 builds because of this (E.g https://builds.apache.org/job/HBase-1.0/487/). Attached patch tries to fix this by waiting for master to be active and initialized. > Fix TestAssignmentManagerOnCluster#testAssignRacingWithSSH() flaky test > --- > > Key: HBASE-12549 > URL: https://issues.apache.org/jira/browse/HBASE-12549 > Project: HBase > Issue Type: Test >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 0.98.9, 0.99.2 > > Attachments: HBASE-12459-addendum.patch, hbase-12549_v1.patch > > > TestAssignmentManagerOnCluster#testAssignRacingWithSSH() runs on average 40 > sec, with 60 sec timeout. This causes flakiness. > The fix is easy. One of the earlier tests increases the > hbase.assignment.maximum.attempts from 3 to 10. Then > testAssignRacingWithSSH() is ran with 10 attempts causing slowness. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12560) [WINDOWS] Append the classpath from Hadoop to HBase classpath in bin/hbase.cmd
[ https://issues.apache.org/jira/browse/HBASE-12560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-12560: -- Attachment: hbase-12560_v1.patch Attaching simple patch. Tested on a windows cluster. > [WINDOWS] Append the classpath from Hadoop to HBase classpath in bin/hbase.cmd > -- > > Key: HBASE-12560 > URL: https://issues.apache.org/jira/browse/HBASE-12560 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 0.98.9, 0.99.2 > > Attachments: hbase-12560_v1.patch > > > On windows, bin/hbase.cmd does not append the classpath from hadoop (only > uses whatever in hbase's lib dir) while on linux bin/hbase does append the > hadoop classpath to the hbase's classpath. > This patch will do the similar thing for bin/hbase.cmd as well. If lzo or > other library is installed in hadoop, this way we can use it from hbase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12560) [WINDOWS] Append the classpath from Hadoop to HBase classpath in bin/hbase.cmd
Enis Soztutar created HBASE-12560: - Summary: [WINDOWS] Append the classpath from Hadoop to HBase classpath in bin/hbase.cmd Key: HBASE-12560 URL: https://issues.apache.org/jira/browse/HBASE-12560 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 2.0.0, 0.98.9, 0.99.2 On windows, bin/hbase.cmd does not append the classpath from hadoop (only uses whatever in hbase's lib dir) while on linux bin/hbase does append the hadoop classpath to the hbase's classpath. This patch will do the similar thing for bin/hbase.cmd as well. If lzo or other library is installed in hadoop, this way we can use it from hbase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12479) Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1
[ https://issues.apache.org/jira/browse/HBASE-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221757#comment-14221757 ] Virag Kothari commented on HBASE-12479: --- I checked and this is caused by HBASE-12549. The pre-commit builds for 1.0 are failing with same error (E.g https://builds.apache.org/job/HBase-1.0/487/) > Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1 > > > Key: HBASE-12479 > URL: https://issues.apache.org/jira/browse/HBASE-12479 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Reporter: Virag Kothari >Assignee: Virag Kothari > Fix For: 0.98.9, 0.99.2 > > Attachments: HBASE-12479-0.98.patch, HBASE-12479-0.98_v2.patch, > HBASE-12479-branch-1.patch > > > Required for zk-less assignment -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12554) TestBaseLoadBalancer may timeout due to lengthy rack lookup
[
https://issues.apache.org/jira/browse/HBASE-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221750#comment-14221750
]
Hudson commented on HBASE-12554:
FAILURE: Integrated in HBase-TRUNK #5810 (See
[https://builds.apache.org/job/HBase-TRUNK/5810/])
HBASE-12554 TestBaseLoadBalancer may timeout due to lengthy rack lookup (tedyu:
rev 7ee4df600bb7ec8d794dd3b4ab2cb933ab15b988)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestBaseLoadBalancer.java
> TestBaseLoadBalancer may timeout due to lengthy rack lookup
> ---
>
> Key: HBASE-12554
> URL: https://issues.apache.org/jira/browse/HBASE-12554
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 0.99.2
>
> Attachments: 12554-v1.txt, 12554-v2.txt, 12554-v3.txt, 12554-v4.txt,
> 12554-v5.txt
>
>
> Here is one of the recent occurrences
> (https://builds.apache.org/job/PreCommit-HBASE-Build/11778/console):
> {code}
> testImmediateAssignment(org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer)
> Time elapsed: 30.019 sec <<< ERROR!
> java.lang.Exception: test timed out after 3 milliseconds
> at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
> at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901)
> at
> java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293)
> at java.net.InetAddress.getAllByName0(InetAddress.java:1246)
> at java.net.InetAddress.getAllByName(InetAddress.java:1162)
> at java.net.InetAddress.getAllByName(InetAddress.java:1098)
> at java.net.InetAddress.getByName(InetAddress.java:1048)
> at org.apache.hadoop.net.NetUtils.normalizeHostName(NetUtils.java:561)
> at org.apache.hadoop.net.NetUtils.normalizeHostNames(NetUtils.java:578)
> at
> org.apache.hadoop.net.CachedDNSToSwitchMapping.resolve(CachedDNSToSwitchMapping.java:109)
> at
> org.apache.hadoop.hbase.master.RackManager.getRack(RackManager.java:66)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.(BaseLoadBalancer.java:273)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.createCluster(BaseLoadBalancer.java:1113)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.randomAssignment(BaseLoadBalancer.java:1175)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.immediateAssignment(BaseLoadBalancer.java:1145)
> at
> org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer.testImmediateAssignment(TestBaseLoadBalancer.java:136)
> {code}
> One possible fix is to submit CachedDNSToSwitchMapping.resolve() to executor
> pool for execution. RackManager.getRack() can set a timeout beyond which
> UNKNOWN_RACK is returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12522) Backport WAL refactoring to branch-1
[
https://issues.apache.org/jira/browse/HBASE-12522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221744#comment-14221744
]
Enis Soztutar commented on HBASE-12522:
---
I've reviewed the the patch for branch-1. It will be good to backport this. I
had a bit hard time understanding the WalProvider vs WalFactory though.
I've noticed that you are using yoda conditionals which is not the practice in
hbase. We do not have to fix that for this patch, but it is good to be
consistent throughout the code base.
{code}
if (null == providerId) {
{code}
Is this a typo?
{code}
+ public void postSync(final long timeInNanos, final int handlerSyncs) {
+source.incrementSyncTime(timeInNanos/100l);
}
{code}
> Backport WAL refactoring to branch-1
>
>
> Key: HBASE-12522
> URL: https://issues.apache.org/jira/browse/HBASE-12522
> Project: HBase
> Issue Type: Task
> Components: wal
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Fix For: 0.99.2
>
>
> backport HBASE-10378 to branch-1.
> This will let us remove the Deprecated stuff in master, allow some baking
> time within the 1.x line, and will give us the option of pulling back follow
> on performance improvements.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12471) Task 4. replace internal ConnectionManager#{delete,get}Connection use with #close, #createConnection (0.98, 0.99) under src/main/java
[
https://issues.apache.org/jira/browse/HBASE-12471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221742#comment-14221742
]
Hadoop QA commented on HBASE-12471:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683000/12471.reapplyv2.txt
against master branch at commit 882324dbcc224072be6a43a74aa6102637e529f8.
ATTACHMENT ID: 12683000
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 49 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ if
(!region.getRegionInfo().getEncodedName().equals(toMove.getRegionInfo().getEncodedName())
+
Assert.assertFalse(destServer.getRegionsInTransitionInRS().containsKey(encodedRegionNameBytes));
+
Assert.assertFalse(curServer.getRegionsInTransitionInRS().containsKey(encodedRegionNameBytes));
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestHCM
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.coprocessor.TestMasterObserver.testRegionTransitionOperations(TestMasterObserver.java:1523)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11790//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11790//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11790//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11790//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11790//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11790//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11790//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11790//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11790//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11790//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11790//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11790//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/11790//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11790//console
This message is automatically generated.
> Task 4. replace internal ConnectionManager#{delete,get}Connection use with
> #close, #createConnection (0.98, 0.99) under src/main/java
> -
>
> Key: HBASE-12471
> URL: https://issues.apache.org/jira/browse/HBASE-12471
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 0.99.2
>
> Attachments:
> 0001-HBASE-12471-Task-4.-replace-internal-ConnectionManag.patch, 12404v9.txt,
> 124471v4.txt, 12471.reapply.txt, 12471.reapplyv2.txt, 12471v10.txt,
> 12471v10.txt, 12471v11.txt, 12471v2.txt, 12471v3.txt, 12471v6.txt,
> 12471v7.txt, 12471v9.txt
>
>
> Let me do thi
[jira] [Updated] (HBASE-12479) Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1
[ https://issues.apache.org/jira/browse/HBASE-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-12479: --- Status: Open (was: Patch Available) > Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1 > > > Key: HBASE-12479 > URL: https://issues.apache.org/jira/browse/HBASE-12479 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Reporter: Virag Kothari >Assignee: Virag Kothari > Fix For: 0.98.9, 0.99.2 > > Attachments: HBASE-12479-0.98.patch, HBASE-12479-0.98_v2.patch, > HBASE-12479-branch-1.patch > > > Required for zk-less assignment -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12479) Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1
[
https://issues.apache.org/jira/browse/HBASE-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221740#comment-14221740
]
Andrew Purtell commented on HBASE-12479:
I get this testing the branch-1 patch locally:
{noformat}
Tests run: 21, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 43.122 sec <<<
FAILURE! - in org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster
testAssignRegionBySSH(org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster)
Time elapsed: 4.823 sec <<< ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster.testAssignRegionBySSH(TestAssignmentManagerOnCluster.java:239)
{noformat}
The 0.98 patch tests out ok.
Holding off commit.
> Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1
>
>
> Key: HBASE-12479
> URL: https://issues.apache.org/jira/browse/HBASE-12479
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Reporter: Virag Kothari
>Assignee: Virag Kothari
> Fix For: 0.98.9, 0.99.2
>
> Attachments: HBASE-12479-0.98.patch, HBASE-12479-0.98_v2.patch,
> HBASE-12479-branch-1.patch
>
>
> Required for zk-less assignment
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12550) Check all storefiles are referenced before splitting
[
https://issues.apache.org/jira/browse/HBASE-12550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221738#comment-14221738
]
Hadoop QA commented on HBASE-12550:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12682999/HBASE-12550-v3.patch
against master branch at commit 882324dbcc224072be6a43a74aa6102637e529f8.
ATTACHMENT ID: 12682999
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 8 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
3807 checkstyle errors (more than the master's current 3805 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ HRegion createDaughterRegionFromSplits(final HRegionInfo hri, int
expectedReferenceFileCount) throws IOException {
+expectedReferenceFileCount !=
FSUtils.getRegionReferenceFileCount(this.parent.getFilesystem(), dir)) {
+ private Pair splitStoreFile(final byte[] family, final StoreFile
sf) throws IOException {
+ public static List getReferenceFilePaths(final FileSystem fs, final
Path familyDir) throws IOException {
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11789//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11789//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11789//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11789//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11789//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11789//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11789//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11789//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11789//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11789//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11789//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11789//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/11789//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11789//console
This message is automatically generated.
> Check all storefiles are referenced before splitting
>
>
> Key: HBASE-12550
> URL: https://issues.apache.org/jira/browse/HBASE-12550
> Project: HBase
> Issue Type: Bug
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Attachments: HBASE-12550-v1.patch, HBASE-12550-v2.patch,
> HBASE-12550-v3.patch, HBASE-12550.patch
>
>
> Make double extra sure all storefiles are referenced before
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12404) Task 5 from parent: Replace internal HTable constructor use with HConnection#getTable (0.98, 0.99)
[
https://issues.apache.org/jira/browse/HBASE-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221717#comment-14221717
]
Hadoop QA commented on HBASE-12404:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683017/12404v15.txt
against master branch at commit 7ee4df600bb7ec8d794dd3b4ab2cb933ab15b988.
ATTACHMENT ID: 12683017
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 120
new or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11793//console
This message is automatically generated.
> Task 5 from parent: Replace internal HTable constructor use with
> HConnection#getTable (0.98, 0.99)
> --
>
> Key: HBASE-12404
> URL: https://issues.apache.org/jira/browse/HBASE-12404
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
> Fix For: 0.99.2
>
> Attachments:
> 0001-HBASE-12404-Task-5-from-parent-Replace-internal-HTab.patch, 12404.txt,
> 12404v10.txt, 12404v11.txt, 12404v12.txt, 12404v12.txt, 12404v13.txt,
> 12404v13.txt, 12404v14.txt, 12404v15.txt, 12404v2.txt, 12404v3.txt,
> 12404v5.txt, 12404v6.txt, 12404v7.txt, 12404v9.txt
>
>
> Do the step 5. from the [~ndimiduk] list in parent issue. Go through src
> code and change all new HTable to instead be connection.getTable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11992) Backport HBASE-11367 (Pluggable replication endpoint) to 0.98
[ https://issues.apache.org/jira/browse/HBASE-11992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221708#comment-14221708 ] Andrew Purtell commented on HBASE-11992: bq. Only one problem I could notice in TableCFsTracker: Alright, holding off commit until [~ram_krish] can comment on this and/or provide an updated patch. > Backport HBASE-11367 (Pluggable replication endpoint) to 0.98 > - > > Key: HBASE-11992 > URL: https://issues.apache.org/jira/browse/HBASE-11992 > Project: HBase > Issue Type: Task >Reporter: Andrew Purtell >Assignee: ramkrishna.s.vasudevan > Fix For: 0.98.9 > > Attachments: HBASE-11992_0.98_1.patch, HBASE-11992_0.98_2.patch, > hbase-11367_0.98.patch > > > ReplicationSource tails the logs for each peer. HBASE-11367 introduces > ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is > run in the same RS process and instantiated per replication peer per region > server. Implementations of this interface handle the actual shipping of WAL > edits to the remote cluster. > This issue is for backporting HBASE-11367 to 0.98. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12550) Check all storefiles are referenced before splitting
[
https://issues.apache.org/jira/browse/HBASE-12550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221706#comment-14221706
]
Hadoop QA commented on HBASE-12550:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12682993/HBASE-12550-v2.patch
against master branch at commit 882324dbcc224072be6a43a74aa6102637e529f8.
ATTACHMENT ID: 12682993
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 8 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
3810 checkstyle errors (more than the master's current 3805 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ HRegion createDaughterRegionFromSplits(final HRegionInfo hri, int
expectedReferenceFileCount) throws IOException {
+expectedReferenceFileCount !=
FSUtils.getRegionReferenceFileCount(this.parent.getFilesystem(), dir)) {
+ private Pair splitStoreFile(final byte[] family, final StoreFile
sf) throws IOException {
+ public static List getReferenceFilePaths(final FileSystem fs, final
Path familyDir) throws IOException {
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11788//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11788//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11788//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11788//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11788//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11788//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11788//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11788//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11788//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11788//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11788//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11788//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/11788//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11788//console
This message is automatically generated.
> Check all storefiles are referenced before splitting
>
>
> Key: HBASE-12550
> URL: https://issues.apache.org/jira/browse/HBASE-12550
> Project: HBase
> Issue Type: Bug
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Attachments: HBASE-12550-v1.patch, HBASE-12550-v2.patch,
> HBASE-12550-v3.patch, HBASE-12550.patch
>
>
> Make double extra sure all storefiles are referenced before
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12550) Check all storefiles are referenced before splitting
[ https://issues.apache.org/jira/browse/HBASE-12550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221704#comment-14221704 ] stack commented on HBASE-12550: --- bq. My speluing is afwul. amlost as bad as myne > Check all storefiles are referenced before splitting > > > Key: HBASE-12550 > URL: https://issues.apache.org/jira/browse/HBASE-12550 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-12550-v1.patch, HBASE-12550-v2.patch, > HBASE-12550-v3.patch, HBASE-12550.patch > > > Make double extra sure all storefiles are referenced before -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12404) Task 5 from parent: Replace internal HTable constructor use with HConnection#getTable (0.98, 0.99)
[ https://issues.apache.org/jira/browse/HBASE-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12404: -- Attachment: 12404v15.txt Fixed my replacement for getRowOrClosest. See what this fixes. > Task 5 from parent: Replace internal HTable constructor use with > HConnection#getTable (0.98, 0.99) > -- > > Key: HBASE-12404 > URL: https://issues.apache.org/jira/browse/HBASE-12404 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack > Fix For: 0.99.2 > > Attachments: > 0001-HBASE-12404-Task-5-from-parent-Replace-internal-HTab.patch, 12404.txt, > 12404v10.txt, 12404v11.txt, 12404v12.txt, 12404v12.txt, 12404v13.txt, > 12404v13.txt, 12404v14.txt, 12404v15.txt, 12404v2.txt, 12404v3.txt, > 12404v5.txt, 12404v6.txt, 12404v7.txt, 12404v9.txt > > > Do the step 5. from the [~ndimiduk] list in parent issue. Go through src > code and change all new HTable to instead be connection.getTable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11992) Backport HBASE-11367 (Pluggable replication endpoint) to 0.98
[
https://issues.apache.org/jira/browse/HBASE-11992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221692#comment-14221692
]
Virag Kothari commented on HBASE-11992:
---
Skimmed the patch and looks fine to me.
Only one problem I could notice in TableCFsTracker:
{code}
@Override
public synchronized void nodeDataChanged(String path) {
if (path.equals(node)) {
super.nodeDataChanged(path);
readTableCFsZnode();
}
}
{code}
This isn't right. Master branch doesn't have readTableCFsZnode() in this
method.
> Backport HBASE-11367 (Pluggable replication endpoint) to 0.98
> -
>
> Key: HBASE-11992
> URL: https://issues.apache.org/jira/browse/HBASE-11992
> Project: HBase
> Issue Type: Task
>Reporter: Andrew Purtell
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.98.9
>
> Attachments: HBASE-11992_0.98_1.patch, HBASE-11992_0.98_2.patch,
> hbase-11367_0.98.patch
>
>
> ReplicationSource tails the logs for each peer. HBASE-11367 introduces
> ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is
> run in the same RS process and instantiated per replication peer per region
> server. Implementations of this interface handle the actual shipping of WAL
> edits to the remote cluster.
> This issue is for backporting HBASE-11367 to 0.98.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-11099) Two situations where we could open a region with smaller sequence number
[
https://issues.apache.org/jira/browse/HBASE-11099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-11099:
---
Fix Version/s: 0.98.9
> Two situations where we could open a region with smaller sequence number
>
>
> Key: HBASE-11099
> URL: https://issues.apache.org/jira/browse/HBASE-11099
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.99.1
>Reporter: Jeffrey Zhong
>Assignee: Stephen Yuan Jiang
> Fix For: 2.0.0, 0.98.9, 0.99.2
>
> Attachments: HBASE-11099.v1-2.0.patch
>
>
> Recently I happened to run into code where we potentially could open region
> with smaller sequence number:
> 1) Inside function: HRegion#internalFlushcache. This is due to we change the
> way WAL Sync where we use late binding(assign sequence number right before
> wal sync).
> The flushSeqId may less than the change sequence number included in the flush
> which may cause later region opening code to use a smaller than expected
> sequence number when we reopen the region.
> {code}
> flushSeqId = this.sequenceId.incrementAndGet();
> ...
> mvcc.waitForRead(w);
> {code}
> 2) HRegion#replayRecoveredEdits where we have following code:
> {code}
> ...
> if (coprocessorHost != null) {
> status.setStatus("Running pre-WAL-restore hook in coprocessors");
> if (coprocessorHost.preWALRestore(this.getRegionInfo(), key,
> val)) {
> // if bypass this log entry, ignore it ...
> continue;
> }
> }
> ...
> currentEditSeqId = key.getLogSeqNum();
> {code}
> If coprocessor skip some tail WALEdits, then the function will return smaller
> currentEditSeqId. In the end, a region may also open with a smaller sequence
> number. This may cause data loss because Master may record a larger flushed
> sequence Id and some WALEdits maybe skipped during recovery if the region
> fail again.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (HBASE-11099) Two situations where we could open a region with smaller sequence number
[
https://issues.apache.org/jira/browse/HBASE-11099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell reopened HBASE-11099:
+1 for 0.98. Reopening. Apologies that I got behind with this. If you provide a
0.98 patch I will commit it
> Two situations where we could open a region with smaller sequence number
>
>
> Key: HBASE-11099
> URL: https://issues.apache.org/jira/browse/HBASE-11099
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.99.1
>Reporter: Jeffrey Zhong
>Assignee: Stephen Yuan Jiang
> Fix For: 2.0.0, 0.98.9, 0.99.2
>
> Attachments: HBASE-11099.v1-2.0.patch
>
>
> Recently I happened to run into code where we potentially could open region
> with smaller sequence number:
> 1) Inside function: HRegion#internalFlushcache. This is due to we change the
> way WAL Sync where we use late binding(assign sequence number right before
> wal sync).
> The flushSeqId may less than the change sequence number included in the flush
> which may cause later region opening code to use a smaller than expected
> sequence number when we reopen the region.
> {code}
> flushSeqId = this.sequenceId.incrementAndGet();
> ...
> mvcc.waitForRead(w);
> {code}
> 2) HRegion#replayRecoveredEdits where we have following code:
> {code}
> ...
> if (coprocessorHost != null) {
> status.setStatus("Running pre-WAL-restore hook in coprocessors");
> if (coprocessorHost.preWALRestore(this.getRegionInfo(), key,
> val)) {
> // if bypass this log entry, ignore it ...
> continue;
> }
> }
> ...
> currentEditSeqId = key.getLogSeqNum();
> {code}
> If coprocessor skip some tail WALEdits, then the function will return smaller
> currentEditSeqId. In the end, a region may also open with a smaller sequence
> number. This may cause data loss because Master may record a larger flushed
> sequence Id and some WALEdits maybe skipped during recovery if the region
> fail again.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-10844) Coprocessor failure during batchmutation leaves the memstore datastructs in an inconsistent state
[ https://issues.apache.org/jira/browse/HBASE-10844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-10844: Attachment: 10844-1-0.98.txt > Coprocessor failure during batchmutation leaves the memstore datastructs in > an inconsistent state > - > > Key: HBASE-10844 > URL: https://issues.apache.org/jira/browse/HBASE-10844 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Devaraj Das >Assignee: Devaraj Das > Attachments: 10844-1-0.98.txt, 10844-1.txt > > > Observed this in the testing with Phoenix. The test in Phoenix - > MutableIndexFailureIT deliberately fails the batchmutation call via the > installed coprocessor. But the update is not rolled back. That leaves the > memstore inconsistent. In particular, I observed that getFlushableSize is > updated before the coprocessor was called but the update is not rolled back. > When the region is being closed at some later point, the assert introduced in > HBASE-10514 in the HRegion.doClose() causes the RegionServer to shutdown > abnormally. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-10844) Coprocessor failure during batchmutation leaves the memstore datastructs in an inconsistent state
[ https://issues.apache.org/jira/browse/HBASE-10844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-10844: Attachment: 10844-1-0.98.txt Patch for 0.98. > Coprocessor failure during batchmutation leaves the memstore datastructs in > an inconsistent state > - > > Key: HBASE-10844 > URL: https://issues.apache.org/jira/browse/HBASE-10844 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Devaraj Das >Assignee: Devaraj Das > Attachments: 10844-1.txt > > > Observed this in the testing with Phoenix. The test in Phoenix - > MutableIndexFailureIT deliberately fails the batchmutation call via the > installed coprocessor. But the update is not rolled back. That leaves the > memstore inconsistent. In particular, I observed that getFlushableSize is > updated before the coprocessor was called but the update is not rolled back. > When the region is being closed at some later point, the assert introduced in > HBASE-10514 in the HRegion.doClose() causes the RegionServer to shutdown > abnormally. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-10844) Coprocessor failure during batchmutation leaves the memstore datastructs in an inconsistent state
[ https://issues.apache.org/jira/browse/HBASE-10844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-10844: Attachment: (was: 10844-1-0.98.txt) > Coprocessor failure during batchmutation leaves the memstore datastructs in > an inconsistent state > - > > Key: HBASE-10844 > URL: https://issues.apache.org/jira/browse/HBASE-10844 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Devaraj Das >Assignee: Devaraj Das > Attachments: 10844-1.txt > > > Observed this in the testing with Phoenix. The test in Phoenix - > MutableIndexFailureIT deliberately fails the batchmutation call via the > installed coprocessor. But the update is not rolled back. That leaves the > memstore inconsistent. In particular, I observed that getFlushableSize is > updated before the coprocessor was called but the update is not rolled back. > When the region is being closed at some later point, the assert introduced in > HBASE-10514 in the HRegion.doClose() causes the RegionServer to shutdown > abnormally. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12556) Create a golden file for testing client API source/binary compatibility
[ https://issues.apache.org/jira/browse/HBASE-12556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221678#comment-14221678 ] Andrew Purtell commented on HBASE-12556: We also might have error-prone (HBASE-12187) at our disposal. There we are waiting at HBASE-12349 on a new release of the tool that would make extension easy. (But if that doesn't materialize we can fork it.) Could bail on "unapproved" changes to public APIs or classes that they depend on. To be defined how that status is determined and changes are approved. > Create a golden file for testing client API source/binary compatibility > --- > > Key: HBASE-12556 > URL: https://issues.apache.org/jira/browse/HBASE-12556 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.99.2 > > > [~lhofhansl] had a suggestion (in some other jira I forgot) for doing a > golden file for the HBase API so that we can compare between releases to > ensure that we are keeping source and binary compatibility as defined in this > document : > https://docs.google.com/document/d/1p5pP7v2OuzSSaomK2S2v7sfKky1Hex6OqwsJO0sZTUY/edit > > I think we can generate a file, commit it to the repo, and create a unit test > to check whether any API's are broken. Adding new InterfaceAudience.Public > interfaces has to modify this file so that it becomes an explicit decision. > The downside is that we have to modify the file every time we add a new API, > but it should be fine since it will force us to think more before committing > to supporting new interfaces within the same major versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12479) Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1
[
https://issues.apache.org/jira/browse/HBASE-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221675#comment-14221675
]
Hadoop QA commented on HBASE-12479:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683011/HBASE-12479-branch-1.patch
against master branch at commit 7ee4df600bb7ec8d794dd3b4ab2cb933ab15b988.
ATTACHMENT ID: 12683011
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 12 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11792//console
This message is automatically generated.
> Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1
>
>
> Key: HBASE-12479
> URL: https://issues.apache.org/jira/browse/HBASE-12479
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Reporter: Virag Kothari
>Assignee: Virag Kothari
> Fix For: 0.98.9, 0.99.2
>
> Attachments: HBASE-12479-0.98.patch, HBASE-12479-0.98_v2.patch,
> HBASE-12479-branch-1.patch
>
>
> Required for zk-less assignment
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12554) TestBaseLoadBalancer may timeout due to lengthy rack lookup
[
https://issues.apache.org/jira/browse/HBASE-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-12554:
---
Resolution: Fixed
Fix Version/s: 0.99.2
2.0.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks for the review, Stack.
> TestBaseLoadBalancer may timeout due to lengthy rack lookup
> ---
>
> Key: HBASE-12554
> URL: https://issues.apache.org/jira/browse/HBASE-12554
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 0.99.2
>
> Attachments: 12554-v1.txt, 12554-v2.txt, 12554-v3.txt, 12554-v4.txt,
> 12554-v5.txt
>
>
> Here is one of the recent occurrences
> (https://builds.apache.org/job/PreCommit-HBASE-Build/11778/console):
> {code}
> testImmediateAssignment(org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer)
> Time elapsed: 30.019 sec <<< ERROR!
> java.lang.Exception: test timed out after 3 milliseconds
> at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
> at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901)
> at
> java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293)
> at java.net.InetAddress.getAllByName0(InetAddress.java:1246)
> at java.net.InetAddress.getAllByName(InetAddress.java:1162)
> at java.net.InetAddress.getAllByName(InetAddress.java:1098)
> at java.net.InetAddress.getByName(InetAddress.java:1048)
> at org.apache.hadoop.net.NetUtils.normalizeHostName(NetUtils.java:561)
> at org.apache.hadoop.net.NetUtils.normalizeHostNames(NetUtils.java:578)
> at
> org.apache.hadoop.net.CachedDNSToSwitchMapping.resolve(CachedDNSToSwitchMapping.java:109)
> at
> org.apache.hadoop.hbase.master.RackManager.getRack(RackManager.java:66)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.(BaseLoadBalancer.java:273)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.createCluster(BaseLoadBalancer.java:1113)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.randomAssignment(BaseLoadBalancer.java:1175)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.immediateAssignment(BaseLoadBalancer.java:1145)
> at
> org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer.testImmediateAssignment(TestBaseLoadBalancer.java:136)
> {code}
> One possible fix is to submit CachedDNSToSwitchMapping.resolve() to executor
> pool for execution. RackManager.getRack() can set a timeout beyond which
> UNKNOWN_RACK is returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12479) Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1
[ https://issues.apache.org/jira/browse/HBASE-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari updated HBASE-12479: -- Attachment: HBASE-12479-branch-1.patch > Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1 > > > Key: HBASE-12479 > URL: https://issues.apache.org/jira/browse/HBASE-12479 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Reporter: Virag Kothari >Assignee: Virag Kothari > Fix For: 0.98.9, 0.99.2 > > Attachments: HBASE-12479-0.98.patch, HBASE-12479-0.98_v2.patch, > HBASE-12479-branch-1.patch > > > Required for zk-less assignment -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12479) Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1
[ https://issues.apache.org/jira/browse/HBASE-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari updated HBASE-12479: -- Attachment: (was: HBASE-12479-branch-1.patch) > Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1 > > > Key: HBASE-12479 > URL: https://issues.apache.org/jira/browse/HBASE-12479 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Reporter: Virag Kothari >Assignee: Virag Kothari > Fix For: 0.98.9, 0.99.2 > > Attachments: HBASE-12479-0.98.patch, HBASE-12479-0.98_v2.patch, > HBASE-12479-branch-1.patch > > > Required for zk-less assignment -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HBASE-10844) Coprocessor failure during batchmutation leaves the memstore datastructs in an inconsistent state
[
https://issues.apache.org/jira/browse/HBASE-10844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj Das reopened HBASE-10844:
-
Thinking about it (and in offline discussions with [~jeffreyz] and [~enis]), it
seems we should remove the 'assert' from the code that causes the RegionServer
to abort. The reason being that coprocessors could fail. The assert was
introduced in HBASE-10514.
{noformat}
// close each store in parallel
for (final Store store : stores.values()) {
+ assert abort? true: store.getFlushableSize() == 0;
{noformat}
Thoughts?
> Coprocessor failure during batchmutation leaves the memstore datastructs in
> an inconsistent state
> -
>
> Key: HBASE-10844
> URL: https://issues.apache.org/jira/browse/HBASE-10844
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Devaraj Das
>Assignee: Devaraj Das
> Attachments: 10844-1.txt
>
>
> Observed this in the testing with Phoenix. The test in Phoenix -
> MutableIndexFailureIT deliberately fails the batchmutation call via the
> installed coprocessor. But the update is not rolled back. That leaves the
> memstore inconsistent. In particular, I observed that getFlushableSize is
> updated before the coprocessor was called but the update is not rolled back.
> When the region is being closed at some later point, the assert introduced in
> HBASE-10514 in the HRegion.doClose() causes the RegionServer to shutdown
> abnormally.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12479) Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1
[ https://issues.apache.org/jira/browse/HBASE-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221666#comment-14221666 ] Andrew Purtell commented on HBASE-12479: Thanks, will commit shortly unless objection > Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1 > > > Key: HBASE-12479 > URL: https://issues.apache.org/jira/browse/HBASE-12479 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Reporter: Virag Kothari >Assignee: Virag Kothari > Fix For: 0.98.9, 0.99.2 > > Attachments: HBASE-12479-0.98.patch, HBASE-12479-0.98_v2.patch, > HBASE-12479-branch-1.patch > > > Required for zk-less assignment -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11992) Backport HBASE-11367 (Pluggable replication endpoint) to 0.98
[ https://issues.apache.org/jira/browse/HBASE-11992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221665#comment-14221665 ] Andrew Purtell commented on HBASE-11992: Thanks for your patience [~ram_krish]. I tested the following scenarios with the latest patch on this issue rebased on top of 7750a1f (HBASE-12346 addendum): https://github.com/apurtell/hbase/commit/aca7161 - Cluster 1 to 2 unidirectional, load 1M rows on 1: All 1M rows OK on 1 and 2 - Clusters 1 and 2 bidirectional, load 1M rows on 1: All 1M rows OK on 1 and 2 - Clusters 1 and 2 bidirectional, load 1M rows on 1 and 2: All 2M rows OK on 1 and 2 - Cluster 1 to 2 unidirectional, load 1M rows on 1, kill RS on 1 and restart during load: All 1M rows OK on 1 and 2 - Cluster 1 to 2 unidirectional, load on 1, kill RS on 2 and restart during load: All 1M rows OK on 1 and 2 - Cluster 1 and 2 bidirectional, load 10M rows on 1 and 2: All 20M rows OK on 1 and 2 I'm going to commit this shortly to 0.98 unless objection. > Backport HBASE-11367 (Pluggable replication endpoint) to 0.98 > - > > Key: HBASE-11992 > URL: https://issues.apache.org/jira/browse/HBASE-11992 > Project: HBase > Issue Type: Task >Reporter: Andrew Purtell >Assignee: ramkrishna.s.vasudevan > Fix For: 0.98.9 > > Attachments: HBASE-11992_0.98_1.patch, HBASE-11992_0.98_2.patch, > hbase-11367_0.98.patch > > > ReplicationSource tails the logs for each peer. HBASE-11367 introduces > ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is > run in the same RS process and instantiated per replication peer per region > server. Implementations of this interface handle the actual shipping of WAL > edits to the remote cluster. > This issue is for backporting HBASE-11367 to 0.98. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12479) Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1
[
https://issues.apache.org/jira/browse/HBASE-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221664#comment-14221664
]
Hadoop QA commented on HBASE-12479:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683009/HBASE-12479-branch-1.patch
against master branch at commit 882324dbcc224072be6a43a74aa6102637e529f8.
ATTACHMENT ID: 12683009
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 15 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11791//console
This message is automatically generated.
> Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1
>
>
> Key: HBASE-12479
> URL: https://issues.apache.org/jira/browse/HBASE-12479
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Reporter: Virag Kothari
>Assignee: Virag Kothari
> Fix For: 0.98.9, 0.99.2
>
> Attachments: HBASE-12479-0.98.patch, HBASE-12479-0.98_v2.patch,
> HBASE-12479-branch-1.patch
>
>
> Required for zk-less assignment
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12554) TestBaseLoadBalancer may timeout due to lengthy rack lookup
[
https://issues.apache.org/jira/browse/HBASE-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221660#comment-14221660
]
Hadoop QA commented on HBASE-12554:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12682980/12554-v5.txt
against master branch at commit 882324dbcc224072be6a43a74aa6102637e529f8.
ATTACHMENT ID: 12682980
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11787//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11787//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11787//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11787//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11787//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11787//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11787//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11787//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11787//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11787//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11787//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11787//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/11787//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11787//console
This message is automatically generated.
> TestBaseLoadBalancer may timeout due to lengthy rack lookup
> ---
>
> Key: HBASE-12554
> URL: https://issues.apache.org/jira/browse/HBASE-12554
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 12554-v1.txt, 12554-v2.txt, 12554-v3.txt, 12554-v4.txt,
> 12554-v5.txt
>
>
> Here is one of the recent occurrences
> (https://builds.apache.org/job/PreCommit-HBASE-Build/11778/console):
> {code}
> testImmediateAssignment(org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer)
> Time elapsed: 30.019 sec <<< ERROR!
> java.lang.Exception: test timed out after 3 milliseconds
> at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
> at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901)
> at
> java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293)
> at java.net.InetAddress.getAllByName0(InetAddress.java:1246)
> at java.net.InetAddress.getAllByName(InetAddress.java:1162)
> at java.net.InetAddress.getAllByName(InetAddress.java:1098)
> at java.net.InetAddress.getByName(InetAddress.java:1048)
> at org.apache.hadoop.net.NetUtils.normalizeHostName(NetUtils.java:561)
> at org.apache.hadoop.net.NetUtils.normalizeHostNames(NetUtils.java:578)
>
[jira] [Updated] (HBASE-12479) Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1
[ https://issues.apache.org/jira/browse/HBASE-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari updated HBASE-12479: -- Attachment: HBASE-12479-branch-1.patch HBASE-12479-0.98_v2.patch Attached is patch for branch-1. The 0.98v2 patch has small update in assignMeta() > Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1 > > > Key: HBASE-12479 > URL: https://issues.apache.org/jira/browse/HBASE-12479 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Reporter: Virag Kothari >Assignee: Virag Kothari > Fix For: 0.98.9, 0.99.2 > > Attachments: HBASE-12479-0.98.patch, HBASE-12479-0.98_v2.patch, > HBASE-12479-branch-1.patch > > > Required for zk-less assignment -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12557) Introduce timeout mechanism for IP to rack resolution
[ https://issues.apache.org/jira/browse/HBASE-12557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221654#comment-14221654 ] Ted Yu commented on HBASE-12557: Browsed through hadoop tests, such as TestJobHistoryParsing, which use custom resolver. In the new test for this JIRA, I can issue long I/O operation in the resolve() method and see if the cancellation works. > Introduce timeout mechanism for IP to rack resolution > - > > Key: HBASE-12557 > URL: https://issues.apache.org/jira/browse/HBASE-12557 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 12557-v1.txt > > > Config parameter, "hbase.util.ip.to.rack.determiner", determines the class > which does IP to rack resolution. > The actual resolution may be lengthy. > This JIRA is continuation of HBASE-12554 where a mock DNSToSwitchMapping is > used for rack resolution. > A timeout parameter, "hbase.ip.to.rack.determiner.timeout", is proposed whose > value governs the duration which RackManager waits before rack resolution is > stopped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12557) Introduce timeout mechanism for IP to rack resolution
[ https://issues.apache.org/jira/browse/HBASE-12557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221643#comment-14221643 ] stack commented on HBASE-12557: --- I'd suggest also breaking dns so you get long timeout and seeing if the cancel actual cancels the ongoing lookup and doesn't leave resources laying around > Introduce timeout mechanism for IP to rack resolution > - > > Key: HBASE-12557 > URL: https://issues.apache.org/jira/browse/HBASE-12557 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 12557-v1.txt > > > Config parameter, "hbase.util.ip.to.rack.determiner", determines the class > which does IP to rack resolution. > The actual resolution may be lengthy. > This JIRA is continuation of HBASE-12554 where a mock DNSToSwitchMapping is > used for rack resolution. > A timeout parameter, "hbase.ip.to.rack.determiner.timeout", is proposed whose > value governs the duration which RackManager waits before rack resolution is > stopped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12557) Introduce timeout mechanism for IP to rack resolution
[ https://issues.apache.org/jira/browse/HBASE-12557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221641#comment-14221641 ] Ted Yu commented on HBASE-12557: Good catch, Andrew. Corrected the typo. As for the test, I am looking among related hadoop tests to see if I can find something. > Introduce timeout mechanism for IP to rack resolution > - > > Key: HBASE-12557 > URL: https://issues.apache.org/jira/browse/HBASE-12557 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 12557-v1.txt > > > Config parameter, "hbase.util.ip.to.rack.determiner", determines the class > which does IP to rack resolution. > The actual resolution may be lengthy. > This JIRA is continuation of HBASE-12554 where a mock DNSToSwitchMapping is > used for rack resolution. > A timeout parameter, "hbase.ip.to.rack.determiner.timeout", is proposed whose > value governs the duration which RackManager waits before rack resolution is > stopped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12557) Introduce timeout mechanism for IP to rack resolution
[ https://issues.apache.org/jira/browse/HBASE-12557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-12557: --- Attachment: 12557-v1.txt > Introduce timeout mechanism for IP to rack resolution > - > > Key: HBASE-12557 > URL: https://issues.apache.org/jira/browse/HBASE-12557 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 12557-v1.txt > > > Config parameter, "hbase.util.ip.to.rack.determiner", determines the class > which does IP to rack resolution. > The actual resolution may be lengthy. > This JIRA is continuation of HBASE-12554 where a mock DNSToSwitchMapping is > used for rack resolution. > A timeout parameter, "hbase.ip.to.rack.determiner.timeout", is proposed whose > value governs the duration which RackManager waits before rack resolution is > stopped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12557) Introduce timeout mechanism for IP to rack resolution
[ https://issues.apache.org/jira/browse/HBASE-12557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-12557: --- Attachment: (was: 12557-v1.txt) > Introduce timeout mechanism for IP to rack resolution > - > > Key: HBASE-12557 > URL: https://issues.apache.org/jira/browse/HBASE-12557 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > > Config parameter, "hbase.util.ip.to.rack.determiner", determines the class > which does IP to rack resolution. > The actual resolution may be lengthy. > This JIRA is continuation of HBASE-12554 where a mock DNSToSwitchMapping is > used for rack resolution. > A timeout parameter, "hbase.ip.to.rack.determiner.timeout", is proposed whose > value governs the duration which RackManager waits before rack resolution is > stopped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12559) Provide StochasticLoadBalancer with online configuration capability
Ted Yu created HBASE-12559: -- Summary: Provide StochasticLoadBalancer with online configuration capability Key: HBASE-12559 URL: https://issues.apache.org/jira/browse/HBASE-12559 Project: HBase Issue Type: Improvement Reporter: Ted Yu StochasticLoadBalancer has many knobs which user can adjust. It would increase productivity by allowing StochasticLoadBalancer to accept online configuration changes. StochasticLoadBalancer should implement PropagatingConfigurationObserver and reload relevant configuration parameters in onConfigurationChange() method. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12557) Introduce timeout mechanism for IP to rack resolution
[ https://issues.apache.org/jira/browse/HBASE-12557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221636#comment-14221636 ] Andrew Purtell commented on HBASE-12557: Spelling error "DETEMINDER" Missing a test that this actually works > Introduce timeout mechanism for IP to rack resolution > - > > Key: HBASE-12557 > URL: https://issues.apache.org/jira/browse/HBASE-12557 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 12557-v1.txt > > > Config parameter, "hbase.util.ip.to.rack.determiner", determines the class > which does IP to rack resolution. > The actual resolution may be lengthy. > This JIRA is continuation of HBASE-12554 where a mock DNSToSwitchMapping is > used for rack resolution. > A timeout parameter, "hbase.ip.to.rack.determiner.timeout", is proposed whose > value governs the duration which RackManager waits before rack resolution is > stopped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12471) Task 4. replace internal ConnectionManager#{delete,get}Connection use with #close, #createConnection (0.98, 0.99) under src/main/java
[
https://issues.apache.org/jira/browse/HBASE-12471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-12471:
--
Attachment: 12471.reapplyv2.txt
Rebase
> Task 4. replace internal ConnectionManager#{delete,get}Connection use with
> #close, #createConnection (0.98, 0.99) under src/main/java
> -
>
> Key: HBASE-12471
> URL: https://issues.apache.org/jira/browse/HBASE-12471
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 0.99.2
>
> Attachments:
> 0001-HBASE-12471-Task-4.-replace-internal-ConnectionManag.patch, 12404v9.txt,
> 124471v4.txt, 12471.reapply.txt, 12471.reapplyv2.txt, 12471v10.txt,
> 12471v10.txt, 12471v11.txt, 12471v2.txt, 12471v3.txt, 12471v6.txt,
> 12471v7.txt, 12471v9.txt
>
>
> Let me do this. A bunch of this was done in HBASE-12404 Let me see if can
> find more.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12550) Check all storefiles are referenced before splitting
[ https://issues.apache.org/jira/browse/HBASE-12550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-12550: -- Attachment: HBASE-12550-v3.patch > Check all storefiles are referenced before splitting > > > Key: HBASE-12550 > URL: https://issues.apache.org/jira/browse/HBASE-12550 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-12550-v1.patch, HBASE-12550-v2.patch, > HBASE-12550-v3.patch, HBASE-12550.patch > > > Make double extra sure all storefiles are referenced before -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12550) Check all storefiles are referenced before splitting
[ https://issues.apache.org/jira/browse/HBASE-12550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221632#comment-14221632 ] Elliott Clark commented on HBASE-12550: --- My speluing is afwul. > Check all storefiles are referenced before splitting > > > Key: HBASE-12550 > URL: https://issues.apache.org/jira/browse/HBASE-12550 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-12550-v1.patch, HBASE-12550-v2.patch, > HBASE-12550-v3.patch, HBASE-12550.patch > > > Make double extra sure all storefiles are referenced before -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12556) Create a golden file for testing client API source/binary compatibility
[ https://issues.apache.org/jira/browse/HBASE-12556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221619#comment-14221619 ] Dima Spivak commented on HBASE-12556: - I like Sean's idea of an automated tool over a golden file; I've briefly explored adding annotation support into our existing jdiff wrapper and could speed this up if there's interest. If we do this, we might also want to have it run regularly on builds.apache.org; as it stands, I run it regularly on my own, but it would help to up its visibility and perhaps spam people if it finds public API deletions where they shouldn't be. > Create a golden file for testing client API source/binary compatibility > --- > > Key: HBASE-12556 > URL: https://issues.apache.org/jira/browse/HBASE-12556 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.99.2 > > > [~lhofhansl] had a suggestion (in some other jira I forgot) for doing a > golden file for the HBase API so that we can compare between releases to > ensure that we are keeping source and binary compatibility as defined in this > document : > https://docs.google.com/document/d/1p5pP7v2OuzSSaomK2S2v7sfKky1Hex6OqwsJO0sZTUY/edit > > I think we can generate a file, commit it to the repo, and create a unit test > to check whether any API's are broken. Adding new InterfaceAudience.Public > interfaces has to modify this file so that it becomes an explicit decision. > The downside is that we have to modify the file every time we add a new API, > but it should be fine since it will force us to think more before committing > to supporting new interfaces within the same major versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12550) Check all storefiles are referenced before splitting
[ https://issues.apache.org/jira/browse/HBASE-12550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221616#comment-14221616 ] stack commented on HBASE-12550: --- Fix this on commit RefernceFileFilter Otherwise good by me. [~mbertozzi]? You good with this? > Check all storefiles are referenced before splitting > > > Key: HBASE-12550 > URL: https://issues.apache.org/jira/browse/HBASE-12550 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-12550-v1.patch, HBASE-12550-v2.patch, > HBASE-12550.patch > > > Make double extra sure all storefiles are referenced before -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-10671) Add missing InterfaceAudience annotations for classes in hbase-common and hbase-client modules
[ https://issues.apache.org/jira/browse/HBASE-10671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221603#comment-14221603 ] Hudson commented on HBASE-10671: FAILURE: Integrated in HBase-1.0 #492 (See [https://builds.apache.org/job/HBase-1.0/492/]) HBASE-10671 Add missing InterfaceAudience annotations for classes in hbase-common and hbase-client modules (enis: rev aa343ebcfe169732d2868006f235c231f1af2e5d) * hbase-common/src/main/java/org/apache/hadoop/hbase/io/hadoopbackport/ThrottledInputStream.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeersZKImpl.java * hbase-common/src/main/java/org/apache/hadoop/hbase/security/UserProvider.java * hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/ConcatenatedLists.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/DelegatingRetryingCallable.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/LimitInputStream.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerFactory.java * hbase-common/src/main/java/org/apache/hadoop/hbase/MetaMutationAnnotation.java * hbase-client/src/main/java/org/apache/hadoop/hbase/exceptions/LockTimeoutException.java * hbase-client/src/main/java/org/apache/hadoop/hbase/exceptions/ConnectionClosingException.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationException.java * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/ServerRpcController.java * hbase-common/src/main/java/org/apache/hadoop/hbase/types/PBType.java * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RegionServerCoprocessorRpcChannel.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/token/AuthenticationTokenIdentifier.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/AbstractPositionedByteRange.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RowTooBigException.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java * hbase-annotations/src/main/java/org/apache/hadoop/hbase/classification/tools/ExcludePrivateAnnotationsStandardDoclet.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/FailureInfo.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesClient.java * hbase-common/src/test/java/org/apache/hadoop/hbase/ClassFinder.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/ChecksumFactory.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/coprocessor/Batch.java * hbase-common/src/main/java/org/apache/hadoop/hbase/NamespaceDescriptor.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationTrackerZKImpl.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/Base64.java * hbase-client/src/main/java/org/apache/hadoop/hbase/exceptions/UnknownProtocolException.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/SimpleByteRange.java * hbase-client/src/main/java/org/apache/hadoop/hbase/KeepDeletedCells.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationStateZKBase.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/HBaseSaslRpcClient.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/RegexStringComparator.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueueInfo.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTableMultiplexer.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/LongComparator.java * hbase-client/src/main/java/org/apache/hadoop/hbase/snapshot/ExportSnapshotException.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedException.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/MurmurHash3.java * hbase-common/src/main/java/org/apache/hadoop/hbase/trace/HBaseHTraceConfiguration.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/util/ByteStringer.java * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcControllerFactory.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/crypto/Encryption.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/ImmutableBytesWritable.java * hbase-protocol/pom.xml * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replicatio
[jira] [Updated] (HBASE-12550) Check all storefiles are referenced before splitting
[ https://issues.apache.org/jira/browse/HBASE-12550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-12550: -- Attachment: HBASE-12550-v2.patch Today I Learned that on Spy'd mocks the full function is called even when you're just asking to throw an exception. > Check all storefiles are referenced before splitting > > > Key: HBASE-12550 > URL: https://issues.apache.org/jira/browse/HBASE-12550 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-12550-v1.patch, HBASE-12550-v2.patch, > HBASE-12550.patch > > > Make double extra sure all storefiles are referenced before -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12554) TestBaseLoadBalancer may timeout due to lengthy rack lookup
[
https://issues.apache.org/jira/browse/HBASE-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221574#comment-14221574
]
Hadoop QA commented on HBASE-12554:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12682960/12554-v3.txt
against master branch at commit 882324dbcc224072be6a43a74aa6102637e529f8.
ATTACHMENT ID: 12682960
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11785//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11785//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11785//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11785//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11785//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11785//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11785//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11785//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11785//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11785//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11785//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11785//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/11785//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11785//console
This message is automatically generated.
> TestBaseLoadBalancer may timeout due to lengthy rack lookup
> ---
>
> Key: HBASE-12554
> URL: https://issues.apache.org/jira/browse/HBASE-12554
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 12554-v1.txt, 12554-v2.txt, 12554-v3.txt, 12554-v4.txt,
> 12554-v5.txt
>
>
> Here is one of the recent occurrences
> (https://builds.apache.org/job/PreCommit-HBASE-Build/11778/console):
> {code}
> testImmediateAssignment(org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer)
> Time elapsed: 30.019 sec <<< ERROR!
> java.lang.Exception: test timed out after 3 milliseconds
> at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
> at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901)
> at
> java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293)
> at java.net.InetAddress.getAllByName0(InetAddress.java:1246)
> at java.net.InetAddress.getAllByName(InetAddress.java:1162)
> at java.net.InetAddress.getAllByName(InetAddress.java:1098)
> at java.net.InetAddress.getByName(InetAddress.java:1048)
> at org.apache.hadoop.net.NetUtils.normalizeHostName(NetUtils.java:561)
> at org.apache.hadoop.net.NetUtils.normalizeHostNames(NetUtils.java:578)
>
[jira] [Created] (HBASE-12558) TestHCM.testClusterStatus Unexpected exception, expected but was
stack created HBASE-12558:
-
Summary: TestHCM.testClusterStatus Unexpected exception,
expected but
was
Key: HBASE-12558
URL: https://issues.apache.org/jira/browse/HBASE-12558
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: stack
Priority: Critical
Fix For: 2.0.0, 0.99.2
Happens for me reliably on mac os x. I looked at fixing it. The listener is not
noticing the publish for whatever reason. Thats where I stopped.
{code}
java.lang.Exception: Unexpected exception,
expected but
was
at junit.framework.Assert.fail(Assert.java:57)
at org.apache.hadoop.hbase.Waiter.waitFor(Waiter.java:193)
at
org.apache.hadoop.hbase.HBaseTestingUtility.waitFor(HBaseTestingUtility.java:3537)
at
org.apache.hadoop.hbase.client.TestHCM.testClusterStatus(TestHCM.java:273)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12474) Incorrect handling of default namespace in user_permission command.
[ https://issues.apache.org/jira/browse/HBASE-12474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221559#comment-14221559 ] stack commented on HBASE-12474: --- The failure is unrelated. Its the udp broadcast stuff. It fails reliably for me on mac os x. We actually tried it three times here: Tests in error: org.apache.hadoop.hbase.client.TestHCM.testClusterStatus(org.apache.hadoop.hbase.client.TestHCM) Run 1: TestHCM.testClusterStatus ยป Unexpected exception, expected Incorrect handling of default namespace in user_permission command. > --- > > Key: HBASE-12474 > URL: https://issues.apache.org/jira/browse/HBASE-12474 > Project: HBase > Issue Type: Bug >Reporter: Srikanth Srungarapu >Assignee: Srikanth Srungarapu >Priority: Minor > Attachments: HBASE-12474.patch, HBASE-12474_v2.patch > > > The command user_permission returns 0 rows if we specify default prefix. It > works fine if we drop the prefix. This is because pattern matching of > listTables doesn't take into account that nameAsString method doesn't include > 'default' in it's output for the relevant tables. This method listTables is > also used by delete, disable and enable commands when invoked with regular > expression. Credits to [~mbertozzi] for coming up with this corner case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12554) TestBaseLoadBalancer may timeout due to lengthy rack lookup
[
https://issues.apache.org/jira/browse/HBASE-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221558#comment-14221558
]
stack commented on HBASE-12554:
---
+1 if passes hadoopqa.
> TestBaseLoadBalancer may timeout due to lengthy rack lookup
> ---
>
> Key: HBASE-12554
> URL: https://issues.apache.org/jira/browse/HBASE-12554
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 12554-v1.txt, 12554-v2.txt, 12554-v3.txt, 12554-v4.txt,
> 12554-v5.txt
>
>
> Here is one of the recent occurrences
> (https://builds.apache.org/job/PreCommit-HBASE-Build/11778/console):
> {code}
> testImmediateAssignment(org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer)
> Time elapsed: 30.019 sec <<< ERROR!
> java.lang.Exception: test timed out after 3 milliseconds
> at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
> at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901)
> at
> java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293)
> at java.net.InetAddress.getAllByName0(InetAddress.java:1246)
> at java.net.InetAddress.getAllByName(InetAddress.java:1162)
> at java.net.InetAddress.getAllByName(InetAddress.java:1098)
> at java.net.InetAddress.getByName(InetAddress.java:1048)
> at org.apache.hadoop.net.NetUtils.normalizeHostName(NetUtils.java:561)
> at org.apache.hadoop.net.NetUtils.normalizeHostNames(NetUtils.java:578)
> at
> org.apache.hadoop.net.CachedDNSToSwitchMapping.resolve(CachedDNSToSwitchMapping.java:109)
> at
> org.apache.hadoop.hbase.master.RackManager.getRack(RackManager.java:66)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.(BaseLoadBalancer.java:273)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.createCluster(BaseLoadBalancer.java:1113)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.randomAssignment(BaseLoadBalancer.java:1175)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.immediateAssignment(BaseLoadBalancer.java:1145)
> at
> org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer.testImmediateAssignment(TestBaseLoadBalancer.java:136)
> {code}
> One possible fix is to submit CachedDNSToSwitchMapping.resolve() to executor
> pool for execution. RackManager.getRack() can set a timeout beyond which
> UNKNOWN_RACK is returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-12557) Introduce timeout mechanism for IP to rack resolution
[ https://issues.apache.org/jira/browse/HBASE-12557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu reassigned HBASE-12557: -- Assignee: Ted Yu > Introduce timeout mechanism for IP to rack resolution > - > > Key: HBASE-12557 > URL: https://issues.apache.org/jira/browse/HBASE-12557 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 12557-v1.txt > > > Config parameter, "hbase.util.ip.to.rack.determiner", determines the class > which does IP to rack resolution. > The actual resolution may be lengthy. > This JIRA is continuation of HBASE-12554 where a mock DNSToSwitchMapping is > used for rack resolution. > A timeout parameter, "hbase.ip.to.rack.determiner.timeout", is proposed whose > value governs the duration which RackManager waits before rack resolution is > stopped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12557) Introduce timeout mechanism for IP to rack resolution
[ https://issues.apache.org/jira/browse/HBASE-12557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-12557: --- Attachment: 12557-v1.txt Proposed patch. > Introduce timeout mechanism for IP to rack resolution > - > > Key: HBASE-12557 > URL: https://issues.apache.org/jira/browse/HBASE-12557 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu > Attachments: 12557-v1.txt > > > Config parameter, "hbase.util.ip.to.rack.determiner", determines the class > which does IP to rack resolution. > The actual resolution may be lengthy. > This JIRA is continuation of HBASE-12554 where a mock DNSToSwitchMapping is > used for rack resolution. > A timeout parameter, "hbase.ip.to.rack.determiner.timeout", is proposed whose > value governs the duration which RackManager waits before rack resolution is > stopped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12557) Introduce timeout mechanism for IP to rack resolution
Ted Yu created HBASE-12557: -- Summary: Introduce timeout mechanism for IP to rack resolution Key: HBASE-12557 URL: https://issues.apache.org/jira/browse/HBASE-12557 Project: HBase Issue Type: Improvement Reporter: Ted Yu Config parameter, "hbase.util.ip.to.rack.determiner", determines the class which does IP to rack resolution. The actual resolution may be lengthy. This JIRA is continuation of HBASE-12554 where a mock DNSToSwitchMapping is used for rack resolution. A timeout parameter, "hbase.ip.to.rack.determiner.timeout", is proposed whose value governs the duration which RackManager waits before rack resolution is stopped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12533) staging directories are not deleted after secure bulk load
[
https://issues.apache.org/jira/browse/HBASE-12533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221552#comment-14221552
]
Jeffrey Zhong commented on HBASE-12533:
---
[~dubislv] Is that possible for you to try to patch to see if the issue is
addressed? The patch should be able to apply to 0.98 code base as well. Thanks.
> staging directories are not deleted after secure bulk load
> --
>
> Key: HBASE-12533
> URL: https://issues.apache.org/jira/browse/HBASE-12533
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.98.6
> Environment: CDH5.2 + Kerberos
>Reporter: Andrejs Dubovskis
>Assignee: Jeffrey Zhong
> Attachments: HBASE-12533.patch
>
>
> We using secure bulk load heavily in our environment. And it was working with
> no problem during some time. But last week I found that clients hangs while
> calling *doBulkLoad*
> After some investigation I found that HDFS keeps more than 1,000,000
> directories in /tmp/hbase-staging directory.
> When directory's content was purged the load process runs successfully.
> According the [hbase
> book|http://hbase.apache.org/book/ch08s03.html#hbase.secure.bulkload]
> {code}
> HBase manages creation and deletion of this directory.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12554) TestBaseLoadBalancer may timeout due to lengthy rack lookup
[
https://issues.apache.org/jira/browse/HBASE-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-12554:
---
Attachment: 12554-v5.txt
> TestBaseLoadBalancer may timeout due to lengthy rack lookup
> ---
>
> Key: HBASE-12554
> URL: https://issues.apache.org/jira/browse/HBASE-12554
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 12554-v1.txt, 12554-v2.txt, 12554-v3.txt, 12554-v4.txt,
> 12554-v5.txt
>
>
> Here is one of the recent occurrences
> (https://builds.apache.org/job/PreCommit-HBASE-Build/11778/console):
> {code}
> testImmediateAssignment(org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer)
> Time elapsed: 30.019 sec <<< ERROR!
> java.lang.Exception: test timed out after 3 milliseconds
> at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
> at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901)
> at
> java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293)
> at java.net.InetAddress.getAllByName0(InetAddress.java:1246)
> at java.net.InetAddress.getAllByName(InetAddress.java:1162)
> at java.net.InetAddress.getAllByName(InetAddress.java:1098)
> at java.net.InetAddress.getByName(InetAddress.java:1048)
> at org.apache.hadoop.net.NetUtils.normalizeHostName(NetUtils.java:561)
> at org.apache.hadoop.net.NetUtils.normalizeHostNames(NetUtils.java:578)
> at
> org.apache.hadoop.net.CachedDNSToSwitchMapping.resolve(CachedDNSToSwitchMapping.java:109)
> at
> org.apache.hadoop.hbase.master.RackManager.getRack(RackManager.java:66)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.(BaseLoadBalancer.java:273)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.createCluster(BaseLoadBalancer.java:1113)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.randomAssignment(BaseLoadBalancer.java:1175)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.immediateAssignment(BaseLoadBalancer.java:1145)
> at
> org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer.testImmediateAssignment(TestBaseLoadBalancer.java:136)
> {code}
> One possible fix is to submit CachedDNSToSwitchMapping.resolve() to executor
> pool for execution. RackManager.getRack() can set a timeout beyond which
> UNKNOWN_RACK is returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12474) Incorrect handling of default namespace in user_permission command.
[
https://issues.apache.org/jira/browse/HBASE-12474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221538#comment-14221538
]
Hadoop QA commented on HBASE-12474:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12682958/HBASE-12474_v2.patch
against master branch at commit 882324dbcc224072be6a43a74aa6102637e529f8.
ATTACHMENT ID: 12682958
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestHCM
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11784//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11784//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11784//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11784//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11784//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11784//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11784//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11784//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11784//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11784//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11784//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11784//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/11784//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11784//console
This message is automatically generated.
> Incorrect handling of default namespace in user_permission command.
> ---
>
> Key: HBASE-12474
> URL: https://issues.apache.org/jira/browse/HBASE-12474
> Project: HBase
> Issue Type: Bug
>Reporter: Srikanth Srungarapu
>Assignee: Srikanth Srungarapu
>Priority: Minor
> Attachments: HBASE-12474.patch, HBASE-12474_v2.patch
>
>
> The command user_permission returns 0 rows if we specify default prefix. It
> works fine if we drop the prefix. This is because pattern matching of
> listTables doesn't take into account that nameAsString method doesn't include
> 'default' in it's output for the relevant tables. This method listTables is
> also used by delete, disable and enable commands when invoked with regular
> expression. Credits to [~mbertozzi] for coming up with this corner case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12554) TestBaseLoadBalancer may timeout due to lengthy rack lookup
[
https://issues.apache.org/jira/browse/HBASE-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221528#comment-14221528
]
stack commented on HBASE-12554:
---
+1
On commit, put public first in below and add class comment why we are putting
in place this mock class:
static public class...
Leave out the other changes, the changes to RackManager and to BaseLoadBalancer
since you don't know if they have an effect (logging that we spent 60 seconds
in lookup could be good ... but you ain't sure the interrupt works). Do such
changes in another JIRA where you can try code against bad dns to see it is
doing the right thing.
> TestBaseLoadBalancer may timeout due to lengthy rack lookup
> ---
>
> Key: HBASE-12554
> URL: https://issues.apache.org/jira/browse/HBASE-12554
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 12554-v1.txt, 12554-v2.txt, 12554-v3.txt, 12554-v4.txt
>
>
> Here is one of the recent occurrences
> (https://builds.apache.org/job/PreCommit-HBASE-Build/11778/console):
> {code}
> testImmediateAssignment(org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer)
> Time elapsed: 30.019 sec <<< ERROR!
> java.lang.Exception: test timed out after 3 milliseconds
> at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
> at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901)
> at
> java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293)
> at java.net.InetAddress.getAllByName0(InetAddress.java:1246)
> at java.net.InetAddress.getAllByName(InetAddress.java:1162)
> at java.net.InetAddress.getAllByName(InetAddress.java:1098)
> at java.net.InetAddress.getByName(InetAddress.java:1048)
> at org.apache.hadoop.net.NetUtils.normalizeHostName(NetUtils.java:561)
> at org.apache.hadoop.net.NetUtils.normalizeHostNames(NetUtils.java:578)
> at
> org.apache.hadoop.net.CachedDNSToSwitchMapping.resolve(CachedDNSToSwitchMapping.java:109)
> at
> org.apache.hadoop.hbase.master.RackManager.getRack(RackManager.java:66)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.(BaseLoadBalancer.java:273)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.createCluster(BaseLoadBalancer.java:1113)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.randomAssignment(BaseLoadBalancer.java:1175)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.immediateAssignment(BaseLoadBalancer.java:1145)
> at
> org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer.testImmediateAssignment(TestBaseLoadBalancer.java:136)
> {code}
> One possible fix is to submit CachedDNSToSwitchMapping.resolve() to executor
> pool for execution. RackManager.getRack() can set a timeout beyond which
> UNKNOWN_RACK is returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12522) Backport WAL refactoring to branch-1
[ https://issues.apache.org/jira/browse/HBASE-12522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221515#comment-14221515 ] Enis Soztutar commented on HBASE-12522: --- [~busbey] is the branch-1 patch that different from master? It should not be. I am going over the master patch in the review board. > Backport WAL refactoring to branch-1 > > > Key: HBASE-12522 > URL: https://issues.apache.org/jira/browse/HBASE-12522 > Project: HBase > Issue Type: Task > Components: wal >Reporter: Sean Busbey >Assignee: Sean Busbey > Fix For: 0.99.2 > > > backport HBASE-10378 to branch-1. > This will let us remove the Deprecated stuff in master, allow some baking > time within the 1.x line, and will give us the option of pulling back follow > on performance improvements. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12554) TestBaseLoadBalancer may timeout due to lengthy rack lookup
[
https://issues.apache.org/jira/browse/HBASE-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-12554:
---
Attachment: 12554-v4.txt
> TestBaseLoadBalancer may timeout due to lengthy rack lookup
> ---
>
> Key: HBASE-12554
> URL: https://issues.apache.org/jira/browse/HBASE-12554
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 12554-v1.txt, 12554-v2.txt, 12554-v3.txt, 12554-v4.txt
>
>
> Here is one of the recent occurrences
> (https://builds.apache.org/job/PreCommit-HBASE-Build/11778/console):
> {code}
> testImmediateAssignment(org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer)
> Time elapsed: 30.019 sec <<< ERROR!
> java.lang.Exception: test timed out after 3 milliseconds
> at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
> at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901)
> at
> java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293)
> at java.net.InetAddress.getAllByName0(InetAddress.java:1246)
> at java.net.InetAddress.getAllByName(InetAddress.java:1162)
> at java.net.InetAddress.getAllByName(InetAddress.java:1098)
> at java.net.InetAddress.getByName(InetAddress.java:1048)
> at org.apache.hadoop.net.NetUtils.normalizeHostName(NetUtils.java:561)
> at org.apache.hadoop.net.NetUtils.normalizeHostNames(NetUtils.java:578)
> at
> org.apache.hadoop.net.CachedDNSToSwitchMapping.resolve(CachedDNSToSwitchMapping.java:109)
> at
> org.apache.hadoop.hbase.master.RackManager.getRack(RackManager.java:66)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.(BaseLoadBalancer.java:273)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.createCluster(BaseLoadBalancer.java:1113)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.randomAssignment(BaseLoadBalancer.java:1175)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.immediateAssignment(BaseLoadBalancer.java:1145)
> at
> org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer.testImmediateAssignment(TestBaseLoadBalancer.java:136)
> {code}
> One possible fix is to submit CachedDNSToSwitchMapping.resolve() to executor
> pool for execution. RackManager.getRack() can set a timeout beyond which
> UNKNOWN_RACK is returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-10671) Add missing InterfaceAudience annotations for classes in hbase-common and hbase-client modules
[ https://issues.apache.org/jira/browse/HBASE-10671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-10671: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed this to 0.99+. Thanks for review Stack. > Add missing InterfaceAudience annotations for classes in hbase-common and > hbase-client modules > -- > > Key: HBASE-10671 > URL: https://issues.apache.org/jira/browse/HBASE-10671 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 0.99.2 > > Attachments: hbase-10671_v1.patch, hbase-10671_v2.patch, > hbase-10671_v3.patch, hbase-10671_v4.patch, hbase-10671_v6-0.99.patch, > hbase-10671_v6.patch > > > In this jira, we'll add missing InterfaceAudience annotations to classes in > the client visible modules (hbase-client and hbase-common). > Parent jira is for deciding on whether some of the classes should be private > or public. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-10671) Add missing InterfaceAudience annotations for classes in hbase-common and hbase-client modules
[ https://issues.apache.org/jira/browse/HBASE-10671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-10671: -- Attachment: hbase-10671_v6-0.99.patch hbase-10671_v6.patch Attaching the committed versions of the patches. > Add missing InterfaceAudience annotations for classes in hbase-common and > hbase-client modules > -- > > Key: HBASE-10671 > URL: https://issues.apache.org/jira/browse/HBASE-10671 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 0.99.2 > > Attachments: hbase-10671_v1.patch, hbase-10671_v2.patch, > hbase-10671_v3.patch, hbase-10671_v4.patch, hbase-10671_v6-0.99.patch, > hbase-10671_v6.patch > > > In this jira, we'll add missing InterfaceAudience annotations to classes in > the client visible modules (hbase-client and hbase-common). > Parent jira is for deciding on whether some of the classes should be private > or public. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12554) TestBaseLoadBalancer may timeout due to lengthy rack lookup
[
https://issues.apache.org/jira/browse/HBASE-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221489#comment-14221489
]
stack commented on HBASE-12554:
---
Doesn't do what I suggestted
> TestBaseLoadBalancer may timeout due to lengthy rack lookup
> ---
>
> Key: HBASE-12554
> URL: https://issues.apache.org/jira/browse/HBASE-12554
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 12554-v1.txt, 12554-v2.txt, 12554-v3.txt
>
>
> Here is one of the recent occurrences
> (https://builds.apache.org/job/PreCommit-HBASE-Build/11778/console):
> {code}
> testImmediateAssignment(org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer)
> Time elapsed: 30.019 sec <<< ERROR!
> java.lang.Exception: test timed out after 3 milliseconds
> at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
> at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901)
> at
> java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293)
> at java.net.InetAddress.getAllByName0(InetAddress.java:1246)
> at java.net.InetAddress.getAllByName(InetAddress.java:1162)
> at java.net.InetAddress.getAllByName(InetAddress.java:1098)
> at java.net.InetAddress.getByName(InetAddress.java:1048)
> at org.apache.hadoop.net.NetUtils.normalizeHostName(NetUtils.java:561)
> at org.apache.hadoop.net.NetUtils.normalizeHostNames(NetUtils.java:578)
> at
> org.apache.hadoop.net.CachedDNSToSwitchMapping.resolve(CachedDNSToSwitchMapping.java:109)
> at
> org.apache.hadoop.hbase.master.RackManager.getRack(RackManager.java:66)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.(BaseLoadBalancer.java:273)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.createCluster(BaseLoadBalancer.java:1113)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.randomAssignment(BaseLoadBalancer.java:1175)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.immediateAssignment(BaseLoadBalancer.java:1145)
> at
> org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer.testImmediateAssignment(TestBaseLoadBalancer.java:136)
> {code}
> One possible fix is to submit CachedDNSToSwitchMapping.resolve() to executor
> pool for execution. RackManager.getRack() can set a timeout beyond which
> UNKNOWN_RACK is returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12404) Task 5 from parent: Replace internal HTable constructor use with HConnection#getTable (0.98, 0.99)
[
https://issues.apache.org/jira/browse/HBASE-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221472#comment-14221472
]
Hadoop QA commented on HBASE-12404:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12682944/12404v14.txt
against master branch at commit 890c067b661aa5912311799867b2defd57d1f9dd.
ATTACHMENT ID: 12682944
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 150
new or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ * https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/InterfaceClassification.html";>Hadoop
+{@link org.apache.hadoop.hbase.client.Table#coprocessorService(Class, byte[],
byte[], org.apache.hadoop.hbase.client.coprocessor.Batch.Call)}, and
+{@link org.apache.hadoop.hbase.client.Table#coprocessorService(Class, byte[],
byte[], org.apache.hadoop.hbase.client.coprocessor.Batch.Call,
org.apache.hadoop.hbase.client.coprocessor.Batch.Callback)}
+ {@link org.apache.hadoop.hbase.client.Table#coprocessorService(Class,
byte[], byte[], org.apache.hadoop.hbase.client.coprocessor.Batch.Call)}
+ or {@link org.apache.hadoop.hbase.client.Table#coprocessorService(Class,
byte[], byte[], org.apache.hadoop.hbase.client.coprocessor.Batch.Call,
org.apache.hadoop.hbase.client.coprocessor.Batch.Callback)}
+method's argument. Calling {@link
org.apache.hadoop.hbase.client.Table#coprocessorService(Class, byte[], byte[],
org.apache.hadoop.hbase.client.coprocessor.Batch.Call)}
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestMultiParallel
org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildBase
org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildOverlap
org.apache.hadoop.hbase.util.TestRegionSplitter
org.apache.hadoop.hbase.coprocessor.TestHTableWrapper
org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildHole
org.apache.hadoop.hbase.master.TestMasterShutdown
org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer
org.apache.hadoop.hbase.util.TestHBaseFsck
org.apache.hadoop.hbase.client.TestHCM
org.apache.hadoop.hbase.master.TestMaster
{color:red}-1 core zombie tests{color}. There are 12 zombie test(s):
at
org.apache.hadoop.hbase.security.access.TestCellACLs.testCoveringCheck(TestCellACLs.java:385)
at
org.apache.hadoop.hbase.security.access.TestAccessController2.testCreateWithCorrectOwner(TestAccessController2.java:86)
at
org.apache.hadoop.hbase.security.access.TestAccessControlFilter.testQualifierAccess(TestAccessControlFilter.java:97)
at
org.apache.hadoop.hbase.security.access.TestScanEarlyTermination.testEarlyScanTermination(TestScanEarlyTermination.java:149)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11782//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11782//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11782//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11782//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11782//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11782//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11782//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/Pr
[jira] [Updated] (HBASE-12073) Shell command user_permission fails on the table created by user if he is not global admin.
[ https://issues.apache.org/jira/browse/HBASE-12073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Srungarapu updated HBASE-12073: Attachment: HBASE-12073_0.99_v3.patch Attaching the patch for branch-1. > Shell command user_permission fails on the table created by user if he is not > global admin. > -- > > Key: HBASE-12073 > URL: https://issues.apache.org/jira/browse/HBASE-12073 > Project: HBase > Issue Type: Bug >Affects Versions: 0.99.1, 0.98.6.1 >Reporter: Srikanth Srungarapu >Assignee: Srikanth Srungarapu >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-12073.patch, HBASE-12073_0.99_v3.patch, > HBASE-12073_master_v3.patch, HBASE-12073_v2.patch > > > The command fails as the changes introduced by HBASE-10892 requires user > (because of newly introduced call to getTableDescriptors) to have global > admin permission. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12522) Backport WAL refactoring to branch-1
[
https://issues.apache.org/jira/browse/HBASE-12522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221464#comment-14221464
]
Jeffrey Zhong commented on HBASE-12522:
---
{quote}
Or do you think Phoenix's secondary index needs to have WAL edits that span
regions?
{quote}
WAL Edits can't span regions because our log SeqId is only guaranteed to
monotonically increase by region. Local index doesn't span edits across
regions. For transaction support, some high level support is needed but not at
the WAL level.
> Backport WAL refactoring to branch-1
>
>
> Key: HBASE-12522
> URL: https://issues.apache.org/jira/browse/HBASE-12522
> Project: HBase
> Issue Type: Task
> Components: wal
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Fix For: 0.99.2
>
>
> backport HBASE-10378 to branch-1.
> This will let us remove the Deprecated stuff in master, allow some baking
> time within the 1.x line, and will give us the option of pulling back follow
> on performance improvements.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12554) TestBaseLoadBalancer may timeout due to lengthy rack lookup
[
https://issues.apache.org/jira/browse/HBASE-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-12554:
---
Attachment: 12554-v3.txt
TestBaseLoadBalancer passes with patch v3.
> TestBaseLoadBalancer may timeout due to lengthy rack lookup
> ---
>
> Key: HBASE-12554
> URL: https://issues.apache.org/jira/browse/HBASE-12554
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 12554-v1.txt, 12554-v2.txt, 12554-v3.txt
>
>
> Here is one of the recent occurrences
> (https://builds.apache.org/job/PreCommit-HBASE-Build/11778/console):
> {code}
> testImmediateAssignment(org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer)
> Time elapsed: 30.019 sec <<< ERROR!
> java.lang.Exception: test timed out after 3 milliseconds
> at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
> at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901)
> at
> java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293)
> at java.net.InetAddress.getAllByName0(InetAddress.java:1246)
> at java.net.InetAddress.getAllByName(InetAddress.java:1162)
> at java.net.InetAddress.getAllByName(InetAddress.java:1098)
> at java.net.InetAddress.getByName(InetAddress.java:1048)
> at org.apache.hadoop.net.NetUtils.normalizeHostName(NetUtils.java:561)
> at org.apache.hadoop.net.NetUtils.normalizeHostNames(NetUtils.java:578)
> at
> org.apache.hadoop.net.CachedDNSToSwitchMapping.resolve(CachedDNSToSwitchMapping.java:109)
> at
> org.apache.hadoop.hbase.master.RackManager.getRack(RackManager.java:66)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.(BaseLoadBalancer.java:273)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.createCluster(BaseLoadBalancer.java:1113)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.randomAssignment(BaseLoadBalancer.java:1175)
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.immediateAssignment(BaseLoadBalancer.java:1145)
> at
> org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer.testImmediateAssignment(TestBaseLoadBalancer.java:136)
> {code}
> One possible fix is to submit CachedDNSToSwitchMapping.resolve() to executor
> pool for execution. RackManager.getRack() can set a timeout beyond which
> UNKNOWN_RACK is returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-10671) Add missing InterfaceAudience annotations for classes in hbase-common and hbase-client modules
[ https://issues.apache.org/jira/browse/HBASE-10671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221435#comment-14221435 ] Hudson commented on HBASE-10671: FAILURE: Integrated in HBase-TRUNK #5809 (See [https://builds.apache.org/job/HBase-TRUNK/5809/]) HBASE-10671 Add missing InterfaceAudience annotations for classes in hbase-common and hbase-client modules (enis: rev 882324dbcc224072be6a43a74aa6102637e529f8) * hbase-common/src/main/java/org/apache/hadoop/hbase/NamespaceDescriptor.java * hbase-client/src/main/java/org/apache/hadoop/hbase/exceptions/PreemptiveFastFailException.java * hbase-common/src/test/java/org/apache/hadoop/hbase/ClassFinder.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/ReadOnlyByteRangeException.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTableMultiplexer.java * hbase-annotations/src/main/java/org/apache/hadoop/hbase/classification/tools/IncludePublicAnnotationsStandardDoclet.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/BoundedCompletionService.java * hbase-client/src/main/java/org/apache/hadoop/hbase/exceptions/UnknownProtocolException.java * hbase-client/src/main/java/org/apache/hadoop/hbase/exceptions/ConnectionClosingException.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/SimpleByteRange.java * hbase-client/src/main/java/org/apache/hadoop/hbase/exceptions/LockTimeoutException.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationStateZKBase.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesClientZKImpl.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityControllerNotReadyException.java * hbase-protocol/pom.xml * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/IPCUtil.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/ExceptionUtil.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/Base64.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/PrettyPrinter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RowTooBigException.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java * hbase-client/src/main/java/org/apache/hadoop/hbase/quotas/InvalidQuotaSettingsException.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java * hbase-common/src/main/java/org/apache/hadoop/hbase/types/PBType.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedException.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueueInfo.java * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/TimeLimitedRpcController.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * hbase-common/src/main/java/org/apache/hadoop/hbase/trace/SpanReceiverHost.java * hbase-client/src/main/java/org/apache/hadoop/hbase/quotas/QuotaExceededException.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/coprocessor/Batch.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/AbstractPositionedByteRange.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/ChecksumUtil.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/ConcatenatedLists.java * hbase-client/src/main/java/org/apache/hadoop/hbase/KeepDeletedCells.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/RegexStringComparator.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionAdminServiceCallable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/token/AuthenticationTokenIdentifier.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeersZKImpl.java * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcControllerFactory.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationTrackerZKImpl.java * hbase-client/src/test/java/org/apache/hadoop/hbase/TestInterfaceAudienceAnnotations.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerFactory.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/FailureInfo.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/DelegatingRetryingCallable.java * hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java * hbase-annotations/src/main/java/org/apache/hadoop/hbase/classification/tools/ExcludePrivateAnnotationsStandardDoclet.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java * hbase-client/src/main/java/org/apache/hadoop/hbase/exce
